[Feature Request] Developer API for Dependency Tree

[wSedlacek]

Description

Make a public API for determining dependencies for a given project and which dependencies have build artifacts.
Cache the dependency tree in the same way build artifacts are so that git submodules do not need to be cloned unless they were modified or not cached.

Motivation

I am working on a solution for using git-meta with Nrwl Nx.

To facilitate this I will need to hook into the dependency graph of Nrwl Nx. Specifically I am looking for an interface that can tell me what dependencies are for a given project so I can clone the submodules for those dependencies.

Additionally because the submodules will not be always cloned this dependency tree needs to be cached in the same way build artifacts are both locally and to Nx Cloud.

If the cache is missed then I need just the partial tree up to the point of a module that is not on disk, then I can open that module check it's dependencies from there.

I would also like another API to get dependents, but this would need to read from a cache because I don't see a way to traverse the tree back up.

Suggested Implementation

I would like to just have a simple import something like this.

import { getDependencies } from '@nrwl/nx';
import { open } from 'git-meta';

const openProjects = new Set<string>();
const isNotOpen = (name: string) => !openProjects.has(name);

const openRepo = async ({ name, path, children, cached }) => {
  if (!cached) {
    await open(path);
    openProjects.add(name);
    const dependencies = await getDependencies(name).toPromise();
    const requests = dependencies.filter(isNotOpen).map(openRepo);
    await Promise.all(requests);
  } else {
    openProjects.add(name);
    const requests = children.map(openRepo);
    await Promise.all(requests);
  }
};

getDependencies('some-project-name')
  .subscribe((dependencies) => dependencies.filter(isNotOpen).forEach(openRepo));

As far as how to get this interface, I don't have enough context into the inner workings of Nrwl Nx at the time of writing, but I would love some feedback on this.

[FrozenPandaz]

That sounds like a cool idea!

We haven't exposed a public API for getting the project graph because we're not quite ready yet. However, the API is pretty stable as it isn't changed often. Of course, the fact that it isn't public, means it is subject to change. It will likely become a public API at some point.

The API you are looking for is likely import { createProjectGraph } from '@nrwl/workspace/src/core/project-graph';.

Let us know what you think! 😄

[wSedlacek]

Thank you for pointing me in the right direction @FrozenPandaz!

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[DrSensor]

no stale

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.