Possible memory leak when BwoB is on

[lberki]

I think I found a memory leak in BwoB that's not something that's fixable using a band-aid, but is somewhat fundamental.

When BwoB is on, we create a RemoteActionFileSystem for each executed action and if one wants to access an input file from Bazel, one has to plumb it through ActionExecutionContext.getPathResolver(). That then returns a Path, which can be used to get the metadata of input files or an InputStream thereof.

If a reference to such a Path is kept after the action finishes execution:

• the Path keeps a reference to its ActionFileSystem
• the ActionFileSystem keeps a reference to its ActionInputMap
• the ActionInputMap keeps a reference to the metadata of every input of the action in question

And ActionInputMap can be huge.

I do know for a fact that we do save Path instances in things like TestAttempt, but I have no idea if those are transformed Paths or whether we store such Paths anywhere else.

We could in theory audit the code, but how nice would it be if we could somehow make it so that this is obviously not an issue.

[meisterT]

cc @coeuvre @tjgq

[coeuvre]

Good point. TestAttempt does hold reference to ActionFileSystem, I don't know the lifetime of TestAttempt though. And yeah, it's not an easy task to audit every usages of Path in the codebase.

[lberki]

FWIW, one principled fix would be to outlaw Artifact.getPath() and move the few methods on Path we need to ActionExecutionContext or some interface returned by it. But that would be a HUGE refactoring.

[lberki]

The lifetime of TestAttempt is at least the execution time of a Bazel command, I think.

[katre]

Is #18145 related to this?

[lberki]

#18145 (comment) would indicate that it's not since it looks the main leak is something related to the UI. But it's strange how the UI can use enough RAM to trigger an OOM, so maybe there is a little reference somewhere that connects these two threads.