Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Filesystem #73

Open
itsWindows11 opened this issue Sep 1, 2024 · 1 comment
Open

Memory Filesystem #73

itsWindows11 opened this issue Sep 1, 2024 · 1 comment

Comments

@itsWindows11
Copy link
Contributor

itsWindows11 commented Sep 1, 2024

Since streams in MemoryFiles are made non-disposable as seen in this code:
https://github.com/Arlodotexe/OwlCore.Storage/blob/main/src%2FMemory%2FMemoryFile.cs#L52-L61

This can cause memory leaks if not properly managed, so I propose adding a way to control the entire memory filesystem tree, this can be a root MemoryFolder with a list of streams corresponding to virtual paths that's private.

using var virtualMemFilesystem = new MemoryFilesystem();

var memFile = new MemoryFile(fullName: "/virtual/path/file.jpg", filesystem: virtualMemFilesystem); // Can also be created through a parent MemoryFolder that's part of this same filesystem.

var stream = await memFile.OpenWriteAsync();

await GetPhysicalPhotoStream().CopyToAsync(stream);

// For as long as `virtualMemFilesystem` isn't disposed, the files will stay intact in memory.

This opens the door to easily providing virtual filesystems that can be easily disposed or managed based on the app's needs, and may also be used to represent a memory drive in the Windows File Explorer for example which is disposed when unmounted, or when recursive copies are available through this library, provides a way to easily create a filesystem tree for installers to copy all at once.

@Arlodotexe
Copy link
Owner

Arlodotexe commented Sep 2, 2024

Note that the internal stream in MemoryFile was passed into the constructor. Here, it's expected that this stream is disposed by the constructor's caller, not internally disposed of by MemoryFile. This allows us to wrap the provided stream with a NonDisposableStreamWrapper and reuse it each time OpenStreamAsync is called, instead of disposing it after the first use and making the IFile instance unusable.

There should be no memory leak here if the caller is properly disposing the provided stream. If there were, we'd want to solve it in a more transparent way than crafting a custom garbage collector for storage / IDisposable.


On the concept of a general Filesystem API, people tend to place here helpers such as file/folder pickers, path tools, etc., and in this case, a memory management system and path aliases.

It's rather vague, and something we try to avoid building inbox. Instead, the storage abstraction focuses heavily on the primitive functionality of what it means to be a "file" or "folder", allowing consumers to leverage the primitives from both the underlying API and the storage abstraction to build more complex helpers and tooling on top of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants