Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: add rosetta2-gc #1165

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

module: add rosetta2-gc #1165

wants to merge 1 commit into from

Conversation

mweinelt
Copy link
Contributor

@mweinelt mweinelt commented Nov 11, 2024

Provides a service to prune the Rosetta 2 JIT bytecode cache.

We have only ever used minsize 0 as a criterion over in nixos/infra, so that it prunes indiscriminately.

The logs look like this:

Mon Nov 11 17:31:20 CET 2024
will purge files >= 0
request has flags: 0x4 type: 0x0 typelimit: 0 min_size 0 max_age 0 desired_amt 0 (size: 144)
Purging files on /System/Volumes/Data returned 0 (OK).  Deleted amount: 3244412928 (1403 files)
Mon Nov 11 19:31:21 CET 2024
will purge files >= 0
request has flags: 0x4 type: 0x0 typelimit: 0 min_size 0 max_age 0 desired_amt 0 (size: 144)
Purging files on /System/Volumes/Data returned 0 (OK).  Deleted amount: 2315005952 (982 files)
Mon Nov 11 21:31:21 CET 2024
will purge files >= 0
request has flags: 0x4 type: 0x0 typelimit: 0 min_size 0 max_age 0 desired_amt 0 (size: 144)
Purging files on /System/Volumes/Data returned 0 (OK).  Deleted amount: 2701418496 (1123 files)

Does darwin have any log rotation service I could configure?

@Enzime
Copy link
Collaborator

Enzime commented Nov 12, 2024

newsyslog runs on macOS and be configured with files in /etc/newsyslog.d

We currently don't use it although eventually it would be great to just have it on by default for all the launchd daemons and agents we manage and then we can just turn on logging by default for them all

@mweinelt mweinelt force-pushed the rosetta2-gc branch 2 times, most recently from 2173c13 to 6092079 Compare November 14, 2024 02:23
Provides a service to prune the Rosetta 2 JIT bytecode cache.
@Enzime
Copy link
Collaborator

Enzime commented Nov 14, 2024

Can you also add a test?

@emilazy
Copy link
Collaborator

emilazy commented Nov 14, 2024

Hmm, so what this service is actually doing is deleting all purgeable space from the disk, which AIUI is essentially cache data that is marked as inessential. It would I guess just purge the Rosetta 2 caches if the path to wherever those are is specified, but currently this has nothing to do with Rosetta 2 per se.

Nominally, it’s fine to do this indiscriminately, because it’s the FS equivalent of flushing disk cache from memory. But it makes me wonder why this service was added to the infra and if there is some known problem it addresses? As with disk cache, this space is meant to be removed and reused automatically if anything else on the system needs the space, and as every Linux user knows it’s not actually a good thing if free -m shows low values. It’s possible that Apple’s implementation here is imperfect, but I’m wondering if this service was added to address a specific issue or was just prophylactically introduced because it looked like disk space was low?

@mweinelt
Copy link
Contributor Author

mweinelt commented Nov 14, 2024

I don't know what the source for the command we're using is. But this one does it similarly and links to a source which is unfortunately down.

https://gist.github.com/thimslugga/17adbee3c45713594c091e80ecd0cf69#file-01-ec2-mac-tuning-sh-L287-L294

The commit it was added in doesn't speak to the reason it was added. But with the throughput a nixos builder has I think it's reasonable to assume this fills up.

As you can see in the log in the op it removes ~3 GB every two hours. Now it is of course hard to say if that is the same 3 GB it recreates in the next 3 hours.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants