Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Reused memory from previous allocations is not reported #348

Open
NightMachinery opened this issue Jan 22, 2022 · 0 comments
Open

Comments

@NightMachinery
Copy link

!pip install -U memory_profiler
%load_ext memory_profiler

for i in range(4):
  n = 10**(5+4)
  %memit np.ones((n,), dtype=np.float32)

None
peak memory: 4018.02 MiB, increment: 3818.63 MiB
peak memory: 4018.07 MiB, increment: 0.00 MiB
peak memory: 4018.07 MiB, increment: 0.00 MiB
peak memory: 4018.07 MiB, increment: 0.00 MiB

As you see, while the memory consumption of the line np.ones((n,), dtype=np.float32) is constant, only the first measurement is correct; the next measurements seem to not notice that the Python interpreter is reusing memory it had previously claimed from the OS and freed in garbage collection.

Is there any way to handle this problem? It makes it difficult to find where memory is being actually used, as opposed to finding the very first big allocation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant