Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible memory leak #266

Open
Yixiao-Zhang opened this issue Sep 11, 2024 · 10 comments
Open

Possible memory leak #266

Yixiao-Zhang opened this issue Sep 11, 2024 · 10 comments
Labels
upstream bug

Comments

@Yixiao-Zhang
Copy link

Describe the bug

The total memory usage keeps increasing when repeatedly calling a function that opens an NC file, does some allocation, and closes that NC file.

To Reproduce

using Printf
using NCDatasets

function create_dummpy_netcdf()
    dset = NCDataset("output.nc", "c")
    close(dset)
end

function output_dummy_netcdf()
    dset = NCDataset("output.nc", "a")
    zeros(1000, 1000)
    close(dset)
end

create_dummpy_netcdf()

for i in 1:1000
    output_dummy_netcdf()

    @info Printf.@sprintf "Max. RSS:  %9.3f MiB\n" Sys.maxrss()/2^20
end

The total memory usage keeps increasing at a rate of 7.5 MiB per iteration. The size of the allocated array in output_dummy_netcdf is 7.6 MiB.

Besides, when we force garbage collection during the iteration, the total memory usage still keeps increasing, although the rate is lower than the case without manual garbage collection.

for i in 1:1000
    output_dummy_netcdf()

    # force garbage collection
    if i % 50 == 0
        GC.gc()
    end

    @info Printf.@sprintf "Max. RSS:  %9.3f MiB\n" Sys.maxrss()/2^20
end

Expected behavior

I expect the total memory usage stays roughly a constant after several iterations. Actually, this is true when I use the "c" mode in output_dummy_netcdf instead of the "a" mode.

Environment

  • operating system: Ubuntu 20.04.6 LTS
  • Julia version: 1.10.4
  • NCDatasets version: 0.14.5
Output of the julia command `versioninfo()` Julia Version 1.10.4 Commit 48d4fd48430 (2024-06-04 10:41 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) CPU: 16 × 11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-15.0.7 (ORCJIT, rocketlake) Threads: 1 default, 0 interactive, 1 GC (on 16 virtual cores)
Output of `using Pkg; Pkg.status(mode=PKGMODE_MANIFEST)` Project TestNCDatasets v0.1.0 Status `~/Documents/IdealizedOceanWorlds.jl/TestNCDatasets.jl/Manifest.toml` [179af706] CFTime v0.1.3 [1fbeeb36] CommonDataModel v0.3.6 [34da2185] Compat v4.16.0 [864edb3b] DataStructures v0.18.20 ⌅ [3c3547ce] DiskArrays v0.3.23 [692b3bcd] JLLWrappers v1.6.0 [8ac3fa9e] LRUCache v1.6.1 [3da0fdf6] MPIPreferences v0.1.11 [85f8d34a] NCDatasets v0.14.5 [6fe1bfb0] OffsetArrays v1.14.1 [bac558e1] OrderedCollections v1.6.3 [21216c6a] Preferences v1.4.3 [0b7ba130] Blosc_jll v1.21.5+0 [6e34b625] Bzip2_jll v1.0.8+1 [0951126a] GnuTLS_jll v3.8.4+0 ⌃ [0234f1f7] HDF5_jll v1.14.2+1 [e33a78d0] Hwloc_jll v2.11.1+0 [1d63c593] LLVMOpenMP_jll v17.0.6+0 [94ce4f54] Libiconv_jll v1.17.0+0 [5ced341a] Lz4_jll v1.10.0+0 [7cb0a576] MPICH_jll v4.2.2+0 [f1f71cc9] MPItrampoline_jll v5.4.0+0 [9237b28f] MicrosoftMPI_jll v10.1.4+2 ⌃ [7243133f] NetCDF_jll v400.902.209+0 ⌅ [4c82536e] Nettle_jll v3.7.2+0 [fe0851c0] OpenMPI_jll v5.0.5+0 [458c3c95] OpenSSL_jll v3.0.15+0 [c2071276] P11Kit_jll v0.24.1+0 [02c8fc9c] XML2_jll v2.13.3+0 [ffd25f8a] XZ_jll v5.4.6+0 [3161d3a3] Zstd_jll v1.5.6+0 [477f73a3] libaec_jll v1.1.2+0 [337d8026] libzip_jll v1.10.1+0 [0dad84c5] ArgTools v1.1.1 [56f22d72] Artifacts [2a0f44e3] Base64 [ade2ca70] Dates [f43a241f] Downloads v1.6.0 [7b1f6079] FileWatching [b77e0a4c] InteractiveUtils [4af54fe1] LazyArtifacts [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 [8f399da3] Libdl [37e2e46d] LinearAlgebra [56ddb016] Logging [d6f4376e] Markdown [ca575930] NetworkOptions v1.2.0 [44cfe95a] Pkg v1.10.0 [de0858da] Printf [3fa0cd96] REPL [9a3f8284] Random [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization [6462fe0b] Sockets [2f01184e] SparseArrays v1.10.0 [10745b16] Statistics v1.10.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [cf7118a7] UUIDs [4ec0a83e] Unicode [e66e0078] CompilerSupportLibraries_jll v1.1.1+0 [781609d7] GMP_jll v6.2.1+6 [deac9b47] LibCURL_jll v8.4.0+0 [e37daf67] LibGit2_jll v1.6.4+0 [29816b5a] LibSSH2_jll v1.11.0+1 [c8ffd9c3] MbedTLS_jll v2.28.2+1 [14a3606d] MozillaCACerts_jll v2023.1.10 [4536629a] OpenBLAS_jll v0.3.23+4 [bea87d4a] SuiteSparse_jll v7.2.1+1 [83775a58] Zlib_jll v1.2.13+1 [8e850b90] libblastrampoline_jll v5.8.0+1 [8e850ede] nghttp2_jll v1.52.0+1 [3f19e933] p7zip_jll v17.4.0+2

Full output

Full Output without Manual GC

[ Info: Max. RSS: 318.418 MiB
[ Info: Max. RSS: 325.723 MiB
[ Info: Max. RSS: 333.457 MiB
[ Info: Max. RSS: 341.191 MiB
[ Info: Max. RSS: 348.668 MiB
[ Info: Max. RSS: 356.402 MiB
[ Info: Max. RSS: 363.879 MiB
[ Info: Max. RSS: 371.613 MiB
[ Info: Max. RSS: 371.613 MiB
[ Info: Max. RSS: 379.348 MiB
[ Info: Max. RSS: 386.824 MiB
[ Info: Max. RSS: 394.559 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 402.035 MiB
[ Info: Max. RSS: 407.242 MiB
[ Info: Max. RSS: 414.977 MiB
[ Info: Max. RSS: 422.453 MiB
[ Info: Max. RSS: 430.188 MiB
[ Info: Max. RSS: 437.664 MiB
[ Info: Max. RSS: 445.398 MiB
[ Info: Max. RSS: 453.133 MiB
[ Info: Max. RSS: 460.609 MiB
[ Info: Max. RSS: 468.344 MiB
[ Info: Max. RSS: 475.820 MiB
[ Info: Max. RSS: 483.555 MiB
[ Info: Max. RSS: 491.289 MiB
[ Info: Max. RSS: 498.766 MiB
[ Info: Max. RSS: 506.500 MiB
[ Info: Max. RSS: 513.977 MiB
[ Info: Max. RSS: 521.711 MiB
[ Info: Max. RSS: 529.445 MiB
[ Info: Max. RSS: 536.922 MiB
[ Info: Max. RSS: 544.656 MiB
[ Info: Max. RSS: 552.133 MiB
[ Info: Max. RSS: 559.867 MiB
[ Info: Max. RSS: 567.602 MiB
[ Info: Max. RSS: 575.078 MiB
[ Info: Max. RSS: 582.812 MiB
[ Info: Max. RSS: 590.289 MiB
[ Info: Max. RSS: 598.023 MiB
[ Info: Max. RSS: 605.500 MiB
[ Info: Max. RSS: 613.234 MiB
[ Info: Max. RSS: 620.969 MiB
[ Info: Max. RSS: 628.445 MiB
[ Info: Max. RSS: 636.180 MiB
[ Info: Max. RSS: 643.656 MiB
[ Info: Max. RSS: 651.391 MiB
[ Info: Max. RSS: 659.125 MiB
[ Info: Max. RSS: 666.602 MiB
[ Info: Max. RSS: 674.336 MiB
[ Info: Max. RSS: 681.812 MiB
[ Info: Max. RSS: 689.547 MiB
[ Info: Max. RSS: 697.281 MiB
[ Info: Max. RSS: 704.758 MiB
[ Info: Max. RSS: 712.492 MiB
[ Info: Max. RSS: 719.969 MiB
[ Info: Max. RSS: 727.703 MiB
[ Info: Max. RSS: 735.438 MiB
[ Info: Max. RSS: 742.914 MiB
[ Info: Max. RSS: 750.648 MiB

...

[ Info: Max. RSS: 4107.625 MiB
[ Info: Max. RSS: 4115.102 MiB
[ Info: Max. RSS: 4122.836 MiB
[ Info: Max. RSS: 4130.570 MiB
[ Info: Max. RSS: 4138.047 MiB
[ Info: Max. RSS: 4145.781 MiB
[ Info: Max. RSS: 4153.258 MiB
[ Info: Max. RSS: 4160.992 MiB
[ Info: Max. RSS: 4168.727 MiB
[ Info: Max. RSS: 4176.203 MiB
[ Info: Max. RSS: 4183.938 MiB
[ Info: Max. RSS: 4191.414 MiB
[ Info: Max. RSS: 4199.148 MiB
[ Info: Max. RSS: 4206.883 MiB
[ Info: Max. RSS: 4214.359 MiB
[ Info: Max. RSS: 4222.094 MiB
[ Info: Max. RSS: 4229.570 MiB
[ Info: Max. RSS: 4237.305 MiB
[ Info: Max. RSS: 4244.781 MiB
[ Info: Max. RSS: 4252.516 MiB
[ Info: Max. RSS: 4260.250 MiB
[ Info: Max. RSS: 4267.727 MiB
[ Info: Max. RSS: 4275.461 MiB
[ Info: Max. RSS: 4282.938 MiB
[ Info: Max. RSS: 4290.672 MiB
[ Info: Max. RSS: 4298.406 MiB
[ Info: Max. RSS: 4305.883 MiB
[ Info: Max. RSS: 4313.617 MiB
[ Info: Max. RSS: 4321.094 MiB
[ Info: Max. RSS: 4328.828 MiB
[ Info: Max. RSS: 4336.562 MiB
[ Info: Max. RSS: 4344.039 MiB
[ Info: Max. RSS: 4351.773 MiB
[ Info: Max. RSS: 4359.250 MiB
[ Info: Max. RSS: 4366.984 MiB
[ Info: Max. RSS: 4374.719 MiB
[ Info: Max. RSS: 4382.195 MiB
[ Info: Max. RSS: 4389.930 MiB
[ Info: Max. RSS: 4397.406 MiB
[ Info: Max. RSS: 4405.141 MiB
[ Info: Max. RSS: 4412.875 MiB
[ Info: Max. RSS: 4420.352 MiB
[ Info: Max. RSS: 4428.086 MiB
[ Info: Max. RSS: 4435.562 MiB
[ Info: Max. RSS: 4443.297 MiB
[ Info: Max. RSS: 4450.773 MiB
[ Info: Max. RSS: 4458.508 MiB
[ Info: Max. RSS: 4466.242 MiB
[ Info: Max. RSS: 4473.719 MiB
[ Info: Max. RSS: 4481.453 MiB
[ Info: Max. RSS: 4488.930 MiB
[ Info: Max. RSS: 4496.664 MiB
[ Info: Max. RSS: 4504.398 MiB
[ Info: Max. RSS: 4511.875 MiB
[ Info: Max. RSS: 4519.609 MiB
[ Info: Max. RSS: 4527.086 MiB
[ Info: Max. RSS: 4534.820 MiB
[ Info: Max. RSS: 4542.555 MiB
[ Info: Max. RSS: 4550.031 MiB
[ Info: Max. RSS: 4557.766 MiB
[ Info: Max. RSS: 4565.242 MiB
[ Info: Max. RSS: 4572.977 MiB
[ Info: Max. RSS: 4580.711 MiB
[ Info: Max. RSS: 4588.188 MiB
[ Info: Max. RSS: 4595.922 MiB
[ Info: Max. RSS: 4603.398 MiB
[ Info: Max. RSS: 4611.133 MiB
[ Info: Max. RSS: 4618.867 MiB
[ Info: Max. RSS: 4626.344 MiB
[ Info: Max. RSS: 4634.078 MiB
[ Info: Max. RSS: 4641.555 MiB
[ Info: Max. RSS: 4649.289 MiB
[ Info: Max. RSS: 4657.023 MiB
[ Info: Max. RSS: 4664.500 MiB
[ Info: Max. RSS: 4672.234 MiB
[ Info: Max. RSS: 4679.711 MiB
[ Info: Max. RSS: 4687.445 MiB
[ Info: Max. RSS: 4694.922 MiB
[ Info: Max. RSS: 4702.656 MiB
[ Info: Max. RSS: 4710.391 MiB
[ Info: Max. RSS: 4717.867 MiB
[ Info: Max. RSS: 4725.602 MiB
[ Info: Max. RSS: 4733.078 MiB
[ Info: Max. RSS: 4740.812 MiB
[ Info: Max. RSS: 4748.547 MiB
[ Info: Max. RSS: 4756.023 MiB
[ Info: Max. RSS: 4763.758 MiB
[ Info: Max. RSS: 4771.234 MiB
[ Info: Max. RSS: 4778.969 MiB
[ Info: Max. RSS: 4786.703 MiB
[ Info: Max. RSS: 4794.180 MiB
[ Info: Max. RSS: 4801.914 MiB
[ Info: Max. RSS: 4809.391 MiB
[ Info: Max. RSS: 4817.125 MiB
[ Info: Max. RSS: 4824.859 MiB
[ Info: Max. RSS: 4832.336 MiB
[ Info: Max. RSS: 4840.070 MiB
[ Info: Max. RSS: 4847.547 MiB
[ Info: Max. RSS: 4855.281 MiB
[ Info: Max. RSS: 4863.016 MiB
[ Info: Max. RSS: 4870.492 MiB
[ Info: Max. RSS: 4878.227 MiB
[ Info: Max. RSS: 4885.703 MiB
[ Info: Max. RSS: 4893.438 MiB
[ Info: Max. RSS: 4900.914 MiB
[ Info: Max. RSS: 4908.648 MiB
[ Info: Max. RSS: 4916.383 MiB
[ Info: Max. RSS: 4923.859 MiB
[ Info: Max. RSS: 4931.594 MiB
[ Info: Max. RSS: 4939.070 MiB
[ Info: Max. RSS: 4946.805 MiB
[ Info: Max. RSS: 4954.539 MiB
[ Info: Max. RSS: 4962.016 MiB
[ Info: Max. RSS: 4969.750 MiB
[ Info: Max. RSS: 4977.227 MiB
[ Info: Max. RSS: 4984.961 MiB
[ Info: Max. RSS: 4992.695 MiB
[ Info: Max. RSS: 5000.172 MiB
[ Info: Max. RSS: 5007.906 MiB
[ Info: Max. RSS: 5015.383 MiB
[ Info: Max. RSS: 5023.117 MiB
[ Info: Max. RSS: 5030.852 MiB
[ Info: Max. RSS: 5038.328 MiB
[ Info: Max. RSS: 5046.062 MiB
[ Info: Max. RSS: 5053.539 MiB
[ Info: Max. RSS: 5061.273 MiB
[ Info: Max. RSS: 5069.008 MiB
[ Info: Max. RSS: 5076.484 MiB
[ Info: Max. RSS: 5084.219 MiB
[ Info: Max. RSS: 5091.695 MiB
[ Info: Max. RSS: 5099.430 MiB
[ Info: Max. RSS: 5107.164 MiB
[ Info: Max. RSS: 5114.641 MiB
[ Info: Max. RSS: 5122.375 MiB
[ Info: Max. RSS: 5129.852 MiB
[ Info: Max. RSS: 5137.586 MiB
[ Info: Max. RSS: 5145.062 MiB
[ Info: Max. RSS: 5152.797 MiB
[ Info: Max. RSS: 5160.531 MiB
[ Info: Max. RSS: 5168.008 MiB
[ Info: Max. RSS: 5175.742 MiB
[ Info: Max. RSS: 5183.219 MiB
[ Info: Max. RSS: 5190.953 MiB
[ Info: Max. RSS: 5198.688 MiB
[ Info: Max. RSS: 5206.164 MiB
[ Info: Max. RSS: 5213.898 MiB
[ Info: Max. RSS: 5221.375 MiB
[ Info: Max. RSS: 5229.109 MiB
[ Info: Max. RSS: 5236.844 MiB
[ Info: Max. RSS: 5244.320 MiB
[ Info: Max. RSS: 5252.055 MiB
[ Info: Max. RSS: 5259.531 MiB
[ Info: Max. RSS: 5267.266 MiB
[ Info: Max. RSS: 5275.000 MiB
[ Info: Max. RSS: 5282.477 MiB
[ Info: Max. RSS: 5290.211 MiB
[ Info: Max. RSS: 5297.688 MiB
[ Info: Max. RSS: 5305.422 MiB
[ Info: Max. RSS: 5313.156 MiB
[ Info: Max. RSS: 5320.633 MiB
[ Info: Max. RSS: 5328.367 MiB
[ Info: Max. RSS: 5335.844 MiB
[ Info: Max. RSS: 5343.578 MiB
[ Info: Max. RSS: 5351.055 MiB
[ Info: Max. RSS: 5358.789 MiB
[ Info: Max. RSS: 5366.523 MiB
[ Info: Max. RSS: 5374.000 MiB
[ Info: Max. RSS: 5381.734 MiB
[ Info: Max. RSS: 5389.211 MiB
[ Info: Max. RSS: 5396.945 MiB
[ Info: Max. RSS: 5404.680 MiB
[ Info: Max. RSS: 5412.156 MiB
[ Info: Max. RSS: 5419.891 MiB
[ Info: Max. RSS: 5427.367 MiB
[ Info: Max. RSS: 5435.102 MiB
[ Info: Max. RSS: 5442.836 MiB
[ Info: Max. RSS: 5450.312 MiB
[ Info: Max. RSS: 5458.047 MiB
[ Info: Max. RSS: 5465.523 MiB
[ Info: Max. RSS: 5473.258 MiB
[ Info: Max. RSS: 5480.992 MiB
[ Info: Max. RSS: 5488.469 MiB
[ Info: Max. RSS: 5496.203 MiB
[ Info: Max. RSS: 5503.680 MiB
[ Info: Max. RSS: 5511.414 MiB
[ Info: Max. RSS: 5519.148 MiB
[ Info: Max. RSS: 5526.625 MiB
[ Info: Max. RSS: 5534.359 MiB
[ Info: Max. RSS: 5541.836 MiB
[ Info: Max. RSS: 5549.570 MiB
[ Info: Max. RSS: 5557.305 MiB
[ Info: Max. RSS: 5564.781 MiB
[ Info: Max. RSS: 5572.516 MiB
[ Info: Max. RSS: 5579.992 MiB
[ Info: Max. RSS: 5587.727 MiB
[ Info: Max. RSS: 5595.203 MiB
[ Info: Max. RSS: 5602.938 MiB
[ Info: Max. RSS: 5610.672 MiB
[ Info: Max. RSS: 5618.148 MiB
[ Info: Max. RSS: 5625.883 MiB
[ Info: Max. RSS: 5633.359 MiB
[ Info: Max. RSS: 5641.094 MiB
[ Info: Max. RSS: 5648.828 MiB
[ Info: Max. RSS: 5656.305 MiB
[ Info: Max. RSS: 5664.039 MiB
[ Info: Max. RSS: 5671.516 MiB
[ Info: Max. RSS: 5679.250 MiB
[ Info: Max. RSS: 5686.984 MiB
[ Info: Max. RSS: 5694.461 MiB
[ Info: Max. RSS: 5702.195 MiB
[ Info: Max. RSS: 5709.672 MiB
[ Info: Max. RSS: 5717.406 MiB
[ Info: Max. RSS: 5725.141 MiB
[ Info: Max. RSS: 5732.617 MiB
[ Info: Max. RSS: 5740.352 MiB
[ Info: Max. RSS: 5747.828 MiB
[ Info: Max. RSS: 5755.562 MiB
[ Info: Max. RSS: 5763.297 MiB
[ Info: Max. RSS: 5770.773 MiB
[ Info: Max. RSS: 5778.508 MiB
[ Info: Max. RSS: 5785.984 MiB
[ Info: Max. RSS: 5793.719 MiB
[ Info: Max. RSS: 5801.453 MiB
[ Info: Max. RSS: 5808.930 MiB
[ Info: Max. RSS: 5816.664 MiB
[ Info: Max. RSS: 5824.141 MiB
[ Info: Max. RSS: 5831.875 MiB
[ Info: Max. RSS: 5839.352 MiB
[ Info: Max. RSS: 5847.086 MiB
[ Info: Max. RSS: 5854.820 MiB
[ Info: Max. RSS: 5862.297 MiB
[ Info: Max. RSS: 5870.031 MiB
[ Info: Max. RSS: 5877.508 MiB
[ Info: Max. RSS: 5885.242 MiB
[ Info: Max. RSS: 5892.977 MiB
[ Info: Max. RSS: 5900.453 MiB
[ Info: Max. RSS: 5908.188 MiB
[ Info: Max. RSS: 5915.664 MiB
[ Info: Max. RSS: 5923.398 MiB
[ Info: Max. RSS: 5931.133 MiB
[ Info: Max. RSS: 5938.609 MiB
[ Info: Max. RSS: 5946.344 MiB
[ Info: Max. RSS: 5953.820 MiB
[ Info: Max. RSS: 5961.555 MiB
[ Info: Max. RSS: 5969.289 MiB
[ Info: Max. RSS: 5976.766 MiB
[ Info: Max. RSS: 5984.500 MiB
[ Info: Max. RSS: 5991.977 MiB
[ Info: Max. RSS: 5999.711 MiB
[ Info: Max. RSS: 6007.445 MiB
[ Info: Max. RSS: 6014.922 MiB
[ Info: Max. RSS: 6022.656 MiB
[ Info: Max. RSS: 6030.133 MiB
[ Info: Max. RSS: 6037.867 MiB
[ Info: Max. RSS: 6045.344 MiB
[ Info: Max. RSS: 6053.078 MiB
[ Info: Max. RSS: 6060.812 MiB
[ Info: Max. RSS: 6068.289 MiB
[ Info: Max. RSS: 6076.023 MiB
[ Info: Max. RSS: 6083.500 MiB
[ Info: Max. RSS: 6091.234 MiB
[ Info: Max. RSS: 6098.969 MiB
[ Info: Max. RSS: 6106.445 MiB
[ Info: Max. RSS: 6114.180 MiB
[ Info: Max. RSS: 6121.656 MiB
[ Info: Max. RSS: 6129.391 MiB
[ Info: Max. RSS: 6137.125 MiB
[ Info: Max. RSS: 6144.602 MiB
[ Info: Max. RSS: 6152.336 MiB
[ Info: Max. RSS: 6159.812 MiB
[ Info: Max. RSS: 6167.547 MiB
[ Info: Max. RSS: 6175.281 MiB
[ Info: Max. RSS: 6182.758 MiB
[ Info: Max. RSS: 6190.492 MiB
[ Info: Max. RSS: 6197.969 MiB
[ Info: Max. RSS: 6205.703 MiB
[ Info: Max. RSS: 6213.438 MiB
[ Info: Max. RSS: 6220.914 MiB
[ Info: Max. RSS: 6228.648 MiB
[ Info: Max. RSS: 6236.125 MiB
[ Info: Max. RSS: 6243.859 MiB
[ Info: Max. RSS: 6251.594 MiB
[ Info: Max. RSS: 6259.070 MiB
[ Info: Max. RSS: 6266.805 MiB
[ Info: Max. RSS: 6274.281 MiB
[ Info: Max. RSS: 6282.016 MiB
[ Info: Max. RSS: 6289.492 MiB
[ Info: Max. RSS: 6297.227 MiB
[ Info: Max. RSS: 6304.961 MiB
[ Info: Max. RSS: 6312.438 MiB
[ Info: Max. RSS: 6320.172 MiB
[ Info: Max. RSS: 6327.648 MiB
[ Info: Max. RSS: 6335.383 MiB
[ Info: Max. RSS: 6343.117 MiB
[ Info: Max. RSS: 6350.594 MiB
[ Info: Max. RSS: 6358.328 MiB
[ Info: Max. RSS: 6365.805 MiB
[ Info: Max. RSS: 6373.539 MiB
[ Info: Max. RSS: 6381.273 MiB
[ Info: Max. RSS: 6388.750 MiB
[ Info: Max. RSS: 6396.484 MiB
[ Info: Max. RSS: 6403.961 MiB
[ Info: Max. RSS: 6411.695 MiB
[ Info: Max. RSS: 6419.430 MiB
[ Info: Max. RSS: 6426.906 MiB
[ Info: Max. RSS: 6434.641 MiB
[ Info: Max. RSS: 6442.117 MiB
[ Info: Max. RSS: 6449.852 MiB
[ Info: Max. RSS: 6457.586 MiB
[ Info: Max. RSS: 6465.062 MiB
[ Info: Max. RSS: 6472.797 MiB
[ Info: Max. RSS: 6480.273 MiB
[ Info: Max. RSS: 6488.008 MiB
[ Info: Max. RSS: 6495.742 MiB
[ Info: Max. RSS: 6503.219 MiB
[ Info: Max. RSS: 6510.953 MiB
[ Info: Max. RSS: 6518.430 MiB
[ Info: Max. RSS: 6526.164 MiB
[ Info: Max. RSS: 6533.641 MiB
[ Info: Max. RSS: 6541.375 MiB
[ Info: Max. RSS: 6549.109 MiB
[ Info: Max. RSS: 6556.586 MiB
[ Info: Max. RSS: 6564.320 MiB
[ Info: Max. RSS: 6571.797 MiB
[ Info: Max. RSS: 6579.531 MiB
[ Info: Max. RSS: 6587.266 MiB
[ Info: Max. RSS: 6594.742 MiB
[ Info: Max. RSS: 6602.477 MiB
[ Info: Max. RSS: 6609.953 MiB
[ Info: Max. RSS: 6617.688 MiB
[ Info: Max. RSS: 6625.422 MiB
[ Info: Max. RSS: 6632.898 MiB
[ Info: Max. RSS: 6640.633 MiB
[ Info: Max. RSS: 6648.109 MiB
[ Info: Max. RSS: 6655.844 MiB
[ Info: Max. RSS: 6663.578 MiB
[ Info: Max. RSS: 6671.055 MiB
[ Info: Max. RSS: 6678.789 MiB
[ Info: Max. RSS: 6686.266 MiB
[ Info: Max. RSS: 6694.000 MiB
[ Info: Max. RSS: 6701.734 MiB
[ Info: Max. RSS: 6709.211 MiB
[ Info: Max. RSS: 6716.945 MiB
[ Info: Max. RSS: 6724.422 MiB
[ Info: Max. RSS: 6732.156 MiB
[ Info: Max. RSS: 6739.633 MiB
[ Info: Max. RSS: 6747.367 MiB
[ Info: Max. RSS: 6755.102 MiB
[ Info: Max. RSS: 6762.578 MiB
[ Info: Max. RSS: 6770.312 MiB
[ Info: Max. RSS: 6777.789 MiB
[ Info: Max. RSS: 6785.523 MiB
[ Info: Max. RSS: 6793.258 MiB
[ Info: Max. RSS: 6800.734 MiB
[ Info: Max. RSS: 6808.469 MiB
[ Info: Max. RSS: 6815.945 MiB
[ Info: Max. RSS: 6823.680 MiB
[ Info: Max. RSS: 6831.414 MiB
[ Info: Max. RSS: 6838.891 MiB
[ Info: Max. RSS: 6846.625 MiB
[ Info: Max. RSS: 6854.102 MiB
[ Info: Max. RSS: 6861.836 MiB
[ Info: Max. RSS: 6869.570 MiB
[ Info: Max. RSS: 6877.047 MiB
[ Info: Max. RSS: 6884.781 MiB
[ Info: Max. RSS: 6892.258 MiB
[ Info: Max. RSS: 6899.992 MiB
[ Info: Max. RSS: 6907.727 MiB
[ Info: Max. RSS: 6915.203 MiB
[ Info: Max. RSS: 6922.938 MiB
[ Info: Max. RSS: 6930.414 MiB
[ Info: Max. RSS: 6938.148 MiB
[ Info: Max. RSS: 6945.883 MiB
[ Info: Max. RSS: 6953.359 MiB
[ Info: Max. RSS: 6961.094 MiB
[ Info: Max. RSS: 6968.570 MiB
[ Info: Max. RSS: 6976.305 MiB
[ Info: Max. RSS: 6983.781 MiB
[ Info: Max. RSS: 6991.516 MiB
[ Info: Max. RSS: 6999.250 MiB
[ Info: Max. RSS: 7006.727 MiB
[ Info: Max. RSS: 7014.461 MiB
[ Info: Max. RSS: 7021.938 MiB
[ Info: Max. RSS: 7029.672 MiB
[ Info: Max. RSS: 7037.406 MiB
[ Info: Max. RSS: 7044.883 MiB
[ Info: Max. RSS: 7052.617 MiB
[ Info: Max. RSS: 7060.094 MiB
[ Info: Max. RSS: 7067.828 MiB
[ Info: Max. RSS: 7075.562 MiB
[ Info: Max. RSS: 7083.039 MiB
[ Info: Max. RSS: 7090.773 MiB
[ Info: Max. RSS: 7098.250 MiB
[ Info: Max. RSS: 7105.984 MiB
[ Info: Max. RSS: 7113.719 MiB
[ Info: Max. RSS: 7121.195 MiB
[ Info: Max. RSS: 7128.930 MiB
[ Info: Max. RSS: 7136.406 MiB
[ Info: Max. RSS: 7144.141 MiB
[ Info: Max. RSS: 7151.875 MiB
[ Info: Max. RSS: 7159.352 MiB
[ Info: Max. RSS: 7167.086 MiB
[ Info: Max. RSS: 7174.562 MiB
[ Info: Max. RSS: 7182.297 MiB
[ Info: Max. RSS: 7189.773 MiB
[ Info: Max. RSS: 7197.508 MiB
[ Info: Max. RSS: 7205.242 MiB
[ Info: Max. RSS: 7212.719 MiB
[ Info: Max. RSS: 7220.453 MiB
[ Info: Max. RSS: 7227.930 MiB
[ Info: Max. RSS: 7235.664 MiB
[ Info: Max. RSS: 7243.398 MiB
[ Info: Max. RSS: 7250.875 MiB
[ Info: Max. RSS: 7258.609 MiB
[ Info: Max. RSS: 7266.086 MiB
[ Info: Max. RSS: 7273.820 MiB
[ Info: Max. RSS: 7281.555 MiB
[ Info: Max. RSS: 7289.031 MiB
[ Info: Max. RSS: 7296.766 MiB
[ Info: Max. RSS: 7304.242 MiB
[ Info: Max. RSS: 7311.977 MiB
[ Info: Max. RSS: 7319.711 MiB
[ Info: Max. RSS: 7327.188 MiB
[ Info: Max. RSS: 7334.922 MiB
[ Info: Max. RSS: 7342.398 MiB
[ Info: Max. RSS: 7350.133 MiB
[ Info: Max. RSS: 7357.867 MiB
[ Info: Max. RSS: 7365.344 MiB
[ Info: Max. RSS: 7373.078 MiB
[ Info: Max. RSS: 7380.555 MiB
[ Info: Max. RSS: 7388.289 MiB
[ Info: Max. RSS: 7396.023 MiB
[ Info: Max. RSS: 7403.500 MiB
[ Info: Max. RSS: 7411.234 MiB
[ Info: Max. RSS: 7418.711 MiB
[ Info: Max. RSS: 7426.445 MiB
[ Info: Max. RSS: 7434.180 MiB
[ Info: Max. RSS: 7441.656 MiB
[ Info: Max. RSS: 7449.391 MiB
[ Info: Max. RSS: 7456.867 MiB
[ Info: Max. RSS: 7464.602 MiB
[ Info: Max. RSS: 7472.078 MiB
[ Info: Max. RSS: 7479.812 MiB
[ Info: Max. RSS: 7487.547 MiB
[ Info: Max. RSS: 7495.023 MiB
[ Info: Max. RSS: 7502.758 MiB
[ Info: Max. RSS: 7510.234 MiB
[ Info: Max. RSS: 7517.969 MiB
[ Info: Max. RSS: 7525.703 MiB
[ Info: Max. RSS: 7533.180 MiB
[ Info: Max. RSS: 7540.914 MiB
[ Info: Max. RSS: 7548.391 MiB
[ Info: Max. RSS: 7556.125 MiB
[ Info: Max. RSS: 7563.859 MiB
[ Info: Max. RSS: 7571.336 MiB
[ Info: Max. RSS: 7579.070 MiB
[ Info: Max. RSS: 7586.547 MiB
[ Info: Max. RSS: 7594.281 MiB
[ Info: Max. RSS: 7602.016 MiB
[ Info: Max. RSS: 7609.492 MiB
[ Info: Max. RSS: 7617.227 MiB
[ Info: Max. RSS: 7624.703 MiB
[ Info: Max. RSS: 7632.438 MiB
[ Info: Max. RSS: 7640.172 MiB
[ Info: Max. RSS: 7647.648 MiB
[ Info: Max. RSS: 7655.383 MiB
[ Info: Max. RSS: 7662.859 MiB
[ Info: Max. RSS: 7670.594 MiB
[ Info: Max. RSS: 7678.328 MiB
[ Info: Max. RSS: 7685.805 MiB
[ Info: Max. RSS: 7693.539 MiB
[ Info: Max. RSS: 7701.016 MiB
[ Info: Max. RSS: 7708.750 MiB
[ Info: Max. RSS: 7716.484 MiB
[ Info: Max. RSS: 7723.961 MiB
[ Info: Max. RSS: 7731.695 MiB
[ Info: Max. RSS: 7739.172 MiB
[ Info: Max. RSS: 7746.906 MiB
[ Info: Max. RSS: 7754.641 MiB
[ Info: Max. RSS: 7762.117 MiB
[ Info: Max. RSS: 7769.852 MiB
[ Info: Max. RSS: 7777.328 MiB
[ Info: Max. RSS: 7785.062 MiB
[ Info: Max. RSS: 7792.539 MiB
[ Info: Max. RSS: 7800.273 MiB
[ Info: Max. RSS: 7808.008 MiB
[ Info: Max. RSS: 7815.484 MiB
[ Info: Max. RSS: 7823.219 MiB
[ Info: Max. RSS: 7830.695 MiB
[ Info: Max. RSS: 7838.430 MiB
[ Info: Max. RSS: 7846.164 MiB

Full Output with Manual GC [ Info: Max. RSS: 316.633 MiB [ Info: Max. RSS: 323.871 MiB [ Info: Max. RSS: 331.605 MiB [ Info: Max. RSS: 339.340 MiB [ Info: Max. RSS: 346.816 MiB [ Info: Max. RSS: 354.551 MiB [ Info: Max. RSS: 362.027 MiB [ Info: Max. RSS: 369.762 MiB [ Info: Max. RSS: 377.496 MiB [ Info: Max. RSS: 384.973 MiB [ Info: Max. RSS: 392.707 MiB [ Info: Max. RSS: 400.184 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 407.918 MiB [ Info: Max. RSS: 413.812 MiB [ Info: Max. RSS: 421.289 MiB [ Info: Max. RSS: 429.023 MiB [ Info: Max. RSS: 436.500 MiB [ Info: Max. RSS: 444.234 MiB [ Info: Max. RSS: 451.969 MiB [ Info: Max. RSS: 459.445 MiB [ Info: Max. RSS: 467.180 MiB [ Info: Max. RSS: 474.656 MiB [ Info: Max. RSS: 482.391 MiB [ Info: Max. RSS: 490.125 MiB [ Info: Max. RSS: 497.602 MiB [ Info: Max. RSS: 505.336 MiB [ Info: Max. RSS: 512.812 MiB [ Info: Max. RSS: 520.547 MiB [ Info: Max. RSS: 528.281 MiB [ Info: Max. RSS: 535.758 MiB [ Info: Max. RSS: 543.492 MiB [ Info: Max. RSS: 550.969 MiB [ Info: Max. RSS: 558.703 MiB [ Info: Max. RSS: 566.438 MiB [ Info: Max. RSS: 573.914 MiB [ Info: Max. RSS: 581.648 MiB [ Info: Max. RSS: 589.125 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 596.945 MiB [ Info: Max. RSS: 604.562 MiB [ Info: Max. RSS: 612.297 MiB [ Info: Max. RSS: 619.773 MiB [ Info: Max. RSS: 627.508 MiB [ Info: Max. RSS: 634.984 MiB [ Info: Max. RSS: 642.719 MiB [ Info: Max. RSS: 650.453 MiB [ Info: Max. RSS: 657.930 MiB [ Info: Max. RSS: 665.664 MiB [ Info: Max. RSS: 673.141 MiB [ Info: Max. RSS: 680.875 MiB [ Info: Max. RSS: 688.609 MiB [ Info: Max. RSS: 696.086 MiB [ Info: Max. RSS: 703.820 MiB [ Info: Max. RSS: 711.297 MiB [ Info: Max. RSS: 719.031 MiB [ Info: Max. RSS: 726.508 MiB [ Info: Max. RSS: 734.242 MiB [ Info: Max. RSS: 741.977 MiB [ Info: Max. RSS: 749.453 MiB [ Info: Max. RSS: 757.188 MiB [ Info: Max. RSS: 764.664 MiB [ Info: Max. RSS: 772.398 MiB [ Info: Max. RSS: 780.133 MiB [ Info: Max. RSS: 787.609 MiB [ Info: Max. RSS: 795.344 MiB [ Info: Max. RSS: 802.820 MiB [ Info: Max. RSS: 810.555 MiB [ Info: Max. RSS: 818.289 MiB [ Info: Max. RSS: 825.766 MiB [ Info: Max. RSS: 833.500 MiB [ Info: Max. RSS: 840.977 MiB [ Info: Max. RSS: 848.711 MiB [ Info: Max. RSS: 856.445 MiB [ Info: Max. RSS: 863.922 MiB [ Info: Max. RSS: 871.656 MiB [ Info: Max. RSS: 879.133 MiB [ Info: Max. RSS: 886.867 MiB [ Info: Max. RSS: 894.602 MiB [ Info: Max. RSS: 902.078 MiB [ Info: Max. RSS: 909.812 MiB [ Info: Max. RSS: 917.289 MiB [ Info: Max. RSS: 925.023 MiB [ Info: Max. RSS: 932.758 MiB [ Info: Max. RSS: 940.234 MiB [ Info: Max. RSS: 947.969 MiB [ Info: Max. RSS: 955.445 MiB [ Info: Max. RSS: 963.180 MiB [ Info: Max. RSS: 970.656 MiB [ Info: Max. RSS: 978.391 MiB [ Info: Max. RSS: 986.125 MiB [ Info: Max. RSS: 993.602 MiB [ Info: Max. RSS: 1001.336 MiB [ Info: Max. RSS: 1008.812 MiB [ Info: Max. RSS: 1016.547 MiB [ Info: Max. RSS: 1024.281 MiB [ Info: Max. RSS: 1031.758 MiB [ Info: Max. RSS: 1039.492 MiB [ Info: Max. RSS: 1046.969 MiB [ Info: Max. RSS: 1054.703 MiB [ Info: Max. RSS: 1062.438 MiB [ Info: Max. RSS: 1069.914 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1077.719 MiB [ Info: Max. RSS: 1078.051 MiB [ Info: Max. RSS: 1085.527 MiB [ Info: Max. RSS: 1093.262 MiB [ Info: Max. RSS: 1100.996 MiB [ Info: Max. RSS: 1108.473 MiB [ Info: Max. RSS: 1116.207 MiB [ Info: Max. RSS: 1123.684 MiB [ Info: Max. RSS: 1131.418 MiB [ Info: Max. RSS: 1139.152 MiB [ Info: Max. RSS: 1146.629 MiB [ Info: Max. RSS: 1154.363 MiB [ Info: Max. RSS: 1161.840 MiB [ Info: Max. RSS: 1169.574 MiB [ Info: Max. RSS: 1177.309 MiB [ Info: Max. RSS: 1184.785 MiB [ Info: Max. RSS: 1192.520 MiB [ Info: Max. RSS: 1199.996 MiB [ Info: Max. RSS: 1207.730 MiB [ Info: Max. RSS: 1215.207 MiB [ Info: Max. RSS: 1222.941 MiB [ Info: Max. RSS: 1230.676 MiB [ Info: Max. RSS: 1238.152 MiB [ Info: Max. RSS: 1245.887 MiB [ Info: Max. RSS: 1253.363 MiB [ Info: Max. RSS: 1261.098 MiB [ Info: Max. RSS: 1268.832 MiB [ Info: Max. RSS: 1276.309 MiB [ Info: Max. RSS: 1284.043 MiB [ Info: Max. RSS: 1291.520 MiB [ Info: Max. RSS: 1299.254 MiB [ Info: Max. RSS: 1306.988 MiB [ Info: Max. RSS: 1314.465 MiB [ Info: Max. RSS: 1322.199 MiB [ Info: Max. RSS: 1329.676 MiB [ Info: Max. RSS: 1337.410 MiB [ Info: Max. RSS: 1345.145 MiB [ Info: Max. RSS: 1352.621 MiB [ Info: Max. RSS: 1360.355 MiB [ Info: Max. RSS: 1367.832 MiB [ Info: Max. RSS: 1375.566 MiB [ Info: Max. RSS: 1383.301 MiB [ Info: Max. RSS: 1390.777 MiB [ Info: Max. RSS: 1398.512 MiB [ Info: Max. RSS: 1405.988 MiB [ Info: Max. RSS: 1413.723 MiB [ Info: Max. RSS: 1421.199 MiB [ Info: Max. RSS: 1428.934 MiB [ Info: Max. RSS: 1436.668 MiB [ Info: Max. RSS: 1444.145 MiB [ Info: Max. RSS: 1451.879 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.602 MiB [ Info: Max. RSS: 1459.957 MiB [ Info: Max. RSS: 1467.691 MiB [ Info: Max. RSS: 1475.426 MiB [ Info: Max. RSS: 1482.902 MiB [ Info: Max. RSS: 1490.637 MiB [ Info: Max. RSS: 1498.113 MiB [ Info: Max. RSS: 1505.848 MiB [ Info: Max. RSS: 1513.582 MiB [ Info: Max. RSS: 1521.059 MiB [ Info: Max. RSS: 1528.793 MiB [ Info: Max. RSS: 1536.270 MiB [ Info: Max. RSS: 1544.004 MiB [ Info: Max. RSS: 1551.738 MiB [ Info: Max. RSS: 1559.215 MiB [ Info: Max. RSS: 1566.949 MiB [ Info: Max. RSS: 1574.426 MiB [ Info: Max. RSS: 1582.160 MiB [ Info: Max. RSS: 1589.895 MiB [ Info: Max. RSS: 1597.371 MiB [ Info: Max. RSS: 1605.105 MiB [ Info: Max. RSS: 1612.582 MiB [ Info: Max. RSS: 1620.316 MiB [ Info: Max. RSS: 1628.051 MiB [ Info: Max. RSS: 1635.527 MiB [ Info: Max. RSS: 1643.262 MiB [ Info: Max. RSS: 1650.738 MiB [ Info: Max. RSS: 1658.473 MiB [ Info: Max. RSS: 1665.949 MiB [ Info: Max. RSS: 1673.684 MiB [ Info: Max. RSS: 1681.418 MiB [ Info: Max. RSS: 1688.895 MiB [ Info: Max. RSS: 1696.629 MiB [ Info: Max. RSS: 1704.105 MiB [ Info: Max. RSS: 1711.840 MiB [ Info: Max. RSS: 1719.574 MiB [ Info: Max. RSS: 1727.051 MiB [ Info: Max. RSS: 1734.785 MiB [ Info: Max. RSS: 1742.262 MiB [ Info: Max. RSS: 1749.996 MiB [ Info: Max. RSS: 1757.730 MiB [ Info: Max. RSS: 1765.207 MiB [ Info: Max. RSS: 1772.941 MiB [ Info: Max. RSS: 1780.418 MiB [ Info: Max. RSS: 1788.152 MiB [ Info: Max. RSS: 1795.887 MiB [ Info: Max. RSS: 1803.363 MiB [ Info: Max. RSS: 1811.098 MiB [ Info: Max. RSS: 1818.574 MiB [ Info: Max. RSS: 1826.309 MiB [ Info: Max. RSS: 1834.043 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1841.676 MiB [ Info: Max. RSS: 1842.328 MiB [ Info: Max. RSS: 1850.062 MiB [ Info: Max. RSS: 1857.539 MiB [ Info: Max. RSS: 1865.273 MiB [ Info: Max. RSS: 1873.008 MiB [ Info: Max. RSS: 1880.484 MiB [ Info: Max. RSS: 1888.219 MiB [ Info: Max. RSS: 1895.695 MiB [ Info: Max. RSS: 1903.430 MiB [ Info: Max. RSS: 1910.906 MiB [ Info: Max. RSS: 1918.641 MiB [ Info: Max. RSS: 1926.375 MiB [ Info: Max. RSS: 1933.852 MiB [ Info: Max. RSS: 1941.586 MiB [ Info: Max. RSS: 1949.062 MiB [ Info: Max. RSS: 1956.797 MiB [ Info: Max. RSS: 1964.531 MiB [ Info: Max. RSS: 1972.008 MiB [ Info: Max. RSS: 1979.742 MiB [ Info: Max. RSS: 1987.219 MiB [ Info: Max. RSS: 1994.953 MiB [ Info: Max. RSS: 2002.688 MiB [ Info: Max. RSS: 2010.164 MiB [ Info: Max. RSS: 2017.898 MiB [ Info: Max. RSS: 2025.375 MiB [ Info: Max. RSS: 2033.109 MiB [ Info: Max. RSS: 2040.844 MiB [ Info: Max. RSS: 2048.320 MiB [ Info: Max. RSS: 2056.055 MiB [ Info: Max. RSS: 2063.531 MiB [ Info: Max. RSS: 2071.266 MiB [ Info: Max. RSS: 2079.000 MiB [ Info: Max. RSS: 2086.477 MiB [ Info: Max. RSS: 2094.211 MiB [ Info: Max. RSS: 2101.688 MiB [ Info: Max. RSS: 2109.422 MiB [ Info: Max. RSS: 2116.898 MiB [ Info: Max. RSS: 2124.633 MiB [ Info: Max. RSS: 2132.367 MiB [ Info: Max. RSS: 2139.844 MiB [ Info: Max. RSS: 2147.578 MiB [ Info: Max. RSS: 2155.055 MiB [ Info: Max. RSS: 2162.789 MiB [ Info: Max. RSS: 2170.523 MiB [ Info: Max. RSS: 2178.000 MiB [ Info: Max. RSS: 2185.734 MiB [ Info: Max. RSS: 2193.211 MiB [ Info: Max. RSS: 2200.945 MiB [ Info: Max. RSS: 2208.680 MiB [ Info: Max. RSS: 2216.156 MiB [ Info: Max. RSS: 2223.891 MiB [ Info: Max. RSS: 2231.367 MiB [ Info: Max. RSS: 2239.102 MiB [ Info: Max. RSS: 2246.836 MiB [ Info: Max. RSS: 2254.312 MiB [ Info: Max. RSS: 2262.047 MiB [ Info: Max. RSS: 2269.523 MiB [ Info: Max. RSS: 2277.258 MiB [ Info: Max. RSS: 2284.992 MiB [ Info: Max. RSS: 2292.469 MiB [ Info: Max. RSS: 2300.203 MiB [ Info: Max. RSS: 2307.680 MiB [ Info: Max. RSS: 2315.414 MiB [ Info: Max. RSS: 2323.148 MiB [ Info: Max. RSS: 2330.625 MiB [ Info: Max. RSS: 2338.359 MiB [ Info: Max. RSS: 2345.836 MiB [ Info: Max. RSS: 2353.570 MiB [ Info: Max. RSS: 2361.047 MiB [ Info: Max. RSS: 2368.781 MiB [ Info: Max. RSS: 2376.516 MiB [ Info: Max. RSS: 2383.992 MiB [ Info: Max. RSS: 2391.727 MiB [ Info: Max. RSS: 2399.203 MiB [ Info: Max. RSS: 2406.938 MiB [ Info: Max. RSS: 2414.672 MiB [ Info: Max. RSS: 2422.148 MiB [ Info: Max. RSS: 2429.883 MiB [ Info: Max. RSS: 2437.359 MiB [ Info: Max. RSS: 2445.094 MiB [ Info: Max. RSS: 2452.828 MiB [ Info: Max. RSS: 2460.305 MiB [ Info: Max. RSS: 2468.039 MiB [ Info: Max. RSS: 2475.516 MiB [ Info: Max. RSS: 2483.250 MiB [ Info: Max. RSS: 2490.984 MiB [ Info: Max. RSS: 2498.461 MiB [ Info: Max. RSS: 2506.195 MiB [ Info: Max. RSS: 2513.672 MiB [ Info: Max. RSS: 2521.406 MiB [ Info: Max. RSS: 2529.141 MiB [ Info: Max. RSS: 2536.617 MiB [ Info: Max. RSS: 2544.352 MiB [ Info: Max. RSS: 2551.828 MiB [ Info: Max. RSS: 2559.562 MiB [ Info: Max. RSS: 2567.039 MiB [ Info: Max. RSS: 2574.773 MiB [ Info: Max. RSS: 2582.508 MiB [ Info: Max. RSS: 2589.984 MiB [ Info: Max. RSS: 2597.719 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB [ Info: Max. RSS: 2605.441 MiB

In case of an error, please paste the full error message and stack trace.
@Yixiao-Zhang Yixiao-Zhang mentioned this issue Sep 11, 2024
Open
@Yixiao-Zhang
Copy link
Author

I suspect this originates from a bug in the netcdf-C library (or even libhdf5); possibly related to #Unidata/netcdf-c#2626). When I use the :netcdf3_classic format, there is no memory leak; using a lower version of libnetcdf (v4.8.1) also solves the issue.

@Alexander-Barth
Copy link
Owner

Thank you for preparing this well-done reproducer!

Indeed, here is an example where I can make Max. RSS increase by just using NetCDF_jll (independently from NCDatasets.jl):

using Printf
using NetCDF_jll: libnetcdf

function create_dummpy_netcdf()
    ncidp = Ref(Cint(0))
    path = "output.nc"
    mode = 0x1000 # NC_NETCDF4
    ccall((:nc_create,libnetcdf),Cint,(Cstring,Cint,Ptr{Cint}),path,mode,ncidp)

    ccall((:nc_close,libnetcdf),Cint,(Cint,),ncidp[])
end

function output_dummy_netcdf()
    ncidp = Ref(Cint(0))
    path = "output.nc"
    mode = 1
    ccall((:nc_open,libnetcdf),Cint,(Cstring,Cint,Ptr{Cint}),path,mode,ncidp)

    zeros(1000, 1000)

    ccall((:nc_close,libnetcdf),Cint,(Cint,),ncidp[])
end

create_dummpy_netcdf()

for i in 1:1000
    output_dummy_netcdf()

    # force garbage collection
    if i % 50 == 0
        GC.gc()
    end

    @info Printf.@sprintf "Max. RSS:  %9.3f MiB\n" Sys.maxrss()/2^20
end

This indeed confirms that the issues is probably in netCDF-C.

@Alexander-Barth Alexander-Barth mentioned this issue Sep 13, 2024
Open
@Yixiao-Zhang Yixiao-Zhang changed the title Possibly memory leak Possible memory leak Sep 13, 2024
@Yixiao-Zhang
Copy link
Author

Thank you for response! Your example clearly demonstrates that the bug is upstream (either netcdf-C or Julia).

I am still surprised that the memory leak is amplified by allocations between ccalls. The actual memory leak due to C function calls seems tiny (several MiBs in the example) and should not do much damage to the program. But when combined with allocations in Julia, Max. RSS may increase to tens of GiBs. What if what happens in libnetcdf was not memory leak but it needed to keep the allocated memory? It seems the memory management by ccall is potentially flawed.

@Alexander-Barth
Copy link
Owner

Alexander-Barth commented Sep 16, 2024 •
edited
Loading

Sorry, for cross-posting, but it is maybe better to put my reply here:

| I am still surprised that the memory leak is amplified by allocations between

Yes, I think that this is probably due to memory fragmentation (generated by the memory leak in netcdf-c). Julia needs a contiguous block of memory for its arrays. Even if netcdf-c leaks just a couple of bytes, such contiguous large blocks will be more and more difficult to find if they happen to spoil a a previously large contiguous block of memory (the memory allocated by zeros(Float64,1000, 1000)). Consider if this block of floats is surrounded by data structures of NetCDF and the leading structure is increased by a call to realloc after the block of floats is deallocated. The remaining block is no longer large enough for a new call to zeros(Float64,1000, 1000).

Of course, this is just a hypothesis :-)

@Yixiao-Zhang
Copy link
Author

Thank you for your response! The hypothesis of memory fragmentation makes sense to me. I still think that the memory management of Julia amplifies the memory leak.

For example, we can try replacing

zeros(Float64, 1000, 1000)

with

p = Libc.malloc(1000 * 1000 * sizeof(Float64))
Libc.free(p)

The two do very similar things. However, only zeros(Float64, 1000, 1000) triggers the spurious memory leak.

As mentioned in Unidata/netcdf-c#2626, the memory leak in the Python code is also tiny compared to the Julia version that I posted. This should also be the case for pure C code.

I regard this as an issue of Julia rather than NCDatasets. I am thinking of bringing this discussion of Julia discourse, because it is beyond my understanding right now.

@Yixiao-Zhang Yixiao-Zhang mentioned this issue Sep 17, 2024
Open
@ezaron
Copy link

ezaron commented Nov 30, 2024

Hi Yixiao and Alexander:

I am curious what your recommended workaround is? I've not looked inside the packages (NCDatasets.jl or NetCDF.jl) to see how the library version is controlled.

-Ed

@Alexander-Barth
Copy link
Owner

Alexander-Barth commented Dec 2, 2024 •
edited
Loading

I think it is preferable to not open/close repeatedly the same file, but rather open/close it once for a program and possibly use sync (even independently of this issue).

If you are using 1000s of different files, then I do not know a workaround.

In any case, I think it is better to ask for work-arounds in the upstream issue Unidata/netcdf-c#2626 . The fine folks at unidata have a better understanding of the C side (as the memory leak can be reproduced in pure C, indentently of julia).

To control the library version, have a look here:
https://alexander-barth.github.io/NCDatasets.jl/stable/issues/#Using-a-custom-NetCDF-library

For example:

# install these packages if necessary
using Preferences, NetCDF_jll

set_preferences!(NetCDF_jll, "libnetcdf_path" => "/path/to/libnetcdf.so.xyz")

This works for any BinaryBuilder JLL (including HDF5...).

If you find a combination of NetCDF/HDF5 which is not affected, please let us know :-)

@Alexander-Barth
Copy link
Owner

Maybe updating HDF5_jll solves the issue?

HDFGroup/hdf5#3396 (comment)

https://github.com/JuliaPackaging/Yggdrasil/blob/b028f2de66a1b360511d0e0b8e32b26b991c5277/H/HDF5/build_tarballs.jl#L9

@ezaron
Copy link

ezaron commented Dec 2, 2024

-- thank you. Yes, I have about 16k files to process. Simplest workaround was just to run a single executable 16 times to process 1K files per run, and separately merge the results afterwards.

@Yixiao-Zhang
Copy link
Author

I use NetCDF3, which does not depend on HDF5, as a workaround.

I remember downgrading netcdf-c to 4.8 worked for me. However, I have multiple machines with several different platforms, and thus using NetCDF3 would be a much simpler workaround for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
upstream bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ezaron @Alexander-Barth @Yixiao-Zhang