Memory thrashing

[jessdecker]

With current main branch (after fixing the build error of not enough arguments for getLowestHash) I noticed that initial blocks sync very fast, but after maybe 20-30 seconds it slows to a crawl. When I hooked up the CPU and memory profiler, it appears to be spending roughly 50 percent of cpu time in garbage collection. Haven't figured out where all the thrashing or leaks are coming from yet, but wanted to raise the issue here for further visibility.

[tijno]

I too suspect some memory issues - the recent changes have pushed node memory requirements from 32gb to 64gb and some have reported even needed 90gb.

But - not sure the initial speed is related to this per se.

Are initial blocks not just fast because they are empty?

eg blocks 0-25 are about 800 tx per block
then upto 5k on avg just 1 tx per block
then form block 10k 1000s of tx per block
peak would be blocks on April 9th 2021 with about 644k tx in a single day about 2k per block average.
currently maybe 100-200 per block

Syncs have always been slow for first few blocks then super fast to about 10k in height.

This has not changed much in recent versions.

[jessdecker]

Thanks, good info. Will probably look at it more over the weekend.

[KevinTMitnick]

I too suspect some memory issues - the recent changes have pushed node memory requirements from 32gb to 64gb and some have reported even needed 90gb.

But - not sure the initial speed is related to this per se.

Are initial blocks not just fast because they are empty?

eg blocks 0-25 are about 800 tx per block then upto 5k on avg just 1 tx per block then form block 10k 1000s of tx per block peak would be blocks on April 9th 2021 with about 644k tx in a single day about 2k per block average. currently maybe 100-200 per block

Syncs have always been slow for first few blocks then super fast to about 10k in height.

This has not changed much in recent versions.

yes，I have this problem too