Beta14 and the new insane plotter! #436
Replies: 9 comments 26 replies
-
i feel like these new parameters need a 3D chart lol! very cool, thanks for the details, was a bit overwhelming this morning! |
Beta Was this translation helpful? Give feedback.
-
@scully_chia on keybase has pushed the limits down another 100MB successfully, to 4508 |
Beta Was this translation helpful? Give feedback.
-
i somehow managed to ruin it... sweet spot is not 4508.... new target is 4520... so |
Beta Was this translation helpful? Give feedback.
-
basicly, to achieve full uniform sorting on all possible buckets - the first bucket after F1 stage, needs to report higher than if you see this, its good:
on the other hand, if you see this, you going to hit
|
Beta Was this translation helpful? Give feedback.
-
I am still struggling with my i7-5820k (6 Core). Should I start the first k32 plot with -r 6 and stagger 60 minutes to the next plot or is it better to plot 6 plots with -r 2 each? |
Beta Was this translation helpful? Give feedback.
-
На что мой компьютер потянет: Какой именно из этих я смогу запустить и мой комп выдержит и не сгорит ! Подскажите пожалуйста, СПАСИБО!! 101.4GiB (k-32,временное пространство: 239GiB) |
Beta Was this translation helpful? Give feedback.
-
Why bother... in the last 3 days the TOTAL size of the network has gone up 1.5 times! Just another pyramid scheme... waste of time and energy! While the return is BINGO, lottery... P.off.. |
Beta Was this translation helpful? Give feedback.
-
Hello. |
Beta Was this translation helpful? Give feedback.
-
Здравствуйте! Hello! |
Beta Was this translation helpful? Give feedback.
-
the 3 new musketeers in the battle of the plots is
-r
-s
&-u
...-r
is threads... and its used only during phase 1, table 2-7...far from all the work in this phase is able to be run with multiple threads - the work that can be, will be... but as you can see on some of the reports that are slipping out, it is far from able to run on all threads constantly...
with my 4core 8thread i7, i was running
-r 8
, and got a total CPU for phase 1 between 230%-250% usage... so, far from 800%...a run on a 16core 32threads xeon, got it to 353.2% with
-r 32
... my philosophy on this subject is to use atleast-r
as many cores you got... if you are running parallel plots, and one plotter is already using them, the others are just going to have to share if they all want to use the cores at the same time - if they stagger in between each others usage, then thats just perfect...another way to look at it, is to keep 1 core clear for each instance of the plotter, as it at least uses 1 thread... so if you doing 12 parallel plots on a 16 core CPU, i guess a good
-r
choice would be 4 on each...-s
is the amount of cache for each thread - and it defaults to 65536 (64K) if you leave it out - ive tried both 32K and 128K, both at a small loss... most CPU's the last decade has had a L1 cache of 64K, so i think this is the best bet to go with... just dont worry about it, unless you want to take-s
on an adventure...-u
is the amount of buckets to run the process with - i think of it as datasets... it splits the entire work into pieces of-u
...-u
ranges from 16-32-64-128... as far as i understand it, more buckets will cause more random IO, which makes sense, as it works on either 16, 32, 64 or 128 files... as none of us testers have ran a single -t hdd run - its kinda up in the air how its going to perform on HDD's - but any kind of SSD does not care about seeking around in the flash chips...and this brings me to these 3 new musketeers little helper, their squire...
-b
-b
is now going to be defaulting to 3072 (3G) instead of the previous-b 2048
(2G) - it gives a good point for gaining speed, and still keeping the memory usage down... its not perfect, but its a pretty good point to go with... i have successfully run-b 2048
without problems, its just a bit slower..."but why is 2048 slower than 3072"...
its cause of the new brilliant work from the developers, that is now able to mimic running the entire plotting process like one big RAM drive... imagine getting the benefits of 400G of RAM, with just a tad more than 4G...
as you start plotting with the new plotter, you will see 2 different kind of sorting mechanisms in use - uniform sorting, which is the fast, all in memory sorting.... if it is able to fit the entire bucket into the allocated RAM, it will run the entire dataset in one go... if it does not have enough RAM to do so, it will resort to quicksorting - which is kinda like the old way of plotter sorting... not entirely... but somewhat... it will break down the dataset and use the drive while sorting the bucket...
the slow quicksorting, is still alot faster than any previous versions of the plotter... my tests shows that when you choke the new plotter from using memory, using as little as possible, you still going to see close to twice the speed...
and if you feed it all the RAM it could ever want, its more than x2 faster...
my testing is close to concluding that giving the new plotter
-b 4608
MiB of RAM, will allow it to run all the buckets using the faster uniform sorting algorithm, as long as you're maximizing the amount of-u
buckets to 128.... if you're going the other way about it, and choke it on the amount of -u buckets down to 16... which is an 8th of 128, you need x8 as much ram allocated for it to fully unfold it wings...-b 36864
(36G)turning this around, and asking how low you can go with memory - ive concluded a
-u 16 -b 10240
(10G) run, but almost all of the buckets results in the slower quicksorting... if you ran this with-u 128
, you should be able to lower-b
to something like 1500 or lower... i personally had a-b 1274
fail, cause it did not have enough RAM to do quicksort on a bucket... so that plot was wasted... i dont know if a-b 1280
would be able to complete everytime, as there is some cryptographic variations... so my recommendations for absolute lowest RAM usage so far, is-b 1500
... the presure point is phase 3, compressing table 5 & 6, second pass...oh, and i almost forgot... this new plotter also brings down the amount of writes needed to complete the task... and it doesnt matter if you choke it one way or the other... all my test runs have used just under 2TiB writes... down from 6.6 at worse case... so yippee-ki-yay for that!
disclaimer: im just a google warrior, all of the above is a matter of opinion, subject to bias, and is to be taken only as suggestions!
chia plots create -k 32 -b 4608 -u 128 -r 8 -t /SSD -2 /SSD -d /HDD
Beta Was this translation helpful? Give feedback.
All reactions