-
Notifications
You must be signed in to change notification settings - Fork 3
Output format for limited memory systems
Valentín Mari edited this page Jul 1, 2018
·
1 revision
In order to improve memory efficiency and to reach higher number of combinations, output Shared Arrays must be transform to local arrays and intermediate evaluation of output data should be done to keep a reduced number of "best results".
For example a standard i7 notebook usually have 8GiB of RAM. With this system a thirty covariate problem will use more than 800GiB of memory to return the whole set of results (imposible to reach even with 10 times the notebook RAM). However by keeping just the best one million results (through local arrays and intermediate evaluations, sortings and cleanings) we can work with just 500MiB of RAM (of course by loosing some performance).