You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is really important for people writing papers about Haskell who use Criterion to perform measurements. I'm helping a team that is using a modified version of Criterion to measure the energy consuption of benchmarks (we plan to file a pull request when it's done).
That sounds great. If you have suggestions or a PR, they would be welcome. We're trying to make a new wave of changes to this library, so now is a good time.
At the very least we could link some of the good blog posts that explain the linear regression methodology.
One thing I would like to clarify to users is the difference between taking the "Mean" and taking the "OLS regression". I have found that people are often not clear on that and it's not self explanatory.
Likewise the "y" and "iters" labels are hard to remember and not super accessible.
This is really important for people writing papers about Haskell who use Criterion to perform measurements.
+💯
One thing I would like to clarify to users is the difference between taking the "Mean" and taking the "OLS regression". I have found that people are often not clear on that and it's not self explanatory.
Also + 💯 . I have some clue about linear regression, but that makes sense if I'm plotting a dependent variable against an independent variable—which are the variables here? The dependent one should be runtime, but what's the independent variable?
Never got a clue until today. #122 suggests it's the number of iterations in a "single" measurement, something I've never seen in benchmarking frameworks. (Also, the paper cited in #122 seems to question the approach).
This is really important for people writing papers about Haskell who use Criterion to perform measurements. I'm helping a team that is using a modified version of Criterion to measure the energy consuption of benchmarks (we plan to file a pull request when it's done).
It's mainly to explain the statistical reasoning behind the
runBenchmark
function fromCriterion.Measurement
.The text was updated successfully, but these errors were encountered: