Skip to content

Commit

Permalink
add comment on benchmarking approach
Browse files Browse the repository at this point in the history
Signed-off-by: martinvuyk <[email protected]>
  • Loading branch information
martinvuyk committed Oct 23, 2024
1 parent 83979c2 commit aea3930
Showing 1 changed file with 13 additions and 2 deletions.
15 changes: 13 additions & 2 deletions stdlib/benchmarks/collections/bench_string.mojo
Original file line number Diff line number Diff line change
Expand Up @@ -223,8 +223,17 @@ fn bench_string_is_valid_utf8[
# ===----------------------------------------------------------------------===#
def main():
seed()
var m = Bench(BenchConfig(num_repetitions=1))
m.bench_function[bench_string_init](BenchId("bench_string_init"))
var m = Bench(BenchConfig(num_repetitions=5))
# NOTE: A proper way to run a benchmark like this is:
# 1. Run the benchmark on nightly branch with num_repetitions=5 and take the
# **median** value for each function, length, and language that is to be
# measured.
# 2. Then run the benchmark on num_repetitions=1 if you want faster results
# during development of your branch.
# 3. When ready to make statements about speed improvements, first run the
# benchmark again with num_repetitions=5 and take the **median** of that.
# 4. Make a table and report the new **median** numbers and the markdown
# percentage improvement over nightly version (new - nightly)/nightly.
alias filenames = (
"UN_charter_EN",
"UN_charter_ES",
Expand All @@ -236,6 +245,8 @@ def main():
alias new_chars = ("A", "Ó", "ل", "И", "")
alias lengths = (10, 30, 50, 100, 1000, 10_000, 100_000, 1_000_000)

m.bench_function[bench_string_init](BenchId("bench_string_init"))

@parameter
for i in range(len(lengths)):
alias length = lengths.get[i, Int]()
Expand Down

0 comments on commit aea3930

Please sign in to comment.