Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add workflow to measure execute time #789

Merged
merged 108 commits into from
Sep 19, 2024
Merged

add workflow to measure execute time #789

merged 108 commits into from
Sep 19, 2024

Conversation

FrancoGiachetta
Copy link
Contributor

@FrancoGiachetta FrancoGiachetta commented Sep 11, 2024

Description

This PR adds a workflow to compare how much time does it take to native to execute a program with the main branch and the branch pointed by the PR.

Referes #786

Checklist

  • Linked to Github Issue
  • Unit tests added
  • Integration tests added.
  • This change requires new documentation.
    • Documentation has been added/updated.

@FrancoGiachetta FrancoGiachetta changed the title add workflow for linux add workflow to measure execute time Sep 11, 2024
@codecov-commenter
Copy link

codecov-commenter commented Sep 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.69%. Comparing base (8faf10f) to head (7d00911).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #789   +/-   ##
=======================================
  Coverage   88.69%   88.69%           
=======================================
  Files         121      121           
  Lines       35512    35512           
=======================================
  Hits        31499    31499           
  Misses       4013     4013           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lambdaclass lambdaclass deleted a comment from github-actions bot Sep 12, 2024
@lambdaclass lambdaclass deleted a comment from github-actions bot Sep 12, 2024
@lambdaclass lambdaclass deleted a comment from github-actions bot Sep 12, 2024
@lambdaclass lambdaclass deleted a comment from github-actions bot Sep 12, 2024
Copy link

github-actions bot commented Sep 18, 2024

Benchmark results base v head

Command Mean [s] Min [s] Max [s] Relative
head fib_2M (JIT) 2.843 ± 0.027 2.812 2.879 1.02 ± 0.01
base fib_2M (JIT) 2.799 ± 0.031 2.753 2.851 1.00 ± 0.01
head fib_2M (AOT) 2.800 ± 0.027 2.771 2.862 1.00 ± 0.01
base fib_2M (AOT) 2.787 ± 0.019 2.766 2.822 1.00

Copy link

github-actions bot commented Sep 18, 2024

Benchmark results base v head

Command Mean [s] Min [s] Max [s] Relative
head factorial_2M.cairo (JIT) 3.431 ± 0.027 3.389 3.467 1.01 ± 0.01
base factorial_2M.cairo (JIT) 3.416 ± 0.040 3.352 3.489 1.00 ± 0.02
head factorial_2M.cairo (AOT) 3.402 ± 0.039 3.357 3.484 1.00
base factorial_2M.cairo (AOT) 3.411 ± 0.032 3.359 3.454 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
head fib_2M.cairo (JIT) 2.902 ± 0.038 2.838 2.952 1.00
base fib_2M.cairo (JIT) 2.916 ± 0.044 2.855 2.986 1.00 ± 0.02
head fib_2M.cairo (AOT) 2.928 ± 0.025 2.891 2.964 1.01 ± 0.02
base fib_2M.cairo (AOT) 2.929 ± 0.017 2.896 2.957 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
head logistic_map.cairo (JIT) 3.360 ± 0.033 3.318 3.422 1.04 ± 0.02
base logistic_map.cairo (JIT) 3.378 ± 0.072 3.266 3.463 1.05 ± 0.03
head logistic_map.cairo (AOT) 3.263 ± 0.026 3.218 3.294 1.01 ± 0.02
base logistic_map.cairo (AOT) 3.232 ± 0.059 3.144 3.302 1.00

Copy link

github-actions bot commented Sep 18, 2024

Benchmark results Main vs HEAD

Command Mean [s] Min [s] Max [s] Relative
head factorial_2M.cairo (JIT) 3.219 ± 0.036 3.172 3.291 1.01 ± 0.01
base factorial_2M.cairo (JIT) 3.200 ± 0.022 3.170 3.237 1.00
head factorial_2M.cairo (AOT) 3.257 ± 0.038 3.220 3.334 1.02 ± 0.01
base factorial_2M.cairo (AOT) 3.210 ± 0.021 3.174 3.248 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
head fib_2M.cairo (JIT) 2.734 ± 0.028 2.690 2.765 1.00 ± 0.02
base fib_2M.cairo (JIT) 2.733 ± 0.029 2.694 2.787 1.00 ± 0.02
head fib_2M.cairo (AOT) 2.799 ± 0.059 2.704 2.884 1.03 ± 0.03
base fib_2M.cairo (AOT) 2.729 ± 0.036 2.676 2.780 1.00
Command Mean [s] Min [s] Max [s] Relative
head logistic_map.cairo (JIT) 3.140 ± 0.015 3.116 3.157 1.04 ± 0.01
base logistic_map.cairo (JIT) 3.173 ± 0.018 3.135 3.198 1.05 ± 0.01
head logistic_map.cairo (AOT) 3.015 ± 0.036 2.970 3.092 1.00
base logistic_map.cairo (AOT) 3.055 ± 0.039 2.985 3.105 1.01 ± 0.02

@edg-l edg-l added the review-ready A PR that is ready for review label Sep 19, 2024
@edg-l edg-l added this pull request to the merge queue Sep 19, 2024
Merged via the queue into main with commit 7faf271 Sep 19, 2024
15 checks passed
@edg-l edg-l deleted the workflow-native-vm branch September 19, 2024 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review-ready A PR that is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants