Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find optimal number of runs for fuzz/invariant tests #943

Closed
rndquu opened this issue Jun 25, 2024 · 11 comments · Fixed by #967
Closed

Find optimal number of runs for fuzz/invariant tests #943

rndquu opened this issue Jun 25, 2024 · 11 comments · Fixed by #967

Comments

@rndquu
Copy link
Member

rndquu commented Jun 25, 2024

Depends on #563

What should be done:

  • when Invariant Testing #563 is ready find out a number of runs so that github CI could run for up to 6 hours (github CI max limit)
  • for already implemented fuzzing tests also find out a number of runs so that github CI could run for up to 6 hours
@zugdev
Copy link
Contributor

zugdev commented Sep 24, 2024

/start

@zugdev
Copy link
Contributor

zugdev commented Sep 24, 2024

Are the workflows properly setup? It doesn't feel like it is.

@0x4007

This comment has been minimized.

This comment has been minimized.

@0x4007

This comment has been minimized.

Copy link

ubiquity-os bot commented Sep 25, 2024

@zugdev the deadline is at Wed, Sep 25, 4:20 AM UTC

@gentlementlegen
Copy link
Member

@0x4007 yes we had a fix for that, kernel and plug-in side.

@zugdev
Copy link
Contributor

zugdev commented Sep 25, 2024

I meant fuzz and invariant workflows though, it seems at least that foundry.toml profiles are wrong. I will check that tomorrow.

@rndquu
Copy link
Member Author

rndquu commented Sep 27, 2024

I meant fuzz and invariant workflows though, it seems at least that foundry.toml profiles are wrong. I will check that tomorrow.

What exactly is wrong?

@zugdev
Copy link
Contributor

zugdev commented Sep 29, 2024

It's not wrong, forget about that.

Copy link

ubiquity-os bot commented Oct 1, 2024

 [ 76.069 WXDAI ] 

@zugdev
Contributions Overview
ViewContributionCountReward
IssueTask175
IssueComment31.069
ReviewComment20
Conversation Incentives
CommentFormattingRelevanceReward
Are the workflows properly setup? It doesn't feel like it is.
0.83
content:
  content:
    p:
      score: 0
      elementCount: 1
  result: 0
regex:
  wordCount: 12
  wordValue: 0.1
  result: 0.83
0.30.249
I meant fuzz and invariant workflows though, it seems at least t…
1.28
content:
  content:
    p:
      score: 0
      elementCount: 1
  result: 0
regex:
  wordCount: 20
  wordValue: 0.1
  result: 1.28
0.60.768
It's not wrong, forget about that.
0.52
content:
  content:
    p:
      score: 0
      elementCount: 1
  result: 0
regex:
  wordCount: 7
  wordValue: 0.1
  result: 0.52
0.10.052
Resolves #943
0
content:
  content:
    p:
      score: 0
      elementCount: 1
  result: 0
regex:
  wordCount: 2
  wordValue: 0
  result: 0
0.3-
You can check at my fork's [actions](https://github.com/zugdev/u…
78.42
content:
  content:
    p:
      score: 0
      elementCount: 11
    a:
      score: 5
      elementCount: 12
    h2:
      score: 1
      elementCount: 2
    ol:
      score: 1
      elementCount: 2
    li:
      score: 0.5
      elementCount: 10
  result: 69
regex:
  wordCount: 93
  wordValue: 0.2
  result: 9.42
0.9-

 [ 0 WXDAI ] 

@0x4007
Contributions Overview
ViewContributionCountReward
Conversation Incentives
CommentFormattingRelevanceReward

 [ 0.088 WXDAI ] 

@gentlementlegen
Contributions Overview
ViewContributionCountReward
IssueComment10.088
Conversation Incentives
CommentFormattingRelevanceReward
@0x4007 yes we had a fix for that, kernel and plug-in side.
0.88
content:
  content:
    p:
      score: 0
      elementCount: 1
  result: 0
regex:
  wordCount: 13
  wordValue: 0.1
  result: 0.88
0.10.088

 [ 47.045 WXDAI ] 

@rndquu
Contributions Overview
ViewContributionCountReward
IssueSpecification146.98
IssueComment10.065
Conversation Incentives
CommentFormattingRelevanceReward
Depends on https://github.com/ubiquity/ubiquity-dollar/issues/56…
15.66
content:
  content:
    p:
      score: 0
      elementCount: 4
    ul:
      score: 1
      elementCount: 1
    li:
      score: 0.5
      elementCount: 2
    a:
      score: 5
      elementCount: 2
  result: 12
regex:
  wordCount: 69
  wordValue: 0.1
  result: 3.66
146.98
What exactly is wrong?
0.65
content:
  content:
    p:
      score: 0
      elementCount: 1
  result: 0
regex:
  wordCount: 4
  wordValue: 0.2
  result: 0.65
0.10.065

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment