Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make holesky-test suite faster #154

Open
samlaf opened this issue Sep 27, 2024 · 5 comments · May be fixed by #160
Open

Make holesky-test suite faster #154

samlaf opened this issue Sep 27, 2024 · 5 comments · May be fixed by #160
Labels
optimization testing Anything to do with test suites

Comments

@samlaf
Copy link
Collaborator

samlaf commented Sep 27, 2024

testnet has 10 min batching intervals, which I think (?) is why the holesky-e2e tests currently take roughly 20 mins to run. preprod cluster has 1 min batching, which should reduce the ci time for that test.

holesky e2e test suite takes >15mins to run. Makes waiting for ci unbearably long, and devex experience pretty bad. We should find a way to optimize these tests.

@bxue-l2
Copy link
Collaborator

bxue-l2 commented Sep 29, 2024

preprod is unstable, it is more difficult to identify.

testnet now is 3 mins

@samlaf
Copy link
Collaborator Author

samlaf commented Sep 29, 2024

Indeed bowen just showed me testnet has been updated to 3 min batches: https://github.com/Layr-Labs/eigenda-devops/blob/master/charts/batcher/values/eigenda-testnet/us-east-1/holesky/values.yaml#L15

what does "difficult to identify" mean?
Do we have other tricks to accelerate this test then? perhaps not waiting for finality?

@bxue-l2
Copy link
Collaborator

bxue-l2 commented Sep 29, 2024

difficult to identify the root cause when there is error. Moreover, oncall has responsibility to fix testnet, whereas it is possible for preprod to dangle for a while

Finality is taking a larger chunk of time now, 3mins vs 12mins. But I think it makes sense to test for finality.

@samlaf samlaf linked a pull request Sep 29, 2024 that will close this issue
@samlaf
Copy link
Collaborator Author

samlaf commented Sep 29, 2024

Update: seems like e2e tests still took 14 mins without waiting for finality (action from #160)... so there's prob something else causing them to be slow.

@samlaf samlaf changed the title run holesky-test on preprod instead of testnet Make holesky-test suite faster Oct 8, 2024
@samlaf samlaf added testing Anything to do with test suites optimization labels Oct 8, 2024
@epociask
Copy link
Collaborator

@samlaf we could trying logging the waitForFinality var when its ingested to EigenDA Client to make sure its actually being set

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optimization testing Anything to do with test suites
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants