-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merkle performance improvements #233
Commits on Apr 21, 2024
-
Cherry-Picked from OffchainLabs/nitro@flatmerkleapril16
Configuration menu - View commit details
-
Copy full SHA for a149ebd - Browse repository at this point
Copy the full SHA a149ebdView commit details -
Configuration menu - View commit details
-
Copy full SHA for dec2e82 - Browse repository at this point
Copy the full SHA dec2e82View commit details -
Add the total time for each step-size.
It can be easy to look at small average step and hash times and miss that the total time is what we're really trying to reduce.
Configuration menu - View commit details
-
Copy full SHA for 3c30265 - Browse repository at this point
Copy the full SHA 3c30265View commit details -
Add some doc comments and tests to merkle.rs
I definitely had some incorrect assumptions about this data structure which made it more difficult to learn. So, I'm documenting how it works and adding some tests. The simple_merkle test is currently failing because the `set` method doesn't allow setting an index larger than the largest currently set leaf's index. There is some debate as to whether or not this is the correct behavior. To run the test, use: ``` $> cargo test -- --include-ignored ```
Configuration menu - View commit details
-
Copy full SHA for 51069f5 - Browse repository at this point
Copy the full SHA 51069f5View commit details -
Allow callers to extend the Merkle Tree by adding leaves.
At this point, the new root hash is eagerly calculated after each call to `extend`.
Configuration menu - View commit details
-
Copy full SHA for 2ecc4f5 - Browse repository at this point
Copy the full SHA 2ecc4f5View commit details -
Extend the memory merkle instead of clearing it.
If this happened frequently, it should really improve the perfomance of the machine. However, it looks like it doesn't happen at all with the benchmark inputs.
Configuration menu - View commit details
-
Copy full SHA for 05b157f - Browse repository at this point
Copy the full SHA 05b157fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 965a202 - Browse repository at this point
Copy the full SHA 965a202View commit details -
Configuration menu - View commit details
-
Copy full SHA for 477e49c - Browse repository at this point
Copy the full SHA 477e49cView commit details
Commits on Apr 22, 2024
-
Fix the implementation of extend.
Previously, it could hit an index out of bounds if the new leafs caused any parent layer to grow beyond its current size.
Configuration menu - View commit details
-
Copy full SHA for cb1d10c - Browse repository at this point
Copy the full SHA cb1d10cView commit details -
Add criterion benchmark for a big merkle tree.
Hopefully, this will allow us to compare this branch's implementation of a merkle tree to the one on merkle-perf-a.
Configuration menu - View commit details
-
Copy full SHA for e644b89 - Browse repository at this point
Copy the full SHA e644b89View commit details
Commits on Apr 23, 2024
-
Include the creation of a fresh tree for each iteration.
The previous implementation was growing the same layers and dirty_indices arrays because the clone isn't deep (I guess.)
Configuration menu - View commit details
-
Copy full SHA for e8868a8 - Browse repository at this point
Copy the full SHA e8868a8View commit details
Commits on Apr 24, 2024
-
Add some profiling and instrumentation code.
There are a few different things going on in this commit. 1. I've added some counters for when methods get called on the Merkle tree. 2. I've added integration with gperftools for profiling specific areas of the code.
Configuration menu - View commit details
-
Copy full SHA for 1f52875 - Browse repository at this point
Copy the full SHA 1f52875View commit details
Commits on Apr 25, 2024
-
This allows me to profile CPU and Heap independently, and to enable and disable the call counters independently.
Configuration menu - View commit details
-
Copy full SHA for 976cb36 - Browse repository at this point
Copy the full SHA 976cb36View commit details
Commits on Apr 26, 2024
-
Add a benchmark for new_advanced.
This part of the code is obviously slow. Let's see if we can improve it.
Configuration menu - View commit details
-
Copy full SHA for 5e14a2b - Browse repository at this point
Copy the full SHA 5e14a2bView commit details -
Actually set the cached merkle back on the instance.
This is why there were all those unexpected "new_advanced" calls on the memory merkle. The resizes were actually setting self.merkle back to None.
Configuration menu - View commit details
-
Copy full SHA for 2f2e173 - Browse repository at this point
Copy the full SHA 2f2e173View commit details -
Configuration menu - View commit details
-
Copy full SHA for fc16ec3 - Browse repository at this point
Copy the full SHA fc16ec3View commit details
Commits on Apr 29, 2024
-
Update the logic for expand to include upper layers.
There was a bug where expanding the lowest layer and calling set on all of the new elements was not sufficient to grow the upper layers. This commit also fixes a warning about the package-level profile override being ineffective.
Configuration menu - View commit details
-
Copy full SHA for 1a52847 - Browse repository at this point
Copy the full SHA 1a52847View commit details
Commits on Apr 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 72bc92f - Browse repository at this point
Copy the full SHA 72bc92fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6f7cafa - Browse repository at this point
Copy the full SHA 6f7cafaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6a3c245 - Browse repository at this point
Copy the full SHA 6a3c245View commit details -
Configuration menu - View commit details
-
Copy full SHA for 36578d7 - Browse repository at this point
Copy the full SHA 36578d7View commit details -
Add the fake stuff for the benchmarks to the Dockerfile.
I have no idea why this is needed. But, it makes `make docker` successful again.
Configuration menu - View commit details
-
Copy full SHA for 97d93d5 - Browse repository at this point
Copy the full SHA 97d93d5View commit details -
Configuration menu - View commit details
-
Copy full SHA for e3d7cf4 - Browse repository at this point
Copy the full SHA e3d7cf4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 521a7d8 - Browse repository at this point
Copy the full SHA 521a7d8View commit details
Commits on May 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for cf50743 - Browse repository at this point
Copy the full SHA cf50743View commit details -
Turn off always_merkelize for now.
The system tests are timing out because the implementation is still too slow for large steps with lots of store and resize memory calls.
Configuration menu - View commit details
-
Copy full SHA for cc71b32 - Browse repository at this point
Copy the full SHA cc71b32View commit details
Commits on May 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 77866ec - Browse repository at this point
Copy the full SHA 77866ecView commit details