[DAPHNE-#77] Avoid stack overflow in long-running DaphneDSL loops. #820
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- As described in #77, DaphneDSL loops with many iterations (e.g., >100k) crash DAPHNE. - The reason is that LowerToLLVMPass creates LLVM::AllocaOp in several places and for various purposes. - These AllocaOps could also be inside a loop, which means that the memory they allocate piles up until the stack is exhausted, because it is only freed at the end of the "scope" (function). - This commit solves the problem by ensuring that all AllocaOps created in LowerToLLVMPass are inserted at the beginning of the function surrounding the currently considered operation. - With that, there are no AllocaOps in loops anymore. - Note that the memory allocated by the AllocaOps can safely be reused by repeated kernel calls in different loop iterations. - Added script-level test cases employing loops with many iterations. - This bug has existed for a long time (partly because it wasn't a blocking issue until recently). - Thanks to @corepointer and @philipportner for investigating the causes (this fix is based upon their insights). - Thanks to @J-Brest and @borkob for providing a work-around for DAPHNE users. - Closes #77.
|
Awesome @pdamme |
|
I've been playing around with various memory profilers to test the issue and the solution. Stack usage looks quite normal with your changes applied. Also the explain_llvm output shows the improvements nicely. Thx for fixing this long standing bug at last @pdamme |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I think, in the future, we could even reduce the number of
AllocaOpsby making different kernel calls use the same memory slots for their result pointers. However, the most important goal for now is to get this bug fixed on main.