[#2300][Bug] Long tail tasks in the Write Stage retry phase results in data loss. #2301
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yl09099
changed the title
yl09099
force-pushed
the
uniffle-2300
branch
2 times, most recently
from
0c3ccff to
de38df3
Compare
yl09099
force-pushed
the
uniffle-2300
branch
from
de38df3 to
355f691
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2301 +/- ##
============================================
+ Coverage 52.27% 52.34% +0.07%
+ Complexity 3569 3362 -207
============================================
Files 543 513 -30
Lines 29797 27585 -2212
Branches 2779 2593 -186
============================================
- Hits 15575 14439 -1136
+ Misses 13227 12206 -1021
+ Partials 995 940 -55 ☔ View full report in Codecov by Sentry. |
zuston
reviewed
Jan 14, 2025
| @@ -118,6 +118,9 @@ public void reportShuffleWriteFailure( | |||
| // Clear the metadata of the completed task, otherwise some of the stage's data will | |||
| // be lost. | |||
| shuffleManager.unregisterAllMapOutput(shuffleId); | |||
| // Need to clear the mapStatus twice to prevent partition data loss due to the | |||
| // long-tail task performed before the stage retry. | |||
| shuffleManager.unregisterAllMapOutput(shuffleId); | |||
There was a problem hiding this comment.
Does it work? Just invoking twice in the same time point.
|
Sorry for the late reply, I'm busy for my work. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The long-tail task in the previous Stage caused data loss, and this bug needs to be fixed.
Why are the changes needed?
Fix: #2300
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Existential UT.