prototype: Parallelize tf unit tests #36294
Draft
+3,154
−325
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
liamcervante
reviewed
Jan 9, 2025
There was a problem hiding this comment.
Nice, I do like this as a starting point but I'd want to push it further. For example we can add provider and variable nodes into the new test graph as well so these also benefit from the up front computation that the runs are also getting.
But, we could definitely do that iteratively. So start with just the run blocks, then in a follow up PR add the variables, then add the providers, etc.
| dependency, ok := prevs[runName] | ||
| if !ok { | ||
| // TODO: should we catch this error, or leave it, as it will still be caught in the test? | ||
| return fmt.Errorf("dependency %q not found for run %q", runName, run.Name) |
There was a problem hiding this comment.
one of the advantages of moving to the graph is exactly this - we can catch these kind of errors right at the beginning and then report this as a kind of compilation error. So definitely leave this as is.
dsa0x
force-pushed
the
sams/tf-test-parallel
branch
from
7ded152 to
eae3add
Compare
dsa0x
force-pushed
the
sams/tf-test-parallel
branch
from
eae3add to
5811743
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #
WIP
Target Release
1.11.x
CHANGELOG entry