[DAR-3513][External] Addition of the item_merge_mode
push
argument
#920
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds the
item_merge_mode
argument topush()
:item_merge_mode: Enum["slot", "series", "channel"]
1: It is available through both CLI & SDK-initiated
push()
with this syntax:darwin dataset push /path/to/files --item_merge_mode [slots | series | channels]
2: Raises an error if
preserve_folders
is set:dataset.push(files_to_upload=files_to_upload, item_merge_mode=item_merge_mode, preserve_folders=True)
3: Throws 1 non-blocking warning for each instance of:
push()
that references a specific file rather than a directory, since these are ignoreditem_merge_mode
rules, results in an empty directoryTechnical Implementation
1: Forks the existing parsing logic so that it's only run if
item_merge_mode
isn't set2: If
item_merge_mode
is set, use parsing logic that creates 1MultiFileItem
object (new class) per dataset item. Run validation & construct + attach a Layout object to eachMultiFileItem
Changes beyond this point would relate to uploading itself, so are beyond the scope of this ticket, and will be coming in the next couple of weeks