Skip to content

Commit

Permalink
test: _process_dataconfig_file
Browse files Browse the repository at this point in the history
Signed-off-by: Abhishek <[email protected]>
  • Loading branch information
Abhishek-TAMU committed Nov 25, 2024
1 parent 086b8a1 commit 5b17f66
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tests/data/test_data_preprocessing_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,7 @@ def test_validate_args_pretokenized(data_args, packing):
],
)
def test_process_dataconfig_file(data_config_path, data_path):
"""Ensure that the train/eval data are properly formatted based on the data args / text field"""
"""Ensure that datasets are formatted and validated correctly based on the arguments passed in config file."""
with open(data_config_path, "r") as f:
yaml_content = yaml.safe_load(f)
yaml_content["datasets"][0]["data_paths"][0] = data_path
Expand Down

0 comments on commit 5b17f66

Please sign in to comment.