Skip to content

Commit

Permalink
Remove scc24 in the main docs
Browse files Browse the repository at this point in the history
  • Loading branch information
arjunsuresh committed Sep 23, 2024
1 parent 213c605 commit 6c23816
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 23 deletions.
40 changes: 20 additions & 20 deletions docs/benchmarks/text_to_image/reproducibility/scc24.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,45 +48,45 @@ or supporting multi-node execution) useful for the community and [MLCommons](htt
=== "MLCommons-Python"
## MLPerf Reference Implementation in Python

{{ mlperf_inference_implementation_readme (4, "sdxl", "reference", extra_variation_tags=",_short,_scc24-base", devices=["ROCm", "CUDA"],scenarios=["Offline"],categories=["Edge"], setup_tips=False) }}
{{ mlperf_inference_implementation_readme (4, "sdxl", "reference", extra_variation_tags=",_short,_scc24-base", devices=["ROCm", "CUDA"],scenarios=["Offline"],categories=["Datacenter"], setup_tips=False) }}

=== "Nvidia"
## Nvidia MLPerf Implementation
{{ mlperf_inference_implementation_readme (4, "sdxl", "nvidia", extra_variation_tags=",_short", scenarios=["Offline"],categories=["Edge"], setup_tips=False, implementation_tips=False) }}
{{ mlperf_inference_implementation_readme (4, "sdxl", "nvidia", extra_variation_tags=",_short", scenarios=["Offline"],categories=["Datacenter"], setup_tips=False, implementation_tips=False) }}

## Submission Commands

### Generate actual submission tree

```bash
cm run script --tags=generate,inference,submission \
--clean \
--preprocess_submission=yes \
--run-checker \
--submitter=<Team Name> \
--tar=yes \
--env.CM_TAR_OUTFILE=submission.tar.gz \
--division=open \
--category=datacenter \
--env.CM_DETERMINE_MEMORY_CONFIGURATION=yes \
--quiet
cm run script --tags=generate,inference,submission \
--clean \
--preprocess_submission=yes \
--run-checker \
--tar=yes \
--env.CM_TAR_OUTFILE=submission.tar.gz \
--division=open \
--category=datacenter \
--env.CM_DETERMINE_MEMORY_CONFIGURATION=yes \
--run_style=test --adr.submission-checker.tags=_short-run \
--quiet \
--submitter=<Team Name>
```

* Use `--hw_name="My system name"` to give a meaningful system name. Examples can be seen [here](https://github.com/mlcommons/inference_results_v3.0/tree/main/open/cTuning/systems)
* Use `--hw_name="My system name"` to give a meaningful system name.

* Use `--hw_notes_extra` option to add additional notes like `--hw_notes_extra="Result taken by NAME" `

<!-- ### Aggregate Results in GitHub
### Aggregate Results in GitHub

If you are collecting results across multiple systems you can generate different submissions and aggregate all of them to a GitHub repository (can be private) and use it to generate a single tar ball which can be uploaded to the [MLCommons Submission UI](https://submissions-ui.mlcommons.org/submission).

Run the following command after **replacing `--repo_url` with your GitHub repository URL**.

```bash
cm run script --tags=push,github,mlperf,inference,submission \
--repo_url=https://github.com/GATEOverflow/mlperf_inference_submissions_v4.1 \
--commit_message="Results on <HW name> added by <Name>" \
cm run script --tags=push,github,mlperf,inference,submission \
--repo_url=https://github.com/mlcommons/cm4mlperf-inference \
--repo_branch=mlperf-inference-results-scc24 \
--commit_message="Results on system <HW Name>" \
--quiet
```

At the end, you can download the github repo and upload to the [MLCommons Submission UI](https://submissions-ui.mlcommons.org/submission). -->
6 changes: 3 additions & 3 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -417,8 +417,8 @@ def mlperf_inference_run_command(spaces, model, implementation, framework, categ

if docker:
docker_cmd_suffix = f" \\\n{pre_space} --docker --quiet"
if "scc24" not in extra_variation_tags:
docker_cmd_suffix += f" \\\n{pre_space} --test_query_count={test_query_count} {extra_docker_input_string} {extra_input_string}"
if test_query_count > 0:
docker_cmd_suffix += f" \\\n{pre_space} --test_query_count={test_query_count}""
if extra_docker_input_string != "" or extra_input_string != "":
docker_cmd_suffix += f" \\\n{pre_space} {extra_docker_input_string} {extra_input_string}"
if "bert" in model.lower() and framework == "deepsparse":
Expand Down Expand Up @@ -456,7 +456,7 @@ def mlperf_inference_run_command(spaces, model, implementation, framework, categ
else:
cmd_suffix = f"\\\n{pre_space} --quiet {extra_input_string}"

if execution_mode == "test" and test_query_count > 0 and "scc24" not in extra_variation_tags:
if execution_mode == "test" and test_query_count > 0:
cmd_suffix += f" \\\n {pre_space} --test_query_count={test_query_count}"

if "bert" in model.lower() and framework == "deepsparse":
Expand Down

0 comments on commit 6c23816

Please sign in to comment.