add benchmarks/u_rho #3

zsweger · 2024-01-13T21:01:07Z

Briefly, what does this PR introduce?

What kind of change does this PR introduce?

Bug fix (issue #__)
New feature (issue #__)
Documentation update
Other: __

Please check if this PR fulfills the following:

Tests for the changes have been added
Documentation has been added / updated
Changes have been communicated to collaborators

Does this PR introduce breaking changes? What changes might users need to make to their code?

No breaking changes.

Does this PR change default behavior?

This PR should create an automatic benchmark using u-channel rho production to benchmark the tracking performance of the B0 trackers

zsweger · 2024-01-13T21:17:37Z

I would like to make this into one of the automatic benchmarks which runs full simulations when there are detector or tracking updates, and then makes the plots which I've written here.
What I've written now runs over reconstruction files from one of the simulation campaigns.
Can the experts please help me dit this benchmark to automatically run the simulations and then process the results?
Also I have the benchmark produce some plots in the u_rho/figures directory. How can I convert these into artifacts?

veprbl · 2024-01-13T21:23:35Z

Current config.yml is suitable for detector_benchmarks, but not physics benchmarks:
https://eicweb.phy.anl.gov/EIC/benchmarks/physics_benchmarks/-/pipelines/78788

You'd need to look at https://github.com/eic/physics_benchmarks/blob/master/benchmarks/diffractive_vm/config.yml for reference, but it's at least a matter of replacing .det_benchmark with .phy_benchmark.

veprbl · 2024-01-13T21:24:53Z

Producing artifacts requires placing your output files to results directory. I think I haven't implemented this for diffractive_vm yet. Ideas are welcome.

zsweger · 2024-01-13T21:30:11Z

The config.yml file has been updated for the physics benchmarks

zsweger · 2024-01-13T21:34:46Z

Producing artifacts requires placing your output files to results directory. I think I haven't implemented this for diffractive_vm yet. Ideas are welcome.

Sorry for the naïve questions, but does this just mean I should rename the figures directory to results?

Updated config.yml to include simulate task

veprbl · 2024-01-13T22:34:53Z

Producing artifacts requires placing your output files to results directory. I think I haven't implemented this for diffractive_vm yet. Ideas are welcome.

Sorry for the naïve questions, but does this just mean I should rename the figures directory to results?

Yes.

veprbl · 2024-01-13T22:38:54Z

benchmarks/u_rho/config.yml

+  stage: collect
+  needs: ["u_channel_rho:simulate"]
+  script:
+    - cp benchmark_output/campaign_23.12.0_combined_60files_eicrecon.edm4eic.plots_figures/*.pdf figures/


One issue here is that the contents of the "current" directory is not preserved, only what's in the $LOCAL_DATA_PATH space. I think it would be simpler to move this line to the line after you call snakemake, and let gitlab output results/ directory as an artifact.

zsweger · 2024-01-13T23:03:35Z

Okay I edited the procedure to immediately create the results directory and copy to it.

Testing removing analysis compilation from u-channel rho pipeline

Reimplemented compilation step

Updated compile step in snakefile

Edited compile rule

Updated file paths

Updated compile path

zsweger · 2024-01-14T21:15:15Z

@veprbl , @wdconinc I don't know why this keeps failing at the compile step of the pipeline. Could you offer any insight?

veprbl · 2024-01-14T22:05:00Z

Should be a matter of including into the top-level Snakefile. I've pushed the changes, let's see if it works.

veprbl · 2024-01-14T22:13:47Z

Produces some pretty plots now: https://eicweb.phy.anl.gov/EIC/benchmarks/physics_benchmarks/-/jobs/2335547/artifacts/file/results/benchmark_rho_dNdu.pdf

zsweger · 2024-06-07T20:35:12Z

Understood. I've checked few things, but nothing stands out. Could you file this bug to https://github.com/eic/EICrecon/issues, please.

Done! eic/EICrecon#1500

Demonstration of benchmark for CA Consortium Meeting, using simulation campaign

add benchmarks/u_rho

4e084b5

zsweger marked this pull request as draft January 13, 2024 21:02

zsweger requested review from KongTu, wdconinc, veprbl and rahmans1 January 13, 2024 21:03

Update config.yml for phy benchmark

0f6d327

zsweger added 2 commits January 13, 2024 15:51

Updated config.yml to include results task

6841030

Update config.yml

91f2abe

Updated config.yml to include simulate task

veprbl reviewed Jan 13, 2024

View reviewed changes

Editing figures directory

b622c99

Zachary W Sweger and others added 7 commits January 13, 2024 18:45

Edited the compile rule in the Snakefile to actually compile the script

33eaf51

Update config.yml

b35ecd8

Testing removing analysis compilation from u-channel rho pipeline

Update config.yml

3890737

Reimplemented compilation step

Update Snakefile

543dc45

Updated compile step in snakefile

Update Snakefile

f44fd82

Edited compile rule

Update Snakefile

873a0ca

Updated file paths

Update Snakefile

0c2496c

Updated compile path

veprbl added 2 commits January 14, 2024 17:01

Snakefile: include benchmarks/u_rho/Snakefile

3f0bead

benchmarks/u_rho/Snakefile: change compile rule to use common definition

0d284a9

Test April campaign

ec78fcf

zsweger added 4 commits July 1, 2024 14:51

Test ReconstructedChargedRealPIDParticles

66fc5f1

Test on live data

95b18da

Test truth PID using chargedParticleAssociations

09db4a9

Test on 100 samples

c141582

veprbl mentioned this pull request Jul 3, 2024

benchmarks/tracking_performances: implement CI eic/detector_benchmarks#29

Merged

zsweger added 23 commits August 7, 2024 21:12

Saeahram's fix to uchannelrho.cxx

ae61bc9

Try 1000 events

17c9415

Test simulation campaign

102f7b8

Update to July sim campaign

fe4daee

Update Snakefile to July campaign

cb83559

Update z axis label sizes

e939bc5

July campaign has fewer files. Reduce to 45

ad0641f

Editing benchmark plot labels

a98485e

Update EPIC label

46d8e3f

Get rid of dN/dt plot

b4a6f2d

Test proton reco fix

3338459

Get rid of dN/dt plot second attempt

72d7324

Update bench status test

73064ef

Update bench plot labels

37a62d1

Update plots top margin

2fc026d

Final top margin fix

1700b77

Test bench with 5k events

2a7aa70

Hello California! (and John!)

4cd11cd

Demonstration of benchmark for CA Consortium Meeting, using simulation campaign

Hello tutorial attendees!

4e5830f

Hello tutorial attendees! :)

2dc2aed

Hello tutorial attendees!

69e8fc4

Try to trigger pipelines

10644c4

Trigger pipelines again

9f28e0c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add benchmarks/u_rho #3

add benchmarks/u_rho #3

zsweger commented Jan 13, 2024

zsweger commented Jan 13, 2024

veprbl commented Jan 13, 2024

veprbl commented Jan 13, 2024

zsweger commented Jan 13, 2024

zsweger commented Jan 13, 2024

veprbl commented Jan 13, 2024

veprbl Jan 13, 2024

zsweger commented Jan 13, 2024

zsweger commented Jan 14, 2024

veprbl commented Jan 14, 2024

veprbl commented Jan 14, 2024

zsweger commented Jun 7, 2024

add benchmarks/u_rho #3

Are you sure you want to change the base?

add benchmarks/u_rho #3

Conversation

zsweger commented Jan 13, 2024

Briefly, what does this PR introduce?

What kind of change does this PR introduce?

Please check if this PR fulfills the following:

Does this PR introduce breaking changes? What changes might users need to make to their code?

Does this PR change default behavior?

zsweger commented Jan 13, 2024

veprbl commented Jan 13, 2024

veprbl commented Jan 13, 2024

zsweger commented Jan 13, 2024

zsweger commented Jan 13, 2024

veprbl commented Jan 13, 2024

veprbl Jan 13, 2024

Choose a reason for hiding this comment

zsweger commented Jan 13, 2024

zsweger commented Jan 14, 2024

veprbl commented Jan 14, 2024

veprbl commented Jan 14, 2024

zsweger commented Jun 7, 2024