Code

Ab tensorize speed up#552

Code

Merged

abaumann merged 5 commits intomasterbroadinstitute/ml4h:masterfrom ab_tensorize_speed_upbroadinstitute/ml4h:ab_tensorize_speed_upCopy head branch name to clipboardFeb 8, 2024

+123-40

Commits

Commits on Feb 6, 2024

• Changed parallelization within tensorize.sh so it runs in parallel within docker rather than parallel docker - about a 10x speed up. Added hdf5-tools to the docker for testing the hd5 files. Added …

  Show description for fe93385
  abaumann

  committed
  fe93385

  Copy full SHA for fe93385

• Made parallel use the NUM_JOBS flag to see how many run in parallel.

  abaumann

  committed
  46c9826

  Copy full SHA for 46c9826

• Fixed defaults for min and max sample id when not provided.

  abaumann

  committed
  f911a0d

  Copy full SHA for f911a0d

Commits on Feb 7, 2024

• Fixed permissions on validate_tensors.sh so it is executable

  abaumann

  committed
  8f1a205

  Copy full SHA for 8f1a205

Commits on Feb 8, 2024

• Removed redundant line in tensorize.sh and update documentation about scaling instance types and debugging tensorization.

  abaumann

  committed
  b96c3a2

  Copy full SHA for b96c3a2