Skip to content

Commit

Permalink
Remove NetCDF from parallel CI test
Browse files Browse the repository at this point in the history
Julia's package-manager-provided NetCDF_jll.jl now links to the
package-manager-provided MPI. That means that if we use NetCDF in
parallel, we have to use a package-manager-provided MPI to avoid weird
errors due to multiple MPI libraries being linked. However, the default
MPI is MPICH, and we have to use OpenMPI for the CI test to avoid
horrible performance when oversubscribing (we do oversubscribe in the
parallel test). It seems like too much of a faff to switch the MPI, and
figure out how to call the package-manager-provided MPI, so instead just
disable NetCDF (which we never use anyway).

This issue may be fixed one day when NCDatasets.jl supports MPI, as it
will (should?) provide functionality to link a system-provided NetCDF
instead of the package-manager-provided one, but that functionality is
still underdevelopment
(Alexander-Barth/NCDatasets.jl#122).
  • Loading branch information
johnomotani committed Sep 16, 2024
1 parent 0458ab0 commit c2b9bbd
Showing 1 changed file with 6 additions and 6 deletions.
12 changes: 6 additions & 6 deletions .github/workflows/parallel_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,16 @@ jobs:
- run: |
touch Project.toml
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.add(["MPI", "MPIPreferences"]); using MPIPreferences; MPIPreferences.use_system_binary()'
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.add(["NCDatasets", "Random", "SpecialFunctions", "Test"]); Pkg.develop(path="moment_kinetics/")'
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.add(["Random", "SpecialFunctions", "Test"]); Pkg.develop(path="moment_kinetics/")'
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.precompile()'
# Need to use openmpi so that the following arguments work:
# * `--mca rmaps_base_oversubscribe 1` allows oversubscription (more processes
# than physical cores).
# * `--mca mpi_yield_when_idle 1` changes a setting to prevent excessively
# terrible performance when oversubscribing.
mpiexec -np 3 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1 --force-optional-dependencies
mpiexec -np 4 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1 --force-optional-dependencies
mpiexec -np 2 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1 --long --force-optional-dependencies
mpiexec -np 3 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1
mpiexec -np 4 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1
mpiexec -np 2 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1 --long
# Note: MPI.jl's default implementation is mpich, which has a similar option
# `--with-device=ch3:sock`, but that needs to be set when compiling mpich.
shell: bash
Expand All @@ -57,14 +57,14 @@ jobs:
export MPILIBPATH=$(find /opt/homebrew/Cellar/open-mpi/ -name libmpi.dylib)
touch Project.toml
julia --project -O3 --check-bounds=no -e "import Pkg; Pkg.add([\"MPI\", \"MPIPreferences\"]); using MPIPreferences; MPIPreferences.use_system_binary(library_names=\"$MPILIBPATH\")"
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.add(["NCDatasets", "Random", "SpecialFunctions", "Test"]); Pkg.develop(path="moment_kinetics/")'
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.add(["Random", "SpecialFunctions", "Test"]); Pkg.develop(path="moment_kinetics/")'
julia --project -O3 --check-bounds=no -e 'import Pkg; Pkg.precompile()'
# Need to use openmpi so that the following arguments work:
# * `--mca rmaps_base_oversubscribe 1` allows oversubscription (more processes
# than physical cores).
# * `--mca mpi_yield_when_idle 1` changes a setting to prevent excessively
# terrible performance when oversubscribing.
mpiexec -np 4 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1 --force-optional-dependencies
mpiexec -np 4 --mca rmaps_base_oversubscribe 1 julia --project -O3 --check-bounds=no moment_kinetics/test/runtests.jl --debug 1
# Note: MPI.jl's default implementation is mpich, which has a similar option
# `--with-device=ch3:sock`, but that needs to be set when compiling mpich.
shell: bash

0 comments on commit c2b9bbd

Please sign in to comment.