Fix Adapt calls, and remove device-side structs #2118
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In efforts to support conversion between cpu and gpu, we need the device-side objects to retain as much as possible so that we can convert back and forth without losing information.
This PR does a few things:
mpicomm
adapt_structure
to return device structs when called with aCUDA.KernelAdaptor
.This should hopefully work for all cases except for CUDA-MPI jobs, since adapting from CPU to GPU with
mpicomm
(a mutable object) cannot reside on the device. This limitation should hopefully be fixed by CliMA/ClimaComms.jl#102.Also, I noticed that the main branch has some typing issues. In particular:
does not apply to device-side objects, since those would need to be
(assuming that their type parameters are synchronized)