Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adapt fmi4cpp for distributed systems #152

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

prudhomm
Copy link
Contributor

@prudhomm prudhomm commented Oct 27, 2024

When deploying thousands of instances of the same FMU (Functional Mock-up Unit) in a distributed system using MPI (Message Passing Interface), it is inefficient for each MPI process to unzip the FMU individually. This pull request introduces an option to control whether the FMU should be unzipped or not. Typically, the MPI process with rank 0 will unzip the FMU once, and the other processes will wait until the FMU is ready to be loaded. This approach reduces redundant file I/O operations and improves initialization performance in large-scale distributed simulations.

@markaren
Copy link
Member

I think this PR can be simplified. If fmupath extension .fmu, unzip, if is_directory assume unzipped folder. No need to add the boolean flag.

@prudhomm
Copy link
Contributor Author

prudhomm commented Oct 27, 2024

@markaren the boolean is necessary if we let the fmu class load the fmu. the unzipFMU is there for the distributed to avoid unzipping hundreds or thousands of the same fmu.

so in parallel using MPI say, (or multithread for that matter)
rank 0 MPI process would

fmu::fmu(path, mpirank == 0 ); // only rank 0 unzips, the other will read the unzip directory and load the dll

does that make sense? The use case is really parallel computing.

@markaren
Copy link
Member

markaren commented Oct 27, 2024

I was thinking that the FMU was unzipped by the user prior. But that function may not be exposed through the public API. That could be a solution though. Expose an unzip function for the user to control. What would be the best solution?

@prudhomm
Copy link
Contributor Author

I think exposing the unzipping function would be more explicit and probably better in the end.
people might not understand why we provide this boolean flag.
so the path to the fmu would be in mpi the unzipped directory and fmu::fmu is kept simple.
However the name of the fmu path should be known by all MPI processes afterwards so that they load correctly

@markaren
Copy link
Member

However the name of the fmu path should be known by all MPI processes afterwards so that they load correctly

Yeah, a solution to this is needed. This PR has issues with that as well. Moving the issue to user-land lets the user deal with it.

@markaren
Copy link
Member

Anyhow, I exposed unzipper in 38799be

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Enhance FMU Constructor for Distributed MPI Environments
2 participants