ModelicaSystem.simulate() does not handle 'outputFormat' and 'variableFilter' options correctly.

[Nobby-n]

Overview

The argument '-override' of (model).exe has the following syntax

(model).exe -override=startTime=0,stopTime=5,stepSize=0.01,tolerance=1e-06,solver=dassl,mdl.param1=0.5,mdl.param2=2.0,outputFormat=csv,variableFilter=. *

ModelicaSystems.simulate() will give the '-override' option if parameters or simulation options are set using setParameters() or setSimulationOptions().

In this case, 'outputFormat' and 'variableFilter' cannot be specified.
To specify 'outputFormat' and 'variableFilter', I added options to the 'simflags' argument as below, but after using setParameters() or/and setSimulationOptions(), '-override' is double argument and simulate() does not work properly.

mod.simulate(simflags='-override=startTime=0,stopTime=5,stepSize=0.01,tolerance=1e-06,solver=dassl,outputFormat=csv,variableFilter=.*')

Operating Environment

Windows 10 Enterprise ver 20H2
OpenModelica 1.16.2 MSL3.2.3
Python 3.9.6
OMPython 3.3.0

Tentative measures

Do not use setParameters(), setSimulationOptions(), and specify all 'override' options using simFlags.

simulate() improvement suggestions

Add arguments to specify 'outputFormat' and 'variableFilter' and add code to append them to the '-override' statement.

The tested 'simulate()' modifications are as follows.

--- C:/venv39/jupyter-lab/Lib/site-packages/OMPython/__init__.py.org	Thu Mar 17 15:14:01 2022
+++ C:/venv39/jupyter-lab/Lib/site-packages/OMPython/__init__patch.py	Fri Apr  1 18:02:55 2022
@@ -1197,13 +1197,30 @@
             return ([self.optimizeOptions.get(x,"NotExist") for x in names])

     # to simulate or re-simulate model
-    def simulate(self,resultfile=None,simflags=None):  # 11
+    ##  override option add
+    # -def simulate(self,resultfile=None,simflags=None):  # 11
+    def simulate(self,resultfile=None,simflags=None,overrideaux=None):  # 11
         """
         This method simulates model according to the simulation options.
+
+        Parameters
+        ----------
+        resultfile : str or None
+            Output file name
+
+        simflags : str or None
+            Other simulation options not '-override' parameters
+
+        overrideaux : str or None
+            Specify 'outputFormat' and 'variableFilter
+
         usage
+        -----
         >>> simulate()
         >>> simulate(resultfile="a.mat")
         >>> simulate(simflags="-noEventEmit -noRestart -override=e=0.3,g=10) set runtime simulation flags
+        >>> simulate(simflags="-noEventEmit -noRestart" ,overrideaux="outputFormat=csv,variableFilter=.*")  # set runtime simulation flags
         """
         if(resultfile is None):
             r=""
@@ -1225,6 +1242,13 @@
             override =" -override=" + values1
         else:
             override =""
+
+        # add override flags not parameters or simulation options
+        if overrideaux:
+            if override:
+                override = override + "," + overrideaux
+            else:
+                override = " -override=" + overrideaux

         if (self.inputFlag):  # if model has input quantities
             for i in self.inputlist:

[adeas31]

You should use ModelicaSystem.resultfile to set a new output format. As for variableFilter, I am not sure how to set this.

@arun3688 is it possible to set this in ModelicaSystem?
@sjoelund is it possible to set variableFilter via simulation flags?

[arun3688]

@Nobby-n To set the result file format when using ModelicaSystem you can set the result file via simulate API, something like below

from OMPython import ModelicaSystem
mod=ModelicaSystem("BouncingBall.mo","BouncingBall")

mod.simulate() // method-1 default result file name will be used
mod.simulate(resultfile="tmpbouncingBall.mat")  // method-2 resultfile name provided by users
mod.simulate(simflags="-noEventEmit -noRestart -override=e=0.3,g=9.71") // method-3 simulationflags provided by users

You cannot use variable filter with ModelicaSystem class, but i will add this feature

[Nobby-n]

@adeas31 san, @arun3688 san
Thank you for your comment.

I understand how to use 'simulate ()' as in your comment.
I think 'method-3' is a good way to use all the '-override' options.
However, in this case I cannot use 'setParameters ()' and 'setSimulationOptions ()' at the same time to specify 'outputFormat' and 'variableFilter'.
I would be happy if 'simulate ()' could be improved in the future.

[arun3688]

@Nobby-n It is not possible to use variable filter as a runtime simulation flags, but however you can use the getSolutions API to filter the results, something like below

>>> from OMPython import ModelicaSystem
>>> mod = ModelicaSystem("C:/OPENMODELICAGIT/OpenModelica/OMCompiler/Examples/BouncingBall.mo", "BouncingBall")
>>> mod.simulate()
LOG_SUCCESS       | info    | The initialization finished successfully without homotopy method.
LOG_SUCCESS       | info    | The simulation finished successfully.
>>> mod.getSolutions() 
('der(h)', 'der(v)', 'e', 'flying', 'foo', 'g', 'h', 'impact', 'time', 'v', 'v_new')
>>> mod.getSolutions(["h", "g"]); ## filter the results 

[Nobby-n]

@arun3688 san

I'd confirmed that I can get only the variables I need using getSolutions() API.

One of the reasons I want to use variableFilter in simulate() API is that I can reduce the file size of the mat file.

For example, in the Modelica.Fluid.Examples.PumpingSystem included in the MSL, I'd confirmed that by specifying variableFilter as follows, the variables output to the mat file can be reduced.

simulate(simflags="variableFilter=userValve.m_flow")

[arun3688]

@Nobby-n Now the variable filter option is added to ModelicaSystem constructor with this commit 2284229, you can use it in following ways ,

>>> from OMPython import ModelicaSystem
>>> omc = ModelicaSystem("BouncingBall.mo", "BouncingBall", variableFilter="impact|h")

(or)
 
>>> from OMPython import ModelicaSystem
>>> omc = ModelicaSystem("BouncingBall.mo", "BouncingBall")
>>> omc.buildModel(variableFilter="h")

You can directly use it in ModelicaSystem() Constructor or you can use it in buildModel()

[joewa]

The approach from @Nobby-n applies the customization (variableFilter or overrides in general) when calling the pre-built model executable whereas the current implementation of @arun3688 applies the variableFilter before when calling buildModel.

I think both approaches are interesting! There might be use cases where calling buildModel each time ModelicaSystem is instantiated is not efficient, e.g. when only the parameters or the input data changes between different simulation runs.

It would be necessary to tell ModelicaSystem not to call buildModel then, e.g. by providing the xmlFileName of an existing model.

    mod = ModelicaSystem(
            fileName=`BouncingBall.mo`, modelName='BouncingBall',
            xmlFileName='BouncingBall_init.xml'
        )

I would appreciate if we can have this in OMPython.

I have already integrated this proposal in my fork of OMPython and I am ready to send a pull request.

What do you think?

[arun3688]

@Nobby-n Please make a PR

[Nobby-n]

@joewa, @arun3688
Thank you for comments.

I'd saw @joewa 's fork.
@joewa, Please send a pull request.
I agree your proposal.

[joewa]

Please note my pull request #165 @arun3688 @Nobby-n
(I don"t know if you are notified by default)

[arun3688]

@joewa yes your PR is noted

[Nobby-n]

@joewa Me too, your pull request is notified.