What strategy for parameter upload for public submission?

[mlocardpaulet]

How it works today:
Currently, when users want to submit their workflow output for public upload, they provide:

• the output of their analysis
• the associated parameter file
  These are kept in our server and made available as is to the community.
  Some parameters are also parsed from the parameter file and visible in the ProteoBench module main figure (and the table under it).

Problems with this strategy:

1. parameter files are all different, and do not always provide all the information that we'd like to retrieve.
2. some parameters that we do not automatically extract from the parameter file are of interest to users. Currently, they can add these to a free-text field, but maybe we can do better?
   Example of specific cases that we had to handle:
   a. ProlineStudio developers can use different software and parameters for peak list generation. This is not something we look for.
   b. In software tools that can use different search engines (i2MassChroQ, MSAngel), each seach engine specific set of parameters is reported in search-engine-specific ways
   c. Some software tools have hidden parameters that the user may know and could report manually if we asked, but can be tricky to find in parameter files (quick modifications in X!Tandem).
   Please provide more examples in this discussion if I forgot some
3. With the way we work today, I don't think that we can keep up and have a parameter parsing that will return homogenised and complete information.

Possible solution(s):
We propose fields where users manually write down important parameters.
If we do that, I think that we need to add a curation step when we review Proteobot PRs, to check that there is no obvious mistake (see other discussion on this topic: https://github.com/orgs/Proteobench/discussions/496).
We can also have an intermediate solution, where we parse parameter files and ask users to check the parsing result and fill in the blanks.
Please comment and/or propose other solution(s)

[enryH]

I would rather go for manipulating the parsed result. In the end we want to be able to allow for comparisons in modules deemed to be important parameters. If we add free fields, these would need to be ignored in the plots. More interesting is now that we expand the parameter parsing if these should also be used in all modules.

[mlocardpaulet]

I don't understand why you say that "If we add free fields, these would need to be ignored in the plots". They could be made visible in the plots, no? We could create one field per parameter that we deem important. And check at submission that they are correctly described (= correspond to what is in the parameter file associated to the public submission).

[enryH]

They can be made visible in the plots. But having structured data allows for further down-stream analysis across parameters set by software packages. Isn't the goal to compare settings which can be optimal across many software implementations?

[mlocardpaulet]

Yes, so you think that if we ask people to manually register their parameters in a form, it cannot be structured?

[enryH]

We will need to structure the form, and if we allow for open options to be typed, they won't be very comparable as a single typo will be hard to resolve.

If someone thinks a parameter is missing, then they should report it. We can allow for a limited number of open parameters. but eventually these should be picked up from the parameter files.