Use Hydra multirun mode to do optimization tasks

[ri-heme]

The hyperparameter optimization can be re-adapted to make use of Hydra's multirun mode (and/or sweeper).

Overview of tasks:

Below a list of tasks to convert these two modules from its current state:

Module and schema

• Define new schemas for this task: OptimizeTaskConfig (inherits from TaskConfig) class in move.conf.schema.
• Create new move.tasks.optimize_hyperparameters module and function.
• Create example experiment config for hyperparameter search. See example YAML in hydra-app-example.

Function

For the function optimize_hyperparamters:

• Log job number
• Create objective function TSV (if it doesn't exist)
• Load pre-processed data
• Split data into training/test sets
• Make dataloaders
• Train model
• Record values of objective function (append to TSV)

Misc.

• Repeat for second optimization, taking a similar approach to the "identify associations" task (see move.tasks.identify_associations), i.e., detect type of task and change the value of the objective function (from accuracy to stabillity).
• Add to move.__main__.
• Re-format tutorial files for random.small dataset.

Open Questions

• Is the best set of hyperparameters automatically selected based on the objective function value (e.g., reconstruction accuracy)?
  • If so, I would suggest we also implement the Optuna plugin with a smarter sampler than greedy grid search.
  • If not, then I suggest just saving the results, and then providing some visualization functions so the users can decide on their hyperparameter set.
• Best latent representation #22

Resources:

• Hydra's documentation on multirun and sweepers
• Example Hydra app I wrote with multirun examples