Experiment Library and Setup (ELIAS)

1. Main functionalities

Config class

Intuitive dataclass extension that addresses issues commonly encountered in research projects. A Config has the following features:

• Easily persistable to a file (JSON or YAML)
• Stored configs are human-readable and editable
• Special support for storing enums and inheritance structures
• Backward compatibility to allow loading older persisted configs

TODO: Overview image of benefits of using Config over regular dataclass

Philosophy for stored config file types:

• Everything should be human-readable
• JSON for configs that may be viewed often (statistics, preprocessing/training configs, evaluation results)
• YAML for configs that have to be edited (run specifications)

Experiment Workflow Utilities

To speed up the experimentation process with utility classes the elias library assumes the following workflow:

Stage	Input	Output	Utility
Data Preprocessing	Processing Configuration Raw Data	Preprocessed Data (.p, .p.gz, .json, .npy, ...) Data Statistics (stats.json) Preprocessing Config (config.json)	DataFolder -> DataManager
Training/Fitting	Preprocessed Data Hyperparameters	Model checkpoints Hyperparameter configs	ModelManager -> RunManager
Evaluation	Trained model Evaluation Config	Evaluation Config Evaluation Results	RunManager -> EvaluationManager
Manual Analysis	Any model/data	Plots, statistics, images	AnalysisFolder -> AnalysisManager