model state manipulation

[Yumcoder-dev]

Suppose we have two model

#[derive(Module, Debug)]
pub struct RegressionModel<B: Backend> {
input_layer: Linear,
output_layer: Linear,
activation: ReLU,
}

let model1: crate::model::RegressionModel =
RegressionModelConfig::new(config.input_feature_len).init(&device);

let model2: crate::model::RegressionModel =
RegressionModelConfig::new(config.input_feature_len).init(&device);

How can we compute the average of the parameters of two models and then load these averaged parameters into a new model?

[Nikaidou-Shinku]

I think #1245 will fix this, once it's merged you can use model1.weight to get the model's parameters and then manipulate them as tensors.

[nathanielsimard]

I think the question is a bit different. You can create functions that work for each parameter of the module. There is the map and visit functions that exist on Module. You could use burn::tensor::container::TensorContainer to aggregate each parameter and then update the module in question. This is the strategy used in burn-train for gradients accumulation, but this can be used to merge modules as well.