AttributeError: 'Namespace' object has no attribute 'residual_dropout_lima'

[cassidylaidlaw]

Hi, thanks for making your code available! I tried to run the reward model training script:

accelerate launch --config_file configs/accelerate_config.yaml src/reward_modeling/training/trainer_rm.py --configs defaults_rm rm-pythia-44m --rng_seed 0

It failed with this error:

  File ".../llm_optimization/src/reward_modeling/training/trainer_rm.py", line 350, in <module>
    main()
  File ".../llm_optimization/src/reward_modeling/training/trainer_rm.py", line 209, in main
    model = get_model(training_conf, tokenizer)
  File ".../lib/python3.10/site-packages/model_training/utils/utils.py", line 354, in get_model
    residual_dropout_lima=conf.residual_dropout_lima,
AttributeError: 'Namespace' object has no attribute 'residual_dropout_lima'. Did you mean: 'residual_dropout'?

I change the config entry for residual_dropout to residual_dropout_lima but I wanted to make sure that this is the correct way to fix the error. It might be good to update the default config in the repo as well.