slurm_scripts For setting up your python virtual enviroment and launching slurm jobs.
training scripts for training and utility stuff.
configs Yaml files for the accelerate configurations and training arguments
Go to slurm_scripts and modify it according to your own paths.
sbatch slurm_scripts/setup_venv
Either use your own dataset and convert it to chatml's conversational format. You can get inspiration from data.
{"messages": [{"role": "system", "content": "You are..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
{"messages": [{"role": "system", "content": "You are..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
{"messages": [{"role": "system", "content": "You are..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}or use some of the ready-made datasets in /scratch/project_462000558/TurkuNLP_workshop/data
Go to configs if you want to change training arguments, as you would with huggingface TrainingArguments
Modify the launch script at slurm_scripts
sbatch slurm_scripts/sft.sh
Full model weight training on a 34B model requires minimum of 2 nodes and atleast 3 nodes is required. Also note that as you increase the amount of nodes, the training becomes more unstable and prone to nccl crashes/hangs.
This work was heavily inspired by:
- https://github.com/pacman100/DHS-LLM-Workshop/tree/main
- https://www.philschmid.de/fine-tune-llms-in-2024-with-trl
- https://github.com/huggingface/alignment-handbook
- https://wandb.ai/capecape/alpaca_ft/reports/How-to-Fine-tune-an-LLM-Part-3-The-Hugging-Face-Trainer--Vmlldzo1OTEyNjMy
My own fork of the alignment handbook is https://github.com/Vmjkom/alignment-handbook (wip). The Alignment handbook implements reinforcement learning techniques, along with the Sft. Furthermore, there is more sophisticated data handling
SftTrainerChat templatingSingle and Multi-node Launchers with SLURMPEFT(Parameter Efficient finetuning -- Lora...etc)