Request for requirements.txt for LLaMA-3.2 Image Training

[tsw123tsw]

Description

Hi, I have been trying to train images using LLaMA-3.2. However, I encountered an issue with the current requirements.txt file, specifically with transformers==4.42.4. The error I faced is as follows:

ValueError: rope_scaling must be a dictionary with two fields, type and factor, got {'factor': 32.0, 'high_freq_factor': 4.0, 'low_freq_factor': 1.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}

Upon inspecting the Transformers code (configuration_llama.py), I noticed that this case is not handled in version 4.42.4. Updating transformers to 4.43.1 resolves this specific problem.

Additionally:

• According to Hugging Face's release notes, LLaMA 3.2 support begins with transformers==4.45.0.
• The config.json file provided with the model weights indicates "transformers_version": "4.44.2".
• However, updating to newer versions of transformers such as 4.45.0 introduces issues with methods like save_checkpoint, as the number of arguments has changed.

In light of this, could you please share the exact requirements.txt you used for image training with LLaMA-3.2 (specifically for the 3B and 1B models)? This would greatly help ensure compatibility and avoid any unnoticed errors during training.

---

Steps to Reproduce

1. Use transformers==4.42.4 and attempt to train LLaMA 32.
2. Encounter the rope_scaling ValueError.

---

Thank you for your assistance! Looking forward to your response.