refactor: Removal of pad_token when padding_free is set#368

Closed

Abhishek-TAMU wants to merge 2 commits intofoundation-model-stack:mainfrom Abhishek-TAMU:tokenization_cleanup

+8-7

Commits on Oct 8, 2024

refactor: Removing assignment of pad_tokens in padding_free tuning
Abhishek-TAMU
committed
Merge branch 'foundation-model-stack:main' into tokenization_cleanup
Abhishek-TAMU
authored