Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PeftSavingCallback #24

Closed
wants to merge 1 commit into from
Closed

Fix PeftSavingCallback #24

wants to merge 1 commit into from

Conversation

lchu6
Copy link
Contributor

@lchu6 lchu6 commented Jan 26, 2024

This PR introduces fix on PeftSavingCallback for both saving PEFT and non-PEFT (i.e. full parameter tuning).

  1. Fix saving peft when original implementation would save peft in root ckpt folder which violates the purpose of this callback.
  2. Fix saving full-parameter-tuning when it is actually not necessary for saving separately. And original implementation would introduce broken ckpt as discussed in model checkpoints are broken when doing full parameter tuning  #23

@lchu6 lchu6 changed the title Fix PeftSavingCallback on full parameter tuning Fix PeftSavingCallback Jan 26, 2024
@Ssukriti
Copy link
Collaborator

Ssukriti commented Mar 7, 2024

@fabianlim does this PR become obsolete with #53 ?

@fabianlim
Copy link
Collaborator

@Ssukriti to my knowledge yes, as these checkpoint manipulation techniques are not required anymore, now that HF trainer will save the checkpoints correct as described in the writeup of #53

@Ssukriti
Copy link
Collaborator

Ssukriti commented Mar 9, 2024

closing unless asked otherwise. PR #53 will be merged soon

@Ssukriti Ssukriti closed this Mar 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants