Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于tuning prompt only #1

Open
zhaijianyang opened this issue Dec 18, 2022 · 3 comments
Open

关于tuning prompt only #1

zhaijianyang opened this issue Dec 18, 2022 · 3 comments

Comments

@zhaijianyang
Copy link

大佬,在tuning prompt only时候,main.py的第79行代码执行后,模型token embedding会产生梯度,这个时候是更新了item, user, token embedding三个东西吧?

@lileipisces
Copy link
Owner

不是79行,是78行,传了一个freezeLM=True,整个模型的梯度就不更新了,只更新user和item embedding,不过这样效果不太好

@zhaijianyang
Copy link
Author

78行后,模型的梯度不更新了,但是,执行79行之后,再查看模型梯度,会发现token embedding的required_grad==True,我看了下源码,resize_token_embedding新建立了一个nn.embedding,所以后面更新了user, item和token三个呀

@lileipisces
Copy link
Owner

厉害👍🏻我原本只是想更新user和item token的😂幸好整个模型没有更新

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants