Replies: 1 comment
-
这两个应该需要补上。 COSINE_WITH_RESTARTS = "cosine_with_restarts"
POLYNOMIAL = "polynomial" 这两个可以考虑合并到 Trainer 里面一起支持 "CosineAnnealingWithWarmupDecay",
"LinearAnnealingWithWarmupDecay", |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
1. LR Scheduler 升级更新
Huggingface的 https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_utils.py
而当前的PaddleNLP的Trainer中只支持下面几个,建议更新。
2. 统一使用Trainer里面的LR Scheduler(与HF对齐)
https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/transformers/optimization.py 有旧的LR Scheduler,建议废弃,未来的新代码需要升级成最新的 Trainer 里面的 Scheduler,这样在使用体验上对习惯HF的用户来说更友好。

Beta Was this translation helpful? Give feedback.
All reactions