Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU
- We are excited to officially release the integration of
peftto make Large Language Model (LLM) fine-tuning with Reinforcement Learning more accessible to anyone! In this post, we explain why this is a competitive alternative to existing fine-tuning approaches.
peftis a general tool that can be applied to many ML use-cases but it’s particularly interesting for RLHF as this method is especially memory-hungry!
- If you want to directly deep dive into the code, check out the example scripts directly on the documentation page of TRL.