🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning

Name: 🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning
Start: 2024-04-05T11:00:00+00:00
End: 2024-04-05T11:25:00+00:00
Location: SMK

Talk 25 min April 05, 2024 at 11:00

Gabriel Martín Blázquez

David Berenstein

If you are GPU-poor you need to become data-rich. I will give an overview of what we learned from looking at Alpaca, LIMA, Dolly, UltraFeedback and Zephyr and how we applied that to fine-tuning a state-of-the-art open source LLM called Notus and Notux by becoming data-rich.