After ElixirConf I found myself increasingly immersed in the world of fine-tuning. This curiosity soon led me on a journey to explore various use cases, eventually drawing me into the realm of synthetic data—a journey that spanned several months.
One pivotal moment in this exploration was a post by Edward Donner that detailed his experience training a language model to mimic his unique texting style. The majority of my time was soon dedicated to cleaning raw text message data in order to create a high-quality dataset for fine-tuning.
While the talk was focused on synthetic data generation, I also delved into the intricacies of fine-tuning with Unsloth, evaluation strategies, and even model deployment and serving with Nx. I owe much of the data generation techniques I shared to Jon Durbin, whose DPO approach in particular proved instrumental to my work. Lastly, a big thank you to Paraxial IO for giving me the opportunity to share my insights!
The python source code for fine tuning with unsloth is on Github for anyone interested. I also put the elixir code for the chat app shown in the talk on Github.
