GenZ LLM

A post-trained language model that responds in GenZ slang, built on top of Qwen2.5-0.5B-Instruct using Supervised Fine-Tuning (SFT) followed by Reinforcement Learning with GRPO.

The fine-tuned model is available on Hugging Face: aidarbek/GenZ-Qwen2.5-0.5B-Instruct

Intro

The goal of this work was mostly to have fun and learn. If you are really interested in making LLMs talk like GenZ, current frontier models are probably doing it a lot better, the only possible advantage this model has is that you probably will save a bit more tokens when prompting models. Another interesting insight, it actually doesn't cost that much to train the model.

Setup & costs

Models were trained in Google Colab notebooks, with the cheapest GPU runtime. Overall, it cost me <$2 and 3-4 hours to train models (using Colab's "pay as you go" plan).

Potentially, if I was lucky and knowledgeable, I could even train the RL model completely for free. My initial RL training was run on a free plan, and I quickly hit the plan limits mid-training. However, a model checkpoint file was created that I could have used, but ultimately, all the data was lost because I hadn’t mounted my Google Drive to the runtime environment, so the results were deleted together with the runtime. I could probably wait until my limits reset, but I decided not to wait and got the cheapest paid plan.

Results

As a result, model learned to use GenZ slang, however, the open question was if the SFT training was enough for a model to learn, and we did't actually need an RL/GRPO training? It's true that both SFT and SFT+RL models generate more or less the same text, with GenZ keywords injected. However, there's a weak evidence that suggests RL training increased the reward (reward defined as simple number of keywords occurence over the constant). I believe it could be stronger with more data, RL training, and bigger test sample sizes. Interestingly, RL model preferred certain words more than the others (e.g. vibes, down horrendous, oof, etc).

Data

The data is synthetically generated by frontier LLMs.

Training data is located in the rl_test folder:

• sft_dataset.csv — dataset used for supervised fine-tuning
• rl_dataset.csv — dataset used for GRPO reinforcement learning
• genz_keywords.json — Gen Z slang keywords used as the RL reward signal

Code

• GenZ_LLM_Training.ipynb — full training pipeline (SFT + GRPO)
• GenZ_LLM_Inference.ipynb — inference demo using the uploaded model