Imagine going to sleep after writing a single Markdown specification and waking up to find that an AI agent ran dozens of LLM fine-tuning experiments overnight on your behalf - discovering optimal LoRA ranks, refining learning rate schedules, tuning batch sizes and committing each verified improvement to Git.

This is no longer a fantasy. Earlier this year, the autoresearch project showcased how autonomous LLM agents can iteratively explore pre-training in a self-contained loop. Taking inspiration from this paradigm, we created autofinetune: applying autonomous research loops to LLM post-training (Supervised Fine-Tuning and Reinforcement Learning via GRPO), using Google’s full AI stack—Tunix, Gemma, and Cloud TPUs orchestrated with Antigravity CLI and Gemini Flash 3.7.

In this post, we’ll explore how the autonomous research loop works for post-training and walk through a couple of real-world LLM finetuning case studies.

The Paradigm Shift: From Manual Tuning to Autonomous Loops

Traditional post-training involves a repetitive, manual cycle:

1. Formulate a hypothesis (e.g., "Will adding attn_vec_einsum to the LoRA target modules improve accuracy?" or "What happens if we change rollout temperature during GRPO?").
2. Edit training scripts and hyperparameters.
3. Launch a job on accelerators.
4. Monitor loss curves and benchmark evaluations.
5. Manually revert failures or record successes in a spreadsheet.

As demonstrated in autoresearch, we can now automate this whole process with the power of AI agents:

• Design the Arena (program.md): human defines the loop, boundary conditions, evaluation criteria, and constraints.
• Provide the Execution Code (run.py): A single, clean, self-contained finetuning script.
• Let the Agent Iterate: An agent follows instructions in program.md: it modifies run.py, runs the training job, measures the target metric, retains winning commits or reverts regressions, and logs results in results.tsv.

Case Study 1: Supervised Fine-Tuning (SFT) on FunctionGemma

In the first experiment in autofinetune, we took the same SFT setup in our previous blog and extended it by creating the autoresearch loop to optimize google/functiongemma-270m-it on the google/mobile-actions dataset.

The Setup

• Hardware: Cloud TPU v5e-1
• Iteration Speed: A few minutes per run
• Total Runs: 20 automated experiments in a couple of hours for quick experimentation
• Objective Metric: Post-training evaluation accuracy on function call generation (accuracy)

The agent was given boundaries in program.md:

• Allowed: LoRA rank/alpha, target projection layers, learning rates, warmup/decay schedules, optimizers (e.g., AdamW/Muon, gradient clipping), batch size, and seeds.
• Disallowed: Changing the dataset, number of epochs, or model architecture.

Sample Trajectory

Here is a sample trajectory from sample_runs/SFT_results.tsv demonstrating how the agent hill climbed.

As you can see, the agent is able to automatically adjust LoRA rank/alpha, optimizer, learning rate, etc. to keep improving the model’s accuracy in terms of generating correct function calls.

Case Study 2: Reinforcement Learning (GRPO) on Gemma for Math Reasoning

Supervised fine-tuning is only a simple test. For our second case study, we took the official GRPO example from the Tunix repository (which trains Gemma 3 1B for math reasoning using GSM8K; the trained model has better numerical accuracy and format accuracy in its answers) and set it up for autonomous RL finetuning. Reinforcement learning is subject to hyperparameter sensitivity, instability, and longer execution times - making this task more challenging and time-consuming.

The Arena Setup

• Hardware: Cloud TPU v6e-1
• Model: Gemma 3 1B
• Dataset: GSM8K
• Iteration Speed: A couple of hours per run
• Run Duration: 40 experiments over 2–3 days
• Objective Metric: to simplify the outer optimization loop for the agent, we chose a single artificial evaluation metric Post_RL_metric, which is simply numerical_accuracy + format_accuracy (you can of course use other metrics, i.e., using different weights).

Sample trajectory

Below is a sample trajectory logged in sample_runs/RL_results.tsv, showing the agent’s progress. The agent was able to identify better LoRA configurations, rollout temperature, KL penalty, system prompt, etc. to improve the total reward by ~10%.

What’s Next?

We hope this project shows you the power of AI agents in the domain of LLM post-training and inspires you to think about how to leverage them to automate your LLM finetuning workflows using Tunix on TPUs. Please check out the code, sample runs, and program.md templates in the autofinetune GitHub repository, explore the Tunix library, and start building your own autonomous post-training lab today!