> ## Documentation Index
> Fetch the complete documentation index at: https://benchgen.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# The autonomous loop

> One message, and the BenchGen AI agent benchmarks a model, trains it on its failures, benchmarks it again and tells you whether it got better, with the price of every step.

This is the job of an ML engineer for an afternoon, done by the agent from one message: measure a model, build a training set from what it got wrong, fine-tune, measure again, decide. You read the verdict in the chat. Nothing in between needs a click.

![The autonomous loop: from one message, the BenchGen agent runs five steps with its tools benchmark-launch, training-launch and benchmark-analyze, the platform GPU runs and signals back, and the chat reports each step and the verdict](https://mintcdn.com/benchgen-8fc81371/7ImVCk0tKA-Gy8LP/images/guides/benchgen-ai-agent/autonomous-loop.svg?fit=max\&auto=format\&n=7ImVCk0tKA-Gy8LP\&q=85\&s=4211402455abe5e1a28b4e06a696b3dd)

One message in, one verdict out: the agent runs five steps with its own tools, the platform GPU signals it back after each heavy step, and every step lands in your chat

## Start it

Open the chat with the BenchGen AI agent and paste:

```text wrap theme={null}
Run benchmark 104 on the base model Qwen2.5-0.5B-Instruct. When that run finishes, train this model on the questions it got wrong. When the training finishes, run benchmark 104 again on the model that training produced. Then compare the two runs: score before and after, which questions got fixed, which got broken, and your verdict: did training really improve the model?
```

Use your own benchmark id and model. The agent names the exact model id it picked, the price and your balance, and starts.

## What happens, step by step

**1. Benchmark the base model**

The agent launches the run and reports the launch in the chat with a link, and says what it will do next.

![The BenchGen AI agent reports that benchmark run 1129 was launched and that it will train, rerun and compare once it finishes](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-01-start.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=5caecb131eb31d2f2ebb2904661f32f5)

Step 1: the run is launched; the agent names what it will do when the run finishes

When the platform finishes scoring, it signals the agent. Wherever you are on the platform, the agent's message reaches you as a notification with a way to the chat.

![A notification on the Evaluations page: BenchGen Agent reports that baseline run 1129 finished with its scores, with an Open chat button](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-04-notification.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=e2de3c30b0243ca6f94349c7475e12a9)

The platform's notification of the agent's message, on the Evaluations page

**2. Build the dataset and fine-tune**

From the questions the model got wrong, the agent assembles the training set with the expected answers, augmented five times, names the estimated price and starts a LoRA training. No click, no question: it continues the plan you gave.

![The agent reports baseline run 1129 scores and that it started training job 387 on the 29 failed examples at 0.0767 credits](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-02-run-finished-training-started.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=1cf4e6063275aec30f05bb25c5e63aff)

Step 2 and 3: the run's scores reported, the training started on its 29 failures, with the price and the balance

![Training Jobs page with the job from-run-1129 in the Training state](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-05-training-jobs.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=d3ce3b988210e6de6d7217024cc1a74e)

The training the agent started, on the Jobs page like any other

**3. Benchmark the trained model**

When the platform signals that the training finished, the agent runs the same benchmark on the model the training produced. On the Evaluations page the two runs sit next to each other: the base model and the trained one, six minutes apart.

![Evaluations page with run 1130 on the model from-run-1129 and run 1129 on Qwen2.5-0.5B-Instruct, both completed](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-05b-evaluations.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=d6eabe237e35095b1eb0b69b81bb004f)

Step 4: run 1130 on the trained model, launched by the agent, next to run 1129 on the base model

**4. Compare and decide**

With both runs in, the agent writes the comparison: score before and after, what changed, and whether the difference means anything. It says so when training did not help, and offers what to do next.

![The agent's comparison table of run 1129 and run 1130 with the verdict that the fine-tuned model did not beat the baseline, and three next-step buttons](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-06-comparison.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=e6d551a12ee628a969bdf8fbecab6ce6)

Step 5: the comparison and the verdict, twelve minutes after the one message

Every message in the chat during the loop is written by the agent, with a sound if the chat is open. You can close the page; the messages wait.

## Who does what

| Step | You | BenchGen AI agent | Platform |
| - | - | - | - |
| Start | One message | Picks the model id, names the price | |
| Benchmark | | Launches, reads the scores, reports | Runs the model, scores, signals |
| Dataset | | Builds it from the failures | |
| Fine-tune | | Launches with the estimate, reports the model | Trains on GPU, bills real tokens, signals |
| Benchmark again | | Launches, reports | Runs, scores, signals |
| Compare | Read the verdict | Writes it | |

Without an agreed plan the agent reports each result and offers the next step as buttons; it starts nothing you did not ask for.

## Watch it

On the BenchGen AI agent's page, the **Activity** tab lists what the agent did on its own for you: each platform event, the message the agent wrote, what it started, what that turn cost, and the chat it is in.

![Activity tab of the BenchGen AI agent with the three events of the loop and the agent's messages](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/loop-07-activity.png?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=b16781407ed26d4f242ede66077eb117)

Activity: the loop above as the platform saw it, newest first, with the cost of each of the agent's turns

## The loop at a glance

![Swimlane diagram: you send one message, the agent benchmarks, builds the dataset, fine-tunes, benchmarks again and compares; the platform runs and scores and signals the agent after each step](https://mintcdn.com/benchgen-8fc81371/NkgiGeeAdO0kdHtp/images/guides/benchgen-ai-agent/autonomous-loop-steps.svg?fit=max\&auto=format\&n=NkgiGeeAdO0kdHtp\&q=85\&s=0ab8b7929de60cb24e19fef2bc377d67)

The loop: you write one message and read one verdict; the agent does the five steps; the platform runs the GPU work and signals the agent when each step is done


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.