One message, and the BenchGen AI agent benchmarks a model, trains it on its failures, benchmarks it again and tells you whether it got better, with the price of every step.
This is the job of an ML engineer for an afternoon, done by the agent from one message: measure a model, build a training set from what it got wrong, fine-tune, measure again, decide. You read the verdict in the chat. Nothing in between needs a click.
One message in, one verdict out: the agent runs five steps with its own tools, the platform GPU signals it back after each heavy step, and every step lands in your chat
Open the chat with the BenchGen AI agent and paste:
Run benchmark 104 on the base model Qwen2.5-0.5B-Instruct. When that run finishes, train this model on the questions it got wrong. When the training finishes, run benchmark 104 again on the model that training produced. Then compare the two runs: score before and after, which questions got fixed, which got broken, and your verdict: did training really improve the model?
Use your own benchmark id and model. The agent names the exact model id it picked, the price and your balance, and starts.
The agent launches the run and reports the launch in the chat with a link, and says what it will do next.
Step 1: the run is launched; the agent names what it will do when the run finishes
When the platform finishes scoring, it signals the agent. Wherever you are on the platform, the agent’s message reaches you as a notification with a way to the chat.
The platform's notification of the agent's message, on the Evaluations page
2
Build the dataset and fine-tune
From the questions the model got wrong, the agent assembles the training set with the expected answers, augmented five times, names the estimated price and starts a LoRA training. No click, no question: it continues the plan you gave.
Step 2 and 3: the run's scores reported, the training started on its 29 failures, with the price and the balance
The training the agent started, on the Jobs page like any other
3
Benchmark the trained model
When the platform signals that the training finished, the agent runs the same benchmark on the model the training produced. On the Evaluations page the two runs sit next to each other: the base model and the trained one, six minutes apart.
Step 4: run 1130 on the trained model, launched by the agent, next to run 1129 on the base model
4
Compare and decide
With both runs in, the agent writes the comparison: score before and after, what changed, and whether the difference means anything. It says so when training did not help, and offers what to do next.
Step 5: the comparison and the verdict, twelve minutes after the one message
Every message in the chat during the loop is written by the agent, with a sound if the chat is open. You can close the page; the messages wait.
On the BenchGen AI agent’s page, the Activity tab lists what the agent did on its own for you: each platform event, the message the agent wrote, what it started, what that turn cost, and the chat it is in.
Activity: the loop above as the platform saw it, newest first, with the cost of each of the agent's turns
The loop: you write one message and read one verdict; the agent does the five steps; the platform runs the GPU work and signals the agent when each step is done
