Benchmark Results

Every run, past or in progress, has a detail page. Open one from Results or Evaluations in the Eval sidebar, or from a Recents entry. While a run is still going you see live logs, covered in Evaluate an Inference Model. This page covers what you see once it’s Completed.

Score Breakdown

The Overall Score tile is followed by the individual metrics the environment’s scoring program reported. These are entirely environment-defined, not a fixed set, so two runs can show completely different tiles:

A math benchmark might report accuracy, exact match, answer rate, and duration.
A routing benchmark might instead report accuracy, correct, total, total cost usd, and avg cost usd per task.

If a scoring program doesn’t emit structured metrics, this card just says score breakdown isn’t available for that run.

Detailed Results

This embeds the HTML report the environment’s own scoring program generated, the same harness and scoring rules described in the Overview. Layout is entirely up to whoever built the environment:

The example above is a simple report: a few stat tiles plus a per-question table with the case ID, the Gold (expected) answer, the model’s Prediction, and a pass/fail mark.
Others go further, for example breaking results down by difficulty, by domain, or by routing target with their own bar charts, on top of the same accuracy-style tiles.

Click Open to view the report on its own page. It only appears once the run has a detailed-results artifact, failed runs show the failure reason and logs instead.

Evaluation details

The sidebar summarizes the run:

Field	What it shows
Status	`Completed`, `Failed`, `Running`, `Deploying`, or `Pending`
Model	The model that was evaluated
Environment	Which benchmark environment it ran against
Created	When the run started
Phase	The benchmark phase this run counts toward, for environments that use phases
Submission	The submitted archive’s filename (shows a no file tag if none was attached)
Run ID	The run’s unique identifier
Generated Dataset	View fine-tune dataset, shown only when the scoring program emitted one
Score	The same overall score shown in Score Breakdown

If you own the run, a Run configuration button below the sidebar opens the sampling parameters and environment variables it used, see Environment Variables.

Files

Every artifact the run produced is downloadable from Files, each row has its own Download button so you can grab exactly the one you need:

File	What’s inside	Format
Submission File	The archive you (or Eval, for a live model) submitted to start the run: your model’s config, or the generated submission for a running/platform model	`.zip`, whatever you uploaded
Prediction Output	Everything the ingestion step wrote for the scoring program to read: the model’s raw responses to every test case, before any judging happens	`.zip`
Scoring Output	Everything the environment’s scoring program wrote while judging those predictions: the raw per-case verdicts it computed before they were rolled up into the Score Breakdown tiles	`.zip`
Detailed Results	The same human-readable report shown inline above, standalone. Has both View (opens it in the browser) and Download	`.html` (sometimes with a `.json` alongside)
Finetune Artifact	A ready-to-train dataset the scoring program packaged from this run’s cases, only listed when it emitted one	`.zip`

None of these have a fixed internal structure. What ends up inside Prediction Output, Scoring Output, and Detailed Results is entirely up to the benchmark’s own scoring_program.zip (and ingestion_program.zip, if it has one), see Bundle Structure. The examples below show what BenchGen’s own built-in templates write, a custom environment can look completely different.

Prediction Output example

BenchGen’s Question-Answer template’s ingestion program writes one predictions.json per run:

{
  "predictions": [
    { "id": "q1", "prediction_text": "Ankara", "no_answer_probability": 0.0 },
    { "id": "q2", "prediction_text": "", "no_answer_probability": 1.0 }
  ],
  "metadata": {
    "execution_time": 42.7,
    "total_questions": 2,
    "answered_questions": 1,
    "status": "success"
  }
}

Scoring Output example

The Code (SWE-style) template’s scoring program reads that predictions.json and writes its own scores.json, with whatever metric keys that benchmark defines:

{
  "patch_generated_rate": 0.82,
  "resolved_rate": 0.0,
  "total_tasks": 50,
  "patches_generated": 41
}

These scores.json keys are exactly what populates the Score Breakdown tiles for that run, matched against the leaderboard columns declared in the environment’s competition.yaml.

Finetune Artifact example

When a scoring program also writes a finetune_artifact.zip, BenchGen extracts it and lists it here. Inside, it’s a small manifest plus the training rows:

finetune_artifact.zip
├── manifest.json   { "kind": "training_rows", "n": 512,
│                     "strategy": "failed_and_near_miss" }
└── rows.jsonl      { "prompt": [...], "answer": "..." }

Below Files, a Fine-tune Dataset panel either lets you save this artifact as a training dataset, or tells you the scoring program didn’t emit one for this run. Turning results into training data is covered in Export Datasets → Train.

Get started

Agents

Eval

Train

Inference

Deploy Benchgen Platform

Score Breakdown

Detailed Results

Evaluation details

Files

Prediction Output example

Scoring Output example

Finetune Artifact example

​Score Breakdown

​Detailed Results

​Evaluation details

​Files

​Prediction Output example

​Scoring Output example

​Finetune Artifact example

Score Breakdown

Detailed Results

Evaluation details

Files

Prediction Output example

Scoring Output example

Finetune Artifact example