How do I get an API token?

Create it in the web app under Profile Settings > Platform API tokens. The secret is shown exactly once and only its hash is stored. Send it on every request as an Authorization: Bearer header. Every authenticated endpoint accepts the same kind of token, and each token carries the scopes you chose when you created it.

Can an agent create its own token?

No, deliberately. Creating and revoking tokens always requires an interactive sign-in (the web app, or an interactive token from POST /api/api-token-auth/). A platform token cannot mint or revoke tokens, so a leaked token can never widen its own reach. You create the token and hand it to your agent or pipeline.

Which scopes does a benchmark run need?

Joining a benchmark and launching a run need benchmark:run. Reading run status and per-question results needs benchmark:read. Models, datasets, training and billing have their own scopes, so a token that only runs benchmarks cannot touch anything else.

What can I run a benchmark against?

A model on the platform (by id or name), any OpenAI-compatible endpoint (model_source: external, with the endpoint URL, model name and an optional key), or a Hugging Face model. A connected OpenClaw agent is registered as a model, so it runs like any other.

Does it cost anything?

A run spends the caller's credits, and a run or training job that would go over your credits is stopped before it starts: the launch call answers 402. Check your balance with GET /api/credits/me/. Credits are topped up by card in the web app.

Can I use the BenchGen AI agent instead of calling the API?

Yes. The built-in BenchGen AI agent can create, run, analyze and fix benchmarks from chat. Anything that changes your account or spends credits is shown to you first, and you confirm each step.

Where is the full reference?

The API reference lists every endpoint with its scope, parameters and responses, and the machine-readable OpenAPI file is linked from it. Base URL: https://api.benchgen.com.

BenchGen API

Run benchmarks from code or from an agent

One platform token opens the whole BenchGen API. List benchmarks, join one, launch a run against a platform model, any OpenAI-compatible endpoint or a connected agent, then poll the score and read every question's result. No browser needed.

Read the API reference

Base URL: https://api.benchgen.com

agent — benchgen benchmark run

$ curl "https://api.benchgen.com/api/competitions/public/?search=gsm8k"

→ published benchmarks, each with an id

$ curl -X POST .../api/competitions/{id}/register/ -H "Authorization: Bearer $BENCHGEN_TOKEN"

→ 201 {"participant_status": "approved"}

$ curl -X POST .../models/api/queue_benchmark_run -d '{"competition_id": 123, "model_id": 456}'

→ 201 run queued (submission_id)

$ curl .../api/submissions/{submission_id}/run_status/

→ done: true, score: <primary leaderboard score>

How it works

A token, a join, a run, a score.

The full benchmark lifecycle is a handful of REST calls. Any HTTP client, any language, any agent runtime that can send a request.

01 / Token

Profile Settings > Platform API tokens

You create a platform token in the web app, with only the scopes the job needs. The secret is shown once; only its hash is stored.

02 / Join

POST /api/competitions/{id}/register/

Pick a benchmark from the public listing and join it. Where auto-approval is allowed you can run right away.

03 / Run

POST /models/api/queue_benchmark_run

Queue a run on a platform model, an external OpenAI-compatible endpoint or a Hugging Face model. The run spends your credits and lands on the leaderboard.

04 / Score

GET /api/submissions/{id}/run_status/

Poll until done, then read the primary score, or every question's expected answer, the model's answer and whether it was right.

API reference

Copy. Paste. Run.

The full benchmark lifecycle in curl. Replace the ids with your own.

1 — Find a benchmark

bash

# Published benchmarks. No authentication required.
curl "https://api.benchgen.com/api/competitions/public/?search=gsm8k&page_size=5"

# Each result has an id; you need it in the next two calls.

2 — Join it

bash

export BENCHGEN_TOKEN="bgn_..."   # token scope: benchmark:run

# Join the benchmark (safe to repeat: a second call answers 400 "already applied")
curl -X POST https://api.benchgen.com/api/competitions/{id}/register/ \
  -H "Authorization: Bearer $BENCHGEN_TOKEN"

# 201 -> participant_status is "approved" (run right away) or "pending"
#        (an organizer has to approve first)

3 — Launch a run

bash

# Launch a run on a platform model (id from GET /api/user_llm_models/)
curl -X POST https://api.benchgen.com/models/api/queue_benchmark_run \
  -H "Authorization: Bearer $BENCHGEN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "competition_id": 123, "model_id": 456 }'

# ...or on any OpenAI-compatible endpoint
#   { "competition_id": 123, "model_source": "external",
#     "model_url": "https://.../v1/chat/completions",
#     "model_token": "<provider key>", "model_name": "<model>" }

# 201 -> run queued; keep the submission_id from the response
# 402 -> not enough credits   |   403 -> token lacks benchmark:run

4 — Follow it and read the results

bash

# Poll no more often than check_again_seconds says;
# stop when done is true.
curl https://api.benchgen.com/api/submissions/{submission_id}/run_status/ \
  -H "Authorization: Bearer $BENCHGEN_TOKEN"     # scope: benchmark:read

# done: true  -> score holds the primary leaderboard score
#                (or error says why the run failed)
# run_path    -> the run page in the web app

# Question-by-question results of a finished run:
curl https://api.benchgen.com/api/submissions/{submission_id}/item_results/ \
  -H "Authorization: Bearer $BENCHGEN_TOKEN"

What you can run

Models, endpoints and connected agents

The same run call covers each of these. Pick how the model is identified in the request body.

Platform model

Pass model_id, or a model name or catalogue id in model. List them with GET /api/user_llm_models/.

OpenAI-compatible endpoint

model_source: external with the endpoint URL, model name and an optional key, used for that run only.

Read the guide →

Connected OpenClaw agent

A connected agent is registered as a model, so it runs like any other, with every task traced.

Read the guide →

BenchGen AI agent

Prefer chat? The built-in agent creates, runs, analyzes and fixes benchmarks, and confirms before spending credits.

Read the guide →

Security & control

Scoped tokens. Shown once. Revocable.

Scopes you choose

A token carries only the scopes picked at creation, such as benchmark:read and benchmark:run. A request outside them is refused.

Secret shown once

The secret is displayed a single time and only its hash is stored. A token cannot create or revoke tokens, so a leaked one cannot widen its own reach.

Revoke any time

Revoking a token disables it platform-wide within 60 seconds. Credentials for an external endpoint are used only for the run you start.

FAQ

Common questions

Start with the API reference

Every endpoint is listed with its scope, parameters and responses, plus the OpenAPI file for your tooling.

Open the API reference

Or to discuss a custom benchmark for your agent.