Connect an OpenClaw Agent

OpenClaw can send trace data to BenchGen through the plugin package @benchgen/benchgen-openclaw. This integration captures:

One trace per message the agent handles
Model calls and tool executions, nested inside that trace
A startup handshake trace, tagged benchgen-setup, used to verify the setup

Prerequisites

OpenClaw gateway version 2026.7.1 or newer. Older builds skip plugins that require a newer plugin API.
Plugin 0.2.0 or newer, for the startup handshake.
BenchGen agent credentials — public key, secret key, and endpoint, from the Observability card on the agent’s Overview tab.

1. Command line

Install the plugin and restart the gateway

openclaw plugins install @benchgen/benchgen-openclaw

Configure the plugin

openclaw benchgen configure

This command prompts for your public key, secret key, and endpoint, validates that the keys reach BenchGen, enables the plugin, and writes plugin-scoped config to plugins.entries.benchgen.

Restart the gateway and check status

openclaw benchgen status
openclaw plugins info benchgen

Enabled: yes and Status: loaded mean the plugin is running. Now verify.

2. Setup prompt

On the Observability card, open the Ask your agent tab, click Copy setup prompt, and paste it into a session with the agent that runs your gateway. The prompt already contains your public key and endpoint; it tells the agent to install the plugin, write the config, and restart the gateway — so the agent’s tool profile has to allow shell execution.

The secret key is not in the prompt, deliberately — a prompt goes into chat history and usually into a model provider’s logs. The agent will ask for it separately.

Then check the result yourself at Verify. An agent reporting success is not the same as a trace arriving.

What the prompt says

Your copy has real values where the placeholders are.

Set up BenchGen tracing in this OpenClaw gateway.

Follow these steps exactly. Do not invent configuration keys — every name below
is read verbatim by the plugin.

## 1. Install the plugin

    openclaw plugins install @benchgen/benchgen-openclaw

## 2. Get the secret key from me

Three values are needed. Two are here; the third is a secret and is NOT in this
prompt:

  publicKey: pk-lf-YOUR-PUBLIC-KEY
  baseUrl:   https://your-benchgen-endpoint
  secretKey: ASK ME FOR IT. It is on the BenchGen agent page I copied this from,
             in the Observability card. Do not guess it, do not invent a
             placeholder, and do not continue without it.

## 3. Write the configuration

Merge this into the gateway's `openclaw.json`, keeping any other entries that
are already under `plugins.entries`:

    {
      "plugins": {
        "entries": {
          "benchgen": {
            "enabled": true,
            "config": {
              "enabled": true,
              "publicKey": "pk-lf-YOUR-PUBLIC-KEY",
              "secretKey": "<the key I give you>",
              "baseUrl": "https://your-benchgen-endpoint"
            }
          }
        }
      }
    }

Both `enabled` flags have to be true — the outer one turns the plugin entry on,
the inner one turns tracing on, and the plugin checks both.

`benchgen` is the plugin's manifest id, which is not its npm package name. An
unknown id under `plugins.entries` is a configuration error the gateway refuses
to start on, so do not change that string.

Instead of putting the keys in the file, you may set `BENCHGEN_PUBLIC_KEY`,
`BENCHGEN_SECRET_KEY` and `BENCHGEN_BASE_URL` in the gateway's environment; the
plugin falls back to those when the config omits them.

## 4. Restart the gateway

The configuration is only read at startup, so nothing happens until you restart
the gateway. This is the most common reason setup appears to fail.

## 5. Verify

On every start the plugin sends a trace named `benchgen.plugin.connected`,
tagged `benchgen-setup`. Tell me to look for it on the Traces tab — that trace
is the gateway confirming it reached BenchGen, and it is the only real proof
this worked.

If nothing arrives, check the credentials themselves:

    curl -s -o /dev/null -w '%{http_code}\n' \
      -u 'pk-lf-YOUR-PUBLIC-KEY:<the key I give you>' \
      'https://your-benchgen-endpoint/api/public/health'

200 means the keys are good and the problem is the config or the restart. 401
means the keys are wrong.

## Do not

- Do not run `openclaw benchgen configure`. It prompts interactively and cannot be
  driven from this conversation; the JSON in step 3 is its exact equivalent.
- Do not print the secret key back to me, and do not write it anywhere except
  the gateway configuration or its environment.

3. Advanced configuration (manual JSON)

Use this only when you need explicit checked-in or templated plugin configuration, or when your only access is the Control UI’s raw config editor (Settings → Advanced → Settings → Raw). Add this to your OpenClaw config, at the root level:

{
  "plugins": {
    "entries": {
      "benchgen": {
        "enabled": true,
        "config": {
          "enabled": true,
          "publicKey": "pk-...",
          "secretKey": "sk-...",
          "baseUrl": "https://your-benchgen-endpoint"
        }
      }
    }
  }
}

Both enabled flags have to be true — the outer one loads the plugin, the inner one turns on streaming. benchgen is the plugin’s manifest id, not its npm package name. Environment variable fallbacks are also supported:

BENCHGEN_PUBLIC_KEY
BENCHGEN_SECRET_KEY
BENCHGEN_BASE_URL

Omit publicKey, secretKey and baseUrl from the config to fall back to them. Restart the gateway after editing.

Back up openclaw.json first. OpenClaw refuses to start on an invalid config — an unknown plugin id or a wrong type stops the gateway, and the Control UI goes down with it.

Verify

Every time the plugin starts with valid configuration it sends one trace of its own:

benchgen: sent startup trace "benchgen.plugin.connected" to https://...

Look for it in the Logs tab of the Control UI, and on your agent’s Traces tab as benchgen.plugin.connected, tagged benchgen-setup. The Observability card flips from Waiting for first trace to Traces arriving when it lands, with no page reload. Wrong credentials fail loudly rather than silently:

benchgen: startup trace failed, streaming is still active: Unauthorized

Streaming stays subscribed either way — a rejected handshake never stops the plugin.

On OpenClaw 2026.7.2 a config edit hot-reloads: the logs show config change detected followed by a fresh handshake, no restart. Only new plugin code needs a restart.

These traces accumulate, since a config edit counts as a start. Filter the Traces tab by tag benchgen-setup to isolate them, or to scan past them. Then send your agent a few ordinary messages. Those produce the real traces.

Configuration reference

Field	Purpose	Environment fallback
`entries.benchgen.enabled`	Loads the plugin	n/a
`config.enabled`	Enables trace streaming	n/a
`config.publicKey`	BenchGen project public key	`BENCHGEN_PUBLIC_KEY`
`config.secretKey`	BenchGen project secret key	`BENCHGEN_SECRET_KEY`
`config.baseUrl`	BenchGen ingest endpoint	`BENCHGEN_BASE_URL`
`config.startupTrace`	Sends the handshake trace on every plugin start, including a config hot reload. Defaults to `true`	n/a

Setting all three variables on the container skips the wizard entirely. Values in openclaw.json always win over their environment fallback.

Troubleshooting

No handshake trace after restarting? Read the gateway logs — each line rules out a different cause:

Log line	Cause
`missing publicKey/secretKey (...); not starting`	No credentials reached the plugin. Check `openclaw benchgen status`, or that the variables are set for the process the gateway actually runs as.
`disabled via config (enabled === false); not starting`	One of the two `enabled` flags is false.
`diagnostics are disabled (config.diagnostics.enabled === false)`	The plugin has nothing to listen to. Enable diagnostics.
`startup trace failed, streaming is still active: ...`	Credentials are present but wrong, or BenchGen is unreachable. Compare all three values against the Observability card.
`Plugin not found: <id>`	Use the manifest id `benchgen`, not the npm package name.
`plugin requires plugin API >=X, but this host is Y; skipping discovery`	The OpenClaw build is too old. Upgrade the image through your hosting panel — the UI Update button only works for git checkouts.
`EPERM: operation not permitted, chmod '/home/node/.openclaw'`	The config directory is not writable by the user the gateway runs as (`node`, uid 1000). Fix on the hosting side.
No benchgen lines at all	The plugin did not load. Check `openclaw plugins info benchgen` for `Status: loaded`.

Gateway will not start after editing config. Unknown keys, wrong types, and invalid values all block startup, and the Control UI is unavailable while the gateway is down. Restore openclaw.json.bak through your hosting panel. The agent says it worked but nothing changed. Treat any claim without visible tool output as unverified — the handshake trace is the only proof that does not depend on the agent’s word. The handshake arrived, but real turns produce no traces. Credentials and transport are fine. The plugin maps OpenClaw’s diagnostics events, so confirm diagnostics are enabled and that traffic is reaching this gateway.

Upgrade the plugin

openclaw plugins update benchgen

From the Control UI, send it as Run: openclaw plugins update benchgen. Restart the gateway afterwards to load the new code. plugins install refuses to overwrite a plugin that is already present — use update, or --force.

Get started

Agents

Eval

Train

Deploy Benchgen Platform

Prerequisites

1. Command line

2. Setup prompt

3. Advanced configuration (manual JSON)

Verify

Configuration reference

Troubleshooting

Upgrade the plugin

​Prerequisites

​1. Command line

​2. Setup prompt

​3. Advanced configuration (manual JSON)

​Verify

​Configuration reference

​Troubleshooting

​Upgrade the plugin

Prerequisites

1. Command line

2. Setup prompt

3. Advanced configuration (manual JSON)

Verify

Configuration reference

Troubleshooting

Upgrade the plugin