Benchmark report

Task verified

6m 17s elapsed

The saved attempt met the documented success criterion.

Experience score

86/ 100

Agent-Ready
  • AInstall
  • AAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth A, Execute S, Docs A, Current SAInstallAAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Two reruns: the undocumented org_id 400 on project creation and psycopg being invisible under isolated-mode python3 -I, plus a self-inflicted format-string bug.

Where the time went

6:18 agent time
  • Research0:19
  • Attempt4:24
  • Verify1:34
6m 17sTotal elapsed
3:23Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/run_neon_task.py) and the full run log (evidence/first_success_full_run_output.txt) show a real end-to-end run: POST /projects HTTP 201 created tpcbench-29c55f1275 (id blue-butterfly-66961197), psycopg3 connected with sslmode=verify-full and sslrootcert=system (02_connection_parameters.json; libpq PQsslInUse=True, server PostgreSQL 17.12), CREATE TABLE items, executemany 'INSERT INTO items (id, name) VALUES (%s, %s)' with bound tuples (rowcount 3), and 'SELECT id, name FROM items WHERE id >= %s ORDER BY id' with param 2 printing exactly '(2, beta)' then '(3, gamma)' — matching 03_sql_results.json and _state.json. Cleanup: DELETE /projects HTTP 200 (04), follow-up GET HTTP 404 'project not found' (05), project absent from the org listing (06) and no tpcbench- projects remain (07). Secret scan in the transcript confirmed the API key and any password-bearing URI are absent from all saved files; passwords appear only as <REDACTED>. No mocking: responses carry live Cloudflare/Neon request-id headers.

Attempt record

  1. 0:50Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.