Benchmark report

Task verified

11m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Team default 'private for new projects' made the first public fetch return a 401 edge-access redirect, costing a doc hunt before PATCH {"sso_login": false} fixed it.

Where the time went

11:14 agent time
  • Starting0:01
  • Research0:25
  • Attempt8:37
  • Verify2:11
11m 13sTotal elapsed
7:05Time to first success
0:00Waited on the owner

Judge reason

Saved HTTP logs prove the full flow against live Netlify API v1 (no dashboard). Site tpcbench-92e466 created via POST /rick-chen/sites/ after resolving the team with GET /accounts (logs/00,01). v1 deployed by documented file-digest method (deploy 6af1cf1331534cbf77c7d8c5, function zip sha256 cff77e34...) and the public URL https://tpcbench-92e466.netlify.app/.netlify/functions/version returned HTTP 200 {"version":"v1","servedAt":"2026-11-09T13:51:08.508Z"} (logs/20-public-v1.txt); v2 deploy 6af1cfd6a97272159dda3d6e served {"version":"v2",...13:51:32.417Z} on the same URL (logs/21). Rollback used the documented POST /sites/{id}/deploys/{v1_id}/restore -> HTTP 200 with id==original v1 deploy, published_at 13:51:41.355Z and available_functions digest cff77e34... (logs/30); rollback.py makes only that one call, no file or zip upload, so v1 was neither rebuilt nor re-uploaded. Four subsequent fetches returned v1 with distinct servedAt values (13:51:42/51/53/54), proving per-request computation; version is a hard-coded literal in code/v1/version.js and code/v2/version.js, not an env var. Cleanup: DELETE /sites/{id} 204, GET site 404, GET /sites returned [] (no tpcbench- left), and the final public fetch returned HTTP 404 Netlify not-found (logs/40-43). Token was read from /tmp/nlpriv/token (deleted at end) and never logged; the only Bearer string in saved files is the docs placeholder 'Bearer YOUR_OAUTH2_ACCESS_TOKEN'.

Attempt record

  1. 0:48Answered from saved key: Netlify personal access tokenno wait
  2. —First success verified

Attempt artifacts saved.