Benchmark report

Task verified

16m 18s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current SSInstallSAuthAExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Fly's default Depot remote builder and all public registries were proxy-blocked, forcing ~70 tool calls to fetch a Go toolchain from GitHub and build a FROM-scratch image.

Where the time went

16:18 agent time
  • Starting0:01
  • Research0:39
  • Attempt13:31
  • Verify2:07
16m 18sTotal elapsed
9:21Time to first success
0:00Waited on the owner

Judge reason

Saved flyctl outputs prove the full cycle on app tpcbench-642619fac2b at https://tpcbench-642619fac2b.fly.dev/. outputs/03-deploy-v1.txt and 04-deploy-v2.txt show release 1 (image img_3mno4wj2kee9vk18) then release 2 (img_2wokpyqr3nyev3g1); transcript polls captured {"version":"v1","servedAt":"2026-11-08T19:01:49.982Z"} then {"version":"v2",...19:03:13.115Z}. Rollback used the documented redeploy-previous-artifact path: fly releases --image --json (05-releases.json) gave release 1's ImageRef, and fly deploy --config fly.toml --image <ref> was run from rollback/ containing only fly.toml; 06-rollback-deploy.txt shows 'Searching for image ... remotely / image found: img_3mno4wj2kee9vk18' — the same image object as release 1, no build and no docker push, so v1 was not rebuilt or re-uploaded. The URL then served {"version":"v1",...19:04:44.468Z} plus two later live 200s with differing servedAt, confirming per-request time. Version is a compiled-in Go const in v1/main.go and v2/main.go, no env var. Cleanup: 09-apps-destroy.txt 'Destroyed app tpcbench-642619fac2b', apps list shows no tpcbench app, and evidence/final-fetch-after-delete.txt shows HTTP/2 502 'bad gateway' on three fetches. Grep of all saved artifacts and the transcript found no token material (token stayed in a private file read by a wrapper and piped to docker login).

Attempt record

  1. 1:14Answered from saved key: Fly.io API token (FLY_API_TOKEN)no wait
  2. —First success verified

Attempt artifacts saved.