Benchmark report

Task verified

7m 2s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Minimal: pooler config returns port 6543/transaction and a tenant-qualified username the API doesn't explain, so the agent had to know session port 5432 itself.

Where the time went

7:03 agent time
  • Starting0:01
  • Research0:24
  • Attempt4:55
  • Verify1:42
7m 2sTotal elapsed
3:03Time to first success
0:00Waited on the owner

Judge reason

Saved evidence shows the full real lifecycle. POST /v1/projects returned HTTP 201 for tpcbench-296022da (ref eiijpnsrlailjgwixyxl); GET pooler config returned the pooler host. Scoped credential was created by the documented SQL path (Supabase Postgres Roles guide, saved at evidence/05) after the agent proved via the live OpenAPI spec (evidence/14) that the only Management API role endpoint is experimental and accepts only {read_only}. evidence/07 shows CREATE ROLE + 3 GRANTs with information_schema.table_privileges listing exactly SELECT on public.items and pg_roles showing rolsuper/createdb/createrole/bypassrls all f. evidence/08 (transcript idx 113 tool result) shows connect as tpcbench_reader_296022da over PGSSLMODE=require on aws-0-us-east-1.pooler.supabase.com, SELECT returning 3 rows, 'ERROR: permission denied for table items' on INSERT, and count still 3. evidence/09 \conninfo reports TLSv1.3 TLS_AES_256_GCM_SHA384 (server pg_stat_ssl ssl=f is the expected pooler artifact). evidence/10 shows REVOKE x3 + DROP ROLE with follow-up counts 0/0; evidence/11 shows the revoked credential now fails with FATAL (EAUTHQUERY) user not found, exit 2. evidence/12 DELETE HTTP 200, evidence/13 follow-up GET 404 'Resource has been removed' plus an empty filtered project list. evidence/15 literal grep of all saved files against the token and both passwords returned 0 matches; the only password-bearing URL is the placeholder in Supabase's own public doc copy. All outputs are reproduced verbatim as real Bash tool results in the transcript; nothing mocked or hard-coded.

Attempt record

  1. 0:58Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.