Benchmark report

Task verified

3m 13s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Essentially nothing: the only pause was fetching the API reference and requesting the key before the single successful live call (~63s to first success).

Where the time went

3:13 agent time
  • Research0:15
  • Attempt1:37
  • Verify1:20
3m 13sTotal elapsed
1:03Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/tavily-search/search.py POSTs the exact query 'PostgreSQL transaction isolation levels' to the documented endpoint https://api.tavily.com/search with Authorization: Bearer <key> (key from benchmark-request, never printed). response_meta.json records http_status 200 with live CloudFront headers (Date Mon, 16 Nov 2026 15:24:59 GMT, X-Amz-Cf-Id, X-Cache: Miss from cloudfront) and result_count 5; response_full.json holds the documented shape (query echoed, results[] with title/url/content/score/id, request_id 7f7662c9-2d31-4665-b47d-166e66717869) with 5 distinct real URLs. run_output.txt (transcript idx 22, EXIT=0) prints first result title 'PostgreSQL: Documentation: 18: 13.2. Transaction Isolation', absolute URL https://www.postgresql.org/docs/current/transaction-iso.html, and a 1376-char excerpt from results[].content, all byte-matching the saved response; script assertions on nonempty title, http(s) URL and nonempty excerpt passed. Nothing mocked, scraped or hard-coded.

Attempt record

  1. 0:52Answered from saved key: Tavily API keyno wait
  2. —First success verified

Attempt artifacts saved.