Benchmark report

Task verified

6m 8s elapsed

The saved attempt met the documented success criterion.

Experience score

83/ 100

Needs Alignment
  • AInstall
  • SAuth
  • AExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute A, Docs A, Current AAInstallSAuthAExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Sending required a four-call template dance (create, update content, publish, send), and an OpenAPI-advertised languageCode field caused a 400 that forced a second run.

Where the time went

6:09 agent time
  • Research0:17
  • Attempt4:05
  • Verify1:46
6m 8sTotal elapsed
3:08Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (loops-task/send-transactional.mjs) makes an authenticated Bearer request to Loops' documented send operation POST https://app.loops.so/api/v1/transactional. Evidence 07-send-transactional.json records the live response: HTTP 200 OK with body {"success":true}, Cloudflare headers (cf-ray a541665b4e6cfb74-PDX, date Mon, 02 Nov 2026 07:24:29 GMT, x-ratelimit-remaining 9) — not a mock. The saved request body {email: delivered@example.com, transactionalId: cmvyxo6321fui0jxohpwibifw, addToAudience: false, dataVariables:{customerName, orderNumber, shipDate}} matches the code's sendBody exactly. Recipient is the provider-documented sandbox domain: I independently fetched https://loops.so/docs/transactional, which states "no emails will be sent to @example.com or @test.com email addresses" while API calls still return success. Sender was discovered through the API (GET /v1/email-messages/cmvywt35s1fc10jx4cqanst6t -> fromName "Prompting Co", fromEmail local part "rick" on the team's verified sending domain), not hard-coded. The template was created via the API (POST /v1/transactional-emails -> 201, POST /v1/email-messages/{id} -> 200, publish -> 200, dataVariables [customerName, orderNumber, shipDate]) with subject "Loops API test: order LB-20261102 confirmed" and a plain-format body, matching the template id used in the send. run-stdout.txt prints "HTTP status: 200 OK", body {"success":true}, and states no message identifier; I confirmed against https://loops.so/docs/api-reference/send-transactional-email that the documented 200 shape is {success: boolean} with no message id, so the "API returned none" statement is correct. Transport was HTTPS REST, not SMTP or dashboard; the earlier failed run aborted at step 05 (HTTP 400 translations) before any send, so exactly one POST /v1/transactional occurred.

Attempt record

  1. 1:38Requested Test recipient email addressdeclined automatically: campaigns don't wait for people
  2. —First success verified

Attempt artifacts saved.