Why benchmark agents at all#
Nodal hosts several kinds of agent, and the owner chooses between them constantly: Opus or Sonnet, Claude or Codex, a premium model or a small local one. Marketing claims do not answer the questions that matter in daily use. Does a more expensive model actually cost less once you count retries? How many tokens does a screenshot loop burn? Does a model that plans once and writes everything in one pass beat a model that iterates? The benchmarks exist to answer those questions with numbers instead of vibes, and they are a good fit for a livestream because anyone can look at the output and judge it.
There are two of them. Both use a short, fixed prompt, a single deliverable you can see, and a report written from the session transcripts. This page covers the Nodal-side write-up: the setup, the numbers and the lessons for building an agent workspace. The creator hub covers the same benchmarks from the stream angle, in front of the camera, on its benchmarks page.
How to read the numbers
These are small experiments on one machine with sessions the owner ran personally. They are useful for comparing approaches, not for ranking models for everyone. Where a run was interrupted or not clean, the notes say so, and dollar figures are equivalent API cost, not money billed to a subscription.
Benchmark one: the low-poly spaceship#
The task#
The agent is given a reference sheet, an image with eight views of a gray and yellow boxy shuttle with two-by-two engines and four landing legs, and one instruction: use the Blender MCP and model this spaceship in a low poly style. Blender runs on the owner’s machine and the agent drives it through a tool server. No asset generators and no other examples may be used. The deliverable is a Blender file and renders.
The first three runs#
The first sessions were meant to compare Claude Opus 5.5 with Claude Opus 5, but the second Opus 5.5 run was started by accident on the same model, so the owner did an Opus 5 run third. Each session wrote a report in the same format, with prompts, a tool-call timeline, token usage and how the model was made. The headline difference is one of method.
| Run | Model | Faces | Tokens for modelling | Approach |
|---|---|---|---|---|
| Session 1 | Claude Opus 5.5 | 672 | about 14,000 | One script built the whole ship; 11 tool calls |
| Session 2 | Claude Opus 5.5 | 939 | about 18,400 | One-shot generation with a helper file |
| Session 3 | Claude Opus 5 | 1,434 | about 78,500 | Build, render, inspect, diagnose, fix: an iterative loop |
The two Opus 5.5 sessions treated the job as a one-shot generation task: every coordinate was derived by reading the reference, a single Python script built the hull, legs, engines and details, and the only fix afterwards was toning down a blown-out engine glow. Total cost for the whole task, including fixed overhead such as the system prompt and the reference image, was about 76,000 and 81,000 tokens. The Opus 5 session behaved like a human modeler. It rebuilt the hull and engines several times, rendered to disk to inspect, and ended with about three times the geometry and a closer silhouette. It cost four to five times as much, and a further 47,000 tokens were spent on a scene the owner then told it to ignore. The Opus 5 report says plainly that asking one question up front would have been cheaper.
Later runs#
The same prompt was later given to Claude Sonnet 5.5, to a ChatGPT-side Codex agent running GPT-6.1 Sol and to GPT-6 Astra, with the same reference and the same Blender setup. Those runs produced larger renders and more polished lighting, because the agents rendered final images with soft shadows and gradients. One of them is shown below. The files are kept alongside the reports for comparison.
Lessons from the spaceship#
- Front-loaded reasoning is cheap. One careful generation beat a long loop on token cost, but the loop produced more detail.
- Viewport screenshots can lag. The Opus 5 session found screenshots trailing camera changes by a step. Rendering to a file and reading the image was both more accurate and slightly cheaper per check.
- A tool can kill the server. One call that reset the file disconnected the Blender tool server and silently reset the scene. Tool pitfalls are findings too.
- Know whether to build on the existing scene. Inspecting an old scene burned over a third of one run for nothing.
Benchmark two: the island#
The task#
One prompt for a self-contained web page: create a single HTML file using Three.js that renders an interactive low-poly 3D scene with procedural island terrain, a stylized water shader, a few trees, a skybox and dynamic lighting with shadows, plus a third-person character controlled with WASD and Space to jump with a simple camera follow, visually polished as a quality showcase. Output only the HTML file. A second version of the prompt asks for a high-quality, non-low-poly version of the same scene.
The efficiency audit#
For the first three Claude runs, the owner built an audit reconstructed from the raw session transcripts, not from self-reports. It counts tokens by category, de-duplicates streamed messages and converts to equivalent API cost.
| Model | Wall clock | API round-trips | Tokens processed | Equivalent cost | File |
|---|---|---|---|---|---|
| Claude Sonnet 5 | 18.5 min | 77 | 10.68 million | $3.45 | 23.8 KB, 692 lines |
| Claude Opus 5.5 | 9.7 min | 12 | 1.48 million | $2.95 | 43.2 KB, 780 lines |
| Claude Opus 5 | 29.5+ min | 48+ | 5.87 million | $6.93 (live snapshot) | 45.8 KB, 1,146 lines |
The surprise is in the middle row. Opus 5.5 processed 7.2 times fewer tokens than Sonnet 5 and used 6.4 times fewer round-trips, for a deliverable 1.8 times larger, and it was the cheapest of the three. The reason is turn count. Sonnet 5 wrote a working but half-size scene in three and a half minutes and then spent nine more in a 74-tool-call loop of screenshots and patches, re-sending around 135,000 tokens of context on every step. That loop, not the generation, made it expensive despite the cheapest per-token rate. Cache reads were 60 percent of its cost and 8 percent of Opus 5.5’s. Opus 5.5 thought for 8.5 minutes, wrote the whole 43 KB scene in one message and then verified with the console and three screenshots. The Opus 5 run was a live snapshot and not a clean comparison, because the owner interrupted it twice and asked for a bug fix the others were never given.
The audit’s own findings are worth quoting in spirit: front-loading reasoning is cheaper than iterating; turn count, not model tier, set the bill; a screenshot loop that re-reads a large context on every step is the specific inefficiency to avoid; and batching many browser actions into one call kept Opus 5 from paying for 89 separate context re-reads.
Later islands#
The island prompt was later run on Sonnet 5.5, GPT-6 Astra, GPT-6 Luna and GPT-6.1 Sol, and the high-quality variant on Sonnet 5.5 and Sol, with a final GPT-6.1 pass that improved the Sonnet 5 output. Each result is a standalone page you can open in a browser, and each agent gave its island a name. Two screenshots from the audit assets:
What these benchmarks mean for Nodal#
Nodal’s job is to make runs like these easy to launch, watch and compare. That is why the usage meter reports the provider’s own numbers rather than an estimate, why a finished turn shows its duration and session tokens, why the Review tab exists and why the same project can host a Codex session, a Claude session and an Ollama session side by side. The benchmark habit also shaped features. Counting tokens made the owner distrust unmeasured claims, which is why the notification and board tooling is deliberately cheap. The lessons about screenshot loops fed into how the browser tools are batched. Read more on the providers page and how it’s built.
Reproducing it#
The prompts are short enough to copy: the low-poly island prompt above, the same prompt minus the low-poly requirement for the high-quality variant, and the one-line spaceship prompt with a reference sheet of your own. For a fair comparison, give each agent a clean start, tell it not to look at other results, avoid interrupting it and record the session transcript so you can count tokens afterward. If you run your own, tell the community on Discord. The stream-side story is on the livestream page and the creator hub’s benchmark page.