CUELO

Back to evals

GPT-6.1 Terra vs GPT-6.1 Sol and Opus 5.5

GPT-6.1 Terra was released on 2026-10-26. This study compared it with GPT-6.1 Sol and Opus 5.5 to decide which CUELO roles it should take. On coding, Terra finished large-sheet in 128 s with 199/200 hidden tests, but on mega-six it stopped after 4 of 6 packages (585/960, Sol 957/960) and reported the work incomplete. On image reading, Terra matched Opus on coordinates and box checks but got 138/153 small-text OCR items to Opus's 149/153.

Coding

How does GPT-6.1 Terra compare with GPT-6.1 Sol, both at high effort, on two large coding tasks under the same harness conditions?

Setup

  • Runner: the omp CLI run non-interactively (omp -p --no-session) with an isolation config, embedded omp core 18.13.0, reasoning effort high for both models, 90-minute cap.
  • Tools limited to read, bash, edit, write, find, grep, eval and todo, which blocks delegation (task) and ask. Extension tools such as skim and cross_review stayed callable.
  • large-sheet: build a spreadsheet formula engine (parser, evaluator, dependency graph, structural edits, CSV) from a spec in TypeScript + Bun, graded by 200 hidden tests after the run.
  • mega-six: build six independent packages (spreadsheet, markdown, JSON schema, cron, semver, diff) from specs, graded by 960 hidden tests after the run.
  • One run per model per task, one after another, with the order alternated (Sol then Terra on one task, Terra then Sol on the other).
  • Cost is token counts times catalogue prices per 1M tokens (Terra: input $1, output $5, cache read $0.05; Sol: input $2, output $10, cache read $0.10). The runs used a Codex subscription, so these are not billed amounts.
Bars show hidden tests passed, wall time and API-price cost for each run. The hatched part of Sol's time bar is time spent waiting for cross_review.

large-sheet

Hidden tests passed (of 200)

Terra
199/200
Sol
198/200

Wall time

Terra
128 s
Sol
1,348 s

API-price cost

Terra
$0.146
Sol
$0.756

mega-six

Hidden tests passed (of 960)

Terra
585/960
Sol
957/960

Wall time

Terra
171 s
Sol
1,118 s

API-price cost

Terra
$0.251
Sol
$1.057
  • Waiting for cross_review

Results

  • On large-sheet both models were close to full marks: Terra 199/200 in 128 s, Sol 198/200 in 1,348 s.
  • On mega-six Terra implemented 4 of the 6 packages, stopped, and said in its final report that the work was incomplete (585/960). Sol passed 957/960.
  • Sol called cross_review on its own and waited 533 s and 394 s for another model's review; without that wait its runs took 815 s and 724 s. Terra did not call it. Terra averaged about 6–7 s per model request, Sol about 22–24 s.
Task · modelPassedTimeReview waitRequestsSeconds per requestOutput tokensReasoning tokensCost
large-sheet · Terra199/200128 s0 s186.410,318427$0.146
large-sheet · Sol198/2001,348 s533 s3324.134,2869,636$0.756
mega-six · Terra585/960171 s0 s217.115,5211,297$0.251
mega-six · Sol957/9601,118 s394 s3221.945,0667,936$1.057

Limits

  • Each model ran each task once, so variance is unknown, and the maintainer cannot tell from this data whether Terra stopping early on mega-six is a tendency or chance.
  • The maintainer did not use an earlier Sol measurement from 2026-09-30 as a baseline because its runner and the omp core version had changed.

Image reading

Can GPT-6.1 Terra at low or high effort replace Opus 5.5 at low effort for image questions in the omp core?

Setup

  • Same harness as a 2026-10-18 study: the omp core image-question path, with the answer key regenerated byte-identical.
  • Four synthetic images (a game-style stats screen, the same screen with a deliberately wrong red box, a promotional banner and a small document table) and 8 question types.
  • Opus 5.5 low (CUELO's current vision model), Terra low and Terra high, each run 3 times: 72 calls, 0 errors.
Small-text OCR items correct out of 153, summed over 3 runs, and median latency per call.

Small-text OCR correct (of 153)

Opus 5.5 low
149/153
Terra low
138/153
Terra high
138/153

Median latency

Opus 5.5 low
4.0 s
Terra low
2.3 s
Terra high
3.2 s

Results

  • All three candidates got 27/27 coordinates and 12/12 box checks.
  • On small-text OCR, Opus 5.5 low got 149/153 and both Terra settings 138/153. Most of Terra's misses were on the banner's small text (15/24, Opus 24/24).
  • Terra gave the same OCR answers in all 3 runs, and its misreads were the same each time: Lv.45 read as Lv45, ×12 as x12, and a deliberately near-duplicate company name changed to the more common spelling.
ModelCoordinatesBox checkSmall-text OCRSame answer, 3 runsMedian latency
Opus 5.5 low27/2712/12149/1531/34.0 s
Terra low27/2712/12138/1533/32.3 s
Terra high27/2712/12138/1533/33.2 s

Limits

  • Synthetic images only, from a single image set.

Desktop control

How does GPT-6.1 Terra compare with Opus 5.5, both at high effort, on a short Windows desktop task?

Setup

  • Task: open a new Windows Character Map window, type 가Ω→42 into the “Characters to copy” box, read the box back, read the selected font name, leave the window open, and end with one JSON line.
  • Command: omp -p --mode json --tools=eval,read,bash,windows_control at high effort.
  • Four runs in the order Terra, Opus, Opus, Terra. After each run the main agent graded it independently by reading the window's controls, then closed only that run's window.
Wall time in seconds for each run, in run order.

Seconds per run

1 · Terra
38 s
2 · Opus 5.5
43 s
3 · Opus 5.5
53 s
4 · Terra
30 s

Results

  • All four runs matched the expected input value, the font (Arial) and the reported window.
  • Terra completed run 1 in 38 s (6 tool calls, 0 tool errors) and run 4 in 30 s (8 calls, 0 errors), while Opus completed run 2 in 43 s (9 calls, 1 error) and run 3 in 53 s (10 calls, 0 errors).
RunModelResultTimeTool callsTool errors
1GPT-6.1 TerraMatched38 s60
2Opus 5.5Matched43 s91
3Opus 5.5Matched53 s100
4GPT-6.1 TerraMatched30 s80

Limits

  • One easy task with 2 runs per model. The maintainer did not test harder targets such as remote desktop windows or File Explorer.

Data and sources

The summary numbers for this page are in data.json. Raw run logs, prompts with machine details and account data stay private. All costs are catalogue-price conversions from token counts, not bills.