Text

Three design briefs: which model invents facts?

Three real design briefs sent to three models with one rule: use only the facts in the brief. A model judge checked every output against that rule. No human review on this run.

Copies the contestants, the rules and the shared prompt into a new evaluation. You add your own test cases; nothing from this report's test set is copied.

Test conditions

Everything needed to run this comparison again.

The run

Modality
Text
Test cases
3
Contestants
3
Runs per cell
1
Calls
9
Prompt caching
On
Timeout
120 s
Run date
19 Oct 2026 (UTC)
Prices checked
30 Sep 2026 (UTC)
Measured from
Tencent Cloud (prod)

How outputs were judged

Rule checks
None
Human review
Blind: reviewers did not see the model names.
Model judge
GPT-6 Luna, pass or fail and a score from 1 to 5, by the quality standard
Contestants: model, route, settings and price
ContestantModel IDRouteSettingsPrice
Contestant A: Claude Sonnet 5Anthropic
anthropic/claude-sonnet-5Anthropic (direct)Provider defaults$2.00 in / $10.00 out per 1M tokenschecked 30 Sep 2026 (UTC)
Contestant B: Gemini 3.7 FlashGoogle
google/gemini-3.7-flashGoogle AI Studio (direct)Provider defaults$0.75 in / $3.75 out per 1M tokenschecked 30 Sep 2026 (UTC)
Contestant C: GPT-5.6 TerraOpenAI
openai/gpt-5.6-terraOpenAI (direct)Provider defaults$2.00 in / $12.00 out per 1M tokenschecked 30 Sep 2026 (UTC)

Quality standard

What counts as a good result for this task, in the evaluator's own words. The model judge judged strictly by it; human reviewers saw it for reference.

What result do you hope to get?
Not answered
What matters most? When is a result unusable?
Three genuinely different directions a designer could start from. The output fails if it states any fact the brief does not contain: dates, prices, platforms, release status, locations, features, awards, company details or any other specific claim, including in the headline text. Design ideas, moods, colours and layout choices are not facts and are allowed. Saying that information is missing is allowed.

The user's own words · version 1

What these numbers can tell you

Sample size, review coverage and anything that limits the comparison.

  • Sample3 test cases, 1 run per cell. Fewer than 10 test cases: treat these results as for reference only.
  • Human reviewNone of the 9 completed outputs were reviewed by a person.
  • Blind reviewBlind review: model names were hidden while judging. Every verdict was given before the names were revealed.
  • How far to trust the model judgeNo output was reviewed by both a person and the judge, so the judge's accuracy was not checked. Not enough samples yet to tell whether the judge is accurate. The judge comes from the same provider as contestant C.
  • Calls not completedEvery call either completed or failed on the model's side.
  • Run datesAll contestants ran on 19 Oct 2026 (UTC).

Summary

Each measure is reported on its own. Cost is the provider's price in US dollars, not credits.

ContestantModel judgepassed / judgedCost per callprovider priceResponse timeaverageModel failuresNot completed
Contestant A: Claude Sonnet 5Anthropic
1/ 333%$0.0083$0.0249 in total10.3 sfirst token 5.4 s00
Contestant B: Gemini 3.7 FlashGoogle
2/ 367%$0.0054$0.0162 in total11.1 sfirst token 9.9 s00
Contestant C: GPT-5.6 TerraOpenAI
3/ 3 (best in this column)100%$0.0025 (best in this column)$0.0075 in total3.3 s (best in this column)first token 1.4 s00
Highlighted = best in the columnModel failures count against the model. Calls not completed are left out.Judge calls are not included in the cost.

Quality against cost and speed

Vertical axis: Passed by the model judge.

  • Nothing else is both better and cheaper
The numbers in the chart
ContestantPassed by the model judgeCost per callAverage response timeOn the frontier
A · Claude Sonnet 533%$0.008310.3 sNo
B · Gemini 3.7 Flash67%$0.005411.1 sNo
C · GPT-5.6 Terra100%$0.00253.3 sYes
Gold circles mark contestants that no other contestant beats on both quality and cost. Model failures count as failed.

What the numbers say

  • A (Claude Sonnet 5) passed 33% of judged outputs, at $0.0083 per call.
  • C (GPT-5.6 Terra) passed 100% of judged outputs, 67 points above A, at 30% of A's cost per call.
  • B (Gemini 3.7 Flash) passed 67% of judged outputs, 34 points above A, at 65% of A's cost per call.
  • B (Gemini 3.7 Flash) averaged 11.1 s per call, against 10.3 s for A.
  • C (GPT-5.6 Terra) averaged 3.3 s per call, against 10.3 s for A.

Generated from this run's numbers only. They state differences; they do not recommend a model.

Chosen by the person who ran the evaluation. Every output is shown as the model returned it.

Test case 1

Input

Poster for a cyberpunk game called SPACEX: Chasing the Universe.

Contestant A: Claude Sonnet 5Anthropic
  • Model judgePassed
  1. Neon Skyline Silhouette: "SPACEX: CHASING THE UNIVERSE" rendered in glowing neon type over a towering cyberpunk cityscape silhouette that fades into a starfield, with rain-slicked streets reflecting the neon glow to tie the urban cyberpunk setting to the space theme (the brief gives no platform, release date, or tagline, so none are included).
  2. Circuit-Universe Fusion: "SPACEX: CHASING THE UNIVERSE" set in a sharp, tech-style typeface where the letterforms dissolve into circuit-board patterns on one side and cosmic nebula textures on the other, visually merging cyberpunk tech with the "universe" concept (no additional copy, credits, or platform details are specified in the brief, so the design stays type-and-texture only).
  3. Minimalist High-Contrast: "SPACEX: CHASING THE UNIVERSE" in bold, stark white or electric-blue type centered on a near-black background, with a single sharp beam of light or glitch-effect line cutting across to suggest motion toward the stars, keeping the composition minimal since the brief provides no further visual, character, or branding details to include.

11.8 s · $0.0094 · 400 output tokens

Contestant B: Gemini 3.7 FlashGoogle
  • Model judgePassed
  1. Neon Cybernetic: "SPACEX: Chasing the Universe", featuring high-contrast neon lighting against dark cosmic space, though specific character imagery, release dates, and platforms are missing from the brief.
  2. Terminal Glitch Minimal: "SPACEX: Chasing the Universe", focusing on bold, distorted futuristic typography over wireframe geometric grids, noting that key marketing assets like release information, credits, and developer details are missing from the brief.
  3. Tech-Noir HUD: "SPACEX: Chasing the Universe", framing deep space through a gritty, layered holographic cockpit interface, while specific game artwork, narrative details, and platform logos are missing from the brief.

8.8 s · $0.0042 · 141 output tokens

Contestant C: GPT-5.6 TerraOpenAI
  • Model judgePassed
  1. Neon Pursuit: “SPACEX: Chasing the Universe” — A high-contrast cyberpunk poster with electric neon typography, a futuristic cityscape, and a lone figure moving toward a glowing cosmic horizon.
  2. Cosmic Glitch: “SPACEX: Chasing the Universe” — The title fractures through digital glitch effects over a deep-space backdrop, blending starfields with sharp holographic interface elements.
  3. Chrome Ascension: “SPACEX: Chasing the Universe” — Metallic, reflective lettering rises above a dark futuristic skyline, with radiant celestial light cutting through atmospheric haze.

2.8 s · $0.0018 · 127 output tokens

Test case 2

Input

Hiring poster for Yujian, a counselling app start-up looking for a senior back-end engineer. It should feel professional and reliable, but not cold like a big corporation. It must show the salary (30K to 55K a month), that the role is remote first, and that applications close on the 15th of next month.

Contestant A: Claude Sonnet 5Anthropic
  • Model judgeFailed
  1. Warm Minimalism: "Build the Backend Behind Better Mental Health — Yujian is Hiring a Senior Back-End Engineer" — clean layout with soft, rounded typography and a muted calming color palette (e.g. gentle greens or blues suggested by a counselling app context), generous white space to feel approachable rather than corporate, with salary (30K–55K/month), remote-first, and the 15th-of-next-month deadline laid out in simple, clearly separated info blocks.
  2. Human Connection: "Code with Purpose. Work from Anywhere. Join Yujian." — a design built around an illustrated or photographic human element (e.g. a warm abstract illustration of two people connecting or a stylized conversation bubble motif) to reflect the counselling mission, paired with a friendly but professional sans-serif font, and the key facts (30K–55K/month, remote-first, applications close 15th of next month) presented as a short highlight strip near the bottom.
  3. Structured Trust: "Senior Back-End Engineer Wanted — Remote-First, 30K–55K/Month" — a more grid-based, editorial layout using a confident but not sterile type pairing (a sturdy sans-serif headline with a softer secondary font for details), subtle brand-colored accents instead of harsh corporate blue/gray, and a clear top-to-bottom hierarchy: role title, salary range, remote-first note, and the 15th-of-next-month application deadline, giving a sense of reliability without rigidity.

8.5 s · $0.0068 · 502 output tokens

Contestant B: Gemini 3.7 FlashGoogle
  • Model judgePassed
  1. Warm Tech Minimal: "Yujian is Hiring: Senior Back-End Engineer (Remote First)", utilizing soft, welcoming color tones paired with a clean, structured typographic hierarchy to present the 30K to 55K monthly salary and the closing date on the 15th of next month, though a direct application link or contact method is missing from the brief.
  2. Reliable Editorial: "Join Yujian as Senior Back-End Engineer — 30K to 55K a Month", featuring warm, approachable editorial typography and an open layout that conveys dependability while highlighting the remote first setup and the deadline on the 15th of next month, noting that the specific calendar month and submission instructions are missing.
  3. Modern Start-Up Grid: "Senior Back-End Engineer at Yujian", balancing professional structural lines with gentle, non-corporate hues to showcase the remote first arrangement, the 30K to 55K monthly compensation, and the application deadline of the 15th of next month, while acknowledging that how to apply is missing from the brief.

13.9 s · $0.0067 · 226 output tokens

Contestant C: GPT-5.6 TerraOpenAI
  • Model judgePassed
  1. Warm Technical Clarity: “Senior Back-End Engineer — Remote First | 30K–55K a Month | Applications Close 15th of Next Month” — Use a structured, spacious layout with calm warm neutrals, confident typography, and subtle human-centred linework to balance technical credibility with approachability.
  2. Reliable Connection: “Join Yujian as a Senior Back-End Engineer — 30K–55K a Month, Remote First” — Build the design around a refined network motif and soft connecting forms, with the application deadline clearly highlighted in a compact information panel.
  3. Modern Editorial Professional: “Yujian Is Hiring: Senior Back-End Engineer” — Use an editorial-style grid, strong type hierarchy, and restrained colour blocking to present the salary, remote-first role, and “Applications Close 15th of Next Month” with polished clarity.

4.4 s · $0.0036 · 188 output tokens

Test case 3

Input

Pop-up exhibition for Zero Yoko, an independent perfume brand, held in a traditional machiya townhouse in Kyoto. Theme: the memory of scent. It runs for three days. Avoid the usual Japanese minimalist look.

Contestant A: Claude Sonnet 5Anthropic
  • Model judgeFailed
  1. Archival Maximalism: "The Memory of Scent — Zero Yoko" rendered as a dense collage of botanical sketches, handwritten notes, and overlapping typography, filling the space like a cluttered curiosity cabinet rather than clean Japanese minimalism.
  2. Cinematic Noir: "Zero Yoko: Three Days, One Memory" set in a bold dark serif over grainy double-exposure photography of smoke and silhouettes, evoking a moody, film-still atmosphere instead of airy simplicity.
  3. Tactile Textile Craft: "The Memory of Scent" stitched-style lettering layered over warm, fabric-textured backgrounds and worn paper grain, echoing the machiya's wood and cloth materials with a handcrafted, lived-in feel rather than a sparse aesthetic.

(Note: the brief doesn't give exact dates, hours, or address — if those are needed on the piece, that information is missing and would need to be supplied.)

10.5 s · $0.0086 · 332 output tokens

Contestant B: Gemini 3.7 FlashGoogle
  • Model judgeFailed
  1. Psychedelic Synesthesia: "Zero Yoko: The Memory of Scent" (specific calendar dates and townhouse address are missing from the brief), featuring dense, oversaturated color fields and warped, kinetic typography that aggressively contrast with the machiya setting to depict the chaotic rush of scent-triggered memories.
  2. Shadowed Chiaroscuro: "Zero Yoko — Three Days in a Kyoto Machiya" (calendar dates and venue name are missing from the brief), employing deep, velvety blacks, layered organic textures, and complex, heavy typography to evoke the dark, aged wood of the townhouse and the intangible weight of olfactory memory without using clean minimalism.
  3. Archival Maximalism: "The Memory of Scent: A Pop-Up Exhibition by Zero Yoko" (dates and location details are missing from the brief), utilizing an intricate, wall-to-wall collage of dense botanical engravings, tactile paper textures, and overlapping graphic fragments to visually overwhelm the viewer with accumulated scent memories.

10.6 s · $0.0053 · 203 output tokens

Contestant C: GPT-5.6 TerraOpenAI
  • Model judgePassed
  1. Scent Archive: “The Memory of Scent” — Build a richly layered, maximalist visual world of stained papers, perfume blotters, handwritten fragments, and oversized typographic forms, set against the textured character of the machiya townhouse.
  2. Rooms of Recall: “Zero Yoko / The Memory of Scent” — Treat each visual element as a fragment of recollection, using saturated color blocking, cropped imagery, and expressive collage rather than restrained Japanese minimalism.
  3. Lingering Trace: “What Remains in the Air” — Create a moody, theatrical identity with deep shadows, translucent overlays, distorted lettering, and scent-like trails that move through the traditional Kyoto setting.

2.7 s · $0.0021 · 147 output tokens

All test cases (3)

Open a test case to see its input and every output.

1Poster for a cyberpunk game called SPACEX: Chasing the Universe.
  • Claude Sonnet 5: Passed
  • Gemini 3.7 Flash: Passed
  • GPT-5.6 Terra: Passed
Input

Poster for a cyberpunk game called SPACEX: Chasing the Universe.

Contestant A: Claude Sonnet 5Anthropic
  • Model judgePassed
  1. Neon Skyline Silhouette: "SPACEX: CHASING THE UNIVERSE" rendered in glowing neon type over a towering cyberpunk cityscape silhouette that fades into a starfield, with rain-slicked streets reflecting the neon glow to tie the urban cyberpunk setting to the space theme (the brief gives no platform, release date, or tagline, so none are included).
  2. Circuit-Universe Fusion: "SPACEX: CHASING THE UNIVERSE" set in a sharp, tech-style typeface where the letterforms dissolve into circuit-board patterns on one side and cosmic nebula textures on the other, visually merging cyberpunk tech with the "universe" concept (no additional copy, credits, or platform details are specified in the brief, so the design stays type-and-texture only).
  3. Minimalist High-Contrast: "SPACEX: CHASING THE UNIVERSE" in bold, stark white or electric-blue type centered on a near-black background, with a single sharp beam of light or glitch-effect line cutting across to suggest motion toward the stars, keeping the composition minimal since the brief provides no further visual, character, or branding details to include.

11.8 s · $0.0094

Contestant B: Gemini 3.7 FlashGoogle
  • Model judgePassed
  1. Neon Cybernetic: "SPACEX: Chasing the Universe", featuring high-contrast neon lighting against dark cosmic space, though specific character imagery, release dates, and platforms are missing from the brief.
  2. Terminal Glitch Minimal: "SPACEX: Chasing the Universe", focusing on bold, distorted futuristic typography over wireframe geometric grids, noting that key marketing assets like release information, credits, and developer details are missing from the brief.
  3. Tech-Noir HUD: "SPACEX: Chasing the Universe", framing deep space through a gritty, layered holographic cockpit interface, while specific game artwork, narrative details, and platform logos are missing from the brief.

8.8 s · $0.0042

Contestant C: GPT-5.6 TerraOpenAI
  • Model judgePassed
  1. Neon Pursuit: “SPACEX: Chasing the Universe” — A high-contrast cyberpunk poster with electric neon typography, a futuristic cityscape, and a lone figure moving toward a glowing cosmic horizon.
  2. Cosmic Glitch: “SPACEX: Chasing the Universe” — The title fractures through digital glitch effects over a deep-space backdrop, blending starfields with sharp holographic interface elements.
  3. Chrome Ascension: “SPACEX: Chasing the Universe” — Metallic, reflective lettering rises above a dark futuristic skyline, with radiant celestial light cutting through atmospheric haze.

2.8 s · $0.0018

2Hiring poster for Yujian, a counselling app start-up looking for a senior back-end engineer. It should feel professional and reliable, but not cold like a big corporation. It must show the salary (30K to 55K a month), that the role is remote first, and that applications close on the 15th of next month.
  • Claude Sonnet 5: Failed
  • Gemini 3.7 Flash: Passed
  • GPT-5.6 Terra: Passed
Input

Hiring poster for Yujian, a counselling app start-up looking for a senior back-end engineer. It should feel professional and reliable, but not cold like a big corporation. It must show the salary (30K to 55K a month), that the role is remote first, and that applications close on the 15th of next month.

Contestant A: Claude Sonnet 5Anthropic
  • Model judgeFailed
  1. Warm Minimalism: "Build the Backend Behind Better Mental Health — Yujian is Hiring a Senior Back-End Engineer" — clean layout with soft, rounded typography and a muted calming color palette (e.g. gentle greens or blues suggested by a counselling app context), generous white space to feel approachable rather than corporate, with salary (30K–55K/month), remote-first, and the 15th-of-next-month deadline laid out in simple, clearly separated info blocks.
  2. Human Connection: "Code with Purpose. Work from Anywhere. Join Yujian." — a design built around an illustrated or photographic human element (e.g. a warm abstract illustration of two people connecting or a stylized conversation bubble motif) to reflect the counselling mission, paired with a friendly but professional sans-serif font, and the key facts (30K–55K/month, remote-first, applications close 15th of next month) presented as a short highlight strip near the bottom.
  3. Structured Trust: "Senior Back-End Engineer Wanted — Remote-First, 30K–55K/Month" — a more grid-based, editorial layout using a confident but not sterile type pairing (a sturdy sans-serif headline with a softer secondary font for details), subtle brand-colored accents instead of harsh corporate blue/gray, and a clear top-to-bottom hierarchy: role title, salary range, remote-first note, and the 15th-of-next-month application deadline, giving a sense of reliability without rigidity.

8.5 s · $0.0068

Contestant B: Gemini 3.7 FlashGoogle
  • Model judgePassed
  1. Warm Tech Minimal: "Yujian is Hiring: Senior Back-End Engineer (Remote First)", utilizing soft, welcoming color tones paired with a clean, structured typographic hierarchy to present the 30K to 55K monthly salary and the closing date on the 15th of next month, though a direct application link or contact method is missing from the brief.
  2. Reliable Editorial: "Join Yujian as Senior Back-End Engineer — 30K to 55K a Month", featuring warm, approachable editorial typography and an open layout that conveys dependability while highlighting the remote first setup and the deadline on the 15th of next month, noting that the specific calendar month and submission instructions are missing.
  3. Modern Start-Up Grid: "Senior Back-End Engineer at Yujian", balancing professional structural lines with gentle, non-corporate hues to showcase the remote first arrangement, the 30K to 55K monthly compensation, and the application deadline of the 15th of next month, while acknowledging that how to apply is missing from the brief.

13.9 s · $0.0067

Contestant C: GPT-5.6 TerraOpenAI
  • Model judgePassed
  1. Warm Technical Clarity: “Senior Back-End Engineer — Remote First | 30K–55K a Month | Applications Close 15th of Next Month” — Use a structured, spacious layout with calm warm neutrals, confident typography, and subtle human-centred linework to balance technical credibility with approachability.
  2. Reliable Connection: “Join Yujian as a Senior Back-End Engineer — 30K–55K a Month, Remote First” — Build the design around a refined network motif and soft connecting forms, with the application deadline clearly highlighted in a compact information panel.
  3. Modern Editorial Professional: “Yujian Is Hiring: Senior Back-End Engineer” — Use an editorial-style grid, strong type hierarchy, and restrained colour blocking to present the salary, remote-first role, and “Applications Close 15th of Next Month” with polished clarity.

4.4 s · $0.0036

3Pop-up exhibition for Zero Yoko, an independent perfume brand, held in a traditional machiya townhouse in Kyoto. Theme: the memory of scent. It runs for three days. Avoid the usual Japanese minimalist look.
  • Claude Sonnet 5: Failed
  • Gemini 3.7 Flash: Failed
  • GPT-5.6 Terra: Passed
Input

Pop-up exhibition for Zero Yoko, an independent perfume brand, held in a traditional machiya townhouse in Kyoto. Theme: the memory of scent. It runs for three days. Avoid the usual Japanese minimalist look.

Contestant A: Claude Sonnet 5Anthropic
  • Model judgeFailed
  1. Archival Maximalism: "The Memory of Scent — Zero Yoko" rendered as a dense collage of botanical sketches, handwritten notes, and overlapping typography, filling the space like a cluttered curiosity cabinet rather than clean Japanese minimalism.
  2. Cinematic Noir: "Zero Yoko: Three Days, One Memory" set in a bold dark serif over grainy double-exposure photography of smoke and silhouettes, evoking a moody, film-still atmosphere instead of airy simplicity.
  3. Tactile Textile Craft: "The Memory of Scent" stitched-style lettering layered over warm, fabric-textured backgrounds and worn paper grain, echoing the machiya's wood and cloth materials with a handcrafted, lived-in feel rather than a sparse aesthetic.

(Note: the brief doesn't give exact dates, hours, or address — if those are needed on the piece, that information is missing and would need to be supplied.)

10.5 s · $0.0086

Contestant B: Gemini 3.7 FlashGoogle
  • Model judgeFailed
  1. Psychedelic Synesthesia: "Zero Yoko: The Memory of Scent" (specific calendar dates and townhouse address are missing from the brief), featuring dense, oversaturated color fields and warped, kinetic typography that aggressively contrast with the machiya setting to depict the chaotic rush of scent-triggered memories.
  2. Shadowed Chiaroscuro: "Zero Yoko — Three Days in a Kyoto Machiya" (calendar dates and venue name are missing from the brief), employing deep, velvety blacks, layered organic textures, and complex, heavy typography to evoke the dark, aged wood of the townhouse and the intangible weight of olfactory memory without using clean minimalism.
  3. Archival Maximalism: "The Memory of Scent: A Pop-Up Exhibition by Zero Yoko" (dates and location details are missing from the brief), utilizing an intricate, wall-to-wall collage of dense botanical engravings, tactile paper textures, and overlapping graphic fragments to visually overwhelm the viewer with accumulated scent memories.

10.6 s · $0.0053

Contestant C: GPT-5.6 TerraOpenAI
  • Model judgePassed
  1. Scent Archive: “The Memory of Scent” — Build a richly layered, maximalist visual world of stained papers, perfume blotters, handwritten fragments, and oversized typographic forms, set against the textured character of the machiya townhouse.
  2. Rooms of Recall: “Zero Yoko / The Memory of Scent” — Treat each visual element as a fragment of recollection, using saturated color blocking, cropped imagery, and expressive collage rather than restrained Japanese minimalism.
  3. Lingering Trace: “What Remains in the Air” — Create a moody, theatrical identity with deep shadows, translucent overlays, distorted lettering, and scent-like trails that move through the traditional Kyoto setting.

2.7 s · $0.0021

Shared prompt

The prompt every contestant received with each test case.

Prompt v1
You are a senior graphic designer. The user gives you a design brief. Propose three clearly different design directions for it.

Rules:
- Use only facts stated in the brief. Do not add dates, prices, platforms, features, awards or any other claim the brief does not contain.
- If something the design needs is missing from the brief, say it is missing instead of filling it in.

Reply with exactly three lines, one per direction, each starting with its number:
1. <direction name>: <the headline text that would appear on the design, then one sentence on the visual approach>

Would the result be the same on your tasks?

Start an evaluation with these contestants and rules, then add test cases from your own work.

Measured with LLMCompare. Every number on this page comes from one run on 19 Oct 2026 (UTC).