Compare AI models on your own tasks

Paste your test cases, pick the models, and run them all at once. LLMCompare shows how well each model handles your task, next to what it costs and how long it takes.

  • Judged by your standardoutputs are checked against what your task needs, not spec sheets
  • No codeeverything runs in the browser
  • No API keyswe call the models and charge credits
Upload a spreadsheet
1 test case × 3 models = 3 calls

You will see the estimated credits before anything runs.

ModelsClaude Sonnet 5Gemini 3.7 FlashGPT-5.6 Terra
Claude Sonnet 5Done

1. Neon Skyline Silhouette: "SPACEX: CHASING THE UNIVERSE" rendered in glowing neon type over a towering cyberpunk cityscape silhouette that fades into a starfield, with rain-slicked streets reflecting the neon glow to…

Response time
11.8s
Output tokens
400
Cost
$0.0094
Gemini 3.7 FlashDone

1. Neon Cybernetic: "SPACEX: Chasing the Universe", featuring high-contrast neon lighting against dark cosmic space, though specific character imagery, release dates, and platforms are missing from the brief. 2.…

Response time
8.8s
Output tokens
141
Cost
$0.0042
GPT-5.6 TerraDone
  1. Neon Pursuit: “SPACEX: Chasing the Universe” — A high-contrast cyberpunk poster with electric neon typography, a futuristic cityscape, and a lone figure moving toward a glowing cosmic horizon.
  2. Cosmic Glitch:…
Response time
2.8s
Output tokens
127
Cost
$0.0018
Replay of test case 1 from our 19 Oct 2026 run, at 3× speed. Open the report

A real model comparison report

This report comes from an evaluation we ran on 19 October 2026: 3 test cases, 3 models, 1 run per test case. Every number can be checked in the full report.

C (GPT-5.6 Terra) passed 100% of judged outputs, 67 points above A, at 30% of A's cost per call.

  • A (Claude Sonnet 5) passed 33% of judged outputs, at $0.0083 per call.
  • B (Gemini 3.7 Flash) passed 67% of judged outputs, 34 points above A, at 65% of A's cost per call.
  • B (Gemini 3.7 Flash) averaged 11.1 s per call, against 10.3 s for A.
  • C (GPT-5.6 Terra) averaged 3.3 s per call, against 10.3 s for A.

Each test case ran once, so the quality result is an observation, not a rate.

The run has 3 test cases. With fewer than 10, read the result as a reference only.

Test conditions

Test cases
3
Models
Claude Sonnet 5, Gemini 3.7 Flash, GPT-5.6 Terra
Runs per test case
1
Prompt caching
On
Run date
19 October 2026
Prices checked
30 September 2026
Quality judged by
Model judge (GPT-6 Luna)
Measured from
Tencent Cloud (prod)
Model
Outputs passed by the model judge
Average response time
Cost per call
Claude Sonnet 5
1of 3
10.3s
$0.0083
Gemini 3.7 Flash
2of 3
11.1s
$0.0054
GPT-5.6 Terra
3of 3Highest
3.3sLowest
$0.0025Lowest
How it is judgedA judge model scored every output against criteria written for this task.
How it is measuredFrom our server (Tencent Cloud (prod)), from sending the request to receiving the complete output.
How it is calculatedThe provider's published price multiplied by the usage it reported. Reasoning tokens are included.

Result for every test case

Verdict shown: Model judge (GPT-6 Luna).

Test case
Claude Sonnet 5Anthropic
Gemini 3.7 FlashGoogle
GPT-5.6 TerraOpenAI
1Poster for a cyberpunk game called SPACEX: Chasing the Universe.
Passed11.8 s
Passed8.8 s
Passed2.8 s
2Hiring poster for Yujian, a counselling app start-up looking for a senior back-end engineer. It should feel professional and reliable, but not cold like a big corporation. It must show the salary (30K to 55K a month), that the role is remote first, and that applications close on the 15th of next month.
Failed8.5 s
Passed13.9 s
Passed4.4 s
3Pop-up exhibition for Zero Yoko, an independent perfume brand, held in a traditional machiya townhouse in Kyoto. Theme: the memory of scent. It runs for three days. Avoid the usual Japanese minimalist look.
Failed10.5 s
Failed10.6 s
Passed2.7 s

Quality against cost

Higher and further left is better.

  • Nothing else is both better and cheaper
The numbers in the chart
ContestantOutputs passed by the model judgeCost per callAverage response timeOn the frontier
A · Claude Sonnet 533%$0.008310.3 sNo
B · Gemini 3.7 Flash67%$0.005411.1 sNo
C · GPT-5.6 Terra100%$0.00253.3 sYes

Cost is the average for one call in this run. A gold ring marks a model that no other model beats on both quality and cost.

Outputs side by side

What each model returned for test case 1.

Test casePoster for a cyberpunk game called SPACEX: Chasing the Universe.
Claude Sonnet 5
  1. Neon Skyline Silhouette: "SPACEX: CHASING THE UNIVERSE" rendered in glowing neon type over a towering cyberpunk cityscape silhouette that fades into a starfield, with rain-slicked streets reflecting the neon glow to tie the urban cyberpunk setting to the space theme (the brief gives no platform, release date, or tagline, so none are included).
  2. Circuit-Universe Fusion: "SPACEX: CHASING THE UNIVERSE" set in a sharp, tech-style typeface where the letterforms dissolve into circuit-board patterns on one side and cosmic nebula textures on the other, visually merging cyberpunk tech with the…

11.8 s, 400 output tokens, 502 reasoning tokens

Gemini 3.7 Flash
  1. Neon Cybernetic: "SPACEX: Chasing the Universe", featuring high-contrast neon lighting against dark cosmic space, though specific character imagery, release dates, and platforms are missing from the brief.
  2. Terminal Glitch Minimal: "SPACEX: Chasing the Universe", focusing on bold, distorted futuristic typography over wireframe geometric grids, noting that key marketing assets like release information, credits, and developer details are missing from the brief.
  3. Tech-Noir HUD: "SPACEX: Chasing the Universe", framing deep space through a gritty, layered holographic cockpit interface,…

8.8 s, 141 output tokens, 940 reasoning tokens

GPT-5.6 Terra
  1. Neon Pursuit: “SPACEX: Chasing the Universe” — A high-contrast cyberpunk poster with electric neon typography, a futuristic cityscape, and a lone figure moving toward a glowing cosmic horizon.
  2. Cosmic Glitch: “SPACEX: Chasing the Universe” — The title fractures through digital glitch effects over a deep-space backdrop, blending starfields with sharp holographic interface elements.
  3. Chrome Ascension: “SPACEX: Chasing the Universe” — Metallic, reflective lettering rises above a dark futuristic skyline, with radiant celestial light cutting through atmospheric haze.

2.8 s, 127 output tokens

Shortened here. The full report has the complete output.

How to compare AI models with LLMCompare

  1. Add your test cases

    Type them, paste them one per line, or upload a spreadsheet. Add the expected result when you have one.

  2. Pick the models

    Choose the models and set their parameters. To test a prompt change, add a second version of the system prompt.

  3. Read the results

    Check the matrix, compare outputs side by side, review them blind, then share the report.

What people compare models for

Latest model comparisons

All model comparison articles

Models and prices

LLMCompare calculates cost from each provider's published price. Every price shows the date it was last checked.

LLMCompare lists 443 models from 65 providers as of 18 November 2026.

See all models and prices
ModelInputOutputCached input
GPT-6 LunaOpenAI$0.10$0.50$0.01
Gemini 3.7 FlashGoogle$0.75$3.75$0.075
GPT-5.6 TerraOpenAI$2.00$12.00$0.20
Claude Sonnet 5Anthropic$2.00$10.00$0.20
Text models that can run now. US dollars per million tokens. Prices checked on 30 September 2026.
ModelPer imageImage output, per million tokens
GPT Image 2.5 SunburstOpenAIDoes not apply$30.00
GPT Image 2.5 FlareOpenAIDoes not apply$30.00
Seedream 5.0 LiteByteDance Seed$0.035Does not apply
Image generation models that can run now. US dollars. Prices checked on 30 September 2026.

LLMCompare plans and credits

LLMCompare charges credits for the model calls in a run. The estimated credits are shown before a run starts, and a call that does not complete is not charged.

See all plans and credit prices
PlanPriceCreditsTest cases per evaluationCalls per run
FreeFree50 a day, up to 500 a month20100
Starter$10.00one-time10,000, no expiry2001,000
Expert$99.00per month120,000 every monthNo limit10,000
Prices in US dollars.

What LLMCompare does

  • LLMCompare is a web tool for comparing AI models on test cases that you supply.
  • LLMCompare sends each test case to every model you select and records the output, the token usage, the cost in US dollars and the response time of every call.
  • LLMCompare compares text models and image generation models. Video generation models are planned.
  • LLMCompare measures quality in three ways: rule checks, human review and scoring by a judge model. The three results are reported separately and are never merged into a single score.
  • Human review in LLMCompare is blind by default: model names are hidden and the outputs are shuffled until you reveal them.
  • LLMCompare calculates cost from each provider's published price. Every price shows the date it was last checked.
  • LLMCompare needs no code and no API keys. LLMCompare calls the models for you and charges credits for the usage.
  • LLMCompare does not switch to another model when a call fails. A timeout, a refusal or an empty response is recorded as that model's result.
  • LLMCompare lists 443 models from 65 providers as of 18 November 2026. 11 of them can be run in an evaluation.

What LLMCompare is not

  • LLMCompare is not a public leaderboard. LLMCompare does not rank models on general benchmarks.
  • LLMCompare is not a chat interface.
  • LLMCompare is not a production monitoring service. LLMCompare is used before launch, to choose a model.
  • LLMCompare is not owned by a model provider.

Questions about comparing AI models

How is LLMCompare different from a model leaderboard?
A leaderboard ranks models on general benchmarks that are the same for everyone. LLMCompare runs the test cases you supply, so the results describe how each model handles your task.
Do I need my own API keys?
No. LLMCompare calls the models with its own accounts and charges credits for the usage.
Which models can I compare?
LLMCompare supports text models and image generation models. The models page lists every supported model with its price and the date the price was checked. See all models and prices
How does LLMCompare calculate cost?
LLMCompare multiplies the usage reported by the provider by the provider's published price on the day of the run. Reasoning tokens are included because providers bill them as output.
Can I compare different versions of a prompt?
Yes. Add a second version of the system prompt and LLMCompare runs the same test cases with each version.
What happens when a model refuses a request or times out?
LLMCompare records the refusal or timeout as that model's result for the test case. LLMCompare does not retry with another model.
Are my test cases private?
Yes. An evaluation is visible only to you until you share it. When you share or publish a report, all of its test cases and outputs become visible to the people who can open it.
Can I share the results?
Yes. You can create a link that anyone with the link can open, or publish the report. People who open a shared report do not need an account.
How do I judge which output is better?
LLMCompare offers three methods: rule checks such as valid JSON or required keywords, blind human review, and scoring by a judge model that follows criteria you write.