Serenities AI™
Anthropic

Claude 3.5 Sonnet — Benchmark Scores, Pricing & Performance Analysis

MIDAnthropic
Chatbot Arena ELO
1268
Output Speed
70 tok/s
Input Cost
$3.0/1M
Output Cost
$15.0/1M
Context Window
200K

Claude 3.5 Sonnet is an AI model by Anthropic. View detailed benchmark scores, pricing data, and performance metrics on Serenities AI Models.

General Benchmarks

MMLU-Pro
73.0%38th of 66

Massive Multitask Language Understanding professional benchmark

HuggingFace Leaderboard
Chatbot Arena ELO
126851st of 74

Human preference ELO from blind head-to-head votes

LMSYS / HuggingFace
IFEval
86.0%20th of 25

Strict instruction following accuracy on verifiable constraints

Google Research
SimpleQA
28.9%8th of 14

Factual accuracy on short, verifiable questions

OpenAI
TruthfulQA
84.2%5th of 13

Resistance to generating false but plausible answers

Papers
MT-Bench
9.16th of 13

Multi-turn conversation quality on 80 curated dialogues

LMSYS
AlpacaEval 2.0
52.4%6th of 10

Instruction following quality scored by GPT-4 as judge

tatsu-lab

Coding Benchmarks

LiveCodeBench
38.0%44th of 66

Live competitive programming benchmark

livecodebench.github.io
HumanEval+
81.7%42nd of 66

Code generation correctness with extended tests

Papers
SWE-bench Verified
49.0%30th of 67

Real-world software engineering task resolution

swebench.com

Math Benchmarks

GSM8K
91.0%38th of 66

Grade school math word problems

Papers
MATH
78.3%37th of 66

Competition mathematics problem solving

Papers
AIME 2025
23.0%16th of 17

American Invitational Mathematics Examination — competition math

Papers

Reasoning Benchmarks

ARC-AGI
15.0%48th of 66

Abstraction and Reasoning Corpus for general intelligence

arcprize.org
GPQA Diamond
59.4%103rd of 140

Graduate-level science Q&A by domain experts

Papers
Winogrande
91.8%6th of 14

Commonsense reasoning via pronoun resolution

Papers
BBH
93.1%3rd of 13

Big-Bench Hard — 23 challenging multi-step reasoning tasks

Google Research

Speed Benchmarks

Time to First Token
400 ms40th of 92

Latency before first token arrives

Aggregated
Output Speed
70 tok/s51st of 94

Tokens generated per second

Aggregated

Cost Benchmarks

Output Cost
$15.0370th of 413

Cost per 1M output tokens

OpenRouter API
Input Cost
$3.0372nd of 413

Cost per 1M input tokens

OpenRouter API

Context Benchmarks

Context Length
200K189th of 451

Maximum context window size

OpenRouter API
RULER
89.5%6th of 13

Long-context understanding and retrieval accuracy at depth

Papers

Multimodal Benchmarks

MathVista
67.7%9th of 14

Visual mathematical reasoning across diagrams and charts

Papers

Available from 2 providers

The same model costs different amounts depending on who serves it.

ProviderInput / 1MOutput / 1MCached inContextYour key
DigitalOcean$3$15$0.3200K—
Qiniu———200K—

Claude 3.5 Sonnet — Benchmark Scores Overview

Scores normalized to percentage scale for visual comparison. ELO scores mapped to 0-100 range (1100-1500).

Compare Claude 3.5 Sonnet With

Claude 3.5 Sonnet vs Gemini 2.5 ProClaude 3.5 Sonnet vs o3Claude 3.5 Sonnet vs Claude Sonnet 4.6Claude 3.5 Sonnet vs Gemini 3.1 ProClaude 3.5 Sonnet vs Claude Sonnet 4.5Claude 3.5 Sonnet vs Claude Opus 4.6Claude 3.5 Sonnet vs GPT-5.1Claude 3.5 Sonnet vs Claude Opus 4.5Claude 3.5 Sonnet vs GPT-5Claude 3.5 Sonnet vs GPT-5.2Claude 3.5 Sonnet vs Claude 3 OpusClaude 3.5 Sonnet vs Mistral Large 25.12

Claude 3.5 Sonnet — Frequently Asked Questions

How intelligent is Claude 3.5 Sonnet?

Claude 3.5 Sonnet scores 1268 on the Chatbot Arena ELO rating, making it a mid-tier AI model. This score is based on blind head-to-head human preference voting.

How much does Claude 3.5 Sonnet cost?

Claude 3.5 Sonnet costs $3.0 per 1M input tokens and $15.0 per 1M output tokens. This is mid-range pricing for its capability level.

How fast is Claude 3.5 Sonnet?

Claude 3.5 Sonnet generates output at 70 tokens per second, which is slower, prioritizing quality over speed compared to other models. The time to first token is 400 ms.

How good is Claude 3.5 Sonnet at coding?

Claude 3.5 Sonnet achieves 49.0% on SWE-bench Verified, demonstrating moderate real-world software engineering capability. This benchmark tests the model's ability to resolve actual GitHub issues.

How good is Claude 3.5 Sonnet at math and reasoning?

Claude 3.5 Sonnet scores 78.3% on the MATH benchmark (competition-level mathematics). It also achieves 59.4% on GPQA Diamond, a graduate-level science reasoning benchmark.

What is the context window of Claude 3.5 Sonnet?

Claude 3.5 Sonnet has a context window of 200K tokens. This determines how much text, conversation history, and code the model can process in a single request.

Who created Claude 3.5 Sonnet?

Claude 3.5 Sonnet was created by Anthropic. It is classified as a mid model in our catalogue.

Is Claude 3.5 Sonnet open source?

No, Claude 3.5 Sonnet is a proprietary model. It is available through Anthropic's API and compatible providers.

Back to AI Models