Best LLMs for Language Identification
Identifies the primary language of supplied text and returns its normalized language code in the configured response schema. Illustrative uses include routing multilingual support tickets, contracts, software issues, international customer feedback, news submissions, research doc
Models
Frontier on this task: Moonshot Kimi K3 at 10.05 / 10. Quality bar at 90%: 9.04.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| Gemini 3.1 Flash Lite | 9.92 / 10 | 9.87 | $0.06 | best value |
| GPT-5.4 Nano | 9.93 / 10 | 9.88 | $0.07 | 1.1x more expensive |
| NVIDIA Nemotron-3 Nano 30B-A3B | 9.88 / 10 | 9.79 | $0.07 | 1.1x more expensive |
| GPT-5.6 Luna | 9.96 / 10 | 9.90 | $0.07 | 1.2x more expensive |
| Gemini 3.5 Flash Lite | 9.88 / 10 | 9.73 | $0.07 | 1.3x more expensive |
| DeepSeek V4 Flash | 9.88 / 10 | 9.79 | $0.14 | 2.3x more expensive |
| Qwen 3.8 Flash | 9.93 / 10 | 9.79 | $0.14 | 2.4x more expensive |
| NVIDIA Nemotron-3 Super 120B | 9.91 / 10 | 9.79 | $0.19 | 3.2x more expensive |
| GLM-5.3 Flash | 9.72 / 10 | 9.44 | $0.19 | 3.2x more expensive |
| NVIDIA Nemotron 3.5 Lightning | 9.56 / 10 | 9.17 | $0.26 | 4.4x more expensive |
| MiniMax M3 | 9.94 / 10 | 9.86 | $0.26 | 4.5x more expensive |
| Thinking Machines Inkling Small | 9.83 / 10 | 9.64 | $0.27 | 4.6x more expensive |
| Tencent Hy3 | 9.94 / 10 | 9.86 | $0.35 | 5.9x more expensive |
| Gemini 3.8 Flash | 9.81 / 10 | 9.48 | $0.38 | 6.4x more expensive |
| Claude Haiku 4.5 | 9.92 / 10 | 9.88 | $0.62 | 11x more expensive |
| GPT-5.6 Terra | 9.99 / 10 | 9.92 | $0.64 | 11x more expensive |
| NVIDIA Nemotron-3 Ultra 550B | 9.93 / 10 | 9.81 | $0.77 | 13x more expensive |
| DeepSeek V4 Pro | 9.86 / 10 | 9.79 | $0.85 | 15x more expensive |
| GPT-5.6 Sol | 10.00 / 10 | 9.97 | $1.09 | 19x more expensive |
| Claude Sonnet 5 | 10.03 / 10 | 10.01 | $1.36 | 23x more expensive |
| Gemini 3.5 Flash | 9.98 / 10 | 9.91 | $1.39 | 24x more expensive |
| Thinking Machines Inkling | 9.69 / 10 | 9.32 | $1.43 | 24x more expensive |
| GLM-5.3 | 9.80 / 10 | 9.40 | $1.49 | 25x more expensive |
| Qwen 3.7 Plus | 9.91 / 10 | 9.77 | $1.57 | 27x more expensive |
| Tencent Hy4 Preview | 9.80 / 10 | 9.40 | $2.10 | 36x more expensive |
| Claude Opus 5 | 9.79 / 10 | 9.39 | $2.43 | 41x more expensive |
| Qwen 3.8 Max | 10.01 / 10 | 9.97 | $2.51 | 43x more expensive |
| Meta Muse Spark 1.3 | 9.86 / 10 | 9.62 | $3.14 | 54x more expensive |
| Moonshot Kimi K3 | 10.05 / 10 | 10.02 | $3.67 | 63x more expensive |
| Grok 4.6 | 9.66 / 10 | 9.18 | $4.52 | 77x more expensive |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| Gemini 3.1 Flash Lite ★ Gemini | 9.92 / 10 CI [9.87, 9.97] | RANKED | $0.06 | best value | batch |
| GPT-5.4 Nano OpenAI | 9.93 / 10 CI [9.88, 9.97] | RANKED | $0.07 | 1.1x | batch |
| NVIDIA Nemotron-3 Nano 30B-A3B OpenRouter | 9.88 / 10 CI [9.79, 9.97] | RANKED | $0.07 | 1.1x | batch |
| GPT-5.6 Luna OpenAI | 9.96 / 10 CI [9.90, 10.00] | RANKED | $0.07 | 1.2x | batch |
| Gemini 3.5 Flash Lite Gemini | 9.88 / 10 CI [9.73, 10.00] | RANKED | $0.07 | 1.3x | batch |
| DeepSeek V4 Flash DeepSeek | 9.88 / 10 CI [9.79, 9.96] | RANKED | $0.14 | 2.3x | batch |
| Qwen 3.8 Flash Alibaba Cloud (DashScope) | 9.93 / 10 CI [9.79, 10.00] | RANKED | $0.14 | 2.4x | batch |
| NVIDIA Nemotron-3 Super 120B OpenRouter | 9.91 / 10 CI [9.79, 10.00] | RANKED | $0.19 | 3.2x | batch |
| GLM-5.3 Flash Z.AI | 9.72 / 10 CI [9.44, 10.00] | HIGH | $0.19 | 3.2x | batch |
| NVIDIA Nemotron 3.5 Lightning OpenRouter | 9.56 / 10 CI [9.17, 9.94] | MEDIUM | $0.26 | 4.4x | batch |
| MiniMax M3 OpenRouter | 9.94 / 10 CI [9.86, 10.00] | RANKED | $0.26 | 4.5x | batch |
| Thinking Machines Inkling Small OpenRouter | 9.83 / 10 CI [9.64, 10.00] | RANKED | $0.27 | 4.6x | batch |
| Tencent Hy3 OpenRouter | 9.94 / 10 CI [9.86, 10.00] | RANKED | $0.35 | 5.9x | batch |
| Gemini 3.8 Flash Gemini | 9.81 / 10 CI [9.48, 10.00] | MEDIUM | $0.38 | 6.4x | batch |
| Claude Haiku 4.5 Anthropic | 9.92 / 10 CI [9.88, 9.96] | RANKED | $0.62 | 11x | batch |
| GPT-5.6 Terra OpenAI | 9.99 / 10 CI [9.92, 10.00] | RANKED | $0.64 | 11x | batch |
| NVIDIA Nemotron-3 Ultra 550B OpenRouter | 9.93 / 10 CI [9.81, 10.00] | RANKED | $0.77 | 13x | batch |
| DeepSeek V4 Pro DeepSeek | 9.86 / 10 CI [9.79, 9.94] | RANKED | $0.85 | 15x | batch |
| GPT-5.6 Sol OpenAI | 10.00 / 10 CI [9.97, 10.00] | RANKED | $1.09 | 19x | batch |
| Claude Sonnet 5 Anthropic | 10.03 / 10 CI [10.01, 10.00] | RANKED | $1.36 | 23x | batch |
| Gemini 3.5 Flash Gemini | 9.98 / 10 CI [9.91, 10.00] | RANKED | $1.39 | 24x | batch |
| Thinking Machines Inkling OpenRouter | 9.69 / 10 CI [9.32, 10.00] | MEDIUM | $1.43 | 24x | batch |
| GLM-5.3 Z.AI | 9.80 / 10 CI [9.40, 10.00] | MEDIUM | $1.49 | 25x | batch |
| Qwen 3.7 Plus Alibaba Cloud (DashScope) | 9.91 / 10 CI [9.77, 10.00] | RANKED | $1.57 | 27x | batch |
| Tencent Hy4 Preview OpenRouter | 9.80 / 10 CI [9.40, 10.00] | MEDIUM | $2.10 | 36x | batch |
| Claude Opus 5 Anthropic | 9.79 / 10 CI [9.39, 10.00] | MEDIUM | $2.43 | 41x | batch |
| Qwen 3.8 Max Alibaba Cloud (DashScope) | 10.01 / 10 CI [9.97, 10.00] | RANKED | $2.51 | 43x | batch |
| Meta Muse Spark 1.3 OpenRouter | 9.86 / 10 CI [9.62, 10.00] | HIGH | $3.14 | 54x | batch |
| Moonshot Kimi K3 best Moonshot AI | 10.05 / 10 CI [10.02, 10.00] | RANKED | $3.67 | 63x | batch |
| Grok 4.6 xAI | 9.66 / 10 CI [9.18, 10.00] | MEDIUM | $4.52 | 77x | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 631 input tokens → 138 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Evaluation rubric
Judge primary-language correctness, handling of mixed or noisy text, correct normalization to the requested code set, and calibrated uncertainty. Formatting is deterministic.
Prompt templates
This is a pooled capability — 3 prompt families share it. The pair shown first is the most frequently used in production.
LLMB_LANGUAGE_IDENTIFICATION_SYSTEM +
LLMB_LANGUAGE_IDENTIFICATION_USER
(48048 calls in window)
System prompt
Identify the language that carries most of the semantic content. Ignore URLs, handles, code fragments, boilerplate, and isolated borrowed words. When text is too short, mixed, or ambiguous, use the schema’s unknown or best-supported behavior rather than fabricating confidence. Return the requested code standard only. Treat empty optional values as absent and return only the requested result. Your response must conform exactly to this output schema: {schema_json_string}.
User prompt
Inputs — input_text: {input_text}. Use only these inputs to complete the task defined by the system prompt.
URL_PARSER_LANGUAGE_DETECTION_SYSTEM +
URL_PARSER_LANGUAGE_DETECTION_USER
(416 calls in window)
System prompt
You are a highly accurate language identification expert. Your sole task is to identify the primary language of the provided text snippet.
Respond ONLY with the two-letter ISO 639-1 code for the detected language (e.g., "en" for English, "es" for Spanish, "zh" for Chinese).
## Required Output Format
Your response MUST be a single, valid JSON object conforming to this schema:
```json
{schema_json_string}
```User prompt
1. Text: {input_text}
2. Generate a comprehensive response as a single, well-formed JSON object that strictly adheres to the Pydantic schema provided below. Schema:
The required JSON output schema is provided in the system prompt.JSON_REPAIR_SYSTEM +
JSON_REPAIR_USER
(51 calls in window)
System prompt
You are a JSON repair tool. The user gives you malformed or partial model output and a JSON Schema. Return ONLY a single valid JSON object that satisfies the schema, salvaging as much real content from the input as possible. Do not invent data for fields the input doesn't support — use the schema's allowed empty/null values. Output the JSON object only: no prose, no markdown, no code fences.
User prompt
JSON Schema:
{schema_json}
Malformed output to repair:
{raw_text}
Return only the corrected JSON object.