The Right-Sized LLM Benchmark Cheapest LLM that's good enough.

Best LLMs for Claim Refinement

Reviews extracted claims and either REFINES (adds missing subject, expands abbreviations, resolves ambiguous references) or DROPS (methodology mentions, promotional content, claims about the source itself rather than the financial subject).

Cost mode:

Category: Infrastructure & Utility · Rail: absolute · Typical I/O: 2982→1508 tokens

Models

Frontier on this task: Tencent Hy3 at 8.47 / 10. Quality bar at 90%: 7.63.

point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.

Cost breakdown

ModelQualityConfidenceCost / 1k runsOverpayMode
Gemini 3.1 Flash Lite ★ Gemini8.47 / 10 CI [8.32, 8.63]RANKED$0.56best valuebatch
MiniMax M3 MiniMax7.67 / 10 CI [7.32, 8.02]MEDIUM$1.392.5xbatch
Tencent Hy3 best OpenRouter8.47 / 10 CI [8.24, 8.71]HIGH$1.803.2xbatch
Qwen 3.5 Flash Alibaba Cloud (DashScope)7.80 / 10 CI [7.57, 8.02]HIGH$2.885.1xbatch
GPT-5.6 Luna OpenAI7.90 / 10 CI [7.40, 8.40]MEDIUM$2.905.2xbatch
Qwen 3.7 Plus Alibaba Cloud (DashScope)8.30 / 10 CI [8.12, 8.49]RANKED$4.427.9xbatch
Gemini 3.5 Flash Gemini8.31 / 10 CI [8.03, 8.59]HIGH$7.4613xbatch
Qwen 3.6 Flash Alibaba Cloud (DashScope)8.10 / 10 CI [7.73, 8.47]MEDIUM$7.7414xbatch
Claude Sonnet 5 Anthropic8.35 / 10 CI [8.05, 8.65]HIGH$8.2615xbatch
Qwen 3.6 Plus Alibaba Cloud (DashScope)8.14 / 10 CI [7.91, 8.37]HIGH$9.1116xbatch
Claude Opus 4.8 Anthropic7.99 / 10 CI [7.69, 8.30]MEDIUM$19.5335xbatch

Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 2982 input tokens → 1508 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.

Prompt templates

This is a pooled capability — 2 prompt families share it. The pair shown first is the most frequently used in production.

CLAIM_REFINEMENT_SYSTEM_PROMPT + CLAIM_REFINEMENT_USER_PROMPT (128181 calls in window)

System prompt

You are a quality reviewer for extracted factual claims. You receive a list of claims that were extracted from a research summary, along with the original summary text.

Your job is to review each claim and either REFINE it or DROP it.

## REFINE a claim when:
- It is a valid factual claim but lacks context — add the missing subject, entity name, or qualifier from the original summary so the claim is self-contained
- It uses abbreviations or ticker symbols without full names — expand them (e.g. "MU" → "Micron Technology (MU)")
- It references "the post", "the author", "the article" — rewrite as a standalone fact
- It has ambiguous references ("the fund", "this strategy", "the product") — resolve from the summary

## DROP a claim when:
- It reports absence of information ("no ESG data is provided", "no macro drivers are stated")
- It describes methodology, tools, platforms, simulation setup, or backtest configuration rather than the financial subject
- It is promotional or marketing content (trials, subscriptions, community invitations)
- It is a mere mention, association, or classification without substantive content
- It is a meta-description of the source (its topic, audience, format, tone, or purpose)
- It is a user comment, opinion, or community reaction rather than a factual data point
- It cannot be made self-contained because the summary lacks sufficient context — a vague claim has no value
- It is a vague procedural statement with no concrete information

## Output rules:
- Return ONLY the refined claims that pass quality review
- Each claim must be understandable on its own without the original summary
- Each claim must convey actionable information about the financial subject being analyzed
- Preserve the original category and source_reference — only modify claim_text
- Preserve numerical precision exactly as stated

## Required Output Format
Your response MUST be a single, valid JSON object conforming to this schema:
```json
{schema_json_string}
```

User prompt

## Original Summary

{summary_text}

## Report Chapter Context

{chapter_descriptions}

## Extracted Claims to Review

{claims_list}

## Instructions

Review each claim above against the original summary. For each claim:
1. If it is a valid factual claim about the financial subject — refine it to be self-contained and clear, then include it
2. If it is NOT a valid claim (methodology, meta-description, vague, promotional, mere mention, missing context that cannot be resolved) — drop it

Return only the claims that pass review, with refined claim_text where needed. Keep the original category and source_reference.

## Output Format

Return ONLY the fields defined in the schema below. Do not add extra fields.

The required JSON output schema is provided in the system prompt.
JSON_REPAIR_SYSTEM + JSON_REPAIR_USER (29 calls in window)

System prompt

You are a JSON repair tool. The user gives you malformed or partial model output and a JSON Schema. Return ONLY a single valid JSON object that satisfies the schema, salvaging as much real content from the input as possible. Do not invent data for fields the input doesn't support — use the schema's allowed empty/null values. Output the JSON object only: no prose, no markdown, no code fences.

User prompt

JSON Schema:
{schema_json}

Malformed output to repair:
{raw_text}

Return only the corrected JSON object.