Best LLMs for Vetted News Site Selection
Identifies authoritative, accessible web sources for researching a subject within specified regions. Prioritises official, recognised, regularly maintained sources; filters out strict-paywall and login-required sites.
Models
Frontier on this task: GPT-5.6 Sol at 8.45 / 10. Quality bar at 90%: 7.61.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| MiniMax M3 | 7.64 / 10 | 7.50 | $4.81 | best value |
| DeepSeek V4 Pro | 7.74 / 10 | 7.31 | $6.48 | 1.3x more expensive |
| Qwen 3.6 Plus | 8.11 / 10 | 7.83 | $13.13 | 2.7x more expensive |
| Meta Muse Spark 1.1 | 7.72 / 10 | 7.42 | $27.29 | 5.7x more expensive |
| GPT-5.6 Sol | 8.45 / 10 | 8.16 | $55.00 | 11x more expensive |
| GPT-5.5 | 8.41 / 10 | 8.03 | $98.77 | 21x more expensive |
| Gemini 3.1 Pro Preview | 6.40 / 10 | 5.97 | $14.34 | 3x more expensive |
| Claude Haiku 4.5 | 6.11 / 10 | 5.63 | $14.55 | 3x more expensive |
| Qwen 3.5 Flash | 7.53 / 10 | 7.26 | $1.97 | 59% cheaper |
| Grok 4.5 | 6.96 / 10 | 6.85 | $34.98 | 7.3x more expensive |
| NVIDIA Nemotron-3 Nano 30B-A3B | 4.59 / 10 | 4.10 | $1.08 | 78% cheaper |
| GPT-5.6 Terra | 7.54 / 10 | 7.05 | $19.79 | 4.1x more expensive |
| GPT-5.6 Luna | 7.57 / 10 | 7.23 | $7.66 | 1.6x more expensive |
| Tencent Hy3 | 6.56 / 10 | 6.19 | $2.84 | 41% cheaper |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| MiniMax M3 ★ MiniMax | 7.64 / 10 CI [7.50, 7.78] | RANKED | $4.81 | best value | batch |
| DeepSeek V4 Pro DeepSeek | 7.74 / 10 CI [7.31, 8.17] | MEDIUM | $6.48 | 1.3x | batch |
| Qwen 3.6 Plus Alibaba Cloud (DashScope) | 8.11 / 10 CI [7.83, 8.39] | HIGH | $13.13 | 2.7x | batch |
| Meta Muse Spark 1.1 Meta | 7.72 / 10 CI [7.42, 8.01] | HIGH | $27.29 | 5.7x | batch |
| GPT-5.6 Sol best OpenAI | 8.45 / 10 CI [8.16, 8.74] | HIGH | $55.00 | 11x | batch |
| GPT-5.5 OpenAI | 8.41 / 10 CI [8.03, 8.78] | MEDIUM | $98.77 | 21x | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 13282 input tokens → 1854 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Prompt templates
The system + user template pair used for this task.
RESEARCH_VETTED_SITES_SELECTOR_SYSTEM +
RESEARCH_VETTED_SITES_SELECTOR_USER
(1799 calls in window)
System prompt
You are an expert information quality analyst specializing in source evaluation.
Your task is to identify authoritative, accessible web sources for research on a specific subject.
Source Selection Criteria:
1. REGIONAL RELEVANCE: Sources must be relevant to the specified regions (e.g., UK sources for UK research, Brazil sources for Brazil research)
2. AUTHORITATIVE: Official, recognized, or highly reputable sources
3. ACCESSIBLE: NOT behind strict paywalls or login requirements
4. RELEVANT: Directly related to the subject matter
5. CURRENT: Actively maintained and updated
Regional Source Selection:
- For UK: prioritize .uk domains, UK news sites (bbc.co.uk, theguardian.com, ft.com, reuters.com/world/uk)
- For Brazil: prioritize .br domains, Brazilian news sites (globo.com, folha.uol.com.br, estadao.com.br)
- For US: prioritize .gov, US news sites (reuters.com, bloomberg.com, cnbc.com, sec.gov)
- For EU: prioritize EU regulatory sites, pan-European news sources
- For multiple regions: select sources covering ALL specified regions, not just one
Source Categories by Subject Type:
- Financial/Company: SEC filings, investor relations, financial news
- Technology: Tech news sites, product documentation, developer resources
- Healthcare: Medical journals (open access), health organizations, research databases
- Government/Policy: Official government sites, regulatory bodies
- Academic: Open access journals, institutional repositories, research databases
AVOID:
- Paywalled sites: wsj.com, ft.com, nytimes.com (except for specific free sections)
- Sites requiring mandatory login
- Sites with aggressive anti-scraping measures
- Low-quality aggregators or content farms
Output a JSON object with:
{{
"vetted_sites": [
{{
"domain": "sec.gov",
"name": "SEC Edgar",
"category": "Government/Financial",
"reasoning": "Official SEC filings, free access, relevant to US financial research",
"is_paywall_free": true
}}
]
}}
Select 5-15 most relevant sources.
CRITICAL: Ensure ALL selected sources are relevant to the specified regions. Do not mix UK sources with Brazil-only research, or vice versa.User prompt
Identify authoritative, accessible web sources for researching: {subject_name}
Subject Type: {subject_type}
Subject Description: {subject_description}
Research Focus: {research_focus}
{chapter_context}
TARGET REGIONS: {regions}
CRITICAL REQUIREMENT: Select sources that are HIGHLY RELEVANT to the target regions specified above.
- If researching UK: select UK news sites, .uk domains, UK-focused international sources
- If researching Brazil: select Brazilian news sites, .br domains, Brazil-focused sources
- If researching multiple regions: select sources that cover ALL specified regions
Prioritize sources that:
1. Are REGIONALLY RELEVANT to: {regions}
2. Are free to access (no paywall)
3. Don't require login for basic content
4. Are authoritative in their domain
{additional_requirements}
REMINDER: Do not select sources irrelevant to the target regions. For example, if researching UK and Brazil, do not select US-only or China-only sources unless they have comprehensive UK/Brazil coverage.