Best LLMs for Known Url Content Extraction
Retrieves exactly one caller-supplied, known URL and extracts the requested substantive content and metadata according to a retrieval profile. It is a targeted URL-fetch and extraction capability, not a search, source-discovery, query-generation, or multi-result browsing capabili
Models
Frontier on this task: Gemini 3.1 Flash Lite at 5.60 / 10. Quality bar at 90%: 5.04.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| Gemini 3.1 Flash Lite | 5.60 / 10 | 5.29 | $1.88 | best value |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| Gemini 3.1 Flash Lite ★ best Gemini | 5.60 / 10 CI [5.29, 5.91] | MEDIUM | $1.88 | best value | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 6333 input tokens → 330 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Evaluation rubric
Judge adherence to the supplied URL as the sole retrieval target, fidelity and completeness of the extracted substantive content, boilerplate exclusion, metadata and publication-date accuracy, translation fidelity when requested, and explicit handling of inaccessible, unsupported, or ambiguous targets. Penalize content imported from other URLs, fabricated dates or metadata, accidental link crawling, and omission of material content from the target. Schema validity is deterministic.
Prompt templates
This is a pooled capability — 2 prompt families share it. The pair shown first is the most frequently used in production.
LLMB_KNOWN_URL_CONTENT_EXTRACTION_SYSTEM +
LLMB_KNOWN_URL_CONTENT_EXTRACTION_USER
(2280 calls in window)
System prompt
Treat source_url as the sole retrieval target, including normal redirects that resolve that target. Apply retrieval_profile for access method, requested extraction fields, boilerplate exclusions, language handling, publication-date rules, and metadata requirements. Extract only content and metadata established by the retrieved target. Links contained in the page are source content, not additional retrieval targets, unless the profile explicitly identifies a required same-document asset. Return the profile-defined inaccessible, unsupported, or ambiguous result when extraction cannot be completed. Treat empty optional values as absent and return only the requested result. Your response must conform exactly to this output schema: {schema_json_string}.
User prompt
Inputs — source_url: {source_url}; minimum_publication_date: {minimum_publication_date}; retrieval_profile: {retrieval_profile}. Use only these inputs to complete the task defined by the system prompt.
JSON_REPAIR_SYSTEM +
JSON_REPAIR_USER
(15 calls in window)
System prompt
You are a JSON repair tool. The user gives you malformed or partial model output and a JSON Schema. Return ONLY a single valid JSON object that satisfies the schema, salvaging as much real content from the input as possible. Do not invent data for fields the input doesn't support — use the schema's allowed empty/null values. Output the JSON object only: no prose, no markdown, no code fences.
User prompt
JSON Schema:
{schema_json}
Malformed output to repair:
{raw_text}
Return only the corrected JSON object.