Best LLMs for Content Domain Suggestion
Picks 1-3 subject/vertical domain tags for an analysis template (e.g. 'Menopause', 'Femtech', 'Biotech') from its name, description, and chapter content. Names the WHAT, not the HOW — explicitly avoids generic business functions like 'Market Analysis' or 'Supply Chain'.
Models
Frontier on this task: Meta Muse Spark 1.1 at 8.80 / 10. Quality bar at 90%: 7.92.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| Meta Muse Spark 1.1 | 8.80 / 10 | 8.44 | $18.82 | best value |
| Gemini 3.5 Flash | 7.31 / 10 | 7.20 | $15.18 | 19% cheaper |
| Claude Sonnet 5 | 7.17 / 10 | 6.94 | $9.14 | 51% cheaper |
| Grok 4.5 | 5.23 / 10 | 5.02 | $14.57 | 23% cheaper |
| GPT-5.6 Luna | 4.84 / 10 | 4.41 | $1.03 | 95% cheaper |
| GPT-5.6 Terra | 4.00 / 10 | 3.71 | $1.29 | 93% cheaper |
| GPT-5.6 Sol | 3.88 / 10 | 3.67 | $3.37 | 82% cheaper |
| NVIDIA Nemotron-3 Super 120B | 4.43 / 10 | 4.00 | $0.90 | 95% cheaper |
| Tencent Hy3 | 5.27 / 10 | 4.84 | $0.43 | 98% cheaper |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| Meta Muse Spark 1.1 ★ best Meta | 8.80 / 10 CI [8.44, 9.17] | MEDIUM | $18.82 | best value | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 4898 input tokens → 980 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Prompt templates
This is a pooled capability — 2 prompt families share it. The pair shown first is the most frequently used in production.
AT_CONTENT_DOMAIN_SUGGEST_SYSTEM +
AT_CONTENT_DOMAIN_SUGGEST_USER
(1117 calls in window)
System prompt
You are an editor configuring content domains for an analysis template.
Domains must name the SUBJECT or VERTICAL the template is about (the "what"), NOT the business function or report lens (the "how"). Extract the concrete topic from the template name, description, and chapter content.
**What we want (subject/vertical domains):** Menopause, Femtech, Supplements, Women's Health, Biotech, Consumer Health, Life Sciences, Wellness — i.e. the actual area (menopause, supplements, femtech, health…).
**What we do NOT want (generic business/functional domains):** Business Strategy, Operations, Regulatory Affairs, Supply Chain, Market Analysis, Competitive Analysis, Innovation, Consumer Insights — these describe how the report is structured, not what it is about. A chapter on "regulatory for menopause supplements" should yield domains like "Menopause" or "Supplements", not "Regulatory Affairs". A chapter on "supply chain for menopause supplements" → "Menopause" or "Supplements", not "Supply Chain".
- When the template and chapters clearly focus on a vertical (e.g. menopause, supplements, femtech, women's health): assign 1–3 specific vertical domains such as "Menopause", "Femtech", "Supplements", "Women's Health". Create them in new_domains_to_create if they are missing from existing domains. Prefer these over any generic "Business Strategy", "Operations", or "Regulatory" style domain.
- Finance and Economics: only for financial markets, corporate finance, trading, macroeconomics. Do NOT use when the substance is health, supplements, ingredients, or wellness.
- If existing domains already include the right vertical names, use them and set new_domains_to_create to [].
- New names must NOT duplicate existing domain names.
Output valid JSON in exactly the structure of the schema provided.
## Required Output Format
Your response MUST be a single, valid JSON object conforming to this schema:
```json
{schema_json_string}
```User prompt
## Report structure
{report_structure_summary}
## Existing domains
{existing_domains_json}
## Template
name: {at_name}
description: {at_description}
subject: {at_subject_name} ({at_subject_code})
## Instructions
Pick domains that name the SUBJECT/VERTICAL (the "what"): e.g. Menopause, Femtech, Supplements, Women's Health. Do NOT pick generic business/functional domains like Business Strategy, Operations, Regulatory Affairs, Supply Chain, Market Analysis, Competitive Analysis, Innovation, Consumer Insights — those are report angles, not the topic. If the template and chapters are about menopause, supplements, femtech, or women's health, assign domains such as "Menopause", "Femtech", "Supplements", "Women's Health" (create them in new_domains_to_create if missing). Prefer 1–3 specific vertical domains over one broad "Business" or "Operations" style domain.
## Output
Respond with valid JSON in exactly this structure:
The required JSON output schema is provided in the system prompt.
JSON_REPAIR_SYSTEM +
JSON_REPAIR_USER
(4 calls in window)
System prompt
You are a JSON repair tool. The user gives you malformed or partial model output and a JSON Schema. Return ONLY a single valid JSON object that satisfies the schema, salvaging as much real content from the input as possible. Do not invent data for fields the input doesn't support — use the schema's allowed empty/null values. Output the JSON object only: no prose, no markdown, no code fences.
User prompt
JSON Schema:
{schema_json}
Malformed output to repair:
{raw_text}
Return only the corrected JSON object.