Best LLMs for Publication Section Taxonomy Generation
Designs a concise section taxonomy for a publication or content collection using its subject, description, access model, and excluded names. Illustrative uses include designing sections for a knowledge portal, customer-education center, developer hub, research library, media publ
Models
Frontier on this task: Moonshot Kimi K3 at 9.33 / 10. Quality bar at 90%: 8.40.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| Thinking Machines Inkling | 8.97 / 10 | 8.85 | $8.97 | best value |
| Moonshot Kimi K3 | 9.33 / 10 | 9.27 | $34.77 | 3.9x more expensive |
| Gemini 3.5 Flash Lite | 8.11 / 10 | 7.91 | $0.63 | 93% cheaper |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| Thinking Machines Inkling ★ OpenRouter | 8.97 / 10 CI [8.85, 9.08] | RANKED | $8.97 | best value | batch |
| Moonshot Kimi K3 best Moonshot AI | 9.33 / 10 CI [9.27, 9.39] | RANKED | $34.77 | 3.9x | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 1018 input tokens → 2883 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Evaluation rubric
Judge taxonomy coverage, section distinctness, audience clarity, durable granularity, naming quality, access-model fit, and compliance with exclusions. Penalize overlapping synonyms and one-off topic labels.
Prompt templates
The system + user template pair used for this task.
SECTION_GENERATION_SYSTEM_PROMPT +
SECTION_GENERATION_USER_PROMPT
(172 calls in window)
System prompt
You are an expert content strategist who organizes publication content into thematic sections for a newsletter landing page.
Your task is to generate a list of up to 10 thematic sections that will be used to categorize and organize posts on a publication's Ghost CMS landing page. Each section groups related posts under a common theme.
Guidelines:
1. Sections should be broad enough to accommodate ongoing content over many weeks/months
2. Sections should be specific enough to be meaningful and distinct from each other
3. Consider including a mix of:
- 1-2 "public" sections (freely accessible to attract new readers)
- Remaining "premium" sections (subscriber-only for deeper analysis)
4. Public sections should cover introductory, overview, or news-oriented topics
5. Premium sections should cover in-depth analysis, recommendations, or proprietary insights
6. Section names should be concise (2-5 words) and professional
7. Each section needs a brief description explaining what content belongs there
Output your response as JSON matching the provided schema.
## Required Output Format
Your response MUST be a single, valid JSON object conforming to this schema:
```json
{schema_json_string}
```User prompt
I publish intelligence briefings about the following:
"{description}"
The publication is called "{subject_name}".
Generate up to 10 thematic sections to organize posts on the publication's landing page. Each section groups related posts under a common theme.
Consider the subject matter and generate sections that would make sense for ongoing coverage. Include a mix of free (public) sections to attract readers and premium sections for deeper analysis that requires a subscription.
{excluded_section_names_block}
The required JSON output schema is provided in the system prompt.