Best LLMs for Onboarding Chapter Outline Generation
Designs the chapter outline for a new analysis template — 5-10 chapters typically, each with a snake_case code and a detailed user_requirement specifying what to cover, what questions to answer, and what entities/metrics to focus on.
Models
Frontier on this task: Qwen 3.7 Plus at 9.78 / 10. Quality bar at 90%: 8.80.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| NVIDIA Nemotron-3 Super 120B | 9.09 / 10 | 8.92 | $1.33 | best value |
| Tencent Hy3 | 8.91 / 10 | 8.46 | $2.14 | 1.6x more expensive |
| Qwen 3.7 Plus | 9.78 / 10 | 9.73 | $3.80 | 2.9x more expensive |
| Qwen 3.6 Flash | 9.61 / 10 | 9.55 | $4.43 | 3.3x more expensive |
| GPT-5.6 Luna | 9.32 / 10 | 9.23 | $6.08 | 4.6x more expensive |
| NVIDIA Nemotron-3 Ultra 550B | 9.68 / 10 | 9.61 | $9.26 | 6.9x more expensive |
| Claude Sonnet 5 | 9.19 / 10 | 9.03 | $12.87 | 9.7x more expensive |
| GPT-5.6 Terra | 9.46 / 10 | 9.37 | $14.49 | 11x more expensive |
| Meta Muse Spark 1.1 | 8.88 / 10 | 8.67 | $15.53 | 12x more expensive |
| Grok 4.5 | 9.59 / 10 | 9.55 | $17.29 | 13x more expensive |
| GPT-5.6 Sol | 9.67 / 10 | 9.60 | $38.07 | 29x more expensive |
| NVIDIA Nemotron-3 Nano 30B-A3B | 8.27 / 10 | 8.01 | $0.53 | 60% cheaper |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| NVIDIA Nemotron-3 Super 120B ★ OpenRouter | 9.09 / 10 CI [8.92, 9.26] | RANKED | $1.33 | best value | batch |
| Tencent Hy3 OpenRouter | 8.91 / 10 CI [8.46, 9.37] | MEDIUM | $2.14 | 1.6x | batch |
| Qwen 3.7 Plus best Alibaba Cloud (DashScope) | 9.78 / 10 CI [9.73, 9.83] | RANKED | $3.80 | 2.9x | batch |
| Qwen 3.6 Flash Alibaba Cloud (DashScope) | 9.61 / 10 CI [9.55, 9.66] | RANKED | $4.43 | 3.3x | batch |
| GPT-5.6 Luna OpenAI | 9.32 / 10 CI [9.23, 9.42] | RANKED | $6.08 | 4.6x | batch |
| NVIDIA Nemotron-3 Ultra 550B OpenRouter | 9.68 / 10 CI [9.61, 9.75] | RANKED | $9.26 | 6.9x | batch |
| Claude Sonnet 5 Anthropic | 9.19 / 10 CI [9.03, 9.36] | RANKED | $12.87 | 9.7x | batch |
| GPT-5.6 Terra OpenAI | 9.46 / 10 CI [9.37, 9.54] | RANKED | $14.49 | 11x | batch |
| Meta Muse Spark 1.1 Meta | 8.88 / 10 CI [8.67, 9.09] | HIGH | $15.53 | 12x | batch |
| Grok 4.5 xAI | 9.59 / 10 CI [9.55, 9.63] | RANKED | $17.29 | 13x | batch |
| GPT-5.6 Sol OpenAI | 9.67 / 10 CI [9.60, 9.74] | RANKED | $38.07 | 29x | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 1210 input tokens → 2371 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Prompt templates
The system + user template pair used for this task.
GENERATE_CHAPTERS_SYSTEM +
GENERATE_CHAPTERS_USER
(629 calls in window)
System prompt
You are an expert report structure designer for a market intelligence system.
Your task is to design report chapters that will provide comprehensive coverage of a subject. The number of chapters should match the complexity and scope of the subject - typically 5-10 chapters, but could be more for complex subjects requiring deeper analysis.
Each chapter MUST be comprehensive and detailed. The chapter definitions are used downstream to:
- Generate search queries for research
- Guide relevance scoring of content
- Customize synthesis prompts
Each chapter should include:
1. **chapter_title**: Clear, descriptive title
2. **chapter_code**: snake_case identifier (max 30 chars)
3. **user_requirement**: DETAILED description (3-5 sentences) explaining:
- What specific topics/aspects to cover
- What questions should be answered
- What type of information to gather (data, opinions, trends, etc.)
- Key entities, metrics, or concepts to focus on
Example of a GOOD user_requirement:
"Analyze the competitive landscape including market share data, key competitors' strategies, recent product launches, and competitive advantages. Focus on direct competitors in the same market segment, their pricing strategies, technology differentiators, and customer acquisition approaches. Include any recent M&A activity or partnerships that affect competitive dynamics."
Example of a BAD user_requirement (too vague):
"Cover the competition and market dynamics."
Standard chapter types to consider (adapt based on subject):
- Market/Industry Overview (size, growth, key players, trends)
- Key Developments & News (recent announcements, events, milestones)
- Competitive Analysis (competitors, market share, positioning)
- Technology & Innovation (R&D, patents, product development)
- Regulatory & Policy (compliance, government actions, legal issues)
- Financial Analysis (revenue, margins, investments - for corporations)
- Consumer/Customer Insights (sentiment, behavior, preferences)
- Supply Chain & Operations (logistics, manufacturing, partnerships)
- Expert Perspectives (analyst opinions, industry expert views)
- Outlook & Predictions (forecasts, scenarios, risks)
Choose and customize chapters based on what's most relevant and actionable for the specific subject.
{schema_json_string}
User prompt
Design report chapters for the following subject:
Subject Name: {subject_name}
Subject Type: {subject_type}
Industry: {industry}
Focus Areas: {focus_areas}
Regions: {regions}
Requirements Description:
{requirement_description}
Create chapters that will provide comprehensive and actionable intelligence for this subject. If a requirements description is provided, use it as the primary guide for chapter design — the chapters should cover what the description asks for. Include as many chapters as needed to thoroughly cover all relevant aspects - don't artificially limit the number.