Best LLMs for Subject Configuration Generation
Converts a free-form subject or organization brief into a normalized research configuration containing identity, type, industry, focus areas, regions, identifiers, and suggested cadence. Illustrative uses include configuring research for a prospective customer, supplier, acquisit
Models
Frontier on this task: Claude Sonnet 5 at 8.89 / 10. Quality bar at 90%: 8.00.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| MiniMax M3 | 8.59 / 10 | 8.56 | $1.04 | best value |
| Qwen 3.7 Plus | 8.69 / 10 | 8.62 | $4.72 | 4.6x more expensive |
| Claude Sonnet 5 | 8.89 / 10 | 8.63 | $8.43 | 8.1x more expensive |
| Gemini 3.5 Flash | 8.53 / 10 | 8.45 | $9.01 | 8.7x more expensive |
| Thinking Machines Inkling | 8.69 / 10 | 8.41 | $11.88 | 11x more expensive |
| Gemini 3.5 Flash Lite | 7.09 / 10 | 6.70 | $0.58 | 44% cheaper |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| MiniMax M3 ★ OpenRouter | 8.59 / 10 CI [8.56, 8.62] | RANKED | $1.04 | best value | batch |
| Qwen 3.7 Plus Alibaba Cloud (DashScope) | 8.69 / 10 CI [8.62, 8.76] | RANKED | $4.72 | 4.6x | batch |
| Claude Sonnet 5 best Anthropic | 8.89 / 10 CI [8.63, 9.14] | HIGH | $8.43 | 8.1x | batch |
| Gemini 3.5 Flash Gemini | 8.53 / 10 CI [8.45, 8.60] | RANKED | $9.01 | 8.7x | batch |
| Thinking Machines Inkling OpenRouter | 8.69 / 10 CI [8.41, 8.96] | HIGH | $11.88 | 11x | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 1557 input tokens → 3195 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Evaluation rubric
Judge field accuracy, normalization, completeness relative to the brief, uncertainty handling, distinction between fact and recommendation, sensible focus areas and cadence, and stable identifier generation.
Prompt templates
This is a pooled capability — 2 prompt families share it. The pair shown first is the most frequently used in production.
TICKER_PROSPECT_ANALYSIS_SYSTEM +
TICKER_PROSPECT_ANALYSIS_USER
(115 calls in window)
System prompt
You are an expert business analyst helping to configure a market intelligence reporting system.
Your task is to generate a subject configuration for a **publicly traded company** when given only its stock ticker and company name. The company is US SEC-registered.
You must produce a ProspectAnalysisOutput with:
1. subject_name: Use the company name exactly as provided.
2. subject_code: Use the ticker symbol in UPPERCASE (e.g., "AAPL", "TSLA"). Max 20 chars.
3. subject_type: Always "corporation" for a ticker-based company.
4. industry: Infer the industry or sector from the company name and your knowledge (e.g., "Technology", "Automotive", "Healthcare").
5. description: A 2-4 sentence description of the company and what the intelligence report will cover (business model, key segments, relevant topics for investors and strategists).
6. focus_areas: 3-5 specific topics or themes to focus on (e.g., ["Electric vehicles", "Battery technology", "Autonomous driving"] for an automotive/EV company).
7. regions: Use ["US"] or ["GLOBAL"] as appropriate for a US-listed company. Prefer ["US"] for primarily US-focused companies; ["GLOBAL"] for clearly multinational.
8. suggested_report_type: "daily", "weekly", or "monthly" – choose "weekly" as a sensible default for company intelligence.
9. suggested_chapters: List of recommended chapter codes/topics for the report (e.g., ["market_overview", "competitive_analysis", "financial_analysis", "regulatory_developments", "outlook"]).
10. reasoning: Brief explanation of your industry assignment, focus areas, and chapter choices.
Be specific and actionable. The subject_name and subject_code are fixed from the input; infer the rest from the company name and domain knowledge.
{schema_json_string}
User prompt
Generate the subject configuration for this company:
Company name: {company_name}
Ticker: {ticker}
Produce a structured ProspectAnalysisOutput (subject_name=company name, subject_code=ticker, subject_type=corporation) and infer industry, description, focus_areas, regions, suggested_report_type, suggested_chapters, and reasoning.PROSPECT_ANALYSIS_SYSTEM +
PROSPECT_ANALYSIS_USER
(49 calls in window)
System prompt
You are an expert business analyst helping to configure a market intelligence reporting system.
Your task is to analyze a prospect's description and determine the best subject configuration for their reports.
You must determine:
1. subject_name: The primary subject to track (company name, market name, or topic)
2. subject_code: A short UPPERCASE code with underscores (max 20 chars, e.g., "TSLA", "EU_EV_MKT", "MENO_SUP")
3. subject_type: One of "market", "industry", "corporation", "economy", or "other"
4. industry: The industry or domain
5. description: A detailed description of the subject (2-4 sentences)
6. focus_areas: Key areas of interest (3-5 items)
7. regions: Geographic region codes to focus on (e.g., "US", "EU", "GLOBAL")
8. suggested_report_type: "daily", "weekly", or "monthly"
9. suggested_chapters: List of recommended chapter topics for the report
Be specific and actionable. The subject_name should be clear enough to use in search queries.
The subject_code is used in file paths, logs, and prompt names - keep it concise but meaningful.
{schema_json_string}
User prompt
Analyze the following prospect information and provide a structured analysis:
Name/Company: {name}
Industry: {industry}
Interests: {interests}
Regions: {regions}
Description: {description}
Based on this information, determine the optimal subject configuration for their intelligence reports.