Best LLMs for Research Community Selection
Selects relevant, accessible communities as research sources for a subject, purpose, regions, and platform profile. Illustrative uses include finding developer forums, customer user groups, professional associations, industry communities, journalist networks, scientific societies
Models
Frontier on this task: GPT-5.6 Terra at 8.02 / 10. Quality bar at 90%: 7.22.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
| Model | Quality score | CI low | Cost / 1k runs | vs best value |
|---|---|---|---|---|
| GPT-5.6 Luna | 7.43 / 10 | 7.21 | $0.46 | best value |
| GPT-5.6 Terra | 8.02 / 10 | 7.76 | $5.32 | 12x more expensive |
| Qwen 3.7 Plus | 7.53 / 10 | 7.19 | $5.74 | 13x more expensive |
| Gemini 3.5 Flash | 7.30 / 10 | 7.07 | $8.47 | 18x more expensive |
| Claude Haiku 4.5 | 4.74 / 10 | 4.50 | $3.34 | 7.3x more expensive |
| DeepSeek V4 Flash | 5.31 / 10 | 5.10 | $2.88 | 6.3x more expensive |
| DeepSeek V4 Pro | 5.38 / 10 | 5.14 | $9.61 | 21x more expensive |
| Gemini 3.1 Flash Lite | 5.12 / 10 | 4.81 | $1.12 | 2.4x more expensive |
| GPT-5.4 Nano | 4.61 / 10 | 4.25 | $1.24 | 2.7x more expensive |
| MiniMax M3 | 6.80 / 10 | 6.62 | $1.47 | 3.2x more expensive |
Cost breakdown
| Model | Quality | Confidence | Cost / 1k runs | Overpay | Mode |
|---|---|---|---|---|---|
| GPT-5.6 Luna ★ OpenAI | 7.43 / 10 CI [7.21, 7.65] | HIGH | $0.46 | best value | batch |
| GPT-5.6 Terra best OpenAI | 8.02 / 10 CI [7.76, 8.28] | HIGH | $5.32 | 12x | batch |
| Qwen 3.7 Plus Alibaba Cloud (DashScope) | 7.53 / 10 CI [7.19, 7.87] | MEDIUM | $5.74 | 13x | batch |
| Gemini 3.5 Flash Gemini | 7.30 / 10 CI [7.07, 7.53] | HIGH | $8.47 | 18x | batch |
Overpay shows how much more you pay than the best-value model that clears the quality bar (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit above the bar. Typical call shape for this task: 5209 input tokens → 3723 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.
Evaluation rubric
Judge subject and purpose relevance, community existence, expected research value, coverage/diversity, accessibility, appropriate specificity, and absence of fabricated activity or policy claims.
Prompt templates
The system + user template pair used for this task.
RESEARCH_SUBREDDIT_SELECTOR_SYSTEM +
RESEARCH_SUBREDDIT_SELECTOR_USER
(376 calls in window)
System prompt
You are an expert in Reddit community analysis and social media research.
Your task is to identify relevant, active Reddit subreddits for researching a specific subject.
Subreddit Selection Criteria:
1. RELEVANT: Directly discusses the subject or related topics
2. ACTIVE: Has regular posts and engagement
3. QUALITY: Well-moderated, substantive discussions (not just memes)
4. ACCESSIBLE: Public subreddits (not private or quarantined)
5. SIZE: Balance between large communities (more content) and niche communities (more focused)
Subreddit Categories:
- Investment/Finance: wallstreetbets, investing, stocks, options, personalfinance
- Technology: technology, programming, specific product subreddits
- Company-Specific: teslamotors, apple, microsoft, etc.
- Industry-Specific: biotech, startups, energy, etc.
- General Discussion: news, worldnews, business
Consider:
- Subject type and industry
- Target audience (retail investors, professionals, enthusiasts, general public)
- Research purpose (sentiment analysis, news discovery, technical discussion)
Output a JSON object with:
{{
"subreddits": [
{{
"name": "wallstreetbets",
"display_name": "WallStreetBets",
"category": "Finance/Investing",
"reasoning": "High activity retail investor sentiment",
"estimated_subscribers": "15M+"
}}
]
}}
Select 3-10 most relevant subreddits.User prompt
Identify relevant Reddit subreddits for researching: {subject_name}
Subject Type: {subject_type}
Subject Description: {subject_description}
Research Purpose: {research_purpose}
{chapter_context}
Regions: {regions}
Focus on subreddits that:
1. Discuss {subject_name} directly or related topics
2. Have active communities (regular posts and comments)
3. Provide {research_purpose} insights
4. Are publicly accessible
{additional_requirements}