Best LLMs for x_post_selection autogenerated — DTP Benchmark
Models
Frontier on this task: GPT-5.5 at 8.47 / 10. Quality bar at 95%: 8.05.
point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; cheapest qualifier first. Greyed rows are MEDIUM+ models whose point estimate clears the bar but whose CI low does not.
Cost breakdown
| Model | Quality | Sample | Blended cost / call | Savings vs best | Mode |
|---|---|---|---|---|---|
| Qwen 3.5 Flash Alibaba Cloud (DashScope) | 8.38 / 10 CI [8.19, 8.57] | n=100 · ranked | $0.000103 | 99% cheaper | sync |
| DeepSeek V4 Flash DeepSeek | 8.43 / 10 CI [8.30, 8.55] | n=100 · ranked | $0.000430 | 97% cheaper | sync |
| Gemini 3.1 Flash Lite Gemini | 8.22 / 10 CI [8.00, 8.43] | n=100 · high | $0.000820 | 95% cheaper | batch |
| Qwen 3.6 Plus Alibaba Cloud (DashScope) | 8.37 / 10 CI [8.22, 8.51] | n=100 · ranked | $0.001066 | 94% cheaper | sync |
| Gemini 3 Flash Preview Gemini | 8.32 / 10 CI [8.13, 8.51] | n=100 · ranked | $0.001640 | 90% cheaper | batch |
| Kimi K2.6 Moonshot AI | 8.18 / 10 CI [8.03, 8.33] | n=100 · ranked | $0.003028 | 82% cheaper | batch |
| Gemini 3.1 Pro Preview Gemini | 8.31 / 10 CI [8.12, 8.50] | n=100 · ranked | $0.006560 | 60% cheaper | batch |
| Claude Sonnet 4.6 Anthropic | 8.41 / 10 CI [8.24, 8.59] | n=100 · ranked | $0.009684 | 41% cheaper | batch |
| Claude Opus 4.7 Anthropic | 8.31 / 10 CI [8.10, 8.53] | n=100 · high | $0.016140 | 2% cheaper | batch |
| GPT-5.5 best OpenAI | 8.47 / 10 CI [8.34, 8.61] | n=99 · ranked | $0.016400 | (anchor) | batch |
Typical call shape for this task: 2968 input tokens → 52 output tokens, EMA-tracked from production traffic. Blended cost = (in × in_price + out × out_price), rounded to 6 decimals.