Models That Get It Right First Time
First-time-right output is workflow-specific
For production use, “good enough” is not simply a high score on answers that a model manages to return. The relevant bar is usable output on the first provider call: the response must arrive, conform to the workflow’s expected structure, and avoid salvage or a second repair call. The current evidence reports answer rates alongside judged quality, but it does not provide clean-output, schema-conformance, repair-call cost, or repair-call latency figures for these comparisons. Answer rate is therefore an important indicator of first-pass completion, not a complete substitute for measuring whether returned output can be consumed without intervention.
The central finding is a completion-versus-conditional-quality trade-off rather than a single first-time winner. A model can score highly on the answers it returns while failing most attempts, making its apparent quality a poor guide to production usability. That distinction is operationally separate from statistical reliability: a model may be measured consistently enough to show that it often fails to return usable work, while the available evidence may still be unable to settle close quality comparisons.
Stronger first-pass choices by workflow
The strongest option depends on the work being performed. For summarization, gpt-5.4-nano scored 9.33 out of 10 and answered 87% of attempts, compared with gpt-5.6-sol at 8.95 out of 10 and 54% answered [^10]. gpt-5.4-nano also scored 9.46 out of 10 for structured extraction, with 76% of attempts answered [^6]. Those figures make it a strong candidate where both returned quality and completion matter, provided that its output passes the deployment’s schema and validation checks.
Gemini-3.5-flash is another useful option across several structured workflows. It answered 77% of attempts for structured fact extraction, 86% for regulatory-filing analysis, 90% for query validation, and 85% for trading recommendations [^6][^5][^3][^8]. The spread across tasks reinforces that routing should be workflow-specific: the same model’s measured completion can be strong for one job and materially less dependable for another.
Referenced analyst writing also separates the alternatives clearly. Claude Opus 4-8 scored 9.02 out of 10 with 72% of attempts answered, while Claude Sonnet 5 scored 8.85 out of 10 but answered only 14% [^12]. When an unanswered request creates a costly interruption, Opus is the more dependable first-pass choice on this evidence. Sonnet is stronger in other workflows, however: it answered 88% of attempts for topic-cluster naming and 89% for summarization [^7][^10]. That is not a contradiction; it shows that a model that is operationally weak for referenced analyst writing can still be a reasonable router for cluster naming or summarization.
The quality-focused analysis tasks produce similar trade-offs. Deepseek-v4-pro scored 8.45 out of 10 for naming topic clusters with 84% of attempts answered, and 9.54 out of 10 for turning content into structured facts with 80% answered [^7][^6]. Qwen3.7-plus was close on cluster naming, at 8.09 out of 10 with 82% answered, and scored 8.65 out of 10 with 88% answered for SEC section analysis [^7][^5]. Claude-haiku-4-5 reached 8.08 out of 10 but answered only 68% of attempts [^7]. These comparisons do not establish one universal winner: the preferred model depends on whether the workflow prioritizes completion, judged quality, or a particular acceptance threshold.
Mixed and uneven results
Grok-4.5 illustrates why aggregate impressions are unsafe. It answered 78% of structured fact-extraction attempts and 50% of SEC-filing-analysis attempts, but only 29% for topic sequencing, 20% for trading recommendations, and 14% for promotional X messages [^6][^18][^16][^8][^9]. It may therefore be viable for some structured work while being a poor first-call choice for the latter workflows unless a fallback is already part of the design.
Editing results are similarly task-dependent. Deepseek-v4-pro scored 8.19 out of 10 ±0.29 for metadata-paragraph improvement after 46 judgments [^19]. Grok-4.5 led markdown line-break repair at 6.67 out of 10 ±0.27 after 30 judgments, ahead of Qwen3.7-plus at 5.87 out of 10 ±0.2 after 46 judgments [^11]. For language detection, gpt-5.5 scored 9.94 out of 10 ±0.01 after 297 judgments [^21]. It also scored 8.3 out of 10 ±0.25 for engagement replies, 8.45 out of 10 ±0.16 for image prompts, and 8.11 out of 10 ±0.18 for publication titles [^17][^20][^13]. These reported spreads and uncertainty figures should not be confused with answer rates: they describe the judged quality results supplied for the editing and creative tasks, not whether every request completed cleanly on the first call.
A stricter quality bar can justify accepting more failed calls if the returned work is materially better, while a tight latency or cost budget can favor a model with a lower answer quality but more reliable completion. The supplied findings do not provide price or latency comparisons for most of these reliability comparisons, so they cannot identify a universal cost-optimal route.
Where first-pass reliability is weak
Several models should not be placed in first-time production roles without a validated retry or fallback path. minimax/minimax-m3 scored 8.49 out of 10 for vetting subreddits, 8.59 out of 10 for validating queries, and 8.41 out of 10 for selecting X posts, yet answered 0% of attempts on each [^2][^23][^3][^15][^4]. It also scored 9.74 out of 10 for generating an author’s voice and 9.36 out of 10 for extracting structured facts, again with 0% answered [^14][^6]. Meta/Muse-Spark-1.1 likewise reached 9.15 out of 10 for analyzing onboarding prospects and 9.02 out of 10 for extracting structured facts, but had 0% answers across the listed tasks [^1][^22][^6].
Those results are operationally decisive even though the conditional quality scores are high. If no answer arrives, the quality of the answer that would have been returned cannot rescue the workflow. The current evidence does not show whether these failures could be recovered economically through a second provider call, nor whether the resulting output would then conform to schema. They do show that a high judged score alone is insufficient grounds for production routing.
Salvage, repair, and the real cost of a call
The first-pass distinction matters because salvage and repair are not the same thing. A response may be returned but require local cleanup, schema repair, or human review; a repair call is more consequential because it means another provider request, with added provider cost and latency. A model with a low headline task price can therefore be expensive per accepted result if it frequently fails to answer or produces output that must be repaired. Conversely, a more expensive model can be cheaper end-to-end when its higher first-pass completion avoids retries and downstream intervention.
The present partial results identify answer rates and judged quality, but do not supply model prices, batch-versus-sync prices, clean rates, salvage rates, or measured repair-call economics for these workflows. It would therefore be inappropriate to name a specific model as the cheapest end-to-end option from these figures alone. Engineers should calculate accepted-output cost as part of deployment testing, including the probability of an unanswered request, the probability of schema failure among returned answers, the cost of a repair call, and its latency. A model can be statistically well measured and still be operationally messy; those are different questions.
Routing recommendation
Use answer rate as a routing constraint, then optimize judged quality and price among models that clear the workflow’s minimum completion and schema bar. On the reported evidence, send summarization and structured extraction to gpt-5.4-nano when its measured answer rates meet the application requirement. Use Gemini-3.5-flash for structured extraction, regulatory-filing analysis, query validation, or trading recommendations when its task-specific variability is acceptable. Prefer Claude Opus 4-8 over Claude Sonnet 5 for referenced analyst writing when unanswered requests are costly; prefer Sonnet for topic-cluster naming or summarization, where its measured completion is stronger.
For specialized editing and creative work, route metadata-paragraph improvement to Deepseek-v4-pro, markdown newline repair to Grok-4.5, and language detection, engagement replies, image prompts, or publication titles to gpt-5.5, subject to the application’s quality bar. Keep minimax/minimax-m3 and Meta/Muse-Spark-1.1 out of first-time production roles unless a tested retry, repair, or fallback path is in place. The same rule applies to Grok-4.5 on topic sequencing, trading recommendations, and promotional X messages, where its reported answer rates are low.
These recommendations are workflow-level defaults, not a universal ranking. The evidence supports clear routing choices where completion gaps are large, but it does not establish how broadly close quality differences generalize beyond the measured work or what a repaired output costs in production. Validate the clean, schema-conforming rate and the actual second-call burden before treating any headline price as an end-to-end cost.
Check the live benchmark at https://llm-bench.kapualabs.com/ to set the quality bar your system actually needs and compare the models for the workflows that matter. Use it to compare batch against synchronous pricing yourself, then include salvage and any repair call rather than relying on headline task cost alone.