Researchers have introduced ThinkJev, a novel approach designed to optimize how language models handle typed decisions, which are choices among a closed set of options that code can act upon. While reading probabilities from a single forward pass is fast, it often fails on multi-step problems like arithmetic, counting, or date comparison. Conversely, allowing the model to think before answering corrects most errors but increases processing time from milliseconds to seconds. ThinkJev addresses this trade-off by building a dual-system architecture from a single open hybrid-thinking model that can answer either directly or after reasoning.
The framework utilizes two distinct systems derived from one model and one cached prompt. System 1 provides a fast response by reading the model's probabilities for option letters immediately, without generating thought tokens. System 2 engages in deeper reasoning, reading the same option letters only after the model has generated a thought process. Because both systems read from the same answer slots on the same prompt, escalating from System 1 to System 2 does not require reprocessing the input or running a second model. Furthermore, because the answer can be read after any number of thought tokens, a single sampled thought provides calibrated answers across every possible thinking budget.
ThinkJev operates without training by using a statistical test on labeled development data to determine when System 1 is reliable enough to answer alone. At a target error rate of 5%, the Qwen3.5-27B model successfully answered 24.5% of 8,359 test decisions using only System 1, achieving a realized error rate of 3.9%. For the remaining decisions requiring deeper thought, the system stops reasoning as soon as the answer reaches a confidence threshold. This adaptive approach reached 81.6% accuracy while using only 72% of the thought tokens required by models that always think, compared to 82.7% accuracy for constant thinking.
The study highlights that relying solely on System 1's confidence to decide when to think is less effective, as the decisions where System 1 is least confident are not always the ones that reasoning fixes. On JevBench, an independent benchmark for typed-decision services, Qwen3.5-27B using ThinkJev thought on only 29.4% of public items and achieved accuracy indistinguishable from two leading commercial thinking APIs. The researchers have made the code and data generators available publicly to support further exploration of this training-free cascade method.
Sources: arXiv cs.AI.