Adaptive Stopping of Streamed Reasoning for Language Model Agents

JunHyeok Oh, Byung-Jun Lee

Korea University

arXiv preprint

0.0 / 0.0 s

Same task, same reasoning model, one clock. Qwen3.8-27B shops for the same item under three stopping rules, replayed in the real WebShop environment with the paper's H200 timing. Full thinking reads every reasoning token and arrives in 103 s; a fixed budget matched to StreamStop's token use takes 43 s; StreamStop stops each decision as soon as the reasoning has settled and buys the right item in 19 s.

Abstract

Reasoning-enabled language model agents can incur substantial computation before acting, even though the amount of reasoning needed varies across decisions. Existing adaptive stopping methods use proxy signals such as confidence or intermediate-answer stability, whose effectiveness can vary across reasoning models. We introduce StreamStop, which learns when to stop from prefix-level action agreement using the available reasoning text. A frozen prefix reader supplies features to a lightweight stop head, while the reasoning model generates the action after stopping. We train the stop head with a discounted action-agreement objective and compute its expected return exactly over stopping times on recorded trajectories. Conditioning on the discount factor enables a single policy to support different trade-offs between reasoning consumption and action agreement. StreamStop keeps both language models frozen and requires no access to the reasoning model's internal states. We also construct a dataset of agent reasoning trajectories with prefix-level action annotations for offline evaluation of stopping decisions. In closed-loop evaluations on WebShop and ScienceWorld, StreamStop generally improves task success over matched fixed-budget stopping while consuming fewer reasoning tokens than Full thinking. On WebShop with Qwen3.8-27B, it uses 42% fewer reasoning tokens per episode with similar observed task success.

How much reasoning does one decision need?

An agent reasons before every action, and the reasoning it needs varies from decision to decision. A fixed thinking budget is therefore a guess: too short, and the action changes; too long, and the agent spends tokens and seconds that do not change what it does. The right question is not how long the model reasons on average but at which point of this reasoning stream the action is already settled.

Panel (a): action agreement with the full-reasoning action as a function of the number of thinking tokens read, for a 1.7B prefix reader and for the 27B reasoning model forced to answer at that point. Panel (b): cumulative distribution over decisions of the sufficient prefix length for the reader and for the reasoning model, together with the full reasoning length.
Where a decision settles. WebShop validation decisions with Qwen3.8-27B as the reasoning model. (a) Agreement with the full-reasoning action after reading the first k thinking tokens, for a small prefix reader (Qwen3-1.7B, scoring the valid candidate actions) and for the reasoning model itself forced to answer at k. (b) Cumulative distribution of the sufficient prefix, the earliest prefix after which every later action agrees with the reference, against the full reasoning length. Thinking-token axes are logarithmic.

A small reader decides when the reasoning is enough

The reasoning model \(\mathcal M\) streams its reasoning in chunks. After each chunk a frozen prefix reader \(\mathcal R\) encodes the prefix read so far, and a lightweight stop head \(\pi_\theta\), conditioned on a discount factor \(\gamma\), makes one binary call:

WAITreceive the next reasoning chunk and read again
STOP\(\mathcal M\) acts from the prefix it has so far; the rest of the reasoning is never produced

The head is trained offline, with no rollouts and no gradients through either language model. Every prefix \(t\) of a recorded trajectory \(\xi\) is labelled with whether the action generated from it agrees with the full-reasoning action, \(y_t\in\{0,1\}\), and the discounted agreement of a stopping policy is summed exactly over the finitely many stopping times, with \(q_t\) the probability of first stopping at prefix \(t\):

\[ J(\theta \mid \xi, \gamma) = \sum_{t=0}^{T} q_t\, \gamma^{e_t}\, y_t, \qquad e_t = t+1 . \]

\(\gamma\) is an input, not a hyperparameter: one head trained with \(\gamma\) sampled from a range covers a whole family of trade-offs, and at deployment the choice is a single number. Lower \(\gamma\) stops earlier; higher \(\gamma\) waits for more agreement.

Left: during agent interaction the frozen reasoning model streams reasoning chunks; a frozen prefix reader encodes the public context and the available prefix into features; a trainable stop head conditioned on the discount factor outputs WAIT or STOP; on STOP the reasoning model generates the action from the current prefix. Right: on a recorded trajectory every prefix is annotated with whether its action agrees with the full-reasoning reference action, and the stop head is trained on the exact expected discounted agreement over stopping times.
StreamStop at a glance. (a) Inference: reasoning model, prefix reader and stop head during one decision. (b) Training: recorded trajectories with per-prefix agreement labels, the exact expected return over stopping times, and one γ-conditioned policy for different trade-offs.
Four panels of action agreement against reasoning tokens per decision on validation decisions: WebShop with Qwen3.8 and with Nemotron, ScienceWorld with Qwen3.8 and with Gemini. StreamStop at a sweep of discount factors is compared with constant budgets, EAT settings, a Dynasor-CoT sweep, no thinking and full thinking.
One head, a whole frontier. Validation action agreement against reasoning tokens per decision for four environment and reasoning-model pairs. The StreamStop curve is a single stop head swept over γ; Const-k stops after k tokens; EAT and Dynasor-CoT are swept over their own settings. For Gemini 3.1 Flash-Lite the observable explicit plan stands in for the hidden native reasoning.

Results

Closed-loop task success on the test sets against the existing stopping rules. The proxy-signal methods rank differently from one reasoning model to the next: EAT is strong on Gemini and weak on Qwen3.8, Dynasor-CoT the other way round on Nemotron. StreamStop stays within three points of Full thinking in every configuration while reading 18–52% fewer reasoning tokens, and on WebShop it beats every other stopping rule by a wide margin.

MethodWebShop
Qwen3.8-27B		WebShop
Nemotron-30B		ScienceWorld
Qwen3.8-27B		ScienceWorld
Gemini 3.1 FL
success %tok/epsuccess %tok/epsuccess %tok/epsuccess %tok/ep
Full thinking (reference)43.05,94534.55,37452.145,38047.110,980
No thinking13.2018.3032.0039.90
Const-M (matched budget)35.72,11932.51,59950.112,56245.46,500
EAT33.22,36929.52,03548.815,09549.28,596
Dynasor-CoT31.72,08236.22,33848.117,86844.67,718
StreamStop, γ = 0.9840.32,67535.52,69349.220,75747.18,757
StreamStop, γ = 0.9942.83,46436.33,26349.221,66546.18,977
StreamStop, γ = 0.99541.84,54037.03,72750.823,68047.09,183

Test-set means over three evaluation seeds (Gemini: eight). tok/ep = reasoning tokens per episode; for Gemini they are explicit-plan tokens in reader-token units. Underlined: best success among the stopping rules (Full thinking excluded). Const-M stops after a per-setting token count chosen on the validation set to match StreamStop's mean consumption at γ = 0.99; EAT and Dynasor-CoT use one setting across all environments. Standard deviations across seeds and the remaining baseline settings are in the paper's Table 2.

Fewer tokens is also less time

Panels (a) to (c): success rate against estimated seconds per decision for WebShop with Qwen3.8, WebShop with Nemotron and ScienceWorld with Qwen3.8, comparing StreamStop at three discount factors with constant budgets, EAT, Dynasor-CoT, no thinking and full thinking. Panel (d): box summaries of the reader update time and the reasoning chunk interval in milliseconds.
Success against seconds per decision. (a–c) Test success (95% Wilson intervals) against the per-decision latency estimated from the recorded token counts with H200 timing. (d) Measured reader update time and reasoning chunk interval (box: interquartile range, line: median, whiskers: 10th–90th percentile). The reader runs in parallel with generation and adds under 2% to the decision latency in every setting.

Fast is not the same as right

Stopping early is only useful if the action survives. In the episode below the fixed budget stops after 352 tokens every time and buys the wrong shirt in 32 s; StreamStop keeps reading when the reasoning is still undecided and spends 102 s, but buys the right one, as Full thinking does in 246 s.

0.0 / 0.0 s
A harder task. Test episode on WebShop with Qwen3.8-27B, replayed in the real environment. Each window shows the page the run is on, the reasoning tokens generated for the current decision, the running token total and a time track (dark = reasoning tokens being generated, tick = an action). Seconds per decision follow the paper's H200 timing model applied to the recorded token counts; environment time is the recorded one.

What the stop head reads

The head's stop probability stays near zero through exploration and rises once the reasoning has committed to a plan. Below, two validation decisions in WebShop: the full reasoning is shown with the stop probability after each chunk as shading, and the point at which each method stops is marked with the action it then takes.

Two WebShop reasoning streams rendered as text with blue shading for the stop probability after each chunk; the StreamStop, Dynasor-CoT, Const-M and EAT-proxy cutoffs are marked, and under each stream the action chosen at each cutoff is listed with a check or cross for agreement with the full-reasoning action.
Two decisions, chunk by chunk. Qwen3.8-27B on WebShop, γ = 0.99. Shading is the stop probability after each reasoning chunk on a log scale (stopping threshold 0.5); the labelled boxes mark where StreamStop, Dynasor-CoT, Const-M and EAT-proxy stop, and the line under each stream gives the action generated at that cutoff, with ✓ / ✗ for agreement with the full-reasoning action. In (a) StreamStop stops once the prefix proposes inspecting a hoodie within the price limit; in (b) it waits through a tentative size choice until the prefix decides to go back to search.

A dataset of reasoning prefixes with their actions

Training and evaluating a stopping rule offline needs something ordinary agent logs do not have: the action the model would have taken from each earlier prefix. We recorded it. For every reasoning stream in the collection, the reasoning model was asked to act from a grid of prefix lengths, so stopping rules can be compared on agreement and tokens without any new generation or environment rollout.

Collection (training and validation tasks, four environment–model pairs)Count
Episodes11,194
Non-empty reasoning streams (decision attempts)123,819
Prefix-action annotations2,096,816

Unsuccessful episodes and invalid action responses are kept. Each record carries the task and public context, the full reasoning text, the full-reasoning action, and per-prefix generated actions with parser results. The release is planned together with the code.