Loading
Life sciences · Preprint
arXiv · September 9, 2026
Early or partial results. Treat as a signal, not a conclusion.
BRACE is a proposed correction method for critic bias in asynchronous RL training for language models. It demonstrates a 2.4% improvement on one task and 2.46× speedup over synchronous training, but lacks peer review, multi-task validation, and clarity on generalizability to other domains or human-relevant outcomes.
Preprint. Language model agents on BrowseComp-Plus task. Intervention: BRACE: anchored Bellman-residual correction with bounded correction horizon and constant-weight Monte-Carlo tail. Compared with: Strongest baseline (unspecified); synchronous training for speed comparison.
BRACE improves mean@1 on BrowseComp-Plus by 2.4% over the strongest baseline BRACE runs 2.46× faster per step than synchronous training BRACE remains stable at 50 updates off-policy
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
This is an algorithmic proposal for asynchronous RL with empirical validation on a single benchmark task, lacking peer review, comparative rigor across multiple domains, and human evaluation.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
Asynchronous reinforcement learning has become the standard way to scale training for language models, but the resulting policy lag biases the critic toward the stale behavior policy. Existing work on asynchronous LLM training corrects the actor and leaves this bias unaddressed, while the off-policy value correction of classical RL does not carry over to long-horizon agentic tasks, since a short correction horizon leaves the regression target free of the reward and a long one lets the product of importance ratios drift exponentially with the trajectory length. We propose BRACE, an anchored Bellman-residual correction for stale value models. BRACE bounds the correction horizon to a prefix of policy tokens and anchors a constant-weight Monte-Carlo tail beyond it, which separates policy correction from reward propagation. BRACE improves mean@1 on BrowseComp-Plus by $2.4\%$ over the strongest baseline, runs $2.46\times$ faster per step than synchronous training, and remains stable $50$ updates off-policy.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.