Large Language Models
1
Progress on provable inference for large language models (FlashProve)
FlashProve is a software-only cryptographic system: a provider commits to model weights and attaches a short proof to each response showing the committed model produced it, verifiable on a CPU without revealing the weights. On Llama 4 Maverick (400B MoE, 4-bit) it proves about 110 tokens/s on one H100, with ~19 MB proofs per 1,024-token request and ~17 s verification. Overhead is still ~70–95x vLLM GPU time, and it covers integer, greedy-decoded models with 1,024-token context. Code and Lean proofs are Apache 2.0.
Why it mattersUsers and auditors today cannot confirm which model actually answered an API call; FlashProve makes that checkable without exposing weights. Being orders of magnitude faster than prior open systems, it moves verifiable inference from theory toward something evaluators and regulators could actually use.
Blog ↗Technical report ↗Code ↗
2
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
The authors test whether a general LLM can read and write network weights directly from natural-language goals, without gradient descent. From raw weights it identified which game PPO policies were trained on; hand-written Breakout weights scored 517.4 vs 25.3 for 500M-step PPO. Editing 0.016% of Qwen3.5-0.8B to remove reasoning loops lifted MATH-500 from 7.9% to 48.1% and IFEval from 21.0% to 62.3%. It is a blog post with no formal paper yet, and it did not help Gemma 3 270M.
Why it mattersIf frontier models can reliably reason about and edit weights, model repair and interpretability could become a prompting task rather than a training run. The results are early and informal, but the edited model is public, so others can check the claims.
Blog ↗Logs ↗Model ↗
3
Order-Only On-Policy Distillation
On-policy distillation (OPD) matches teacher token probabilities, which mix preference order with confidence that a smaller student may not reproduce. O³PD instead trains the student to reproduce the ordering of the teacher's top-K tokens, making the objective temperature-invariant. Across six teacher–student pairs it beats OPD on the HMMT25/AIME24/AIME25 average in every pair, by up to 2.9 points (up to 5.6 on AIME25).
Why it mattersOn-policy distillation is now a standard way to train small reasoning models, and this is a near drop-in change to its loss. The finding that ranking, not calibrated probability, carries the transferable knowledge is a useful design principle for distillation pipelines.
arXiv ↗PDF ↓
4
FLAMIA: FLAsh MIcroscaling Attention for Fast and Causally Safe Training on Blackwell
Blackwell's fast low-precision matmuls are hard to use for attention due to quantization overhead, causal leakage through shared scaling factors, and forward/backward numeric mismatch. FLAMIA provides MXFP8 and mixed NVFP4/FP8 attention kernels that reuse quantized operands, guard against scale-induced leakage and keep passes consistent, with document-local scaling for packed sequences. The MXFP8 path is 1.10–1.40x faster on B200 and 1.06–1.42x on B300 than the best baseline, with stable Llama-style pretraining.
Why it mattersLow-precision attention is one of the last big pieces of training that has not moved to FP8/FP4. Naming and fixing the causal-leakage problem from shared scales matters for anyone training long-context models on Blackwell.
arXiv ↗PDF ↓
AI Agents
5
CUA-Master: Scaling Computer-use Agents through Open-World Recursive Improvements
CUA-Master tackles narrow interaction data and saturating self-improvement with a nested double loop: an outer world-data-model flywheel guided by observed weaknesses, and an inner loop that mines real workflows from tutorial videos into reusable primitives for verifiable tasks, plus snapshot-rollback retries. CUA-Master-27B reports 80.2 on OSWorld-Verified, 36.9 on OSWorld 2.0, 43.9 on WeaveBench and 25.0 on Agents' Last Exam.
Why it mattersAn 80.2 on OSWorld-Verified from a 27B model pushes open-sized computer-use agents close to saturating that benchmark. Mining real workflows from tutorial videos is a scalable data source that avoids the diminishing returns of purely self-generated experience.
arXiv ↗PDF ↓
6
CoworkOS: Scaling Agent Coworkers for Real-world Automation
CoworkOS is a general harness that separates task strategy (model) from execution management (runtime), with a three-layer design for direct or delegated, sync or async subagent work, a shared application layer for co-editing documents, slides, CAD and Blender scenes, and persistent hierarchical memory. It is reported competitive with Codex and DeepSeek Harness across 12 benchmarks; with a GLM-5.3 backbone it ran five rounds of automated post-training on Qwen3.6-35B-A3B, raising its 8-benchmark average from 33.19 to 42.95.
Why it mattersThe paper is a concrete blueprint for long-running agent harnesses where humans and agents edit the same artifacts. The automated post-training case study is a notable example of agents doing multi-day ML engineering work end to end.
arXiv ↗PDF ↓
7
SWE-Trail: History-Guided Trajectory Collection for Coding Agent Training
Independent sampling of coding-agent trajectories becomes redundant, while feeding past trajectories as advice causes train-test mismatch. SWE-Trail uses history only to choose where to branch: at a promising state the agent proposes next actions, each is briefly executed from the same checkpoint, and the best route continues, with history kept out of context. A 124B student from Ling-3.0-flash-base gains 2.33 points on SWE-bench Multilingual and 1.60 on SWE-bench Pro over independent sampling.
Why it mattersData collection, not model architecture, is increasingly the bottleneck for coding agents. Using history to steer exploration while keeping it out of the agent's context is a clean way to get more diverse trajectories without a train-test mismatch.
arXiv ↗PDF ↓
