Large Language Models
1
BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design
Existing FP4 KV-cache systems lose significant accuracy with vanilla quantization, while accurate 4-bit vector-codebook methods slow decoding. BitDecoding-FP4 tackles three error sources (query quantization, underflow in probability scaling and key outliers) and removes the softmax bottleneck with a transposed softmax fused with probability quantization. It stays within 0.5 points of BF16 KV accuracy with NVFP4 (1 point with MXFP4) and delivers 2.4–3.0x higher peak serving throughput on NVIDIA B300 with Qwen3-4B and Qwen3-30B-A3B.
Why it mattersKV cache memory is the main bottleneck for long-context serving, and native FP4 hardware is now shipping. A near-lossless FP4 KV path with open code could become a default serving optimization on Blackwell-class GPUs.
arXiv ↗PDF ↓Code ↗
2
DUET: Dual-Sparse Inference for Block Diffusion Language Models
Diffusion LMs decode in parallel but pay for attention over a growing prefix and for running every block position through the full Transformer at each denoising step. Duet is training-free: it reuses a compact, stable subset of prefix KV entries across steps and uses shallow layers to predict which positions will update, sending only those queries deeper. At 32K context it gives 2.41x end-to-end (4.72x per-step) speedup on SDAR-8B-Chat-b32 and 1.34x on LLaDA2.1-mini with competitive accuracy.
Why it mattersDiffusion LMs promise parallel decoding but have been held back by inference cost at long context. Training-free sparsity that more than doubles end-to-end speed makes them more practical to deploy.
arXiv ↗PDF ↓
3
CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse
Noting that LLMs keep re-deriving the same reasoning, the authors build CPU-executable agentic reasoning caches (ARCs) from LLM reasoning traces. They report performance on par with frontier LLMs on difficult math and science tasks. ARCs are described as explainable, continually trainable, robust to input perturbations and 16–291x cheaper to invoke than the source model.
Why it mattersTurning reasoning traces into reusable, CPU-executable artifacts is a different take on test-time compute: pay once, reuse cheaply. If the claims hold up, it points to large cost savings for repetitive reasoning workloads.
arXiv ↗PDF ↓
AI Agents
4
Test-Time Compute Scales Better with Parallel Agent Teams
The paper compares ways to spend inference compute on agents when a verifier can score attempts: best-of-N independent agents versus one large collaborating team. It proposes parallel agent teams, running many small teams in parallel and keeping the best team's result. Across 12 long-horizon tasks, including GPU kernel development and formal proofs, with a 27B model, it beats both baselines at a fixed 100-agent budget and at equal dollar cost against a frontier model; a statistical model explains why.
Why it mattersIt gives a concrete, verifier-based recipe for spending agent compute, with open code. The result that a mid-size 27B model with the right team structure can match frontier cost-efficiency matters for anyone running long-horizon agents.
arXiv ↗PDF ↓Code ↗
5
Authorization Drift in LLM Agents: Evaluating Boundary Compliance in Multi-Step Tool Use
The paper defines authorization drift, agents exceeding their authorized boundaries while pursuing a user's goal with no adversary present, across data, tool, scope and time. The Authorization Drift Benchmark has 648 samples in 7 domains, each pairing a compliant path with a tempting unauthorized shortcut. Across 13 LLMs, 3 frameworks and 25,207 trajectories, more capable models drift more (0.43–0.61 for frontier models), yet 86.18% of tasks have a clean-success run, and drift is driven mainly by early data over-collection.
Why it mattersIt shows a safety failure that appears without any adversary and grows with capability, which standard success-rate benchmarks miss. The legal mapping makes it directly relevant for teams deploying agents on personal or regulated data.
arXiv ↗PDF ↓
6
Knowing When to Intervene: Causal Evaluation of AI Agents
Static benchmarks cannot tell whether deployment-time interventions such as verifying an agent's output or steering it are worth it. The authors frame intervention value as a causal question and estimate dynamic causal effects of randomized interventions within trajectories from a relatively small dataset. They use the estimates to build safety leaderboards for audit prioritization and find, on tau-bench Retail, ToolSandbox and ImpossibleBench, that intervention effects vary substantially across actions and states.
Why it mattersIt moves agent evaluation from 'how good is the agent' to 'where should humans or monitors step in', using proper causal inference. That is the question operators actually face when budgeting oversight.
arXiv ↗PDF ↓
7
MemoryDojo: Benchmarking Poisoning Through LLM Agent Memory
Memory helps LLM agents on long multi-session tasks but creates an attack surface, since malicious instructions can be stored and acted on later. MemoryDojo provides 155 multi-session trajectories across 4 domains and 5 memory systems. All tested systems are vulnerable to memory poisoning, with large differences in downstream effects, and a memory-aware poison optimization process further increases attack efficacy.
Why it mattersPersistent memory is shipping in mainstream agents, and this gives the first systematic benchmark for poisoning it. Together with recent work on benign memory causing unsafe behavior, it argues memory needs its own security layer.
arXiv ↗PDF ↓
8
From Procedures to Permissions: Learning Precise Subagent Authorization from Agent Skills
Multi-agent systems control tools and environments but do not specify what resources each delegated subtask needs. The authors use Agent Skills, which describe the operations, tools and files per step, to synthesize tasks and procedures, run them through subagent delegation, and train on verified trajectories. On ClawArena-Team with Qwen3.5-35B-A3B, tool permission precision rises 13.4 points, workspace permission precision 24.1 points and task success 11.6 points.
Why it mattersLeast-privilege delegation is a practical gap in multi-agent systems. Reusing existing skill documents as free supervision for permissions is a cheap, deployable idea that pairs naturally with the authorization-drift findings.
arXiv ↗PDF ↓
9
ENGRAM: From Agent Experience to Weights through Verified Consolidation
ENGRAM studies moving agent experience from external memory into weights, separating which experiences to train on from when their records can stop being retrieved. A demonstration-aware gate admits skills the student has mastered or that have enough successful teacher demonstrations, and a verification gate releases records only if the updated model without retrieval matches thresholds against the base model with retrieval. The authors report a much smaller retrievable footprint near retain-all performance, plus fewer input tokens and model calls; the abstract gives no specific figures.
Why it mattersDeciding when an agent can safely 'forget' external memory after learning from it is an under-explored but practical problem for continual learning. The verification-before-release design is a sensible safeguard against silent capability loss.
arXiv ↗PDF ↓
Voice Agents
10
Benchmarking Speaker Embedders in the Sub-Second Regime for Real-Time Voice AI Agents
In real-time voice agents, background speech shares the target's spectral characteristics, passes through noise suppression and causes false acceptances. The paper benchmarks five open-source speaker embedders (TitaNet-Large, TitaNet-Small, ECAPA-TDNN, CAM++, SpeakerNet) under a sub-second budget, using a controlled set built by injecting a second speaker into real call silences plus a human-annotated production set. Denoising before embedding generally raises EER, truncating below the full voice-activity segment hurts every model, and about half of the TitaNet embedding is redundant.
Why it mattersBackground speakers triggering false barge-ins is a real production pain for voice agents. The paper gives concrete, counter-intuitive engineering guidance (skip denoising, use full VAD chunks, favor precision) rather than another model.
arXiv ↗PDF ↓