Large Language Models
1
SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization
Instead of counting how often the router picks experts on domain-labeled data, the authors train a sparse autoencoder on an MoE layer's output. Because that output is a router-weighted sum of expert outputs and the encoder is affine, each latent's activation splits exactly across experts. On layers 4, 30 and 46 of Qwen3-30B-A3B, expert contribution is far more concentrated than routing, and most latents have only one or two dominant experts. Auto-interpretation reveals fine specialization within domains, such as separate experts for Java and front-end web code, or geometry versus arithmetic and polynomials.
Why it mattersIt gives an exact, routing-independent way to attribute features to MoE experts, replacing coarse domain-frequency analyses. The finding of fine-grained, uneven specialization matters for expert pruning, merging and targeted fine-tuning. Code is released.
arXiv ↗PDF ↓Code ↗
2
On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer
The paper studies PACT, where a larger model prefills the prompt and a smaller model generates, to balance capability and inference cost. The authors build PLatour, an open-source framework that links frozen models with lightweight zero-initialized linear bridges trained on task rewards, supporting hybrid-attention architectures. PLatour beats open-source baselines and Mostik's reported math and coding results with a much smaller sender model. But controlled experiments show some gains are matched by uninformative signals, self-communication or receiver-only training, so the authors propose PACT-Check to measure how much gain truly comes from communication.
Why it mattersLarge-small model collaboration is an appealing route to cheaper inference, but this paper shows reported gains can be illusory. PACT-Check offers a needed sanity check for any cross-model communication claim.
arXiv ↗PDF ↓Code ↗
AI Agents
3
The components of agent memory
Anthropic splits agent memory into layers: searchable transcripts, notes Claude writes, wiki-style representations and context injected into every session. On CL-Bench, persistent memory lifts Opus 5.5's mean reward from 0.655 to 0.838 and Sonnet 5's from 0.306 to 0.610. Offline 'dreaming' reviews logs to correct notes: in Pokémon Red, fixing a wrong landmark note let Claude reach Cerulean City while the uncorrected run stalled. On a corpus of about 350,000 files, a dreamed wiki plus search reached 61% versus 53% for search alone, and rigid rules like 'record only failures' reduced memory gains for stronger models.
Why it mattersIt is one of the most detailed public empirical looks at how frontier agents should write, curate and consolidate long-term memory. The finding that rigid memory rules hurt stronger models is directly actionable for agent builders.
Anthropic ↗
4
Tools Need Not Speak in Text: Latent Context Compression for Agents
The paper argues LLM agents need not read tool outputs as plain text. It introduces Coca-Cola, which compresses tool observations into latent tokens placed directly in the agent's input embedding space. Action-Preserving Training aligns next-action predictions under full and compressed observations so the latent tokens keep decision-relevant information. With web-search agents on four QA benchmarks and two model families, it cuts token use by 70.01–89.24% and end-to-end runtime by 24.45–55.97% while improving task performance.
Why it mattersTool outputs dominate context in search and coding agents, so compressing them in latent space attacks the main cost driver. Training compression to preserve actions, not text, is a useful design principle.
arXiv ↗PDF ↓Code ↗
5
Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training
The authors observe that variants of a shared agent harness fix different failures, and their usefulness shifts during training. Instead of searching for one best harness, Harness-S builds a fixed pool of specialized sibling harnesses, each targeting a different repair direction. When the base harness fails on every attempt, the policy rolls out under selected siblings and the successful trajectories feed reinforcement learning, with a router picking the top-k siblings via behavior profiles and multi-LLM voting. On AppWorld, SWE-Gym and DBBench, the trained policy beats policy-only training under the original harness.
Why it mattersIt treats the agent harness as a training-time resource rather than a fixed scaffold, recovering learning signal from tasks that would otherwise give zero reward. Gains transfer back to the original harness, which is what matters in deployment.
arXiv ↗PDF ↓Code ↗
6
SwarmEvolve: Latent Agent Swarms for Evolutionary Discovery and Optimization
SwarmEvolve is an LLM-guided evolutionary search framework that replaces one LLM's long reasoning per round with swarms of cheap agents. The agents reason in latent space and share information through KV caches. A novelty filter picks the most distinct ideas using distances between agents' hidden states, and a summarizer agent combines the swarm's outputs. Across 21 math-discovery and scientific-computing problems it achieves the best average score, reaching in a median 3–8 minutes what other frameworks reach in about 60, and with Qwen3 8B it improves some solutions AlphaEvolve first found with frontier LLMs.
Why it mattersIt suggests AlphaEvolve-style discovery does not require frontier models if many small agents communicate in latent space. Latent KV-cache sharing is an emerging multi-agent pattern worth watching.
arXiv ↗PDF ↓
Model Releases
7
Introducing Inkling 2 Small
Thinking Machines Lab launched Inkling 2 Small, a Mixture-of-Experts model with 594B total and 17B active parameters, image input via a 292M Perception Encoder-based vision encoder, and variable effort settings. It was pre-trained on 8.4T tokens followed by a joint RL run of 1,075 steps. The lab reports 72.1% on DeepSWE v1.1, 81.6% on MMMU Pro, 83.8% on ScreenSpot Pro and 31 on the Artificial Analysis Intelligence Index. It is available now via Tinker with closed weights, and the lab plans to release full weights under Apache 2.0 next week after external safety testing and red-teaming.
Why it mattersIf the weights ship as promised, this becomes one of the strongest Apache-licensed agentic coding and vision models. Releasing weights only after external red-teaming is also a notable process for an open release.
Blog ↗Model card ↗
8
ERNIE 5.5 Flash Base
Baidu released open weights for ERNIE 5.5 Flash Base, a 280B-total, 14B-active MoE pre-trained on 33.5T tokens with 128K context and designed to extend to 1M tokens. It is a base model without post-training, shipped in FP8 and BF16 under Baidu's own ERNIE 5.5 Flash Base Model License. Baidu reports 78.4 on MMLU-Pro (vs. 69.2 for DeepSeek-V4-Flash-Base) and 60.2 on GPQA, while trailing on some tests such as ChineseSimpleQA. The architecture builds on its Virtual Head Attention, Multi-View MoE and ECHO papers, and a post-trained version is expected later.
Why it mattersStrong open base models are scarce, and this one gives researchers a clean starting point for their own post-training. Note the custom license rather than a standard open-source one.
Hugging Face ↗Baidu blog ↗
Speech
9
Gabor Gabor: E-Gaborformer, a Strictly Causal Speech Recognizer with Gabor-Dictionary Convolutions and Delta-Rule Memory
E-Gaborformer is a strictly causal streaming speech encoder combining Gated DeltaNet-2 recurrent layers with sliding-window attention at a 3:1 ratio, with temporal convolutions built from a fixed set of causal Gabor atoms. Its streaming state stays at about 19 MB per call, whereas a same-size causal-attention encoder exceeds 1 GB after ten minutes. A 0.6B Hebrew-English production transducer shows 28–31% lower WER than NVIDIA's Nemotron 3.5 ASR Streaming on English voice-agent traffic. The same weights run strictly causal or with 480 ms lookahead without accuracy loss, and a two-class RNN-T formulation trains 2.6x faster.
Why it mattersConstant-memory streaming ASR directly addresses the cost of long voice-agent calls. Beating a strong NVIDIA streaming baseline on real voice-agent traffic makes linear-recurrent encoders a serious production option.
arXiv ↗PDF ↓
10
Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models
Flow-matching speech foundation models need 20–50 iterative steps per request, and classifier-free guidance doubles each step. The authors show distribution matching distillation (DMD) does not suit every task: in separation, wrong student outputs drift outside the teacher's training distribution. Route-DMD routes such tasks to a regression loss and uses distribution matching elsewhere, initializing the student with consistency distillation and replacing CFG with adaptive projected guidance. The 4-step student matches the 32-step teacher on generation, editing and processing tasks with a 4.5x end-to-end speedup.
Why it mattersFew-step distillation is key to making unified speech generation and editing models usable in real time. Routing losses by task type is a simple fix for a real failure mode of DMD outside generation.
arXiv ↗PDF ↓
