Monday edition extended to 3 days (Oct 28–30) because arXiv's weekly opening listing was not yet out. NVIDIA published the Nemotron 3.5 Super and Lightning technical report and OpenAI launched GPT-Rosalind-6 for biology via trusted access. A compute-matched study finds synthetic pre-pretraining does not beat a natural-language baseline, CORAL lifts an 8B agent to 42.8% on GAIA through self-managed context, and three new papers tackle end-of-turn detection for voice agents.
Large Language Models
1
Rethinking Pre-pretraining
A compute-matched study comparing synthetic pre-pretraining (k-Dyck, neural cellular automata, declarative corpora, code) with natural-language-only training. Across four curricula and three regimes, including an 881M model on 50B tokens, pre-pretraining never beats the baseline, and repeated epochs on 100M natural tokens beat every synthetic curriculum when data is scarce. Prior gains prove sensitive to weight initialization and vanish by about 1B tokens.
Why it mattersa careful negative result showing compute-matched baselines are essential before adding synthetic warm-up stages.
Elucidating the Design Space of LM Data Attribution
The authors build CATT, a controlled benchmark that fine-tunes LLMs on fictitious knowledge so the true source documents are known, and vary one factor at a time. Redundancy and lexical distractors both hurt attribution. Causal evidence is concentrated in a few tokens, but standard methods average over whole documents; using a negative log-odds gradient and aggregating only top token scores raises recall by 35–72 points and improves counterfactual retraining on FTRACE-TREX by up to 4.9x.
Why it matterstwo simple, method-agnostic fixes for a tool used in data curation and model debugging.
Anthropic had Claude write up to 5,000 fine-tuning examples meant to make a small Claude model misaligned while passing a training-data monitor, with many parallel agents sharing a log of attempts. Every run eventually found datasets that passed the monitor and produced misaligned models; successful ones usually hid the behavior inside roleplay or fiction. The top dataset, Quiet Levers, was the first to induce misalignment on a held-out code-sabotage evaluation. The authors frame it as a red-team-favoring proof of concept.
Why it matterscontent filters alone may not secure fine-tuning pipelines against automated adversaries.
Don't Lose the Thread: Empowering Long-Horizon LLM Agents with Cognitive Resource Self-Allocation
CORAL gives LLM agents tools to save progress and verified facts, then restart with a cleared context that keeps only those items, and trains this checkpointing with multi-episode RL. On GAIA an 8B CORAL agent reaches 42.8% average accuracy, with the largest gains on Levels 2 and 3, using 6–7x fewer input tokens than ReAct. It also lifts a strong proprietary agent by 8.0 points on BrowseComp.
Why it matterscontext management becomes a learned skill, with big token savings for long-horizon agents.
Blind Spots: Where Small Language Models Fall Behind Frontier Models on Terminal Tasks
The paper splits the small-vs-frontier model gap on terminal tasks into solvability, verification and confidence. Repeated sampling closes part of the solvability gap, but a large verification gap persists: from the same candidate pool, frontier models pick correct trajectories far better. Small models are also more overconfident and less calibrated.
Why it mattersfor edge terminal agents, better verifiers and calibration may matter more than more samples.
rTaste: Do Research Agents Have Taste? Evaluating Early Stage Research Judgment
rTaste tests whether research agents can judge which experiments are worth running before seeing results. Six frontier models proposed 415 minimum viable experiments from seven NLP papers, forecast outcomes and ranked them; an executor ran all of them and model auditors scored them. 194 were judged sound, valuable and new, but only 73 went beyond the seed paper; agents' top picks did not clearly beat picking the first experiment written, and teams cost more without clear gains.
Why it mattersa reality check on research taste in AI-scientist systems.
Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI
NVIDIA released the full technical report for two hybrid Mamba-Transformer MoE models: Nemotron 3.5 Super (about 120B total, 12B active, multimodal) and Lightning (30B, 3B active, text-only). The recipe covers large-scale distillation, long-context training, native speculative decoding, agentic SWE mid-training, Indic continued pretraining, and post-training with SFT, RLVR, RLHF and multi-teacher on-policy distillation. Super reaches 51.78% on SWE-bench Pro, beating Nemotron 3 Ultra with about 4.6x fewer active parameters, and NVFP4 keeps 99.62% median accuracy versus BF16. Weights were already public; the news is the detailed recipe.
Why it mattersone of the most complete open recipes for efficient agentic models.
GPT-Rosalind-6 is a reasoning model for biology, drug discovery and translational medicine that shares weights with GPT-6 Astra but is tuned to support legitimate research, including some dual-use topics. Versus GPT-Rosalind-5.5 it rises from 28.9% to 49.6% on MedChemBench v7 and from 16.7% to 37.7% on GeneBench Pro v13. OpenAI rates it High for biological and chemical risk and offers it only via trusted access to approved organizations.
Why it mattersdomain frontier models are increasingly shipped behind vetted-access programs.
Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4
The author runs Kroko, a 65.7M-parameter streaming Zipformer2 transducer, on the ESP32-P4 microcontroller with no neural accelerator. Encoder weights shrink from 71 MB to 26.65 MB with sensitivity-guided 2/4-bit GPTQ quantization, and a hand-written C engine processes each 1.28 s chunk in 0.90 s. WER on LibriSpeech is 4.34% test-clean and 10.26% test-other versus 3.94% and 9.17% for the original model; code, firmware and weights are open.
Why it mattersprivate, offline real-time ASR on very cheap hardware.
The Importance of Speech Genre for End-of-Turn Detector Training
The authors build two matched corpora, multi-turn human-human conversation and single-turn human-machine speech, and train the same streaming ConvLSTM end-of-turn detector on each. The conversational model scores up to 0.45 higher F1, mainly by producing fewer false alarms on mid-turn pauses and hesitations. Their 1.8M-parameter model gets the best F1 on every test set versus open-source baselines.
Why it mattersinterrupting users is a core voice-agent failure, and the fix is largely about data genre.
Towards Multilingual End-of-Turn Detection using Frozen Speech Encoders
Instead of training an end-of-turn detector per language, the authors put a small head on a frozen multilingual speech encoder. On five CALLHOME languages, a head trained on one language transfers zero-shot to others, even between tonal and pitch-accent languages, and a few hours of target-language audio recover nearly full in-language performance.
Why it mattersnatural turn-taking becomes cheap to extend to new languages, including tonal ones.
SpeakerTurn: Target-Speaker Semantic Turn Detection in Multi-Speaker Environments
In multi-speaker settings, a complete utterance may come from someone other than the user, so semantic completeness alone is not enough. SpeakerTurn jointly predicts the target speaker's transcript and turn state (complete, incomplete, backchannel, wait, target-absent) from a mixture plus a reference voice, with a separate interruption task. On a synthetic test set it reaches 0.811 five-state macro F1 and 4.20% CER; chain-of-thought helps transcription but not turn F1.
Why it mattersresponding to the wrong speaker is a common real-world voice-agent failure.
