291 posts in total. Keep on posting.

2026

• 06-28 中

  Moebius：为 MoE 大模型推理服务实现无缝运行时并行策略切换

  Moebius 允许在 MoE 模型推理服务过程中于专家并行和张量并行之间实时切换，每次切换仅需 215-434 ms、额外内存开销仅 2.4%，RL rollout 速度提升 1.16-1.25 倍。

• 06-27 EN

  JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

  JetSpec resolves the causality-efficiency dilemma in speculative decoding by training a causal parallel draft head that generates all nodes of a candidate tree in one forward pass while preserving branch-wise autoregressive conditioning through a tree-causal attention mask — achieving up to 9.64x speedup on MATH-500.

• 06-27 中

  JetSpec：用并行树草稿突破推测解码的扩展上限

  JetSpec 通过训练一个因果并行草稿头，在单次前向传播中生成推测解码候选树的全部节点，同时借助树因果注意力掩码保留分支级因果条件依赖——将 MATH-500 端到端加速比提升至 9.64 倍。

• 06-26 EN

  SigmaScale: Learning to Scale Weight Matrices for Better SVD-Based LLM Compression

  SigmaScale learns per-layer row and column scaling vectors to reshape weight-matrix singular-value spectra before truncated SVD, improving compression quality in the mild-to-moderate regime without requiring specialized hardware.

• 06-26 中

  SigmaScale 阅读笔记：通过学习缩放矩阵改进 SVD 大语言模型压缩

  SigmaScale 通过梯度下降学习每层权重矩阵的行列缩放向量，在截断 SVD 之前重塑奇异值谱，从而在温和到中等压缩比下超越基于解析缩放的现有方法，且不依赖任何专用硬件。

• 06-25 EN

  ReMP: Low-Downtime Runtime Model-Parallelism Reconfiguration for LLM Serving

  ReMP turns static TP/PP topology into a runtime-adjustable resource, achieving topology switches in 1–7 seconds (100× faster than restart) through shared CPU weight stores, two-dimensional KV cache migration, and pre-built MPU state snapshots — enabling adaptive LLM serving under dynamic workloads.

• 06-25 中

  ReMP：LLM 推理服务中的低停机运行时并行拓扑重配置

  ReMP 将 TP/PP 拓扑从启动时的静态参数变成可在线切换的动态资源，通过 CPU 共享权重存储、二维 KV Cache 迁移和预构建 MPU 状态快照，在 7B 到 70B 参数规模上将拓扑切换时间从分钟级压缩到 1-7 秒，速度提升达 100 倍。

• 06-24 EN

  SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

  SparDA introduces a fourth per-layer Forecast projection that decouples KV block selection from attention computation, enabling lookahead CPU-to-GPU prefetch and a compact GQA-level indexer—delivering up to 1.7x decode speedup and 5.3x throughput over sparse attention baselines on 128K-context inference.

• 06-24 中

  SparDA：稀疏解耦注意力，让长上下文推理又快又准

  SparDA 通过引入第四个逐层 Forecast 投影，将 KV 块选择从注意力计算中解耦，实现提前一层预测并异步预取 CPU KV 缓存，在 128K 上下文推理中实现最高 1.7 倍解码加速和 5.3 倍吞吐量提升。

• 06-23 EN

  Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback

  Critique-GRPO integrates natural language critiques into online RL loops to overcome the three core failure modes of purely numerical reward feedback—performance plateaus, failed self-reflection, and persistent failures—achieving up to +21.6% Pass@1 improvements over GRPO on challenging math and reasoning benchmarks.

• 06-23 中

  Critique-GRPO：用自然语言批评反馈突破强化学习训练瓶颈

  Critique-GRPO 将自然语言批评反馈引入在线强化学习循环，解决纯数值奖励训练的三大结构性瓶颈——性能平台期、无效的自发自我反思与持续性失败——在 AIME 2024 等高难度推理基准上实现最高 +26.7% 的 Pass@1 提升。

• 06-22 EN

  MRAgent: Why Memory Should Be Reconstructed, Not Retrieved

  MRAgent replaces passive top-k retrieval with active, multi-step graph traversal over a Cue-Tag-Content associative memory, achieving up to 23% improvement on long-horizon conversational benchmarks while using 5x fewer tokens than competing methods.