349 posts in total. Keep on posting.

2026

• 05-30 中

  VeriCache：让有损 KV 缓存实现无损 LLM 推理

  VeriCache 把 KV 缓存压缩重新包装成投机解码问题：用压缩 KV 生成草稿 token，再用完整 KV 校验，保证输出与全量 KV 推理完全一致，同时实现最高 4× 吞吐量提升。

• 05-29 EN

  IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression

  How double-sided KL-aware whitening, adaptive heterogeneous rank allocation, and loss-aware remapping combine to push SVD-based LLM compression to a new state of the art — with 4.34× decode throughput and minimal quality loss even at 60% parameter removal.

• 05-29 中

  IO-SVD：基于输入输出双侧白化的自适应秩LLM压缩方法

  KL散度感知的双侧白化 + 贪婪异构秩分配 + 损失感知量化重映射，三招组合将SVD压缩推到新的SOTA——在LLaMA-7B 80%保留率下PPL降至5.59，同时带来4.34倍解码吞吐提升。

• 05-28 EN

  Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving

  How Moonshot AI's Kimi serving platform redesigned LLM infrastructure around KV cache disaggregation—achieving 525% throughput gains for long-context workloads while maintaining strict TTFT and TBT SLO compliance.

• 05-28 中

  Mooncake：以 KV Cache 为核心的大模型推理服务解耦架构

  Moonshot AI（Kimi）如何将整个 LLM 服务系统围绕 KV Cache 的调度、复用与迁移重新设计——在长文本场景下实现 525% 的吞吐量提升，同时满足严格的 TTFT 和 TBT 延迟 SLO。

• 05-27 EN

  GQA: Grouped-Query Attention — Bridging Multi-Head Quality and Multi-Query Speed

  A deep technical review of Grouped-Query Attention (GQA), the architecture behind LLaMA 2/3, Mistral and most modern LLMs. We dissect the KV-cache bandwidth bottleneck, the uptraining recipe that converts an MHA checkpoint to GQA for only 5% of the original training cost, and the design space of group counts G that lets a system trade quality for decode speed almost continuously.

• 05-27 中

  GQA 分组查询注意力：用分组 KV 头桥接多头质量与多查询速度

  GQA（Grouped-Query Attention，分组查询注意力）是 LLaMA 2/3、Mistral 以及当下主流大模型背后的注意力变体。本文从 KV 缓存的带宽瓶颈出发，深入剖析 GQA 的形式定义、从 MHA 检查点出发只需 5% 训练成本的迁移训练（uptraining）配方、以及分组数 G 这一旋钮如何让系统在质量与解码速度之间近乎连续地权衡。

• 05-26 EN

  SimPO: Simple Preference Optimization with a Reference-Free Reward

  SimPO replaces DPO's reference-model-dependent implicit reward with a length-normalized average log probability, eliminates the reference model entirely, adds a target reward margin to the Bradley-Terry objective, and achieves up to +6.4 points on AlpacaEval 2 and +7.5 on Arena-Hard — all while keeping response length controlled. The Gemma-2-9B-it SimPO model ranked #1 on Chatbot Arena among all <10B models.

• 05-26 中

  SimPO：无需参考模型的简洁偏好优化

  SimPO 将 DPO 依赖参考模型的隐式奖励，替换为长度归一化的平均对数概率，彻底移除参考模型，并在 Bradley-Terry 目标中加入目标奖励边距。最终在 AlpacaEval 2 上超越 DPO 最高 +6.4 分、在 Arena-Hard 上超越最高 +7.5 分，且不引入回答长度膨胀。基于 Gemma-2-9B-it 的 SimPO 模型在 Chatbot Arena 人类真实投票中排名全部 10B 以下模型第一。

• 05-25 EN

  CodeAct: Executable Code Actions Elicit Better LLM Agents

  CodeAct proposes using executable Python code as the single unified action space for LLM agents, replacing fragmented JSON/text tool calls. With control flow, data reuse, existing libraries, and automated error feedback, agents using CodeAct achieve up to 20% higher success rates across 17 LLMs — and fine-tuned CodeActAgent rivals closed-source models on agent benchmarks.

• 05-25 中

  CodeAct：用可执行代码驱动更强的 LLM Agent

  CodeAct 提出用可执行 Python 代码作为 LLM Agent 的统一动作空间，取代碎片化的 JSON/文本工具调用。借助控制流、变量复用、现有软件库和自动错误反馈，CodeAct Agent 在 17 个大模型上的成功率提升最高达 20%——而微调后的 CodeActAgent 7B 模型在 Agent 基准上可比肩百亿规模闭源模型。

• 05-24 EN

  FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

  FlashAttention-2 pushes GPU attention kernels from 25-40% to 50-73% of theoretical A100 peak by fixing three concrete inefficiencies in FA1: unnecessary non-matmul FLOPs, underutilized sequence-length parallelism, and a warp-communication bottleneck. This review unpacks every algorithmic change with full derivations, the GPU execution model, and why each fix matters.