293 posts in total. Keep on posting.

2026

• 05-28 中

  Mooncake：以 KV Cache 为核心的大模型推理服务解耦架构

  Moonshot AI（Kimi）如何将整个 LLM 服务系统围绕 KV Cache 的调度、复用与迁移重新设计——在长文本场景下实现 525% 的吞吐量提升，同时满足严格的 TTFT 和 TBT 延迟 SLO。

• 05-27 EN

  GQA: Grouped-Query Attention — Bridging Multi-Head Quality and Multi-Query Speed

  A deep technical review of Grouped-Query Attention (GQA), the architecture behind LLaMA 2/3, Mistral and most modern LLMs. We dissect the KV-cache bandwidth bottleneck, the uptraining recipe that converts an MHA checkpoint to GQA for only 5% of the original training cost, and the design space of group counts G that lets a system trade quality for decode speed almost continuously.

• 05-27 中

  GQA 分组查询注意力：用分组 KV 头桥接多头质量与多查询速度

  GQA（Grouped-Query Attention，分组查询注意力）是 LLaMA 2/3、Mistral 以及当下主流大模型背后的注意力变体。本文从 KV 缓存的带宽瓶颈出发，深入剖析 GQA 的形式定义、从 MHA 检查点出发只需 5% 训练成本的迁移训练（uptraining）配方、以及分组数 G 这一旋钮如何让系统在质量与解码速度之间近乎连续地权衡。

• 05-26 EN

  SimPO: Simple Preference Optimization with a Reference-Free Reward

  SimPO replaces DPO's reference-model-dependent implicit reward with a length-normalized average log probability, eliminates the reference model entirely, adds a target reward margin to the Bradley-Terry objective, and achieves up to +6.4 points on AlpacaEval 2 and +7.5 on Arena-Hard — all while keeping response length controlled. The Gemma-2-9B-it SimPO model ranked #1 on Chatbot Arena among all <10B models.

• 05-26 中

  SimPO：无需参考模型的简洁偏好优化

  SimPO 将 DPO 依赖参考模型的隐式奖励，替换为长度归一化的平均对数概率，彻底移除参考模型，并在 Bradley-Terry 目标中加入目标奖励边距。最终在 AlpacaEval 2 上超越 DPO 最高 +6.4 分、在 Arena-Hard 上超越最高 +7.5 分，且不引入回答长度膨胀。基于 Gemma-2-9B-it 的 SimPO 模型在 Chatbot Arena 人类真实投票中排名全部 10B 以下模型第一。

• 05-25 EN

  CodeAct: Executable Code Actions Elicit Better LLM Agents

  CodeAct proposes using executable Python code as the single unified action space for LLM agents, replacing fragmented JSON/text tool calls. With control flow, data reuse, existing libraries, and automated error feedback, agents using CodeAct achieve up to 20% higher success rates across 17 LLMs — and fine-tuned CodeActAgent rivals closed-source models on agent benchmarks.

• 05-25 中

  CodeAct：用可执行代码驱动更强的 LLM Agent

  CodeAct 提出用可执行 Python 代码作为 LLM Agent 的统一动作空间，取代碎片化的 JSON/文本工具调用。借助控制流、变量复用、现有软件库和自动错误反馈，CodeAct Agent 在 17 个大模型上的成功率提升最高达 20%——而微调后的 CodeActAgent 7B 模型在 Agent 基准上可比肩百亿规模闭源模型。

• 05-24 EN

  FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

  FlashAttention-2 pushes GPU attention kernels from 25-40% to 50-73% of theoretical A100 peak by fixing three concrete inefficiencies in FA1: unnecessary non-matmul FLOPs, underutilized sequence-length parallelism, and a warp-communication bottleneck. This review unpacks every algorithmic change with full derivations, the GPU execution model, and why each fix matters.

• 05-24 中

  FlashAttention-2：更好的并行策略与线程块工作划分

  FlashAttention-2 把 GPU 注意力核的效率从 A100 理论峰值的 25-40% 提升到 50-73%，靠的是精准修复 FA1 的三个具体瓶颈：多余的非矩阵乘 FLOP、序列维度并行度不足，以及 warp 内通信瓶颈。这篇笔记从 GPU 硬件原理出发，完整拆解每一项改动的数学推导、实现原理和设计边界。

• 05-23 EN

  DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

  DeepSeek-R1 shows that reasoning in large language models can emerge from pure reinforcement learning — without any human-annotated reasoning traces. This review unpacks GRPO, the multi-stage training pipeline, distillation to smaller models, and why this paper changed how the field thinks about post-training.

• 05-23 中

  DeepSeek-R1：用强化学习激发大语言模型的推理能力

  DeepSeek-R1 证明了大语言模型的推理能力可以从纯强化学习中涌现，不需要任何人类标注的推理轨迹。本文深入拆解 GRPO 算法、四阶段训练流水线、蒸馏方法及背后的设计取舍。

• 05-22 EN

  DoRA: Weight-Decomposed Low-Rank Adaptation — Technical Review

  DoRA (ICML 2024 Oral) decomposes pretrained weights into magnitude and direction components, then applies LoRA only to the directional part. This decomposition mirrors how full fine-tuning actually updates weights, closing the accuracy gap between LoRA and full FT without adding any inference overhead.