Page 19 / 25
291 posts in total. Keep on posting.
Showing posts 217–228 of 291. Each entry opens locally on this site; legacy Hexo posts link back to their original article at the bottom for reference.
2026
- 中
DistServe：通过 Prefill/Decoding 解耦实现面向 Goodput 的大模型服务优化 — 深度阅读笔记
1. 这篇论文为什么值得认真读 先给一句结论： DistServe 的价值在于：它把大模型服务里最痛的“延迟-成本矛盾”从调度层面硬扛，升级成“架构层面解耦”，然后再用算法做资源与并行策略的联合优化。 很多服务系统论文会给你“吞吐更高”的漂亮数字，但实际线上产品最在意的是： 用户是否足够快看到首个 token（TTFT）？ 后续生成是否足够流畅（TPOT）？ 在满足服务质量（SLO）的前提下，每块 GPU 的产出值不值？ DistServe 不是只追求 token/s，而是明确优化 goodput（满足 SLO 约束下的单位 GPU 可服务请求率）。这点非常“工程真相”。 论文报告的核心收益是： 相比主流方案，最高 7.4× 请求率提升； 或在同样请求率下，实现 12.6× 更严格 SLO； 且 >90% 请求满足延迟约束。 这是“性能数字 + 用户体验 + 成本效率”三者同时进步，而不是单指标提升。
- EN
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — Deep Technical Review
1. Why This Paper Matters If I explain this paper in one sentence: DistServe improves LLM serving under latency SLOs by separating prefill and decoding onto different GPU pools, then jointly optimizing their resources and placement to maximize goodput per GPU. This sounds simple, but it addresses one of the deepest frustrations in real-world LLM systems engineering: product wants both fast first response and smooth generation, infrastructure wants high utilization and low cost, and existing colocated serving designs often force a painful compromise. The paper makes this problem concrete, quantifies why the compromise happens, proposes a practical architecture, and validates it with strong end-to-end gains: up to 7.4× higher request rate at target SLO attainment, or 12.6× tighter SLOs at fixed rate, while keeping latency constraints satisfied for >90% of requests. That combination (clear diagnosis + design + measurable gains + deployment details) is why this is a serious ML systems paper.
- EN
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models — In-Depth Technical Review
1. Why This Paper Matters If you only remember one sentence from this review, I want it to be this: SmoothQuant is important because it turns a seemingly annoying numerical issue—activation outliers—into a clean systems trick that real hardware can actually use. Large language models are expensive for two reasons: they store a huge amount of weights, and they repeatedly move those weights and activations through matrix multiplications. That means memory footprint, memory bandwidth, and integer-kernel friendliness are not side details. They are central engineering constraints.
- 中
SmoothQuant：大型语言模型的精准高效训练后量化 — 深度阅读笔记
SmoothQuant 通过把激活值中的离群难度迁移到权重中，让大模型能够更稳定地实现高效 W8A8 推理。
- EN
ORPO: Monolithic Preference Optimization without Reference Model — In-Depth Technical Review
ORPO merges supervised fine-tuning and preference optimization into one objective, removing the need for a separate reference model while keeping strong alignment performance.
- 中
ORPO：不用参考模型的一体化偏好优化 — 深度阅读笔记
ORPO 将监督微调和偏好优化合并到一个目标中，在不依赖参考模型的情况下实现更简单的对齐训练。
- EN
Switch Transformers: Scaling to Trillion-Parameter Sparse Models — In-Depth Technical Review
Switch Transformer routes each token to a single expert, making trillion-parameter sparse models practical with major training-speed gains and simpler MoE scaling.
- 中
Switch Transformers：用简单高效的稀疏性扩展到万亿参数模型 — 深度阅读笔记
Switch Transformer 将每个 token 只路由到一个专家，在相近计算量下实现稀疏万亿参数模型的高效训练。
- EN
AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration — In-Depth Technical Review
AWQ uses activation-aware scaling to protect salient weights during low-bit quantization, enabling strong LLM accuracy with efficient deployment on edge and server hardware.
- 中
AWQ：感知激活值的大模型权重量化压缩与加速 — 深度阅读笔记
AWQ 利用激活感知缩放保护最关键的权重，在低比特量化下仍保持较强精度，并适合真实设备部署。
- 中
GPipe：微批次流水线并行的大规模模型训练 — 深度阅读笔记
GPipe 提出了微批次流水线并行方法，实现大规模神经网络的高效训练。本文从零讲解流水线调度算法、梯度累积、重计算内存优化，以及在 AmoebaNet 和 Transformer 上的实验结果。
- EN
GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism — In-Depth Technical Review
GPipe introduces micro-batch pipeline parallelism for efficient training of large neural networks. This review covers the pipeline scheduling algorithm, gradient accumulation, re-materialization for memory optimization, and experimental results on AmoebaNet and Transformer models.