Page 31 / 36
432 posts in total. Keep on posting.
Showing posts 361–372 of 432. Each entry opens locally on this site; legacy Hexo posts link back to their original article at the bottom for reference.
2026
- 中
SmoothQuant:大型语言模型的精准高效训练后量化 — 深度阅读笔记
SmoothQuant 通过把激活值中的离群难度迁移到权重中,让大模型能够更稳定地实现高效 W8A8 推理。
- EN
ORPO: Monolithic Preference Optimization without Reference Model — In-Depth Technical Review
ORPO merges supervised fine-tuning and preference optimization into one objective, removing the need for a separate reference model while keeping strong alignment performance.
- 中
ORPO:不用参考模型的一体化偏好优化 — 深度阅读笔记
ORPO 将监督微调和偏好优化合并到一个目标中,在不依赖参考模型的情况下实现更简单的对齐训练。
- EN
Switch Transformers: Scaling to Trillion-Parameter Sparse Models — In-Depth Technical Review
Switch Transformer routes each token to a single expert, making trillion-parameter sparse models practical with major training-speed gains and simpler MoE scaling.
- 中
Switch Transformers:用简单高效的稀疏性扩展到万亿参数模型 — 深度阅读笔记
Switch Transformer 将每个 token 只路由到一个专家,在相近计算量下实现稀疏万亿参数模型的高效训练。
- EN
AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration — In-Depth Technical Review
AWQ uses activation-aware scaling to protect salient weights during low-bit quantization, enabling strong LLM accuracy with efficient deployment on edge and server hardware.
- 中
AWQ:感知激活值的大模型权重量化压缩与加速 — 深度阅读笔记
AWQ 利用激活感知缩放保护最关键的权重,在低比特量化下仍保持较强精度,并适合真实设备部署。
- 中
GPipe:微批次流水线并行的大规模模型训练 — 深度阅读笔记
GPipe 提出了微批次流水线并行方法,实现大规模神经网络的高效训练。本文从零讲解流水线调度算法、梯度累积、重计算内存优化,以及在 AmoebaNet 和 Transformer 上的实验结果。
- EN
GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism — In-Depth Technical Review
GPipe introduces micro-batch pipeline parallelism for efficient training of large neural networks. This review covers the pipeline scheduling algorithm, gradient accumulation, re-materialization for memory optimization, and experimental results on AmoebaNet and Transformer models.
- EN
Layer Pruning for Efficient Large Language Models — In-Depth Technical Review
Layer pruning removes redundant layers from LLMs to reduce compute and memory costs. Covers layer importance metrics, pruning strategies, and fine-tuning recovery.
- EN
Constitutional AI: Harmlessness from AI Feedback — In-Depth Technical Review
Constitutional AI trains harmless AI assistants using AI-generated feedback instead of human labels. Covers the critique-revision pipeline, RLAIF, and comparison with RLHF.
- EN
Chain-of-Thought Prompting Elicits Reasoning in LLMs — In-Depth Technical Review
Chain-of-Thought prompting enables LLMs to perform complex reasoning by generating intermediate steps. Covers few-shot CoT, zero-shot CoT, and analysis across arithmetic, commonsense, and symbolic tasks.