Page 32 / 36
432 posts in total. Keep on posting.
Showing posts 373–384 of 432. Each entry opens locally on this site; legacy Hexo posts link back to their original article at the bottom for reference.
2026
- EN
Ring Attention: Blockwise Transformers for Near-Infinite Context — In-Depth Technical Review
Ring Attention enables near-infinite context length by distributing attention computation across devices in a ring topology. Covers blockwise computation, online softmax, and memory analysis.
- EN
Mamba: Linear-Time Sequence Modeling with Selective State Spaces — In-Depth Technical Review
Mamba introduces selective state space models as an alternative to Transformers with linear-time complexity. Covers selective scan, hardware-aware algorithms, and language modeling results.
- EN
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection — In-Depth Technical Review
GaLore reduces memory requirements for LLM training through gradient low-rank projection. Covers the mathematical foundation, subspace switching, and memory savings analysis.
- EN
Alpa: Automating Inter- and Intra-Operator Parallelism — In-Depth Technical Review
Alpa automates the search for optimal parallelism strategies combining data, tensor, and pipeline parallelism. Covers the ILP formulation, inter-operator DP, and compilation framework.
- EN
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — In-Depth Technical Review
GPTQ enables efficient post-training quantization of large language models to 3-4 bits with minimal accuracy loss. Covers the layer-wise quantization algorithm, Hessian-based error correction, and practical deployment.
- EN
Proximal Policy Optimization Algorithms — In-Depth Technical Review
PPO is one of the most influential RL algorithms. This review covers policy gradients, TRPO, the clipped surrogate objective, and PPO's role in RLHF/LLM alignment.
- 中
近端策略优化算法(PPO)— 深度阅读笔记
PPO(近端策略优化)是深度学习时代最具影响力的强化学习算法之一。本文从零开始详细讲解策略梯度、TRPO 到 PPO 裁剪目标的完整推导,覆盖 MuJoCo、Atari 实验分析,以及 PPO 在 RLHF/LLM 对齐中的核心作用。
- EN
MiRA: A Subgoal-driven Framework for Improving Long-Horizon LLM Agents — Technical Review
A detailed technical review of Google DeepMind's paper 'A Subgoal-driven Framework for Improving Long-Horizon LLM Agents', analyzing how MiRA uses milestone-based subgoal decomposition and potential-based reward shaping to overcome planning bottlenecks in long-horizon web navigation.
- EN
Attention Is All You Need: The Transformer — In-Depth Technical Review
An in-depth technical review of the Transformer architecture from "Attention Is All You Need", covering self-attention mechanics, positional encoding, multi-head attention, training details, and lasting impact.
- EN
BitNet: Scaling 1-bit Transformers for Large Language Models — In-Depth Technical Review
An in-depth technical review of BitNet, covering 1-bit weight quantization, BitLinear layer design, scaling laws for binary Transformers, and practical deployment implications.
- EN
ZeRO: Shattering the Memory Wall — How DeepSpeed Trains Trillion-Parameter Models
A technical review of ZeRO (Zero Redundancy Optimizer), analyzing how partitioning optimizer states, gradients, and parameters across data-parallel processes enables training of trillion-parameter models.
- EN
MetaGPT: When LLM Agents Form a Software Company — Multi-Agent Collaboration Done Right
A technical review of MetaGPT, analyzing how encoding human software development workflows (SOP) into multi-agent systems with structured communication reduces errors in automated code generation.