Page 33 / 36
432 posts in total. Keep on posting.
Showing posts 385–396 of 432. Each entry opens locally on this site; legacy Hexo posts link back to their original article at the bottom for reference.
2026
- EN
FlashAttention: The IO-Aware Algorithm That Made Transformers Actually Fast
A technical review of FlashAttention, analyzing how IO-aware tiling and kernel fusion achieve exact attention computation that is both faster and more memory-efficient than standard implementations.
- EN
LoRA: Fine-Tuning Giant Models with Pocket Change — The Low-Rank Revolution
A technical review of LoRA (Low-Rank Adaptation), analyzing how injecting trainable low-rank decomposition matrices enables parameter-efficient fine-tuning of large language models with minimal overhead.
- EN
Megatron-LM: NVIDIA's Blueprint for Training Billion-Parameter Models at Scale
A technical review of Megatron-LM's efficient large-scale training system, analyzing how tensor, pipeline, and data parallelism are composed to train trillion-parameter models on GPU clusters.
- EN
PaRO: Smarter Partitioning for Distributed Training — Beyond ZeRO's One-Size-Fits-All
A technical review of PaRO, analyzing how partial redundancy optimization in data-parallel training reduces memory overhead while minimizing communication costs through selective parameter partitioning.
- EN
Speculative Decoding: Making LLM Inference 2-3× Faster Without Losing a Single Token
A technical review of Speculative Decoding, analyzing how using a smaller draft model to propose tokens and a larger model to verify them achieves 2-3x inference speedup with mathematically identical output distributions.
- EN
InstructGPT: The RLHF Recipe That Turned GPT-3 Into a Helpful Assistant
A detailed technical review of InstructGPT (OpenAI, 2022), analyzing how Reinforcement Learning from Human Feedback (RLHF) with a three-stage pipeline — SFT, reward modeling, and PPO — transformed next-token prediction into instruction-following behavior that aligned with human intent.
- EN
AutoGen: Microsoft's Framework for Building Multi-Agent Conversations That Actually Work
A technical review of AutoGen, examining how multi-agent conversation frameworks with customizable agents enable complex LLM applications through cooperative dialogue patterns.
- EN
Generative Agents: 25 AI Characters Living in a Simulated Town — Believable Human Behavior from LLMs
A technical review of Generative Agents, analyzing how LLM-powered agents with memory, reflection, and planning create believable simulations of human behavior in interactive sandbox environments.
- EN
SWE-agent: Turning LLMs Into Autonomous Software Engineers That Fix Real GitHub Issues
A technical review of SWE-agent, analyzing how Agent-Computer Interface (ACI) design principles enable LLM agents to autonomously resolve real-world GitHub issues.
- EN
Self-Refine: Teaching LLMs to Critique and Improve Their Own Output — No Extra Training Needed
A technical review of Self-Refine, analyzing how iterative self-feedback loops enable LLMs to progressively improve their own outputs without external training or human supervision.
- EN
DeepSeekMath: How 120B Tokens of Math Data and GRPO Rival GPT-4 on Competition Problems
A detailed technical review of DeepSeekMath, analyzing how continued pretraining on math corpora combined with Group Relative Policy Optimization (GRPO) enables a 7B open model to rival frontier systems on challenging math benchmarks.
- EN
Reflexion: LLM Agents That Learn from Failure Through Verbal Self-Reflection
A technical review of Reflexion, exploring how language agents use verbal self-reflection as reinforcement signals to iteratively improve performance on coding, reasoning, and decision-making tasks.