331 posts in total. Keep on posting.

2026

• 08-26 EN

  HqeKV: Making Quantization and Eviction Cooperate Inside a Single KV Cache Compression Framework

  HqeKV shows that quantization and eviction don't have to be rival KV-cache compression strategies — a five-way action space (three quantization precisions, eviction, or full precision), a joint key-value importance metric, and a search-based ratio allocator let one framework beat single-strategy baselines at matched memory budgets.

• 08-26 中

  HqeKV 阅读笔记：让量化和淘汰在一个 KV Cache 压缩框架里协同工作

  HqeKV 说明量化和淘汰并不必然是相互竞争的 KV cache 压缩策略——通过五选一的动作空间（三档量化精度、淘汰、或全精度）、一个联合 K-V 重要性度量，以及基于搜索的比例分配器，一个框架就能在相同内存预算下同时超过纯量化和纯淘汰的基线方法。

• 08-25 EN

  Best-Practice Critic Optimization: Making a Learned Critic Beat Group Sampling in LLM RL

  BPCO shows that a critic-based actor-critic recipe can match or exceed GRPO-style group sampling in LLM reinforcement learning, once the value head is range-bounded, the critic target is decoupled from bootstrapping, advantage normalization is removed, and GAE is made length-adaptive.

• 08-25 中

  Best-Practice Critic Optimization：让训练出来的 Critic 打赢分组采样

  BPCO 证明了：只要把价值头限幅、把 critic 目标和自举解耦、去掉优势归一化、并让 GAE 随长度自适应，一个训练得当的 critic 完全可以在 LLM 强化学习中匹配甚至超过 GRPO 式的分组采样。

• 08-23 EN

  FreeToken: Turning Your Gaming PC Into a Frontier-Model Serving Box

  FreeToken treats a personal machine's GPU, CPU, host memory, and PCIe link as one elastic MoE-serving platform, using a bandwidth-derived q* split between cache-fill and in-place CPU execution to serve a 753B model on a single workstation GPU — but the trick only pays off once sparse MoE activation has already made the problem edge-shaped in the first place.

• 08-23 中

  FreeToken 阅读笔记：把家用主机变成能跑千亿参数模型的推理平台

  FreeToken 把个人电脑的 GPU、CPU、内存、PCIe 链路统一当成一个可弹性调度的推理平台，用一条由带宽实测值推导出的 q* 公式在「传到 GPU 上算」和「直接在 CPU 上就地算」之间做最优分配，最终能在单张工作站 GPU 上跑起 7530 亿参数的模型——但这一切的前提，是稀疏 MoE 激活已经先把问题变成了「edge 形状」。

• 08-23 EN

  KV-Pipe: Turning a KV-Cache Trick Into a Pipeline-Parallel Load Balancer

  KV-Pipe repurposes cross-layer KV sharing — normally a memory-compression trick for inference — as a knob for balancing pipeline-parallel training stages, getting up to 9.2% higher MFU on an 8-stage pipeline for free, but the mechanism only pays off when your bottleneck is architecturally lopsided in the first place.

• 08-23 中

  KV-Pipe 阅读笔记：把 KV Cache 压缩技巧变成流水线并行的「负载均衡器」

  KV-Pipe 把跨层 KV 共享——原本只是推理阶段的显存压缩手段——改造成训练阶段流水线并行的负载均衡工具，在 8 级流水线上白拿 9.2% 的 MFU 提升；但这个机制只有在瓶颈本身就「结构性偏斜」时才真正好用。

• 08-22 EN

  CacheRoute: Why Your LLM Load Balancer Is Fighting Your KV Cache

  CacheRoute plans prefix-affinity routing and expected load together for LLM serving, reaching 93.2% served KV-cache hit rate and 2.3x the SLO capacity of the strongest baseline on a 70B model — but it can also make things worse, and the paper is unusually honest about exactly when.

• 08-22 中

  CacheRoute 阅读笔记：为什么你的负载均衡器一直在跟 KV Cache 打架

  CacheRoute 把前缀亲和路由和期望负载放在一起做离线规划，在 70B 模型上把服务端 KV 命中率做到 93.2%、SLO 容量做到最强 baseline 的 2.3 倍，但这篇笔记的重点其实是它同样诚实地报告了这套方法在什么情况下会反而变差。

• 08-21 EN

  CARE-LoRA: Fixing LoRA-FA's Frozen-Subspace Problem Without Paying for Full Activations

  CARE-LoRA replaces LoRA's stored full activation with a compressed activation plus a cheap least-squares reconstruction matrix, letting both LoRA factors stay trainable at close to LoRA-FA's memory cost while closing most of LoRA-FA's accuracy gap.

• 08-21 中

  CARE-LoRA：不用冻结子空间，也能省下 LoRA 的激活内存

  阅读笔记：CARE-LoRA 用一个压缩激活加一个轻量重建矩阵替代 LoRA 需要保存的完整激活，让两个 LoRA 矩阵都能继续训练，内存开销却接近只训练一半参数的 LoRA-FA。