Page 6 / 25
291 posts in total. Keep on posting.
Showing posts 61–72 of 291. Each entry opens locally on this site; legacy Hexo posts link back to their original article at the bottom for reference.
2026
- 中
LACE-SVD 阅读笔记：均匀秩预算和局部重建为什么不够用了
LACE-SVD 指出，当 SVD 压缩比例足够激进之后，问题就不再是「怎么把每个矩阵近似好」，而变成了「怎么把有限的参数预算分给不同层」和「怎么防止误差沿残差流累积放大」；把秩分配与校准损失挂钩、再对残差流误差做门控修正，把 LLaMA-7B 在 60% 压缩比下的 WikiText-2 困惑度从 Dobi-SVD 的 46.18 降到了 32.57。
- EN
GIFT: Why the Coordinate System You Quantize In Matters More Than the Quantizer
GIFT shows that FP8 gradient communication error in LLM pretraining is partly a coordinate-system problem, not just a numerical-format problem, and fixes it by quantizing gradients in a cheap K-FAC-whitened space instead of raw Euclidean coordinates.
- 中
GIFT 阅读笔记:量化用的坐标系,比量化器本身更重要
GIFT 指出 LLM 预训练里 FP8 梯度通信的误差,有一部分根本不是数值格式的问题,而是坐标系的问题——把梯度先变换到一个用 K-FAC 曲率近似「白化」过的坐标系里再量化,比直接在原始欧氏坐标里量化更保真。
- EN
COBS: What Block-Sparse Attention Selectors Are Actually Missing (A Second-Order Fix)
COBS shows that every existing cacheable block selector for sparse attention (NSA, CSA, mean-pool, Quest) scores blocks using only a first-order approximation of the true attention mass, and recovers most of the gap to dense attention by caching a compressed second-order (covariance) term instead.
- 中
COBS 阅读笔记:块稀疏注意力的选择器到底漏掉了什么(一个二阶修正)
COBS 指出现有所有可缓存的块选择器(NSA、CSA、mean-pool、Quest)对注意力质量的估计都只用到了一阶信息;论文把块内 key 协方差的二阶项找回来,在长上下文检索基准上补齐了大部分与稠密注意力的差距。
- EN
RIPO: Why PPO's Ratio Clipping Is the Wrong Metric, and What Riemannian Geometry Fixes
RIPO shows PPO-Clip silently measures policy change with the wrong (Euclidean) metric, so it over-constrains rare exploration tokens and under-constrains dominant ones — and fixes this with a clipping rule derived directly from the policy's Riemannian (KL-induced) geometry, beating GRPO by up to 60% on AIME24.
- 中
RIPO 阅读笔记：PPO 的比率裁剪为什么量错了尺度,黎曼几何给出的修正
RIPO 指出 PPO-Clip 隐含地用欧氏距离衡量策略变化,导致稀有探索 token 被过度压制、主导 token 又被约束不足;论文用策略的黎曼(KL 诱导)几何重新推导出裁剪边界,在 AIME24 上比 GRPO 最高提升 60%。
- EN
Long-Horizon-Terminal-Bench: Dense Reward Grading Exposes Agent Long-Horizon Execution Limits
LHTB is a 46-task benchmark that replaces binary pass/fail with subtask-level dense rewards, revealing that even the strongest frontier models (GPT-5.5 at 15.2%) cannot reliably finish long-horizon terminal workflows — with 79% of failures caused by time-budget exhaustion rather than local reasoning errors.
- 中
Long-Horizon-Terminal-Bench 阅读笔记：密集奖励评估揭露 Agent 长程执行瓶颈
LHTB 用 46 个容器化终端任务替代二元通过/失败评分，以子任务级密集奖励量化 Agent 进度——结果发现即使最强模型 GPT-5.5 也只有 15.2% 通过率，79% 的失败源于超时而非单步推理错误。
- EN
DPPO: Fixing PPO's Ratio Clipping by Constraining Divergence Instead of a Single Token's Probability
DPPO shows that PPO's ratio-clipping trust region is a noisy single-token proxy that over-penalizes rare tokens and under-penalizes common ones — and replaces it with a cheap, direct estimate of policy divergence that stabilizes and speeds up LLM RL fine-tuning.
- 中
DPPO 阅读笔记：与其信任一个采样 token 的比率，不如直接约束策略散度
DPPO 指出 PPO 的比率裁剪信任域本质上是对单个采样 token 的高噪声估计——它会过度惩罚罕见 token、又对高频 token 惩罚不足；论文用一个几乎零开销的散度直接估计取而代之，显著提升了大模型强化学习的稳定性与效率。
- EN
FlashSVD v1.5: Why Low-Rank LLMs Don't Get Faster on Their Own
FlashSVD v1.5 shows that SVD-compressed transformers stay slow at serving time not because low-rank arithmetic is weak, but because a naive runtime shatters every layer into hundreds of tiny kernel launches — and that a unified dense-KV, packed-MLP, per-layer-graph-replay runtime turns the same fixed checkpoints into up to 2.55x faster decoding.