Publications
All Publications
Comprehensive list — accepted papers, under-review submissions, preprints, and books. For the most current bibliographic information, please refer to my Google Scholar profile.
30 Total
22 Published
8 Under Review
Conference Paper
- NeurIPS 2026 Workshop RTCA2026When Does a Speech Model Know to Hand Off? Reading Competence from Hidden States for Real-Time Escalation Ruoyu He, Jisen Li, Anthony Ji, Changyi Yang, Zetian Li, Runduo Ma, Zhen Zhang,
- NeurIPS2026Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers Yuhong Chou, Zehao Liu, Yuqi Pan, Qian Liu, Xianwei Chen, , Jibin Wu
- EMNLP2026Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding Fengxiang Bie, Yuqing Jian, Yifan Yu, , Zelei Shao, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu, Tianyi Zhang
arXiv:2609.09338.
- NeurIPS2026OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization , Donglin Zhuang*, Jisen Li, Ziyan Chen, Shuaiwen Leon Song, Ben Athiwaratkun, Xiaoxia Wu
500+ GitHub stars within the first month of release.
- COLM2026I-DLM: Introspective Diffusion Language Models Yifan Yu, Yuqing Jian, Junxiong Wang, , Donglin Zhuang, Xinyu Fang, Sri Yanamandra, Xiaoxia Wu, Qingyang Wu, Shuaiwen Leon Song, Tri Dao, Ben Athiwaratkun, James Zou, Fan Lai, Chenfeng Xu
Introspective strided decoding for diffusion language models.
- COLM2026Squeeze Evolve: Unified Multi-Model Orchestration for Verifier-Free Evolution Monishwaran Maheswaran, Leon Lakhani, , Shijia Yang, Junxiong Wang, Coleman Hooper, Yuezhou Hu, Rishabh Tiwari, Jue Wang, Harman Singh, Qingyang Wu, Yuqing Jian, Ce Zhang, Kurt Keutzer, Tri Dao, Xiaoxia Wu, Ben Athiwaratkun, James Zou, Chenfeng Xu
Unified multi-model orchestration for verifier-free evolution.
- ICLR2026
- ICML2026Aurora: When RL Meets Adaptive Speculative Training — A Unified Training-Serving System Junxiong Wang*, Fengxiang Bie*, Jisen Li, Zelei Shao, Yubo Wang, Yinghui Liu, Qingyang Wu, Avner May, Sri Yanamandra, Ce Zhang, Tri Dao, Percy Liang, Ben Athiwaratkun, Shuaiwen Leon Song, , Chenfeng Xu, Xiaoxia Wu
- MLSys2026KITTY: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost Haojun Xia, Xiaoxia Wu, Jisen Li, Robert Wu, Junxiong Wang, Jue Wang, Chenxi Li, Aman Singhal, Alay Dilipbhai Shah, Alpay Ariyak, Donglin Zhuang, , Ben Athiwaratkun, Zhen Zheng, Shuaiwen Leon Song
- Together AI Blog2026CoderForge-Preview: SOTA open dataset for training efficient coding agents Alpay Ariyak*, Junda Zhang, Junxiong Wang, Shang Zhu, Federico Bianchi, Sanjana Srivastava, Ashwinee Panda, Siddhant Bharti, Chenfeng Xu, John Heo, Xiaoxia Shirley Wu, James Zou, Percy Liang, Leon Song, Ce Zhang, Ben Athiwaratkun, , Qingyang Wu*
Project core lead.
- NeurIPS 2025 Workshop on Efficient Reasoning2025Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time (CREST) Zhenyu Zhang, Xiaoxia Wu, , Qingyang Wu, Yineng Zhang, Pragaash Ponnusamy, Harikaran Subbaraj, Jue Wang, Shuaiwen Leon Song, Ben Athiwaratkun
Ratings refer to the parent conference.
- ICML2025
- NeurIPS2024CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang, Xiaojie Li, , Shuaiwen Leon Song, Jianlong Wu, Liqiang Nie, Bernard Ghanem
- USENIX ATC2024Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPU Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shiyang Chen, Zhewei Yao, Stephen Youn, Arash Bakhtiari, Michael Wyatt, Donglin Zhuang, , Olatunji Ruwase, Yuxiong He, Shuaiwen Leon Song
- VLDB2024
- ICSE2020
Journal Paper
- IEEE TPAMI2025RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Models Fengxiang Bie, Yibo Yang, , Adam Ghanem, Minjia Zhang, Zhewei Yao, Xiaoxia Wu, Connor Holmes, Pareesa Golnari, David A. Clifton, Yuxiong He, Dacheng Tao, Shuaiwen Leon Song
Survey Papers track.
- Sensors (MDPI)2021Binary Neural Network for Automated Visual Surface Defect Detection Wenzhe Liu, Jiehua Zhang, Zhou Su, , Li Liu
Special Issue on Intelligent Sensing and Monitoring for Industrial Process.
Under Review
- arXiv + Under Review2026Tail-Likelihood Reinforcement Learning Shrinivas Ramasubramanian, Daman Arora, Fahim Tajwar, Guanning Zeng, Qingyang Wu, , Chenfeng Xu, Haiwen Feng, Yuda Song, Aarti Singh, Ruslan Salakhutdinov, J. Andrew Bagnell, Jeff Schneider, Andrea Zanette
Tail-likelihood objective for preserving rare high-reward rollouts and improving Best-of-k inference-time scaling.
- arXiv + Under Review2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus Zunhai Su, Bohan Sun, Xialie Zhuang, Shuibai Zhang, He Xiao, Jing Xiong, Hengyuan Zhang, , Tiantian Zhang, Ngai Wong, Chuan-Wei Kuo
Featured in an in-depth interview by Synced (机器之心 Pro), reaching 3,839 reads.
- arXiv + Under Review2026
- arXiv + Under Review · COLM 20262026Sketched Linear Contrastive Learning: Approximation, Optimization, and Statistical Scaling Ziyan Chen, , Dingxuan Zhou
- arXiv + Under Review2026From One-Pass SGD to Data Reuse: Mini-Batch Scaling Laws in Sketched Linear Regression Ziyan Chen, , Dingxuan Zhou
- arXiv + Under Review2026
- Ph.D. Thesis · University of Sydney2026Efficient Compression Algorithm-System Co-Design for Large-Scale Model Training and Inference
Doctor of Philosophy (Engineering) thesis at The University of Sydney.
- arXiv + Under Review2026Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient , Yibo Yang, Ziyan Chen, Fengxiang Bie, Haojun Xia, Xiaoxia Wu, Robert Wu, Ben Athiwaratkun, Bernard Ghanem, Shuaiwen Leon Song
Preprint Paper
- arXiv2026QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs Weili Xu, Jisen Li, Yuqing Jian, Chenxi Li, Zhizhou Sha, Yifan Yu, Qingyang Wu, Chenfeng Xu, , Tianyi Zhang, Ben Athiwaratkun
- arXiv2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, , Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, Yuxiong He
- arXiv2023DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang, Conglong Li, Shiyang Chen, Chengming Zhang, Masahiro Tanaka, Xiaoxia Wu, Jeff Rasley, Ammar Ahmad Awan, Connor Holmes, Martin Cai, Adam Ghanem, , et al.
Book
- Tianjin Univ. Press