Qizhen Weng 翁祈桢

My research interests broadly span AI Infrastructure, Machine Learning Systems, and Cloud Computing, with a special focus on GPU cluster management and large-scale model training, inference, and fine-tuning.

I joined Shanghai AI Laboratory after receiving the Ph.D. degree (2022) in Computer Science and Engineering from The Hong Kong University of Science and Technology, supervised by Prof. Wei Wang. I had been a Research Intern (2020) on cluster management in Alibaba for over two years. Before that, I obtained the B.Eng. degree (2017) from Shanghai Jiao Tong University and studied (2015) in UC Berkeley.

News & Highlights

Mar 21, 2024	💡Openings: I’m currently recruiting highly motivated students who can intern in Shanghai for 3+ months. If you’re excited about advancing AI through large language models (e.g., InternLM [en/zh]), please fill in this form. Experience with deep learning frameworks, distributed systems, or reinforcement learning is a plus but not required.

Selected Publications (Full)

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey

Jiangfei Duan, Shuo Zhang, Zerui Wang, Lijuan Jiang, Wenwen Qu, Qinghao Hu, Guoteng Wang, Qizhen Weng, Hang Yan, Xingcheng Zhang, and 6 more authors

arXiv preprint arXiv:2407.20018, 2024

Bib HTML PDF

@article{duan2024LLMSurvey,
  title = {Efficient Training of Large Language Models on Distributed Infrastructures: A Survey},
  author = {Duan, Jiangfei and Zhang, Shuo and Wang, Zerui and Jiang, Lijuan and Qu, Wenwen and Hu, Qinghao and Wang, Guoteng and Weng, Qizhen and Yan, Hang and Zhang, Xingcheng and Qiu, Xipeng and Lin, Dahua and Wen, Yonggang and Jin, Xin and Zhang, Tianwei and Sun, Peng},
  journal = {arXiv preprint arXiv:2407.20018},
  publisher = {arXiv},
  year = {2024},
}

InternLM2 Technical Report

Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, and 90 more authors

arXiv preprint arXiv:2403.17297, 2024

Bib HTML PDF Code

@article{cai2024Internlm2,
  title = {InternLM2 Technical Report},
  author = {Cai, Zheng and Cao, Maosong and Chen, Haojiong and Chen, Kai and Chen, Keyu and Chen, Xin and Chen, Xun and Chen, Zehui and Chen, Zhi and Chu, Pei and Dong, Xiaoyi and Duan, Haodong and Fan, Qi and Fei, Zhaoye and Gao, Yang and Ge, Jiaye and Gu, Chenya and Gu, Yuzhe and Gui, Tao and Guo, Aijia and Guo, Qipeng and He, Conghui and Hu, Yingfan and Huang, Ting and Jiang, Tao and Jiao, Penglong and Jin, Zhenjiang and Lei, Zhikai and Li, Jiaxing and Li, Jingwen and Li, Linyang and Li, Shuaibin and Li, Wei and Li, Yining and Liu, Hongwei and Liu, Jiangning and Hong, Jiawei and Liu, Kaiwen and Liu, Kuikun and Liu, Xiaoran and Lv, Chengqi and Lv, Haijun and Lv, Kai and Ma, Li and Ma, Runyuan and Ma, Zerun and Ning, Wenchang and Ouyang, Linke and Qiu, Jiantao and Qu, Yuan and Shang, Fukai and Shao, Yunfan and Song, Demin and Song, Zifan and Sui, Zhihao and Sun, Peng and Sun, Yu and Tang, Huanze and Wang, Bin and Wang, Guoteng and Wang, Jiaqi and Wang, Jiayu and Wang, Rui and Wang, Yudong and Wang, Ziyi and Wei, Xingjian and Weng, Qizhen and Wu, Fan and Xiong, Yingtong and Xu, Chao and Xu, Ruiliang and Yan, Hang and Yan, Yirong and Yang, Xiaogui and Ye, Haochen and Ying, Huaiyuan and Yu, Jia and Yu, Jing and Zang, Yuhang and Zhang, Chuyu and Zhang, Li and Zhang, Pan and Zhang, Peng and Zhang, Ruijie and Zhang, Shuo and Zhang, Songyang and Zhang, Wenjian and Zhang, Wenwei and Zhang, Xingcheng and Zhang, Xinyue and Zhao, Hui and Zhao, Qian and Zhao, Xiaomeng and Zhou, Fengzhe and Zhou, Zaida and Zhuo, Jingming and Zou, Yicheng and Qiu, Xipeng and Qiao, Yu and Lin, Dahua},
  journal = {arXiv preprint arXiv:2403.17297},
  publisher = {arXiv},
  year = {2024},
}

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference

Suyi Li, Hanfeng Lu, Tianyuan Wu, Minchen Yu, Qizhen Weng, Xusheng Chen, Yizhou Shan, Binhang Yuan, and Wei Wang

arXiv preprint arXiv:2401.11240, 2024

Bib HTML PDF

@article{li2024CaraServe,
  title = {CaraServe: CPU-Assisted and Rank-Aware {LoRA} Serving for Generative {LLM} Inference},
  author = {Li, Suyi and Lu, Hanfeng and Wu, Tianyuan and Yu, Minchen and Weng, Qizhen and Chen, Xusheng and Shan, Yizhou and Yuan, Binhang and Wang, Wei},
  journal = {arXiv preprint arXiv:2401.11240},
  publisher = {arXiv},
  year = {2024},
}

Beware of Fragmentation: Scheduling GPU-Sharing Workloads with Fragmentation Gradient Descent

Qizhen Weng, Lingyun Yang, Yinghao Yu, Wei Wang, Xiaochuan Tang, Guodong Yang, and Liping Zhang

In 2023 USENIX Annual Technical Conference (ATC), 2023

Bib HTML PDF Code Poster Slides

@inproceedings{weng2023FGD,
  title = {Beware of Fragmentation: Scheduling {GPU}-Sharing Workloads with Fragmentation Gradient Descent},
  author = {Weng, Qizhen and Yang, Lingyun and Yu, Yinghao and Wang, Wei and Tang, Xiaochuan and Yang, Guodong and Zhang, Liping},
  booktitle = {2023 {USENIX} Annual Technical Conference (ATC)},
  publisher = {{USENIX} Association},
  year = {2023},
  isbn = {978-1-939133-35-9},
  address = {Boston, MA},
  pages = {995--1008},
  url = {https://www.usenix.org/conference/atc23/presentation/weng},
}

MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters

Qizhen Weng, Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding

In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2022

Bib HTML PDF Code Slides

@inproceedings{weng2022MLaaS,
  title = {{MLaaS} in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous {GPU} Clusters},
  author = {Weng, Qizhen and Xiao, Wencong and Yu, Yinghao and Wang, Wei and Wang, Cheng and He, Jian and Li, Yong and Zhang, Liping and Lin, Wei and Ding, Yu},
  booktitle = {19th USENIX Symposium on Networked Systems Design and Implementation (NSDI)},
  pages = {945--960},
  year = {2022},
}

Metis: Learning to Schedule Long-Running Applications in Shared Container Clusters at Scale

Luping Wang, Qizhen Weng, Wei Wang, Chen Chen, and Bo Li

In International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2020

Bib HTML PDF Code Slides

@inproceedings{wang2020Metis,
  title = {Metis: Learning to Schedule Long-Running Applications in Shared Container Clusters at Scale},
  author = {Wang, Luping and Weng, Qizhen and Wang, Wei and Chen, Chen and Li, Bo},
  booktitle = {International Conference for High Performance Computing, Networking, Storage and Analysis (SC)},
  pages = {1--17},
  year = {2020},
  organization = {IEEE},
}

Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments

Chen Chen, Qizhen Weng, Wei Wang, Baochun Li, and Bo Li

In 11th ACM Symposium on Cloud Computing (SoCC), 2020

Bib HTML PDF

@inproceedings{chen2020Semi,
  title = {Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments},
  author = {Chen, Chen and Weng, Qizhen and Wang, Wei and Li, Baochun and Li, Bo},
  booktitle = {11th ACM Symposium on Cloud Computing (SoCC)},
  pages = {431--446},
  year = {2020},
}