Qizhen Weng 翁祈桢

My research interests broadly span AI Infrastructure, Machine Learning Systems, and Cloud Computing, with a special focus on GPU cluster management and large-scale model training, inference, and fine-tuning.

I joined Shanghai AI Laboratory after receiving the Ph.D. degree (2022) in Computer Science and Engineering from The Hong Kong University of Science and Technology, supervised by Prof. Wei Wang. I had been a Research Intern (2020) on cluster management in Alibaba for over two years. Before that, I obtained the B.Eng. degree (2017) from Shanghai Jiao Tong University and studied (2015) in UC Berkeley.

News & Highlights

Mar 21, 2024	💡Openings: I’m currently recruiting highly motivated students who can intern in Shanghai AI Laboratory for 3+ months. If you’re excited about advancing AI through large language models (e.g., InternLM [en/zh]), please fill in this form. Experience with deep learning frameworks, distributed systems, or reinforcement learning is a plus but not required.

Selected Publications (Full)

CaraServe: CPU-Assisted and Rank-Aware LoRA Serving for Generative LLM Inference

Suyi Li, Hanfeng Lu, Tianyuan Wu, Minchen Yu, Qizhen Weng, Xusheng Chen, Yizhou Shan, Binhang Yuan, and Wei Wang

arXiv preprint arXiv:2401.11240, 2024

Bib HTML PDF

@article{li2024caraserve,
  title = {CaraServe: CPU-Assisted and Rank-Aware {LoRA} Serving for Generative {LLM} Inference},
  author = {Li, Suyi and Lu, Hanfeng and Wu, Tianyuan and Yu, Minchen and Weng, Qizhen and Chen, Xusheng and Shan, Yizhou and Yuan, Binhang and Wang, Wei},
  journal = {arXiv preprint arXiv:2401.11240},
  publisher = {arXiv},
  year = {2024},
}

Beware of Fragmentation: Scheduling GPU-Sharing Workloads with Fragmentation Gradient Descent

Qizhen Weng, Lingyun Yang, Yinghao Yu, Wei Wang, Xiaochuan Tang, Guodong Yang, and Liping Zhang

In 2023 USENIX Annual Technical Conference (ATC), 2023

Bib HTML PDF Code Poster Slides

@inproceedings{weng2023FGD,
  title = {Beware of Fragmentation: Scheduling {GPU}-Sharing Workloads with Fragmentation Gradient Descent},
  author = {Weng, Qizhen and Yang, Lingyun and Yu, Yinghao and Wang, Wei and Tang, Xiaochuan and Yang, Guodong and Zhang, Liping},
  booktitle = {2023 {USENIX} Annual Technical Conference (ATC)},
  publisher = {{USENIX} Association},
  year = {2023},
  isbn = {978-1-939133-35-9},
  address = {Boston, MA},
  pages = {995--1008},
  url = {https://www.usenix.org/conference/atc23/presentation/weng},
}

MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters

Qizhen Weng, Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding

In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2022

Bib HTML PDF Code Slides

@inproceedings{weng2022MLaaS,
  title = {{MLaaS} in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous {GPU} Clusters},
  author = {Weng, Qizhen and Xiao, Wencong and Yu, Yinghao and Wang, Wei and Wang, Cheng and He, Jian and Li, Yong and Zhang, Liping and Lin, Wei and Ding, Yu},
  booktitle = {19th USENIX Symposium on Networked Systems Design and Implementation (NSDI)},
  pages = {945--960},
  year = {2022},
}

Metis: Learning to schedule long-running applications in shared container clusters at scale

Luping Wang, Qizhen Weng, Wei Wang, Chen Chen, and Bo Li

In International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2020

Bib HTML PDF Code Slides

@inproceedings{wang2020Metis,
  title = {Metis: Learning to schedule long-running applications in shared container clusters at scale},
  author = {Wang, Luping and Weng, Qizhen and Wang, Wei and Chen, Chen and Li, Bo},
  booktitle = {International Conference for High Performance Computing, Networking, Storage and Analysis (SC)},
  pages = {1--17},
  year = {2020},
  organization = {IEEE},
}

Semi-dynamic load balancing: efficient distributed learning in non-dedicated environments

Chen Chen, Qizhen Weng, Wei Wang, Baochun Li, and Bo Li

In 11th ACM Symposium on Cloud Computing (SoCC), 2020

Bib HTML PDF

@inproceedings{chen2020Semi,
  title = {Semi-dynamic load balancing: efficient distributed learning in non-dedicated environments},
  author = {Chen, Chen and Weng, Qizhen and Wang, Wei and Li, Baochun and Li, Bo},
  booktitle = {11th ACM Symposium on Cloud Computing (SoCC)},
  pages = {431--446},
  year = {2020},
}