Publications

Ruihao Gong* Equal Contribution , Shihao Bai* Equal Contribution , Siyu Wu* Equal Contribution , Yunqian Fan, Zaijun Wang, Xiuhong Li, Hailong Yang, Xianglong Liu

Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2

DOI

Conference paper

OMNIBAL: Towards Fast Instruction-Tuning for Vision-Language Models via Omniverse Computation Balance

Yongqiang Yao, Jingru Tan, Feizhao Zhang, Jiahao Hu, Yazhe Niu, Bo Li, Xin Jin, Ruihao Gong📧 Corresponding Author , Pengfei Liu, Dahua Lin, Ningyi Xu

Proceedings of the 42nd International Conference on Machine Learning (ICML)

Conference paper

HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration

Yushi Huang, Zining Wang, Ruihao Gong📧 Corresponding Author , Jing Liu, Xinjie Zhang, Jinyang Guo, Xianglong Liu, Jun Zhang

Proceedings of the 42nd International Conference on Machine Learning (ICML)

Conference paper

DA-KD: Difficulty-Aware Knowledge Distillation for Efficient Large Language Models

Changyi He, Yifu Ding, Jinyang Guo, Ruihao Gong, Haotong Qin, Xianglong Liu

Proceedings of the 42nd International Conference on Machine Learning (ICML)

2024

Conference paper

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

Ruihao Gong* Equal Contribution , Yang Yong* Equal Contribution , Shiqiao Gu* Equal Contribution , Yushi Huang* Equal Contribution , Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track

DOI

Conference paper

TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models

Yushi Huang* Equal Contribution , Ruihao Gong* Equal Contribution , Jing Liu, Tianlong Chen, Xianglong Liu

The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Conference paper

Selective Focus: Investigating Semantics Sensitivity in Post-training Quantization for Lane Detection

Yunqian Fan, Xiuying Wei, Ruihao Gong, Yuqing Ma, Xiangguo Zhang, Qi Zhang, Xianglong Liu

Proceedings of the AAAI Conference on Artificial Intelligence

Conference paper

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

Ruihao Gong, Yang Yong, Zining Wang, Jinyang Guo, Xiuying Wei, Yuqing Ma, Xianglong Liu

Proceedings of the AAAI Conference on Artificial Intelligence

Journal article

Rectify representation bias in vision-language models for long-tailed recognition

Bo Li, Yongqiang Yao, Jingru Tan, Ruihao Gong, Jianwei Lu, Ye Luo

Neural Networks

DOI

Conference paper

QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models

Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, Bohan Zhuang

The Twelfth International Conference on Learning Representations

PDF Code

Conference paper

PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models

Zining Wang, Jinyang Guo, Ruihao Gong, Yang Yong, Aishan Liu, Yushi Huang, Jiaheng Liu, Xianglong Liu

ACM Multimedia 2024

Conference paper

PRoof: A Comprehensive Hierarchical Profiling Framework for Deep Neural Networks with Roofline Analysis

Siyu Wu, Hailong Yang, Xin You, Ruihao Gong, Yi Liu, Zhongzhi Luan, Depei Qian

Proceedings of the 53rd International Conference on Parallel Processing

DOI

Conference paper

Compressing Large Language Models by Joint Sparsification and Quantization

Jinyang Guo, Jianyu Wu, Zining Wang, Jiaheng Liu, Ge Yang, Yifu Ding, Ruihao Gong, Haotong Qin, Xianglong Liu

Forty-first International Conference on Machine Learning

2023

Conference paper

Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling

Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong📧 Corresponding Author , Jinyang Guo, Xianglong Liu

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

PDF Code DOI

Conference paper

Lossy and Lossless (L2) Post-training Model Size Compression

Yumeng Shi, Shihao Bai, Xiuying Wei, Ruihao Gong📧 Corresponding Author , Jianlei Yang📧 Corresponding Author

Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

PDF Code

Conference paper

Exploring the Relationship Between Architectural Design and Adversarially Robust Generalization

Aishan Liu, Shiyu Tang, Siyuan Liang, Ruihao Gong, Boxi Wu, Xianglong Liu, Dacheng Tao

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Conference paper

Annealing-Based Label-Transfer Learning for Open World Object Detection

Yuqing Ma, Hainan Li, Zhange Zhang, Jinyang Guo, Shanghang Zhang, Ruihao Gong, Xianglong Liu

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Conference paper

SysNoise: Exploring and Benchmarking Training-Deployment System Inconsistency

Yan Wang* Equal Contribution , Yuhang Li* Equal Contribution , Ruihao Gong* Equal Contribution , Aishan Liu* Equal Contribution , Jian Hu, Yongqiang Yao, Yunchen Zhang, Fengwei Yu, Xianglong Liu

Proceedings of Machine Learning and Systems

PDF Code Dataset Project

Conference paper

Exploiting Subgraph Similarities for Efficient Auto-tuning of Tensor Programs

Mingzhen Li, Hailong Yang, Shanjun Zhang, Fengwei Yu, Ruihao Gong, Yi Liu, Zhongzhi Luan, Depei Qian

Proceedings of the 52nd International Conference on Parallel Processing

DOI

Journal article

Discrepant Semantic Diffusion Boosts Transfer Learning Robustness

Yajun Gao, Shihao Bai, Xiaowei Zhao, Ruihao Gong, Yan Wu, Yuqing Ma

Electronics

DOI

2022

Journal article

Distribution-Sensitive Information Retention for Accurate Binary Neural Network

Haotong Qin, Xiangguo Zhang, Ruihao Gong, Yifu Ding, Yi Xu, Xianglong Liu

International Journal of Computer Vision

DOI

Conference paper

Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models

Xiuying Wei, Yunchen Zhang, Xiangguo Zhang, Ruihao Gong, Shanghang Zhang, Qi Zhang, Fengwei Yu, Xianglong Liu

Thirty-Sixth Conference on Neural Information Processing Systems

PDF

Conference paper

QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization

Xiuying Wei* Equal Contribution , Ruihao Gong* Equal Contribution , Yuhang Li, Xianglong Liu, Fengwei Yu

International Conference on Learning Representations

PDF Code Project

Conference paper

NNLQP: A Multi-Platform Neural Network Latency Query and Prediction System with An Evolving Database

Liang Liu, Mingzhu Shen, Ruihao Gong, Fengwei Yu, Hailong Yang

51 International Conference on Parallel Processing - ICPP

Code DOI

Conference paper

Generating Transferable Adversarial Examples against Vision Transformers

Yuxuan Wang, Jiakai Wang, Zixin Yin, Ruihao Gong, Jingyi Wang, Aishan Liu, Xianglong Liu

Proceedings of the 30th ACM International Conference on Multimedia

DOI

2021

Conference paper

Once Quantization-Aware Training: High Performance Extremely Low-Bit Architecture Search

Mingzhu Shen, Feng Liang, Ruihao Gong, Yuhang Li, Chuming Li, Chen Lin, Fengwei Yu, Junjie Yan, Wanli Ouyang

Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

PDF

Conference paper

MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing

Yuhang Li, Feng Zhu, Ruihao Gong, Mingzhu Shen, Xin Dong, Fengwei Yu, Shaoqing Lu, Shi Gu

Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

Conference paper

MQBench: Towards Reproducible and Deployable Model Quantization Benchmark

Yuhang Li* Equal Contribution , Mingzhu Shen* Equal Contribution , Jian Ma* Equal Contribution , Yan Ren* Equal Contribution , Mingxin Zhao* Equal Contribution , Qi Zhang* Equal Contribution , Ruihao Gong* Equal Contribution , Fengwei Yu, Junjie Yan

Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks

PDF Code Project

Conference paper

A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration

Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, Shi Gu

Proceedings of the 38th International Conference on Machine Learning

Conference paper

Diversifying Sample Generation for Accurate Data-Free Quantization

Xiangguo Zhang, Haotong Qin, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, Xianglong Liu

The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Conference paper

RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

Shiyu Tang* Equal Contribution , Ruihao Gong* Equal Contribution , Yan Wang* Equal Contribution , Aishan Liu* Equal Contribution , Jiakai Wang, Xinyun Chen, Fengwei Yu, Xianglong Liu, Dawn Song, Alan Yuille, Philip H. S. Torr, Dacheng Tao

Arxiv

PDF Dataset Project

Conference paper