DA-KD: Difficulty-Aware Knowledge Distillation for Efficient Large Language Models

Jan 1, 2025·

Changyi He

,

Yifu Ding

,

Jinyang Guo

Ruihao Gong

Ruihao Gong

,

Haotong Qin

,

Xianglong Liu

· 0 min read

Type

Conference paper

Publication

Proceedings of the 42nd International Conference on Machine Learning (ICML)

Last updated on Jan 1, 2025

Ruihao Gong

Authors

← AtomNet: Designing Tiny Models from Operators Under Extreme MCU Constraints Apr 1, 2025

HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration Jan 1, 2025 →