1 month ago · ai

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders

Introduction AdaSPEC is a new method that speeds up large language models by using a small draft model for the initial generation pass, followed by verificatio...

1 month ago · ai

[Paper] CanKD: Cross-Attention-based Non-local operation for Feature-based Knowledge Distillation

We propose Cross-Attention-based Non-local Knowledge Distillation (CanKD), a novel feature-based knowledge distillation framework that leverages cross-attention...