· ai
AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
Introduction AdaSPEC is a new method that speeds up large language models by using a small draft model for the initial generation pass, followed by verificatio...