[Paper] KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
LLMs are widely adopted in production, pushing inference systems to their limits. Disaggregated LLM serving (e.g., PD separation and KV state disaggregation) im...