#KV-Cache
MorphServe: Making Model Precision Elastic for Bursty LLM Serving
Juncheng Yang
Most LLM servers treat a deployed model as fixed: its weights have one precision, its KV cache has one memory budget, and both remain unchanged until the process restarts.