Seminars
H-Spec: Parallel Speculative Decoding Without A Drafter-Side KV Cache
Speculative decoding accelerates LLM inference with a lightweight draft model, but block diffusion drafters pay per-request memory and KV-write overhead for a separate drafter-side KV cache that grows with concurrency. Weifan will present H-Spec, a hybrid Mamba-attention parallel drafter that reuses target KVs in place and injects target hidden states only at the last input position, removing the drafter-side cache while preserving draft quality.
Houston, We Have a Problem: Systems Challenges of Large Scale Spacecraft Deployments
Cheap launches have moved satellites from a few expensive fault-tolerant high-orbit craft to constellations of commodity SmallSats, but flight software still carries the ad-hoc infrastructure built for one-off missions. Haoda will present Orbital, which derives tight worst-case execution time bounds for hard real-time multi-tenant scheduling, and Radshield, a programming model that replicates execution to protect commodity hardware from radiation-induced silent data corruption.
KV Cache on Flash: What to Write When Writes Wear Out
Long agentic sessions make the KV cache the dominant memory cost in LLM serving, and flash offers capacity HBM cannot, but every block written spends a fraction of the drive’s lifetime, so a flash tier needs an admission policy and not only an eviction policy. Anshvardhan will present a study of that one-shot admission decision on a 222-day production trace of agentic coding traffic, including a learned prefill-time scorer and a two-tier implementation inside vLLM.
RouteWise: Latency–Cost Optimization for Multi-Provider LLM Routing
The same open-weight LLM is now served by many providers with very different pricing models and time-varying latencies, so routing requests well can cut cost and latency without touching model quality. Haoran will present RouteWise, a multi-provider router that unifies heterogeneous pricing and jointly optimizes cost and latency, cutting cost by 26.6% and mean time to first token by 58.4% against OpenRouter’s automatic routing.
zcIO: Enabling Transparent Zero-copy for NVMe/TCP
NVMe-over-Fabrics via TCP has become a datacenter standard for disaggregated storage, yet it remains severely CPU-bound because traditional zero-copy TCP stacks cannot meet the strict page-alignment requirements of efficient hardware-to-software handoffs. Junghan will present zcIO, a Linux NVMe/TCP stack that uses record-aware networking to replace expensive memory copies with low-overhead page remapping, improving remote storage performance by up to 2.3x.
Do LLMs Have a Sense of Time? Zero-Shot Survival Curve Prediction with Frontier Language Models
Survival curves are the standard framework for modeling time-to-event outcomes in clinical biostatistics, yet they exist only at the population level, never as a per-instance label a model could learn to reproduce. Emma will ask whether frontier LLMs can nonetheless generate valid, personal survival curves zero-shot, benchmarking four models on emergency department revisit and hospital readmission prediction.
AI-Native Computing Systems: From Computing for AI to AI for Computing
As AI evolves from individual models into dynamic, autonomous systems, it poses new challenges for computer architecture while also becoming capable of joining the design process itself. Zishen will present work toward AI-native computing systems from two directions: cross-layer co-design for physical and neuro-symbolic AI, and AI agents that reason about, evaluate, and design computer architectures.
Rethinking Bursty Workloads and KV Cache Hierarchies for Efficient LLM Serving
Akira will present two recent projects on serving bursty, prefix-heavy LLM inference workloads: one showing how bursty arrivals can improve a cluster’s time-per-output-token and what that implies for request routing, and another on tuning the private vs. shared split of a distributed KV cache to optimize time-to-first-token—together demonstrating that intuitive systems choices can leave performance gains on the table.
AI and Dependability in Computing Systems: Friend or Foe?
Saurabh will examine both sides of AI’s effect on the reliability and security of computing systems—a foe when it complicates reasoning about probabilistic programs and enables phishing at scale, a friend when it powers causal root-cause diagnosis and automatic vulnerability patching—and highlight recent work from the NSF Center Chorus on streaming ML with probabilistic accuracy guarantees and secure decentralized ML.
Network-aware co-design for distributed machine learning systems
As ML training scales across racks, data-center buildings, regions, and even federated clients, periodic synchronization leaves compute idle and prolongs iterations. Minghao will argue for network-aware co-design—jointly optimizing algorithms, parallelism and pipeline strategies, and heterogeneity handling—and present THC, ScaleAcross Explorer, and FIELDING as works that cut synchronization costs across increasingly large deployments.