Seminars
Inference and AI Infrastructure (Special Event for #BosTechWeek)
The market focus has shifted from building LLMs and training to how to serve these models and efficient inference. In this meet up we are discussing the AI inference stack and how optimizing it is a multi dimensional problem, the role of hardware and software co-design and how applications are the only deliverable. We also deep dive into MLCommons Chakra, a framework for capturing AI application execution graphs.
Firefly: Scalable, Ultra-Accurate Clock Synchronization for Datacenters
Achieving the sub-10ns clock synchronization required by cloud-based financial exchanges is increasingly difficult because existing methods are often vulnerable to jitter, drift, and the complexities of large-scale network paths. Yuliang will showcase Firefly, a software-driven system that leverages a distributed consensus algorithm and a novel layered synchronization technique to provide resilient, high-precision time alignment across modern datacenters.
High-Dimensional Gradient-Free Optimization for Neuroscience, Interpretability and LLMs: Why It Works and How to Make It Better
Evolution strategies (ES) provide a vital gradient-free alternative for solving complex, high-dimensional optimization problems in fields like neuroscience and LLM fine-tuning where traditional backpropagation is often unavailable or inefficient. Binxu Wang will explore the geometric properties that enable these methods to succeed and demonstrate how identifying task-irrelevant parameter directions can be leveraged to further accelerate optimization in modern large-scale models.
Collapsing Towers of Interpreters for Security
Staged metaprogramming offers an alternative to traditional macros by using annotations to control execution timing, allowing programs to be specialized for known inputs by delaying specific computations. Cameron Wong will demonstrate how this technique can derive compilers from interpreters and reify low-level hardware behaviors to detect side-channel vulnerabilities, while also discussing future applications in address sanitization and decompilation.
Practical End-to-End Privacy and Data Use Policy Enforcement
While modern data is governed by strict privacy policies, developers often lack the practical abstractions needed to ensure their code complies, leading to frequent manual errors and costly violations. Malte Schwarzkopf will introduce Sesame, a framework that utilizes policy containers and static analysis to automate end-to-end privacy enforcement with minimal developer effort and low performance overhead.
The Collapse of Constrained Physical AI
As AI transitions from digital screens to autonomous physical systems like flight and construction, moving large-scale intelligence from the cloud to the edge presents significant technical and safety hurdles. Jason Jabbour will show these deployment constraints and share recent collaborative research that restores safety and task success within compressed Vision-Language-Action (VLA) models.
Open Source Software (OSS) Inference Stack
Open source inference systems have become critically important as models get bigger and bigger and agentic applications create demanding workloads. In this talk, we will discuss the key trends in model architecture and hardware accelerator server design and how open-source inference systems like vLLM and llm-d optimize performance against these trends.
Enhancing Performance Guarantees in Large-Scale Systems
Performance stability is fundamentally challenging because application-defined resources remain invisible to the operating system, leading to unpredictable degradation that escapes traditional system-level monitoring. Yigong Hu will present his research on making application-defined resources first-class entities for performance reasoning and control.
Learning-Directed Operating Systems
Modern applications often face performance bottlenecks due to rigid OS policies, even when resources are plentiful. Aditya Akella will present LDOS (Learning-Directed Operating System), which replaces manual policy design with data-driven, system-wide optimization.
Resilient Composition for Fault-Tolerant Cloud Applications
As cloud programming grows complex, new abstractions are vital for building correct, simple and efficient applications. Tianyu Li will introduce Resilient Composition, a method that ensures fault tolerance for applications built from distributed components.