Abstract

M* is a modular serving system for composite multimodal models, including models for image generation, speech, and robotic planning. Its Walk Graph abstraction represents model components as a dataflow graph and requests as graph traversals, supporting flexible component placement and distributed runtime optimizations. Compared with vLLM-Omni, M* achieves 20% lower average end-to-end latency on BAGEL text-to-image workloads and up to 2.9x lower real-time factor and 2.7x higher throughput on Qwen3-Omni text-to-speech workloads. It also accelerates V-JEPA 2-AC robotic planning rollouts by up to 12.5x over the baseline.

Article

Article URL

Article

pdf

BibTeX

@article{jha-atindra2026,
  title={M*: A Modular, Extensible, Serving System for Multimodal Models},
  author={Atindra Jha and Naomi Sagan and Keisuke Kamahori and Irmak Sivgin and Rohan Sanda and Steven Gao and Mark Horowitz and Luke Zettlemoyer and Olivia Hsu and Jure Leskovec and Baris Kasikci and Stephanie Wang},
  journal={to appear in Conference on Neural Information Processing Systems (NeurIPS)},
  year={2026},
  month={December}
}