In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks. Four years and 3,000 contributors later, that project is @vllm_project, one of the largest open-source inference engines, running on ~500,000 GPUs at any given time. He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.) Their first user, Berkeley neighbor @arena, cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production. In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team. They're hiring across eng, marketing, and devrel in SF and Singapore: https://a16zjobs.substack.com/p/inferact-raised-one-of-the-largest @simon_mo_ @woosuk_k @KaichaoYou @rogerw0108