Simon Mo
@simon_mo_
building @inferact for @vllm_project
Joined July 2018
368 Following 4.3K Followers
❤️ vLLM is PagedAttention, but quickly expanded to the platform for open source models, frontier hardwares, and efficient inference for all workload at speed of light! Today, the PA kernel is no longer used as all kernels support it natively; the scheduler and engine remained.
Show more
