vLLM

Pinned Loading

vllm vllm Public

A high-throughput and memory-efficient inference and serving engine for LLMs

Python 31.7k 4.8k
llm-compressor llm-compressor Public

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python 786 65

vllm Public
A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-project/vllm’s past year of commit activity

Python 31,708 Apache-2.0 4,819 1,265 (7 issues need help) 403 Updated Dec 11, 2024
llm-compressor Public
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vllm-project/llm-compressor’s past year of commit activity

Python 786 Apache-2.0 65 25 35 Updated Dec 12, 2024
buildkite-ci Public

vllm-project/buildkite-ci’s past year of commit activity

HCL 7 16 0 3 Updated Dec 10, 2024
vllm-blog-source Public

vllm-project/vllm-blog-source’s past year of commit activity

SCSS 1 MIT 2 0 0 Updated Dec 4, 2024
flash-attention Public Forked from Dao-AILab/flash-attention
Fast and memory-efficient exact attention

vllm-project/flash-attention’s past year of commit activity

C++ 34 BSD-3-Clause 1,371 0 5 Updated Dec 2, 2024
vllm-project.github.io Public

vllm-project/vllm-project.github.io’s past year of commit activity

HTML 4 MIT 5 0 0 Updated Oct 31, 2024
vllm-nccl Public archive
Manages vllm-nccl dependency

vllm-project/vllm-nccl’s past year of commit activity

Python 16 Apache-2.0 2 2 0 Updated Jun 3, 2024
dashboard Public
vLLM performance dashboard

vllm-project/dashboard’s past year of commit activity

Python 20 Apache-2.0 4 0 0 Updated Apr 26, 2024