Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 65.3k 11.9k

2. llm-d/llm-d llm-d/llm-d Public

   Achieve state of the art inference performance with modern accelerators on Kubernetes

   Shell 2.2k 262

3. vllm-project/llm-compressor vllm-project/llm-compressor Public

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   Python 2.4k 315