Discussions

• 1 You must be logged in to vote

  🙏

  How to load the model successfully through multi-card in vllm?

  SmallHappyJerry asked Mar 27, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Increase prefix cache memory space

  M0rpheus-0 asked Mar 24, 2025 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  0.8.2 version in docker

  burakaytan asked Mar 25, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Pipeline Parallelism Support

  egebeysel asked Mar 25, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Cant Run VLLM using https://github.com/vllm-project/vllm/blob/main/benchmarks/README.md

  SSWAMIN1SSS started Mar 25, 2025 in General

  

  

  0
• 8 You must be logged in to vote

  📣

  GitHub discussion is not used anymore, please use the forum for discussion.

  youkaichao announced Mar 20, 2025 in Announcements

  

  

  

  

  3
• 7 You must be logged in to vote

  🙏

  Extremely high CPU memory (RAM) usage when running vLLM on k8s

  BrianPark314 asked Nov 22, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 4 You must be logged in to vote

  🙏

  Generation speed suddenly became very slow, what might be the cause?

  zhuwenfei-wintech asked Sep 27, 2024 in Q&A · Unanswered

  

  

  1
• 2 You must be logged in to vote

  🙌

  CoreAgent - Simplest Agent Framework (based on vLLM API)

  ComposerKevin started Mar 20, 2025 in Show and tell

  

  

  0
• 1 You must be logged in to vote

  🙏

  Plugins and addons

  carelesswhisp asked Mar 20, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  XGrammar falling to Outlines

  michele-aindo asked Mar 20, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  GPU-parallel inference

  vadimcn asked Jan 19, 2025 in Q&A · Unanswered

  

  

  

  

  4
• 5 You must be logged in to vote

  🙏

  Which tests do I have to run before submitting a pool request for a new feature?

  Settheworldonfireiii asked Mar 19, 2025 in Q&A · Unanswered

  

  

  0
• 5 You must be logged in to vote

  💬

  Gemma3 supported release

  burakaytan started Mar 17, 2025 in General

  

  

  8
• 17 You must be logged in to vote

  🙏

  Is there a way to set max-concurrency

  tommasofavaron1 asked Jan 31, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  production metrics like gpu-utilization and gpu-memory-utilization

  geraldstanje started Mar 17, 2025 in General

  

  

  0
• 2 You must be logged in to vote

  💬

  from v0.7.3 to v0.7.4-pre, HUGE VRAM increase

  DefinitlyEvil started Mar 16, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  Export state/prefix-cache & reuse

  M0rpheus-0 asked Mar 16, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Single-Node Multi-GPU Inferencing For Large Models

  evetsagg started Mar 15, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  [NCCL Error] Tensor Parallelism with multi GPUs. Please

  jooe0824 asked Aug 13, 2024 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  The legacy LLMEngine in offline inference

  RookieChenTaoYu asked Mar 14, 2025 in Q&A · Unanswered

  

  

  0
• 0 You must be logged in to vote

  🙏

  Help serving vllm with kuberay

  torsteinelv asked Jan 11, 2025 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  🙏

  An error occurred when pip3 install vllm package.

  tangrenwang asked Mar 13, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Mqllmengine

  baturgezici started Mar 13, 2025 in General

  

  

  0
• 3 You must be logged in to vote

  🙏

  Can I run VLLM with 5090+5070Ti for Llama 70B Q4 (needs approximately 42 GB) inference? Or do I need identical GPUs?

  jayavanth asked Mar 12, 2025 in Q&A · Unanswered

  

  

  0