High-throughput LLM inference server with OpenAI-compatible API. Serves Llama, Mistral, Qwen, and 30+ model families with PagedAttention. pip install vllm.
Use it now
Keep it with up to 9 other tools you're setting up, then share them all as one link.
Install the CLI
Install it as its own instructions say.
Its page: docs.vllm.ai
Call the API
Auth None needed
Its page: docs.vllm.ai
Compare
Press Compare to add it to the tray at the bottom of the window, then pick up to 2 others anywhere on the site.
- Access
- APICLI
- Auth
- None
- Skills
- 3
Skills
OpenAI-Compatible Serving
Serve any HuggingFace model as an OpenAI-compatible API endpoint with full streaming and function calling.
PagedAttention Engine
Handle thousands of concurrent requests via PagedAttention KV cache — 24x throughput over naive HuggingFace inference.
Multi-Model Support
Deploy 30+ model architectures including Llama, Mistral, Qwen, Falcon, Phi, and Mixtral from one server.