Platform
GA
Open source
vLLM
High-throughput open-model serving in production
Release2023-06
Context—
Max output—
ParametersUndisclosed
Local / VRAM16–80 GB
LicenseOpen source
Overview
vLLM is a model in the Platform category from vLLM Project. It was released in 2023-06. The vendor is based in US. Best for: High-throughput open-model serving in production.
Strengths
- PagedAttention
- Continuous batching
- OpenAI-compatible
Limits
- Requires a GPU
- Setup knowledge
Specifications
| Vendor | vLLM Project |
|---|---|
| Category | Platform |
| Release | 2023-06 |
| Status | GA |
| License | Open source |
| License name | Apache-2.0 |
| Commercial use | Allowed |
| Input modalities | Text, Image |
| Output modalities | Text |
| Prompt types | Text |
| Negative prompts | No |
| Max outputs | 1 |
| Local deployment | Yes |
| VRAM | 16–80 GB |
| System RAM | 32–256 GB |
| API access | Yes |
| Pricing | Free |
| Confidence | High |
| Vendor country | US |
| Vendor type | Platform |
Pricing & hardware
Free
This model can be self-hosted or run locally. Approximate GPU memory: 16–80 GB. Approximate system RAM: 32–256 GB. API access is available.