# Deploy open-source LLMs

Deploy GLM 5.2, DeepSeek, Qwen 3.7, or any open-source LLM as a production API in minutes.

streamingtokens/sec **0**  
request  
_POST_ /v1/chat/completions · llama-3.1-8b-instruct

response  
Serverless GPUs change the economics of serving open-source models. Endpoints boot in seconds, scale with traffic, and go to zero when idle.

## Production LLM serving, without the infrastructure

Everything you need to run open-source LLMs at scale.

**Cold starts in seconds**  
Beam boots containers in seconds — even with big model weights loaded — so scaling up doesn't keep your users waiting.

**Elastic Scaling**  
Endpoints scale from zero to thousands of GPUs the moment traffic spikes, and back down when it stops.

**OpenAI-compatible out of the box**  
The managed vLLM integration exposes an OpenAI-compatible API so your existing SDKs, clients, and tooling work by swapping one base URL.

**Any model, any GPU**  
Serve Llama, Mistral, Qwen, or your own fine-tuned weights on 4090s through B300s. Switching hardware is a one-line code change.

**Weights cached in PVs**  
Mount a storage volume and load weights from disk instead of pulling from Hugging Face on every boot. Storage is included, free of charge.

**Per-second billing**  
Pay for the seconds your model is actually serving requests.

> “Beam's infrastructure makes productizing our AI straightforward. We can scale up or down when needed, test new GPU options, and keep our users happy.”  
> — Dave Carlton, Founder, Hooktheory

>99.99% production uptime  
100,000s of inference requests daily  
40% lower latency after a one-line GPU swap

## Deploy an LLM in three steps

**01**  
### Define your app in code  
Use the managed vLLM integration — or wrap any inference code in an endpoint decorator — and declare GPU, memory, and image in Python or TS.

**02**  
### Deploy with one command  
beam deploy ships it to serverless GPUs. No Dockerfiles, no Kubernetes, no YAML.

**03**  
### Call it from any OpenAI client  
Point your existing SDK at the endpoint URL. Auth, autoscaling, and telemetry are handled for you.

```bash
# 1. Deploy the server
$ beam deploy app.py:llama
=> Deployed 🎉
=> https://llama-3-1-8b-abc123.app.beam.cloud

# 2. Call it from any OpenAI client
from openai import OpenAI

client = OpenAI(
    base_url="https://llama-3-1-8b-abc123.app.beam.cloud/v1",
    api_key="YOUR_BEAM_TOKEN",
)

response = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

## Frequently asked questions

**Which models can I serve?**  
Any open-source or custom model — GLM 5.2, DeepSeek, Qwen, or your own fine-tuned weights. Use the vLLM integration for an OpenAI-compatible server, or wrap any inference code in an endpoint decorator.

**Do I pay while the endpoint is idle?**  
No. Endpoints scale to zero when idle and billing is per-second, so you only pay while requests are being processed. If you're latency-sensitive, you can keep containers warm and control exactly how long they stay up.

**Am I billed for cold starts?**  
No. Beam only charges for the time it takes to load your application code — never for server spin-up or container image pulls.

**Can I keep a server running 24/7?**  
Yes. By default apps spin down after each request, but you control the idle timeout with keep_warm_seconds — or set min_containers to keep capacity always on. Many teams with latency-sensitive workloads run their LLM servers around the clock on Beam.

**Is the API OpenAI-compatible?**  
Yes, when you use Beam's vLLM integration. Your deployed endpoint speaks the OpenAI API, so existing SDKs and client libraries work by changing only the base URL.

**Can I stream tokens back to my users?**  
Yes. The vLLM integration streams through the standard OpenAI API.

**How do I monitor my deployment?**  
The dashboard shows request volume, latency, container counts, and resource utilization per deployment, with streaming logs for every task. Deployments are versioned, so you can roll back to a previous version as needed.

**Can I run this in my own cloud?**  
Yes. Beam's engine is open source (Beta9), and you can self-host it on your own VPC or connect your own hardware — same SDK, same decorators, your infrastructure.