Why self-host at all when APIs are this cheap
Token prices keep falling, and for most projects a hosted API is the sane choice. So why are people still firing up their own inference boxes? Three reasons keep coming up: the data cannot leave the building (legal, medical, internal code), the workload is bursty enough that per-token billing stings, or the model needs custom quantization, adapters and tooling that a managed endpoint will not run for you.
The open-model side has also matured. Llama, Qwen and DeepSeek-R1 distills cover most of what people actually ask chatbots to do, and a well-quantized 32B model answers like a paid mid-tier API did two years ago. The blocker is rarely software. It is picking the right amount of VRAM without overpaying for it. That is the part this guide works through, with prices checked on the providers' own pages in November 2026.
The VRAM math, without the hand-waving
Two numbers decide everything: weight size and the KV cache. Weights in FP16 take roughly 2 GB per billion parameters. A 4-bit quant (Q4-class, the everyday choice) lands near 0.55 to 0.6 GB per billion, plus a slice of working memory. The KV cache grows with context length and stacks on top; on a card that "just fits" the weights, a long conversation will still throw OOM.
- 8B class, Q4: about 5 to 6 GB of weights. An 8 GB card runs it, 12 GB runs it comfortably with real context.
- 14B class, Q4: around 9 to 10 GB. 12 GB is workable, 16 GB is relaxed.
- 32B class, Q4: roughly 19 to 21 GB before cache. This is the 24 GB card's job, and the reason RTX 4090 and RTX 3090 machines are still rented around the clock.
- 70B class, Q4: 40 GB and change. One 48 GB card (A40, L40S, RTX PRO 6000) or two 24 GB cards in tensor parallel. At 8-bit it wants 70 GB, which is A100/H100 80 GB territory.
- Large MoE models: 100B+ parameters, even quantized, means multi-GPU or an H200-class card with 141 GB. Budget accordingly or distill first.
Rule of thumb worth memorizing: match the card to the quantized weights plus 20 percent for cache and overhead. Renting "a bit more VRAM" costs far less than a crashed batch job at 3am.
Hourly pods: RunPod and Vast.ai, priced mid-November
For bursty work, per-hour GPU pods are hard to beat. RunPod splits its fleet into Community Cloud (cheaper, hosted by third parties) and Secure Cloud (RunPod-managed). Vast.ai is a marketplace over individual hosts, which pushes prices down further and reliability questions up. Rates below are from the providers' pricing data as of November 18-19, 2026, US dollars per GPU-hour:
- RunPod Community, RTX 4090 (24 GB): $0.34/hr, about $248 for a 730-hour month. Secure Cloud doubles it to $0.89/hr.
- Vast.ai on-demand, RTX 4090: $0.37/hr, with interruptible spot at $0.27/hr (roughly $197/month if you can survive preemption).
- Vast.ai, A100 SXM 80 GB: $0.44/hr on-demand. RunPod Community lists the same GPU at $1.39/hr.
- RunPod Community, H100 SXM: $2.69/hr; Vast.ai marketplace had H100 SXM at $1.73/hr.
- RunPod Community, RTX PRO 6000 (96 GB): $1.69/hr for the current king of single-card VRAM.
Notice the 70B trap: two RTX 4090s on Vast.ai cost $0.74/hr combined, while one A100 80 GB is $0.44/hr. Fewer, bigger cards win for models that fit. Marketplace spot pricing covers 28 GPU models on Vast.ai and none on RunPod; if your job checkpoints well, spot is the cheapest compute left in the market.
Serverless: pay for busy seconds
RunPod Serverless bills flex workers from $0.58/hr for 16 GB-class GPUs while requests are actually flowing, and nothing when idle. An RTX 4090 PRO worker is $1.34/hr. The arithmetic favors serverless hard at low utilization: 120 busy hours a month on the 4090 worker is $160.80, and a 16 GB flex worker doing the same 120 hours is $69.60. The same card rented as an always-on pod is $248 and up. Past roughly 200 to 300 busy hours, the pod wins; below that, serverless is not even close.
The catch: cold starts add latency, and you are managing containers, not SSH-ing into a box. For a chat product with spiky traffic, that trade is usually fine. For agent pipelines that hold long contexts alive, keep a pod.
Flat monthly GPU servers: Hetzner, Contabo, ServaRica
When inference runs 24/7, hourly billing becomes the expensive option. Three fixed-price routes checked on their official pages in November 2026:
Hetzner GEX45
A dedicated root server with an NVIDIA RTX PRO 4000 Blackwell SFF GPU, 64 GB DDR4 and 2x512 GB NVMe, at 214 EUR per month excluding VAT plus a 209 EUR one-time setup fee. First-year effective cost works out to about 231 EUR per month, or roughly 0.32 EUR per hour. The card's 20 GB VRAM puts 8B and 14B models at ease and 32B quantized within reach, but 70B is not its job. For European data residency at a fixed price, nothing else is close.
Contabo GPU VPS
Launched in September 2026: a dedicated RTX PRO 6000 with 96 GB of GDDR7, 18 vCPUs, 96 GB RAM and 900 GB NVMe at $1,199 per month excluding VAT, with 15 percent off 12-month terms (about $1,019) and 20 percent off 24-month terms ($959.20). That is a serious single card for 70B quantized with room for cache, cheaper per month than 730 hours of RunPod's community RTX PRO 6000 at $1.69/hr ($1,233.70). The flip side: zero hourly flexibility, and 24-month commitments on hardware that depreciates fast.
ServaRica Montreal
Announced October 23, 2026: KVM with full PCI passthrough (no shared vGPU games), Tesla P40 24 GB at $139/month with 10 dedicated cores, 40 GB RAM and 500 GB NVMe, or Tesla V100 32 GB HBM2 at $179/month. Annual pricing drops the P40 to $1,529 a year, effectively $127.42 a month, which is a floor of about $0.19 per hour for a whole dedicated GPU. Up to three GPUs per VM, 10 Gbps networking with 100 TB as an option. The P40 is old silicon with no tensor cores worth mentioning, fine for llama.cpp-class inference, wrong for serious training.
The crossover is easy to compute: any flat GPU server priced under 730 hours times the hourly pod rate wins for always-on workloads. ServaRica's P40 at $0.19/hr and Hetzner at 0.32 EUR/hr both clear that bar against $0.34-plus pods; Contabo's RTX PRO 6000 only wins against the same card billed hourly elsewhere.
What the sticker price hides
- Tax: a chunk of these prices exclude VAT, which moves the real bill 19 to 25 percent in the EU.
- Marketplace neighbors: Vast.ai hosts are individuals; a "4090" can share power and cooling with a mining rig. Check the host's reliability score before committing to long runs.
- Preemption: spot instances vanish when a higher bidder shows up. Fine for checkpointed fine-tunes, ruinous for a live product.
- Multi-GPU VRAM does not pool: 2x24 GB is 48 GB only for frameworks that tensor-parallel properly. The listing that shouts "96 GB total" across four cards is not one big card.
- Setup fees and commitments: Hetzner's 209 EUR setup and Contabo's 12/24-month discounts only pay off if the box actually runs that long. Hardware refresh cycles in this market are short.
A short decision path
- Under ~100 busy hours a month, small models: serverless. The $69.60/month math above is hard to argue with.
- Bursty fine-tuning and experiments: hourly pods, spot where checkpoints allow. Vast.ai for price, RunPod Secure when the data matters.
- 8B to 32B models, always-on, Europe: Hetzner GEX45 at 214 EUR, if 20 GB VRAM covers your quant.
- 70B quantized, always-on, any region: Contabo's flat $1,199 RTX PRO 6000 or an 80 GB A100 on Vast.ai at $0.44/hr ($321.20/month), whichever your cash flow prefers.
- Rock-bottom 24/7 inference for Qwen-32B-class quants: ServaRica's P40 at an effective $127 a month, with eyes open about its age.
Prices above are a dated snapshot, not a checkout quote. Verify region, tax and term on the order page before you commit; in this market, six weeks is a long time.