Best GPU Servers for AI Inference
Inference workloads need high throughput per dollar, not raw training speed. L40S, L4, and A10 hit the sweet spot between VRAM, compute, and cost. Compare live pricing across all providers.
87
Options
$0.34
From /hr
16
Providers
6
GPU types
What to look for
- →L40S offers 48 GB GDDR6 with Ada Lovelace architecture — 2× the inference throughput of A100 per dollar
- →L4 is purpose-built for inference: low power, 24 GB VRAM, runs 7B–13B models efficiently
- →A10 / A10G are workhorses for high-concurrency serving at a fraction of H100 cost
- →RTX 4090 is viable for small models and prototyping at consumer pricing
87 results
Density:
| Provider | GPU | GPUs↕ | VRAM↕ | Cores↕ | RAM↕ | $/mo↑ | $/hr↕ | $/GB·hr | Location | |
|---|---|---|---|---|---|---|---|---|---|---|
| RTX 4090 | 1× | 24 GB | — | — | $248/mo | $0.34 | $0.014 | Global | View | |
| L4 | 1× | 24 GB | — | — | $321/mo | $0.44 | $0.018 | Global | View | |
| L4 | 1× | 24 GB | 8 | 60 GB | $358/mo | $0.49 | $0.020 | 🇺🇸US | View | |
| RTX 4090 | 1× | 24 GB | 8 | 48 GB | $365/mo | $0.50 | $0.021 | 🇺🇸US | View | |
| A10 | 1× | 24 GB | 30 | 200 GB | $438/mo | $0.60 | $0.025 | 🇺🇸US | View | |
| RTX 4090 | 1× | 24 GB | 12 | 70 GB | $467/mo | $0.64 | $0.027 | 🇪🇺EU | View | |
| L40 | 1× | 48 GB | — | — | $504/mo | $0.69 | $0.014 | Global | View | |
| L4 | 1× | 24 GB | 8 | 48 GB | €548/mo | €0.75 | €0.031 | 🇫🇷EU-FR | View | |
| A100 PCIe | 1× | 80 GB | 4 | 24 GB | $569/mo | $0.78 | $0.0097 | 🇺🇸US | View | |
| L40S | 1× | 48 GB | — | — | $577/mo | $0.79 | $0.016 | Global | View | |
| RTX 4090 | 1× | 24 GB | 8 | 48 GB | $584/mo | $0.80 | $0.033 | 🇪🇺EU | View | |
| L40 | 1× | 48 GB | 28 | 58 GB | $767/mo | $1.05 | $0.022 | 🇪🇺EU | View | |
| A100 SXM4 | 1× | 40 GB | 30 | 200 GB | $803/mo | $1.10 | $0.028 | 🇺🇸US | View | |
| A100 | 1× | 80 GB | — | — | $869/mo | $1.19 | $0.015 | Global | View | |
| L40S | 1× | 48 GB | 16 | 100 GB | $913/mo | $1.25 | $0.026 | 🇺🇸US | View | |
| L40S | 1× | 48 GB | 16 | 120 GB | $942/mo | $1.29 | $0.027 | 🇺🇸US | View | |
| L40 | 1× | 48 GB | 12 | 96 GB | $942/mo | $1.29 | $0.027 | 🇺🇸US | View | |
| A100 SXM4 | 1× | 80 GB | 30 | 200 GB | $942/mo | $1.29 | $0.016 | 🇺🇸US | View | |
| A100 | 1× | 40 GB | 16 | 117 GB | $986/mo | $1.35 | $0.034 | 🇺🇸US | View | |
| A100 | 1× | 40 GB | 16 | 120 GB | $986/mo | $1.35 | $0.034 | 🇺🇸US | View | |
| A100 | 1× | 80 GB | — | — | $1015/mo | $1.39 | $0.017 | Global | View | |
| A100 PCIe | 1× | 80 GB | 28 | 120 GB | $1073/mo | $1.47 | $0.018 | 🇪🇺EU | View | |
| L40S | 1× | 48 GB | 16 | 120 GB | $1088/mo | $1.49 | $0.031 | 🇺🇸US | View | |
| L40S | 1× | 48 GB | 12 | 96 GB | €1088/mo | €1.49 | €0.031 | 🇫🇷EU-FR | View | |
| L4 | 2× | 24 GB | 16 | 96 GB | €1095/mo | €1.50 | €0.031 | 🇫🇷EU-FR | View | |
| L40S | 1× | 48 GB | 8 | 64 GB | $1146/mo | $1.57 | $0.033 | 🇺🇸US | View | |
| A100 SXM4 | 1× | 80 GB | 31 | 240 GB | $1146/mo | $1.57 | $0.020 | 🇪🇺EU | View | |
| L4 | 2× | 24 GB | — | 192 GB | $1204/mo | — | — | 🇺🇸US | View | |
| L40S | 1× | 48 GB | 16 | 120 GB | $1234/mo | $1.69 | $0.035 | 🇺🇸US | View | |
| L4 | 2× | 24 GB | — | 192 GB | $1241/mo | — | — | 🇺🇸US | View | |
| A100 PCIe | 1× | 40 GB | 8 | 45 GB | $1248/mo | $1.71 | $0.043 | 🇺🇸US | View | |
| L4 | 2× | 24 GB | — | 192 GB | $1279/mo | — | — | 🇺🇸US | View | |
| A100 | 1× | 80 GB | 16 | 117 GB | $1307/mo | $1.79 | $0.022 | 🇺🇸US | View | |
| A100 PCIe | 1× | 40 GB | 24 | 192 GB | $1307/mo | $1.79 | $0.045 | 🇺🇸US | View | |
| A100 PCIe | 1× | 80 GB | 16 | 120 GB | $1307/mo | $1.79 | $0.022 | 🇪🇺EU | View | |
| A100 SXM4 | 1× | 80 GB | 12 | 120 GB | $1307/mo | $1.79 | $0.022 | 🇺🇸US | View | |
| A100 | 1× | 80 GB | 20 | 160 GB | $1380/mo | $1.89 | $0.024 | 🇺🇸US | View | |
| L4 | 4× | 24 GB | 32 | 240 GB | $1431/mo | $1.96 | $0.020 | 🇺🇸US | View | |
| A100 PCIe | 1× | 80 GB | 24 | 192 GB | $1453/mo | $1.99 | $0.025 | 🇺🇸US | View | |
| A100 SXM4 | 1× | 80 GB | 20 | 180 GB | $1453/mo | $1.99 | $0.025 | 🇺🇸US | View | |
| RTX 4090 | 4× | 24 GB | 32 | 192 GB | $1460/mo | $2.00 | $0.021 | 🇺🇸US | View | |
| A100 SXM4 | 1× | 80 GB | 30 | 200 GB | $1504/mo | $2.06 | $0.026 | 🇺🇸US | View | |
| A100 PCIe | 1× | 80 GB | 12 | 90 GB | $1679/mo | $2.30 | $0.029 | 🇺🇸US | View | |
| A100 | 1× | 80 GB | 22 | 184 GB | $1869/mo | $2.56 | $0.032 | 🇫🇮FI-01 | View | |
| A100 SXM4 | 2× | 80 GB | 60 | 400 GB | $1883/mo | $2.58 | $0.016 | 🇺🇸US | View | |
| L40S | 2× | 48 GB | 24 | 192 GB | €2175/mo | €2.98 | €0.031 | 🇫🇷EU-FR | View | |
| A100 | 1× | 80 GB | 32 | 192 GB | $2183/mo | $2.99 | $0.037 | 🇪🇺EU | View | |
| L4 | 4× | 24 GB | 32 | 192 GB | €2190/mo | €3.00 | €0.031 | 🇫🇷EU-FR | View | |
| A100 SXM4 | 1× | 80 GB | 12 | 90 GB | $2321/mo | $3.18 | $0.040 | 🇺🇸US | View | |
| A100 SXM4 | 2× | 80 GB | 24 | 240 GB | $2613/mo | $3.58 | $0.022 | 🇺🇸US | View | |
| RTX 4090 | 4× | 24 GB | 32 | 256 GB | $2920/mo | $4.00 | $0.042 | 🇺🇸US | View | |
| L40 | 4× | 48 GB | 112 | 232 GB | $3066/mo | $4.20 | $0.022 | 🇪🇺EU | View | |
| A100 PCIe | 4× | 40 GB | 64 | 512 GB | $3270/mo | $4.48 | $0.028 | 🇺🇸US | View | |
| L40S | 2× | 48 GB | — | 384 GB | $3505/mo | — | — | 🇺🇸US | View | |
| L40S | 2× | 48 GB | — | 384 GB | $3505/mo | — | — | 🇺🇸US | View | |
| L40S | 2× | 48 GB | — | 384 GB | $3505/mo | — | — | 🇺🇸US | View | |
| A100 | 2× | 80 GB | 44 | 368 GB | $3738/mo | $5.12 | $0.032 | 🇫🇮FI-01 | View | |
| RTX 4090 | 8× | 24 GB | 120 | 706 GB | $3738/mo | $5.12 | $0.027 | 🇪🇺EU | View | |
| L40 | 4× | 48 GB | 48 | 384 GB | $3767/mo | $5.16 | $0.027 | 🇺🇸US | View | |
| A100 SXM4 | 4× | 80 GB | 120 | 800 GB | $3767/mo | $5.16 | $0.016 | 🇺🇸US | View | |
| L40S | 4× | 48 GB | 64 | 512 GB | $4088/mo | $5.60 | $0.029 | 🇺🇸US | View | |
| L40S | 4× | 48 GB | 48 | 384 GB | €4351/mo | €5.96 | €0.031 | 🇫🇷EU-FR | View | |
| L4 | 8× | 24 GB | 64 | 384 GB | €4380/mo | €6.00 | €0.031 | 🇫🇷EU-FR | View | |
| RTX 4090 | 8× | 24 GB | 64 | 384 GB | $4672/mo | $6.40 | $0.033 | 🇪🇺EU | View | |
| A100 PCIe | 4× | 80 GB | 64 | 480 GB | $5227/mo | $7.16 | $0.022 | 🇪🇺EU | View | |
| A100 SXM4 | 4× | 80 GB | 48 | 480 GB | $5227/mo | $7.16 | $0.022 | 🇺🇸US | View | |
| RTX 4090 | 8× | 24 GB | 64 | 512 GB | $5840/mo | $8.00 | $0.042 | 🇺🇸US | View | |
| L40 | 8× | 48 GB | 252 | 464 GB | $6132/mo | $8.40 | $0.022 | 🇪🇺EU | View | |
| L40S | 8× | 48 GB | 128 | 800 GB | $7300/mo | $10.00 | $0.026 | 🇺🇸US | View | |
| A100 | 4× | 80 GB | 88 | 736 GB | $7475/mo | $10.24 | $0.032 | 🇫🇮FI-01 | View | |
| A100 SXM4 | 8× | 80 GB | 240 | 1800 GB | $7534/mo | $10.32 | $0.016 | 🇺🇸US | View | |
| A100 | 8× | 40 GB | 128 | 936 GB | $7884/mo | $10.80 | $0.034 | 🇪🇺EU | View | |
| L40S | 8× | 48 GB | 128 | 1024 GB | $8176/mo | $11.20 | $0.029 | 🇺🇸US | View | |
| A100 PCIe | 8× | 80 GB | 128 | 1024 GB | $8176/mo | $11.20 | $0.017 | 🇺🇸US | View | |
| A100 PCIe | 8× | 80 GB | 252 | 1440 GB | $8585/mo | $11.76 | $0.018 | 🇪🇺EU | View | |
| L40S | 8× | 48 GB | 128 | 960 GB | $8702/mo | $11.92 | $0.031 | 🇺🇸US | View | |
| L40S | 8× | 48 GB | 96 | 768 GB | €8702/mo | €11.92 | €0.031 | 🇫🇷EU-FR | View | |
| A100 SXM4 | 8× | 80 GB | 252 | 1920 GB | $9169/mo | $12.56 | $0.020 | 🇪🇺EU | View | |
| L40S | 8× | 48 GB | 128 | 960 GB | $9870/mo | $13.52 | $0.035 | 🇺🇸US | View | |
| A100 | 8× | 80 GB | 128 | 936 GB | $10,453.6/mo | $14.32 | $0.022 | 🇺🇸US | View | |
| A100 SXM4 | 8× | 80 GB | 96 | 960 GB | $10,453.6/mo | $14.32 | $0.022 | 🇺🇸US | View | |
| A100 PCIe | 8× | 80 GB | 32 | 192 GB | $10,453.6/mo | $14.32 | $0.022 | 🇺🇸US | View | |
| A100 | 8× | 80 GB | 160 | 1280 GB | $11,037.6/mo | $15.12 | $0.024 | 🇺🇸US | View | |
| A100 SXM4 | 8× | 80 GB | 160 | 1440 GB | $11,621.6/mo | $15.92 | $0.025 | 🇺🇸US | View | |
| A100 SXM4 | 8× | 80 GB | 240 | 1600 GB | $12,030.4/mo | $16.48 | $0.026 | 🇺🇸US | View | |
| A100 PCIe | 8× | 80 GB | 96 | 720 GB | $13,432/mo | $18.40 | $0.029 | 🇺🇸US | View | |
| A100 | 8× | 80 GB | 176 | 1472 GB | $14,950.4/mo | $20.48 | $0.032 | 🇫🇮FI-01 | View |