Cloud GPU rental
AWS EC2, Azure GPU VMs, Oracle Cloud, CoreWeave and Nebius are infrastructure candidates. GPU availability and networking differ by region and offering.
GPU procurement options
Procurement comparison
| Compare | Own hardwareBuy GPUs | Cloud GPUsRent capacity | Serverless GPUsBring your engine | Managed deploymentUse a supported engine | Model APIBuy model usage |
|---|---|---|---|---|---|
| Cost & commitment | |||||
| Upfront hardware purchase | Required | None | None | None | None |
| Billing basis | Hardware + operations | Allocated GPU time | Metered compute | Capacity + fees | Tokens / requests |
| Cost while idle | Ownership costs remain | While allocated | Warm capacity / storage | While provisioned | Usually no token charge |
| Features & control | |||||
| Choose GPU hardware | You configure | Provider catalog | Provider catalog | Depends | Unavailable |
| Bring model weights | You configure | You configure | You configure | Depends | Depends |
| Customize vLLM / SGLang | You configure | You configure | You configure | Depends | Unavailable |
| Private networking | You configure | Depends | Depends | Depends | Depends |
| Dedicated GPU capacity | Included | Depends | Depends | Depends | Depends |
| Operations & availability | |||||
| Engine maintenance | Your team | Your team | Your team | Provider¹ | Provider |
| Automatic scaling | You configure | You configure | Included² | Depends | Provider-managed |
| Cold-start exposure | At startup | At startup | After scale-to-zero | If scaled to zero | Provider-dependent |
| Serving recovery owner | Your team | Your team | Shared³ | Provider¹ | Provider |
| Redundancy | You configure | You configure | Depends | Depends | Provider-dependent |
| Contractual uptime SLA | Your service design | Check contract | Check contract | Check contract | Check contract |
¹ Supported managed engines; custom containers can shift maintenance back to you. ² You set scaling limits and concurrency. ³ The platform recovers infrastructure; you fix engine and application failures.
No route guarantees zero downtime. Cold starts are startup delays, not outages. An SLA is a contractual commitment, not a record of measured availability. See provider documentation for specific offerings.
Cost vs. utilization
Illustrative scenario—not market prices. Assumptions below.
Scroll the chart on smaller screens. Exact values below.
- Own hardware
- Cloud GPUs
- Serverless GPUs
- Managed deployment
- Model API
Scenario assumptions & exact values
One model, one fixed input/output mix and the same useful throughput across every route. At 100% utilization, the workload runs for 730 hours and produces 730 million output tokens. Real throughput, quality and latency must be measured before comparing quotes.
- Owned: $1,200/month for amortized hardware and fixed operating costs, plus $0.20 per busy hour.
- Cloud rental: one always-allocated equivalent GPU at $2/hour × 730 hours.
- Serverless: $4 per busy equivalent GPU-hour, scaling to zero with no warm idle time.
- Managed deployment: a flat $1,800/month for equivalent provisioned capacity.
- Model API: $5 per million output tokens, with input charges assumed included for this fixed mix.
All rates are invented to explain the cost shapes. These are partial costs: staff, networking, storage, taxes, redundancy and startup overhead are excluded unless stated. The chart is not a total-cost forecast or a vendor ranking.
| Route | 0% | 25% | 50% | 75% | 100% |
|---|---|---|---|---|---|
| Own hardware | $1,200.00 | $1,236.50 | $1,273.00 | $1,309.50 | $1,346.00 |
| Cloud GPUs | $1,460.00 | $1,460.00 | $1,460.00 | $1,460.00 | $1,460.00 |
| Serverless GPUs | $0.00 | $730.00 | $1,460.00 | $2,190.00 | $2,920.00 |
| Managed deployment | $1,800.00 | $1,800.00 | $1,800.00 | $1,800.00 | $1,800.00 |
| Model API | $0.00 | $912.50 | $1,825.00 | $2,737.50 | $3,650.00 |
Provider examples
Responsibilities vary by deployment mode.
Modal
Shared Endpoints provide managed model access. Dedicated Endpoints start with optimized serving configurations. With custom deployments, you configure vLLM or SGLang and Modal handles infrastructure.
Read Modal’s deployment options ↗Baseten & Fireworks
Baseten provides built-in engines and custom containers; with a custom server, you own the serving stack. Fireworks operates its managed inference stack, including its bring-your-own-cloud offering.
Baseten options ↗Fireworks options ↗Cloud model services
Services such as Amazon Bedrock provide model APIs with private connectivity options. Managed hosting such as SageMaker can also run your own container, where you supply the inference code.
Bedrock private access ↗SageMaker custom containers ↗Private deployment
Check network access, isolation and data location separately.
Private network access
A private endpoint controls how you connect. It does not by itself mean the model runs in your account or on dedicated hardware.
Dedicated capacity
Ask what is dedicated: a model deployment, GPUs or physical servers. Dedicated capacity does not automatically mean customer-owned infrastructure.
Your cloud account
A bring-your-own-cloud service may run in your environment while a provider operates the engine. Confirm administrative access and what leaves the environment.
Approved data location
Confirm the processing region and where prompts, outputs, logs and backups are stored. Approval applies to the specific service and configuration.
Provider examples are starting points, not endorsements. Features, regions, capacity and commercial terms need confirmation for your deployment. Sources reviewed November 11, 2026.