GPU Cloud Hosting for AI/ML: How to Choose and Deploy

TL;DR — GPU cloud hosting lets you run AI training and inference without buying hardware. The key decisions: which GPU (T4 for inference, A100 for training), how to price it (on-demand vs reserved), and where to run it. RaidFrame offers NVIDIA T4, L40S, and A100 instances with per-hour billing and no long-term commitment.

Why you need GPU cloud

Training a large language model on CPU takes weeks. On an A100, it takes hours. Running inference on a fine-tuned model without a GPU means 2-3 second response times. With a T4, it's 50-200ms.

If you're building anything with AI — image generation, embeddings, fine-tuning, real-time inference — you need GPUs. Buying them makes no sense for most teams. Renting them by the hour does.

Which GPU for which workload?

Rules of thumb

• Inference only (serving a trained model) → T4. Cheapest, fast enough for production.
• Fine-tuning (LoRA, QLoRA on 7B-13B models) → L40S. 48GB VRAM handles most fine-tuning jobs.
• Training from scratch (or full fine-tuning on 70B+) → A100. The only option for serious training.
• Not sure → Start with T4. Upgrade if you hit VRAM limits.

How to deploy a GPU workload on RaidFrame

Step 1: Create a Dockerfile

FROM nvidia/cuda:12.3.1-runtime-ubuntu22.04
 
RUN apt-get update && apt-get install -y python3 python3-pip
COPY requirements.txt .
RUN pip3 install -r requirements.txt
 
COPY . /app
WORKDIR /app
 
CMD ["python3", "serve.py"]

Step 2: Configure your service

services:
  inference:
    type: web
    build:
      dockerfile: Dockerfile
    port: 8080
    resources:
      gpu: t4
      cpu: 4
      memory: 16GB
    scaling:
      min: 1
      max: 5
      target_rps: 50

Step 3: Deploy

rf deploy

Detecting stack...     Python (PyTorch)
GPU:                   NVIDIA T4 (16GB VRAM)
Building...            ████████████████████ 100% (2m 12s)
Deploying...           ✓ 1 GPU instance healthy
 
✓ Live at https://inference-abc123.raidframe.app

Scaling GPU workloads

Auto-scaling on queue depth

For batch processing (image generation, video encoding):

services:
  gpu-worker:
    type: worker
    resources:
      gpu: l40s
    scaling:
      min: 0
      max: 10
      target_queue_depth: 5
      scale_to_zero:
        idle_timeout: 10m

Scale to zero when idle. Spin up GPU instances when jobs arrive. Scale down when the queue empties.

Auto-scaling on requests

For real-time inference APIs:

services:
  inference:
    type: web
    resources:
      gpu: t4
    scaling:
      min: 1
      max: 20
      target_rps: 100

Multi-GPU training

Distributed training across multiple GPUs:

services:
  trainer:
    type: worker
    resources:
      gpu: a100-80g
      gpu_count: 8
      cpu: 32
      memory: 256GB
    command: torchrun --nproc_per_node=8 train.py

Common patterns

Model serving with FastAPI

from fastapi import FastAPI
from transformers import pipeline
 
app = FastAPI()
model = pipeline("text-generation", model="meta-llama/Llama-2-7b", device="cuda")
 
@app.post("/generate")
async def generate(prompt: str, max_tokens: int = 100):
    result = model(prompt, max_new_tokens=max_tokens)
    return {"text": result[0]["generated_text"]}

Embedding API

from sentence_transformers import SentenceTransformer
 
model = SentenceTransformer("all-MiniLM-L6-v2", device="cuda")
 
@app.post("/embed")
async def embed(texts: list[str]):
    embeddings = model.encode(texts)
    return {"embeddings": embeddings.tolist()}

GPU pricing compared

RaidFrame pricing is competitive with specialists, without the AWS/GCP complexity. No VPC setup, no IAM roles, no quota requests.

FAQ

Do I need to install CUDA drivers?

No. Use NVIDIA's official CUDA Docker images as your base. RaidFrame instances have the GPU drivers pre-installed.

Can I reserve GPU capacity?

Yes. Reserved instances are available at a discount for 1-month and 3-month terms. Contact sales for pricing.

How fast are GPUs provisioned?

Most instances are available within 60 seconds. A100 80GB instances may take a few minutes during peak demand.

Can I use AMD GPUs?

NVIDIA GPUs only at this time. ROCm support is on the roadmap.

What about spot/preemptible instances?

Spot GPU instances are available at 50-70% discount. Workloads must be checkpoint-friendly — instances can be reclaimed with 30 seconds notice.

Related reading

• Auto-Scaling Cloud Infrastructure
• Deploy Any Docker App on RaidFrame
• Cloud Cost Optimization