Ollama infrastructure

Best Ollama Hosting: Managed GPU, VPS and Self-Hosted Options

Ollama can run as a local model runtime on your own hardware and can also access Ollama-hosted cloud models through the same CLI and API. The right deployment depends on model size, uptime requirements, network access needs, data privacy constraints, and budget.

GPU cloud sizing workflow from model artifact and quantization through VRAM and context capacity to total cost including compute, storage and network egress — GPU sizing and cost planning workflow. No vendor performance ranking or real-time pricing is implied.

Ollama deployment models compared

Each deployment model has different implications for memory, persistence, networking, and operational responsibility. Choose based on the workload, not the simplest option.

Deployment	Best fit	Memory	Persistence	Network	Tradeoff
Local workstation (Mac / PC)	Private local inference, development, prototyping	Limited to installed RAM/VRAM	Runs while the machine is on and Ollama is started	localhost only by default	Uptime depends on the workstation. No remote access without additional networking.
Dedicated local machine (e.g. Mac mini, Linux box)	Always-on local inference, LAN model server	Can be sized for the workload at purchase time	Survives reboots if Ollama is configured as a system service	LAN access; internet exposure requires reverse proxy / VPN	You own all maintenance, power, cooling, backups, and recovery.
CPU VPS	Small quantized models, agent orchestration, light embedding workloads	Typically 4–64 GB system RAM; no GPU VRAM	Persistent remote server with provider-managed uptime	Internet-accessible; firewalling and auth required	Large-model inference will be slow. CPU-only means no GPU acceleration.
GPU VPS / cloud GPU	Larger models, faster inference, remote GPU-accelerated workloads	24–141+ GB VRAM depending on instance type	Persistent if using dedicated instances; ephemeral on spot/serverless	Internet-accessible; firewalling and auth required	Higher hourly cost. You manage OS, drivers, runtime, and recovery.
Ollama cloud models (preview)	Models too large for local hardware; quick access without GPU provisioning	Runs on Ollama-hosted infrastructure; no local GPU required	Managed by Ollama; requires internet and Ollama v0.12+ with sign-in	Uses the standard Ollama API through localhost; inference runs remotely	Data leaves your machine. Pricing and availability may change during preview.

Ollama memory sizing

Ollama loads model weights into GPU memory (or system RAM on CPU-only setups). The Ollama FAQ documents that required RAM scales with the combination of model size, OLLAMA_NUM_PARALLEL (concurrent request slots, default 1), and OLLAMA_CONTEXT_LENGTH (default 4096 tokens). Increasing either parallel slots or context length increases the memory allocation beyond the model weights alone.

Model weights are the baseline. Use the VRAM calculator to estimate the weight footprint at different quantizations.
Context and concurrency add to the baseline. Longer context windows and more parallel slots each require additional memory for KV cache allocation.
Multiple loaded models share GPU memory. Ollama defaults to loading up to 3× the number of GPUs (or 3 on CPU). Control this with OLLAMA_MAX_LOADED_MODELS.
OS and application reserve. Leave headroom for the operating system, Ollama runtime, and other services running on the same machine.

TEST PENDING: Actual idle RAM, loaded RAM, model load time, and per-token throughput for specific models on specific hardware have not been measured by Agent Infra Guide. These are future Evidence Layer test specifications.

Ollama model storage

Ollama stores downloaded models locally. Default locations:

macOS: ~/.ollama/models
Linux: /usr/share/ollama/.ollama/models
Windows: C:\Users\%username%\.ollama\models

Override with the OLLAMA_MODELS environment variable. On a VPS or server, ensure the storage volume has enough space for the models you intend to keep — quantized large models can be 10–50+ GB each. On Linux, the ollama user needs read/write access to the custom directory.

Ollama service persistence and recovery

For a server that should survive reboots and process failures:

System service: Run Ollama as a systemd service (Linux) or launchd agent (macOS) so it restarts automatically after reboot or crash.
Keep-alive behavior: By default, Ollama unloads a model from memory after 5 minutes of inactivity. Set OLLAMA_KEEP_ALIVE to a longer duration (e.g. "24h") or a negative value to keep models loaded indefinitely.
After unexpected termination: Ollama reloads the model on the next request. The reload time depends on model size and storage speed. There is no persistent inference state to recover — in-flight requests are lost.
Model version management: ollama pull updates to the latest version of a model tag. Pin a specific model digest if you need reproducible inference across updates.

Exposing Ollama to the network

By default Ollama binds to 127.0.0.1:11434 and is only accessible from the local machine. To serve other machines:

LAN access: Set OLLAMA_HOST=0.0.0.0:11434 to bind to all interfaces. This makes the API available to any machine on the local network. Use firewall rules to restrict access to trusted hosts.
Internet-exposed server: Do not expose an unrestricted Ollama API directly to the public internet. Use a reverse proxy (nginx, Caddy, Traefik) with authentication, TLS, and rate limiting. Consider a VPN (WireGuard, Tailscale) for authenticated private access without public exposure.
Cross-origin requests: Set OLLAMA_ORIGINS to allow specific origins if web applications need direct API access.

Ollama cloud models (preview)

As of Ollama v0.12, Ollama supports cloud models that run on Ollama-hosted infrastructure while using the same CLI and API interface. This means an application can use the Ollama API without changes while selected models execute remotely. Access requires signing in with ollama signin.

Currently available cloud models include large models like deepseek-v3.1:671b-cloud and qwen3-coder:480b-cloud that would be impractical to run on most local hardware. This is a preview feature — available models, pricing, and data-handling policies may change.

Architecture difference: Ollama cloud models are not the same as self-hosting Ollama on a VPS or GPU cloud. With cloud models, inference runs on Ollama's servers and your data leaves your machine. With self-hosted Ollama, you control the hardware, data stays on your infrastructure, and you manage the full operational stack.

When to host Ollama on a remote server

You want one model server shared by multiple agents or applications.
The inference hardware should remain online independently of your laptop or workstation.
You want centralized model downloads and runtime configuration.
Your agent and model server should scale independently.
You need inference accessible from multiple locations or networks.

Sources

Ollama FAQ — configuration and runtime behavior— Memory scaling, environment variables, model storage, context, concurrency, and keep-alive behavior
Ollama cloud models announcement— Cloud-model behavior, available models, and sign-in requirement

Estimate model memory →Calculate hosting cost →Compare GPU cloud providers →Build your own AI server →Local AI vs cloud GPU →RAM requirements guide →LM Studio vs Ollama →

Agent Infrastructure Recommendation Kit

Planning Ollama infrastructure for production use?

The Agent Infrastructure Recommendation Kit helps you define requirements, size memory and storage, compare local, VPS, and GPU deployment options, and document a defensible infrastructure recommendation — before you commit to hardware or a provider.

Get the Recommendation Kit — $19