Llama 3.2 infrastructure

Llama 3.2 Hardware Requirements: RAM and VRAM for 1B and 3B

Llama 3.2's text models are small enough to run on almost any recent laptop, but their 128K context window is not small. At long context the KV cache, not the weights, decides the hardware. This page works through AIG's three layers — parametric estimate → actual artifact fit → workload planning — for the 1B and 3B models.

Sizing framework: three layers

1. Parametric estimate: parameter count × bits per weight ÷ 8. For models this small the result is under 1.6 GB at 4-bit — rarely the binding constraint.
2. Actual model artifact fit: the file you will load. Note that Ollama's two default Llama 3.2 tags use different quantizations (Layer 2).
3. Conservative workload planning: artifact + KV cache for your configured context + runtime overhead + OS reserve + one cache per concurrent request. For Llama 3.2 this layer dominates.

Layer 1: Parametric weight estimates

CALCULATED. Parameter count × bit width ÷ 8. The 11B and 90B Llama 3.2 Vision models are separate multimodal models and are not covered here.

Layer 2: Actual model artifact sizes

The two default Ollama tags are not quantized the same way. llama3.2:1b ships as Q8_0 — 8-bit, which is why the 1B file is 1.3 GB rather than the ~0.6 GB 4-bit floor suggests. llama3.2 (3B) ships as Q4_K_M at 2.0 GB, about 5 bits per weight once quantization scales are included. If you are comparing the two, compare like for like.

DOCUMENTED (sizes and quantizations) and CALCULATED (bits per weight = size × 8 ÷ parameter count). Other quantizations of both models are listed on Ollama's tags page.

Layer 3: Workload planning — the KV cache is the real requirement

Both models use grouped-query attention with 8 key/value heads. The 1B has 16 layers with a head dimension of 64; the 3B has 28 layers with a head dimension of 128. At the default 16-bit cache precision that is 2 (K and V) × layers × 8 × head dimension × 2 bytes per token: 32 KiB for 1B and 112 KiB for 3B.

CALCULATED from config.json, using the Ollama artifact sizes from Layer 2. Runtime allocations beyond the KV cache are covered by the headroom column, not modelled separately.

• The default context depends on your hardware: Ollama's context-length documentation picks it from available VRAM — 4K below 24 GiB, 32K from 24 to 48 GiB, and 256K at 48 GiB and above (its FAQ still says 4,096). Ollama caps the requested context at the model's supported limit, so a 48 GiB+ GPU's 256K default is capped at Llama 3.2's 128K window. You can set OLLAMA_CONTEXT_LENGTH or num_ctx explicitly for your workload; check the effective context and GPU offload with ollama ps and plan from the matching row. The documentation does not say how Apple-silicon unified memory is counted, so verify the effective context on a Mac too.
• Concurrency multiplies the cache: Ollama documents that memory scales with OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH. Four parallel 32K requests on the 3B model add about 15 GB of f16 cache — the same as a single 128K request.
• KV cache quantization: OLLAMA_KV_CACHE_TYPE=q8_0 roughly halves cache memory (the q8_0 column), which brings 3B at 128K from about 17 GB to about 9.5 GB. Ollama requires Flash Attention for quantized cache and applies it to every loaded model.
• OS and application reserve: on an 8 GB Mac or phone the OS and other apps share the same memory, so the 4K rows are the realistic operating range.

Which Llama 3.2 model fits 4 GB, 8 GB or 16 GB?

4 GB GPU: both models at 4K context; the 3B with about 1.5 GB to spare. 8 GB GPU or Mac: both models up to 32K context, or 1B at the full 128K. 16 GB: the 3B at long context — up to the full 128K with a q8_0 cache, but not with the default f16 cache, which needs about 17 GB. Above 16 GB, the question is usually whether a larger model would serve you better than more context on a 3B one.

Can Llama 3.2 run on a CPU, Raspberry Pi or phone?

Yes — this is the use case the 1B and 3B models were built for. With Ollama or llama.cpp on the CPU they use the same memory as above, drawn from system RAM: an 8 GB machine runs either at short context. Speed is limited by memory bandwidth, so a 1.3–2 GB model generates tokens much faster on the same CPU than a 7B or 14B model does. For broader CPU and unified-memory sizing, see the local AI RAM requirements guide.

Llama 3.2 1B vs 3B: which do you need?

The memory difference at short context is about 1 GB, so the 3B is the default choice anywhere it fits. Choose the 1B when you need very long context on small hardware: it stores 32 KiB per token against the 3B's 112 KiB, so 128K of context costs 4.3 GB instead of 15 GB. For larger Llama models — 3.1 8B, 70B and 405B — see the Llama hardware requirements guide.