Quick Fix (TL;DR)

Ollama defaults to a context window of 2,048 or 4,096 tokens regardless of model capabilities. To expand context for AI coding agents without silent token truncation, create a custom Modelfile with your desired num_ctx limit:

# 1. Create a Modelfile targeting your base model
cat << 'MODFILE' > Modelfile
FROM qwen2.5-coder:14b
PARAMETER num_ctx 32768
MODFILE

# 2. Build and register the 32k context variant
ollama create qwen2.5-coder:14b-32k -f Modelfile

# 3. Verify the model loads with 32,768 context tokens
ollama run qwen2.5-coder:14b-32k "Explain the difference between mutex and semaphore in Linux C"

---

1. Why Ollama Silently Truncates Context

When an AI coding agent like Continue.dev, Cline, or Aider sends a 16,000-token prompt containing your project tree, open tabs, and file contents, Ollama does not return an HTTP error. It accepts the request, truncates the prompt from the front, and processes only the final 2,048 or 4,096 tokens.

The silent failure occurs because Ollama sets a conservative default context length (num_ctx) to prevent unconstrained KV cache memory allocations on low-memory workstations.

When truncation happens:

• System prompts defining code style rules disappear.
• Architecture summaries and imported file headers are dropped.
• The model hallucinates non-existent functions because it lost the declarations declared earlier in the file.

---

2. KV Cache VRAM Consumption Matrix

Expanding context requires dedicated VRAM for the Key-Value (KV) cache. With Grouped-Query Attention (GQA) used in modern architectures like Qwen 2.5 and Llama 3.3, memory scales linearly with context length.

The table below shows measured VRAM requirements for FP16 weights plus standard FP16 KV cache on Ubuntu 24.04 with CUDA 12.4:

Model Architecture	Base Weights (Q4_K_M)	4k Context VRAM	16k Context VRAM	32k Context VRAM	64k Context VRAM	Recommended GPU
Qwen2.5-Coder-7B	4.7 GB	5.2 GB	6.4 GB	7.9 GB	11.2 GB	RTX 3060 (12GB) / RTX 4070 (12GB)
Llama-3.1-8B	4.9 GB	5.5 GB	6.8 GB	8.3 GB	12.1 GB	RTX 4070 Super (12GB)
Qwen2.5-Coder-14B	9.0 GB	9.9 GB	11.8 GB	14.3 GB	19.8 GB	RTX 4080 (16GB) / RTX 4090 (24GB)
DeepSeek-R1-Distill-32B	19.8 GB	21.1 GB	23.4 GB	26.5 GB (Offloads)	32.8 GB	2x RTX 3090 (48GB)
Qwen2.5-Coder-32B	19.8 GB	21.0 GB	23.3 GB	26.4 GB (Offloads)	32.6 GB	2x RTX 4090 (48GB)

Note: If context allocation exceeds available VRAM, Ollama automatically offloads layers to system RAM, dropping generation speed from 45 tokens/sec to under 4 tokens/sec.

---

3. Three Methods to Configure num_ctx

Select the configuration pattern that matches your workflow architecture:

Method A: Modelfile Definition (Recommended for Coding Agents)

Creating a named alias guarantees that every tool connecting to Ollama receives the extended context window automatically.

Create a file named Modelfile:

FROM deepseek-r1:14b

# Set context window to 32k tokens
PARAMETER num_ctx 32768

# Set temperature for deterministic programming output
PARAMETER temperature 0.2

Build the alias:

ollama create deepseek-r1:14b-32k -f Modelfile

Point your agent (such as Cursor or Cline) to deepseek-r1:14b-32k.

Method B: REST API Payload Parameter

If you interact with Ollama through Python or shell scripts, inject num_ctx directly into the options object:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:14b",
  "prompt": "Refactor this Go handler for concurrency safety",
  "options": {
    "num_ctx": 32768,
    "temperature": 0.1
  },
  "stream": false
}'

Method C: Agent Extension Configuration (Continue.dev)

If using Continue.dev in VS Code or JetBrains, update ~/.continue/config.json to enforce num_ctx on the client side:

{
  "models": [
    {
      "title": "Local Qwen Coder 14B (32k)",
      "provider": "ollama",
      "model": "qwen2.5-coder:14b",
      "requestOptions": {
        "num_ctx": 32768
      }
    }
  ]
}

---

4. Hardware Optimization: Flash Attention & Parallel Slots

Enabling Flash Attention reduces KV cache memory consumption and accelerates inference at long sequence lengths.

Edit the systemd service unit:

sudo systemctl edit ollama.service

Add the following environment variables inside the override block:

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_KEEP_ALIVE=24h"

Save and reload the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Parameter Breakdown:

• OLLAMA_FLASH_ATTENTION=1: Activates memory-efficient attention kernels on NVIDIA Ampere, Ada Lovelace, and Blackwell GPUs.
• OLLAMA_NUM_PARALLEL=1: Restricts concurrent processing slots to 1. By default, Ollama can spin up multiple parallel sequence slots, duplicating KV cache allocations and triggering out-of-memory errors on 32k+ context.
• OLLAMA_KEEP_ALIVE=24h: Keeps model weights mapped in GPU VRAM instead of unloading them after 5 minutes of idle time.

---

5. Verification: Confirming Active Context Allocation

Verify that your running instance actually loaded the full context allocation instead of falling back to default values.

Run the Ollama runner with debug logging enabled:

OLLAMA_DEBUG=1 ollama run qwen2.5-coder:14b-32k "" 2>&1 | grep -i "n_ctx"

Expected diagnostic output:

llama_init_from_model: n_ctx = 32768
llama_init_from_model: n_ctx_per_seq = 32768
llama_kv_cache_init: kv_size = 32768, type = f16

If n_ctx displays 2048 or 4096, your client application is overriding the Modelfile parameter with its own lower default value. Check your client agent settings to resolve the conflict.