Quick Start (TL;DR)

To install Ollama and run a quantized DeepSeek R1 distilled model immediately on Linux with NVIDIA GPU acceleration, run:

# 1. Install Ollama binary & systemd service
curl -fsSL https://ollama.com/install.sh | sh

# 2. Run the 8B distilled model (Requires ~6GB VRAM, fits RTX 3060/4060)
ollama run deepseek-r1:8b

Once loaded, the interactive prompt handles streaming responses including DeepSeek R1’s <think>...</think> reasoning trace.

---

1. DeepSeek R1 Model Sizes & Hardware VRAM Matrix

DeepSeek R1 was trained at a full 671B parameter scale (Mixture-of-Experts), then distilled into smaller dense architectures using Qwen-2.5 and Llama-3. For local workstation deployment, select the distilled parameter size that fits your GPU’s unshared VRAM:

Model Tag	Base Architecture	Context Window	Min VRAM (Q4_K_M)	Recommended GPU
deepseek-r1:1.5b	Qwen-2.5-Math-1.5B	32k	1.8 GB	Any modern GPU / CPU only
deepseek-r1:7b	Qwen-2.5-7B	32k	5.2 GB	RTX 3060 (6GB mobile), RTX 2060
deepseek-r1:8b	Llama-3.1-8B	32k	5.8 GB	RTX 3060 / 4060 (8GB)
deepseek-r1:14b	Qwen-2.5-14B	32k	9.6 GB	RTX 3060 (12GB), RTX 4070 (12GB)
deepseek-r1:32b	Qwen-2.5-32B	32k	20.5 GB	RTX 3090 / RTX 4090 (24GB)
deepseek-r1:70b	Llama-3.3-70B	32k	43.0 GB	2x RTX 3090 / 2x RTX 4090 (48GB total)
deepseek-r1:671b	MoE (37B active)	128k	420+ GB	Multi-node cluster (8x H100 80GB)

Rule of thumb: The 14B model (deepseek-r1:14b) delivers the highest reasoning performance-per-watt for standard 12GB–16GB consumer GPUs, while 32B matches proprietary reasoning models for complex programming tasks.

---

2. Linux System Preparation & GPU Driver Verification

Before downloading large model weights, verify that your NVIDIA drivers and CUDA runtime are exposed to system services:

nvidia-smi

Ensure your Driver Version is $\ge 535$ and CUDA Version is $\ge 12.0$. If running on Ubuntu/Debian without drivers:

sudo apt update && sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers install
sudo reboot

---

3. Configuring the Ollama Service for Performance

By default, Ollama binds only to 127.0.0.1 and uses conservative memory settings. To expose the inference API to your local network, enable Flash Attention, and allow concurrent streams, configure a systemd override.

Create /etc/systemd/system/ollama.service.d/override.conf:

sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf > /dev/null <<EOF
[Service]
# Allow LAN access for web UIs and IDE extensions
Environment="OLLAMA_HOST=0.0.0.0:11434"

# Enable Flash Attention (reduces VRAM consumption for long reasoning chains)
Environment="OLLAMA_FLASH_ATTENTION=1"

# Keep model loaded in VRAM for 30 minutes after last request
Environment="OLLAMA_KEEP_ALIVE=30m"

# Number of parallel request slots
Environment="OLLAMA_NUM_PARALLEL=2"
EOF

Reload and restart the daemon:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Verify the service is active and running:

systemctl status ollama --no-pager

---

4. Expanding the Context Window (Crucial for Reasoning)

By default, Ollama limits inference context (num_ctx) to 2,048 tokens unless specified. Because DeepSeek R1 outputs hundreds or thousands of tokens inside <think>...</think> tags before returning the final solution, a 2k context window will truncate code generation.

Create a custom Modelfile to set num_ctx to 32,768:

tee Modelfile > /dev/null <<EOF
FROM deepseek-r1:14b

# Set context window to 32k tokens
PARAMETER num_ctx 32768

# Recommended sampling parameters from DeepSeek team
PARAMETER temperature 0.6
PARAMETER top_p 0.95
EOF

Build the custom model:

ollama create deepseek-r1-32k -f Modelfile

Launch the expanded context model:

ollama run deepseek-r1-32k

---

5. Integrating with Coding Tools & APIs

Ollama provides an OpenAI-compatible endpoint at http://localhost:11434/v1.

Verification via cURL

Test the API and inspect the reasoning trace:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1:8b",
    "messages": [
      {"role": "user", "content": "Write a Python function to find all anagrams in an array."}
    ],
    "stream": false
  }'

Cursor & Continue Extension Setup

• Provider: OpenAI Compatible
• Base URL: http://localhost:11434/v1
• API Key: ollama (any placeholder string)
• Model Name: deepseek-r1:8b or deepseek-r1-32k

---

6. Troubleshooting Common Linux Errors

1. Slow Token Generation (< 3 tokens/sec) & 100% CPU Usage

• Root Cause: The model failed to offload layers to the GPU and fell back to CPU system RAM.
• Diagnosis: Check GPU layer distribution with ollama ps:

  ollama ps

  Look at the PROCESSOR column. If it says 100% CPU or 50%/50% CPU/GPU:
• Fix: Ensure your user or the ollama system user has access to GPU compute groups:

  sudo usermod -a -G video,render ollama
  sudo systemctl restart ollama

2. CUDA Out of Memory (OOM) on Launch

• Root Cause: The requested model layer weights plus context KV-cache exceed physical GPU VRAM.
• Fix: Either downscale from 14b to 8b, or reduce num_ctx in your Modelfile from 32768 to 16384. Each 8k context window expansion adds approximately 1.2GB–2.0GB to VRAM usage depending on model dimensions.

3. Model Skips <think> and Gives Shallow Answers

• Root Cause: Setting temperature=0 or injecting aggressive system prompts that demand “Answer directly without thinking”.
• Fix: Keep temperature between 0.5 and 0.7. DeepSeek R1 models rely on native reasoning tokens to structure complex answers; do not force zero-shot format constraints into the system message.

---

Summary Reference

• Lightweight / Laptop (8GB VRAM): ollama run deepseek-r1:8b
• Mid-tier Workstation (12GB VRAM): ollama run deepseek-r1:14b
• High-end Workstation (24GB VRAM): ollama run deepseek-r1:32b
• System Config Location: /etc/systemd/system/ollama.service.d/override.conf
• API Endpoint: http://localhost:11434/v1