Quick Start (TL;DR)

Fine-tuning DeepSeek R1 distilled reasoning models locally requires preserving the <think>...</think> internal monologue while fitting model weights into consumer VRAM. Run the following commands on Ubuntu 24.04 with an NVIDIA GPU (16GB or 24GB VRAM) to install Unsloth and start 4-bit QLoRA fine-tuning:

# 1. Create a dedicated virtual environment
conda create -n unsloth_env python=3.11 pytorch-cuda=12.4 pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers -y
conda activate unsloth_env

# 2. Install Unsloth and training dependencies
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps trl peft accelerate bitsandbytes datasets

# 3. Launch training script
python train_r1.py

---

1. Hardware & VRAM Allocation Matrix

Unsloth uses manual Triton kernel rewrites to cut memory consumption by up to 70% compared to standard Hugging Face PEFT. This enables fine-tuning DeepSeek R1 models on a single workstation GPU:

Base Model	Architecture	Quantization	Min VRAM Required	Max Sequence Length	Recommended GPU
DeepSeek-R1-Distill-Qwen-1.5B	Qwen 2.5	4-bit QLoRA	5.5 GB	8,192 tokens	RTX 3060 (6GB mobile / 12GB)
DeepSeek-R1-Distill-Qwen-7B	Qwen 2.5	4-bit QLoRA	9.2 GB	4,096 tokens	RTX 4070 (12GB) / RTX 3060 (12GB)
DeepSeek-R1-Distill-Llama-8B	Llama 3.1	4-bit QLoRA	10.1 GB	4,096 tokens	RTX 4070 Ti Super (16GB)
DeepSeek-R1-Distill-Qwen-14B	Qwen 2.5	4-bit QLoRA	18.8 GB	4,096 tokens	RTX 3090 / RTX 4090 (24GB)
DeepSeek-R1-Distill-Qwen-32B	Qwen 2.5	4-bit QLoRA	36.5 GB	2,048 tokens	2x RTX 3090 / A6000 (48GB)

---

2. Dataset Formatting: Preserving <think> Reasoning Tags

Standard Supervised Fine-Tuning (SFT) breaks reasoning models if you replace the model’s intermediate thinking trace with direct answers. The dataset must format the reasoning block explicitly:

{
  "prompt": "Write a Python function to validate whether an IP address string is an IPv4 or IPv6 format.",
  "response": "<think>\nTo validate an IP address in Python, I should consider using the standard ipaddress module.\n1. Attempt ipaddress.ip_address(address).\n2. Check if isinstance(obj, IPv4Address) or isinstance(obj, IPv6Address).\n3. Handle ValueError exceptions cleanly.\n</think>\nimport ipaddress\n\ndef validate_ip(address: str) -> str:\n    try:\n        ip = ipaddress.ip_address(address)\n        return \"IPv4\" if ip.version == 4 else \"IPv6\"\n    except ValueError:\n        return \"Invalid\""
}

Format your input JSONL file with prompt and response columns matching this convention before loading.

---

3. The Complete Training Script (train_r1.py)

Save the following code as train_r1.py. This script initializes 4-bit loading, applies LoRA adapters targeting all attention projection matrices, and runs training:

import torch
from unsloth import FastLanguageModel
from datasets import load_dataset
from trl import SFTTrainer
from transformers import TrainingArguments

# Configuration
max_seq_length = 4096
dtype = None # Auto-detect (Float16 or Bfloat16)
load_in_4bit = True

# 1. Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/DeepSeek-R1-Distill-Qwen-14B",
    max_seq_length=max_seq_length,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
)

# 2. Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16, # Rank
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0, # Unsloth optimizes dropout=0 for speed
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
)

# 3. Load dataset
dataset = load_dataset("json", data_files="dataset.jsonl", split="train")

def format_prompts(examples):
    texts = []
    for prompt, response in zip(examples["prompt"], examples["response"]):
        text = f"<｜User｜>{prompt}<｜Assistant｜>{response}"
        texts.append(text)
    return {"text": texts}

dataset = dataset.map(format_prompts, batched=True)

# 4. Set training hyperparameters
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=2,
    packing=False,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=120,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="linear",
        seed=3407,
        output_dir="outputs",
    ),
)

# 5. Execute training
trainer_stats = trainer.train()
print(f"Training complete. Peak memory: {torch.cuda.max_memory_allocated() / 1e9:.2f} GB")

---

4. Exporting to GGUF & Serving in Ollama

Once training finishes, export the merged weights directly into GGUF format for deployment on local workstations with Ollama or llama.cpp:

# Export 4-bit quantized GGUF
model.save_pretrained_gguf(
    "deepseek-r1-14b-custom",
    tokenizer,
    quantization_method="q4_k_m"
)

Create a Modelfile to register the custom model with Ollama:

FROM ./deepseek-r1-14b-custom-unsloth.Q4_K_M.gguf

PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER num_ctx 8192

SYSTEM """You are a domain-specific reasoning assistant. Always formulate your inner monologue inside <think>...</think> before presenting your final answer."""

Register the model:

ollama create deepseek-r1:14b-custom -f Modelfile
ollama run deepseek-r1:14b-custom "Validate this IPv6 CIDR block"

The model produces structured reasoning steps followed by the formatted code response, retaining its analytical capabilities within your specialized problem domain.