Building a Production-Grade MLOps Home Lab on Windows — K8s, LLM, RAG & GitLab CI

TL;DR — I set up a complete MLOps stack on my Windows 11 PC using Multipass + k3s. This is the real guide — including every error I hit and how I fixed it. No fluff, no perfect screenshots. Just what actually worked.

Why Build a Home Lab?

Cloud bills add up fast when you're learning. A local home lab gives you:

Real Kubernetes — not Minikube toy mode
Full MLOps stack — MLflow, Minio, Airflow, Ollama, Qdrant
CI/CD with GitLab — actual pipelines, not tutorials
Zero cost — runs on hardware you already own
Safe sandbox — break things without consequences

The goal wasn't just to have services running. The goal was to practice the full DevOps + MLOps workflow end to end: push code → pipeline triggers → Terraform provisions → services deploy → metrics appear in Grafana.

My Setup

Resource	Value
OS	Windows 11 Pro
RAM	32 GB
CPU	8 cores
Disk	500 GB SSD
Hypervisor	Hyper-V (native Windows Pro)
VM Manager	Multipass
Kubernetes	k3s

Architecture decision: I kept Windows as my daily driver and ran everything inside a single Ubuntu VM via Multipass. Clean separation, easy to pause/resume, no dual boot headaches.

The Stack

Windows 11 (daily driver)
│
├── 🌐 GitLab.com (SaaS — free tier)
│    └── Pipelines + Container Registry
│
└── Multipass → vm-k3s (10 GB RAM / 4 CPU / 80 GB)
     │
     ├── ☸️  k3s (Kubernetes)
     │
     ├── ⚙️  MLOps
     │    ├── MLflow      — experiment tracking
     │    ├── Minio       — S3-compatible artifact storage
     │    └── Airflow     — pipeline orchestration
     │
     ├── 🤖 LLM Stack
     │    ├── Ollama      — run LLMs locally (CPU)
     │    └── LiteLLM    — unified OpenAI-compatible API
     │
     ├── 🔍 RAG Stack
     │    ├── Qdrant      — vector database
     │    └── LangChain   — RAG orchestration
     │
     ├── 📊 Observability
     │    ├── Prometheus  — metrics
     │    ├── Grafana     — dashboards
     │    └── Loki        — centralized logs
     │
     └── 🔐 HashiCorp Vault — secrets management

Step 1 — Enable Hyper-V and Install Tools

First, enable Hyper-V on Windows Pro (required for Multipass):

# Run as Administrator
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All
# Reboot when prompted

After reboot, install all tools via winget:

winget install Canonical.Multipass
winget install Git.Git
winget install Microsoft.VisualStudioCode
winget install Hashicorp.Terraform
winget install Helm.Helm
winget install Kubernetes.kubectl

Why winget over Chocolatey? I originally used choco install multipass and hit this error:
Exception calling "Start": "The specified executable is not a valid application for this OS platform."
Winget installs the proper signed installer directly. Use winget.

Step 2 — Create the VM

multipass launch `
  --name vm-k3s `
  --cpus 4 `
  --memory 10G `
  --disk 80G `
  22.04

RAM tip: I originally tried 16 GB and got:
Failed to allocate 16384 MB of RAM: Insufficient system resources
Windows was already consuming ~20 GB. 10 GB for the VM is the sweet spot on a 32 GB machine — leaves your OS comfortable and gives k3s plenty of room.

Check your actual free RAM before creating the VM:

Get-CimInstance Win32_OperatingSystem | Select-Object FreePhysicalMemory, TotalVisibleMemorySize

Enter the VM:

multipass shell vm-k3s
# Prompt becomes: ubuntu@vm-k3s:~$

Step 3 — Install Docker + k3s

Inside the VM:

# Docker
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker ubuntu
sudo systemctl enable docker && sudo systemctl start docker

# k3s — lightweight Kubernetes
curl -sfL https://get.k3s.io | sh -s - \
  --write-kubeconfig-mode 644 \
  --disable traefik \
  --docker

sleep 20
sudo k3s kubectl get nodes

Configure kubectl:

mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown ubuntu:ubuntu ~/.kube/config
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
source ~/.bashrc

kubectl get nodes
# NAME      STATUS   ROLES                  AGE   VERSION
# vm-k3s    Ready    control-plane,master   30s   v1.29.x

Step 4 — Helm + Namespaces

# Install Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# Add repos
helm repo add grafana         https://grafana.github.io/helm-charts
helm repo add prometheus      https://prometheus-community.github.io/helm-charts
helm repo add open-telemetry  https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo add hashicorp       https://helm.releases.hashicorp.com
helm repo update

# Create namespaces
for ns in mlops llm rag monitoring logging vault; do
  kubectl create namespace $ns
done

Step 5 — Connect GitLab CI

I used GitLab.com SaaS instead of self-hosting GitLab. This saved 6 GB of RAM — GitLab CE alone needs 6+ GB. Free tier is more than enough for a home lab.

Create a project on gitlab.com, grab the registration token from Settings → CI/CD → Runners, then:

# Install GitLab Runner
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install -y gitlab-runner
sudo usermod -aG docker gitlab-runner

# Register
sudo gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.com" \
  --registration-token "YOUR_TOKEN_HERE" \
  --executor "docker" \
  --docker-image "alpine:latest" \
  --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
  --docker-privileged \
  --description "homelab-runner" \
  --tag-list "homelab,k8s,mlops,terraform" \
  --run-untagged true

sudo gitlab-runner start

Your runner appears green in GitLab within seconds. Every push now triggers real CI/CD on your local machine.

Step 6 — Deploy Minio (The Right Way)

This is where I hit my first major blocker.

What I tried first:

helm install minio bitnami/minio \
  --namespace mlops \
  --set auth.rootUser=minioadmin \
  --set auth.rootPassword=minioadmin123

What happened:

Failed to pull image "docker.io/bitnami/minio:2025.7.23-debian-12-r3": not found
Error: ErrImagePull → ImagePullBackOff

Bitnami generates Helm chart tags that reference Docker images which don't yet exist on Docker Hub. Classic timing issue.

The fix — use the official Minio image directly:

cat <<'EOF' | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: minio
  namespace: mlops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: minio
  template:
    metadata:
      labels:
        app: minio
    spec:
      containers:
      - name: minio
        image: quay.io/minio/minio:latest
        command: ["minio", "server", "/data", "--console-address", ":9001"]
        env:
        - name: MINIO_ROOT_USER
          value: "minioadmin"
        - name: MINIO_ROOT_PASSWORD
          value: "minioadmin123"
        ports:
        - containerPort: 9000
          name: api
        - containerPort: 9001
          name: console
---
apiVersion: v1
kind: Service
metadata:
  name: minio
  namespace: mlops
spec:
  type: NodePort
  ports:
  - name: api
    port: 9000
    nodePort: 30900
  - name: console
    port: 9001
    nodePort: 30901
  selector:
    app: minio
EOF

quay.io/minio/minio is Minio's own registry — always up to date, no tag mismatch issues.

Step 7 — Deploy MLflow

MLflow needs Minio as its artifact backend. I used SQLite to keep it simple (no PostgreSQL dependency for a home lab):

cat <<'EOF' | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mlflow
  namespace: mlops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mlflow
  template:
    metadata:
      labels:
        app: mlflow
    spec:
      initContainers:
      - name: create-minio-bucket
        image: quay.io/minio/mc:latest
        command: ["/bin/sh", "-c"]
        args:
          - |
            mc alias set minio http://minio:9000 minioadmin minioadmin123
            mc mb minio/mlflow --ignore-existing
      containers:
      - name: mlflow
        image: ghcr.io/mlflow/mlflow:latest
        command:
          - mlflow
          - server
          - --host=0.0.0.0
          - --port=5000
          - --backend-store-uri=sqlite:///mlflow.db
          - --default-artifact-root=s3://mlflow/
          - --serve-artifacts
        env:
        - name: MLFLOW_S3_ENDPOINT_URL
          value: "http://minio:9000"
        - name: AWS_ACCESS_KEY_ID
          value: "minioadmin"
        - name: AWS_SECRET_ACCESS_KEY
          value: "minioadmin123"
        - name: AWS_DEFAULT_REGION
          value: "us-east-1"
        ports:
        - containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
  name: mlflow
  namespace: mlops
spec:
  type: NodePort
  ports:
  - port: 5000
    nodePort: 30500
  selector:
    app: mlflow
EOF

The initContainer automatically creates the mlflow bucket in Minio before the server starts — no manual setup needed.

Step 8 — Run a Local LLM with Ollama

This is where it gets interesting. Running a real LLM on your local machine, inside Kubernetes, on CPU only.

cat <<'EOF' | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ollama
  namespace: llm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ollama
  template:
    metadata:
      labels:
        app: ollama
    spec:
      containers:
      - name: ollama
        image: ollama/ollama:latest
        ports:
        - containerPort: 11434
        env:
        - name: OLLAMA_NUM_PARALLEL
          value: "1"
        - name: OLLAMA_MAX_LOADED_MODELS
          value: "1"
        resources:
          requests:
            memory: "2Gi"
            cpu: "1"
          limits:
            memory: "4Gi"
            cpu: "3"
        volumeMounts:
        - name: ollama-data
          mountPath: /root/.ollama
      volumes:
      - name: ollama-data
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: ollama
  namespace: llm
spec:
  type: NodePort
  ports:
  - port: 11434
    nodePort: 31434
  selector:
    app: ollama
EOF

Critical: always set resource limits on shared VMs. Without limits, Ollama will consume all available RAM and OOMKill your other pods. I learned this the hard way.

Choosing the Right Model for CPU

Model	RAM needed	Good for
Mistral 7B Q4	4.3 GB	Too heavy for 4 GB limit
Phi-3 Mini	3.5 GB	Still too heavy
llama3.2:1b	1.3 GB	✅ Perfect for CPU home lab
gemma2:2b	1.6 GB	✅ Good alternative

OLLAMA_POD=$(kubectl get pod -n llm -l app=ollama -o jsonpath='{.items[0].metadata.name}')

# Pull the model
kubectl exec -n llm $OLLAMA_POD -- ollama pull llama3.2:1b

# Test it
kubectl exec -n llm $OLLAMA_POD -- ollama run llama3.2:1b "Explain RAG in 2 sentences"

Test via API:

VM_IP=$(hostname -I | awk '{print $1}')
curl -s http://$VM_IP:31434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2:1b",
    "prompt": "What is MLOps?",
    "stream": false
  }' | python3 -c "import sys,json; print(json.load(sys.stdin)['response'])"

Step 9 — Observability Stack

# Prometheus + Grafana
helm install kube-prometheus prometheus/kube-prometheus-stack \
  --namespace monitoring \
  --set grafana.service.type=NodePort \
  --set grafana.service.nodePort=30300 \
  --set grafana.adminPassword=admin123 \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

# Loki (log aggregation) + Promtail (log shipper)
helm install loki grafana/loki-stack \
  --namespace logging \
  --set grafana.enabled=false \
  --set promtail.enabled=true

Access Grafana:

VM_IP=$(hostname -I | awk '{print $1}')
echo "Grafana: http://$VM_IP:30300"
# Login: admin / admin123

Add Loki as a data source in Grafana:

Settings → Data Sources → Add → Loki
URL: http://loki.logging:3100

Now you have unified logs + metrics in one dashboard.

Step 10 — Vault for Secrets Management

helm install vault hashicorp/vault \
  --namespace vault \
  --set server.dev.enabled=true \
  --set server.dev.devRootToken=root \
  --set ui.enabled=true \
  --set ui.serviceType=NodePort \
  --set ui.serviceNodePort=30820

# Store your first secret
VAULT_POD=$(kubectl get pod -n vault -l app.kubernetes.io/name=vault -o jsonpath='{.items[0].metadata.name}')
kubectl exec -n vault $VAULT_POD -- vault secrets enable kv-v2
kubectl exec -n vault $VAULT_POD -- vault kv put secret/homelab \
  minio_key=minioadmin \
  minio_secret=minioadmin123

VM_IP=$(hostname -I | awk '{print $1}')
echo "Vault UI: http://$VM_IP:30820  (token: root)"

In your GitLab CI pipeline, reference Vault secrets instead of hardcoding them in variables.

The Full Picture — All Services Running

VM_IP=$(hostname -I | awk '{print $1}')
echo "=== Your Home Lab ==="
echo "MLflow   : http://$VM_IP:30500"
echo "Minio    : http://$VM_IP:30901"
echo "Grafana  : http://$VM_IP:30300  (admin/admin123)"
echo "Vault    : http://$VM_IP:30820  (token: root)"
echo "Ollama   : http://$VM_IP:31434"

Lessons Learned

1. Bitnami Helm charts break on image tags
Don't use bitnami/minio — it references images that don't exist yet. Use quay.io/minio/minio:latest directly.

2. Always set Kubernetes resource limits on shared VMs
Without limits, one greedy pod (looking at you, Ollama) will OOMKill everything else. Set limits.memory always.

3. RAM planning matters more than you think
On a 32 GB machine, Windows itself consumes ~20 GB. That leaves 12 GB for your VM. Budget carefully — 10 GB for the VM is the realistic sweet spot.

4. GitLab SaaS > self-hosted for a home lab
Self-hosting GitLab CE needs 6+ GB of RAM just to idle. GitLab.com free tier gives you unlimited private repos, 400 CI/CD minutes/month, and a container registry. Use it.

5. Start small with LLMs on CPU
Forget Mistral 7B on CPU without a GPU. llama3.2:1b is surprisingly capable for RAG experiments and uses only 1.3 GB. Add GPU passthrough later if you need more power.

6. Use winget not choco for Multipass on Windows
Chocolatey's Multipass package uses an installer that fails on recent Windows builds. winget install Canonical.Multipass works every time.

What's Next

This setup is a solid foundation. Here's what I'm building on top of it:

Kubeflow Pipelines — proper ML pipeline orchestration on K8s
OpenTelemetry Collector — unified traces/metrics/logs routing
Datadog integration — ship everything to cloud observability
Terraform IaC — replace all kubectl apply with proper infrastructure as code
RAG pipeline — Qdrant + LangChain + Ollama end-to-end

Quick Reference

# VM management (Windows PowerShell)
multipass list                    # list VMs
multipass shell vm-k3s            # enter VM
multipass suspend vm-k3s          # pause (saves RAM)
multipass start vm-k3s            # resume

# Inside the VM
kubectl get pods -A               # all pods
kubectl top pods -A               # RAM/CPU usage
free -mh                          # available RAM
watch kubectl get pods -A         # live monitoring

Resources

Built this through caffeine and kubectl describe pod debugging. If you hit issues I didn't cover, drop a comment — happy to help.

If this saved you time, leave a ❤️ — it helps others find it.

推荐订阅源

DEV Community