GPU Kubernetes on a Mini PC? Yes. Here's the Full Stack.

Description

Can you run LLMs on Kubernetes from a mini PC under your desk? Yes — and here's the full stack in 90 seconds. 🔧 Custom active cooling for the NVIDIA Tesla T4 (N3rdware bracket + 5V fan swap) 🖥️ GPU passthrough on Proxmox — including the 4 kernel parameters most people miss on the MS-02 ☸️ Kubernetes cluster provisioned with Cluster API (GPU + CPU worker pools) ⚡ NVIDIA GPU Operator: drivers, device plugin, DCGM monitoring — single Helm install 🔀 GPU time-slicing: 1 physical Tesla T4 → 4 schedulable GPU slots in Kubernetes 🤖 LLM inference with Ollama — Llama 3.1 8B + DeepSeek-R1 7B 🌐 Gateway API Inference Extension: KV-cache-aware routing, A/B testing, load shedding 📂 GitHub repo (all configs + manifests): https://github.com/isItObservable/K8s-LLM #Kubernetes #GPU #LLM #Proxmox #Homelab #NVIDIA #CloudNative #IsItObservable