Home on zolty.systems

Deploying a private ChatGPT alternative on a homelab k3s cluster using Open WebUI, LiteLLM proxy, and four AWS Bedrock models -- with OAuth2 for access control.

AI coding tools have gone from curiosity to default for most developers.

A catalog of AI-caused production incidents in a homelab Kubernetes cluster, the guardrail architecture built to prevent them, and an honest assessment of what still goes wrong.

How I shipped a 109-day project in 8 hours: multi-model planning with ChatGPT, Gemini, GPT Codex, and Claude Opus, then Claude Sonnet 4.6 implementing overnight via GitHub Copilot agent PRs.

How I built a multiplayer D&D platform with an AI Dungeon Master powered by AWS Bedrock, a 5-tier lore memory system, and a Dwarf Fortress-style world simulation engine.

Production failure database accumulated across 40+ incidents running a 7-node k3s cluster on Proxmox.

The full 506-line execution plan used to ship a 109-day software project in 8 hours using multi-model planning and the GitHub Copilot agent workflow.

How I stopped AI assistants from suggesting wrong tool versions and builds by adding a single environment instructions file to every repository.

A $20/month Copilot sub is the best AI tooling investment right now.

How the AI instruction system evolved from a 10-line file to a multi-layered architecture of copilot-instructions, path-scoped rules, and a living failure database -- and why every production incident makes the AI smarter.

How a new cluster node triggered Longhorn replica I/O errors that silently corrupted Prometheus WAL and Loki TSDB files, leaving Grafana showing No data for 26 hours.

Why I self-hosted Wiki.js as the primary knowledge layer for the cluster, how the AI assistant writes to it after completing work, and how it became the most useful thing I built.

A multi-repo AI skill audit found 401KB of duplicated content consuming 100K tokens.

Rolling k3s upgrade from v1.29.0 to v1.34.4 across 8 nodes with interleaved Longhorn upgrades, broken SSH, and unexpected Traefik pinning.

How I use Claude Opus 4.6, GitHub Copilot, and AWS Bedrock to build and operate a production homelab cluster -- and why AI without memory is a liability.

Full hardware benchmark results for four Lenovo M920q Proxmox hosts — NVMe SMART health, fio disk IOPS, sysbench CPU and memory bandwidth, iperf3 10GbE throughput, and the dramatic thermal impact of custom 3D-printed cooling.

Connecting Jellyseerr's API to Claude so I can say 'download 100GB of family-friendly anime that I might like' and have the cluster handle the rest — including scheduled runs that queue content automatically.

Upgrading the homelab to 10GbE with used Mellanox ConnectX-3 NICs — including the firmware flash that bricked a NIC and the cold boot recovery procedure.

A comprehensive look at the monitoring stack powering the homelab: kube-prometheus-stack for metrics, Loki for logs, custom dashboards, alert tuning, and the performance benchmarks I ran to validate the platform.

A complete guide to passing through Intel UHD 630 iGPU from Proxmox to a k3s VM for Jellyfin hardware transcoding — IOMMU, VFIO, VA-API, and all the troubleshooting.

Deploying Jellyfin with GPU transcoding, the *arr suite, automated seedbox sync with rclone, and NFS storage — a complete self-hosted media pipeline on k3s.

How I migrated all k3s nodes, MetalLB pool, and load balancer IPs from a flat network to a dedicated Server VLAN — and the etcd recovery procedure I needed when things went wrong.

One week into running a homelab k3s cluster in production, here are the top 10 failures I have encountered and the lessons each one taught me about Kubernetes operations.

How I built an AI alert responder that analyzes Prometheus alerts using AWS Bedrock and suggests remediation actions directly in Slack, plus multi-user dev workspaces on k3s.

Lessons from deploying a Vue.js frontend with 7 FastAPI backend services, NATS messaging, PostgreSQL, and Redis on a homelab k3s cluster.

A deep dive into GitHub Actions Runner Controller (ARC) on k3s — from initial setup to scaling, RBAC configuration, and all the gotchas I hit along the way.

Taking an Angular SPA and 7 Flask microservices from local development to Kubernetes — covering MQTT brokers, multi-service deployments, and the joys of container networking.

Deploying Home Assistant on k3s with hostNetwork and split routing, plus building an automated Proxmox watchdog that power-cycles unresponsive hosts via smart plugs.

Day two of the cluster build: deploying Cardboard (TCG price tracker), Trade Bot (automated trading), a cluster dashboard, monitoring with Prometheus, and dev workspaces — all in one marathon session.

From bare Proxmox hosts to a fully operational k3s HA cluster in a single day, using Terraform for VM provisioning and Ansible for configuration management.

How I chose the Lenovo ThinkCentre M920q as the foundation for a production-grade k3s homelab cluster, and why tiny form factor PCs are the best kept secret in homelabbing.

Introducing zolty.systems -- a blog about homelab infrastructure, Kubernetes, and the lessons learned from running production workloads at home.