Autonomous operations · homelab

The homelab that finds its own faults.

One Proxmox box running a rack of self-hosted services, with watchers that check its state on a schedule, diagnose what they find, and bring you a researched fix to approve. Below is the full changelog of what it has actually caught and done — generated from the system's own records and redeployed within a minute of every fix.

241 fixes logged ·86 closed without a human ·refreshed 12 Nov 2026, 00:46 UTC ·see the full changelog →
241
Fixes logged
86
Self-resolved
14
Root-caused
23
Ways it notices
For readers new to home labs

One small computer, running your own software instead of renting it.

A "homelab" is a computer at home that runs the services most people pay a subscription for elsewhere — a streaming server instead of a cut of five streaming apps, your own file storage instead of a cloud drive, your own DNS and smart-home brain instead of letting a vendor hold them. Everything below runs on one machine, not a rack or a cloud account.

1 mini PC, fanless, on a shelf 4 cores · 32 GB RAM · 94 GB disk run by Proxmox (free, open-source) 0 services run as separate sandboxed containers/VMs on it

Each service runs in its own container or VM. A crash in one does not affect the others, and each gets its own share of processing power and memory rather than sharing a pool with everything else. Proxmox is what lets one small box behave like several separate machines.

Proxmox — the hypervisor MEDIA LXC 100 — media stack LXC 102 — jellyfin HOME VM 101 — home assistant LXC 113 — frigate VM 200 — gerry assistant ACCESS & EDGE LXC 104 — adguard LXC 108 — seerr LXC 109 — jfa-go LXC 110 — cloudflared DATA & SAFETY LXC 111 — rclone OBSERVABILITY LXC 112 — observ
Every guest currently running on the host, grouped by function — not a sample. LXC 100 alone holds about fifteen Docker containers of its own (the media stack, diagrammed separately below).
What it can do

Five things, and it can prove each one.

Every number below is read from the system's own registries when this page is built — the action vocabulary from the code that enforces it, the services from the file that decides how loudly each one pages, the guests from a capture taken off the hypervisor six-hourly. None of it is typed in by hand, since a feature list written by hand tends to drift out of sync with the system it describes.

23

ways it notices

Each watches something the others structurally cannot see — a dead indexer is a state, not an error line; a remote outage writes no local log at all.

acquisition-sli, backlog, config-export-monitor, dep-update, dependency-watch, failure-watcher, gap-audit, guest-resource, ha-watch, healthcheck-down, image-scan, indexer-watch, jellyfin-user-audit, ma-ha-provider-watch, media-malware-sweep, new-log-template, notify-watch, recording-watchdog, repo-audit, resource-watch, seerr-issue, spec-check, stuck-acquisition

16

things it may do

A closed vocabulary. An action outside this list cannot be selected, regardless of what the model proposes. Widening it is a reviewed change to the code, not a one-off judgement made during an incident.

delete-duplicate-series-timer, prune-build-cache, prune-images, restart-container, restart-guest, restart-service, restore-series-timer, search-missing-episodes, set-cpulimit, set-indexer-reject-blocklisted-hashes, set-memory, set-radarr-indexer-reject-blocklisted-hashes, set-record-new-only, start-container, stop-container, vacuum-journal

87

rules it checks against itself

Each was written in response to a specific failure, and is first checked by making it fail on purpose. A check that has never been seen to fail has only been written, not tested.

0 health checks · 41 workflows

0

services it looks after

Graded by who notices when they stop, which decides how loudly a fault is reported and whether a change may be applied without asking.

0 reachable from outside · 0 household · 0 LAN-critical · 0 back-office

13

machines it can describe exactly

Full configuration, installed packages and mounts, captured from the hypervisor and compared against the committed copy. A change made by hand shows up as a failed check rather than as a surprise during the next restore.

plus 11 application settings that used to exist only as a note asking someone to remember them

What it runs

A rack's worth of services — on a single small box.

Media, home automation, networking, and the platform to run them — around a dozen self-hosted services, each sized to a tight CPU and memory allowance and checked regularly by a resource watchdog that flags anything running short.

1 Proxmox host0 services every one CPU-limited + memory-floored right-sized weekly0 open inbound ports

Media

  • Jellyfin — streams films, TV and live TV & DVR to any screen in the house
  • Seerr — anyone with a Jellyfin account can request a title themselves
  • Radarr / Sonarr / Prowlarr — automatically find and fetch what's requested
  • Bazarr + Audiobookshelf — subtitles, plus a dedicated audiobook app

Home

  • Home Assistant — lighting, heating and presence automation
  • Frigate — camera NVR with on-device AI (person / car detection)
  • Room-level routines & scenes, driven by the same box
  • ntfy — push notifications and live dashboards to your phone

Access & edge

  • Traefik — reverse proxy in front of every service, one hostname each
  • Cloudflare Tunnel — reaches the internet with zero open ports on the router
  • JFA-Go — user accounts and invites for Jellyfin, self-service
  • AdGuard Home — whole-network DNS + DHCP with ad-blocking

Data & safety

  • qBittorrent + Gluetun — downloads routed through a VPN with a kill-switch
  • rclone — nightly encrypted backups shipped offsite, automatically
  • Library malware sweep — quarantines anything that isn't actually media
  • Recyclarr — keeps quality/format rules enforced without manual upkeep

Observability

  • Grafana + Loki — every service's logs, searchable in one place
  • Healthchecks.io — a dead-man's switch on every scheduled job
  • Uptime Kuma — active checks that each service is really responding
  • A researched fix pushed to your phone, not a raw alert

How it ships

  • GitOps via GitHub Actions — merging a change is the deploy
  • Bitwarden — secrets rendered in at deploy time, never stored in git
  • Smoke-tested behind an approval gate before it goes live
  • One-command guided rollback if a deploy goes wrong
How it connects

Three paths, picked from one corner of the map.

The service map above has five groups — media, home, access and edge, data and safety, observability. These three diagrams zoom into two of them: the media stack and the network edge. Home Assistant, the backups, and the observability stack each work differently and aren't pictured here — that's the next section.

MEDIA STACK

Seerryou request a title Radarr / Sonarrdecides what's missing Prowlarrsearches every indexer split by each indexer's tag untagged indexers — direct vpn-tagged — via gluetun:8888 flaresolverr — solves Cloudflare flaresolverr-vpn — solves Cloudflare grabs the chosen release qBittorrentinside gluetun's netns — VPN + kill-switch downloads, then imports Jellyfinlibrary — Seerr marks it fulfilled
Most indexers go out the house IP; the ones tagged vpn route through gluetun's proxy instead, each solved by its own FlareSolverr — because a Cloudflare-issued cookie is bound to the IP that solved the challenge, so a VPN-routed request replayed on the house IP (or vice versa) just gets a fresh 403.

NETWORK & ACCESS

someone outside the house the internet Cloudflare edge encrypted, outbound-only cloudflaredLXC 110 — zero open router ports a device on the LAN DNS: *.lan, *.media.lan AdGuard HomeLXC 104 — resolves, doesn't proxy connects directly Traefik Jellyfin Seerr JFA-Go + every other service
Two different ways in, one reverse proxy. The Tunnel is a client, not a listener — it dials out to Cloudflare, so there is nothing for a port scan to find on the router. A LAN device skips the tunnel entirely once AdGuard has resolved the name.

LIVE TV

custom EPG scraper6-hourly, 9-day guide xtream providerupstream tuliproxaggregates M3U + EPG threadfinthe one tuner — tuner: 1 Jellyfinlive TV & DVR
One provider connection does more than it looks: live viewing, a finished recording, and watching a recording that is still being written all cost zero extra tuner slots — only two genuinely simultaneous streams (recording one channel while watching another live) ever contend for the single connection.
Away from the media stack

Home Assistant, and what a decision actually cost to get right.

The rest of the house doesn't route through Seerr or Prowlarr at all. This is the other half: what a household member notices day to day, what Home Assistant automates on its own, and a few decisions that looked fine until they were measured.

For the household

  • Request a film or show with your Jellyfin login — nothing separate to ask for
  • The camera reports a person or a car, not every leaf that moves — Frigate's detection runs on-device
  • A password reset or an invite is self-service through JFA-Go, not a message to the admin
  • One phone channel, five priorities — a push only arrives when someone can't do something they expect to; a quiet day is a daily digest, not silence

Home Assistant automations

  • Lighting, heating and presence routines run room by room, from the same box as everything else
  • Dashboard cards can be re-pointed at a different physical device without editing the dashboard — an indirection layer built for exactly that swap
  • The bedroom TV's picture mode is wired to dim automatically — the current Wi-Fi path is unreliable, so a Zigbee IR upgrade is already in transit to replace it
  • Config lives in Home Assistant itself and is still exported to git on every change, so a restore has a file to read rather than a memory to rely on

Decisions worth knowing about

  • Capping qBittorrent to 10 active torrents made the library 11× faster — the bottleneck on one spinning disk was seek contention, not bandwidth
  • Turning on Bazarr's audio-sync thresholds stopped Jellyfin playback stutter: every subtitle was getting a full re-sync twice a day regardless of match quality
  • The offsite backup's Shared Drive role is load-bearing, not cosmetic — downgrade it and deleted backups pile up as invisible trash, measured at 940 GiB of trash against 415 GiB of live data
  • Maintainerr's settings API redacts secrets on a GET, so re-posting the full object silently destroys its own API keys — twice, before this was caught. PATCH with only the changed field is now the only way anything touches it
The changelog

Every fix, the same eight questions answered each time.

What caught it, what the check's own threshold is, how that check gets triggered, what the agent was allowed to do about it, how it arrived at a recommendation, what stops the fix from breaking something else, how it was actually implemented, and what proves it worked. Pulled directly from the system's own issues and pull requests — a step with nothing recorded says so, rather than being smoothed over. Internal specifics (hosts, addresses, paths) are removed; the reasoning behind each fix is not.

watchers' pulse, last 30 days no run checked, nothing needed took action

Loading the feed…

The roster

One job each.

Each watcher checks one kind of state on a schedule and files what it finds as an issue. None of them applies a change.

log-triageclusters error logs into deduplicated, root-caused issues
failure-watcherturns silent failures into auto-closing issues
drift-checkpinned-vs-running container digests, every day
dep-updatewatches the version layer, proposes safe upgrades
resource-watchhost load & memory; right-sizes and alerts
renovatedependency & container-image bumps, digest-pinned
recording-watchdogre-arms a stalled DVR recording within 2 min
seerr-reconcilerevery request ends fulfilled — or flagged
autohealrestarts a container the moment it goes unhealthy
malware-sweepquarantines non-media payloads in the library
smoke-test gateblocks a bad deploy before it ever lands
dependency-watchevery service the box leans on, internal and remote, every 15 minutes
stuck-acquisitionan acquisition loop burning its own inventory while every step succeeds
indexer-watchindexer health, which is a state and so writes no error line
ha-watchHome Assistant config rot: orphaned entities, dead automations
triage agentreads the issue, diagnoses it against live state, proposes a declared remedy
How it decides

Autonomous where it’s safe. Gated where it isn’t.

Six rules decide what an agent may change on its own, and what has to wait for a person.

01

Git is the source of truth

Every change is a branch, a review, a merge. Nothing is configured by hand and forgotten. The repository can rebuild the box from scratch.

02

Rehearse before prod

Anything destructive is replayed on a throwaway clone of production first. The dangerous surprises get caught on a copy, not on the live system the household depends on.

03

Diagnose, don’t blind-fix

Agents find root causes and propose. A human approves anything that changes state. Confident-but-wrong auto-fixes were tried once — and deliberately removed.

04

Defence in depth

Secrets are removed at the log shipper and again before the repository. A leaked credential is treated as a bug to fix at the source, checked at more than one layer — never as a fire drill.

05

Gated by blast radius

Every change is classed by how much it can break and who it touches. A back-office tweak lands instantly; anything that reaches the network, the household, or an outside user needs a live preflight and an explicit go-ahead — and fails closed if a check is unsure.

06

Right-sized on purpose

One small box, not a cluster. Every service runs on a deliberately tight CPU and memory budget, rebalanced on a schedule — capacity is a decision, not an afterthought. Over-building for scale it will never have is treated as its own failure mode.

How it reaches you

A researched fix reaches your phone, and one tap records the decision.

The hard part is the last step: getting a problem that needs a person's judgement in front of a person, with enough context to decide in seconds, and making sure silence means the system is healthy rather than that nobody is listening. The tap records a decision; the change itself still goes through the same review as anything else.

Detect01 De-noise02 Researchthe fix03 Escalate04 Applygated05 A person is only asked at step 04 — and step 05 happens only if they approve.
01

Detect

Agents cluster errors, watch resources, probe reachability, and track versions — turning noise into deduplicated, root-caused issues.

02

De-noise

A one-off blip auto-closes as a transient; a fixed issue reopens only if it comes back sustained. The queue stays honest — no flapping, no silent pile-up.

03

Research the fix

Before anything reaches you, per-tool experts ground the fix in the official docs and the community grounds it in real-world experience: root cause, live state, upstream behaviour. What arrives is a recommendation with its evidence attached.

04

Escalate

The researched fix lands on your phone with one-tap Approve / Reject. A daily digest arrives even when everything’s green — so a quiet day is proof the loop is alive, and a missing digest is itself the alarm.

05

Apply — gated

Approve records the decision and routes it into the reviewed, rehearsed path. It never auto-applies a change on its own — the human stays in the loop where it counts.

The actual shape of a decision push. The button carries a one-time signed token — never a credential.

Researched
A root-caused recommendation, not a raw alert to go investigate.
Silence = healthy
A daily heartbeat makes "no news" trustworthy instead of ambiguous.
Human-gated
One tap records intent; the apply still goes through review + rehearsal.
Hardened
Signed, token-free taps, on a channel that watches its own health.