Originally published at woitzik.dev

I assumed my homelab repo was clean. No one had ever flagged anything in review (there is no one else reviewing it), CI was green, and I generally try to use Vault and ExternalSecrets for anything sensitive.

Then I ran a full-history gitleaks detect against it. It found 12 distinct secrets committed in plaintext — including the OIDC private key that signs SSO tokens for half the cluster.

This is the scanning setup I put in place afterward, the baseline strategy that let me adopt secret scanning without getting blocked by my own history on every commit, and the remediation plan for the leaks themselves.

View the complete homelab infrastructure source on GitHub 🐙

What Gitleaks Found

gitleaks detect --no-banner -v

Twelve real findings, plus one already-hashed password (lower severity but still shouldn't be hand-committed) and one false positive in ROADMAP.md (documentation text that happened to match a generic API key pattern).

The real findings, by severity:

None of these were exposed by a public repo (this one is private), but "private repo" is not a security control — it's a single permission setting away from being public, and anyone with read access to the repo (or its history, forever) has all of this regardless.

Why a Private Repo Doesn't Make This Fine

The honest reason these accumulated: early in the project, before Vault and ExternalSecrets were set up, every new service got a quick secrets.yml with the actual values inline, "just to get it working." Once Vault was running, new services went through it — but nobody went back and migrated the old ones. Each individually felt low-risk at the time. Twelve of them, four months later, is a real exposure if the repo's access list ever changes.

This is the same drift pattern as the Terraform-vs-RouterOS-firewall divergence I wrote about separately: each shortcut is locally reasonable, the accumulated state is not.

Setting Up Gitleaks Without Getting Blocked by History

The naive approach — turn on gitleaks protect in pre-commit and call it done — fails immediately. Every single future commit gets blocked by the 12 pre-existing leaks, because gitleaks scans the whole working tree, not just your diff. You'd have to fix all 12 before you could make any other commit, including the commit that adds the scanning.

The fix is a baseline file:

gitleaks detect --baseline-path .gitleaks-baseline.json --no-banner -v

A baseline is a snapshot of currently-known findings. Anything in the baseline is allowed to keep existing; anything new fails the hook. Generate it once:

gitleaks detect --report-format json --report-path .gitleaks-baseline.json

Commit that baseline file. From this point forward, gitleaks only blocks genuinely new secrets — exactly what you want when adopting scanning on a repo with history older than the scanning itself.

The Three-Layer Hook Setup

One scan layer is not enough — a single missed git commit --no-verify or a commit made from a machine without the hooks installed slips through. Three layers, increasing scope, decreasing frequency:

# .pre-commit-config.yaml
- id: gitleaks-staged
  name: Gitleaks (staged changes)
  description: >-
    Blocks committing NEW secrets. Uses .gitleaks-baseline.json so the
    12 pre-existing leaks don't block every commit until they're fully
    remediated — only genuinely new secrets fail this.
  entry: bash -c 'gitleaks protect --staged --baseline-path .gitleaks-baseline.json --no-banner -v'
  language: system
  always_run: true
  pass_filenames: false
  stages: [pre-commit]

- id: gitleaks-full-repo
  name: Gitleaks (full history, pre-push only)
  description: Re-scans the entire repo and history before any push, against the same baseline.
  entry: bash -c 'gitleaks detect --baseline-path .gitleaks-baseline.json --no-banner -v'
  language: system
  always_run: true
  pass_filenames: false
  stages: [pre-push]

gitleaks protect --staged at commit time — fast, scans only what's staged, catches a secret before it ever enters history.

gitleaks detect at push time — re-scans the entire repo (slower, but only runs once per push, not once per commit). This catches anything that slipped past the first layer, for example a commit made with git commit --no-verify.

CI runs the same gitleaks detect command as a third, environment-independent layer — catches anything pushed from a machine that never had the hooks installed at all.

Allowlisting Real False Positives

The ROADMAP.md false positive needed an explicit allowlist entry, not a baseline bypass — baseline entries are meant for things you intend to fix, allowlist entries are for things that were never secrets in the first place:

# .gitleaks.toml
[extend]
useDefault = true

[allowlist]
description = "Known false positives"
regexes = [
  # ROADMAP.md doc text listing which services use which OIDC client auth
  # method — matches the generic-api-key pattern but is plain documentation,
  # not a secret.
  '''Proxmox/PBS/Grafana/Headscale use `client_secret_basic`''',
]

Be specific with allowlist regexes. A broad pattern here defeats the entire point of scanning — match the exact false-positive string, not a category of strings that happens to include it.

The Remediation Plan

Finding the leaks and remediating them are two different projects. Remediation means: rotate the actual credential (not just remove it from the file — the old value is still valid until rotated), and move the new value into Vault behind an ExternalSecret so it never gets hand-committed again.

The tricky part is ordering. Some of these credentials are dependencies of each other:

1. Garage RPC secret + admin token + S3 keys
   ↳ Everything else's backups depend on Garage being internally consistent.
     Rotating the S3 key also invalidates Terraform's own state backend
     credential (terraform/stacks/network/local_backend.hcl uses the same
     key) — update both in the same pass or Terraform loses access to its
     own state.

2. Authelia OIDC issuer private key
   ↳ Highest blast radius if left exposed (signs every SSO session).
     After rotating, every service trusting the old key should be checked
     for unexpected active sessions.

3. Everything else, any order
   ↳ Cloudflare Tunnel token (rotate in Cloudflare dashboard first, update
     second — order matters for tokens with an external source of truth).
   ↳ Headscale OIDC client secret must be rotated in lockstep with
     Authelia's matching client config — they're a pair.

A secret with downstream dependents must be rotated with the dependents in mind, not in isolation. Rotating Garage's S3 key without immediately updating the Terraform backend config doesn't remove a vulnerability — it breaks Terraform's access to its own state.

Confirming Remediation Actually Worked

After moving a secret to Vault and rotating the credential, re-run the same scan:

gitleaks detect --baseline-path .gitleaks-baseline.json --no-banner -v

The secret will still show up — it's in history, and the baseline still lists it. That's expected; the baseline isn't meant to disappear until every listed item has actually been fixed. Only regenerate the baseline once all 12 are addressed, as a final confirmation step that nothing was missed in the process — not as a way to make individual items "go away" faster.

What This Doesn't Fix

Scanning catches secrets in files. It does not:

• Scrub git history. The old values remain readable to anyone with repo access, forever, unless you rewrite history (git filter-repo) — which has its own risks if anyone else has a clone.
• Replace rotation. A secret found and removed from the current file tree is still valid until you change the actual credential at its source (Cloudflare dashboard, Garage admin CLI, Postgres ALTER USER, etc.).
• Catch secrets gitleaks' default ruleset doesn't recognize. Custom internal token formats need custom regex rules — useDefault = true covers known formats (AWS keys, generic API key patterns, JWTs) but not everything.

The same baseline-adoption pattern applies directly to any enterprise repo with years of history and no prior secret scanning — which describes most codebases that predate a security initiative. The Vault + ExternalSecrets target architecture this remediation moves toward is the same pattern covered in External Secrets Operator + HashiCorp Vault — that's where these 12 secrets are headed.