fscotto/infra
1
0
Fork 0
mirror of https://github.com/fscotto/infra.git synced 2026-09-27 19:03:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add Atlas media and storage services [Phase 1] (#9)

Browse Source 
* Add Atlas media and storage services

* Document Atlas backend phase one and WireGuard deployment

* Enable Atlas NAS management and document bootstrap workflow

* Harden Atlas network, SSH, firewall, and sharing

* Rotate Ansible Vault secrets

* Allow configurable Aegis SSH users and authorized keys

* Manage Aegis SSH authorized key fragments

* Manage SSH authorized key fragments for infrastructure hosts

* Harden Rocky storage and sharing configuration

* Verify WireGuard handshakes and restore Podman networking
This commit is contained in:
Fabio Scotto di Santolo
2026-09-15 22:39:28 +02:00
committed by GitHub
parent 73bf2cd62a
commit 160d63c02d
50 changed files with 2071 additions and 460 deletions
Download Patch File Download Diff File Expand all files Collapse all files

76
AGENTS.md
View File

@@ -53,6 +53,14 @@ Ansible-driven personal infrastructure repo for Fedora and Void desktops, Fedora
- AI coding agents: `ansible-playbook ansible/site.yml --limit <host> --tags ai_agents --check --diff`
- Mail bootstrap: `sh -n scripts/bootstrap_mail.sh` and `shellcheck scripts/bootstrap_mail.sh`
- Server compose render: `podman-compose -f /opt/docker/server/docker-compose.yml config` and `systemctl status podman-compose-server`
- Atlas media stack:
`ansible-playbook ansible/site.yml --limit atlas --tags storage,sharing,containers --check --diff`
- Atlas network/share hardening:
`ansible-playbook ansible/site.yml --limit atlas --tags hardening,sharing --check --diff`
- Atlas phase-one rootless services:
`ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff`
- Prometheus/Atlas WireGuard overlay:
`ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard --check --diff`
- DuckDNS config only: `ansible-playbook ansible/site.yml --limit prometheus --tags duckdns --check --diff`
## Conventions
@@ -104,12 +112,13 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
- `rocky_server` is a child of both `platform_rocky` and `server`; `prometheus` is its active target.
- The target must already provide `server_username` with local sudo access before the profile runs.
- The Rocky profile installs Podman and podman-compose, uses firewalld, preserves SELinux enforcement, and renders the
same server Compose stack with a `podman-compose-server` systemd unit. It does not start, enable, transfer data,
update DNS, or cut over traffic; activating the stack is a manual step.
existing Nginx Proxy Manager/Gitea Compose stack with a `podman-compose-server` systemd unit. PostgreSQL and
Navidrome are no longer part of the desired Prometheus configuration. The role does not stop or remove legacy
containers, delete `/opt/postgres/data`, start the Compose stack, update DNS, or cut over traffic.
- Firewalld enables SSH, Cockpit (`9090/tcp`), HTTP and HTTPS. Nginx Proxy Manager publishes `80/tcp` and
`443/tcp`; bind its administration interface only to `127.0.0.1:81` and use `npm-tunnel` from Ikaros or Nymph.
Nextcloud remains disabled; do not provision `/srv/nextcloud` directories.
- `scripts/migrate_prometheus_data.sh` is the separate, source-host-run migration path. It dry-runs by
- `scripts/migrate_prometheus_data.sh` is the separate, source-host-run NPM/Gitea migration path. It dry-runs by
default and requires explicit source-stack quiescing before copying persistent Docker data with rsync.
- Atlas-only OpenZFS, NFS, Samba, and Syncthing stay selected through Atlas host variables and must not
leak into `rocky_server`. Cockpit plus its Navigator and Podman extensions are selected explicitly for
@@ -119,19 +128,49 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
- `atlas` is a remote Rocky Linux 9 NAS. Keep its connection, LAN, pool and mountpoint values in
`host_vars/atlas.yml`. Bootstrap it once with `-e atlas_connection_username=<existing-admin>`;
subsequent runs use the dedicated Atlas account.
- The pool is pre-existing: never add pool creation, disk partitioning, RAIDZ creation, rollback,
or destruction to the Atlas profile.
- `atlas_manage_storage` and `atlas_manage_firewall` remain false until their placeholders are
replaced; only then may the profile manage datasets, shares and LAN-restricted firewall rules.
- Atlas requires `vault_atlas_authorized_ssh_keys`, `vault_atlas_admin_password_hash` for Cockpit
and, when storage is enabled, `vault_atlas_samba_password`. Never print these values.
- Atlas uses NFSv4 for Linux and SMB for Windows/WSL, restricted to the configured LAN. Snapshot,
Borg/Hetzner offsite backup, Prometheus pull and USB backup automation are intentionally deferred.
- The pool is normally pre-existing. A one-time bootstrap may create it only when `atlas_create_pool=true`
is explicitly supplied and `atlas_zpool_disks` contains exactly four real `/dev/disk/by-id/...` paths.
Never partition, force, destroy, roll back, or modify the vdev layout of an existing pool.
- `atlas_manage_storage`, `atlas_manage_sharing`, and `atlas_manage_firewall` are enabled in Atlas host vars as
the declared steady state; set one false only for a deliberate suspension. `atlas_manage_media_stack` remains false
until the future rootful Immich stack has its required Vault inputs and target validation.
- Atlas requires `vault_atlas_admin_password_hash` for Cockpit and, while sharing is enabled,
`vault_atlas_samba_password`. The future rootful media stack also requires
`vault_atlas_immich_db_password`. Never print these values.
- Atlas creates the complete declared hierarchy only under the verified existing or explicitly bootstrapped pool: `work`, `archive`,
`archive/app_data`, `archive/app_data/navidrome`, `archive/app_data/syncthing`, `media`, `media/music`,
`media/photobook`, `backups`, `backups/services`, and `backup_prometheus`. `backups/services` has a `500G`
refreservation. There is no separate legacy `zpool/syncthing` dataset.
- The `immich` system account is fixed to UID/GID `1100`, has no login shell or `wheel` membership, and receives only
the `video` and `render` supplementary groups. Immich's rootful Quadlets run as `1100:1100`; Server and ML receive
`/dev/dri`, while the Photobook external library is read-only at `/external/photobook`.
- Atlas applies persistent kernel network hardening: redirects and source routes are rejected, martians logged, reverse-path filtering remains loose for WireGuard, and IPv4 forwarding is disabled. SSH permits only the declared administrator using public-key authentication; root login, passwords, agent and remote forwarding
are disabled, while local forwarding remains available for private administrative tunnels. Photobook is exported only to the configured Aegis IP with all access squashed to UID/GID
`1100`. Targeted SELinux is enforced persistently; a required reboot is reported but never initiated automatically. The primary LAN interface is assigned explicitly to the managed firewalld zone, and firewall rules are applied before NFS or SMB are started; their service state and TCP listeners are then verified. SMB3 exposes `Archive` to Vault-backed authorized accounts on mandatory encrypted, signed SMB3 over TCP/445 only and admits the configured LAN without host-specific exclusions.
- Atlas NPM and Immich share a rootful Podman network. NPM publishes HTTP/HTTPS, but its administration port remains
bound to `127.0.0.1:81`; do not expose it directly to the LAN or Internet.
- `profile_backend_phase1` is limited to rootless Navidrome and Syncthing user Quadlets on Atlas. Official Navidrome
`0.63.2` uses SQLite below `/data` and does not support `ND_DATABASE_URL` or an external PostgreSQL backend; do not
recreate the obsolete Prometheus `navidromedb` service. The role requires the storage role's `zpool/media/music`,
`zpool/archive/app_data`, `zpool/archive/app_data/navidrome`, and `zpool/archive/app_data/syncthing` datasets at
their exact paths. It never creates the pool.
- Keep `backend_phase1_start_services` false until the stopped Prometheus Navidrome data directory has been copied to
Atlas and its SQLite database verified. The playbook renders the target but never migrates or deletes application
data; after cutover, set the flag true to enable and start Navidrome and Syncthing.
- Phase 1 must not change Prometheus' existing NPM deployment. NPM continues to be managed exactly by `profile_server`;
use `10.0.0.2:4533` for Navidrome and `10.0.0.2:8384` for the Syncthing GUI. Syncthing does not use host networking:
its GUI, transfer, QUIC and discovery ports are explicitly published only on `10.0.0.2`; native transfer/discovery does not use the HTTP proxy.
- `wireguard_overlay` manages the required `wg0` path between Prometheus and Atlas, persists private keys only on their
respective hosts, exchanges only derived public keys, and verifies a real peer handshake. The initial run must
include both hosts. Prometheus
opens `51820/udp`; after creating the WireGuard firewalld zone, restore Prometheus' rootful Podman networking with
`podman network reload --all` so the existing proxy stack retains container DNS. The Atlas backend role admits
service ports only in the WireGuard firewalld zone.
## Atlas NAS TODO
- Replace every Atlas `CHANGEME` value, provide the required Vault variables and validate the first
remote bootstrap on the real Rocky Linux 9 host. Enable `atlas_manage_storage` first and
`atlas_manage_firewall` only after confirming the pool, mountpoints, LAN subnet and firewalld zone.
- Provide the required Vault variables and validate the first remote bootstrap on the real Rocky Linux 9 host.
Before the first apply, confirm the pool, mountpoints, LAN subnet and firewalld zone. Keep
`atlas_manage_media_stack` disabled until `/dev/dri`, the container paths and the Immich database secret are validated.
- Validate the complete baseline on the target: OpenZFS kmod loading, existing pool import, dataset
mounts, SSH reconnect, Cockpit and all selected 45Drives plugins, NFSv4, SMB and Syncthing.
- Finalize dataset properties and the shared UID/GID, group and POSIX ACL model; test the same files
@@ -140,10 +179,11 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
or manual operations, not as the only source of configuration, and never automate snapshot rollback.
- Manage the Syncthing star topology, device IDs, folders, folder modes, ignore rules and protected GUI
or API access for the selected clients.
- Add Tailscale or WireGuard and corresponding LAN/VPN-only firewalld rules before enabling remote
services; never expose SSH, Cockpit, NFS, SMB or Syncthing through public port forwarding.
- Add the least-privilege Prometheus backup flow: remote dump generation, dedicated SSH identity,
pinned host key, atomic pull, verification, retention and an Atlas systemd service/timer.
- Validate the managed WireGuard path and its LAN/VPN-only firewalld rules before enabling remote services;
never expose SSH, Cockpit, NFS, SMB or Syncthing through public port forwarding.
- Add the Atlas-initiated least-privilege Prometheus backup pull: Prometheus exposes only prepared
read-only dumps through a dedicated account and Atlas retains the private SSH key, pinned host key,
atomic pull, verification, retention and systemd service/timer.
- Add the encrypted offsite backup with Borg to a Hetzner Storage Box: use a dedicated SSH identity,
pin the host key, keep Borg repository credentials and encryption material in Vault, use
snapshot-consistent sources, and manage retries, logging, pruning, repository checks and restores.