23 Commits

Author SHA1 Message Date
Fabio Scotto di Santolo
fa1c8c0b82 Document Atlas NAS backups and monitoring 2026-09-25 14:06:28 +02:00
Fabio Scotto di Santolo
0b6efc9ad8 Harden Atlas USB backup cleanup and document restorecon workflow 2026-09-25 08:02:44 +02:00
Fabio Scotto di Santolo
361ee77d72 Add manual Atlas USB backup and reminder services 2026-09-24 08:55:52 +02:00
Fabio Scotto di Santolo
d4e40d423a Document deferred Cerberus management node plan 2026-09-23 09:01:18 +02:00
Fabio Scotto di Santolo
0a5c2ac1a4 Add progress logging to Atlas Borg backups 2026-09-22 21:21:21 +02:00
Fabio Scotto di Santolo
48a7f57f7e Add Cockpit storage management to Atlas and Prometheus 2026-09-22 20:49:18 +02:00
Fabio Scotto di Santolo
defa98c968 Install Cockpit Podman plugin on Atlas 2026-09-22 15:00:10 +02:00
Fabio Scotto di Santolo
21e41f4fc1 Prepare Atlas encryption dependencies and update Vault placeholders 2026-09-22 14:37:25 +02:00
Fabio Scotto di Santolo
e10c6694f8 Document populated Atlas archive and pending Borg evaluation 2026-09-21 22:15:42 +02:00
Fabio Scotto di Santolo
4c10af3187 Add openssl to Fedora base packages 2026-09-21 12:07:50 +02:00
Fabio Scotto di Santolo
3ac732751c Add pinned Java 25 to Mise on WSL workstation 2026-09-21 11:57:51 +02:00
Fabio Scotto di Santolo
e837b0059b Document validated Atlas Borg backup activation 2026-09-18 10:09:12 +02:00
Fabio Scotto di Santolo
7e498514dd Add encrypted Borg backups for Atlas 2026-09-18 10:00:45 +02:00
Fabio Scotto di Santolo
e7836ea25f Manage Atlas ZFS snapshots and scrubs 2026-09-17 23:10:53 +02:00
Fabio Scotto di Santolo
de2c24d15c Mount the shared Archive in Syncthing 2026-09-17 19:41:23 +02:00
Fabio Scotto di Santolo
bd13cb65cc Move temporary Navidrome and Syncthing services to Atlas 2026-09-17 10:18:23 +02:00
Fabio Scotto di Santolo
77afdda0a3 Move WireGuard gateway to Aegis 2026-09-17 09:08:25 +02:00
Fabio Scotto di Santolo
add75d74e9 refactor(atlas): separate service and host backup datasets 2026-09-17 00:06:46 +02:00
Fabio Scotto di Santolo
e46a2b22d7 Update Atlas NAS validation status and remaining TODOs 2026-09-16 10:25:38 +02:00
Fabio Scotto di Santolo
a97c64b08f Configure Aegis host DNS and NFS support 2026-09-16 10:19:06 +02:00
Fabio Scotto di Santolo
160d63c02d Add Atlas media and storage services [Phase 1] (#9)
* Add Atlas media and storage services

* Document Atlas backend phase one and WireGuard deployment

* Enable Atlas NAS management and document bootstrap workflow

* Harden Atlas network, SSH, firewall, and sharing

* Rotate Ansible Vault secrets

* Allow configurable Aegis SSH users and authorized keys

* Manage Aegis SSH authorized key fragments

* Manage SSH authorized key fragments for infrastructure hosts

* Harden Rocky storage and sharing configuration

* Verify WireGuard handshakes and restore Podman networking
2026-09-15 22:39:28 +02:00
Fabio Scotto di Santolo
73bf2cd62a feature: add WSL container tooling 2026-09-15 16:38:33 +02:00
Fabio Scotto di Santolo
e8d059c8ed Install ChatGPT desktop on Fedora desktops 2026-09-11 21:27:48 +02:00
68 changed files with 3956 additions and 413 deletions

209
AGENTS.md
View File

@@ -48,6 +48,8 @@ Ansible-driven personal infrastructure repo for Fedora and Void desktops, Fedora
- Rocky server after activation: `ansible-playbook ansible/site.yml --limit <host> --check --diff`
- Atlas NAS: `ansible-playbook ansible/site.yml --limit atlas --check --diff`
- Aegis IoT: `ansible-playbook ansible/site.yml --limit aegis --check --diff`
- Aegis NFS client layer: `ansible-playbook ansible/site.yml --limit aegis --tags nfs --list-tasks`
- Aegis host DNS: `ansible-playbook ansible/site.yml --limit aegis --tags dns --check --diff`
- Focused checks:
- Emacs is disabled by default; temporary Emacs check: `ansible-playbook ansible/site.yml --limit <host> --tags emacs --check --diff -e emacs_enabled=true`
- AI coding agents: `ansible-playbook ansible/site.yml --limit <host> --tags ai_agents --check --diff`
@@ -57,10 +59,20 @@ Ansible-driven personal infrastructure repo for Fedora and Void desktops, Fedora
`ansible-playbook ansible/site.yml --limit atlas --tags storage,sharing,containers --check --diff`
- Atlas network/share hardening:
`ansible-playbook ansible/site.yml --limit atlas --tags hardening,sharing --check --diff`
- Atlas phase-one rootless services:
`ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff`
- Prometheus/Atlas WireGuard overlay:
`ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard --check --diff`
- Atlas ZFS snapshot retention and scrub timers:
`ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff`
- Atlas encrypted Borg backup:
`ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff`
- Atlas Borg progress logging only:
`ansible-playbook ansible/site.yml --limit atlas --tags borg_logging --check --diff`
- Atlas manual offline USB backup and 45Drives Alerts reminder:
`ansible-playbook ansible/site.yml --limit atlas --tags usb_backup,usb_reminder --check --diff`
- Atlas pool, disk, capacity, temperature, and job monitoring:
`ansible-playbook ansible/site.yml --limit atlas --tags monitoring --check --diff`
- Atlas explicit post-restore SELinux relabeling:
`ansible-playbook ansible/site.yml --limit atlas --tags restorecon --check -e '{"atlas_restorecon_paths":["/zpool/archive"]}'`
- Prometheus/Aegis WireGuard gateway:
`ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff`
- DuckDNS config only: `ansible-playbook ansible/site.yml --limit prometheus --tags duckdns --check --diff`
## Conventions
@@ -137,10 +149,10 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
- Atlas requires `vault_atlas_admin_password_hash` for Cockpit and, while sharing is enabled,
`vault_atlas_samba_password`. The future rootful media stack also requires
`vault_atlas_immich_db_password`. Never print these values.
- Atlas creates the complete declared hierarchy only under the verified existing or explicitly bootstrapped pool: `work`, `archive`,
`archive/app_data`, `archive/app_data/navidrome`, `archive/app_data/syncthing`, `media`, `media/music`,
`media/photobook`, `backups`, `backups/services`, and `backup_prometheus`. `backups/services` has a `500G`
refreservation. There is no separate legacy `zpool/syncthing` dataset.
- Atlas creates the complete declared hierarchy only under the verified existing or explicitly bootstrapped pool: `archive`,
`services`, `services/data`, `services/data/navidrome`, `services/data/syncthing`, `media`, `media/music`,
`media/photobook`, `backup`, `backup/hosts`, and `backup/hosts/prometheus`. `backup` has a `500G`
reservation covering its descendants. `archive` is the SMB-shared raw-data namespace; container state is never beneath it.
- The `immich` system account is fixed to UID/GID `1100`, has no login shell or `wheel` membership, and receives only
the `video` and `render` supplementary groups. Immich's rootful Quadlets run as `1100:1100`; Server and ML receive
`/dev/dri`, while the Photobook external library is read-only at `/external/photobook`.
@@ -149,52 +161,136 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
`1100`. Targeted SELinux is enforced persistently; a required reboot is reported but never initiated automatically. The primary LAN interface is assigned explicitly to the managed firewalld zone, and firewall rules are applied before NFS or SMB are started; their service state and TCP listeners are then verified. SMB3 exposes `Archive` to Vault-backed authorized accounts on mandatory encrypted, signed SMB3 over TCP/445 only and admits the configured LAN without host-specific exclusions.
- Atlas NPM and Immich share a rootful Podman network. NPM publishes HTTP/HTTPS, but its administration port remains
bound to `127.0.0.1:81`; do not expose it directly to the LAN or Internet.
- `profile_backend_phase1` is limited to rootless Navidrome and Syncthing user Quadlets on Atlas. Official Navidrome
`0.63.2` uses SQLite below `/data` and does not support `ND_DATABASE_URL` or an external PostgreSQL backend; do not
recreate the obsolete Prometheus `navidromedb` service. The role requires the storage role's `zpool/media/music`,
`zpool/archive/app_data`, `zpool/archive/app_data/navidrome`, and `zpool/archive/app_data/syncthing` datasets at
their exact paths. It never creates the pool.
- Keep `backend_phase1_start_services` false until the stopped Prometheus Navidrome data directory has been copied to
Atlas and its SQLite database verified. The playbook renders the target but never migrates or deletes application
data; after cutover, set the flag true to enable and start Navidrome and Syncthing.
- Phase 1 must not change Prometheus' existing NPM deployment. NPM continues to be managed exactly by `profile_server`;
use `10.0.0.2:4533` for Navidrome and `10.0.0.2:8384` for the Syncthing GUI. Syncthing does not use host networking:
its GUI, transfer, QUIC and discovery ports are explicitly published only on `10.0.0.2`; native transfer/discovery does not use the HTTP proxy.
- `wireguard_overlay` manages the required `wg0` path between Prometheus and Atlas, persists private keys only on their
respective hosts, exchanges only derived public keys, and verifies a real peer handshake. The initial run must
include both hosts. Prometheus
opens `51820/udp`; after creating the WireGuard firewalld zone, restore Prometheus' rootful Podman networking with
`podman network reload --all` so the existing proxy stack retains container DNS. The Atlas backend role admits
service ports only in the WireGuard firewalld zone.
- `profile_backend_phase1` temporarily runs rootless Navidrome and Syncthing on Atlas until Uranus replaces
them. It binds only to Atlas' LAN IP, never `wg0`; Navidrome and the Syncthing GUI admit only Aegis as
the source-NAT gateway, while native Syncthing ports admit the configured LAN. It initializes fresh
state only and never migrates or deletes source application data.
- `wireguard_overlay` manages `wg0` between Prometheus (`10.0.0.1`) and Aegis (`10.0.0.2`). It persists private
keys only on their respective hosts, exchanges only derived public keys through Ansible, and verifies a real peer
handshake. Prometheus opens `51820/udp`; Aegis is the LAN gateway. Its persistent IPv4 forwarding, narrowly scoped
WireGuard-to-LAN firewalld policy, and source masquerading permit Prometheus to reach LAN services without a static
route on the router. Prometheus includes `192.168.178.0/24` in Aegis' peer `AllowedIPs`; add the Uranus VIP there
when it is assigned. After a firewalld reload, restore Prometheus' rootful Podman networking with
`podman network reload --all` so the existing proxy stack retains container DNS.
## Atlas NAS TODO
- Provide the required Vault variables and validate the first remote bootstrap on the real Rocky Linux 9 host.
Before the first apply, confirm the pool, mountpoints, LAN subnet and firewalld zone. Keep
`atlas_manage_media_stack` disabled until `/dev/dri`, the container paths and the Immich database secret are validated.
- Validate the complete baseline on the target: OpenZFS kmod loading, existing pool import, dataset
mounts, SSH reconnect, Cockpit and all selected 45Drives plugins, NFSv4, SMB and Syncthing.
- Finalize dataset properties and the shared UID/GID, group and POSIX ACL model; test the same files
through both NFS and SMB before considering multiprotocol access complete.
- Add Ansible-managed ZFS snapshot retention and scrub timers. Use Cockpit Scheduler for visibility
or manual operations, not as the only source of configuration, and never automate snapshot rollback.
- Manage the Syncthing star topology, device IDs, folders, folder modes, ignore rules and protected GUI
or API access for the selected clients.
- Validate the managed WireGuard path and its LAN/VPN-only firewalld rules before enabling remote services;
never expose SSH, Cockpit, NFS, SMB or Syncthing through public port forwarding.
- Add the Atlas-initiated least-privilege Prometheus backup pull: Prometheus exposes only prepared
Completed validation: the existing RAIDZ2 pool and datasets, SELinux, LAN firewall, SSH, Cockpit with
the selected 45Drives plugins, encrypted SMB3 `Archive`, the Aegis-only NFSv4 `photobook` export, and the
Prometheus--Aegis WireGuard gateway are operational. The gateway handshake, forwarding, source masquerading,
and TCP reachability to Atlas were verified. Temporary Navidrome and Syncthing are available through their
manual NPM Proxy Hosts; Syncthing uses `/data/Org` backed by the SMB-shared Archive dataset. Aegis has also
validated NFSv4.2 read, write, delete, and `all_squash` mapping to UID/GID `1100` end-to-end. The ZFS
snapshot timers are active and the first recursive hourly snapshot completed successfully; the first
scheduled retention prune and monthly scrub remain runtime checks.
### Priority 1 - Data protection
- [x] Deploy Ansible-managed recursive ZFS snapshots with 24 hourly, 30 daily, 8 weekly, and 12 monthly
generations, plus a monthly scrub on the first Sunday at 03:00. The timers and first hourly snapshot were
verified on Atlas. Still observe the first scheduled retention prune and scrub; Cockpit Scheduler is for
visibility or manual operations only, and snapshot rollback is never automated.
- [x] Activate and validate the encrypted offsite Borg backup to the Hetzner Storage Box. Atlas uses the
dedicated SSH identity, pinned ED25519 host key, Vault-backed `repokey` encryption, and a locked
non-login `borg` account with no sudo or supplementary groups. The initial snapshot-consistent backup,
Borg repository check, and temporary-directory restore completed successfully; the restored `Archive`
tree matched the live data, and temporary snapshots and mounts were removed. The exported recovery key
was copied offline. Daily backup retries and logging, 30 daily, 8 weekly and 12 monthly archives,
compaction, and monthly repository checks are enabled. Future runs report a ZFS-based estimated
percentage, and a post-exit helper handles host-namespace temporary snapshot cleanup. The active
run predates the new progress logging and still requires an observed final cleanup result.
- [x] Populate `/zpool/archive` with the currently available data so offsite and offline backup tests run
against a representative load.
- [ ] Run and evaluate Borg against the populated pool: duration, repository capacity, deduplication, and
a subsequent incremental archive must be observed before considering the offsite path fully validated.
- [x] Add the UUID-bound offline USB backup with versioned rsync, locking, capacity checks, verification,
safe unmounting and a tested restore procedure; never trigger it for an arbitrary USB disk. The
LUKS/ext4 identities were read-only verified; the manual service and 45Drives Alerts reminder timer were
deployed on Atlas. Interactive LUKS unlock is part of the manual service; only the reminder is
scheduled for the first Saturday of each month at 10:00 Europe/Rome via the existing 45Drives
notifier. A manual test produced an Alerts notification, not an email. The first USB attempt failed
on a `security.selinux` xattr and was interrupted; the xattr filter is deployed and the temporary
recursive snapshot and open LUKS mapper were cleaned up. A later run reported checksum verification
and published the USB version, but failed while removing host-namespace ZFS snapshot mounts. Those
exact mounts and snapshots were cleaned up. An `ExecStopPost` helper now removes only the named
temporary snapshot after the backup process exits. A new full run checksum-verified and published a
USB version; the service ended successfully, the mapper closed, no temporary USB snapshot remained,
and the pool was healthy. On 2026-09-25 an independent, read-only USB restore test copied one file from
the published `atlas/latest` version into `/var/tmp` and matched contents, owner, mode, size, mtime and
POSIX ACL. The temporary copy and mount were removed, the mapper closed, and the pool remained healthy.
- [x] Test restores independently from a ZFS snapshot, Borg, and the offline USB backup before relying on
any backup path. The earlier Borg temporary-directory restore passed. On 2026-09-25 a separate,
read-only ZFS snapshot test restored one file to `/var/tmp`, confirmed matching contents, ownership,
mode, mtime and ACL, then removed its temporary copy and on-demand mount. This is a file-level smoke
test, not full dataset recovery. An independent USB file restore passed on 2026-09-25 with matching
content and metadata; full disaster recovery remains a separate Priority 2 task.
- [x] Add monitoring and alerting for pool health, scrub/resilver, SMART data, temperatures, free space,
snapshot/local-backup growth, Hetzner Storage Box quota, and failed maintenance or backup timers.
The half-hourly Atlas health monitor and systemd final-failure hooks are deployed. A live probe
found no issues; the service and timer succeeded, and a labelled 45Drives Alerts test notification
was submitted. Alerts are deduplicated; email delivery is not claimed. The Storage Box quota probe
runs `df -m` over the dedicated pinned-key SSH identity and does not open the Borg repository.
Detailed archive size and deduplication remain part of the pending Borg evaluation.
### Priority 2 - NAS operability and recovery
- [ ] Document and test disaster recovery: rebuild Atlas with Ansible, import the existing pool, restore
from snapshot/USB/Hetzner, preserve Vault and Borg recovery material offline, and define RPO/RTO.
- [ ] Define a controlled Rocky kernel/OpenZFS update and reboot procedure.
- [ ] Add the Atlas-initiated least-privilege Prometheus backup pull: Prometheus exposes only prepared
read-only dumps through a dedicated account and Atlas retains the private SSH key, pinned host key,
atomic pull, verification, retention and systemd service/timer.
- Add the encrypted offsite backup with Borg to a Hetzner Storage Box: use a dedicated SSH identity,
pin the host key, keep Borg repository credentials and encryption material in Vault, use
snapshot-consistent sources, and manage retries, logging, pruning, repository checks and restores.
- Add the UUID-bound offline USB backup with versioned rsync, locking, capacity checks, verification,
safe unmounting and a tested restore procedure; never trigger it for an arbitrary USB disk.
- Add monitoring and alerting for pool health, scrub/resilver, SMART data, temperatures, free space and
failed backup timers, plus a controlled Rocky kernel/OpenZFS update and reboot procedure.
- Document and test disaster recovery: rebuild Atlas with Ansible, import the existing pool, restore
from snapshot/USB/Hetzner, preserve Vault and Borg recovery material offline, and define RPO/RTO.
- Optionally design iCloud photo ingestion as a separate workflow after the storage and backup layers
are validated; do not make it a dependency of the Atlas baseline.
- [ ] Decide whether a common SMB/NFS namespace is required. `Archive` (SMB) and `photobook` (NFS) are
intentionally distinct today; only if a shared namespace is selected, finalize its UID/GID, group,
and POSIX ACL model and test the same files through both protocols.
### Priority 3 - Service expansion
- [ ] After data protection and recovery are validated, populate `/zpool/media/music` and validate Navidrome.
- [ ] Design and deploy Nextcloud as another explicitly temporary Atlas service before Uranus. Give it
separate persistent application, database, and cache storage; keep credentials in Vault; publish it only
through NPM over the Prometheus--Aegis gateway; and define backup, upgrade, and eventual Uranus-migration
procedures before exposing user data. Do not deploy Nextcloud before the data-protection checklist is complete.
- [ ] Keep `atlas_manage_media_stack` disabled until the future Immich deployment has validated `/dev/dri`,
container paths, and the required Vault database secret.
### Priority 4 - Optional workflows
- [ ] After data protection is validated, move iCloudPD photo ingestion from Aegis to Atlas as a
temporary service until Uranus is ready. Plan to store photos in `/zpool/archive/Pictures` and
persistent application/MFA state outside `Archive`; validate permissions, SELinux, backups and
recovery before cutover. Keep the current Aegis service and Photobook NFS export unchanged until
the Atlas workflow is tested, then retire them explicitly if no longer needed.
## Cerberus Management Node (Deferred)
`cerberus` is postponed until the office in the new house is physically set up. It is not an inventory
host and this section is a design and implementation backlog, not authorization to provision it early.
The planned node is a Lenovo ThinkCentre M700 Tiny with an Intel Core i3-6100T, 8 GB RAM, a 256 GB SSD,
and native 1 Gbps Ethernet. It will connect to a multi-input KVM switch using a passive DisplayPort-to-HDMI
cable, sharing the monitor and peripherals with Ikaros. Fedora Sericea (immutable Fedora with the Sway
Wayland compositor) is the intended OS. Cerberus is an isolated management plane: a dedicated Toolbox
environment will run Ansible for future `uranus` cluster provisioning. Rootless Podman will host Grafana,
Prometheus, and Loki. The 256 GB local SSD is the hot tier retaining metrics and logs for 30 days; scheduled,
validated exports of older historical data will use a dedicated Atlas NFS dataset as cold storage.
### Implementation plan
- [ ] Confirm the office, KVM switch, passive DisplayPort-to-HDMI path, shared monitor/peripherals, and native
1 Gbps Ethernet are physically operational before adding Cerberus to inventory.
- [ ] Install and update Fedora Sericea with Sway; document the immutable-host lifecycle and keep host changes
declarative rather than treating the base OS as a mutable workstation.
- [ ] Model Cerberus as its own host with independent platform, role, desktop, network, and storage inputs;
do not repurpose Ikaros variables or make it a Uranus cluster member.
- [ ] Provision an isolated Toolbox-based Ansible controller with the required collections and a reproducible
project checkout; define its least-privilege SSH access, known-host handling, and Vault workflow without
storing secrets in the image or repository.
- [ ] Define the explicit Uranus provisioning workflow from Cerberus, including inventory boundaries,
validation-only runs, and separate approval for any destructive cluster operation.
- [ ] Design rootless Podman/Quadlet services for Grafana, Prometheus, and Loki, including persistent local
state, service ownership, LAN exposure/authentication, resource limits, updates, and backups.
- [ ] Size and enforce a 30-day local hot-retention policy for metrics and logs on the 256 GB SSD; validate
actual disk growth and alert before capacity exhaustion.
- [ ] Create and validate a dedicated Atlas NFS cold-storage dataset and least-privilege export for Cerberus;
do not use a broad existing share or couple it to unrelated Atlas application state.
- [ ] Implement scheduled, idempotent exports of data older than 30 days to the Atlas NFS cold tier, with
locking, capacity checks, integrity verification, retention rules, failure monitoring, and a tested restore.
- [ ] Validate management-plane recovery: rebuild Cerberus, restore observability history from Atlas, and
confirm that Uranus provisioning can resume without depending on unreproducible local state.
## Coding Agent Notes
- Shared agent definitions and lifecycle flags live in `ai_agents` in `ansible/inventory/group_vars/all.yml`.
@@ -221,14 +317,19 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
- `aegis` is a remote Fedora IoT Raspberry Pi 4 node. Bootstrap it once with
`ansible/bootstrap/aegis.bu`; the remaining configuration is applied by `profile_aegis` over SSH.
- Fedora IoT is immutable. Do not add it to mutable Fedora package or shared dotfile roles.
- `profile_aegis` owns rootful Podman Quadlets, persistent container state under `/var/lib`, the
Podman auto-update timer, LAN-restricted firewalld rules, and SSH hardening. Keep
`aegis_lan_subnet` and `aegis_adguard_web_port` host-specific; SSH permits only the declared
- `profile_aegis` owns the `nfs-utils` and `wireguard-tools` rpm-ostree layers and reports the required reboot
without initiating it. `wireguard_overlay` then configures Aegis as the WireGuard LAN gateway with persistent IPv4
forwarding, a scoped inter-zone policy, and source masquerading. It also owns rootful Podman Quadlets, persistent container
state under `/var/lib`, the Podman auto-update timer, LAN-restricted firewalld rules, and SSH hardening. Keep
`aegis_lan_subnet`, `aegis_adguard_web_port`, and `aegis_network_connection_uuid` host-specific;
SSH permits only the declared
key-authenticated users, never root or password authentication. Keep Apple IDs and other
credentials in Vault and use `no_log` for their rendering.
- `aegis_adguard_web_port` defaults to `80`. The initial AdGuard Home wizard port `3000` is intentionally unmanaged: open and close it manually only while
completing initial setup. Disable the local systemd-resolved stub through `profile_aegis` before
AdGuard binds port 53; keep
`/etc/resolv.conf` linked to `/run/systemd/resolve/resolv.conf` so Aegis retains router-provided DNS.
`/etc/resolv.conf` linked to `/run/systemd/resolve/resolv.conf`. LAN clients may use AdGuard, but
Aegis must use the independent upstream DNS declared by `aegis_host_dns_servers` so Greenboot does
not depend on the AdGuard container during startup.
- iCloudPD requires post-deployment interactive MFA initialization; its cookie/configuration state is
persisted in `/var/lib/icloudpd/config`.

View File

@@ -95,6 +95,27 @@ Nota sullo stato attuale del playbook principale:
- `ansible/site.yml` applica il profilo server Rocky a `prometheus` con DNF, systemd, dotfiles server e firewalld
- `ansible/site.yml` applica il profilo NAS Rocky su `atlas` tramite SSH remoto
## Nodo pianificato e posticipato: Cerberus
`cerberus` e un nodo di management **posticipato**, in attesa dell'allestimento
fisico dell'ufficio nella nuova casa. Non e ancora presente nell'inventory e non
esistono ruoli o playbook che lo prendano come target.
L'hardware previsto e un Lenovo ThinkCentre M700 Tiny (Intel Core i3-6100T,
8 GB di RAM e SSD da 256 GB) con Ethernet nativa a 1 Gbps. Condividera monitor
e periferiche di Ikaros tramite uno switch KVM a ingressi multipli, usando un
cavo passivo DisplayPort-HDMI per il collegamento video. Il sistema operativo
previsto e Fedora Sericea, la variante Fedora immutabile con compositor Wayland
Sway.
Cerberus sara un management plane isolato: Ansible verra eseguito in un ambiente
Toolbox dedicato per il provisioning del futuro cluster `uranus`, anziche da
Ikaros o da un host non gestito. Lo stack di osservabilita rootless Podman
eseguira Grafana, Prometheus e Loki. L'SSD locale sara l'hot storage, con
metriche e log conservati per 30 giorni; esportazioni programmate trasferiranno
i dati storici piu vecchi su un dataset Atlas montato via NFS come cold storage.
Il piano di implementazione, con prerequisiti espliciti, e in `AGENTS.md`.
## Desktop
Target operativi:
@@ -246,90 +267,252 @@ ansible-playbook ansible/site.yml --limit prometheus -e server_username=myuser -
## NAS
`atlas` e un NAS Rocky Linux 9 raggiunto tramite SSH. Normalmente il pool ZFS esiste gia e il profilo
gestisce solo i dataset figli. Un bootstrap RAIDZ2 una tantum e disponibile solo con conferma esplicita
(`atlas_create_pool=true`) e quattro percorsi reali e verificati `/dev/disk/by-id/...` in
`atlas_zpool_disks`. Non partiziona, forza, distrugge, esegue rollback o modifica il layout vdev di un
pool esistente. I client Linux usano NFSv4, quelli Windows/WSL SMB; entrambi restano limitati alla LAN
configurata.
`atlas` è un NAS Rocky Linux 9 raggiunto via SSH. Normalmente il pool esiste già e il profilo gestisce
solo i dataset figli. La creazione iniziale del RAIDZ2 richiede esplicitamente `atlas_create_pool=true`
e quattro percorsi `/dev/disk/by-id/...` verificati in `atlas_zpool_disks`. Il ruolo non partiziona,
forza, distrugge, ripristina né modifica il layout vdev di un pool esistente. I client Linux usano NFSv4,
quelli Windows/WSL SMB; l'accesso è limitato alla LAN configurata.
Per il primo avvio fornire `vault_atlas_authorized_ssh_keys`, `vault_atlas_admin_password_hash`,
`vault_atlas_samba_password` e `vault_atlas_immich_db_password`. Eseguire il bootstrap tramite
l'amministratore esistente:
Per il primo avvio servono `vault_atlas_admin_password_hash`, `vault_atlas_samba_password` e
`vault_atlas_immich_db_password`; il primo è un hash compatibile con `/etc/shadow`, non una password
Cockpit in chiaro. Il bootstrap usa l'amministratore preesistente:
```bash
ansible-playbook ansible/site.yml --limit atlas \
-e atlas_connection_username=<existing-admin>
```
`vault_atlas_admin_password_hash` deve essere un hash compatibile con `/etc/shadow`, non una
password Cockpit in chiaro. Le esecuzioni successive usano `atlas_admin_username`. Atlas dichiara
abilitati storage, condivisioni e regole firewall LAN. Prima della prima applicazione verificare pool e
mountpoint esistenti, subnet LAN e zona firewalld attiva. `atlas_manage_media_stack` resta disabilitato
finche non saranno validati `/dev/dri`, i percorsi dei container e il segreto del database Immich.
Le esecuzioni successive usano `atlas_admin_username`. Storage, condivisioni e firewall LAN sono
abilitati; prima dell'applicazione verificare pool, mountpoint, subnet e zona firewalld. La creazione
del pool è protetta da un gate esplicito e avviene solo se è assente. Atlas non fa più parte della VPN
WireGuard: la vecchia interfaccia è stata ritirata manualmente dopo la verifica del collegamento tra
Prometheus e Aegis. Le chiavi SSH autorizzate sono in file separati sotto
`~/.ssh/authorized_keys.d/`. `atlas_manage_media_stack` resta disabilitato finché `/dev/dri`, percorsi
dei container e segreto del database Immich non sono validati.
Con la gestione storage attiva, Atlas crea l'intera gerarchia sotto il pool `zpool` esistente o creato esplicitamente:
`work`, `archive`, `archive/app_data`, i dataset applicativi separati
`archive/app_data/navidrome` e `archive/app_data/syncthing`, `media`, `media/music`,
`media/photobook`, `backups`, `backups/services` e `backup_prometheus`. I dataset applicativi e
di archivio usano `zstd`; media, Syncthing e backup dei servizi usano `lz4`;
`backups/services` mantiene inoltre una `refreservation` di `500G`.
Atlas impone SELinux targeted in modo persistente e segnala, senza avviarlo, l’eventuale reboot necessario per attivarlo. Assegna esplicitamente l’interfaccia LAN primaria alla zona firewalld gestita e applica hardening persistente del kernel di rete: rifiuta redirect e source-route, registra i martian, usa reverse-path filtering loose per WireGuard e disabilita il forwarding IPv4. SSH consente solo l’amministratore dichiarato tramite chiave pubblica; root, password, agent e forwarding
remoto sono disabilitati, mentre il forwarding locale resta disponibile per tunnel amministrativi privati. SMB3 pubblica `Archive` solo agli account Samba configurati con password in Vault e
ammette la LAN configurata su SMB3 cifrato e firmato, esclusivamente su TCP/445. NFSv4 esporta soltanto
`media/photobook` all'IP configurato di Aegis su TCP/2049, con `all_squash` verso UID/GID anonimi `1100`.
Sotto `zpool` Atlas crea `archive` (SMB), `services/data` con i dataset applicativi
`services/data/navidrome` e `services/data/syncthing`, `media`, `media/music`, `media/photobook` e
`backup/hosts/prometheus`. Archivio e applicazioni usano `zstd`; media, Syncthing e backup host usano
`lz4`. `backup` ha una riserva di `500G` che copre i discendenti. SELinux targeted è persistente;
l'eventuale riavvio necessario viene segnalato, non eseguito. Atlas assegna l'interfaccia primaria
alla zona firewalld gestita, rifiuta redirect e source route, registra i martian, mantiene il reverse-path
filter loose e disabilita il forwarding IPv4. SSH consente soltanto l'amministratore dichiarato con
chiave pubblica: root, password, agent forwarding e remote forwarding sono disabilitati, mentre il
forwarding locale resta disponibile per i tunnel amministrativi. SMB3 espone `Archive` agli account
autorizzati da Vault sulla LAN, solo su TCP/445 con cifratura e firma obbligatorie. NFSv4 espone
soltanto `media/photobook` all'IP di Aegis su TCP/2049, con `all_squash` verso UID/GID `1100`.
L'account di sistema `immich` usa UID/GID `1100`, shell senza login, nessuna appartenenza a `wheel` e
i gruppi supplementari `video` e `render`. I Quadlet rootful di Immich Server, ML, cache compatibile
Redis, PostgreSQL e NPM condividono una rete Podman. Immich viene eseguito come `1100:1100`; Server e
ML ricevono `/dev/dri` e Photobook e montato in sola lettura su `/external/photobook`. NPM pubblica `80` e
`443`, mentre l'amministrazione resta vincolata a `127.0.0.1:81` per l'accesso tramite tunnel SSH.
L'account di sistema `immich` usa UID/GID `1100`, non ha shell di login né gruppo `wheel` e riceve i
gruppi `video` e `render`. Lo stack Immich futuro prevede Quadlet rootful per Server, ML, cache,
PostgreSQL e NPM su una rete Podman comune. Immich gira come `1100:1100`, Server e ML ricevono
`/dev/dri` e Photobook è montato in sola lettura su `/external/photobook`. NPM pubblica `80` e `443`;
l'interfaccia amministrativa resta su `127.0.0.1:81`, raggiungibile via tunnel SSH.
La fase 1 e limitata ai Quadlet utente rootless di Navidrome e Syncthing su Atlas. E abilitata nella
configurazione host di Atlas e puo essere impostata a `false` solo per una sospensione intenzionale. Navidrome ufficiale `0.63.2` usa il database SQLite sotto `/data` e
non supporta `ND_DATABASE_URL` ne un backend PostgreSQL esterno. Il servizio obsoleto `navidromedb`
e quindi rimosso da Prometheus invece di essere replicato su Atlas. Il ruolo deriva i percorsi dal
pool `zpool`, montato in `/zpool`: musica in sola lettura da `/zpool/media/music`, stato
applicativo Navidrome e `navidrome.db` in `/zpool/archive/app_data/navidrome` e dati Syncthing in
`/zpool/archive/app_data/syncthing`. `profile_atlas` crea questi dataset quando
`atlas_manage_storage` e attivo; il ruolo backend verifica i mountpoint esatti prima di avviare i
container. Il ruolo backend non crea mai il pool. Il ruolo separato `wireguard_overlay`
gestisce `wg0` tra Prometheus (`10.0.0.1`) e Atlas (`10.0.0.2`), genera una sola volta le chiavi
private sui rispettivi host e scambia tramite Ansible soltanto quelle pubbliche. Solo Prometheus apre
pubblicamente `51820/udp`. Le porte backend sono ammesse esclusivamente nella zona firewalld WireGuard.
Atlas ospita temporaneamente Navidrome e Syncthing rootless fino alla sostituzione con Uranus. I
servizi sono inizializzati **ex novo**, senza migrare lo stato precedente, rispettivamente sotto
`/zpool/services/data/navidrome` e `/zpool/services/data/syncthing`; la musica in
`/zpool/media/music` viene popolata separatamente. Sono vincolati all'indirizzo LAN di Atlas
(`192.168.178.55`), mai a WireGuard. `wireguard_overlay` collega invece Prometheus (`10.0.0.1`)
e Aegis (`10.0.0.2`): le chiavi private restano sui rispettivi host e Ansible scambia solo le pubbliche.
Prometheus apre `51820/udp`; Aegis inoltra soltanto il traffico overlay→LAN dichiarato e applica
source NAT, evitando interfacce VPN su Atlas/Uranus e route statiche sul router. Navidrome (`4533/tcp`)
e la GUI Syncthing (`8384/tcp`) ammettono solo Aegis, mentre le porte native Syncthing sono limitate
alla LAN. Dopo la verifica dei servizi, configurare manualmente i Proxy Host NPM verso
`http://192.168.178.55:4533` e `http://192.168.178.55:8384`. Il peer Prometheus include la LAN
negli `AllowedIPs`; aggiungere la VIP Uranus quando esisterà. Dopo il reload di firewalld, Ansible
ricarica le reti Podman rootful di Prometheus per conservare DNS e connettività del proxy.
`backend_phase1_start_services` resta falso durante il trasferimento dello stato applicativo, quindi
la prima esecuzione reale del backend genera i Quadlet senza creare un database Atlas vuoto. Dopo aver
arrestato Navidrome su Prometheus, copiare l'intera directory `/opt/navidrome/data/` in
`/zpool/archive/app_data/navidrome/`, preservando `navidrome.db` e gli eventuali file SQLite laterali.
Impostare quindi questa variabile a vero e rieseguire il ruolo per abilitare e avviare Navidrome e
Syncthing. Il playbook non copia e non elimina mai i dati applicativi.
Validare e generare i servizi Atlas con:
Validare il gateway con:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags storage
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1
ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff
```
Per il cutover, arrestare il vecchio Navidrome prima di copiare la sua directory dati, verificare
l'ownership dell'account `admin` su Atlas e confermare la presenza del database SQLite copiato prima
di impostare `backend_phase1_start_services: true` in `host_vars/atlas.yml`. Conservare i dati sorgente
e il container legacy `navidromedb` fermo finche Navidrome su Atlas e una prova di restore non sono
stati validati.
La prima esecuzione reale WireGuard deve includere entrambi i peer. Se Aegis ha appena installato il
layer `wireguard-tools`, riavviarlo manualmente e rieseguire senza `--check`: il ruolo attende un
handshake effettivo.
Restano da completare retention delle snapshot, topologia Syncthing, validazione WireGuard/firewall,
pull di backup da Prometheus, backup cifrati con Borg su una Hetzner Storage Box, backup USB,
monitoraggio e test di disaster recovery. Il backlog operativo dettagliato e in `AGENTS.md`.
Gli snapshot ZFS ricorsivi coprono l'intero pool: 24 orari al minuto 05, 30 giornalieri alle 00:15,
8 settimanali la domenica alle 01:00 e 12 mensili il primo giorno alle 02:00. La retention elimina
solo gli snapshot con prefisso gestito `atlas-auto` e non esegue rollback. Lo scrub OpenZFS mensile è
previsto la prima domenica alle 03:00; il timer settimanale incompatibile è disabilitato. Il primo
snapshot orario ricorsivo è riuscito; la prima pulizia pianificata e il primo scrub schedulato
richiedono ancora una verifica a runtime.
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff
```
Il backup Borg cifrato usa il sub-account Hetzner `u660064-sub1`, il repository relativo `./borg-data`
e Borg remoto 1.4 su SSH porta 23. La chiave ED25519 del server è fissata; una chiave client dedicata
appartiene all'account `borg`, bloccato e senza login, sudo o gruppi supplementari. La chiave privata
resta in `/etc/atlas-borg`; la passphrase proviene da `vault_atlas_borg_passphrase` ed è resa in un
file `0600`. Solo il wrapper root crea snapshot e mount; avvia il client come `borg` con il minimo
accesso temporaneo in lettura, senza concedergli gestione ZFS o sudo.
Il backup giornaliero parte alle 04:30 con un ritardo casuale fino a 30 minuti. Crea uno snapshot ZFS
ricorsivo temporaneo e ricostruisce tutti i dataset sotto `/zpool` in un albero di bind mount in sola
lettura, per inserirli in un unico archivio coerente. Il wrapper smonta ricorsivamente l'albero privato;
un helper `ExecStopPost` mirato rimuove eventuali mount dello snapshot nel namespace host e lo snapshot
temporaneo dopo l'uscita del processo. Borg conserva 30 archivi giornalieri, 8 settimanali e 12
mensili, poi compatta il repository. Il controllo completo di metadati e repository si svolge il 15
di ogni mese alle 06:00. Le operazioni usano un lock comune, journal e retry systemd limitati. Le
nuove esecuzioni riportano al massimo una riga di avanzamento al minuto: percentuale **stimata**,
dataset, file elaborati e byte originali/compressi/deduplicati. Il denominatore è la somma dei
`logicalreferenced` ZFS dello snapshot, non un totale Borg: può superare il 100% e non comprende
retention, compattazione o controlli. Le righe di progresso non riportano i nomi dei file; eventuali
warning possono farlo. Seguire il job con `sudo journalctl -fu atlas-borg-backup.service`; modifiche
all'helper non cambiano un'esecuzione già avviata.
Attivazione iniziale esplicita:
1. Inserire una passphrase unica in `secrets/vault.yml` con `ansible-vault edit`.
2. Generare e mostrare solo la chiave pubblica con
`ansible-playbook ansible/site.yml --limit atlas --tags borg_key`.
3. Installarla nel sub-account Hetzner, poi applicare con
`ansible-playbook ansible/site.yml --limit atlas --tags packages,borg`.
4. Copiare `secrets/recovery/atlas-borg-repokey.export` su un supporto davvero offline: la copia
locale ignorata da Git non è di per sé un backup offline.
Il ruolo inizializza solo un repository `repokey` assente, non accetta password SSH né host key non
fissate e non avvia manualmente il primo backup. Validazione:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff
```
L'attivazione iniziale è riuscita: backup e controllo del repository, restore completo in una
directory temporanea confrontato con l'albero `Archive`, esportazione offline della chiave di recupero
e pulizia di snapshot/mount temporanei. Il 2026-09-25 un test separato da snapshot ZFS giornaliero ha
copiato un file di `/zpool/archive` in `/var/tmp`, verificando contenuto, proprietario, modalità,
mtime e ACL POSIX; copia e mount temporanei sono stati rimossi senza interrompere Borg. Non è un test
di ripristino dell'intero dataset.
Il backup USB offline è distribuito come **servizio solo manuale** (`atlas_manage_usb_backup: true`):
Ansible non formatta, sblocca, monta né avvia automaticamente il disco. Il disco esistente è stato
verificato in sola lettura il 2026-09-23: UUID LUKS `577b3c43-ea37-4611-81a9-39d555cdfbd4`,
UUID ext4 interno `758e2d2e-a427-4797-aad9-39c3a9f17c7e`, mapper `zpool-backup`. All'ispezione
era montato in `/mnt/zpool-backup`; il servizio richiede invece che il mapper **non sia montato** prima
dell'avvio. Se serve, `systemd-ask-password` chiede interattivamente la passphrase LUKS tramite
l'agente di `systemctl start` e la passa direttamente a `cryptsetup`, senza salvarla, esporla negli
argomenti o memorizzarla nella cache. Lo script monta il disco privatamente, crea uno snapshot ZFS
ricorsivo, copia tutti i dataset in `atlas/snapshots/<timestamp>/` con `rsync --link-dest`, verifica
con un dry-run basato sui checksum, aggiorna atomicamente `atlas/latest`, smonta e chiude LUKS. Un
errore non sostituisce `latest` né cancella versioni complete precedenti. Borg e USB possono operare
contemporaneamente su snapshot distinti, ma la lettura concorrente può ridurre il throughput.
La copia USB conserva le ACL ma non gli attributi estesi generici, compreso `security.selinux`: la
policy della destinazione deve ricreare le etichette dopo un restore. Per un percorso esplicito:
```bash
ansible-playbook ansible/site.yml --limit atlas --tags restorecon \
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
```
Il task accetta solo percorsi sotto la radice del pool Atlas, esegue `restorecon -RFv` solo su quelli
indicati ed è altrimenti inattivo; non va lanciato sull'intero pool durante i run ordinari. Le vecchie
versioni USB non vengono eliminate automaticamente senza una retention deliberata. Il controllo di
capacità include il trasferimento stimato e una riserva libera di 10 GiB. Dopo un backup riuscito,
scollegare fisicamente il disco per renderlo davvero offline.
Validare la configurazione senza avviare il backup e, separatamente, un eventuale relabel pianificato:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags usb_backup,usb_reminder --check --diff
ansible-playbook ansible/site.yml --limit atlas --tags restorecon --check \
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
```
Prima dell'avvio manuale smontare in sicurezza `/mnt/zpool-backup`, se ancora montato. Con il mapper
chiuso, `sudo systemctl start atlas-usb-backup.service` chiede la passphrase e avvia il backup; né la
password LUKS né un keyfile vanno in Ansible. Seguire con
`sudo journalctl -fu atlas-usb-backup.service`. **Non esiste un timer di backup USB.** Soltanto
`atlas-usb-reminder.timer` è schedulato il primo sabato del mese alle 10:00 `Europe/Rome`: invia un
promemoria al notifier 45Drives Houston, senza avviare il backup. Un test manuale ha prodotto una
notifica in 45Drives Alerts, **non un'email**; il log conferma l'invio della notifica, non la consegna
di posta. Il primo evento pianificato era il 2026-10-03 alle 10:00 CEST. Controllare timer e risultato
con `systemctl list-timers atlas-usb-reminder.timer` e in 45Drives Alerts.
Il primo tentativo USB del 2026-09-23 fallì su `security.selinux` e, dopo l'interruzione, lasciò
snapshot e mapper aperti. Applicato il filtro rsync, furono rimossi lo snapshot fallito, il mapper
smontato e lo stato failed; non rimase una copia valida di quel tentativo. Un run del 2026-09-24
pubblicò una versione verificata ma fallì nella distruzione dello snapshot a causa di mount
`.zfs/snapshot` aperti nel namespace host. Dopo la pulizia non forzata, è stato aggiunto un helper
`ExecStopPost` mirato e testato con uno snapshot usa-e-getta. Un run successivo del 2026-09-24 ha
verificato i checksum, pubblicato la versione ed è terminato con successo: mapper chiuso, nessuno
snapshot USB temporaneo e pool sano. Il 2026-09-25 un test di restore indipendente ha aperto il disco
in sola lettura, montato ext4 con `ro,noload`, copiato un file di 5.707.945 byte da `atlas/latest` in
una directory vuota sotto `/var/tmp` e confrontato contenuto, proprietario, modalità, dimensione,
mtime e ACL POSIX. Il test ha rimosso copia e mount temporanei, chiuso LUKS e lasciato il pool sano
mentre Borg continuava. È un test su file, non un esercizio completo di disaster recovery.
Il monitoraggio Atlas è eseguito ogni 30 minuti da `atlas-health-monitor.timer`. Sonde in sola
lettura controllano stato/errori del pool e dei vdev, scrub/resilver, SMART dei quattro dischi del
pool e dell'NVMe di sistema, temperature dei dischi e CPU, spazio di sistema/pool/snapshot, crescita
di `zpool/backup` e quota Hetzner tramite `df -m` via SSH con l'account `borg` e la chiave fissata.
La query remota non apre il repository Borg né il suo lock. Gli alert di crescita richiedono una
baseline di circa 24 ore. Sono controllati anche attivazione e freschezza dei timer; hook systemd
`OnFailure` segnalano errori di snapshot, scrub, Borg, USB, promemoria e monitoraggio. Il monitor non
riavvia Borg; avvisa solo se un run supera 14 giorni. Soglie e percorsi stabili dei dischi sono nelle
variabili host. Gli avvisi usano 45Drives Houston con deduplicazione; **la consegna email non è stata
verificata**. Il controllo live del 2026-09-25 non ha trovato problemi; la notifica di prova è stata
inviata e lo Storage Box risultava occupato al 22%. Dimensione dell'archivio Borg e deduplicazione
dettagliata richiedono ancora la fine del backup in corso.
```bash
ansible-playbook ansible/site.yml --limit atlas --tags monitoring --check --diff
sudo /usr/local/libexec/atlas-health-monitor --dry-run
sudo journalctl -u atlas-health-monitor.service -n 100 --no-pager
systemctl list-timers atlas-health-monitor.timer
```
`--dry-run` non invia alert e non modifica lo stato del monitor. Un controllo reale si avvia con
`sudo systemctl start atlas-health-monitor.service`, senza avviare servizi di backup. Per una prova
etichettata di 45Drives Alerts usare
`sudo /usr/local/libexec/atlas-health-monitor --test-notification`.
### Timer systemd di Atlas
Tutti i nove timer gestiti sono abilitati. Gli orari sono locali ad Atlas (`Europe/Rome`); Borg e
monitoraggio aggiungono il ritardo casuale indicato. Tutti hanno `Persistent=true`: un evento perso
viene recuperato quando il timer torna attivo.
| Timer | Pianificazione (`OnCalendar`) | Azione |
| --- | --- | --- |
| `atlas-zfs-snapshot-hourly.timer` | `*-*-* *:05:00` — ogni ora al minuto 05 | Snapshot ricorsivo orario e retention |
| `atlas-zfs-snapshot-daily.timer` | `*-*-* 00:15:00` — ogni giorno alle 00:15 | Snapshot ricorsivo giornaliero e retention |
| `atlas-zfs-snapshot-weekly.timer` | `Sun *-*-* 01:00:00` — domenica alle 01:00 | Snapshot ricorsivo settimanale e retention |
| `atlas-zfs-snapshot-monthly.timer` | `*-*-01 02:00:00` — primo giorno del mese alle 02:00 | Snapshot ricorsivo mensile e retention |
| `zfs-scrub-monthly@zpool.timer` | `Sun *-*-01..07 03:00:00` — prima domenica alle 03:00 | Scrub ZFS |
| `atlas-borg-backup.timer` | `*-*-* 04:30:00` — ogni giorno alle 04:30, più 0–30 min casuali | Backup cifrato offsite |
| `atlas-borg-check.timer` | `*-*-15 06:00:00` — giorno 15 alle 06:00, più 0–30 min casuali | Controllo repository Borg |
| `atlas-usb-reminder.timer` | `Sat *-*-01..07 10:00:00 Europe/Rome` — primo sabato alle 10:00 | Solo promemoria 45Drives Alerts |
| `atlas-health-monitor.timer` | `*:0/30` — ogni mezz'ora, più 0–5 min casuali | Controlli di salute in sola lettura |
`atlas-usb-backup.service` **non ha timer** e va avviato manualmente. Il timer del fornitore
`zfs-scrub-weekly@zpool.timer` è disabilitato a favore dello scrub mensile. Il futuro pull del backup
Prometheus non ha ancora un timer, perché non è implementato. Durante un backup Borg attivo,
`systemctl list-timers` può mostrare `-` per il prossimo evento senza che il timer sia disabilitato.
Per vedere la pianificazione corrente: `systemctl list-timers --all` su Atlas.
Nextcloud è previsto come servizio temporaneo su Atlas prima di Uranus, ma solo dopo la validazione
della protezione dei dati: richiede storage applicativo, database e cache separati, segreti Vault,
pubblicazione solo tramite NPM e Aegis, procedure di backup, aggiornamento e migrazione. Non
distribuirlo prima di completare la checklist di protezione dei dati.
La destinazione futura per l'importazione foto iCloud è Atlas, non Aegis. Dopo la validazione dei
backup, pianificare una migrazione esplicita di iCloudPD con foto sotto `/zpool/archive/Pictures` e
stato applicativo/MFA fuori da `Archive`; testare permessi, SELinux, backup e restore prima del
cutover. L'attuale iCloudPD su Aegis e l'export NFS Photobook restano configurati fino
all'approvazione e alla verifica di questa migrazione separata. Anche il servizio Atlas sarà
temporaneo in attesa di Uranus.
Il pull dei backup di Prometheus, la valutazione delle dimensioni degli archivi Borg e i test completi
di disaster recovery restano da fare. Il backlog prioritizzato è in `AGENTS.md`.
---

378
README.md
View File

@@ -63,6 +63,31 @@ nymph is allowed to break
`ikaros` is the stable personal Fedora/GNOME desktop. `nymph` is the laptop and gets the same shared desktop dotfiles while GNOME itself stays close to the Fedora defaults. The legacy `void` and `desktop` groups are compatibility parents; the main axes are `platform_*`, `role_*`, and `desktop_*`.
The official ChatGPT desktop RPM is enabled only on `ikaros` and `nymph`. The
playbook configures OpenAI's signed RPM repository and imports its pinned RPM
signing key before installation; subsequent updates are handled by DNF.
## Deferred planned node: Cerberus
`cerberus` is a **postponed** management-plane node, pending the physical setup
of the office in the new house. It is not yet an inventory host and no role or
playbook targets it.
The planned hardware is a Lenovo ThinkCentre M700 Tiny (Intel Core i3-6100T,
8 GB RAM, and a 256 GB SSD) with native 1 Gbps Ethernet. It will share Ikaros'
monitor and peripherals through a multi-input KVM switch, using a passive
DisplayPort-to-HDMI cable for its video connection. Fedora Sericea, the
immutable Fedora variant with the Sway Wayland compositor, is the intended
operating system.
Cerberus will be an isolated management plane: Ansible will run from a
dedicated Toolbox environment to provision the future `uranus` cluster, rather
than from Ikaros or an unmanaged host. Its rootless Podman observability stack
will run Grafana, Prometheus, and Loki. The local SSD is the hot tier and
retains metrics and logs for 30 days; scheduled exports will place older
historical data on an NFS-mounted Atlas dataset as the cold tier. The detailed,
implementation-gated plan is maintained in `AGENTS.md`.
## Desktop profiles
- `ikaros`: stable Fedora Workstation + GNOME desktop.
@@ -107,21 +132,19 @@ the Compose stack, update DNS, or perform a cutover.
The server profile installs platform-specific packages, Podman and podman-compose, declared systemd
services, and firewalld. The manually activated `podman-compose-server` unit contains the existing
Nginx Proxy Manager and Gitea services. The desired Compose file no longer includes Navidrome,
Syncthing, or the obsolete Navidrome PostgreSQL database. Navidrome and Syncthing belong to Atlas;
official Navidrome uses SQLite instead. Applying the profile does not stop or remove legacy
containers and does not delete `/opt/postgres/data`.
Syncthing, or the obsolete Navidrome PostgreSQL database; their temporary Atlas deployment is managed
by `profile_backend_phase1`. Applying the profile does not stop or remove legacy containers and does
not delete `/opt/postgres/data`.
Firewalld enables SSH, Cockpit (`9090/tcp`), HTTP and HTTPS. Nginx Proxy Manager publishes only
`80/tcp` and `443/tcp`; its administration interface is bound to `127.0.0.1:81` and can be reached
from Ikaros or Nymph with the `npm-tunnel` Bash alias. Nextcloud remains disabled and the profile
does not provision any `/srv/nextcloud` directories.
The Atlas phase-one work does not change this NPM deployment or its persistent data. Once WireGuard
and the Atlas services are active, configure the current NPM proxy hosts with Navidrome upstream
`http://10.0.0.2:4533` and Syncthing GUI upstream `http://10.0.0.2:8384`. Only the Syncthing web GUI
uses NPM; synchronization traffic remains on explicitly published native ports bound only to the Atlas
WireGuard address. Configure both Syncthing authentication and an appropriate NPM access policy before
publishing its GUI.
NPM remains managed only by `profile_server`. Its WireGuard peer is Aegis (`10.0.0.2`), which forwards
selected requests to LAN addresses and source-NATs them so no static route is required on the router.
Use an Atlas LAN address for any current NAS-backed upstream; when Uranus receives its VIP, add that VIP
to Prometheus' Aegis peer `AllowedIPs` and declare the corresponding proxy target separately.
Server identity comes from `server_username`, `server_user_group`, and `server_user_home` in `ansible/inventory/group_vars/server.yml`. `server_username` defaults to `username`, but it can be overridden, for example:
@@ -189,12 +212,19 @@ ansible/bootstrap/generate-aegis-ign.sh --write IMAGE DEVICE
The controller manages it remotely as `pi@aegis`; unlike local desktop profiles, Aegis is
intentionally an SSH inventory target. `profile_aegis` manages rootful Podman Quadlets for AdGuard
Home and iCloudPD, persistent data under `/var/lib`, the Podman auto-update timer, LAN-restricted
firewalld rules, SSH key-only access for `pi`, and `wake-ikaros`. Set the host-local
`aegis_lan_subnet` and `aegis_adguard_web_port` values before applying it. The playbook permits
firewalld rules, SSH key-only access for `pi`, the `nfs-utils` and `wireguard-tools` rpm-ostree layers,
and `wake-ikaros`. `wireguard_overlay` makes Aegis the internal endpoint and LAN gateway for Prometheus:
it enables persistent IPv4 forwarding, installs a scoped WireGuard-to-LAN firewalld policy, and source-NATs
forwarded tunnel traffic so the router needs no static route. A new layered package deployment requires a manual reboot; the
role reports this condition but never reboots Aegis automatically. Set the host-local
`aegis_lan_subnet`, `aegis_adguard_web_port`, and `aegis_network_connection_uuid` values before
applying it. The playbook permits
AdGuard Home HTTP on port `80`; the initial wizard port `3000` is intentionally unmanaged and must be
opened and closed manually during initial setup. The profile disables the local systemd-resolved DNS
stub and points `/etc/resolv.conf` to its full resolver data, freeing port 53
for AdGuard while retaining DNS learned from the router. Define
stub and points `/etc/resolv.conf` to its full resolver data, freeing port 53 for AdGuard. LAN clients
may use AdGuard on Aegis, while Aegis itself uses the independent upstream DNS declared by
`aegis_host_dns_servers`; this prevents Greenboot from depending on the AdGuard container during
startup. Reboot Aegis after changing its NetworkManager DNS profile. Define
`vault_aegis_icloudpd_apple_id` in Vault before applying it. iCloudPD still requires interactive MFA
initialization after its first deployment.
@@ -209,6 +239,21 @@ ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit aegis --check --diff --ask-become-pass
```
Apply only the independent host DNS configuration, then reboot Aegis manually:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit aegis --tags dns --ask-become-pass
```
Layer the Aegis NFS and WireGuard client tools independently, then reboot Aegis manually when the role reports
that the new deployment is ready:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit aegis --tags nfs --ask-become-pass
```
## NAS
`atlas` is a Rocky Linux 9 NAS reached through SSH. Normally its pool already exists and the profile
@@ -219,17 +264,9 @@ clients use NFSv4 and Windows/WSL clients use SMB; both are restricted to the co
For the first run, provide `vault_atlas_admin_password_hash`, `vault_atlas_samba_password`, and
`vault_atlas_immich_db_password`. Bootstrap the host through its
existing administrator. Open `51820/udp` towards Prometheus in the provider firewall first, then
include both WireGuard peers in the same idempotent playbook run:
```bash
ansible-playbook ansible/site.yml --limit prometheus,atlas \
-e atlas_connection_username=<existing-admin> \
-e atlas_create_pool=true
```
The explicit pool gate is safe to repeat: the role creates the RAIDZ2 pool only when it is absent.
WireGuard waits for a real peer handshake before the play continues.
existing administrator. The explicit pool gate is safe to repeat: the role creates the RAIDZ2 pool only when
it is absent. Atlas no longer participates in the WireGuard overlay; its old interface is retired manually only after
Prometheus and Aegis have completed the replacement handshake.
`vault_atlas_admin_password_hash` must be an `/etc/shadow`-compatible hash, not a clear-text
Cockpit password. Subsequent runs use `atlas_admin_username`. Atlas declares storage, sharing, and its
@@ -239,11 +276,11 @@ paths, and the Immich database secret are validated. Atlas reads its declared SS
separate files below `~/.ssh/authorized_keys.d/`.
With storage management enabled, Atlas creates the complete dataset hierarchy below the existing or
explicitly bootstrapped `zpool`: `work`, `archive`, `archive/app_data`, the separate `archive/app_data/navidrome` and
`archive/app_data/syncthing` application datasets, `media`, `media/music`, `media/photobook`,
`backups`, `backups/services`, and `backup_prometheus`. Application/archive datasets use `zstd`,
while media, Syncthing and service-backup datasets use `lz4`; `backups/services` also has a `500G`
refreservation. Atlas enforces targeted SELinux persistently and reports, without initiating, any reboot required to activate it. It assigns its primary LAN interface explicitly to the managed firewalld zone and applies persistent kernel network hardening: redirects and source routes are rejected, martians logged, reverse-path filtering remains loose for WireGuard, and IPv4 forwarding is disabled. SSH permits only the declared administrator using public-key authentication; root login, passwords,
explicitly bootstrapped `zpool`: SMB-shared `archive`, private `services/data` with separate
`services/data/navidrome` and `services/data/syncthing` application datasets, `media`, `media/music`,
`media/photobook`, and `backup/hosts/prometheus`. Application/archive datasets use `zstd`, while media,
Syncthing, and host-backup datasets use `lz4`; `backup` has a `500G` reservation covering its descendants.
Atlas enforces targeted SELinux persistently and reports, without initiating, any reboot required to activate it. It assigns its primary LAN interface explicitly to the managed firewalld zone and applies persistent kernel network hardening: redirects and source routes are rejected, martians logged, reverse-path filtering remains loose for WireGuard, and IPv4 forwarding is disabled. SSH permits only the declared administrator using public-key authentication; root login, passwords,
agent and remote forwarding are disabled, while local forwarding remains available for private administrative tunnels. SMB3 exposes `Archive` only to the configured Vault-backed
Samba accounts on encrypted, signed SMB3 over TCP/445 only and admits the configured LAN without host-specific
exclusions. NFSv4 exports only `media/photobook` to the configured Aegis IP over TCP/2049, using
@@ -255,53 +292,243 @@ and NPM Quadlets share one Podman network. Immich runs as `1100:1100`; Server an
and Photobook is mounted read-only at `/external/photobook`. NPM publishes ports `80` and `443`; its
administration interface remains restricted to `127.0.0.1:81` for SSH-tunnel access.
Phase 1 is limited to rootless Navidrome and Syncthing user Quadlets on Atlas. It is enabled in the
Atlas host configuration and can be set to `false` only for a deliberate suspension. Official Navidrome `0.63.2` uses its SQLite database below `/data`; it does
not support `ND_DATABASE_URL` or an external PostgreSQL backend. The obsolete `navidromedb` service
was therefore removed from Prometheus instead of being reproduced on Atlas. The role derives all
storage paths from the `zpool` mounted at `/zpool`: music is read-only at
`/zpool/media/music`, Navidrome application state and `navidrome.db` are stored at
`/zpool/archive/app_data/navidrome`, and Syncthing persists at
`/zpool/archive/app_data/syncthing`. `profile_atlas` creates these datasets when
`atlas_manage_storage` is enabled; the backend role verifies their exact mountpoints before starting
containers. The backend role never creates the pool. The separate `wireguard_overlay` role manages `wg0`
between Prometheus (`10.0.0.1`) and Atlas (`10.0.0.2`), generating private keys once
on their respective hosts and exchanging only public keys through Ansible. Prometheus alone opens
`51820/udp` publicly. When the WireGuard zone is created, Ansible reloads firewalld and immediately
reloads Prometheus' rootful Podman networks so the existing proxy stack retains container DNS and
connectivity. Backend ports are admitted only in the WireGuard firewalld zone.
Atlas temporarily hosts rootless Navidrome and Syncthing until Uranus replaces them. They bind only to
Atlas' LAN address (`192.168.178.55`); WireGuard remains exclusively between Prometheus (`10.0.0.1`)
and Aegis (`10.0.0.2`). Their state is initialized ex novo in `/zpool/services/data/navidrome` and
`/zpool/services/data/syncthing`; no source application state is migrated. The music library at
`/zpool/media/music` is populated separately.
`backend_phase1_start_services` stays false during the application-state transfer, so the first real
backend run renders the Quadlets without creating an empty Atlas database. After stopping Navidrome
on Prometheus, copy the complete `/opt/navidrome/data/` directory into
`/zpool/archive/app_data/navidrome/`, preserving `navidrome.db` and any SQLite sidecar files. Then set
this variable to true and rerun the role to enable and start Navidrome and Syncthing. The playbook
never copies or deletes application data.
The separate `wireguard_overlay` role manages `wg0` between Prometheus (`10.0.0.1`) and Aegis
(`10.0.0.2`), generating private keys once on their respective hosts and exchanging only public keys
through Ansible. Prometheus alone opens `51820/udp`. Aegis forwards only the declared overlay-to-LAN
traffic and source-NATs it, so Atlas and future Uranus nodes require neither a VPN interface nor a router
static route. Atlas permits Navidrome (`4533/tcp`) and the Syncthing GUI (`8384/tcp`) only from Aegis;
Syncthing native ports are limited to the LAN. Configure NPM manually with
`http://192.168.178.55:4533` and `http://192.168.178.55:8384` after the services are healthy.
Prometheus' peer includes the LAN subnet in `AllowedIPs`; add the Uranus VIP there when it exists.
When the WireGuard zone is created, Ansible reloads firewalld and immediately reloads Prometheus'
rootful Podman networks so the existing proxy stack retains container DNS and connectivity.
Validate and render the Atlas services with:
Validate the gateway with:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags storage
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1
ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff
```
For the cutover, stop the old Navidrome writer before copying its data directory, verify ownership by
the Atlas `admin` account and confirm that the copied SQLite database is present before changing
`backend_phase1_start_services: true` in `host_vars/atlas.yml`. Keep the source data and the stopped
legacy `navidromedb` container until Navidrome on Atlas and a restore test have been validated.
The first real WireGuard run must include both peers. If Fedora IoT has just layered `wireguard-tools`,
reboot Aegis manually and rerun the command without `--check`; the role then waits for a real peer
handshake.
Snapshot retention, Syncthing topology, WireGuard/firewall validation, Prometheus backup pulls,
encrypted Borg backups to a Hetzner Storage Box, USB backup, monitoring, and disaster-recovery tests
remain follow-up work. The detailed operational backlog is kept in `AGENTS.md`.
Atlas declares recursive, systemd-timed ZFS snapshots for the complete pool hierarchy: 24 hourly
snapshots at minute 05, 30 daily snapshots at 00:15, 8 weekly snapshots on Sunday at 01:00, and 12
monthly snapshots on the first day at 02:00. The retention helper prunes only snapshots carrying its
managed `atlas-auto` prefix and never rolls back a dataset. The OpenZFS monthly scrub timer is scheduled
for the first Sunday at 03:00; the conflicting weekly scrub timer is disabled explicitly. The first recursive
hourly snapshot completed successfully on Atlas; retention pruning and the first scheduled scrub still await
live runtime evidence. Validate this layer independently with:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff
```
Atlas also declares an encrypted Borg backup to the dedicated Hetzner Storage Box sub-account
`u660064-sub1`. The repository is the sub-account-relative `./borg-data` path and uses the explicitly
selected remote Borg 1.4 binary over SSH port 23. The ED25519 server key is pinned; a dedicated client
key is generated for the locked, non-login `borg` system account, and its private half never leaves
`/etc/atlas-borg`. The account has no sudo or supplementary groups and owns only its SSH identity,
passphrase, cache, and Borg state. Borg receives its passphrase through a mode `0600` file rendered from
`vault_atlas_borg_passphrase`.
The daily backup starts at 04:30 with up to 30 minutes of randomized delay. It creates a temporary,
recursive ZFS snapshot and reconstructs every dataset below `/zpool` as a read-only bind-mounted tree,
so parent and child datasets enter one consistent Borg archive. The wrapper recursively unmounts its
private source tree; a narrowly scoped `ExecStopPost` helper removes any remaining host-namespace ZFS
snapshot mounts and the named temporary snapshot after the backup process exits. Only the root wrapper
performs snapshot and mount operations; it launches the Borg client as `borg` with temporary read-search
capability and no ZFS, sudo, or pool-management privileges. Borg retains 30 daily, 8 weekly, and 12
monthly archives, then compacts the standard
read-write repository. A full metadata and repository check runs as `borg` on the fifteenth day of each
month at 06:00. Both operations use a common lock, journal logging, and bounded systemd retries.
New backup runs also log the create phase and a compact progress line at most once per minute: an
**estimated** percentage, dataset, files processed, and original/compressed/deduplicated bytes. The
denominator is the summed ZFS `logicalreferenced` size of the backup's own recursive snapshot, not a
Borg-reported total: the estimate can exceed 100% and does not cover retention, compaction, or checks.
Progress lines omit individual filenames; warnings may still name affected files.
Follow the current run with
`sudo journalctl -fu atlas-borg-backup.service` on Atlas; changes to the helper do not alter a run
already in progress.
Initial activation remains explicit:
1. Add a strong unique `vault_atlas_borg_passphrase` with `ansible-vault edit secrets/vault.yml`.
2. Generate and display only the dedicated public key with
`ansible-playbook ansible/site.yml --limit atlas --tags borg_key`.
3. Install that public key in the Hetzner sub-account, then apply with
`ansible-playbook ansible/site.yml --limit atlas --tags packages,borg`.
4. Copy the ignored `secrets/recovery/atlas-borg-repokey.export` file to genuinely offline storage.
The controller-side copy is not an offline backup by itself.
The role initializes only the missing `repokey` repository and never accepts an unpinned host key or
password authentication. It does not start the first backup manually. Validate the rendered state with:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff
```
Atlas runtime activation is complete: the initial backup and repository check succeeded, a full restore
to a temporary directory was validated against the live `Archive` tree, the recovery-key export was copied
to offline storage, and the temporary snapshot and bind mounts were cleaned up.
On 2026-09-25 a separate ZFS restore smoke test copied a small file from an automatic daily
`zpool/archive` snapshot to `/var/tmp`, then confirmed matching contents, ownership, mode, mtime and
POSIX ACL. The temporary copy and on-demand snapshot mount were removed; Borg kept running. This
does not validate a full dataset recovery.
The offline USB backup is deployed as a manual-only service (`atlas_manage_usb_backup: true`):
Ansible never formats, unlocks, mounts, backs up to, or schedules the disk. Atlas' existing USB disk was verified
read-only on 2026-09-23 as LUKS UUID `577b3c43-ea37-4611-81a9-39d555cdfbd4`, containing ext4 UUID
`758e2d2e-a427-4797-aad9-39c3a9f17c7e` through mapper `zpool-backup`. It was mounted at
`/mnt/zpool-backup` at inspection time. The service deliberately requires the verified mapper to be
**not mounted** before starting. When necessary, `systemd-ask-password` requests the LUKS passphrase
through the `systemctl start` password agent; it is piped directly to `cryptsetup` without saving it,
passing it as a command argument, or caching it. The service then mounts the disk privately, takes a recursive ZFS snapshot,
copies every dataset to a versioned `atlas/snapshots/<timestamp>/` directory using `rsync --link-dest`,
verifies the result with a checksum-based dry run, atomically updates `atlas/latest`, unmounts and closes
LUKS. A failed run never replaces `latest` or removes an earlier complete version. Borg and the USB
backup may run concurrently from separate snapshots; both reading the same pool can reduce throughput.
The USB copy preserves ACLs but not generic extended attributes; `security.selinux` is also intentionally
excluded because the target SELinux policy must recreate labels during a restore. Do not restore data into
service paths without relabeling. After restoring an explicit dataset path, apply its destination policy with:
```bash
ansible-playbook ansible/site.yml --limit atlas --tags restorecon \
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
```
The task accepts only paths below the Atlas pool mount root, runs `restorecon -RFv` only for the paths
provided at invocation, and is otherwise a no-op. It must not be used on the whole pool during routine runs.
Old USB versions are not pruned automatically, to avoid deleting the only offline
copy without an explicitly chosen retention policy; capacity checks include an estimated transfer size
and a 10 GiB free-space reserve. The disk must be physically disconnected after a successful backup
to make the copy offline.
To check the USB backup and reminder configuration without starting a backup, run:
```bash
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
ansible-playbook ansible/site.yml --limit atlas --tags usb_backup,usb_reminder --check --diff
```
To validate a planned, explicit post-restore relabel operation without changing labels, run:
```bash
ansible-playbook ansible/site.yml --limit atlas --tags restorecon --check \
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
```
Before the first **manual** service start, safely unmount the currently mounted
`/mnt/zpool-backup`; never run it on an arbitrary mounted disk. Future starts
can begin with the mapper closed: `sudo systemctl start atlas-usb-backup.service` prompts for the
passphrase interactively and then performs the backup. Neither the LUKS password nor a key file belongs
in Ansible. Inspect the run with
`sudo journalctl -fu atlas-usb-backup.service`. There is intentionally no timer. Independently test a
read-only mount and restore from `atlas/latest` into an empty temporary directory before marking the
USB recovery path complete. Only `atlas-usb-reminder.timer` is enabled, for the first Saturday of each
month at 10:00 Europe/Rome. Its warning notification uses the existing 45Drives Houston notifier.
A manual test confirmed a notification in 45Drives Alerts, **not** an email. The reminder service log
reports notification submission, not email delivery; the role does not depend on SMTP/OAuth settings.
The reminder never starts the backup. Check its schedule with
`systemctl list-timers atlas-usb-reminder.timer` and the result in 45Drives Alerts.
The timer was verified active with its first scheduled run at 2026-10-03 10:00 CEST. No email
delivery is claimed.
The first manual USB attempt on 2026-09-23 did not complete: rsync was denied while removing
`security.selinux` on the USB filesystem, then the interrupted service left its recursive
`atlas-usb-20260923T185748Z-2469168` snapshot and the `zpool-backup` LUKS mapper open. The
rsync xattr filter was deployed afterward. The incomplete USB directory was absent on inspection;
the exact failed snapshot was removed, the verified and unmounted mapper closed, and the service
failed state cleared. A final check found no remnant snapshot, mount, mapper, or staging directory.
The failed attempt was not a valid backup, and no USB restore had been tested at that point.
On 2026-09-24 a later run reported a checksum-verified, published USB version and closed the LUKS
mapper, but the service failed while destroying its temporary ZFS snapshot: OpenZFS still had
on-demand `.zfs/snapshot` mounts open in the host namespace. Those exact temporary snapshots were
unmounted normally and removed; no force or rollback was used. The backup service now records its
snapshot name and runs a narrowly scoped `ExecStopPost` cleanup after the private backup process
exits. The cleanup helper was tested with a disposable recursive snapshot and an active snapshot
mount. A complete run on 2026-09-24 later checksum-verified and published a new USB version; the
service ended successfully, the LUKS mapper closed, no temporary USB snapshot remained, and the pool
was healthy. On 2026-09-25 an independent restore test opened the configured USB disk read-only, mounted
ext4 with `ro,noload`, restored a 5,707,945-byte file from the published `atlas/latest` version to an
empty `/var/tmp` directory, and matched its content, owner, mode, size, mtime, and POSIX ACL against
the USB source. The test removed its temporary copy and mount, closed the LUKS mapper, and left the
pool healthy while Borg continued running. This is a file-level recovery smoke test, not a full dataset
or disaster-recovery exercise.
Atlas health monitoring runs every 30 minutes through `atlas-health-monitor.timer`. Its read-only probes
check pool/vdev state and errors, scrub/resilver status, four pool disks and the system NVMe via SMART,
disk and CPU temperatures, system/pool/snapshot space, local `zpool/backup` growth, and the Hetzner
Storage Box quota via `df -m` over the dedicated `borg` account's pinned-key SSH connection. The remote
query never opens the Borg repository or its lock. Growth alerts compare against a roughly 24-hour
baseline and therefore begin only after enough samples exist. The monitor also checks
maintenance/backup timer activation and freshness; systemd `OnFailure` hooks report snapshot,
scrub, Borg, USB, reminder, and monitoring services when they enter the failed state. The ongoing
initial Borg run is never restarted by the monitor; only a run exceeding 14 days raises a warning.
Thresholds and stable disk paths are declared in Atlas host variables. Alerts use the existing 45Drives
Houston notifier and repeated issues are deduplicated; **email delivery is not verified**. The
2026-09-25 live probe found no issues and a labelled test notification was submitted. The Storage Box
reported 22% used. Detailed Borg archive size and deduplication still require the active run to finish.
```bash
ansible-playbook ansible/site.yml --limit atlas --tags monitoring --check --diff
sudo /usr/local/libexec/atlas-health-monitor --dry-run
sudo journalctl -u atlas-health-monitor.service -n 100 --no-pager
systemctl list-timers atlas-health-monitor.timer
```
`--dry-run` sends no alerts and does not change monitor state. A real check is
`sudo systemctl start atlas-health-monitor.service`; do not start the backup services merely to test
monitoring. For a labelled 45Drives Alerts delivery test, use
`sudo /usr/local/libexec/atlas-health-monitor --test-notification`.
### Atlas systemd timers
All nine managed timers below are enabled. Times are local to Atlas (`Europe/Rome`); Borg and monitoring
add the indicated randomized delay. Every timer has `Persistent=true`, so a missed calendar run is
scheduled after the timer becomes active again.
| Timer | Schedule (`OnCalendar`) | Action |
| --- | --- | --- |
| `atlas-zfs-snapshot-hourly.timer` | `*-*-* *:05:00` — every hour at :05 | Recursive hourly snapshot and retention |
| `atlas-zfs-snapshot-daily.timer` | `*-*-* 00:15:00` — daily at 00:15 | Recursive daily snapshot and retention |
| `atlas-zfs-snapshot-weekly.timer` | `Sun *-*-* 01:00:00` — Sunday at 01:00 | Recursive weekly snapshot and retention |
| `atlas-zfs-snapshot-monthly.timer` | `*-*-01 02:00:00` — first day of the month at 02:00 | Recursive monthly snapshot and retention |
| `zfs-scrub-monthly@zpool.timer` | `Sun *-*-01..07 03:00:00` — first Sunday at 03:00 | ZFS scrub |
| `atlas-borg-backup.timer` | `*-*-* 04:30:00` — daily at 04:30, plus 0–30 min random delay | Encrypted offsite backup |
| `atlas-borg-check.timer` | `*-*-15 06:00:00` — 15th of the month at 06:00, plus 0–30 min random delay | Borg repository check |
| `atlas-usb-reminder.timer` | `Sat *-*-01..07 10:00:00 Europe/Rome` — first Saturday at 10:00 | 45Drives Alerts reminder only |
| `atlas-health-monitor.timer` | `*:0/30` — every half-hour, plus 0–5 min random delay | Read-only health checks |
`atlas-usb-backup.service` has **no timer**: the encrypted USB backup must be started manually.
The vendor's `zfs-scrub-weekly@zpool.timer` is intentionally disabled in favor of the monthly scrub.
The future Prometheus backup pull has no timer yet because that workflow is not implemented. While a
Borg backup is still running, `systemctl list-timers` may show `-` for its next trigger; this does not
mean the timer has been disabled. Inspect the current schedule on Atlas with
`systemctl list-timers --all`.
A temporary Nextcloud deployment on Atlas is also planned before Uranus: it requires separately
declared persistent application, database, and cache storage, Vault-backed credentials, NPM-only
publishing through Aegis, and defined backup, upgrade, and eventual migration procedures. Do not deploy
it before the data-protection checklist is complete.
The desired future iCloud photo-ingestion host is Atlas, not Aegis. After data-protection validation,
plan an explicit iCloudPD migration with photos under `/zpool/archive/Pictures` and application/MFA
state outside `Archive`, then test permissions, SELinux, backups and recovery before cutting over.
The current Aegis iCloudPD service and Atlas Photobook NFS export remain configured until that
separate migration is approved and validated; the eventual Atlas service is temporary until Uranus.
Prometheus backup pulls, Borg archive-size evaluation, and full disaster-recovery tests remain follow-up
work. The prioritized operational backlog is kept in `AGENTS.md`.
## How layering works
@@ -334,9 +561,10 @@ ansible-playbook ansible/site.yml --limit <host> --tags emacs -e emacs_enabled=t
## AI coding agents
The shared npm-managed agents are OpenCode, Claude Code, Codex, Gemini CLI, and
GitHub Copilot; IBM Bob is also managed on `deadalus`. Each agent has its own
lifecycle flags in `ansible/inventory/group_vars/all.yml`, so one agent can be
installed, configured, or removed without affecting the others:
GitHub Copilot; IBM Bob is also managed on `deadalus`. Codex Relay is installed
only on `ikaros`. Each agent has its own lifecycle flags in
`ansible/inventory/group_vars/all.yml`, so one agent can be installed,
configured, or removed without affecting the others:
```yaml
ai_agents:
@@ -387,8 +615,8 @@ ansible-playbook ansible/site.yml --limit deadalus --tags ai_agents --check --di
| `profile_workstation_dev_wsl` | WSL development setup. |
| `profile_server` | Server setup. |
| `profile_atlas` | Rocky Linux 9 NAS setup. |
| `profile_backend_phase1` | Rootless Navidrome and Syncthing on Atlas. |
| `wireguard_overlay` | Prometheus/Atlas WireGuard overlay. |
| `profile_backend_phase1` | Temporary rootless Atlas Navidrome and Syncthing services. |
| `wireguard_overlay` | Prometheus/Aegis WireGuard LAN gateway. |
| `profile_aegis` | Fedora IoT always-on LAN node. |
| `dotfiles_common` | Shared user dotfiles. |
@@ -400,8 +628,8 @@ platform_void -> packages_void + services_runit
platform_void & graphical_desktop -> profile_desktop_common + profile_desktop_sway + profile_desktop_niri + profile_desktop_host
platform_fedora -> packages_fedora + services_systemd
platform_rocky -> packages_rocky + services_systemd
wireguard_overlay -> wireguard_overlay (after platform_rocky)
role_aegis -> profile_aegis
wireguard_overlay -> wireguard_overlay (after Aegis profile and platform_rocky)
atlas -> profile_atlas
role_backend_phase1 -> profile_backend_phase1 (after atlas)
rocky_server -> dotfiles_common + profile_server (after platform_rocky)
@@ -506,7 +734,7 @@ ansible-playbook ansible/site.yml --list-tags
| `sharing` | Atlas NFSv4 and SMB3 configuration. |
| `storage` | Atlas child ZFS datasets. |
| `tmux` | tmux configuration and plugins. |
| `wireguard` | Prometheus/Atlas WireGuard overlay. |
| `wireguard` | Prometheus/Aegis WireGuard LAN gateway. |
| `wsl` | WSL bootstrap and configuration. |
## Bootstrapping a new machine

View File

@@ -93,6 +93,8 @@ common_dotfiles:
# AI coding-agent lifecycle controls are independent for every agent.
# Installation and removal are mutually exclusive per agent; the common
# pre-task enforces this before any changes are made.
codex_relay_install_enabled: false
ai_agents:
opencode:
npm_package: opencode-ai
@@ -109,6 +111,11 @@ ai_agents:
install_enabled: true
deploy_enabled: true
uninstall_enabled: false
codex_relay:
npm_package: codex-relay
install_enabled: "{{ codex_relay_install_enabled }}"
deploy_enabled: false
uninstall_enabled: false
gemini_cli:
npm_package: "@google/gemini-cli"
install_enabled: true

View File

@@ -9,6 +9,7 @@ fedora_packages_base:
- jq
- make
- nodejs
- openssl
- ripgrep
fedora_manage_docker_repo: true
@@ -58,3 +59,13 @@ fedora_dev_packages:
fedora_flatpak_remote_name: flathub
fedora_flatpak_remote_url: https://dl.flathub.org/repo/flathub.flatpakrepo
chatgpt_desktop_enabled: false
chatgpt_desktop_supported_fedora_versions:
- "43"
- "44"
chatgpt_desktop_supported_architectures:
- x86_64
- aarch64
chatgpt_desktop_rpm_gpg_key: RPM-GPG-KEY-chatgpt-3BFA0E4AE8B8CC16A2D9BA684A3B4A566C4660E4.asc
chatgpt_desktop_rpm_repository: https://persistent.oaistatic.com/codex-app-prod/linux/rpm/$basearch

View File

@@ -40,4 +40,3 @@ server_firewalld_services:
server_firewalld_ports: []
server_sshd_service_name: sshd
server_compose_selinux_mount_option: Z
server_syncthing_enabled: false

View File

@@ -75,21 +75,6 @@ server_directories:
group: root
mode: "0755"
server_syncthing_enabled: true
server_syncthing_directories:
- path: /opt/syncthing/config
owner: root
group: root
mode: "0755"
- path: /srv/syncthing
owner: root
group: root
mode: "0755"
- path: /srv/syncthing/data
owner: "1000"
group: "1000"
mode: "0755"
server_sshd_settings:
PermitRootLogin: "no"

View File

@@ -3,14 +3,19 @@ enabled_services:
- docker
workstation_dev_wsl_packages:
- buildah
- helm
# Fedora ships the kubectl client binary in the kubernetes-client package.
- kubernetes-client
- mise
- pinentry-curses
- python3-pip
- tmux
# Java 11 and Maven are managed by Mise on this Fedora WSL profile. Keep their
# versions pinned; update them deliberately.
# Java 11, Java 25 and Maven are managed by Mise on this Fedora WSL profile.
# Keep their versions pinned; update them deliberately.
workstation_mise_java_version: temurin-11.0.31+11
workstation_mise_java_25_version: 25.0.2
workstation_mise_maven_version: 3.9.16
workstation_is_wsl: true

View File

@@ -9,6 +9,32 @@ ansible_ssh_use_tty: false
aegis_lan_subnet: 192.168.178.0/24
aegis_adguard_web_port: 80
aegis_network_connection_name: FRITZ!Box 7530 NR
aegis_network_connection_uuid: a52fda3d-3eb6-351f-bf04-753edcb76944
aegis_host_dns_servers:
- 192.168.178.1
aegis_host_dns_search_domains:
- fritz.box
# Aegis is the internal WireGuard endpoint and NAT gateway for the LAN.
wireguard_overlay_enabled: true
wireguard_address: 10.0.0.2/24
wireguard_enable_ipv4_forwarding: true
wireguard_reload_rootful_podman_networks: true
wireguard_forwarding_policies:
- name: aegis-wireguard-to-lan
ingress_zone: wireguard
egress_zone: public
source: 10.0.0.0/24
destination: "{{ aegis_lan_subnet }}"
masquerade: true
wireguard_peers:
- name: prometheus
host: prometheus
endpoint: "{{ hostvars['prometheus']['ansible_host'] }}:{{ hostvars['prometheus']['wireguard_listen_port'] }}"
allowed_ips:
- 10.0.0.0/24
persistent_keepalive: 25
aegis_ssh_authorized_keys:
- name: ikaros
key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINrIxXjA3ffPwziKGR5gzc4gAoBehQPlnEMcXF4Wl0ZS ikaros"

View File

@@ -35,35 +35,112 @@ atlas_zpool_disks:
- /dev/disk/by-id/ata-ST4000VN006-3CW104_WW6A6VJK
- /dev/disk/by-id/ata-ST4000VN006-3CW104_WW6AC1LM
atlas_create_pool: false
atlas_zfs_dataset_work: work
atlas_zfs_dataset_archive: archive
atlas_zfs_dataset_app_data: archive/app_data
atlas_zfs_dataset_navidrome: archive/app_data/navidrome
atlas_zfs_dataset_syncthing: archive/app_data/syncthing
atlas_zfs_dataset_services: services
atlas_zfs_dataset_app_data: services/data
atlas_zfs_dataset_navidrome: services/data/navidrome
atlas_zfs_dataset_syncthing: services/data/syncthing
atlas_zfs_dataset_media: media
atlas_zfs_dataset_music: media/music
atlas_zfs_dataset_backup_prometheus: backup_prometheus
atlas_zfs_dataset_backup: backup
atlas_zfs_dataset_host_backups: backup/hosts
atlas_zfs_dataset_backup_prometheus: backup/hosts/prometheus
atlas_zfs_backup_reservation: 500G
atlas_zfs_dataset_photobook: media/photobook
atlas_zfs_dataset_backups: backups
atlas_zfs_dataset_service_backups: backups/services
atlas_mount_root: /zpool
atlas_manage_storage: true
atlas_manage_zfs_snapshots: true
atlas_zfs_snapshot_prefix: atlas-auto
atlas_zfs_snapshot_policies:
- name: hourly
calendar: "*-*-* *:05:00"
keep: 24
- name: daily
calendar: "*-*-* 00:15:00"
keep: 30
- name: weekly
calendar: "Sun *-*-* 01:00:00"
keep: 8
- name: monthly
calendar: "*-*-01 02:00:00"
keep: 12
atlas_manage_zfs_scrub: true
atlas_zfs_scrub_calendar: "Sun *-*-01..07 03:00:00"
atlas_manage_borg_backup: true
atlas_borg_repository_host: u660064-sub1.your-storagebox.de
atlas_borg_repository_user: u660064-sub1
atlas_borg_repository_port: 23
atlas_borg_repository_path: ./borg-data
atlas_borg_remote_path: borg-1.4
# Verified against Hetzner's published ED25519 fingerprint on 2026-09-17:
# SHA256:XqONwb1S0zuj5A1CDxpOSuD2hnAArV1A3wKY7Z3sdgM
atlas_borg_host_key: >-
[u660064-sub1.your-storagebox.de]:23 ssh-ed25519
AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
atlas_borg_backup_calendar: "*-*-* 04:30:00"
atlas_borg_check_calendar: "*-*-15 06:00:00"
atlas_borg_randomized_delay: 30m
atlas_borg_keep_daily: 30
atlas_borg_keep_weekly: 8
atlas_borg_keep_monthly: 12
atlas_manage_usb_backup: true
# Read-only lsblk verification on Atlas, 2026-09-23. Never store the LUKS password here.
atlas_usb_backup_luks_uuid: 577b3c43-ea37-4611-81a9-39d555cdfbd4
atlas_usb_backup_fs_uuid: 758e2d2e-a427-4797-aad9-39c3a9f17c7e
atlas_usb_backup_mapper_name: zpool-backup
atlas_manage_usb_reminder: true
atlas_usb_reminder_calendar: "Sat *-*-01..07 10:00:00 Europe/Rome"
atlas_manage_monitoring: true
# Physical pool disks and the system NVMe; the disconnected USB disk is intentionally excluded.
atlas_monitor_smart_devices:
- { name: pool-1, path: "{{ atlas_zpool_disks[0] }}", warning_c: 50, critical_c: 55 }
- { name: pool-2, path: "{{ atlas_zpool_disks[1] }}", warning_c: 50, critical_c: 55 }
- { name: pool-3, path: "{{ atlas_zpool_disks[2] }}", warning_c: 50, critical_c: 55 }
- { name: pool-4, path: "{{ atlas_zpool_disks[3] }}", warning_c: 50, critical_c: 55 }
- name: system-nvme
path: /dev/disk/by-id/nvme-Patriot_M.2_P320_256GB_P320ADB26011606111
warning_c: 70
critical_c: 85
atlas_monitor_timers:
- { name: atlas-zfs-snapshot-hourly.timer, max_age_hours: 3 }
- { name: atlas-zfs-snapshot-daily.timer, max_age_hours: 36 }
- { name: atlas-zfs-snapshot-weekly.timer, max_age_hours: 216 }
- { name: atlas-zfs-snapshot-monthly.timer, max_age_hours: 960 }
- { name: zfs-scrub-monthly@zpool.timer, max_age_hours: 960 }
- { name: atlas-borg-backup.timer, max_age_hours: 48 }
- { name: atlas-borg-check.timer, max_age_hours: 960 }
# The first manual USB reminder is not due until October; activation is checked, not age.
- { name: atlas-usb-reminder.timer, max_age_hours: 0 }
atlas_monitor_failure_units:
- atlas-zfs-snapshot@.service
- zfs-scrub@zpool.service
- atlas-borg-backup.service
- atlas-borg-check.service
- atlas-usb-backup.service
- atlas-usb-reminder.service
- atlas-health-monitor.service
atlas_monitor_remote_capacity:
user: "{{ atlas_borg_repository_user }}"
host: "{{ atlas_borg_repository_host }}"
run_as: "{{ atlas_borg_username }}"
ssh_wrapper: "{{ atlas_borg_ssh_wrapper_path }}"
warning_percent: 80
critical_percent: 90
growth_warning_gib_day: 500
atlas_manage_sharing: true
atlas_manage_media_stack: false
# Planned after data-protection validation: move iCloudPD photo ingestion from
# Aegis to Atlas, with photos under /zpool/archive/Pictures and persistent
# application/MFA state outside Archive. Do not deploy or cut over yet.
wireguard_overlay_enabled: true
wireguard_address: 10.0.0.2/24
wireguard_peers:
- name: prometheus
host: prometheus
endpoint: "{{ hostvars['prometheus']['ansible_host'] }}:{{ hostvars['prometheus']['wireguard_listen_port'] }}"
allowed_ips:
- 10.0.0.0/24
persistent_keepalive: 25
# WireGuard is retired on Atlas. These rootless services are a temporary home
# until Uranus replaces them.
backend_phase1_enabled: true
backend_phase1_start_services: false
backend_phase1_wireguard_address: 10.0.0.2
backend_phase1_start_services: true
backend_phase1_bind_address: "{{ ansible_host }}"
backend_phase1_firewalld_zone: "{{ atlas_firewalld_zone }}"
backend_phase1_npm_source_ip: "{{ atlas_aegis_ip }}"
backend_phase1_syncthing_native_subnet: "{{ atlas_lan_subnet }}"
rocky_manage_openzfs_repo: true
rocky_manage_syncthing_binary: false
@@ -73,13 +150,21 @@ rocky_podman_packages:
host_packages:
- cockpit
- cockpit-podman
- cockpit-storaged
- realmd
- pcp
- python3-pcp
- cryptsetup
- nfs-utils
- policycoreutils
- policycoreutils-python-utils
- python3-libselinux
- setroubleshoot-server
- samba
- samba-client
- samba-common-tools
- borgbackup
- zfs
atlas_nfs_exports:
@@ -113,4 +198,5 @@ atlas_firewalld_rich_rules:
host_enabled_services:
- sshd
- cockpit.socket
- pmlogger.service
- zfs.target

View File

@@ -1,6 +1,9 @@
---
hostname: ikaros
codex_relay_install_enabled: true
chatgpt_desktop_enabled: true
host_packages:
- openssh-clients
- openssh-server
@@ -229,7 +232,7 @@ gnome_gsettings_settings:
- schema: org.gnome.shell
key: app-picker-layout
value: >-
[{'System': <{'position': <0>}>, 'Utilities': <{'position': <1>}>, 'org.fedoraproject.MediaWriter.desktop': <{'position': <2>}>, 'org.gnome.Contacts.desktop': <{'position': <3>}>, 'org.gnome.Calendar.desktop': <{'position': <4>}>, 'org.gnome.Calculator.desktop': <{'position': <5>}>, 'org.gnome.Weather.desktop': <{'position': <6>}>, 'org.gnome.clocks.desktop': <{'position': <7>}>, 'org.gnome.Maps.desktop': <{'position': <8>}>, 'org.gnome.SimpleScan.desktop': <{'position': <9>}>, 'org.gnome.TextEditor.desktop': <{'position': <10>}>, 'org.gnome.Settings.desktop': <{'position': <11>}>, 'org.gnome.Boxes.desktop': <{'position': <12>}>, 'org.gnome.Showtime.desktop': <{'position': <13>}>, 'org.gnome.Snapshot.desktop': <{'position': <14>}>, 'org.gnome.Characters.desktop': <{'position': <15>}>, 'org.gnome.Tour.desktop': <{'position': <16>}>, 'org.gnome.Yelp.desktop': <{'position': <17>}>, 'org.gnome.Software.desktop': <{'position': <18>}>, 'be.alexandervanhee.gradia.desktop': <{'position': <19>}>, 'com.github.jeromerobert.pdfarranger.desktop': <{'position': <20>}>, 'org.onlyoffice.desktopeditors.desktop': <{'position': <21>}>, 'org.gnome.Meld.desktop': <{'position': <22>}>, 'com.mattjakeman.ExtensionManager.desktop': <{'position': <23>}>}, {'org.remmina.Remmina.desktop': <{'position': <0>}>, 'ch.protonmail.protonmail-bridge.desktop': <{'position': <1>}>, 'syncthing-start.desktop': <{'position': <2>}>, 'syncthing-ui.desktop': <{'position': <3>}>, 'emacs.desktop': <{'position': <4>}>, 'emacs-client.desktop': <{'position': <5>}>, 'org.gnome.gitlab.somas.Apostrophe.desktop': <{'position': <6>}>, 'mpv.desktop': <{'position': <7>}>, 'htop.desktop': <{'position': <8>}>, 'org.mozilla.thunderbird.desktop': <{'position': <9>}>}]
[{'System': <{'position': <0>}>, 'Utilities': <{'position': <1>}>, 'org.fedoraproject.MediaWriter.desktop': <{'position': <2>}>, 'org.gnome.Contacts.desktop': <{'position': <3>}>, 'org.gnome.Calendar.desktop': <{'position': <4>}>, 'org.gnome.Calculator.desktop': <{'position': <5>}>, 'org.gnome.Weather.desktop': <{'position': <6>}>, 'org.gnome.clocks.desktop': <{'position': <7>}>, 'org.gnome.Maps.desktop': <{'position': <8>}>, 'org.gnome.SimpleScan.desktop': <{'position': <9>}>, 'org.gnome.TextEditor.desktop': <{'position': <10>}>, 'org.gnome.Settings.desktop': <{'position': <11>}>, 'org.gnome.Boxes.desktop': <{'position': <12>}>, 'org.gnome.Showtime.desktop': <{'position': <13>}>, 'org.gnome.Snapshot.desktop': <{'position': <14>}>, 'org.gnome.Characters.desktop': <{'position': <15>}>, 'org.gnome.Tour.desktop': <{'position': <16>}>, 'org.gnome.Yelp.desktop': <{'position': <17>}>, 'org.gnome.Software.desktop': <{'position': <18>}>, 'be.alexandervanhee.gradia.desktop': <{'position': <19>}>, 'com.github.jeromerobert.pdfarranger.desktop': <{'position': <20>}>, 'org.onlyoffice.desktopeditors.desktop': <{'position': <21>}>, 'org.gnome.Meld.desktop': <{'position': <22>}>, 'com.mattjakeman.ExtensionManager.desktop': <{'position': <23>}>}, {'org.remmina.Remmina.desktop': <{'position': <0>}>, 'ch.protonmail.protonmail-bridge.desktop': <{'position': <1>}>, 'syncthing-start.desktop': <{'position': <2>}>, 'syncthing-ui.desktop': <{'position': <3>}>, 'emacs.desktop': <{'position': <4>}>, 'emacs-client.desktop': <{'position': <5>}>, 'org.gnome.gitlab.somas.Apostrophe.desktop': <{'position': <6>}>, 'mpv.desktop': <{'position': <7>}>, 'htop.desktop': <{'position': <8>}>, 'com.raspberrypi.rpi-imager.desktop': <{'position': <9>}>}]
- schema: org.gnome.shell
key: disabled-extensions
value: >-
@@ -241,7 +244,7 @@ gnome_gsettings_settings:
- schema: org.gnome.shell
key: favorite-apps
value: >-
['org.gnome.Nautilus.desktop', 'org.mozilla.firefox.desktop', 'com.spotify.Client.desktop', 'org.gnome.Evolution.desktop', 'org.telegram.desktop.desktop', 'org.gnome.Ptyxis.desktop', 'virt-manager.desktop']
['org.gnome.Nautilus.desktop', 'org.mozilla.firefox.desktop', 'com.spotify.Client.desktop', 'org.gnome.Evolution.desktop', 'org.telegram.desktop.desktop', 'chatgpt.desktop', 'org.gnome.Ptyxis.desktop', 'virt-manager.desktop']
- schema: org.gnome.shell
key: remember-mount-password
value: "true"

View File

@@ -1,6 +1,8 @@
---
hostname: nymph
chatgpt_desktop_enabled: true
# Fedora Workstation/GNOME laptop target. GNOME is provided by the base Fedora
# Workstation install; Ansible deploys shared desktop dotfiles but leaves GNOME
# settings unmanaged for now.

View File

@@ -21,15 +21,23 @@ wireguard_listen_port: 51820
wireguard_enable_ipv4_forwarding: true
wireguard_reload_rootful_podman_networks: true
wireguard_peers:
- name: atlas
host: atlas
- name: aegis
host: aegis
# TODO: Aggiungere VIP di Uranus in futuro
allowed_ips:
- 10.0.0.2/32
- 192.168.178.0/24
host_packages:
- cockpit
- cockpit-navigator
- cockpit-podman
- cockpit-storaged
- realmd
- pcp
- python3-pcp
- setroubleshoot-server
host_enabled_services:
- cockpit.socket
- pmlogger.service

View File

@@ -99,7 +99,7 @@ all:
wireguard_overlay:
hosts:
prometheus:
atlas:
aegis:
role_backend_phase1:
hosts:

View File

@@ -0,0 +1,28 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGpypFUBEACi1Vvzq9pIpA6lj7chbqELuxJtVuzUzxrasa6ZU0yF4yhq7jf8
3YkJRHwbezBKeQyzJ5lkX0EhXS8aXxUhMAm3PFpAlwcInfKzmV7atJwvaxIw6Rmd
GYe9fBWKjTN/SmPIjtyxrTznZY97+TfD1AeGZpLaJ8fsnhrC+HkiN2TACiTocgpe
hFiP0OWK7mWZeTWnY2scpIYXP1Ro7nQv4KacmY4JacTQ7m/HM0Qej/3olhuEv2Cw
lMVWw57/oHhmTllfLDQOogFQyIVqaaR98y/Eu6cAabSfcsqAAZ2A8vfHYD27z28J
vLO2PZEJd5ThlnX4Zqv0eIpZdBj//8Sl/MSqTshFZ1NDsRoqwdqw284X5MpnOJ4k
4Sc2Se8tJxt/nCeibH3dJ504Fb1X/mnOqhCAQ6pVJz4RB5HRlFPSkxVPyag1v1m/
7T4vie+OR4eqFQNz6mudrOoMmeVIfyL5fbe4cOr4fk/FyvEE2xMgkFatPqXn7vM9
og+zremPCfwRAFpBPyX74VowFY7llcdaj/w8K5T8PzM14Hb3E4ZKizMluKmTvTq9
WE1/eSQJLLQqXD5VmtmdUaC/VyE/1ZlIxcA1LWqvEQ327UXREvX/nHsrkKrl956W
jzkiHFUTsD1NJ0dMfs+csOt8Furb5jZj+HsMmCm9jLdfz5b/4WKLPbvxIwARAQAB
tBZDb2RleCBMaW51eCBSZXBvc2l0b3J5iQJRBBMBCgA7FiEEO/oOSui4zBai2bpo
SjtKVmxGYOQFAmpypFUCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQ
SjtKVmxGYORlCQ/9FyikZo8HQcJBP9E/oXVPds/fQnIFB2qJR2z3DrfYEonNt/ev
SAySkPPq4/mEOjaI0pFlDDGSaps+FTcJFgoVRTasBIF7JJivvjW9ap8iWEbhhVLe
IrFLbMLpUcTRntUx7R4fVMJ/1/cGn+NWZmNwS9ORorzSyCH0IAgCw1Xc3ZrjuMbF
VjdToMC1TiXXCEmlYpQakmQ3Ay1cH0FHC2BBNn1MNVkJdPhpZIZCdhaMPHfYFpyo
pg8wFvZ5iIcvlbMgyuy8CPJVRWUcYy2dOhEOGnYJnXRPkE3E1hf8YOHNzRlduH89
6lT9qcEK2+fpLfrVGoc4zscLZ+Ey+Ko6iQRdVE1j67+wNR3hX8ukue574v1N/xxu
i575jumSE19lEj1sH4+P4gFHOtTbF0JhKKzLctbga0IAwTPKhnt3qzj1U5Yj/MZS
uEVjrLhdRauOuFBXUclgyVf2w/lE85UUOdlcollsYA6Huq7xDamqf8SslZQGre3E
I+lhpqJR1cOwDMUzzcl40uTyhrxXXd/bk4QSlhZbwHR25Pnt+ZMtWavlQWS0eDEV
8djuXAURCmx5WOqAFB/TJe1mn5EvyWg4VFzrY/NVNOpzgY5+Xp7J28z7f637r712
Eu9j4imVcdPigwS+jf/0f81i2o9b82Y26TN8+EtDLCY841MJ1lrjDrX/dno=
=Y+3h
-----END PGP PUBLIC KEY BLOCK-----

View File

@@ -130,6 +130,56 @@
loop_control:
label: "{{ item.key }}"
- name: Validate ChatGPT desktop architecture on Fedora
tags: [packages, chatgpt]
ansible.builtin.assert:
that:
- ansible_facts['architecture'] in chatgpt_desktop_supported_architectures
- ansible_facts['distribution_major_version'] in chatgpt_desktop_supported_fedora_versions
fail_msg: >-
ChatGPT desktop supports Fedora {{ chatgpt_desktop_supported_fedora_versions | join(' and ') }}
on x86_64 or aarch64; detected Fedora {{ ansible_facts['distribution_major_version'] }}
on {{ ansible_facts['architecture'] }}.
when: chatgpt_desktop_enabled | bool
- name: Install ChatGPT desktop RPM signing key
tags: [packages, chatgpt]
ansible.builtin.copy:
src: "{{ chatgpt_desktop_rpm_gpg_key }}"
dest: "/etc/pki/rpm-gpg/{{ chatgpt_desktop_rpm_gpg_key }}"
owner: root
group: root
mode: "0644"
when: chatgpt_desktop_enabled | bool
- name: Import ChatGPT desktop RPM signing key
tags: [packages, chatgpt]
ansible.builtin.rpm_key:
state: present
key: "/etc/pki/rpm-gpg/{{ chatgpt_desktop_rpm_gpg_key }}"
when: chatgpt_desktop_enabled | bool
- name: Configure ChatGPT desktop RPM repository
tags: [packages, chatgpt]
ansible.builtin.yum_repository:
name: openai-chatgpt
description: ChatGPT
baseurl: "{{ chatgpt_desktop_rpm_repository }}"
enabled: true
gpgcheck: true
repo_gpgcheck: true
gpgkey: "file:///etc/pki/rpm-gpg/{{ chatgpt_desktop_rpm_gpg_key }}"
state: present
when: chatgpt_desktop_enabled | bool
- name: Install ChatGPT desktop on selected Fedora hosts
tags: [packages, chatgpt]
ansible.builtin.dnf:
name: chatgpt
state: latest
update_cache: true
when: chatgpt_desktop_enabled | bool
- name: Define Fedora Flatpak applications
tags: [packages]
ansible.builtin.set_fact:

View File

@@ -1,5 +1,12 @@
---
aegis_hostname: aegis
aegis_layered_packages:
- nfs-utils
- wireguard-tools
aegis_network_connection_name: ""
aegis_network_connection_uuid: ""
aegis_host_dns_servers: []
aegis_host_dns_search_domains: []
aegis_adguard_image: docker.io/adguard/adguardhome:latest
aegis_icloudpd_image: docker.io/boredazfcuk/icloudpd:latest
aegis_icloudpd_folder_structure: '{:%Y/%m/%d}'

View File

@@ -1,4 +1,18 @@
---
- name: Manage Aegis layered packages
tags: [aegis, packages, nfs, wireguard]
community.general.rpm_ostree_pkg:
name: "{{ aegis_layered_packages }}"
state: present
register: aegis_layered_packages_result
when: aegis_layered_packages | length > 0
- name: Report reboot required for Aegis layered packages
tags: [aegis, packages, nfs, wireguard]
ansible.builtin.debug:
msg: Reboot Aegis to activate the newly layered packages, then rerun the playbook.
when: aegis_layered_packages_result.needs_reboot | default(false)
- name: Require Aegis iCloudPD Apple ID
tags: [aegis, icloudpd]
ansible.builtin.assert:
@@ -8,15 +22,86 @@
no_log: true
- name: Require completed Aegis network placeholders
tags: [aegis, firewall, services]
tags: [aegis, dns, firewall, network, services]
ansible.builtin.assert:
that:
- aegis_lan_subnet != 'CHANGEME_LAN_SUBNET'
- aegis_firewalld_zone | length > 0
- aegis_adguard_web_port | int > 0
- aegis_adguard_web_port | int < 65536
- aegis_network_connection_name | length > 0
- aegis_network_connection_uuid | length > 0
- aegis_host_dns_servers | length > 0
- ansible_facts["default_ipv4"]["address"] not in aegis_host_dns_servers
- aegis_ssh_allowed_users | length > 0
fail_msg: Define the Aegis LAN subnet, firewalld zone, AdGuard web port, and SSH users.
fail_msg: >-
Define the Aegis LAN subnet, firewalld zone, AdGuard web port, independent host DNS,
NetworkManager connection, and SSH users. Aegis must not use its own address as upstream DNS.
- name: Verify the declared Aegis NetworkManager connection exists
tags: [aegis, dns, network, services]
ansible.builtin.command:
argv:
- nmcli
- --get-values
- connection.id
- connection
- show
- uuid
- "{{ aegis_network_connection_uuid }}"
register: aegis_network_connection
changed_when: false
failed_when: >-
aegis_network_connection.rc != 0
or aegis_network_connection.stdout != aegis_network_connection_name
- name: Read the current Aegis host DNS configuration
tags: [aegis, dns, network, services]
ansible.builtin.command:
argv:
- nmcli
- --get-values
- ipv4.ignore-auto-dns,ipv4.dns,ipv4.dns-search,ipv6.ignore-auto-dns
- connection
- show
- uuid
- "{{ aegis_network_connection_uuid }}"
register: aegis_host_dns_current
changed_when: false
- name: Configure independent upstream DNS for the Aegis host
tags: [aegis, dns, network, services]
ansible.builtin.command:
argv:
- nmcli
- connection
- modify
- uuid
- "{{ aegis_network_connection_uuid }}"
- ipv4.ignore-auto-dns
- "yes"
- ipv4.dns
- "{{ aegis_host_dns_servers | join(',') }}"
- ipv4.dns-search
- "{{ aegis_host_dns_search_domains | join(',') }}"
- ipv6.ignore-auto-dns
- "yes"
register: aegis_host_dns_profile
when: >-
aegis_host_dns_current.stdout_lines !=
[
'yes',
aegis_host_dns_servers | join(','),
aegis_host_dns_search_domains | join(','),
'yes'
]
changed_when: true
- name: Report Aegis reboot required for host DNS changes
tags: [aegis, dns, network, services]
ansible.builtin.debug:
msg: Reboot Aegis to activate its independent upstream DNS before testing another OS update.
when: aegis_host_dns_profile.changed | default(false)
- name: Set Aegis hostname
tags: [aegis, services]

View File

@@ -47,31 +47,101 @@ atlas_hardening_sysctls:
net.ipv6.conf.all.accept_source_route: "0"
net.ipv6.conf.default.accept_source_route: "0"
atlas_zfs_pool: CHANGEME_ZFS_POOL
atlas_zfs_dataset_work: work
atlas_zfs_dataset_archive: archive
atlas_zfs_dataset_app_data: archive/app_data
atlas_zfs_dataset_navidrome: archive/app_data/navidrome
atlas_zfs_dataset_syncthing: archive/app_data/syncthing
atlas_zfs_dataset_services: services
atlas_zfs_dataset_app_data: services/data
atlas_zfs_dataset_navidrome: services/data/navidrome
atlas_zfs_dataset_syncthing: services/data/syncthing
atlas_zfs_dataset_media: media
atlas_zfs_dataset_music: media/music
atlas_zfs_dataset_backup_prometheus: backup_prometheus
atlas_zfs_dataset_backup: backup
atlas_zfs_dataset_host_backups: backup/hosts
atlas_zfs_dataset_backup_prometheus: backup/hosts/prometheus
atlas_zfs_backup_reservation: 500G
atlas_zfs_dataset_photobook: media/photobook
atlas_zfs_dataset_backups: backups
atlas_zfs_dataset_service_backups: backups/services
atlas_zfs_service_backups_refreservation: 500G
atlas_mount_root: /CHANGEME_ATLAS_MOUNT_ROOT
atlas_manage_zfs_snapshots: false
atlas_zfs_snapshot_prefix: atlas-auto
atlas_zfs_snapshot_policies: []
atlas_manage_zfs_scrub: false
atlas_zfs_scrub_calendar: ""
atlas_manage_borg_backup: false
atlas_borg_username: borg
atlas_borg_group: borg
atlas_borg_home: /var/lib/atlas-borg
atlas_borg_repository_host: CHANGEME_BORG_HOST
atlas_borg_repository_user: CHANGEME_BORG_USER
atlas_borg_repository_port: 23
atlas_borg_repository_path: ./borg-data
atlas_borg_remote_path: borg-1.4
atlas_borg_host_key: ""
atlas_borg_ssh_private_key_path: /etc/atlas-borg/id_ed25519
atlas_borg_known_hosts_path: /etc/atlas-borg/known_hosts
atlas_borg_passphrase_path: /etc/atlas-borg/passphrase
atlas_borg_ssh_wrapper_path: /usr/local/libexec/atlas-borg-ssh
atlas_borg_passphrase: "{{ vault_atlas_borg_passphrase | default('') }}"
atlas_borg_encryption_mode: repokey
atlas_borg_archive_prefix: atlas
atlas_borg_snapshot_prefix: atlas-borg
atlas_borg_compression: auto,zstd,3
atlas_borg_backup_calendar: ""
atlas_borg_check_calendar: ""
atlas_borg_randomized_delay: 30m
atlas_borg_keep_daily: 30
atlas_borg_keep_weekly: 8
atlas_borg_keep_monthly: 12
atlas_borg_config_dir: /var/lib/atlas-borg
atlas_borg_cache_dir: /var/cache/atlas-borg
atlas_borg_lock_path: /var/lib/atlas-borg/backup.lock
atlas_borg_recovery_export_path: "{{ playbook_dir }}/../secrets/recovery/atlas-borg-repokey.export"
# Manual-only offline backup. No USB device is formatted or mounted by Ansible.
atlas_manage_usb_backup: false
atlas_usb_backup_luks_uuid: ""
atlas_usb_backup_fs_uuid: ""
atlas_usb_backup_mapper_name: atlas-usb-backup
atlas_usb_backup_min_free_bytes: 10737418240
atlas_usb_backup_snapshot_prefix: atlas-usb
atlas_manage_usb_reminder: false
atlas_usb_reminder_calendar: ""
atlas_usb_reminder_notifier: /opt/45drives/houston/houston-notify
# Read-only health probes and 45Drives Alerts; disabled outside Atlas host vars.
atlas_manage_monitoring: false
atlas_monitor_calendar: "*:0/30"
atlas_monitor_notifier: "{{ atlas_usb_reminder_notifier }}"
atlas_monitor_smart_devices: []
atlas_monitor_timers: []
atlas_monitor_failure_units: []
atlas_monitor_remote_capacity: {}
atlas_monitor_pool_warning_percent: 80
atlas_monitor_pool_critical_percent: 90
atlas_monitor_root_warning_percent: 80
atlas_monitor_root_critical_percent: 90
atlas_monitor_snapshot_warning_percent: 10
atlas_monitor_snapshot_critical_percent: 20
atlas_monitor_snapshot_growth_warning_gib_day: 100
atlas_monitor_backup_growth_warning_gib_day: 100
atlas_monitor_cpu_warning_c: 85
atlas_monitor_cpu_critical_c: 95
atlas_monitor_borg_max_runtime_days: 14
# Explicit post-restore relabeling only; never relabel datasets during ordinary runs.
atlas_restorecon_paths: []
atlas_archive_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_archive }}"
atlas_services_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_services }}"
atlas_app_data_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_app_data }}"
atlas_navidrome_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_navidrome }}"
atlas_syncthing_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_syncthing }}"
atlas_media_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_media }}"
atlas_music_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_music }}"
atlas_work_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_work }}"
atlas_backup_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_backup }}"
atlas_host_backups_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_host_backups }}"
atlas_backup_prometheus_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_backup_prometheus }}"
atlas_photobook_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_photobook }}"
atlas_backups_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_backups }}"
atlas_service_backups_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_service_backups }}"
atlas_45drives_repo_url: https://repo.45drives.com/repofiles/rocky/45drives-enterprise.repo
atlas_45drives_repo_file: /etc/yum.repos.d/45drives-enterprise.repo

View File

@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Turn Borg's JSON progress stream into bounded, readable journal entries."""
import argparse
import json
import sys
import time
def size(value):
if not isinstance(value, (int, float)):
return "unknown"
return f"{value / (1024 ** 3):.2f} GiB"
parser = argparse.ArgumentParser()
parser.add_argument("--estimated-total-bytes", type=int, required=True)
args = parser.parse_args()
if args.estimated_total_bytes <= 0:
parser.error("estimated total must be positive")
last_progress = 0.0
for line in sys.stdin:
try:
event = json.loads(line)
except json.JSONDecodeError:
print(line.rstrip(), flush=True)
continue
kind = event.get("type")
if kind == "archive_progress":
now = time.monotonic()
if now - last_progress < 60 and not event.get("finished"):
continue
path = event.get("path") or ""
parts = path.split("/")
dataset = parts[1] if len(parts) > 1 and parts[0] == "source" else "unknown"
original_size = event.get("original_size")
if isinstance(original_size, (int, float)) and original_size >= 0:
percent = original_size / args.estimated_total_bytes * 100
estimated_progress = (
f"{percent:.1f}%" if percent < 100 else ">=100% (ZFS estimate exceeded)"
)
else:
estimated_progress = "unknown"
print(
"Borg create progress: "
f"estimated={estimated_progress} dataset={dataset} "
f"files={event.get('nfiles', 'unknown')} "
f"original={size(original_size)} "
f"compressed={size(event.get('compressed_size'))} "
f"deduplicated={size(event.get('deduplicated_size'))}",
flush=True,
)
last_progress = now
elif kind == "log_message":
print(f"Borg {event.get('levelname', 'INFO')}: {event.get('message', '')}", flush=True)
elif kind == "progress_message" and event.get("message"):
print(f"Borg: {event['message']}", flush=True)

View File

@@ -0,0 +1,396 @@
#!/usr/bin/python3
"""Read-only Atlas health probes with deduplicated 45Drives Alerts."""
import argparse
import fcntl
import json
import os
import re
import subprocess
import sys
import tempfile
import time
from datetime import datetime, timezone
from pathlib import Path
CONFIG_PATH = Path("/etc/atlas-health-monitor.json")
STATE_DIR = Path("/var/lib/atlas-health-monitor")
STATE_PATH = STATE_DIR / "state.json"
GIB = 1024**3
def run(*argv, timeout=40):
return subprocess.run(argv, capture_output=True, text=True, timeout=timeout, check=False)
def issue(issues, key, severity, message):
issues[key] = {"severity": severity, "message": message}
def notify(config, event, severity, subject, message):
now = datetime.now(timezone.utc)
payload = {
"timestamp": now.isoformat(timespec="seconds"),
"unixtime": int(now.timestamp()),
"event": event,
"severity": severity,
"subject": subject,
"email_message": message,
}
result = run(config["notifier"], json.dumps(payload, ensure_ascii=False), timeout=30)
if result.returncode:
raise RuntimeError(f"45Drives notifier exited {result.returncode}: {result.stderr.strip()}")
def parse_fields(text):
return dict(line.split("=", 1) for line in text.splitlines() if "=" in line)
def systemd_fields(unit, *properties):
result = run("systemctl", "show", unit, *(f"-p{item}" for item in properties))
if result.returncode:
raise RuntimeError(f"systemctl show {unit} exited {result.returncode}")
return parse_fields(result.stdout)
def unix_time(text):
if not text or text == "n/a":
return None
result = run("date", "-d", text, "+%s")
if result.returncode:
raise ValueError(f"Cannot parse systemd timestamp: {text}")
return int(result.stdout.strip())
def check_pool(config, issues, measurements):
pool = config["pool"]
listing = run("zpool", "list", "-H", "-p", "-o", "size,alloc,capacity,health", pool)
if listing.returncode:
issue(issues, "pool.probe", "critical", f"Cannot query ZFS pool {pool}")
return
try:
size, alloc, capacity, health = listing.stdout.strip().split("\t")
size, alloc, capacity = int(size), int(alloc), int(capacity)
except (ValueError, TypeError):
issue(issues, "pool.probe", "critical", "Invalid ZFS pool capacity response")
return
measurements.update(pool_size_bytes=size, pool_alloc_bytes=alloc, pool_capacity_percent=capacity)
if health != "ONLINE":
issue(issues, "pool.health", "critical", f"ZFS pool {pool} state is {health}")
if capacity >= config["pool_critical_percent"]:
issue(issues, "pool.capacity", "critical", f"ZFS pool {pool} is {capacity}% full")
elif capacity >= config["pool_warning_percent"]:
issue(issues, "pool.capacity", "warning", f"ZFS pool {pool} is {capacity}% full")
status = run("zpool", "status", "-P", pool)
if status.returncode:
issue(issues, "pool.status", "critical", f"Cannot query detailed ZFS status for {pool}")
return
bad_vdevs = []
for line in status.stdout.splitlines():
match = re.match(r"^\s*(\S+)\s+(ONLINE|DEGRADED|FAULTED|OFFLINE|UNAVAIL|REMOVED)\s+(\d+)\s+(\d+)\s+(\d+)", line)
if match:
name, state, reads, writes, checksums = match.groups()
if state != "ONLINE" or any(int(value) for value in (reads, writes, checksums)):
bad_vdevs.append(f"{name}: {state}, READ={reads}, WRITE={writes}, CKSUM={checksums}")
if bad_vdevs:
issue(issues, "pool.vdevs", "critical", "ZFS vdev errors: " + "; ".join(bad_vdevs))
errors = re.search(r"^errors:\s*(.*)$", status.stdout, re.MULTILINE)
if not errors or errors.group(1).strip() != "No known data errors":
issue(issues, "pool.data_errors", "critical", "ZFS status reports data errors; inspect zpool status -v")
if re.search(r"^\s*scan:\s*resilver in progress", status.stdout, re.MULTILINE | re.IGNORECASE):
issue(issues, "pool.resilver", "warning", "ZFS resilver is in progress; inspect zpool status")
scan = re.search(r"^\s*scan:\s*(.*)$", status.stdout, re.MULTILINE)
if scan and re.search(r"\bwith [1-9][0-9]* errors\b", scan.group(1)):
issue(issues, "pool.scan_errors", "critical", f"ZFS scan reported errors: {scan.group(1)}")
def check_capacity(config, issues, measurements):
pool = config["pool"]
listing = run("zfs", "list", "-H", "-p", "-o", "name,usedbysnapshots", "-r", pool)
if listing.returncode:
issue(issues, "snapshot.probe", "warning", "Cannot query ZFS snapshot space")
else:
try:
snapshots = sum(int(line.split("\t")[1]) for line in listing.stdout.splitlines())
measurements["snapshots_bytes"] = snapshots
size = measurements.get("pool_size_bytes")
if size:
percent = snapshots * 100 // size
measurements["snapshots_percent"] = percent
if percent >= config["snapshot_critical_percent"]:
issue(issues, "snapshot.capacity", "critical", f"Snapshots use {percent}% of pool size")
elif percent >= config["snapshot_warning_percent"]:
issue(issues, "snapshot.capacity", "warning", f"Snapshots use {percent}% of pool size")
except (ValueError, IndexError):
issue(issues, "snapshot.probe", "warning", "Invalid ZFS snapshot-space response")
backup = run("zfs", "list", "-H", "-p", "-o", "used", config["backup_dataset"])
if backup.returncode:
issue(issues, "backup.capacity_probe", "warning", "Cannot query local backup dataset space")
else:
try:
measurements["backup_bytes"] = int(backup.stdout.strip())
except ValueError:
issue(issues, "backup.capacity_probe", "warning", "Invalid local backup space response")
try:
filesystem = os.statvfs("/")
total = filesystem.f_blocks * filesystem.f_frsize
available = filesystem.f_bavail * filesystem.f_frsize
used_percent = (total - available) * 100 // total
measurements["root_capacity_percent"] = used_percent
if used_percent >= config["root_critical_percent"]:
issue(issues, "root.capacity", "critical", f"Atlas system filesystem is {used_percent}% full")
elif used_percent >= config["root_warning_percent"]:
issue(issues, "root.capacity", "warning", f"Atlas system filesystem is {used_percent}% full")
except (OSError, ZeroDivisionError):
issue(issues, "root.capacity_probe", "warning", "Cannot query Atlas system filesystem space")
def check_remote_capacity(config, issues, measurements):
"""Query only the Storage Box quota; do not open or inspect the Borg repository."""
remote = config["remote_capacity"]
try:
result = run("runuser", "-u", remote["run_as"], "--", remote["ssh_wrapper"],
f"{remote['user']}@{remote['host']}", "df", "-m", timeout=65)
if result.returncode:
raise ValueError(f"SSH df exited {result.returncode}")
lines = result.stdout.strip().splitlines()
if len(lines) != 2:
raise ValueError("Unexpected Storage Box df output")
fields = lines[1].split()
if len(fields) < 5:
raise ValueError("Incomplete Storage Box df output")
total_mib, used_mib, available_mib = (int(value) for value in fields[1:4])
percent = int(fields[4].rstrip("%"))
if total_mib <= 0 or not 0 <= percent <= 100 or available_mib < 0:
raise ValueError("Invalid Storage Box quota values")
except (OSError, ValueError, subprocess.TimeoutExpired):
issue(issues, "remote.capacity_probe", "warning", "Cannot query Hetzner Storage Box quota via pinned-key SSH")
return
measurements.update(remote_capacity_percent=percent, remote_bytes=used_mib * 1024**2,
remote_available_bytes=available_mib * 1024**2)
if percent >= remote["critical_percent"]:
issue(issues, "remote.capacity", "critical", f"Hetzner Storage Box quota is {percent}% full")
elif percent >= remote["warning_percent"]:
issue(issues, "remote.capacity", "warning", f"Hetzner Storage Box quota is {percent}% full")
def check_smart(config, issues, measurements):
for device in config["smart_devices"]:
name, path = device["name"], device["path"]
try:
result = run("smartctl", "-j", "-a", path, timeout=60)
data = json.loads(result.stdout)
status = int(data.get("smartctl", {}).get("exit_status", result.returncode))
except (subprocess.TimeoutExpired, json.JSONDecodeError, ValueError) as exc:
issue(issues, f"smart.{name}.probe", "critical", f"SMART probe failed for {name}: {type(exc).__name__}")
continue
if status:
severity = "critical" if status & 0b00001111 else "warning"
issue(issues, f"smart.{name}.status", severity, f"SMART reported exit status {status} for {name}")
passed = data.get("smart_status", {}).get("passed")
if passed is False:
issue(issues, f"smart.{name}.health", "critical", f"SMART self-assessment failed for {name}")
elif passed is None:
issue(issues, f"smart.{name}.health", "warning", f"SMART self-assessment unavailable for {name}")
temperature = data.get("temperature", {}).get("current")
if isinstance(temperature, (int, float)):
measurements[f"smart_{name}_c"] = temperature
if temperature >= device["critical_c"]:
issue(issues, f"smart.{name}.temperature", "critical", f"{name} temperature is {temperature} C")
elif temperature >= device["warning_c"]:
issue(issues, f"smart.{name}.temperature", "warning", f"{name} temperature is {temperature} C")
else:
issue(issues, f"smart.{name}.temperature", "warning", f"Temperature unavailable for {name}")
for attribute in data.get("ata_smart_attributes", {}).get("table", []):
attribute_id = attribute.get("id")
if attribute_id in (5, 187, 197, 198):
raw = attribute.get("raw", {}).get("value", 0)
if isinstance(raw, int) and raw > 0:
severity = "critical" if attribute_id in (197, 198) else "warning"
issue(issues, f"smart.{name}.ata_{attribute_id}", severity,
f"{name} SMART attribute {attribute_id} raw count is {raw}")
nvme = data.get("nvme_smart_health_information_log", {})
if isinstance(nvme, dict):
if int(nvme.get("critical_warning", 0)):
issue(issues, f"smart.{name}.nvme_warning", "critical", f"{name} NVMe critical warning is nonzero")
if int(nvme.get("media_errors", 0)):
issue(issues, f"smart.{name}.nvme_media", "critical", f"{name} NVMe media errors are nonzero")
def check_cpu(config, issues, measurements):
sensors = []
for hwmon in Path("/sys/class/hwmon").glob("hwmon*"):
try:
if (hwmon / "name").read_text().strip() != "coretemp":
continue
sensors.extend(int(path.read_text().strip()) / 1000 for path in hwmon.glob("temp*_input"))
except (OSError, ValueError):
continue
if not sensors:
issue(issues, "cpu.temperature_probe", "warning", "CPU temperature sensors are unavailable")
return
hottest = max(sensors)
measurements["cpu_max_c"] = hottest
if hottest >= config["cpu_critical_c"]:
issue(issues, "cpu.temperature", "critical", f"CPU temperature is {hottest:g} C")
elif hottest >= config["cpu_warning_c"]:
issue(issues, "cpu.temperature", "warning", f"CPU temperature is {hottest:g} C")
def check_jobs(config, issues, measurements, now):
for timer in config["timers"]:
name = timer["name"]
try:
fields = systemd_fields(name, "ActiveState", "UnitFileState", "LastTriggerUSec", "ActiveEnterTimestamp")
if fields.get("ActiveState") != "active" or fields.get("UnitFileState") != "enabled":
issue(issues, f"timer.{name}", "critical", f"Timer {name} is not active and enabled")
max_age = int(timer["max_age_hours"]) * 3600
if max_age:
last = unix_time(fields.get("LastTriggerUSec"))
if last is None:
last = unix_time(fields.get("ActiveEnterTimestamp"))
if last is not None and now - last > max_age:
issue(issues, f"timer.{name}.stale", "warning",
f"Timer {name} has not fired in {int((now-last)/3600)} hours")
except (RuntimeError, ValueError, subprocess.TimeoutExpired):
issue(issues, f"timer.{name}.probe", "warning", f"Cannot query timer {name}")
for unit in config["failure_units"]:
if unit.endswith("@.service"):
continue
try:
fields = systemd_fields(unit, "ActiveState", "Result", "ExecMainStartTimestamp")
state = fields.get("ActiveState")
if state == "failed" or (state == "inactive" and fields.get("Result") not in (None, "", "success")):
issue(issues, f"service.{unit}", "critical", f"Service {unit} failed: {fields.get('Result')}")
if unit == "atlas-borg-backup.service" and fields.get("ActiveState") == "activating":
started = unix_time(fields.get("ExecMainStartTimestamp"))
if started is not None and now - started > config["borg_max_runtime_days"] * 86400:
issue(issues, "backup.borg_long_running", "warning",
"Borg has run longer than its configured limit")
except (RuntimeError, ValueError, subprocess.TimeoutExpired):
issue(issues, f"service.{unit}.probe", "warning", f"Cannot query service {unit}")
def check_growth(config, issues, measurements, samples, now):
previous = [sample for sample in samples if 20 * 3600 <= now - sample.get("time", now) <= 48 * 3600]
if previous:
baseline = min(previous, key=lambda sample: abs(now - sample["time"] - 86400))
days = (now - baseline["time"]) / 86400
for name, threshold in (("snapshots", config["snapshot_growth_warning_gib_day"]),
("backup", config["backup_growth_warning_gib_day"]),
("remote", config["remote_capacity"]["growth_warning_gib_day"])):
current, old = measurements.get(f"{name}_bytes"), baseline.get(f"{name}_bytes")
if isinstance(current, int) and isinstance(old, int) and days > 0:
growth_gib_day = (current - old) / GIB / days
measurements[f"{name}_growth_gib_day"] = round(growth_gib_day, 1)
if growth_gib_day >= threshold:
issue(issues, f"{name}.growth", "warning",
f"Local {name} usage grew {growth_gib_day:.1f} GiB/day over {days:.1f} days")
def allowed_failure_unit(config, unit):
for allowed in config["failure_units"]:
if allowed == unit:
return True
if allowed.endswith("@.service") and unit.startswith(allowed[:-9] + "@") and unit.endswith(".service"):
return True
return False
def load_state():
if not STATE_PATH.exists():
return {"active": {}, "samples": []}
with STATE_PATH.open(encoding="utf-8") as stream:
state = json.load(stream)
if not isinstance(state.get("active"), dict) or not isinstance(state.get("samples"), list):
raise ValueError("Invalid Atlas monitor state; refusing to overwrite it")
return state
def save_state(state):
with tempfile.NamedTemporaryFile("w", dir=STATE_DIR, prefix=".state-", delete=False,
encoding="utf-8") as stream:
path = Path(stream.name)
os.chmod(path, 0o600)
json.dump(state, stream, sort_keys=True)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(path, STATE_PATH)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--dry-run", action="store_true", help="probe without notifications or state changes")
parser.add_argument("--test-notification", action="store_true", help="submit a labelled test alert")
parser.add_argument("--job-failed", metavar="UNIT", help="notify about a failed configured service")
args = parser.parse_args()
with CONFIG_PATH.open(encoding="utf-8") as stream:
config = json.load(stream)
if args.test_notification:
notify(config, "atlas_monitor_test", "warning", "Test monitoraggio Atlas",
"Notifica di prova: il monitoraggio Atlas raggiunge 45Drives Alerts. Non conferma l'invio email.")
print("Atlas monitor test submitted to 45Drives Alerts; email delivery is not verified.")
return 0
if args.job_failed:
if not allowed_failure_unit(config, args.job_failed):
raise ValueError("Unconfigured Atlas failure unit")
notify(config, "atlas_job_failed", "critical", f"Job Atlas fallito: {args.job_failed}",
f"Il servizio {args.job_failed} e' fallito. Controlla: "
f"sudo journalctl -u {args.job_failed} -n 100 --no-pager")
print(f"Atlas job failure submitted to 45Drives Alerts: {args.job_failed}")
return 0
now = int(time.time())
issues, measurements = {}, {}
check_pool(config, issues, measurements)
check_capacity(config, issues, measurements)
check_remote_capacity(config, issues, measurements)
check_smart(config, issues, measurements)
check_cpu(config, issues, measurements)
check_jobs(config, issues, measurements, now)
if args.dry_run:
print(json.dumps({"issues": issues, "measurements": measurements}, sort_keys=True))
return 0
STATE_DIR.mkdir(mode=0o700, exist_ok=True)
with (STATE_DIR / "monitor.lock").open("w") as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
state = load_state()
check_growth(config, issues, measurements, state["samples"], now)
active, failed_notifications = state["active"], []
for key, details in issues.items():
old = active.get(key)
if old is None or old.get("severity") != details["severity"]:
try:
notify(config, "atlas_health_issue", details["severity"],
f"Atlas: {key}", details["message"])
active[key] = details
print(f"ALERT {details['severity']} {key}: {details['message']}", flush=True)
except (RuntimeError, subprocess.TimeoutExpired) as exc:
failed_notifications.append(key)
print(f"NOTIFICATION FAILED {key}: {exc}", file=sys.stderr, flush=True)
for key in set(active) - set(issues):
print(f"RECOVERED {key}", flush=True)
del active[key]
state["samples"] = [sample for sample in state["samples"] if now - sample.get("time", 0) < 48 * 3600]
state["samples"].append({"time": now, **{key: value for key, value in measurements.items()
if key in ("snapshots_bytes", "backup_bytes", "remote_bytes")}})
save_state(state)
print(f"Atlas health: issues={len(issues)} notifications_failed={len(failed_notifications)} "
f"pool={measurements.get('pool_capacity_percent', 'unknown')}% "
f"remote={measurements.get('remote_capacity_percent', 'unknown')}% "
f"snapshots={measurements.get('snapshots_bytes', 'unknown')} bytes "
f"backup={measurements.get('backup_bytes', 'unknown')} bytes", flush=True)
return 1 if failed_notifications else 0
if __name__ == "__main__":
try:
sys.exit(main())
except (OSError, RuntimeError, ValueError, subprocess.TimeoutExpired) as error:
print(f"Atlas health monitor failed: {error}", file=sys.stderr)
sys.exit(1)

View File

@@ -28,6 +28,18 @@
name: smb
state: restarted
- name: Restart Atlas Borg timers
ansible.builtin.systemd:
name: "{{ item }}"
state: restarted
daemon_reload: true
loop:
- atlas-borg-backup.timer
- atlas-borg-check.timer
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Restart Atlas media Quadlets
ansible.builtin.systemd:
name: "{{ item }}"

View File

@@ -0,0 +1,538 @@
---
- name: Validate Atlas Borg backup configuration
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.assert:
that:
- atlas_manage_storage | bool
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
- atlas_mount_root.startswith('/')
- atlas_borg_username is match('^[a-z_][a-z0-9_-]*$')
- atlas_borg_group is match('^[a-z_][a-z0-9_-]*$')
- atlas_borg_username not in ['root', atlas_admin_username]
- atlas_borg_group != 'wheel'
- atlas_borg_home.startswith('/var/lib/')
- atlas_borg_repository_host is match('^[A-Za-z0-9.-]+$')
- atlas_borg_repository_user is match('^[A-Za-z0-9_-]+$')
- atlas_borg_repository_port | int > 0
- atlas_borg_repository_port | int < 65536
- atlas_borg_repository_path is match('^\./[A-Za-z0-9][A-Za-z0-9._/-]*$')
- "'/../' not in ('/' ~ atlas_borg_repository_path ~ '/')"
- atlas_borg_remote_path is match('^borg-[0-9]+\.[0-9]+$')
- atlas_borg_host_key.startswith(
'[' ~ atlas_borg_repository_host ~ ']:' ~ (atlas_borg_repository_port | string) ~ ' ssh-ed25519 '
)
- atlas_borg_ssh_private_key_path.startswith('/etc/atlas-borg/')
- atlas_borg_known_hosts_path.startswith('/etc/atlas-borg/')
- atlas_borg_passphrase_path.startswith('/etc/atlas-borg/')
- atlas_borg_ssh_wrapper_path.startswith('/usr/local/libexec/')
- atlas_borg_encryption_mode == 'repokey'
- atlas_borg_archive_prefix is match('^[a-z0-9][a-z0-9_-]*$')
- atlas_borg_snapshot_prefix is match('^[a-z0-9][a-z0-9_-]*$')
- atlas_borg_keep_daily | int > 0
- atlas_borg_keep_weekly | int > 0
- atlas_borg_keep_monthly | int > 0
fail_msg: >-
Atlas Borg needs a safe relative repository path, a pinned ED25519 host
key, positive retention counts, and valid dedicated SSH settings.
when: atlas_manage_borg_backup | bool
- name: Create the Atlas Borg system group
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.group:
name: "{{ atlas_borg_group }}"
system: true
state: present
when: atlas_manage_borg_backup | bool
- name: Create the least-privilege Atlas Borg account
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.user:
name: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
groups: []
append: false
comment: Atlas Borg backup service
home: "{{ atlas_borg_home }}"
create_home: false
shell: /sbin/nologin
password_lock: true
system: true
state: present
when: atlas_manage_borg_backup | bool
- name: Read Atlas Borg account group membership
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.command:
argv:
- id
- -nG
- "{{ atlas_borg_username }}"
register: atlas_borg_account_groups
changed_when: false
check_mode: false
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Require the Atlas Borg account to have no supplementary groups
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.assert:
that:
- atlas_borg_account_groups.stdout.split() == [atlas_borg_group]
fail_msg: >-
The Atlas Borg service account must belong only to its private primary
group and must never receive wheel or other supplementary membership.
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Validate Atlas Borg systemd calendars
tags: [atlas, storage, backup, borg]
ansible.builtin.command:
argv:
- systemd-analyze
- calendar
- "{{ item }}"
loop:
- "{{ atlas_borg_backup_calendar }}"
- "{{ atlas_borg_check_calendar }}"
changed_when: false
check_mode: false
when: atlas_manage_borg_backup | bool
- name: Create Atlas Borg configuration directory
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.file:
path: /etc/atlas-borg
state: directory
owner: root
group: "{{ atlas_borg_group }}"
mode: "0750"
when: atlas_manage_borg_backup | bool
- name: Generate the dedicated Atlas Borg SSH identity
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.command:
argv:
- ssh-keygen
- -q
- -t
- ed25519
- -N
- ""
- -C
- atlas-borg@atlas
- -f
- "{{ atlas_borg_ssh_private_key_path }}"
creates: "{{ atlas_borg_ssh_private_key_path }}"
when: atlas_manage_borg_backup | bool
- name: Protect the Atlas Borg private SSH identity
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.file:
path: "{{ atlas_borg_ssh_private_key_path }}"
owner: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
mode: "0600"
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Set permissions on the Atlas Borg public SSH identity
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.file:
path: "{{ atlas_borg_ssh_private_key_path }}.pub"
owner: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
mode: "0644"
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Read the dedicated Atlas Borg public SSH identity
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.slurp:
src: "{{ atlas_borg_ssh_private_key_path }}.pub"
register: atlas_borg_public_key
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Report the public SSH identity to install in the Hetzner sub-account
tags: [atlas, storage, backup, borg, borg_key]
ansible.builtin.debug:
msg: "{{ atlas_borg_public_key.content | b64decode | trim }}"
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Pin the Hetzner Storage Box SSH host key
tags: [atlas, storage, backup, borg]
ansible.builtin.copy:
content: "{{ atlas_borg_host_key }}\n"
dest: "{{ atlas_borg_known_hosts_path }}"
owner: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
mode: "0600"
when: atlas_manage_borg_backup | bool
- name: Create Atlas Borg state directories
tags: [atlas, storage, backup, borg]
ansible.builtin.file:
path: "{{ item }}"
state: directory
owner: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
mode: "0700"
loop:
- "{{ atlas_borg_config_dir }}"
- "{{ atlas_borg_cache_dir }}"
when: atlas_manage_borg_backup | bool
- name: Create the shared Atlas Borg operation lock
tags: [atlas, storage, backup, borg]
ansible.builtin.copy:
content: ""
dest: "{{ atlas_borg_lock_path }}"
owner: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
mode: "0600"
force: false
when: atlas_manage_borg_backup | bool
- name: Require the Atlas Borg encryption passphrase from Vault
tags: [atlas, storage, backup, borg]
ansible.builtin.assert:
that:
- atlas_borg_passphrase | length >= 20
fail_msg: >-
Define vault_atlas_borg_passphrase with a strong unique value in the
encrypted Vault before activating the Borg repository.
no_log: true
when: atlas_manage_borg_backup | bool
- name: Install the Atlas Borg passphrase
tags: [atlas, storage, backup, borg]
ansible.builtin.copy:
content: "{{ atlas_borg_passphrase }}\n"
dest: "{{ atlas_borg_passphrase_path }}"
owner: "{{ atlas_borg_username }}"
group: "{{ atlas_borg_group }}"
mode: "0600"
diff: false
no_log: true
when: atlas_manage_borg_backup | bool
- name: Install the Atlas Borg backup helper
tags: [atlas, storage, backup, borg, borg_logging]
ansible.builtin.template:
src: atlas-borg-backup.sh.j2
dest: /usr/local/sbin/atlas-borg-backup
owner: root
group: root
mode: "0750"
when: atlas_manage_borg_backup | bool
- name: Install the Atlas Borg snapshot cleanup helper
tags: [atlas, storage, backup, borg, borg_logging]
ansible.builtin.template:
src: atlas-borg-snapshot-cleanup.sh.j2
dest: /usr/local/sbin/atlas-borg-snapshot-cleanup
owner: root
group: root
mode: "0750"
when: atlas_manage_borg_backup | bool
- name: Install the Atlas Borg check helper
tags: [atlas, storage, backup, borg]
ansible.builtin.template:
src: atlas-borg-check.sh.j2
dest: /usr/local/sbin/atlas-borg-check
owner: root
group: "{{ atlas_borg_group }}"
mode: "0750"
when: atlas_manage_borg_backup | bool
- name: Create the local libexec directory for the Atlas Borg SSH wrapper
tags: [atlas, storage, backup, borg, borg_logging]
ansible.builtin.file:
path: "{{ atlas_borg_ssh_wrapper_path | dirname }}"
state: directory
owner: root
group: root
mode: "0755"
when: atlas_manage_borg_backup | bool
- name: Install the Atlas Borg progress formatter
tags: [atlas, storage, backup, borg, borg_logging]
ansible.builtin.copy:
src: atlas-borg-progress.py
dest: /usr/local/libexec/atlas-borg-progress
owner: root
group: root
mode: "0755"
when: atlas_manage_borg_backup | bool
- name: Install the capability-dropping Atlas Borg SSH wrapper
tags: [atlas, storage, backup, borg]
ansible.builtin.template:
src: atlas-borg-ssh.sh.j2
dest: "{{ atlas_borg_ssh_wrapper_path }}"
owner: root
group: root
mode: "0755"
when: atlas_manage_borg_backup | bool
- name: Install Atlas Borg systemd units
tags: [atlas, storage, backup, borg, borg_logging]
ansible.builtin.template:
src: "{{ item }}.j2"
dest: "/etc/systemd/system/{{ item }}"
owner: root
group: root
mode: "0644"
loop:
- atlas-borg-backup.service
- atlas-borg-backup.timer
- atlas-borg-check.service
- atlas-borg-check.timer
notify: Restart Atlas Borg timers
when: atlas_manage_borg_backup | bool
- name: Verify dedicated SSH access to the Hetzner Storage Box
tags: [atlas, storage, backup, borg]
ansible.builtin.command:
argv:
- ssh
- -T
- -i
- "{{ atlas_borg_ssh_private_key_path }}"
- -p
- "{{ atlas_borg_repository_port | string }}"
- -o
- BatchMode=yes
- -o
- IdentitiesOnly=yes
- -o
- StrictHostKeyChecking=yes
- -o
- "UserKnownHostsFile={{ atlas_borg_known_hosts_path }}"
- "{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}"
- pwd
register: atlas_borg_ssh_probe
become: true
become_user: "{{ atlas_borg_username }}"
changed_when: false
failed_when: false
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Require the dedicated public key on the Hetzner sub-account
tags: [atlas, storage, backup, borg]
ansible.builtin.assert:
that:
- atlas_borg_ssh_probe.rc == 0
fail_msg: >-
Install the reported Atlas Borg public key in the Hetzner sub-account
before rerunning the Borg tasks. Password authentication is never used.
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Probe the remote Atlas Borg repository path
tags: [atlas, storage, backup, borg]
ansible.builtin.command:
argv:
- ssh
- -T
- -i
- "{{ atlas_borg_ssh_private_key_path }}"
- -p
- "{{ atlas_borg_repository_port | string }}"
- -o
- BatchMode=yes
- -o
- IdentitiesOnly=yes
- -o
- StrictHostKeyChecking=yes
- -o
- "UserKnownHostsFile={{ atlas_borg_known_hosts_path }}"
- "{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}"
- stat
- "{{ atlas_borg_repository_path }}"
register: atlas_borg_repository_path_probe
become: true
become_user: "{{ atlas_borg_username }}"
changed_when: false
failed_when: false
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Probe the Atlas Borg repository
tags: [atlas, storage, backup, borg]
ansible.builtin.command:
argv:
- /usr/bin/borg
- --remote-path
- "{{ atlas_borg_remote_path }}"
- info
- >-
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
environment:
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
register: atlas_borg_repository_probe
become: true
become_user: "{{ atlas_borg_username }}"
changed_when: false
failed_when: false
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- atlas_borg_repository_path_probe.rc == 0
- name: Reject an existing path that is not the configured Borg repository
tags: [atlas, storage, backup, borg]
ansible.builtin.assert:
that:
- atlas_borg_repository_probe.rc == 0
fail_msg: >-
The remote repository path already exists but Borg could not open it.
Refusing to initialize over existing data; verify the path, passphrase,
and repository state manually.
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- atlas_borg_repository_path_probe.rc == 0
- name: Initialize the encrypted Atlas Borg repository
tags: [atlas, storage, backup, borg]
ansible.builtin.command:
argv:
- /usr/bin/borg
- --remote-path
- "{{ atlas_borg_remote_path }}"
- init
- --encryption
- "{{ atlas_borg_encryption_mode }}"
- >-
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
environment:
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
become: true
become_user: "{{ atlas_borg_username }}"
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- atlas_borg_repository_path_probe.rc != 0
- name: Verify the encrypted Atlas Borg repository
tags: [atlas, storage, backup, borg]
ansible.builtin.command:
argv:
- /usr/bin/borg
- --remote-path
- "{{ atlas_borg_remote_path }}"
- info
- >-
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
environment:
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
become: true
become_user: "{{ atlas_borg_username }}"
changed_when: false
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Check for the local Atlas Borg recovery-key export
tags: [atlas, storage, backup, borg]
ansible.builtin.stat:
path: "{{ atlas_borg_recovery_export_path }}"
register: atlas_borg_recovery_export
delegate_to: localhost
become: false
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- name: Export the Atlas Borg recovery key for offline preservation
tags: [atlas, storage, backup, borg]
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode
- not atlas_borg_recovery_export.stat.exists
no_log: true
block:
- name: Create the local recovery-material directory
ansible.builtin.file:
path: "{{ atlas_borg_recovery_export_path | dirname }}"
state: directory
mode: "0700"
delegate_to: localhost
become: false
- name: Export the encrypted Borg repository key on Atlas
ansible.builtin.command:
argv:
- /usr/bin/borg
- --remote-path
- "{{ atlas_borg_remote_path }}"
- key
- export
- >-
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
- "{{ atlas_borg_config_dir }}/atlas-borg-repokey.export"
environment:
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
become: true
become_user: "{{ atlas_borg_username }}"
- name: Fetch the encrypted Borg recovery key from Atlas
ansible.builtin.fetch:
src: "{{ atlas_borg_config_dir }}/atlas-borg-repokey.export"
dest: "{{ atlas_borg_recovery_export_path }}"
flat: true
- name: Protect the local Borg recovery-key export
ansible.builtin.file:
path: "{{ atlas_borg_recovery_export_path }}"
mode: "0600"
delegate_to: localhost
become: false
always:
- name: Remove the temporary recovery-key export from Atlas
ansible.builtin.file:
path: "{{ atlas_borg_config_dir }}/atlas-borg-repokey.export"
state: absent
- name: Enable Atlas Borg backup and check timers
tags: [atlas, storage, backup, borg]
ansible.builtin.systemd:
name: "{{ item }}"
enabled: true
state: started
daemon_reload: true
loop:
- atlas-borg-backup.timer
- atlas-borg-check.timer
when:
- atlas_manage_borg_backup | bool
- not ansible_check_mode

View File

@@ -14,6 +14,21 @@
- name: Import Atlas storage tasks
ansible.builtin.import_tasks: storage.yml
- name: Import Atlas ZFS maintenance tasks
ansible.builtin.import_tasks: zfs_maintenance.yml
- name: Import Atlas Borg backup tasks
ansible.builtin.import_tasks: borg_backup.yml
- name: Import Atlas offline USB backup tasks
ansible.builtin.import_tasks: usb_backup.yml
- name: Import Atlas health monitoring tasks
ansible.builtin.import_tasks: monitoring.yml
- name: Import Atlas post-restore SELinux relabeling tasks
ansible.builtin.import_tasks: restorecon.yml
- name: Import Atlas file sharing tasks
ansible.builtin.import_tasks: sharing.yml

View File

@@ -0,0 +1,201 @@
---
- name: Validate Atlas health monitoring policy
tags: [atlas, monitoring]
ansible.builtin.assert:
that:
- atlas_manage_storage | bool
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
- atlas_monitor_calendar | length > 0
- atlas_monitor_smart_devices | length > 0
- atlas_monitor_timers | length > 0
- atlas_monitor_failure_units | length > 0
- atlas_monitor_remote_capacity.user == atlas_borg_repository_user
- atlas_monitor_remote_capacity.host == atlas_borg_repository_host
- atlas_monitor_remote_capacity.run_as == atlas_borg_username
- atlas_monitor_remote_capacity.ssh_wrapper == atlas_borg_ssh_wrapper_path
- >-
0 < atlas_monitor_remote_capacity.warning_percent | int
< atlas_monitor_remote_capacity.critical_percent | int < 100
- atlas_monitor_remote_capacity.growth_warning_gib_day | int > 0
- atlas_monitor_notifier.startswith('/opt/45drives/houston/')
- 0 < atlas_monitor_pool_warning_percent | int < atlas_monitor_pool_critical_percent | int < 100
- 0 < atlas_monitor_root_warning_percent | int < atlas_monitor_root_critical_percent | int < 100
- 0 < atlas_monitor_snapshot_warning_percent | int < atlas_monitor_snapshot_critical_percent | int < 100
- atlas_monitor_snapshot_growth_warning_gib_day | int > 0
- atlas_monitor_backup_growth_warning_gib_day | int > 0
- 0 < atlas_monitor_cpu_warning_c | int < atlas_monitor_cpu_critical_c | int
- atlas_monitor_borg_max_runtime_days | int > 0
fail_msg: >-
Atlas health monitoring needs real devices, job units, a valid calendar,
positive ordered thresholds, and the existing Houston notifier.
when: atlas_manage_monitoring | bool
- name: Validate monitored Atlas SMART devices
tags: [atlas, monitoring]
ansible.builtin.assert:
that:
- item.name is match('^[a-z0-9][a-z0-9_-]*$')
- item.path.startswith('/dev/disk/by-id/')
- 0 < item.warning_c | int < item.critical_c | int
fail_msg: "Every monitored disk needs a stable by-id path and ordered temperature thresholds."
loop: "{{ atlas_monitor_smart_devices }}"
loop_control:
label: "{{ item.name }}"
when: atlas_manage_monitoring | bool
- name: Validate monitored Atlas timer names and age thresholds
tags: [atlas, monitoring]
ansible.builtin.assert:
that:
- item.name is match('^[a-zA-Z0-9@_.-]+\\.timer$')
- item.max_age_hours | int >= 0
loop: "{{ atlas_monitor_timers }}"
loop_control:
label: "{{ item.name }}"
when: atlas_manage_monitoring | bool
- name: Validate monitored Atlas failure unit names
tags: [atlas, monitoring]
ansible.builtin.assert:
that:
- item is match('^[a-zA-Z0-9@_.-]+\\.service$')
loop: "{{ atlas_monitor_failure_units }}"
when: atlas_manage_monitoring | bool
- name: Validate Atlas health monitor calendar
tags: [atlas, monitoring]
ansible.builtin.command:
argv: [systemd-analyze, calendar, "{{ atlas_monitor_calendar }}"]
changed_when: false
check_mode: false
when: atlas_manage_monitoring | bool
- name: Install SMART tooling for Atlas health checks
tags: [atlas, monitoring, packages]
ansible.builtin.dnf:
name: smartmontools
state: present
when: atlas_manage_monitoring | bool
- name: Inspect the existing 45Drives notifier for monitoring
tags: [atlas, monitoring]
ansible.builtin.stat:
path: "{{ atlas_monitor_notifier }}"
register: atlas_monitor_notifier_file
when: atlas_manage_monitoring | bool
- name: Require the existing 45Drives notifier for monitoring
tags: [atlas, monitoring]
ansible.builtin.assert:
that:
- atlas_monitor_notifier_file.stat.executable | default(false)
fail_msg: "The existing 45Drives Houston notifier must be executable."
when: atlas_manage_monitoring | bool
- name: Create private Atlas health monitor state directory
tags: [atlas, monitoring]
ansible.builtin.file:
path: /var/lib/atlas-health-monitor
state: directory
owner: root
group: root
mode: "0700"
when: atlas_manage_monitoring | bool
- name: Install Atlas health monitor configuration
tags: [atlas, monitoring]
ansible.builtin.template:
src: atlas-health-monitor.json.j2
dest: /etc/atlas-health-monitor.json
owner: root
group: root
mode: "0600"
when: atlas_manage_monitoring | bool
- name: Install Atlas health monitor helper
tags: [atlas, monitoring]
ansible.builtin.copy:
src: atlas-health-monitor.py
dest: /usr/local/libexec/atlas-health-monitor
owner: root
group: root
mode: "0750"
when: atlas_manage_monitoring | bool
- name: Install Atlas health monitoring units
tags: [atlas, monitoring]
ansible.builtin.template:
src: "{{ item }}.j2"
dest: "/etc/systemd/system/{{ item }}"
owner: root
group: root
mode: "0644"
loop:
- atlas-health-monitor.service
- atlas-health-monitor.timer
- atlas-monitor-failure@.service
when: atlas_manage_monitoring | bool
- name: Create failure hook directories for monitored Atlas jobs
tags: [atlas, monitoring]
ansible.builtin.file:
path: "/etc/systemd/system/{{ item }}.d"
state: directory
owner: root
group: root
mode: "0755"
loop: "{{ atlas_monitor_failure_units }}"
when: atlas_manage_monitoring | bool
- name: Notify 45Drives Alerts when an Atlas job fails
tags: [atlas, monitoring]
ansible.builtin.template:
src: atlas-monitor-failure.conf.j2
dest: "/etc/systemd/system/{{ item }}.d/atlas-monitor.conf"
owner: root
group: root
mode: "0644"
loop: "{{ atlas_monitor_failure_units }}"
when: atlas_manage_monitoring | bool
- name: Reload systemd after installing Atlas monitoring
tags: [atlas, monitoring]
ansible.builtin.systemd:
daemon_reload: true
when:
- atlas_manage_monitoring | bool
- not ansible_check_mode
- name: Enable the Atlas health monitoring timer
tags: [atlas, monitoring]
ansible.builtin.systemd:
name: atlas-health-monitor.timer
enabled: true
state: started
when:
- atlas_manage_monitoring | bool
- not ansible_check_mode
- name: Validate the deployed Atlas health monitoring units
tags: [atlas, monitoring]
ansible.builtin.command:
argv:
- systemd-analyze
- verify
- atlas-health-monitor.service
- atlas-health-monitor.timer
- atlas-monitor-failure@.service
changed_when: false
when:
- atlas_manage_monitoring | bool
- not ansible_check_mode
- name: Probe Atlas health without sending notifications
tags: [atlas, monitoring]
ansible.builtin.command:
argv: [/usr/local/libexec/atlas-health-monitor, --dry-run]
register: atlas_monitor_dry_run
changed_when: false
when:
- atlas_manage_monitoring | bool
- not ansible_check_mode

View File

@@ -0,0 +1,27 @@
---
- name: Validate requested Atlas post-restore relabel paths
tags: [atlas, restorecon, recovery]
ansible.builtin.assert:
that:
- item is string
- item.startswith(atlas_mount_root ~ '/')
- item != atlas_mount_root
fail_msg: >-
Post-restore relabeling accepts only explicit paths below the Atlas pool
mount root. Do not relabel the whole pool during routine provisioning.
loop: "{{ atlas_restorecon_paths }}"
when: atlas_restorecon_paths | length > 0
- name: Restore SELinux labels on explicitly restored Atlas paths
tags: [atlas, restorecon, recovery]
ansible.builtin.command:
argv:
- restorecon
- -RFv
- "{{ item }}"
register: atlas_restorecon_result
changed_when: atlas_restorecon_result.stdout | length > 0
loop: "{{ atlas_restorecon_paths }}"
when:
- atlas_restorecon_paths | length > 0
- not ansible_check_mode

View File

@@ -3,12 +3,6 @@
tags: [atlas, storage]
ansible.builtin.set_fact:
atlas_zfs_datasets:
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_work }}"
mountpoint: "{{ atlas_work_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
mountpoint: "{{ atlas_work_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_archive }}"
mountpoint: "{{ atlas_archive_mountpoint }}"
owner: "{{ atlas_admin_username }}"
@@ -16,6 +10,13 @@
properties:
compression: zstd
mountpoint: "{{ atlas_archive_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_services }}"
mountpoint: "{{ atlas_services_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
compression: zstd
mountpoint: "{{ atlas_services_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_app_data }}"
mountpoint: "{{ atlas_app_data_mountpoint }}"
owner: "{{ atlas_admin_username }}"
@@ -51,11 +52,27 @@
properties:
compression: lz4
mountpoint: "{{ atlas_music_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_backup }}"
mountpoint: "{{ atlas_backup_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
compression: lz4
mountpoint: "{{ atlas_backup_mountpoint }}"
reservation: "{{ atlas_zfs_backup_reservation }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_host_backups }}"
mountpoint: "{{ atlas_host_backups_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
compression: lz4
mountpoint: "{{ atlas_host_backups_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_backup_prometheus }}"
mountpoint: "{{ atlas_backup_prometheus_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
compression: lz4
mountpoint: "{{ atlas_backup_prometheus_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_photobook }}"
mountpoint: "{{ atlas_photobook_mountpoint }}"
@@ -64,21 +81,6 @@
properties:
compression: lz4
mountpoint: "{{ atlas_photobook_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_backups }}"
mountpoint: "{{ atlas_backups_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
compression: lz4
mountpoint: "{{ atlas_backups_mountpoint }}"
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_service_backups }}"
mountpoint: "{{ atlas_service_backups_mountpoint }}"
owner: "{{ atlas_admin_username }}"
group: "{{ atlas_admin_group }}"
properties:
compression: lz4
mountpoint: "{{ atlas_service_backups_mountpoint }}"
refreservation: "{{ atlas_zfs_service_backups_refreservation }}"
- name: Require completed Atlas storage placeholders
tags: [atlas, storage]

View File

@@ -0,0 +1,145 @@
---
- name: Validate Atlas offline USB backup configuration
tags: [atlas, storage, backup, usb_backup]
ansible.builtin.assert:
that:
- atlas_manage_storage | bool
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
- atlas_mount_root.startswith('/')
- atlas_usb_backup_luks_uuid is match('^[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}$')
- atlas_usb_backup_fs_uuid is match('^[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}$')
- atlas_usb_backup_luks_uuid != atlas_usb_backup_fs_uuid
- atlas_usb_backup_mapper_name is match('^[a-z][a-z0-9_-]*$')
- atlas_usb_backup_min_free_bytes | int > 0
- atlas_usb_backup_snapshot_prefix is match('^[a-z0-9][a-z0-9_-]*$')
- atlas_usb_backup_snapshot_prefix != atlas_borg_snapshot_prefix
- atlas_usb_backup_snapshot_prefix != atlas_zfs_snapshot_prefix
fail_msg: >-
The manual Atlas USB backup needs verified LUKS and ext4 UUIDs, a safe
mapper name, positive free-space reserve, and a unique snapshot prefix.
when: atlas_manage_usb_backup | bool
- name: Install rsync for the Atlas offline USB backup
tags: [atlas, storage, backup, usb_backup]
ansible.builtin.dnf:
name: rsync
state: present
when: atlas_manage_usb_backup | bool
- name: Install the manual Atlas offline USB backup helper
tags: [atlas, storage, backup, usb_backup]
ansible.builtin.template:
src: atlas-usb-backup.sh.j2
dest: /usr/local/sbin/atlas-usb-backup
owner: root
group: root
mode: "0750"
when: atlas_manage_usb_backup | bool
- name: Install the Atlas USB snapshot cleanup helper
tags: [atlas, storage, backup, usb_backup]
ansible.builtin.template:
src: atlas-usb-snapshot-cleanup.sh.j2
dest: /usr/local/sbin/atlas-usb-snapshot-cleanup
owner: root
group: root
mode: "0750"
when: atlas_manage_usb_backup | bool
- name: Install the manual Atlas offline USB backup service
tags: [atlas, storage, backup, usb_backup]
ansible.builtin.template:
src: atlas-usb-backup.service.j2
dest: /etc/systemd/system/atlas-usb-backup.service
owner: root
group: root
mode: "0644"
when: atlas_manage_usb_backup | bool
- name: Reload systemd for the Atlas offline USB backup service
tags: [atlas, storage, backup, usb_backup]
ansible.builtin.systemd:
daemon_reload: true
when:
- atlas_manage_usb_backup | bool
- not ansible_check_mode
- name: Validate the 45Drives Atlas USB reminder configuration
tags: [atlas, backup, usb_reminder]
ansible.builtin.assert:
that:
- atlas_manage_usb_backup | bool
- atlas_usb_reminder_calendar | length > 0
- atlas_usb_reminder_notifier.startswith('/opt/45drives/houston/')
fail_msg: >-
Enable the manual USB backup and declare a systemd calendar before
enabling its 45Drives Alerts reminder.
when: atlas_manage_usb_reminder | bool
- name: Validate the Atlas USB reminder calendar
tags: [atlas, backup, usb_reminder]
ansible.builtin.command:
argv:
- systemd-analyze
- calendar
- "{{ atlas_usb_reminder_calendar }}"
changed_when: false
check_mode: false
when: atlas_manage_usb_reminder | bool
- name: Inspect the existing 45Drives notifier
tags: [atlas, backup, usb_reminder]
ansible.builtin.stat:
path: "{{ atlas_usb_reminder_notifier }}"
register: atlas_usb_reminder_notifier_file
when: atlas_manage_usb_reminder | bool
- name: Require the configured 45Drives notifier for USB reminders
tags: [atlas, backup, usb_reminder]
ansible.builtin.assert:
that:
- atlas_usb_reminder_notifier_file.stat.executable | default(false)
fail_msg: >-
The existing 45Drives Houston notifier must be executable.
when: atlas_manage_usb_reminder | bool
- name: Install the 45Drives Atlas USB reminder helper
tags: [atlas, backup, usb_reminder]
ansible.builtin.template:
src: atlas-usb-reminder.py.j2
dest: /usr/local/libexec/atlas-usb-reminder
owner: root
group: root
mode: "0750"
when: atlas_manage_usb_reminder | bool
- name: Install the 45Drives Atlas USB reminder service
tags: [atlas, backup, usb_reminder]
ansible.builtin.template:
src: atlas-usb-reminder.service.j2
dest: /etc/systemd/system/atlas-usb-reminder.service
owner: root
group: root
mode: "0644"
when: atlas_manage_usb_reminder | bool
- name: Install the 45Drives Atlas USB reminder timer
tags: [atlas, backup, usb_reminder]
ansible.builtin.template:
src: atlas-usb-reminder.timer.j2
dest: /etc/systemd/system/atlas-usb-reminder.timer
owner: root
group: root
mode: "0644"
when: atlas_manage_usb_reminder | bool
- name: Enable only the Atlas USB notification reminder timer
tags: [atlas, backup, usb_reminder]
ansible.builtin.systemd:
name: atlas-usb-reminder.timer
enabled: true
state: started
daemon_reload: true
when:
- atlas_manage_usb_reminder | bool
- not ansible_check_mode

View File

@@ -0,0 +1,175 @@
---
- name: Validate Atlas ZFS snapshot policy
tags: [atlas, storage, snapshots]
ansible.builtin.assert:
that:
- atlas_manage_storage | bool
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
- atlas_zfs_snapshot_prefix is match('^[a-z0-9][a-z0-9_-]*$')
- atlas_zfs_snapshot_policies | length > 0
- >-
(atlas_zfs_snapshot_policies | map(attribute='name') | unique | list | length)
== (atlas_zfs_snapshot_policies | length)
fail_msg: >-
Enable Atlas storage and declare a non-empty snapshot policy with a safe
prefix and unique policy names before managing automatic snapshots.
when: atlas_manage_zfs_snapshots | bool
- name: Validate Atlas ZFS snapshot policy entries
tags: [atlas, storage, snapshots]
ansible.builtin.assert:
that:
- item.name is match('^[a-z][a-z0-9_-]*$')
- item.keep | int > 0
- item.calendar | length > 0
fail_msg: >-
Every Atlas snapshot policy needs a safe name, a positive retention
count, and a systemd calendar expression.
loop: "{{ atlas_zfs_snapshot_policies }}"
loop_control:
label: "{{ item.name | default('unnamed') }}"
when: atlas_manage_zfs_snapshots | bool
- name: Validate Atlas ZFS snapshot calendars
tags: [atlas, storage, snapshots]
ansible.builtin.command:
argv:
- systemd-analyze
- calendar
- "{{ item.calendar }}"
loop: "{{ atlas_zfs_snapshot_policies }}"
loop_control:
label: "{{ item.name }}: {{ item.calendar }}"
changed_when: false
check_mode: false
when: atlas_manage_zfs_snapshots | bool
- name: Install Atlas ZFS snapshot and retention helper
tags: [atlas, storage, snapshots]
ansible.builtin.template:
src: atlas-zfs-snapshot.sh.j2
dest: /usr/local/sbin/atlas-zfs-snapshot
owner: root
group: root
mode: "0750"
when: atlas_manage_zfs_snapshots | bool
- name: Install Atlas ZFS snapshot systemd service
tags: [atlas, storage, snapshots]
ansible.builtin.template:
src: atlas-zfs-snapshot@.service.j2
dest: /etc/systemd/system/atlas-zfs-snapshot@.service
owner: root
group: root
mode: "0644"
when: atlas_manage_zfs_snapshots | bool
- name: Install Atlas ZFS snapshot systemd timers
tags: [atlas, storage, snapshots]
ansible.builtin.template:
src: atlas-zfs-snapshot.timer.j2
dest: "/etc/systemd/system/atlas-zfs-snapshot-{{ item.name }}.timer"
owner: root
group: root
mode: "0644"
loop: "{{ atlas_zfs_snapshot_policies }}"
loop_control:
label: "{{ item.name }}"
when: atlas_manage_zfs_snapshots | bool
- name: Enable Atlas ZFS snapshot systemd timers
tags: [atlas, storage, snapshots]
ansible.builtin.systemd:
name: "atlas-zfs-snapshot-{{ item.name }}.timer"
enabled: true
state: started
daemon_reload: true
loop: "{{ atlas_zfs_snapshot_policies }}"
loop_control:
label: "{{ item.name }}"
when:
- atlas_manage_zfs_snapshots | bool
- not ansible_check_mode
- name: Validate Atlas ZFS scrub policy
tags: [atlas, storage, scrub]
ansible.builtin.assert:
that:
- atlas_manage_storage | bool
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
- atlas_zfs_scrub_calendar | length > 0
fail_msg: >-
Enable Atlas storage and declare a systemd calendar expression before
managing periodic ZFS scrubs.
when: atlas_manage_zfs_scrub | bool
- name: Validate Atlas ZFS scrub calendar
tags: [atlas, storage, scrub]
ansible.builtin.command:
argv:
- systemd-analyze
- calendar
- "{{ atlas_zfs_scrub_calendar }}"
changed_when: false
check_mode: false
when: atlas_manage_zfs_scrub | bool
- name: Require OpenZFS scrub systemd units
tags: [atlas, storage, scrub]
ansible.builtin.command:
argv:
- systemctl
- cat
- "{{ item }}"
loop:
- "zfs-scrub@{{ atlas_zfs_pool }}.service"
- "zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer"
- "zfs-scrub-weekly@{{ atlas_zfs_pool }}.timer"
loop_control:
label: "{{ item }}"
changed_when: false
check_mode: false
when: atlas_manage_zfs_scrub | bool
- name: Create Atlas ZFS scrub timer override directory
tags: [atlas, storage, scrub]
ansible.builtin.file:
path: "/etc/systemd/system/zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer.d"
state: directory
owner: root
group: root
mode: "0755"
when: atlas_manage_zfs_scrub | bool
- name: Configure Atlas ZFS monthly scrub schedule
tags: [atlas, storage, scrub]
ansible.builtin.template:
src: atlas-zfs-scrub-timer.conf.j2
dest: >-
/etc/systemd/system/zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer.d/override.conf
owner: root
group: root
mode: "0644"
when: atlas_manage_zfs_scrub | bool
- name: Disable the conflicting weekly OpenZFS scrub timer
tags: [atlas, storage, scrub]
ansible.builtin.systemd:
name: "zfs-scrub-weekly@{{ atlas_zfs_pool }}.timer"
enabled: false
state: stopped
daemon_reload: true
when:
- atlas_manage_zfs_scrub | bool
- not ansible_check_mode
- name: Enable the Atlas monthly OpenZFS scrub timer
tags: [atlas, storage, scrub]
ansible.builtin.systemd:
name: "zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer"
enabled: true
state: started
daemon_reload: true
when:
- atlas_manage_zfs_scrub | bool
- not ansible_check_mode

View File

@@ -0,0 +1,41 @@
[Unit]
Description=Back up Atlas ZFS datasets to the encrypted Borg repository
Documentation=man:borg-create(1) man:borg-prune(1) man:borg-compact(1)
Requires=zfs.target
Wants=network-online.target
After=zfs.target network-online.target
StartLimitIntervalSec=6h
StartLimitBurst=3
ConditionFileIsExecutable=/usr/local/sbin/atlas-borg-backup
ConditionPathExists={{ atlas_borg_passphrase_path }}
ConditionPathExists={{ atlas_borg_ssh_private_key_path }}
ConditionPathExists={{ atlas_borg_known_hosts_path }}
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/atlas-borg-backup
ExecStopPost=+/usr/local/sbin/atlas-borg-snapshot-cleanup
User=root
Group=root
UMask=0077
SuccessExitStatus=1
Restart=on-failure
RestartSec=30m
TimeoutStartSec=infinity
RuntimeDirectory=atlas-borg
RuntimeDirectoryMode=0750
Nice=15
IOSchedulingClass=best-effort
IOSchedulingPriority=7
NoNewPrivileges=true
PrivateMounts=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths={{ atlas_borg_cache_dir }} {{ atlas_borg_config_dir }} /run/atlas-borg /run/lock
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictRealtime=true
LockPersonality=true

View File

@@ -0,0 +1,212 @@
#!/usr/bin/env bash
set -Eeuo pipefail
export LC_ALL=C.utf8
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
export BORG_CACHE_DIR={{ atlas_borg_cache_dir | quote }}
export BORG_CONFIG_DIR={{ atlas_borg_config_dir | quote }}
export BORG_PASSCOMMAND={{ ('cat ' ~ atlas_borg_passphrase_path) | quote }}
export BORG_RSH={{ atlas_borg_ssh_wrapper_path | quote }}
readonly pool={{ atlas_zfs_pool | quote }}
readonly mount_root={{ atlas_mount_root | quote }}
readonly repository={{ ('ssh://' ~ atlas_borg_repository_user ~ '@' ~ atlas_borg_repository_host
~ ':' ~ (atlas_borg_repository_port | string) ~ '/' ~ atlas_borg_repository_path) | quote }}
readonly remote_path={{ atlas_borg_remote_path | quote }}
readonly archive_prefix={{ atlas_borg_archive_prefix | quote }}
readonly snapshot_prefix={{ atlas_borg_snapshot_prefix | quote }}
readonly compression={{ atlas_borg_compression | quote }}
readonly stage=/run/atlas-borg/source
readonly snapshot_marker=/run/atlas-borg/snapshot-name
readonly borg_user={{ atlas_borg_username | quote }}
readonly borg_group={{ atlas_borg_group | quote }}
readonly borg_home={{ atlas_borg_home | quote }}
readonly borg_lock={{ atlas_borg_lock_path | quote }}
readonly progress_filter=/usr/local/libexec/atlas-borg-progress
snapshot_name=""
mounted_targets=()
# Invoked through the EXIT trap below.
# shellcheck disable=SC2329
cleanup() {
local status=$?
local cleanup_status=0
local index
local source_mount_failed=false
trap - EXIT HUP INT TERM
set +e
{% raw %}
for ((index = ${#mounted_targets[@]} - 1; index >= 0; index--)); do
{% endraw %}
if mountpoint -q "${mounted_targets[$index]}" && ! umount -R "${mounted_targets[$index]}"; then
printf 'Source snapshot mount cleanup failed: %s\n' "${mounted_targets[$index]}" >&2
source_mount_failed=true
fi
if mountpoint -q "${mounted_targets[$index]}"; then
printf 'Source snapshot mount is still active: %s\n' "${mounted_targets[$index]}" >&2
source_mount_failed=true
else
rmdir -- "${mounted_targets[$index]}" 2>/dev/null || true
fi
done
if [[ "$source_mount_failed" == false ]]; then
if [[ -d "$stage" ]]; then
rmdir -- "$stage" 2>/dev/null || cleanup_status=2
fi
else
cleanup_status=2
printf 'Source bind mount cleanup failed; keeping the snapshot for recovery\n' >&2
fi
if ((status == 0 && cleanup_status != 0)); then
status=$cleanup_status
fi
exit "$status"
}
trap cleanup EXIT
trap 'exit 143' HUP INT TERM
run_as_borg() {
setpriv \
--reuid "$borg_user" \
--regid "$borg_group" \
--clear-groups \
--inh-caps=-all,+dac_read_search \
--ambient-caps=-all,+dac_read_search \
--bounding-set=-all,+dac_read_search \
-- env HOME="$borg_home" USER="$borg_user" LOGNAME="$borg_user" "$@"
}
exec 8>"$borg_lock"
flock 8
exec 9>/run/lock/atlas-zfs-snapshot.lock
zpool list -H -o name "$pool" >/dev/null
rm -rf "$stage"
mkdir -p "$stage"
chown root:"$borg_group" /run/atlas-borg "$stage"
chmod 0750 /run/atlas-borg "$stage"
flock 9
while IFS= read -r stale_snapshot; do
stale_suffix="${stale_snapshot#"${pool}@${snapshot_prefix}-"}"
if [[ "$stale_suffix" =~ ^[0-9]{8}T[0-9]{6}Z$ ]]; then
zfs destroy -r "$stale_snapshot"
printf 'Removed stale Borg source snapshot %s\n' "$stale_snapshot"
fi
done < <(
zfs list -H -t snapshot -o name -r "$pool" |
grep -E "^${pool}@${snapshot_prefix}-[0-9]{8}T[0-9]{6}Z$" || true
)
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
readonly timestamp
snapshot_name="${snapshot_prefix}-${timestamp}"
readonly snapshot_name
printf '%s\n' "$snapshot_name" >"$snapshot_marker"
zfs snapshot -r "${pool}@${snapshot_name}"
flock -u 9
printf 'Created recursive Borg source snapshot %s@%s\n' "$pool" "$snapshot_name"
while IFS=$'\t' read -r dataset dataset_mountpoint mounted; do
if [[ "$mounted" != yes ]]; then
printf 'Dataset %s is not mounted; refusing an incomplete backup\n' "$dataset" >&2
exit 65
fi
if [[ "$dataset_mountpoint" != "$mount_root" && "$dataset_mountpoint" != "$mount_root/"* ]]; then
printf 'Dataset %s has unexpected mountpoint %s\n' "$dataset" "$dataset_mountpoint" >&2
exit 65
fi
dataset_suffix="${dataset#"$pool"}"
source_path="${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}"
target_path="${stage}${dataset_suffix}"
mkdir -p "$target_path"
mount --bind "$source_path" "$target_path"
mounted_targets+=("$target_path")
mount -o remount,bind,ro "$target_path"
done < <(zfs list -H -o name,mountpoint,mounted -s name -r "$pool")
estimated_source_bytes=0
while IFS=$'\t' read -r source_snapshot logical_bytes; do
if [[ "$source_snapshot" == *"@${snapshot_name}" ]]; then
[[ "$logical_bytes" =~ ^[0-9]+$ ]] || {
printf 'Invalid logical size for Borg source snapshot %s\n' "$source_snapshot" >&2
exit 74
}
estimated_source_bytes=$((estimated_source_bytes + logical_bytes))
fi
done < <(zfs list -H -p -t snapshot -o name,logicalreferenced -r "$pool")
((estimated_source_bytes > 0)) || {
printf 'Could not estimate the Borg source snapshot size\n' >&2
exit 74
}
printf 'Estimated Borg source logical size: %s bytes (ZFS; progress percentage is approximate)\n' \
"$estimated_source_bytes"
archive="${archive_prefix}-${timestamp}"
readonly archive
borg_status=0
printf 'Starting Borg archive %s from snapshot %s@%s\n' "$archive" "$pool" "$snapshot_name"
set +e
(
cd /run/atlas-borg
run_as_borg borg --remote-path "$remote_path" --lock-wait 600 --log-json --progress create \
--show-rc \
--stats \
--checkpoint-interval 900 \
--compression "$compression" \
"${repository}::${archive}" \
source 2>&1
) | /usr/bin/python3 -u "$progress_filter" --estimated-total-bytes "$estimated_source_bytes"
create_pipeline_status=("${PIPESTATUS[@]}")
set -e
create_status=${create_pipeline_status[0]}
if ((create_pipeline_status[1] != 0)); then
printf 'Borg progress logging failed with status %s\n' "${create_pipeline_status[1]}" >&2
exit 2
fi
if ((create_status >= 2)); then
exit "$create_status"
fi
borg_status=$create_status
printf 'Borg archive %s created; applying retention\n' "$archive"
set +e
run_as_borg borg --remote-path "$remote_path" --lock-wait 600 prune \
--show-rc \
--list \
--glob-archives "${archive_prefix}-*" \
--keep-daily {{ atlas_borg_keep_daily | int }} \
--keep-weekly {{ atlas_borg_keep_weekly | int }} \
--keep-monthly {{ atlas_borg_keep_monthly | int }} \
"$repository"
prune_status=$?
set -e
if ((prune_status >= 2)); then
exit "$prune_status"
fi
if ((prune_status > borg_status)); then
borg_status=$prune_status
fi
printf 'Borg retention complete; compacting repository\n'
set +e
run_as_borg borg --remote-path "$remote_path" --lock-wait 600 compact \
--show-rc \
"$repository"
compact_status=$?
set -e
if ((compact_status >= 2)); then
exit "$compact_status"
fi
if ((compact_status > borg_status)); then
borg_status=$compact_status
fi
printf 'Borg backup %s completed with status %s\n' "$archive" "$borg_status"
exit "$borg_status"

View File

@@ -0,0 +1,12 @@
[Unit]
Description=Schedule the encrypted Atlas Borg backup
[Timer]
OnCalendar={{ atlas_borg_backup_calendar }}
Persistent=true
RandomizedDelaySec={{ atlas_borg_randomized_delay }}
AccuracySec=1min
Unit=atlas-borg-backup.service
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,32 @@
[Unit]
Description=Check the encrypted Atlas Borg repository
Documentation=man:borg-check(1)
Wants=network-online.target
After=network-online.target atlas-borg-backup.service
ConditionFileIsExecutable=/usr/local/sbin/atlas-borg-check
ConditionPathExists={{ atlas_borg_passphrase_path }}
ConditionPathExists={{ atlas_borg_ssh_private_key_path }}
ConditionPathExists={{ atlas_borg_known_hosts_path }}
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/atlas-borg-check
User={{ atlas_borg_username }}
Group={{ atlas_borg_group }}
UMask=0077
SuccessExitStatus=1
TimeoutStartSec=infinity
Nice=15
IOSchedulingClass=best-effort
IOSchedulingPriority=7
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths={{ atlas_borg_cache_dir }} {{ atlas_borg_config_dir }}
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictRealtime=true
LockPersonality=true

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -Eeuo pipefail
export LC_ALL=C
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
export BORG_CACHE_DIR={{ atlas_borg_cache_dir | quote }}
export BORG_CONFIG_DIR={{ atlas_borg_config_dir | quote }}
export BORG_PASSCOMMAND={{ ('cat ' ~ atlas_borg_passphrase_path) | quote }}
export BORG_RSH={{ atlas_borg_ssh_wrapper_path | quote }}
readonly repository={{ ('ssh://' ~ atlas_borg_repository_user ~ '@' ~ atlas_borg_repository_host
~ ':' ~ (atlas_borg_repository_port | string) ~ '/' ~ atlas_borg_repository_path) | quote }}
readonly remote_path={{ atlas_borg_remote_path | quote }}
readonly archive_prefix={{ atlas_borg_archive_prefix | quote }}
readonly borg_lock={{ atlas_borg_lock_path | quote }}
exec 8>"$borg_lock"
flock 8
exec borg --remote-path "$remote_path" --lock-wait 600 check \
--show-rc \
--glob-archives "${archive_prefix}-*" \
"$repository"

View File

@@ -0,0 +1,12 @@
[Unit]
Description=Schedule checks of the encrypted Atlas Borg repository
[Timer]
OnCalendar={{ atlas_borg_check_calendar }}
Persistent=true
RandomizedDelaySec={{ atlas_borg_randomized_delay }}
AccuracySec=1min
Unit=atlas-borg-check.service
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,51 @@
#!/usr/bin/env bash
set -Eeuo pipefail
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
readonly pool={{ atlas_zfs_pool | quote }}
readonly mount_root={{ atlas_mount_root | quote }}
readonly snapshot_prefix={{ atlas_borg_snapshot_prefix | quote }}
readonly marker=/run/atlas-borg/snapshot-name
[[ -e "$marker" ]] || exit 0
[[ -f "$marker" && ! -L "$marker" ]] || {
printf 'Unsafe Atlas Borg snapshot marker; leaving snapshots unchanged\n' >&2
exit 2
}
IFS= read -r snapshot_name <"$marker"
[[ "$snapshot_name" =~ ^${snapshot_prefix}-[0-9]{8}T[0-9]{6}Z$ ]] || {
printf 'Invalid Atlas Borg snapshot marker; leaving snapshots unchanged\n' >&2
exit 2
}
exec 9>/run/lock/atlas-zfs-snapshot.lock
flock 9
if zfs list -H -t snapshot -o name "${pool}@${snapshot_name}" >/dev/null 2>&1; then
# The private bind mounts are gone, but ZFS may leave its on-demand
# .zfs/snapshot mounts in the host namespace until explicitly unmounted.
snapshot_mounts=()
snapshot_sources=()
while IFS=$'\t' read -r dataset dataset_mountpoint; do
[[ "$dataset_mountpoint" == "$mount_root" || "$dataset_mountpoint" == "$mount_root/"* ]] || continue
snapshot_mounts+=("${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}")
snapshot_sources+=("${dataset}@${snapshot_name}")
done < <(zfs list -H -o name,mountpoint -s name -r "$pool")
{% raw %}
for ((index = ${#snapshot_mounts[@]} - 1; index >= 0; index--)); do
{% endraw %}
mounted_source="$(findmnt -rn -M "${snapshot_mounts[$index]}" -o SOURCE || true)"
[[ -n "$mounted_source" ]] || continue
[[ "$mounted_source" == "${snapshot_sources[$index]}" ]] || {
printf 'Unexpected source on Atlas Borg snapshot mount: %s\n' \
"${snapshot_mounts[$index]}" >&2
exit 2
}
umount "${snapshot_mounts[$index]}"
done
zfs destroy -r "${pool}@${snapshot_name}"
printf 'Removed recursive Atlas Borg source snapshot %s@%s after backup exit\n' \
"$pool" "$snapshot_name"
fi

View File

@@ -0,0 +1,19 @@
#!/usr/bin/env bash
set -euo pipefail
# Borg receives CAP_DAC_READ_SEARCH only for local snapshot traversal. Drop it
# before starting the network transport so SSH runs as the plain service user.
exec setpriv \
--inh-caps=-all \
--ambient-caps=-all \
-- /usr/bin/ssh \
-i {{ atlas_borg_ssh_private_key_path | quote }} \
-p {{ atlas_borg_repository_port | int }} \
-o BatchMode=yes \
-o IdentitiesOnly=yes \
-o StrictHostKeyChecking=yes \
-o UserKnownHostsFile={{ atlas_borg_known_hosts_path | quote }} \
-o ConnectTimeout=30 \
-o ServerAliveInterval=60 \
-o ServerAliveCountMax=3 \
"$@"

View File

@@ -0,0 +1,20 @@
{
"pool": {{ atlas_zfs_pool | to_json }},
"backup_dataset": {{ (atlas_zfs_pool ~ '/' ~ atlas_zfs_dataset_backup) | to_json }},
"notifier": {{ atlas_monitor_notifier | to_json }},
"smart_devices": {{ atlas_monitor_smart_devices | to_json }},
"timers": {{ atlas_monitor_timers | to_json }},
"failure_units": {{ atlas_monitor_failure_units | to_json }},
"remote_capacity": {{ atlas_monitor_remote_capacity | to_json }},
"pool_warning_percent": {{ atlas_monitor_pool_warning_percent | int }},
"pool_critical_percent": {{ atlas_monitor_pool_critical_percent | int }},
"root_warning_percent": {{ atlas_monitor_root_warning_percent | int }},
"root_critical_percent": {{ atlas_monitor_root_critical_percent | int }},
"snapshot_warning_percent": {{ atlas_monitor_snapshot_warning_percent | int }},
"snapshot_critical_percent": {{ atlas_monitor_snapshot_critical_percent | int }},
"snapshot_growth_warning_gib_day": {{ atlas_monitor_snapshot_growth_warning_gib_day | int }},
"backup_growth_warning_gib_day": {{ atlas_monitor_backup_growth_warning_gib_day | int }},
"cpu_warning_c": {{ atlas_monitor_cpu_warning_c | int }},
"cpu_critical_c": {{ atlas_monitor_cpu_critical_c | int }},
"borg_max_runtime_days": {{ atlas_monitor_borg_max_runtime_days | int }}
}

View File

@@ -0,0 +1,25 @@
[Unit]
Description=Check Atlas pool, disks, capacity, temperatures and maintenance jobs
Wants=houston-dbus.service network-online.target
After=zfs.target houston-dbus.service network-online.target
ConditionFileIsExecutable=/usr/local/libexec/atlas-health-monitor
[Service]
Type=oneshot
ExecStart=/usr/local/libexec/atlas-health-monitor
User=root
Group=root
UMask=0077
StateDirectory=atlas-health-monitor
StateDirectoryMode=0700
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/var/lib/atlas-health-monitor
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictRealtime=true
LockPersonality=true

View File

@@ -0,0 +1,11 @@
[Unit]
Description=Schedule Atlas health checks
[Timer]
OnCalendar={{ atlas_monitor_calendar }}
Persistent=true
RandomizedDelaySec=5min
Unit=atlas-health-monitor.service
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,2 @@
[Unit]
OnFailure=atlas-monitor-failure@%n.service

View File

@@ -0,0 +1,19 @@
[Unit]
Description=Submit a 45Drives Alert for failed Atlas job %I
Requires=houston-dbus.service
After=houston-dbus.service
ConditionFileIsExecutable=/usr/local/libexec/atlas-health-monitor
[Service]
Type=oneshot
ExecStart=/usr/local/libexec/atlas-health-monitor --job-failed %I
User=root
Group=root
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
RestrictAddressFamilies=AF_UNIX
RestrictRealtime=true
LockPersonality=true

View File

@@ -0,0 +1,30 @@
[Unit]
Description=Run a manual, UUID-bound offline USB backup of Atlas ZFS datasets
Requires=zfs.target
After=zfs.target
ConditionFileIsExecutable=/usr/local/sbin/atlas-usb-backup
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/atlas-usb-backup
ExecStopPost=+/usr/local/sbin/atlas-usb-snapshot-cleanup
User=root
Group=root
UMask=0077
TimeoutStartSec=infinity
RuntimeDirectory=atlas-usb-backup
RuntimeDirectoryMode=0700
Nice=15
IOSchedulingClass=best-effort
IOSchedulingPriority=7
PrivateMounts=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/run/atlas-usb-backup /run/lock
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX
RestrictRealtime=true
LockPersonality=true

View File

@@ -0,0 +1,242 @@
#!/usr/bin/env bash
set -Eeuo pipefail
export LC_ALL=C.utf8
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
readonly pool={{ atlas_zfs_pool | quote }}
readonly mount_root={{ atlas_mount_root | quote }}
readonly luks_uuid={{ atlas_usb_backup_luks_uuid | quote }}
readonly fs_uuid={{ atlas_usb_backup_fs_uuid | quote }}
readonly mapper_name={{ atlas_usb_backup_mapper_name | quote }}
readonly snapshot_prefix={{ atlas_usb_backup_snapshot_prefix | quote }}
readonly min_free_bytes={{ atlas_usb_backup_min_free_bytes | int }}
readonly mapper="/dev/mapper/${mapper_name}"
readonly outer="/dev/disk/by-uuid/${luks_uuid}"
readonly runtime_dir=/run/atlas-usb-backup
readonly snapshot_marker="${runtime_dir}/snapshot-name"
readonly source_dir="${runtime_dir}/source"
readonly usb_mount="${runtime_dir}/target"
readonly backup_root="${usb_mount}/atlas"
snapshot_name=""
mapper_opened_by_script=false
usb_mounted=false
published=false
partial=""
mounted_targets=()
# shellcheck disable=SC2329
cleanup() {
local status=$?
local cleanup_status=0
local index
local source_mount_failed=false
trap - EXIT HUP INT TERM
set +e
if [[ -n "$partial" && "$published" == false && "$usb_mounted" == true ]]; then
rm -rf -- "$partial" || cleanup_status=2
fi
if [[ "$usb_mounted" == true ]]; then
umount "$usb_mount" || cleanup_status=2
fi
{% raw %}
for ((index = ${#mounted_targets[@]} - 1; index >= 0; index--)); do
{% endraw %}
if mountpoint -q "${mounted_targets[$index]}" && ! umount -R "${mounted_targets[$index]}"; then
printf 'Source snapshot mount cleanup failed: %s\n' "${mounted_targets[$index]}" >&2
source_mount_failed=true
fi
if mountpoint -q "${mounted_targets[$index]}"; then
printf 'Source snapshot mount is still active: %s\n' "${mounted_targets[$index]}" >&2
source_mount_failed=true
else
rmdir -- "${mounted_targets[$index]}" 2>/dev/null || true
fi
done
if [[ "$source_mount_failed" == false ]]; then
rmdir -- "$source_dir" 2>/dev/null || true
else
cleanup_status=2
printf 'Source bind mount cleanup failed; keeping the snapshot for recovery\n' >&2
fi
if [[ "$usb_mounted" == true || "$mapper_opened_by_script" == true ]] &&
! mountpoint -q "$usb_mount" &&
! findmnt -rn -S "$mapper" >/dev/null; then
cryptsetup close "$mapper_name" || cleanup_status=2
fi
rmdir -- "$usb_mount" 2>/dev/null || true
if ((status == 0 && cleanup_status != 0)); then
status=$cleanup_status
fi
exit "$status"
}
trap cleanup EXIT
trap 'exit 143' HUP INT TERM
exec 8>/run/lock/atlas-usb-backup.lock
flock -n 8 || { printf 'Atlas USB backup is already running\n' >&2; exit 75; }
exec 9>/run/lock/atlas-zfs-snapshot.lock
zpool list -H -o name "$pool" >/dev/null
[[ -b "$outer" ]] || { printf 'Configured LUKS UUID is not connected\n' >&2; exit 66; }
[[ "$(blkid -s TYPE -o value "$outer")" == crypto_LUKS ]] || {
printf 'Configured outer UUID is not a LUKS container\n' >&2
exit 65
}
[[ "$(blkid -s UUID -o value "$outer")" == "$luks_uuid" ]] || exit 65
if ! cryptsetup status "$mapper_name" >/dev/null; then
printf 'Requesting the LUKS passphrase for the configured USB disk\n'
systemd-ask-password -n --no-tty --timeout=300 \
--id="atlas-usb-backup:${luks_uuid}" \
'Atlas offline USB backup LUKS passphrase:' |
cryptsetup open --type luks2 --key-file - "$outer" "$mapper_name"
mapper_opened_by_script=true
fi
backing_device="$(cryptsetup status "$mapper_name" | awk '$1 == "device:" { print $2 }')"
[[ -n "$backing_device" && "$(readlink -f "$backing_device")" == "$(readlink -f "$outer")" ]] || {
printf 'The unlocked mapper does not belong to the configured LUKS UUID\n' >&2
exit 65
}
[[ "$(blkid -s TYPE -o value "$mapper")" == ext4 ]] || {
printf 'The unlocked USB filesystem is not ext4\n' >&2
exit 65
}
[[ "$(blkid -s UUID -o value "$mapper")" == "$fs_uuid" ]] || {
printf 'The unlocked USB filesystem UUID does not match\n' >&2
exit 65
}
if findmnt -rn -S "$mapper" >/dev/null; then
printf 'The USB filesystem is already mounted elsewhere\n' >&2
exit 65
fi
[[ ! -e "$source_dir" && ! -e "$usb_mount" ]] || {
printf 'USB backup staging directories already exist; inspect them manually\n' >&2
exit 65
}
mkdir -m 0700 "$usb_mount"
mount -t ext4 -o nodev,nosuid,noexec "$mapper" "$usb_mount"
usb_mounted=true
[[ "$(readlink -f "$(findmnt -nro SOURCE --target "$usb_mount")")" == "$(readlink -f "$mapper")" ]] || {
printf 'Mounted USB source does not match the verified mapper\n' >&2
exit 65
}
for path in "$backup_root" "$backup_root/snapshots"; do
[[ ! -L "$path" ]] || { printf 'Unsafe symlink in USB backup destination\n' >&2; exit 65; }
mkdir -p -- "$path"
[[ -d "$path" ]] || exit 65
chown root:root -- "$path"
chmod 0700 -- "$path"
done
free_bytes="$(df -B1 --output=avail "$usb_mount" | tail -n 1 | tr -d ' ')"
if ((free_bytes < min_free_bytes)); then
printf 'USB free space (%s bytes) is below the required reserve (%s bytes)\n' \
"$free_bytes" "$min_free_bytes" >&2
exit 73
fi
mkdir -m 0700 "$source_dir"
flock 9
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
snapshot_name="${snapshot_prefix}-${timestamp}-$$"
printf '%s\n' "$snapshot_name" >"$snapshot_marker"
zfs snapshot -r "${pool}@${snapshot_name}"
flock -u 9
printf 'Created recursive USB source snapshot %s@%s\n' "$pool" "$snapshot_name"
while IFS=$'\t' read -r dataset dataset_mountpoint mounted; do
if [[ "$mounted" != yes ]]; then
printf 'Dataset %s is not mounted; refusing an incomplete backup\n' "$dataset" >&2
exit 65
fi
if [[ "$dataset_mountpoint" != "$mount_root" && "$dataset_mountpoint" != "$mount_root/"* ]]; then
printf 'Dataset %s has unexpected mountpoint %s\n' "$dataset" "$dataset_mountpoint" >&2
exit 65
fi
dataset_suffix="${dataset#"$pool"}"
source_path="${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}"
target_path="${source_dir}${dataset_suffix}"
mkdir -p "$target_path"
mount --bind "$source_path" "$target_path"
mounted_targets+=("$target_path")
mount -o remount,bind,ro "$target_path"
done < <(zfs list -H -o name,mountpoint,mounted -s name -r "$pool")
previous=""
if [[ -e "$backup_root/latest" || -L "$backup_root/latest" ]]; then
[[ -L "$backup_root/latest" ]] || { printf 'latest is not a symlink\n' >&2; exit 65; }
previous="$(readlink -e "$backup_root/latest")"
[[ -n "$previous" && "$previous" == "$backup_root/snapshots/"* && -d "$previous" ]] || {
printf 'latest does not point to a complete snapshot on the USB disk\n' >&2
exit 65
}
fi
backup_name="${timestamp}-$$"
candidate_partial="${backup_root}/snapshots/.incomplete-${backup_name}"
complete="${backup_root}/snapshots/${backup_name}"
[[ ! -e "$candidate_partial" && ! -L "$candidate_partial" && ! -e "$complete" && ! -L "$complete" ]] || exit 65
mkdir -m 0700 "$candidate_partial"
partial="$candidate_partial"
printf 'Copying the consistent pool tree to USB backup %s\n' "$backup_name"
# Preserve POSIX ACLs, ownership, modes, timestamps, hard links, and sparse
# files. Do not preserve generic xattrs: Rocky 9's rsync 3.2.7 fails when
# combining xattrs with --link-dest, while SELinux labels were intentionally
# excluded because restores must relabel for their destination host.
rsync_args=(-aHAS --numeric-ids "--info=progress2,stats2")
estimate_args=(-aHAS --numeric-ids --dry-run --stats)
if [[ -n "$previous" ]]; then
rsync_args+=("--link-dest=$previous")
estimate_args+=("--link-dest=$previous")
fi
# The rsync dry run estimates changed file bytes after link-dest deduplication.
# Metadata and filesystem allocation still require the separate free-space reserve.
estimate="$(rsync "${estimate_args[@]}" "${source_dir}/" "${partial}/")"
transfer_bytes="$(printf '%s\n' "$estimate" | awk -F: \
'/^Total transferred file size:/ { gsub(/[^0-9]/, "", $2); print $2 }')"
[[ "$transfer_bytes" =~ ^[0-9]+$ ]] || {
printf 'Could not determine the USB transfer size\n' >&2
exit 74
}
if ((free_bytes - transfer_bytes < min_free_bytes)); then
printf 'Insufficient USB space: %s bytes free, %s estimated transfer, %s reserved\n' \
"$free_bytes" "$transfer_bytes" "$min_free_bytes" >&2
exit 73
fi
rsync "${rsync_args[@]}" "${source_dir}/" "${partial}/"
printf 'Verifying USB backup %s with a checksum-based dry run\n' "$backup_name"
verification="${runtime_dir}/verification.out"
rsync -aHAS --numeric-ids \
--checksum --dry-run --delete --itemize-changes \
"${source_dir}/" "${partial}/" >"$verification"
if [[ -s "$verification" ]]; then
printf 'USB verification found mismatches; refusing to publish the backup\n' >&2
exit 74
fi
free_bytes="$(df -B1 --output=avail "$usb_mount" | tail -n 1 | tr -d ' ')"
if ((free_bytes < min_free_bytes)); then
printf 'USB backup completed below the free-space reserve; refusing to publish it\n' >&2
exit 73
fi
mv -- "$partial" "$complete"
partial=""
ln -s "snapshots/${backup_name}" "${backup_root}/.latest-${backup_name}"
mv -Tf -- "${backup_root}/.latest-${backup_name}" "${backup_root}/latest"
published=true
sync -f "$complete"
sync -f "$backup_root"
printf 'USB backup %s verified and published; unmounting and closing LUKS\n' "$backup_name"

View File

@@ -0,0 +1,31 @@
#!/usr/bin/python3
"""Submit a manual-backup reminder through Atlas' existing Houston notifier."""
import json
import subprocess
from datetime import datetime, timezone
now = datetime.now(timezone.utc)
message = {
"timestamp": now.isoformat(timespec="seconds"),
"unixtime": int(now.timestamp()),
"event": "atlas_usb_backup_reminder",
"severity": "warning",
"subject": "Promemoria backup USB offline Atlas",
"email_message": (
"Collega il disco USB di backup ad Atlas ed esegui manualmente il backup offline.\n"
"Il promemoria non avvia il backup. Controlla che il disco non sia\n"
"montato; poi esegui:\n\n"
" sudo systemctl start atlas-usb-backup.service\n\n"
"Verifica l'esito con:\n"
" sudo journalctl -u atlas-usb-backup.service -n 100 --no-pager\n\n"
"Dopo la riuscita, scollega fisicamente il disco."
),
}
subprocess.run(
[{{ atlas_usb_reminder_notifier | to_json }}, json.dumps(message)],
check=True,
)
print("Atlas USB backup reminder submitted to 45Drives Alerts; email delivery is not verified.", flush=True)

View File

@@ -0,0 +1,22 @@
[Unit]
Description=45Drives Alerts reminder to run the manual Atlas offline USB backup
Requires=houston-dbus.service
After=houston-dbus.service
ConditionFileIsExecutable=/usr/local/libexec/atlas-usb-reminder
[Service]
Type=oneshot
ExecStart=/usr/local/libexec/atlas-usb-reminder
User=root
Group=root
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_UNIX
RestrictRealtime=true
LockPersonality=true

View File

@@ -0,0 +1,10 @@
[Unit]
Description=Remind the administrator to run the manual Atlas offline USB backup
[Timer]
OnCalendar={{ atlas_usb_reminder_calendar }}
Persistent=true
Unit=atlas-usb-reminder.service
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,51 @@
#!/usr/bin/env bash
set -Eeuo pipefail
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
readonly pool={{ atlas_zfs_pool | quote }}
readonly mount_root={{ atlas_mount_root | quote }}
readonly snapshot_prefix={{ atlas_usb_backup_snapshot_prefix | quote }}
readonly marker=/run/atlas-usb-backup/snapshot-name
[[ -e "$marker" ]] || exit 0
[[ -f "$marker" && ! -L "$marker" ]] || {
printf 'Unsafe Atlas USB snapshot marker; leaving snapshots unchanged\n' >&2
exit 2
}
IFS= read -r snapshot_name <"$marker"
[[ "$snapshot_name" =~ ^${snapshot_prefix}-[0-9]{8}T[0-9]{6}Z-[0-9]+$ ]] || {
printf 'Invalid Atlas USB snapshot marker; leaving snapshots unchanged\n' >&2
exit 2
}
exec 9>/run/lock/atlas-zfs-snapshot.lock
flock 9
if zfs list -H -t snapshot -o name "${pool}@${snapshot_name}" >/dev/null 2>&1; then
# ZFS can leave its on-demand .zfs/snapshot mounts in the host namespace
# even after the backup's private bind mounts and process have exited.
snapshot_mounts=()
snapshot_sources=()
while IFS=$'\t' read -r dataset dataset_mountpoint; do
[[ "$dataset_mountpoint" == "$mount_root" || "$dataset_mountpoint" == "$mount_root/"* ]] || continue
snapshot_mounts+=("${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}")
snapshot_sources+=("${dataset}@${snapshot_name}")
done < <(zfs list -H -o name,mountpoint -s name -r "$pool")
{% raw %}
for ((index = ${#snapshot_mounts[@]} - 1; index >= 0; index--)); do
{% endraw %}
mounted_source="$(findmnt -rn -M "${snapshot_mounts[$index]}" -o SOURCE || true)"
[[ -n "$mounted_source" ]] || continue
[[ "$mounted_source" == "${snapshot_sources[$index]}" ]] || {
printf 'Unexpected source on Atlas USB snapshot mount: %s\n' \
"${snapshot_mounts[$index]}" >&2
exit 2
}
umount "${snapshot_mounts[$index]}"
done
zfs destroy -r "${pool}@${snapshot_name}"
printf 'Removed recursive Atlas USB source snapshot %s@%s after backup exit\n' \
"$pool" "$snapshot_name"
fi

View File

@@ -0,0 +1,6 @@
[Timer]
OnCalendar=
OnCalendar={{ atlas_zfs_scrub_calendar }}
Persistent=true
RandomizedDelaySec=0
AccuracySec=1min

View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
set -euo pipefail
export LC_ALL=C
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
if [[ $# -ne 1 ]]; then
printf 'Usage: %s <policy>\n' "$0" >&2
exit 64
fi
readonly pool={{ atlas_zfs_pool | quote }}
readonly prefix={{ atlas_zfs_snapshot_prefix | quote }}
readonly period="$1"
case "$period" in
{% for policy in atlas_zfs_snapshot_policies %}
{{ policy.name | quote }})
keep={{ policy.keep | int }}
;;
{% endfor %}
*)
printf 'Unknown Atlas ZFS snapshot policy: %s\n' "$period" >&2
exit 64
;;
esac
readonly keep
zpool list -H -o name "$pool" >/dev/null
exec 9>/run/lock/atlas-zfs-snapshot.lock
flock 9
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
readonly timestamp
readonly snapshot_prefix="${pool}@${prefix}-${period}-"
readonly snapshot="${snapshot_prefix}${timestamp}"
zfs snapshot -r "$snapshot"
printf 'Created recursive ZFS snapshot %s\n' "$snapshot"
snapshot_listing="$(zfs list -H -t snapshot -o name -s creation -r "$pool")"
managed_snapshots=()
while IFS= read -r snapshot_name; do
if [[ "$snapshot_name" == "$snapshot_prefix"* ]]; then
snapshot_suffix="${snapshot_name#"$snapshot_prefix"}"
if [[ "$snapshot_suffix" =~ ^[0-9]{8}T[0-9]{6}Z$ ]]; then
managed_snapshots+=("$snapshot_name")
fi
fi
done <<< "$snapshot_listing"
{% raw %}
managed_snapshot_count="${#managed_snapshots[@]}"
{% endraw %}
prune_count=$((managed_snapshot_count - keep))
if ((prune_count <= 0)); then
printf 'Retaining %d of %d managed %s snapshots\n' \
"$managed_snapshot_count" "$keep" "$period"
exit 0
fi
for ((index = 0; index < prune_count; index++)); do
candidate="${managed_snapshots[$index]}"
if [[ "$candidate" != "$snapshot_prefix"* ]]; then
printf 'Refusing to destroy unexpected snapshot: %s\n' "$candidate" >&2
exit 65
fi
zfs destroy -r "$candidate"
printf 'Pruned recursive ZFS snapshot %s\n' "$candidate"
done

View File

@@ -0,0 +1,11 @@
[Unit]
Description=Schedule {{ item.name }} ZFS snapshots for {{ atlas_zfs_pool }}
[Timer]
OnCalendar={{ item.calendar }}
Persistent=true
AccuracySec=1min
Unit=atlas-zfs-snapshot@{{ item.name }}.service
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,25 @@
[Unit]
Description=Create and retain %i ZFS snapshots for {{ atlas_zfs_pool }}
Documentation=man:zfs-snapshot(8) man:zfs-destroy(8)
Requires=zfs.target
After=zfs.target
ConditionFileIsExecutable=/usr/local/sbin/atlas-zfs-snapshot
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/atlas-zfs-snapshot %i
User=root
Group=root
UMask=0077
Nice=10
IOSchedulingClass=best-effort
IOSchedulingPriority=7
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictRealtime=true
LockPersonality=true

View File

@@ -5,14 +5,18 @@ backend_phase1_username: "{{ atlas_admin_username }}"
backend_phase1_user_group: "{{ atlas_admin_group }}"
backend_phase1_user_home: "{{ atlas_admin_home }}"
backend_phase1_quadlet_dir: "{{ backend_phase1_user_home }}/.config/containers/systemd"
backend_phase1_wireguard_interface: wg0
backend_phase1_wireguard_address: CHANGEME_ATLAS_WIREGUARD_ADDRESS
backend_phase1_wireguard_firewalld_zone: wireguard
backend_phase1_bind_address: CHANGEME_ATLAS_BIND_ADDRESS
backend_phase1_firewalld_zone: public
backend_phase1_npm_source_ip: CHANGEME_AEGIS_IP
backend_phase1_syncthing_native_subnet: CHANGEME_LAN_SUBNET
backend_phase1_music_dir: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_music }}"
backend_phase1_archive_dir: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_archive }}"
backend_phase1_syncthing_folder_path: "{{ backend_phase1_archive_dir }}/Org"
backend_phase1_app_data_root: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_app_data }}"
backend_phase1_navidrome_data_dir: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_navidrome }}"
backend_phase1_syncthing_root: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_syncthing }}"
backend_phase1_music_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_music }}"
backend_phase1_archive_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_archive }}"
backend_phase1_app_data_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_app_data }}"
backend_phase1_navidrome_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_navidrome }}"
backend_phase1_syncthing_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_syncthing }}"

View File

@@ -7,17 +7,22 @@
ansible.builtin.assert:
that:
- not (atlas_manage_media_stack | bool)
- backend_phase1_wireguard_interface in ansible_facts.interfaces
- backend_phase1_wireguard_address != 'CHANGEME_ATLAS_WIREGUARD_ADDRESS'
- backend_phase1_bind_address != 'CHANGEME_ATLAS_BIND_ADDRESS'
- backend_phase1_bind_address is match('^[0-9]{1,3}(\.[0-9]{1,3}){3}$')
- backend_phase1_firewalld_zone | length > 0
- backend_phase1_npm_source_ip != 'CHANGEME_AEGIS_IP'
- backend_phase1_syncthing_native_subnet != 'CHANGEME_LAN_SUBNET'
- backend_phase1_music_dir.startswith('/')
- backend_phase1_archive_dir.startswith('/')
- backend_phase1_syncthing_folder_path.startswith(backend_phase1_archive_dir + '/')
- backend_phase1_app_data_root.startswith('/')
- backend_phase1_navidrome_data_dir.startswith(backend_phase1_app_data_root + '/')
- backend_phase1_syncthing_root.startswith(backend_phase1_app_data_root + '/')
fail_msg: >-
Disable the rootful media-stack gate and provide the active
WireGuard interface/address and absolute ZFS-backed paths before
Disable the rootful media-stack gate and provide the Atlas LAN bind
address, firewall sources, and absolute ZFS-backed paths before
enabling phase one. This role does not manage Prometheus or migrate
Navidrome application data.
application data.
- name: Read the rootless service account
ansible.builtin.getent:
@@ -49,6 +54,8 @@
loop:
- dataset: "{{ backend_phase1_music_dataset }}"
mountpoint: "{{ backend_phase1_music_dir }}"
- dataset: "{{ backend_phase1_archive_dataset }}"
mountpoint: "{{ backend_phase1_archive_dir }}"
- dataset: "{{ backend_phase1_app_data_dataset }}"
mountpoint: "{{ backend_phase1_app_data_root }}"
- dataset: "{{ backend_phase1_navidrome_dataset }}"
@@ -101,6 +108,8 @@
mode: "0750"
- path: "{{ backend_phase1_syncthing_root }}"
mode: "0750"
- path: "{{ backend_phase1_syncthing_folder_path }}"
mode: "0770"
loop_control:
label: "{{ item.path }}"
@@ -132,19 +141,35 @@
DBUS_SESSION_BUS_ADDRESS: "unix:path=/run/user/{{ backend_phase1_uid }}/bus"
when: not ansible_check_mode
- name: Permit phase-one services only through the WireGuard zone
- name: Permit NPM access to phase-one web interfaces through Aegis
ansible.posix.firewalld:
port: "{{ item }}"
zone: "{{ backend_phase1_wireguard_firewalld_zone }}"
rich_rule: >-
rule family="ipv4" source address="{{ backend_phase1_npm_source_ip }}"
port port="{{ item }}" protocol="tcp" accept
zone: "{{ backend_phase1_firewalld_zone }}"
state: enabled
permanent: true
immediate: true
loop:
- "{{ backend_phase1_navidrome_port }}/tcp"
- "{{ backend_phase1_syncthing_gui_port }}/tcp"
- "{{ backend_phase1_syncthing_transfer_port }}/tcp"
- "{{ backend_phase1_syncthing_transfer_port }}/udp"
- "{{ backend_phase1_syncthing_discovery_port }}/udp"
- "{{ backend_phase1_navidrome_port }}"
- "{{ backend_phase1_syncthing_gui_port }}"
- name: Permit native Syncthing traffic from the LAN
ansible.posix.firewalld:
rich_rule: >-
rule family="ipv4" source address="{{ backend_phase1_syncthing_native_subnet }}"
port port="{{ item.port }}" protocol="{{ item.protocol }}" accept
zone: "{{ backend_phase1_firewalld_zone }}"
state: enabled
permanent: true
immediate: true
loop:
- port: "{{ backend_phase1_syncthing_transfer_port }}"
protocol: tcp
- port: "{{ backend_phase1_syncthing_transfer_port }}"
protocol: udp
- port: "{{ backend_phase1_syncthing_discovery_port }}"
protocol: udp
- name: Start rootless phase-one Quadlets
become_user: "{{ backend_phase1_username }}"

View File

@@ -8,7 +8,7 @@ Image={{ backend_phase1_navidrome_image }}
UserNS=keep-id
User={{ backend_phase1_uid }}
Group={{ backend_phase1_gid }}
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_navidrome_port }}:4533
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_navidrome_port }}:4533
Environment=ND_LOGLEVEL=info
Environment=ND_SCANSCHEDULE=1h
Environment=ND_SESSIONTIMEOUT=24h

View File

@@ -9,15 +9,17 @@ HostName=atlas-syncthing
UserNS=keep-id
User={{ backend_phase1_uid }}
Group={{ backend_phase1_gid }}
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_gui_port }}:{{ backend_phase1_syncthing_gui_port }}
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}/udp
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_discovery_port }}:{{ backend_phase1_syncthing_discovery_port }}/udp
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_gui_port }}:{{ backend_phase1_syncthing_gui_port }}
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}/udp
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_discovery_port }}:{{ backend_phase1_syncthing_discovery_port }}/udp
Environment=HOME=/var/syncthing
Environment=STHOMEDIR=/var/syncthing/config
Environment=STGUIADDRESS=0.0.0.0:{{ backend_phase1_syncthing_gui_port }}
Environment=TZ={{ backend_phase1_timezone }}
Volume={{ backend_phase1_syncthing_root }}:/var/syncthing:Z
# Archive is intentionally shared with Samba; use a shared SELinux container label.
Volume={{ backend_phase1_archive_dir }}:/data:z
NoNewPrivileges=true
DropCapability=all

View File

@@ -20,15 +20,7 @@
owner: "{{ item.owner }}"
group: "{{ item.group }}"
mode: "{{ item.mode }}"
loop: >-
{{
(server_directories | default([]))
+ (
server_syncthing_directories | default([])
if server_syncthing_enabled | default(true) | bool
else []
)
}}
loop: "{{ server_directories | default([]) }}"
loop_control:
label: "{{ item.path }}"

View File

@@ -66,6 +66,17 @@
changed_when: false
failed_when: false
- name: Check whether the pinned Java 25 version is installed with Mise
tags: [packages, mise, java, wsl]
ansible.builtin.command:
cmd: "mise where java@{{ workstation_mise_java_25_version }}"
become_user: "{{ username }}"
environment:
HOME: "{{ user_home }}"
register: workstation_mise_java_25_where
changed_when: false
failed_when: false
- name: Check whether the pinned Maven version is installed with Mise
tags: [packages, mise, maven, wsl]
ansible.builtin.command:
@@ -86,6 +97,7 @@
HOME: "{{ user_home }}"
when: >-
workstation_mise_java_where.rc != 0 or
workstation_mise_java_25_where.rc != 0 or
workstation_mise_maven_where.rc != 0
- name: Ensure WSL boot configuration file exists

View File

@@ -10,6 +10,12 @@ wireguard_mtu: 1420
wireguard_firewalld_zone: wireguard
wireguard_public_firewalld_zone: public
wireguard_enable_ipv4_forwarding: false
# Legacy zone-level masquerading; policy-level masquerading is required for inter-zone forwarding.
wireguard_enable_masquerade: false
# A list of narrowly scoped inter-zone forwarding rules, rendered as persistent
# firewalld policies. Each item requires name, ingress_zone, egress_zone,
# source, and destination.
wireguard_forwarding_policies: []
wireguard_reload_rootful_podman_networks: false
wireguard_handshake_retries: 12
wireguard_handshake_delay: 5

View File

@@ -14,10 +14,44 @@
Configure this host's WireGuard address and peers, and run the first
key bootstrap against every peer in the same play.
- name: Install WireGuard userspace tools
- name: Validate WireGuard forwarding policies
ansible.builtin.assert:
that:
- item.name is defined
- item.ingress_zone is defined
- item.egress_zone is defined
- item.source is defined
- item.destination is defined
fail_msg: >-
Every WireGuard forwarding policy requires name, ingress_zone,
egress_zone, source, and destination.
loop: "{{ wireguard_forwarding_policies }}"
loop_control:
label: "{{ item.name | default('unnamed policy') }}"
- name: Install WireGuard userspace tools on mutable hosts
ansible.builtin.dnf:
name: "{{ wireguard_packages }}"
state: present
when: "'platform_fedora_iot' not in group_names"
- name: Require WireGuard userspace tools in the booted deployment
ansible.builtin.command:
argv:
- wg
- --version
changed_when: false
failed_when: false
register: wireguard_userspace_tools
- name: Require active WireGuard userspace tools
ansible.builtin.assert:
that:
- wireguard_userspace_tools.rc == 0
fail_msg: >-
WireGuard userspace tools are not present in the booted deployment.
On Fedora IoT, reboot after rpm-ostree layers wireguard-tools, then
rerun the WireGuard play.
- name: Create private WireGuard configuration directory
ansible.builtin.file:
@@ -124,12 +158,33 @@
permanent: true
register: wireguard_firewalld_zone_result
- name: Reload firewalld after creating the WireGuard zone
- name: Create the firewalld policy directory
ansible.builtin.file:
path: /etc/firewalld/policies
state: directory
owner: root
group: root
mode: "0755"
when: wireguard_forwarding_policies | length > 0
- name: Render WireGuard forwarding policies
ansible.builtin.template:
src: wireguard-forwarding-policy.xml.j2
dest: "/etc/firewalld/policies/{{ item.name }}.xml"
owner: root
group: root
mode: "0644"
loop: "{{ wireguard_forwarding_policies }}"
loop_control:
label: "{{ item.name }}"
register: wireguard_forwarding_policy_result
- name: Reload firewalld after WireGuard firewall changes
ansible.builtin.systemd:
name: firewalld.service
state: reloaded
when:
- wireguard_firewalld_zone_result is changed
- wireguard_firewalld_zone_result is changed or (wireguard_forwarding_policy_result | default({})) is changed
- not ansible_check_mode
- name: Restore rootful Podman networking after firewalld reload
@@ -142,7 +197,7 @@
register: wireguard_podman_network_reload
changed_when: wireguard_podman_network_reload.stdout_lines | length > 0
when:
- wireguard_firewalld_zone_result is changed
- wireguard_firewalld_zone_result is changed or (wireguard_forwarding_policy_result | default({})) is changed
- wireguard_reload_rootful_podman_networks | bool
- not ansible_check_mode
@@ -154,6 +209,14 @@
permanent: true
immediate: true
- name: Manage legacy WireGuard zone masquerading
ansible.posix.firewalld:
zone: "{{ wireguard_firewalld_zone }}"
masquerade: true
state: "{{ 'enabled' if wireguard_enable_masquerade | bool else 'disabled' }}"
permanent: true
immediate: true
- name: Permit this host's public WireGuard listener
ansible.posix.firewalld:
port: "{{ wireguard_listen_port }}/udp"

View File

@@ -11,7 +11,8 @@ ListenPort = {{ wireguard_listen_port }}
{% for peer in wireguard_peers %}
[Peer]
# {{ peer.name }}
PublicKey = {{ hostvars[peer.host].wireguard_public_key }}
{% if peer.comment is defined %}# {{ peer.comment }}
{% endif %}PublicKey = {{ hostvars[peer.host].wireguard_public_key }}
AllowedIPs = {{ peer.allowed_ips | join(', ') }}
{% if peer.endpoint is defined %}
Endpoint = {{ peer.endpoint }}

View File

@@ -0,0 +1,15 @@
<?xml version="1.0" encoding="utf-8"?>
<policy target="CONTINUE">
<short>WireGuard forwarding: {{ item.name }}</short>
<description>Managed WireGuard-to-LAN forwarding policy.</description>
{% if item.masquerade | default(false) %}
<masquerade/>
{% endif %}
<ingress-zone name="{{ item.ingress_zone }}"/>
<egress-zone name="{{ item.egress_zone }}"/>
<rule family="ipv4">
<source address="{{ item.source }}"/>
<destination address="{{ item.destination }}"/>
<accept/>
</rule>
</policy>

View File

@@ -73,13 +73,6 @@
- packages_rocky
- services_systemd
- name: Configure WireGuard overlay
hosts: wireguard_overlay
become: true
roles:
- wireguard_overlay
- name: Configure Aegis Fedora IoT profile
hosts: role_aegis
become: true
@@ -87,6 +80,13 @@
roles:
- profile_aegis
- name: Configure WireGuard overlay
hosts: wireguard_overlay
become: true
roles:
- wireguard_overlay
- name: Configure Atlas NAS profile
hosts: atlas
become: true

View File

@@ -56,24 +56,6 @@ services:
- "3000:3000"
- "127.0.0.1:222:22"
{% if server_syncthing_enabled | default(true) | bool %}
syncthing:
image: docker.io/syncthing/syncthing:2
container_name: syncthing
hostname: syncthing
restart: unless-stopped
expose:
- "8384"
volumes:
- "/opt/syncthing/config:/var/syncthing{{ ':' ~ selinux_volume_option if selinux_volume_option else '' }}"
- "/srv/syncthing/data:/data{{ ':' ~ selinux_volume_option if selinux_volume_option else '' }}"
ports:
- "22000:22000/tcp"
- "22000:22000/udp"
- "21027:21027/udp"
networks:
- web
{% endif %}
networks:
web:

View File

@@ -1,3 +1,6 @@
[tools]
java = "temurin-11.0.31+11"
java = ["25.0.2", "temurin-11.0.31+11"]
maven = "3.9.16"
[env]
JAVA_HOME = "{{ env.HOME }}/.local/share/mise/installs/java/25.0.2"

1
secrets/.gitignore vendored
View File

@@ -2,3 +2,4 @@
.vault_pass*
vault.local.yml
vault.local.yml*
/recovery/

View File

@@ -1,78 +1,83 @@
$ANSIBLE_VAULT;1.1;AES256
31386434333363613930316363393564373332303236306233643365386639346631336232373361
3732633931376438313835373537666438383664306266380a633531316432343166323361303465
39306136386664356163346266303963373839373763343136346135633236333333326331313063
6264306265373865310a373735393632373861333433326632303931633732636535326264346537
39633766326361666432396432623132616666363462373237623664363664373362346366383230
65326665303762356233356531306331333231376163353638363936626562623937323166623065
62633165383033303362336238343037353930396430336537363565333132666532323662653262
65373531313264353938666161373064393239653165666462336665323962336134343432316134
32373361613539356262313333663964623034313230356439626139383539326261663765373034
36633933366662656464306561393130373238313730336638663932633935313037363133636266
36663935303735623132663464376633306431396662306166393831313566323238363865303162
66303732376530653435613966373832333161333137366230613166303061333433336461386234
63393030616662373666643164663862653037383336323766646330653131623930353265336231
39646335353635616465663763306262353931316562616437313362616136623735646535616431
33363439623361343231363663363535663265366166346439623935323632366336363135373462
62646662316538323734333334393566333463653833326162663065646634363336373865623932
33646533636638386534366561663930613536663935663638313137646537626431393035653466
36623435613966323166363630383531656436373366343364346439343034333934623836336636
38653534343563393434646430306662343435653465363439326261373537666233353731306436
35323739336361343665633239366633393530346335396635316238323435663466316235376536
38303839313433643038353236616632626363653339346334303136306138336461383831626536
39303966393034333739363061363865326165303236656438316537396463383935393764346135
66323632323833353833643434316566326366633562666262386232653730303038613336663264
62636332383834393861373665393364653362636339346632303463386565633830336363393065
31383530613161366436323163393366386635316562633436383134623061353937326363396439
38636438396637616362373937666361303536376164636533653536306338356263313965623336
32633461623861643138613734313164633562613932386163643062353636376266633166373838
39613438653531316333663736366161313832613830353566356461393435656234383037353335
32343661376266353538643531313239306432646335383734623233373063316430383362643531
35393263343534363936373361316265653934383735663933663365363564663966646335353337
63366365383466393765336533363130643236313331633537346237353631623334633330376164
32616530623032663761643437336536323332343130393339313232623364656334613233323833
64346637343738386335343631333035636337633732333662663763326362636432646136353064
30393061326435323837633632333765623931663265326137373135303035623464333366653566
66613133396339376264313964353932396136613538383264363865633536373839663465333437
37613765626131343330653063663764346364316264363635653438396331363263623562333735
30646563303439663639376430336336383761363365353838613036333032306434623661636138
63616439636437663732386164346365643834376631303263376563323662373734653631613730
31373039656238303462643930303531613931373065316435303661343862623034653364653736
37613835616630316133333130633631653639336266313438633166316539373433333661613839
32356139366232313336343062323265366563633735383661393335386163653133313664613264
63303739323863333439316461366137656434666366306466356633306663623730653939663430
39313764663534316562326435316264633236373834323665356266346532323565323532333438
33306237623430613463393164383332356533303433343465343930636563363862373330613632
38643937613263313435626166313464316133373338373261666331653436373063393162636339
66356434663239646334383433626566336433653265336332323866633666646132663836313537
66663766306438396131623533613834626331653731323137303539303835306632663132616363
64303438353836613136623562626664326364656133383865383730373762666539383036396337
66393636343561333435313032353939316138306336316337353861383238326136373265663433
62366566343866626336646466386561353061343735306565383437333931613635393034363430
37633765663239643435623066313331353862393966306563393838386334393162306562363062
61633363386564373834383432323861323364623365306439353631323463326461383039356466
34396464653233626435656463326439303665633532656639376633353932666630626564616564
34326638383634353033326232646339393638663637313136653763336265616635326666336530
33663261376262626136346265656130653831636662306132393837306135643831353534626636
34383762386665666363313932336632326230646439663366663037323562633630373137333232
37393164656137303131313738396131613561306332356436303436636338623233343637363332
36613137316337666461333237373266326238303531396432383461616239316630346230303735
63636535383766353338643932366339333130663632386337643932636630316432313465393766
39613065623631316165313134386231616165366266323634643632626230303861376461323433
35346137326563646332303334313530383464373838373633363635373766333362386466323836
32613236646561343365656239346239353866663336393930356238653463336361333033643331
61323430663031383239353363616666336637666230663633616464616638303966343631386632
32333735643936353638666336303133356435396338653465323234316234626536666635333238
38313661303466333464666538363938656631396666643566343763396638336663376430646532
38666234653330646262633062356238343536336637356432313137313561363937363936383364
30643937383833663339643862363234643765386164316138636565643434373734383339363138
36323163633837326132393365333236633264386664373234313061373835346634663137383837
62666561353532303663346365343131316233633163323938623066656332383030393864363536
38383939383935613432613837333863313239653831333438383133343763633838353964353161
61323462343835613937653465633563306462613631323762656437626133336638396663646362
30323661383134653336366234663333336261353162373030626266656336356233316265636661
34303865313433633138363936373561636537353831373033303163646436303932626138356633
63656364353163313037613262396338636230646330666331616534313466306361363433656132
66633231626665303165346339373764666264313838313063323732653837383736633235363064
37353632336238623366313432376163653535656134633634313065356533343933666135396633
30613134646132613637656461303431613064393438363231383464663765316638
61353065386233646137323235306631353635663530363237636231316265643562353465323430
6165646466623962313835313537633137633766373930380a316335323962616265643136346666
63336133336131346336383534356637623831363138323165633262386333363535393365383233
6234393835653439370a313963313365373633323464343263383661383336363662633133643232
34366634383862363635653034313531623330396639616462343630326162316535643465653532
36326534333637376462353561343964633636366331363833313263353133383636623537303663
35393032316439336666343161653439643638376134363535656262343963393365623432336433
35383934313762313037326430316666363731666231336534326661353034333063643364343230
65333739303566366263333565333465613136646237623937393733623438613832393634663463
39376131313234333039633735613233373931613232653036663665316636303961653834366339
36353730316132316233303964303839363161346564396163336137663134353062363733656430
37643339326661653031376265646132623162373562393437373437313732396537383939333666
62353036316633306666313461663033303830393765396131643035353730383931646239663935
32626461316364386135303761383837613063336466363162323332663764616464373565383231
61346463336566346533326535376439643133613762383633396131323632356533636139336365
62393838316634623932643034376631333539343965383436613364643962363834346337353334
32656439366439313734353963343133333533653839613632323338336131373566613835393536
31663433616334373432376531346435336530303936356461303163646463613661643161313661
66663866343565616631616338353737356164353562366164383736346131666662623132333466
39383865653631373232393433663430643961646265386166333137643966303834363262373636
62396434373363353636376133666133663162653265313139313732353639336232333862643036
64386231336561396537326139346566306434633934343038663165396665363032383466633662
62336163633964363435386630343966333162333730336138333239646631633132663931376462
33663139356261313065376636613930353735396131306538306664646135636336643032623131
38346264333331353633326535326431626563323036313665643337353563333339646430386564
31613435383036313430316366323636663735326336393338353835323861333564363832656462
35336435623261326363633033316130393062616339353263643062633331646137376135656365
35636139336564346164616235616431326531333433646330386134323932373339646536356464
66343533326534326165323564663533653666633035343163633832393361336462343937623165
62383931326630363036396333313931393836366439653433623165666166356338653364336534
35333936653833386163633738326164386166613561333530633937343230363366333662666539
39333361633933663735303438663239303536363433313962643137386533633539326365383765
37636538386339333935386132353265353031643662616330316463623661663738353433313830
36373963633166333464653338343830373063323536383364393033393235326639613662343737
38663362636331343061646465313237313431373433353361353265333766633463353632646536
31323231306138323031396630656538363930373439336234343963616334363632653738316465
63653938373830336362313238656266613362636634616537653863336132343931616262396130
66393239303866656232653832343132366537333537343635666563343639323433383163613335
39613533376634316133633430303535306266656333626264343733666335393661666561396633
39346265316137326465326635396362333565393133623637633132616232326263663662343137
33363733306135363361643031306265363733656362386666306334333035393839636533343363
35396638616636633639343930373136376339346162393061393765363837646365383866636131
33653465666239393133616232636231333332396138376332393664343364643835306530393238
34663237303530303837663535646263393931373531393039356336316561653130356262636562
38336362326639653237626634376334666565653036353236313634376364626338646538386536
38626636386466373566646166393963643164343536373236396138303532393161363335386638
32633032393737626363613463323366366637616361313537356136626661626633613739323338
35383963666431343566356562333234663936376562616638636261303466633539376334303331
39303834663234663063356233313962326664383839393832303462643636393034383434303465
64333635376135326333356435373734643430623736373234643335343130383066326436356664
63346663326364343634303930343338336139313864316165366232643537366635653764353763
31363863633261643263303433373330366161323166366462336332313135366338393334653764
66353733653137663835663731373364613030373334663061313433373861613665363236633130
65613965366636343465336533613438373466383737373366653965633437323562643966396431
39303033643438633762633263326132663466643438656366363431616237633031333936313831
30323930383233313032323638356333626230333764363662313662646536643839353032353462
30326166653937353130623133303533343934633565393831623033303234316330353432313266
30636536633933376365623665616262663236383731633633346232613366333137396139306363
35633336643266326335303261666666653536666630613639376336373237646134306462616537
33343561373162666332613634643837343566646161373065366637653135613632353334636363
63363232303963646530333366663862323264326536643337323266396566316233613630303637
66646366376466373931613734363931316230323063373666653062373364396433633762633762
38613933323733653238383935623230383562646563363833653838636165626365646537383639
33666535656363393562316336633439636138373365623431393965653765306138646234663938
65653133663663393731646337386535333261643932336132396237323930306136643534353930
65636438396432623034626561613137336138623265393064383034623863303166356138393564
37373164626634653662326234333539663735323464613334616130643937373730363263633366
31393937326432386165343338313031376565313866363731643534313233303064373935303538
31343832336230393636653432653162336361383963633766343461653466316337353931333363
63313137303564336630343937356564643763383764613362366634373362666465626334336539
64366533376165306532343461613265366266383862323032333465336161663161376630316465
30306562666163646235656664653635366461366435663961623635383437663564356563346462
31636234663765623838333237393239373564366262613637363938653463396530613963643837
38636634376637366332623035313465393762653865623130336263343663303066366135616639
63333964356466613038303263366462346261353030646532366361393965306435613131316463
65366266376637323764643239323730366565633335666638666334663635373961303637383861
35313431646434656562333937663837393038386361616630626532636339306432353434656165
33663261383166386432383465666136376237346565303164363461666663346130346162316338
62373061353034316234303835663439396434343738303764376665336239626238386436386234
61306166383637366266393730323732386163366261393630336431633862353761343763363665
61323039396234393835303633363339373633653334343766653032313230343464326664356566
3462623830666664626633373966363866333337383730313066

View File

@@ -3,14 +3,13 @@ vault_duckdns_token: "CHANGEME"
vault_personal_full_name: "REPLACE_ME"
vault_git_email: "REPLACE_ME"
vault_git_signing_key: "REPLACE_ME"
vault_icloud_email: "REPLACE_ME"
vault_protonmail_email: "REPLACE_ME"
vault_icloud_mail_password: "REPLACE_ME"
vault_git_work_email: "REPLACE_ME"
vault_git_work_gpg: "REPLACE_ME"
vault_openai_api_key: "REPLACE_ME"
vault_ikaros_authorized_ssh_keys:
- "ssh-ed25519 REPLACE_ME"
vault_aegis_icloudpd_apple_id: "REPLACE_ME"
vault_atlas_admin_password_hash: "REPLACE_WITH_A_SHADOW_COMPATIBLE_HASH"
vault_atlas_samba_password: "REPLACE_ME"
vault_atlas_immich_db_password: "REPLACE_ME"
vault_atlas_borg_passphrase: "REPLACE_WITH_A_STRONG_UNIQUE_PASSPHRASE"