mirror of
https://github.com/fscotto/infra.git
synced 2026-09-27 19:03:47 +00:00
Compare commits
8 Commits
main
...
2ab5ba6818
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2ab5ba6818 | ||
|
|
a347880d4c | ||
|
|
b05a2b4e74 | ||
|
|
75d9081a11 | ||
|
|
b4b3bd10b8 | ||
|
|
db10d1296e | ||
|
|
8c35ef63c9 | ||
|
|
64aebe8c34 |
206
AGENTS.md
206
AGENTS.md
@@ -48,8 +48,6 @@ Ansible-driven personal infrastructure repo for Fedora and Void desktops, Fedora
|
||||
- Rocky server after activation: `ansible-playbook ansible/site.yml --limit <host> --check --diff`
|
||||
- Atlas NAS: `ansible-playbook ansible/site.yml --limit atlas --check --diff`
|
||||
- Aegis IoT: `ansible-playbook ansible/site.yml --limit aegis --check --diff`
|
||||
- Aegis NFS client layer: `ansible-playbook ansible/site.yml --limit aegis --tags nfs --list-tasks`
|
||||
- Aegis host DNS: `ansible-playbook ansible/site.yml --limit aegis --tags dns --check --diff`
|
||||
- Focused checks:
|
||||
- Emacs is disabled by default; temporary Emacs check: `ansible-playbook ansible/site.yml --limit <host> --tags emacs --check --diff -e emacs_enabled=true`
|
||||
- AI coding agents: `ansible-playbook ansible/site.yml --limit <host> --tags ai_agents --check --diff`
|
||||
@@ -59,20 +57,10 @@ Ansible-driven personal infrastructure repo for Fedora and Void desktops, Fedora
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags storage,sharing,containers --check --diff`
|
||||
- Atlas network/share hardening:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags hardening,sharing --check --diff`
|
||||
- Atlas ZFS snapshot retention and scrub timers:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff`
|
||||
- Atlas encrypted Borg backup:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff`
|
||||
- Atlas Borg progress logging only:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags borg_logging --check --diff`
|
||||
- Atlas manual offline USB backup and 45Drives Alerts reminder:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags usb_backup,usb_reminder --check --diff`
|
||||
- Atlas pool, disk, capacity, temperature, and job monitoring:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags monitoring --check --diff`
|
||||
- Atlas explicit post-restore SELinux relabeling:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags restorecon --check -e '{"atlas_restorecon_paths":["/zpool/archive"]}'`
|
||||
- Prometheus/Aegis WireGuard gateway:
|
||||
`ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff`
|
||||
- Atlas phase-one rootless services:
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff`
|
||||
- Prometheus/Atlas WireGuard overlay:
|
||||
`ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard --check --diff`
|
||||
- DuckDNS config only: `ansible-playbook ansible/site.yml --limit prometheus --tags duckdns --check --diff`
|
||||
|
||||
## Conventions
|
||||
@@ -149,10 +137,10 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
|
||||
- Atlas requires `vault_atlas_admin_password_hash` for Cockpit and, while sharing is enabled,
|
||||
`vault_atlas_samba_password`. The future rootful media stack also requires
|
||||
`vault_atlas_immich_db_password`. Never print these values.
|
||||
- Atlas creates the complete declared hierarchy only under the verified existing or explicitly bootstrapped pool: `archive`,
|
||||
`services`, `services/data`, `services/data/navidrome`, `services/data/syncthing`, `media`, `media/music`,
|
||||
`media/photobook`, `backup`, `backup/hosts`, and `backup/hosts/prometheus`. `backup` has a `500G`
|
||||
reservation covering its descendants. `archive` is the SMB-shared raw-data namespace; container state is never beneath it.
|
||||
- Atlas creates the complete declared hierarchy only under the verified existing or explicitly bootstrapped pool: `work`, `archive`,
|
||||
`archive/app_data`, `archive/app_data/navidrome`, `archive/app_data/syncthing`, `media`, `media/music`,
|
||||
`media/photobook`, `backups`, `backups/services`, and `backup_prometheus`. `backups/services` has a `500G`
|
||||
refreservation. There is no separate legacy `zpool/syncthing` dataset.
|
||||
- The `immich` system account is fixed to UID/GID `1100`, has no login shell or `wheel` membership, and receives only
|
||||
the `video` and `render` supplementary groups. Immich's rootful Quadlets run as `1100:1100`; Server and ML receive
|
||||
`/dev/dri`, while the Photobook external library is read-only at `/external/photobook`.
|
||||
@@ -161,136 +149,49 @@ The dotfile vars follow the same split: `desktop_common_dotfiles` carries mode-i
|
||||
`1100`. Targeted SELinux is enforced persistently; a required reboot is reported but never initiated automatically. The primary LAN interface is assigned explicitly to the managed firewalld zone, and firewall rules are applied before NFS or SMB are started; their service state and TCP listeners are then verified. SMB3 exposes `Archive` to Vault-backed authorized accounts on mandatory encrypted, signed SMB3 over TCP/445 only and admits the configured LAN without host-specific exclusions.
|
||||
- Atlas NPM and Immich share a rootful Podman network. NPM publishes HTTP/HTTPS, but its administration port remains
|
||||
bound to `127.0.0.1:81`; do not expose it directly to the LAN or Internet.
|
||||
- `profile_backend_phase1` temporarily runs rootless Navidrome and Syncthing on Atlas until Uranus replaces
|
||||
them. It binds only to Atlas' LAN IP, never `wg0`; Navidrome and the Syncthing GUI admit only Aegis as
|
||||
the source-NAT gateway, while native Syncthing ports admit the configured LAN. It initializes fresh
|
||||
state only and never migrates or deletes source application data.
|
||||
- `wireguard_overlay` manages `wg0` between Prometheus (`10.0.0.1`) and Aegis (`10.0.0.2`). It persists private
|
||||
keys only on their respective hosts, exchanges only derived public keys through Ansible, and verifies a real peer
|
||||
handshake. Prometheus opens `51820/udp`; Aegis is the LAN gateway. Its persistent IPv4 forwarding, narrowly scoped
|
||||
WireGuard-to-LAN firewalld policy, and source masquerading permit Prometheus to reach LAN services without a static
|
||||
route on the router. Prometheus includes `192.168.178.0/24` in Aegis' peer `AllowedIPs`; add the Uranus VIP there
|
||||
when it is assigned. After a firewalld reload, restore Prometheus' rootful Podman networking with
|
||||
`podman network reload --all` so the existing proxy stack retains container DNS.
|
||||
- `profile_backend_phase1` is limited to rootless Navidrome and Syncthing user Quadlets on Atlas. Official Navidrome
|
||||
`0.63.2` uses SQLite below `/data` and does not support `ND_DATABASE_URL` or an external PostgreSQL backend; do not
|
||||
recreate the obsolete Prometheus `navidromedb` service. The role requires the storage role's `zpool/media/music`,
|
||||
`zpool/archive/app_data`, `zpool/archive/app_data/navidrome`, and `zpool/archive/app_data/syncthing` datasets at
|
||||
their exact paths. It never creates the pool.
|
||||
- Keep `backend_phase1_start_services` false until the stopped Prometheus Navidrome data directory has been copied to
|
||||
Atlas and its SQLite database verified. The playbook renders the target but never migrates or deletes application
|
||||
data; after cutover, set the flag true to enable and start Navidrome and Syncthing.
|
||||
- Phase 1 must not change Prometheus' existing NPM deployment. NPM continues to be managed exactly by `profile_server`;
|
||||
use `10.0.0.2:4533` for Navidrome and `10.0.0.2:8384` for the Syncthing GUI. Syncthing does not use host networking:
|
||||
its GUI, transfer, QUIC and discovery ports are explicitly published only on `10.0.0.2`; native transfer/discovery does not use the HTTP proxy.
|
||||
- `wireguard_overlay` manages the required `wg0` path between Prometheus and Atlas, persists private keys only on their
|
||||
respective hosts, and exchanges only derived public keys. The initial run must include both hosts. Prometheus
|
||||
opens `51820/udp`; the Atlas backend role admits service ports only in the WireGuard firewalld zone.
|
||||
|
||||
## Atlas NAS TODO
|
||||
Completed validation: the existing RAIDZ2 pool and datasets, SELinux, LAN firewall, SSH, Cockpit with
|
||||
the selected 45Drives plugins, encrypted SMB3 `Archive`, the Aegis-only NFSv4 `photobook` export, and the
|
||||
Prometheus--Aegis WireGuard gateway are operational. The gateway handshake, forwarding, source masquerading,
|
||||
and TCP reachability to Atlas were verified. Temporary Navidrome and Syncthing are available through their
|
||||
manual NPM Proxy Hosts; Syncthing uses `/data/Org` backed by the SMB-shared Archive dataset. Aegis has also
|
||||
validated NFSv4.2 read, write, delete, and `all_squash` mapping to UID/GID `1100` end-to-end. The ZFS
|
||||
snapshot timers are active and the first recursive hourly snapshot completed successfully; the first
|
||||
scheduled retention prune and monthly scrub remain runtime checks.
|
||||
|
||||
### Priority 1 - Data protection
|
||||
- [x] Deploy Ansible-managed recursive ZFS snapshots with 24 hourly, 30 daily, 8 weekly, and 12 monthly
|
||||
generations, plus a monthly scrub on the first Sunday at 03:00. The timers and first hourly snapshot were
|
||||
verified on Atlas. Still observe the first scheduled retention prune and scrub; Cockpit Scheduler is for
|
||||
visibility or manual operations only, and snapshot rollback is never automated.
|
||||
- [x] Activate and validate the encrypted offsite Borg backup to the Hetzner Storage Box. Atlas uses the
|
||||
dedicated SSH identity, pinned ED25519 host key, Vault-backed `repokey` encryption, and a locked
|
||||
non-login `borg` account with no sudo or supplementary groups. The initial snapshot-consistent backup,
|
||||
Borg repository check, and temporary-directory restore completed successfully; the restored `Archive`
|
||||
tree matched the live data, and temporary snapshots and mounts were removed. The exported recovery key
|
||||
was copied offline. Daily backup retries and logging, 30 daily, 8 weekly and 12 monthly archives,
|
||||
compaction, and monthly repository checks are enabled. Future runs report a ZFS-based estimated
|
||||
percentage, and a post-exit helper handles host-namespace temporary snapshot cleanup. The active
|
||||
run predates the new progress logging and still requires an observed final cleanup result.
|
||||
- [x] Populate `/zpool/archive` with the currently available data so offsite and offline backup tests run
|
||||
against a representative load.
|
||||
- [ ] Run and evaluate Borg against the populated pool: duration, repository capacity, deduplication, and
|
||||
a subsequent incremental archive must be observed before considering the offsite path fully validated.
|
||||
- [x] Add the UUID-bound offline USB backup with versioned rsync, locking, capacity checks, verification,
|
||||
safe unmounting and a tested restore procedure; never trigger it for an arbitrary USB disk. The
|
||||
LUKS/ext4 identities were read-only verified; the manual service and 45Drives Alerts reminder timer were
|
||||
deployed on Atlas. Interactive LUKS unlock is part of the manual service; only the reminder is
|
||||
scheduled for the first Saturday of each month at 10:00 Europe/Rome via the existing 45Drives
|
||||
notifier. A manual test produced an Alerts notification, not an email. The first USB attempt failed
|
||||
on a `security.selinux` xattr and was interrupted; the xattr filter is deployed and the temporary
|
||||
recursive snapshot and open LUKS mapper were cleaned up. A later run reported checksum verification
|
||||
and published the USB version, but failed while removing host-namespace ZFS snapshot mounts. Those
|
||||
exact mounts and snapshots were cleaned up. An `ExecStopPost` helper now removes only the named
|
||||
temporary snapshot after the backup process exits. A new full run checksum-verified and published a
|
||||
USB version; the service ended successfully, the mapper closed, no temporary USB snapshot remained,
|
||||
and the pool was healthy. On 2026-09-25 an independent, read-only USB restore test copied one file from
|
||||
the published `atlas/latest` version into `/var/tmp` and matched contents, owner, mode, size, mtime and
|
||||
POSIX ACL. The temporary copy and mount were removed, the mapper closed, and the pool remained healthy.
|
||||
- [x] Test restores independently from a ZFS snapshot, Borg, and the offline USB backup before relying on
|
||||
any backup path. The earlier Borg temporary-directory restore passed. On 2026-09-25 a separate,
|
||||
read-only ZFS snapshot test restored one file to `/var/tmp`, confirmed matching contents, ownership,
|
||||
mode, mtime and ACL, then removed its temporary copy and on-demand mount. This is a file-level smoke
|
||||
test, not full dataset recovery. An independent USB file restore passed on 2026-09-25 with matching
|
||||
content and metadata; full disaster recovery remains a separate Priority 2 task.
|
||||
- [x] Add monitoring and alerting for pool health, scrub/resilver, SMART data, temperatures, free space,
|
||||
snapshot/local-backup growth, Hetzner Storage Box quota, and failed maintenance or backup timers.
|
||||
The half-hourly Atlas health monitor and systemd final-failure hooks are deployed. A live probe
|
||||
found no issues; the service and timer succeeded, and a labelled 45Drives Alerts test notification
|
||||
was submitted. Alerts are deduplicated; email delivery is not claimed. The Storage Box quota probe
|
||||
runs `df -m` over the dedicated pinned-key SSH identity and does not open the Borg repository.
|
||||
Detailed archive size and deduplication remain part of the pending Borg evaluation.
|
||||
|
||||
### Priority 2 - NAS operability and recovery
|
||||
- [ ] Document and test disaster recovery: rebuild Atlas with Ansible, import the existing pool, restore
|
||||
from snapshot/USB/Hetzner, preserve Vault and Borg recovery material offline, and define RPO/RTO.
|
||||
- [ ] Define a controlled Rocky kernel/OpenZFS update and reboot procedure.
|
||||
- [ ] Add the Atlas-initiated least-privilege Prometheus backup pull: Prometheus exposes only prepared
|
||||
- Provide the required Vault variables and validate the first remote bootstrap on the real Rocky Linux 9 host.
|
||||
Before the first apply, confirm the pool, mountpoints, LAN subnet and firewalld zone. Keep
|
||||
`atlas_manage_media_stack` disabled until `/dev/dri`, the container paths and the Immich database secret are validated.
|
||||
- Validate the complete baseline on the target: OpenZFS kmod loading, existing pool import, dataset
|
||||
mounts, SSH reconnect, Cockpit and all selected 45Drives plugins, NFSv4, SMB and Syncthing.
|
||||
- Finalize dataset properties and the shared UID/GID, group and POSIX ACL model; test the same files
|
||||
through both NFS and SMB before considering multiprotocol access complete.
|
||||
- Add Ansible-managed ZFS snapshot retention and scrub timers. Use Cockpit Scheduler for visibility
|
||||
or manual operations, not as the only source of configuration, and never automate snapshot rollback.
|
||||
- Manage the Syncthing star topology, device IDs, folders, folder modes, ignore rules and protected GUI
|
||||
or API access for the selected clients.
|
||||
- Validate the managed WireGuard path and its LAN/VPN-only firewalld rules before enabling remote services;
|
||||
never expose SSH, Cockpit, NFS, SMB or Syncthing through public port forwarding.
|
||||
- Add the Atlas-initiated least-privilege Prometheus backup pull: Prometheus exposes only prepared
|
||||
read-only dumps through a dedicated account and Atlas retains the private SSH key, pinned host key,
|
||||
atomic pull, verification, retention and systemd service/timer.
|
||||
- [ ] Decide whether a common SMB/NFS namespace is required. `Archive` (SMB) and `photobook` (NFS) are
|
||||
intentionally distinct today; only if a shared namespace is selected, finalize its UID/GID, group,
|
||||
and POSIX ACL model and test the same files through both protocols.
|
||||
|
||||
### Priority 3 - Service expansion
|
||||
- [ ] After data protection and recovery are validated, populate `/zpool/media/music` and validate Navidrome.
|
||||
- [ ] Design and deploy Nextcloud as another explicitly temporary Atlas service before Uranus. Give it
|
||||
separate persistent application, database, and cache storage; keep credentials in Vault; publish it only
|
||||
through NPM over the Prometheus--Aegis gateway; and define backup, upgrade, and eventual Uranus-migration
|
||||
procedures before exposing user data. Do not deploy Nextcloud before the data-protection checklist is complete.
|
||||
- [ ] Keep `atlas_manage_media_stack` disabled until the future Immich deployment has validated `/dev/dri`,
|
||||
container paths, and the required Vault database secret.
|
||||
|
||||
### Priority 4 - Optional workflows
|
||||
- [ ] After data protection is validated, move iCloudPD photo ingestion from Aegis to Atlas as a
|
||||
temporary service until Uranus is ready. Plan to store photos in `/zpool/archive/Pictures` and
|
||||
persistent application/MFA state outside `Archive`; validate permissions, SELinux, backups and
|
||||
recovery before cutover. Keep the current Aegis service and Photobook NFS export unchanged until
|
||||
the Atlas workflow is tested, then retire them explicitly if no longer needed.
|
||||
|
||||
## Cerberus Management Node (Deferred)
|
||||
`cerberus` is postponed until the office in the new house is physically set up. It is not an inventory
|
||||
host and this section is a design and implementation backlog, not authorization to provision it early.
|
||||
|
||||
The planned node is a Lenovo ThinkCentre M700 Tiny with an Intel Core i3-6100T, 8 GB RAM, a 256 GB SSD,
|
||||
and native 1 Gbps Ethernet. It will connect to a multi-input KVM switch using a passive DisplayPort-to-HDMI
|
||||
cable, sharing the monitor and peripherals with Ikaros. Fedora Sericea (immutable Fedora with the Sway
|
||||
Wayland compositor) is the intended OS. Cerberus is an isolated management plane: a dedicated Toolbox
|
||||
environment will run Ansible for future `uranus` cluster provisioning. Rootless Podman will host Grafana,
|
||||
Prometheus, and Loki. The 256 GB local SSD is the hot tier retaining metrics and logs for 30 days; scheduled,
|
||||
validated exports of older historical data will use a dedicated Atlas NFS dataset as cold storage.
|
||||
|
||||
### Implementation plan
|
||||
- [ ] Confirm the office, KVM switch, passive DisplayPort-to-HDMI path, shared monitor/peripherals, and native
|
||||
1 Gbps Ethernet are physically operational before adding Cerberus to inventory.
|
||||
- [ ] Install and update Fedora Sericea with Sway; document the immutable-host lifecycle and keep host changes
|
||||
declarative rather than treating the base OS as a mutable workstation.
|
||||
- [ ] Model Cerberus as its own host with independent platform, role, desktop, network, and storage inputs;
|
||||
do not repurpose Ikaros variables or make it a Uranus cluster member.
|
||||
- [ ] Provision an isolated Toolbox-based Ansible controller with the required collections and a reproducible
|
||||
project checkout; define its least-privilege SSH access, known-host handling, and Vault workflow without
|
||||
storing secrets in the image or repository.
|
||||
- [ ] Define the explicit Uranus provisioning workflow from Cerberus, including inventory boundaries,
|
||||
validation-only runs, and separate approval for any destructive cluster operation.
|
||||
- [ ] Design rootless Podman/Quadlet services for Grafana, Prometheus, and Loki, including persistent local
|
||||
state, service ownership, LAN exposure/authentication, resource limits, updates, and backups.
|
||||
- [ ] Size and enforce a 30-day local hot-retention policy for metrics and logs on the 256 GB SSD; validate
|
||||
actual disk growth and alert before capacity exhaustion.
|
||||
- [ ] Create and validate a dedicated Atlas NFS cold-storage dataset and least-privilege export for Cerberus;
|
||||
do not use a broad existing share or couple it to unrelated Atlas application state.
|
||||
- [ ] Implement scheduled, idempotent exports of data older than 30 days to the Atlas NFS cold tier, with
|
||||
locking, capacity checks, integrity verification, retention rules, failure monitoring, and a tested restore.
|
||||
- [ ] Validate management-plane recovery: rebuild Cerberus, restore observability history from Atlas, and
|
||||
confirm that Uranus provisioning can resume without depending on unreproducible local state.
|
||||
- Add the encrypted offsite backup with Borg to a Hetzner Storage Box: use a dedicated SSH identity,
|
||||
pin the host key, keep Borg repository credentials and encryption material in Vault, use
|
||||
snapshot-consistent sources, and manage retries, logging, pruning, repository checks and restores.
|
||||
- Add the UUID-bound offline USB backup with versioned rsync, locking, capacity checks, verification,
|
||||
safe unmounting and a tested restore procedure; never trigger it for an arbitrary USB disk.
|
||||
- Add monitoring and alerting for pool health, scrub/resilver, SMART data, temperatures, free space and
|
||||
failed backup timers, plus a controlled Rocky kernel/OpenZFS update and reboot procedure.
|
||||
- Document and test disaster recovery: rebuild Atlas with Ansible, import the existing pool, restore
|
||||
from snapshot/USB/Hetzner, preserve Vault and Borg recovery material offline, and define RPO/RTO.
|
||||
- Optionally design iCloud photo ingestion as a separate workflow after the storage and backup layers
|
||||
are validated; do not make it a dependency of the Atlas baseline.
|
||||
|
||||
## Coding Agent Notes
|
||||
- Shared agent definitions and lifecycle flags live in `ai_agents` in `ansible/inventory/group_vars/all.yml`.
|
||||
@@ -317,19 +218,14 @@ validated exports of older historical data will use a dedicated Atlas NFS datase
|
||||
- `aegis` is a remote Fedora IoT Raspberry Pi 4 node. Bootstrap it once with
|
||||
`ansible/bootstrap/aegis.bu`; the remaining configuration is applied by `profile_aegis` over SSH.
|
||||
- Fedora IoT is immutable. Do not add it to mutable Fedora package or shared dotfile roles.
|
||||
- `profile_aegis` owns the `nfs-utils` and `wireguard-tools` rpm-ostree layers and reports the required reboot
|
||||
without initiating it. `wireguard_overlay` then configures Aegis as the WireGuard LAN gateway with persistent IPv4
|
||||
forwarding, a scoped inter-zone policy, and source masquerading. It also owns rootful Podman Quadlets, persistent container
|
||||
state under `/var/lib`, the Podman auto-update timer, LAN-restricted firewalld rules, and SSH hardening. Keep
|
||||
`aegis_lan_subnet`, `aegis_adguard_web_port`, and `aegis_network_connection_uuid` host-specific;
|
||||
SSH permits only the declared
|
||||
- `profile_aegis` owns rootful Podman Quadlets, persistent container state under `/var/lib`, the
|
||||
Podman auto-update timer, LAN-restricted firewalld rules, and SSH hardening. Keep
|
||||
`aegis_lan_subnet` and `aegis_adguard_web_port` host-specific; SSH permits only the declared
|
||||
key-authenticated users, never root or password authentication. Keep Apple IDs and other
|
||||
credentials in Vault and use `no_log` for their rendering.
|
||||
- `aegis_adguard_web_port` defaults to `80`. The initial AdGuard Home wizard port `3000` is intentionally unmanaged: open and close it manually only while
|
||||
completing initial setup. Disable the local systemd-resolved stub through `profile_aegis` before
|
||||
AdGuard binds port 53; keep
|
||||
`/etc/resolv.conf` linked to `/run/systemd/resolve/resolv.conf`. LAN clients may use AdGuard, but
|
||||
Aegis must use the independent upstream DNS declared by `aegis_host_dns_servers` so Greenboot does
|
||||
not depend on the AdGuard container during startup.
|
||||
`/etc/resolv.conf` linked to `/run/systemd/resolve/resolv.conf` so Aegis retains router-provided DNS.
|
||||
- iCloudPD requires post-deployment interactive MFA initialization; its cookie/configuration state is
|
||||
persisted in `/var/lib/icloudpd/config`.
|
||||
|
||||
305
README.it.md
305
README.it.md
@@ -95,27 +95,6 @@ Nota sullo stato attuale del playbook principale:
|
||||
- `ansible/site.yml` applica il profilo server Rocky a `prometheus` con DNF, systemd, dotfiles server e firewalld
|
||||
- `ansible/site.yml` applica il profilo NAS Rocky su `atlas` tramite SSH remoto
|
||||
|
||||
## Nodo pianificato e posticipato: Cerberus
|
||||
|
||||
`cerberus` e un nodo di management **posticipato**, in attesa dell'allestimento
|
||||
fisico dell'ufficio nella nuova casa. Non e ancora presente nell'inventory e non
|
||||
esistono ruoli o playbook che lo prendano come target.
|
||||
|
||||
L'hardware previsto e un Lenovo ThinkCentre M700 Tiny (Intel Core i3-6100T,
|
||||
8 GB di RAM e SSD da 256 GB) con Ethernet nativa a 1 Gbps. Condividera monitor
|
||||
e periferiche di Ikaros tramite uno switch KVM a ingressi multipli, usando un
|
||||
cavo passivo DisplayPort-HDMI per il collegamento video. Il sistema operativo
|
||||
previsto e Fedora Sericea, la variante Fedora immutabile con compositor Wayland
|
||||
Sway.
|
||||
|
||||
Cerberus sara un management plane isolato: Ansible verra eseguito in un ambiente
|
||||
Toolbox dedicato per il provisioning del futuro cluster `uranus`, anziche da
|
||||
Ikaros o da un host non gestito. Lo stack di osservabilita rootless Podman
|
||||
eseguira Grafana, Prometheus e Loki. L'SSD locale sara l'hot storage, con
|
||||
metriche e log conservati per 30 giorni; esportazioni programmate trasferiranno
|
||||
i dati storici piu vecchi su un dataset Atlas montato via NFS come cold storage.
|
||||
Il piano di implementazione, con prerequisiti espliciti, e in `AGENTS.md`.
|
||||
|
||||
## Desktop
|
||||
|
||||
Target operativi:
|
||||
@@ -267,252 +246,90 @@ ansible-playbook ansible/site.yml --limit prometheus -e server_username=myuser -
|
||||
|
||||
## NAS
|
||||
|
||||
`atlas` è un NAS Rocky Linux 9 raggiunto via SSH. Normalmente il pool esiste già e il profilo gestisce
|
||||
solo i dataset figli. La creazione iniziale del RAIDZ2 richiede esplicitamente `atlas_create_pool=true`
|
||||
e quattro percorsi `/dev/disk/by-id/...` verificati in `atlas_zpool_disks`. Il ruolo non partiziona,
|
||||
forza, distrugge, ripristina né modifica il layout vdev di un pool esistente. I client Linux usano NFSv4,
|
||||
quelli Windows/WSL SMB; l'accesso è limitato alla LAN configurata.
|
||||
`atlas` e un NAS Rocky Linux 9 raggiunto tramite SSH. Normalmente il pool ZFS esiste gia e il profilo
|
||||
gestisce solo i dataset figli. Un bootstrap RAIDZ2 una tantum e disponibile solo con conferma esplicita
|
||||
(`atlas_create_pool=true`) e quattro percorsi reali e verificati `/dev/disk/by-id/...` in
|
||||
`atlas_zpool_disks`. Non partiziona, forza, distrugge, esegue rollback o modifica il layout vdev di un
|
||||
pool esistente. I client Linux usano NFSv4, quelli Windows/WSL SMB; entrambi restano limitati alla LAN
|
||||
configurata.
|
||||
|
||||
Per il primo avvio servono `vault_atlas_admin_password_hash`, `vault_atlas_samba_password` e
|
||||
`vault_atlas_immich_db_password`; il primo è un hash compatibile con `/etc/shadow`, non una password
|
||||
Cockpit in chiaro. Il bootstrap usa l'amministratore preesistente:
|
||||
Per il primo avvio fornire `vault_atlas_authorized_ssh_keys`, `vault_atlas_admin_password_hash`,
|
||||
`vault_atlas_samba_password` e `vault_atlas_immich_db_password`. Eseguire il bootstrap tramite
|
||||
l'amministratore esistente:
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas \
|
||||
-e atlas_connection_username=<existing-admin>
|
||||
```
|
||||
|
||||
Le esecuzioni successive usano `atlas_admin_username`. Storage, condivisioni e firewall LAN sono
|
||||
abilitati; prima dell'applicazione verificare pool, mountpoint, subnet e zona firewalld. La creazione
|
||||
del pool è protetta da un gate esplicito e avviene solo se è assente. Atlas non fa più parte della VPN
|
||||
WireGuard: la vecchia interfaccia è stata ritirata manualmente dopo la verifica del collegamento tra
|
||||
Prometheus e Aegis. Le chiavi SSH autorizzate sono in file separati sotto
|
||||
`~/.ssh/authorized_keys.d/`. `atlas_manage_media_stack` resta disabilitato finché `/dev/dri`, percorsi
|
||||
dei container e segreto del database Immich non sono validati.
|
||||
`vault_atlas_admin_password_hash` deve essere un hash compatibile con `/etc/shadow`, non una
|
||||
password Cockpit in chiaro. Le esecuzioni successive usano `atlas_admin_username`. Atlas dichiara
|
||||
abilitati storage, condivisioni e regole firewall LAN. Prima della prima applicazione verificare pool e
|
||||
mountpoint esistenti, subnet LAN e zona firewalld attiva. `atlas_manage_media_stack` resta disabilitato
|
||||
finche non saranno validati `/dev/dri`, i percorsi dei container e il segreto del database Immich.
|
||||
|
||||
Sotto `zpool` Atlas crea `archive` (SMB), `services/data` con i dataset applicativi
|
||||
`services/data/navidrome` e `services/data/syncthing`, `media`, `media/music`, `media/photobook` e
|
||||
`backup/hosts/prometheus`. Archivio e applicazioni usano `zstd`; media, Syncthing e backup host usano
|
||||
`lz4`. `backup` ha una riserva di `500G` che copre i discendenti. SELinux targeted è persistente;
|
||||
l'eventuale riavvio necessario viene segnalato, non eseguito. Atlas assegna l'interfaccia primaria
|
||||
alla zona firewalld gestita, rifiuta redirect e source route, registra i martian, mantiene il reverse-path
|
||||
filter loose e disabilita il forwarding IPv4. SSH consente soltanto l'amministratore dichiarato con
|
||||
chiave pubblica: root, password, agent forwarding e remote forwarding sono disabilitati, mentre il
|
||||
forwarding locale resta disponibile per i tunnel amministrativi. SMB3 espone `Archive` agli account
|
||||
autorizzati da Vault sulla LAN, solo su TCP/445 con cifratura e firma obbligatorie. NFSv4 espone
|
||||
soltanto `media/photobook` all'IP di Aegis su TCP/2049, con `all_squash` verso UID/GID `1100`.
|
||||
Con la gestione storage attiva, Atlas crea l'intera gerarchia sotto il pool `zpool` esistente o creato esplicitamente:
|
||||
`work`, `archive`, `archive/app_data`, i dataset applicativi separati
|
||||
`archive/app_data/navidrome` e `archive/app_data/syncthing`, `media`, `media/music`,
|
||||
`media/photobook`, `backups`, `backups/services` e `backup_prometheus`. I dataset applicativi e
|
||||
di archivio usano `zstd`; media, Syncthing e backup dei servizi usano `lz4`;
|
||||
`backups/services` mantiene inoltre una `refreservation` di `500G`.
|
||||
Atlas impone SELinux targeted in modo persistente e segnala, senza avviarlo, l’eventuale reboot necessario per attivarlo. Assegna esplicitamente l’interfaccia LAN primaria alla zona firewalld gestita e applica hardening persistente del kernel di rete: rifiuta redirect e source-route, registra i martian, usa reverse-path filtering loose per WireGuard e disabilita il forwarding IPv4. SSH consente solo l’amministratore dichiarato tramite chiave pubblica; root, password, agent e forwarding
|
||||
remoto sono disabilitati, mentre il forwarding locale resta disponibile per tunnel amministrativi privati. SMB3 pubblica `Archive` solo agli account Samba configurati con password in Vault e
|
||||
ammette la LAN configurata su SMB3 cifrato e firmato, esclusivamente su TCP/445. NFSv4 esporta soltanto
|
||||
`media/photobook` all'IP configurato di Aegis su TCP/2049, con `all_squash` verso UID/GID anonimi `1100`.
|
||||
|
||||
L'account di sistema `immich` usa UID/GID `1100`, non ha shell di login né gruppo `wheel` e riceve i
|
||||
gruppi `video` e `render`. Lo stack Immich futuro prevede Quadlet rootful per Server, ML, cache,
|
||||
PostgreSQL e NPM su una rete Podman comune. Immich gira come `1100:1100`, Server e ML ricevono
|
||||
`/dev/dri` e Photobook è montato in sola lettura su `/external/photobook`. NPM pubblica `80` e `443`;
|
||||
l'interfaccia amministrativa resta su `127.0.0.1:81`, raggiungibile via tunnel SSH.
|
||||
L'account di sistema `immich` usa UID/GID `1100`, shell senza login, nessuna appartenenza a `wheel` e
|
||||
i gruppi supplementari `video` e `render`. I Quadlet rootful di Immich Server, ML, cache compatibile
|
||||
Redis, PostgreSQL e NPM condividono una rete Podman. Immich viene eseguito come `1100:1100`; Server e
|
||||
ML ricevono `/dev/dri` e Photobook e montato in sola lettura su `/external/photobook`. NPM pubblica `80` e
|
||||
`443`, mentre l'amministrazione resta vincolata a `127.0.0.1:81` per l'accesso tramite tunnel SSH.
|
||||
|
||||
Atlas ospita temporaneamente Navidrome e Syncthing rootless fino alla sostituzione con Uranus. I
|
||||
servizi sono inizializzati **ex novo**, senza migrare lo stato precedente, rispettivamente sotto
|
||||
`/zpool/services/data/navidrome` e `/zpool/services/data/syncthing`; la musica in
|
||||
`/zpool/media/music` viene popolata separatamente. Sono vincolati all'indirizzo LAN di Atlas
|
||||
(`192.168.178.55`), mai a WireGuard. `wireguard_overlay` collega invece Prometheus (`10.0.0.1`)
|
||||
e Aegis (`10.0.0.2`): le chiavi private restano sui rispettivi host e Ansible scambia solo le pubbliche.
|
||||
Prometheus apre `51820/udp`; Aegis inoltra soltanto il traffico overlay→LAN dichiarato e applica
|
||||
source NAT, evitando interfacce VPN su Atlas/Uranus e route statiche sul router. Navidrome (`4533/tcp`)
|
||||
e la GUI Syncthing (`8384/tcp`) ammettono solo Aegis, mentre le porte native Syncthing sono limitate
|
||||
alla LAN. Dopo la verifica dei servizi, configurare manualmente i Proxy Host NPM verso
|
||||
`http://192.168.178.55:4533` e `http://192.168.178.55:8384`. Il peer Prometheus include la LAN
|
||||
negli `AllowedIPs`; aggiungere la VIP Uranus quando esisterà. Dopo il reload di firewalld, Ansible
|
||||
ricarica le reti Podman rootful di Prometheus per conservare DNS e connettività del proxy.
|
||||
La fase 1 e limitata ai Quadlet utente rootless di Navidrome e Syncthing su Atlas. E abilitata nella
|
||||
configurazione host di Atlas e puo essere impostata a `false` solo per una sospensione intenzionale. Navidrome ufficiale `0.63.2` usa il database SQLite sotto `/data` e
|
||||
non supporta `ND_DATABASE_URL` ne un backend PostgreSQL esterno. Il servizio obsoleto `navidromedb`
|
||||
e quindi rimosso da Prometheus invece di essere replicato su Atlas. Il ruolo deriva i percorsi dal
|
||||
pool `zpool`, montato in `/zpool`: musica in sola lettura da `/zpool/media/music`, stato
|
||||
applicativo Navidrome e `navidrome.db` in `/zpool/archive/app_data/navidrome` e dati Syncthing in
|
||||
`/zpool/archive/app_data/syncthing`. `profile_atlas` crea questi dataset quando
|
||||
`atlas_manage_storage` e attivo; il ruolo backend verifica i mountpoint esatti prima di avviare i
|
||||
container. Il ruolo backend non crea mai il pool. Il ruolo separato `wireguard_overlay`
|
||||
gestisce `wg0` tra Prometheus (`10.0.0.1`) e Atlas (`10.0.0.2`), genera una sola volta le chiavi
|
||||
private sui rispettivi host e scambia tramite Ansible soltanto quelle pubbliche. Solo Prometheus apre
|
||||
pubblicamente `51820/udp`. Le porte backend sono ammesse esclusivamente nella zona firewalld WireGuard.
|
||||
|
||||
Validare il gateway con:
|
||||
`backend_phase1_start_services` resta falso durante il trasferimento dello stato applicativo, quindi
|
||||
la prima esecuzione reale del backend genera i Quadlet senza creare un database Atlas vuoto. Dopo aver
|
||||
arrestato Navidrome su Prometheus, copiare l'intera directory `/opt/navidrome/data/` in
|
||||
`/zpool/archive/app_data/navidrome/`, preservando `navidrome.db` e gli eventuali file SQLite laterali.
|
||||
Impostare quindi questa variabile a vero e rieseguire il ruolo per abilitare e avviare Navidrome e
|
||||
Syncthing. Il playbook non copia e non elimina mai i dati applicativi.
|
||||
|
||||
Validare e generare i servizi Atlas con:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff
|
||||
```
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags storage
|
||||
|
||||
La prima esecuzione reale WireGuard deve includere entrambi i peer. Se Aegis ha appena installato il
|
||||
layer `wireguard-tools`, riavviarlo manualmente e rieseguire senza `--check`: il ruolo attende un
|
||||
handshake effettivo.
|
||||
|
||||
Gli snapshot ZFS ricorsivi coprono l'intero pool: 24 orari al minuto 05, 30 giornalieri alle 00:15,
|
||||
8 settimanali la domenica alle 01:00 e 12 mensili il primo giorno alle 02:00. La retention elimina
|
||||
solo gli snapshot con prefisso gestito `atlas-auto` e non esegue rollback. Lo scrub OpenZFS mensile è
|
||||
previsto la prima domenica alle 03:00; il timer settimanale incompatibile è disabilitato. Il primo
|
||||
snapshot orario ricorsivo è riuscito; la prima pulizia pianificata e il primo scrub schedulato
|
||||
richiedono ancora una verifica a runtime.
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff
|
||||
```
|
||||
ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard
|
||||
|
||||
Il backup Borg cifrato usa il sub-account Hetzner `u660064-sub1`, il repository relativo `./borg-data`
|
||||
e Borg remoto 1.4 su SSH porta 23. La chiave ED25519 del server è fissata; una chiave client dedicata
|
||||
appartiene all'account `borg`, bloccato e senza login, sudo o gruppi supplementari. La chiave privata
|
||||
resta in `/etc/atlas-borg`; la passphrase proviene da `vault_atlas_borg_passphrase` ed è resa in un
|
||||
file `0600`. Solo il wrapper root crea snapshot e mount; avvia il client come `borg` con il minimo
|
||||
accesso temporaneo in lettura, senza concedergli gestione ZFS o sudo.
|
||||
|
||||
Il backup giornaliero parte alle 04:30 con un ritardo casuale fino a 30 minuti. Crea uno snapshot ZFS
|
||||
ricorsivo temporaneo e ricostruisce tutti i dataset sotto `/zpool` in un albero di bind mount in sola
|
||||
lettura, per inserirli in un unico archivio coerente. Il wrapper smonta ricorsivamente l'albero privato;
|
||||
un helper `ExecStopPost` mirato rimuove eventuali mount dello snapshot nel namespace host e lo snapshot
|
||||
temporaneo dopo l'uscita del processo. Borg conserva 30 archivi giornalieri, 8 settimanali e 12
|
||||
mensili, poi compatta il repository. Il controllo completo di metadati e repository si svolge il 15
|
||||
di ogni mese alle 06:00. Le operazioni usano un lock comune, journal e retry systemd limitati. Le
|
||||
nuove esecuzioni riportano al massimo una riga di avanzamento al minuto: percentuale **stimata**,
|
||||
dataset, file elaborati e byte originali/compressi/deduplicati. Il denominatore è la somma dei
|
||||
`logicalreferenced` ZFS dello snapshot, non un totale Borg: può superare il 100% e non comprende
|
||||
retention, compattazione o controlli. Le righe di progresso non riportano i nomi dei file; eventuali
|
||||
warning possono farlo. Seguire il job con `sudo journalctl -fu atlas-borg-backup.service`; modifiche
|
||||
all'helper non cambiano un'esecuzione già avviata.
|
||||
|
||||
Attivazione iniziale esplicita:
|
||||
|
||||
1. Inserire una passphrase unica in `secrets/vault.yml` con `ansible-vault edit`.
|
||||
2. Generare e mostrare solo la chiave pubblica con
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags borg_key`.
|
||||
3. Installarla nel sub-account Hetzner, poi applicare con
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags packages,borg`.
|
||||
4. Copiare `secrets/recovery/atlas-borg-repokey.export` su un supporto davvero offline: la copia
|
||||
locale ignorata da Git non è di per sé un backup offline.
|
||||
|
||||
Il ruolo inizializza solo un repository `repokey` assente, non accetta password SSH né host key non
|
||||
fissate e non avvia manualmente il primo backup. Validazione:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff
|
||||
```
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff
|
||||
|
||||
L'attivazione iniziale è riuscita: backup e controllo del repository, restore completo in una
|
||||
directory temporanea confrontato con l'albero `Archive`, esportazione offline della chiave di recupero
|
||||
e pulizia di snapshot/mount temporanei. Il 2026-09-25 un test separato da snapshot ZFS giornaliero ha
|
||||
copiato un file di `/zpool/archive` in `/var/tmp`, verificando contenuto, proprietario, modalità,
|
||||
mtime e ACL POSIX; copia e mount temporanei sono stati rimossi senza interrompere Borg. Non è un test
|
||||
di ripristino dell'intero dataset.
|
||||
|
||||
Il backup USB offline è distribuito come **servizio solo manuale** (`atlas_manage_usb_backup: true`):
|
||||
Ansible non formatta, sblocca, monta né avvia automaticamente il disco. Il disco esistente è stato
|
||||
verificato in sola lettura il 2026-09-23: UUID LUKS `577b3c43-ea37-4611-81a9-39d555cdfbd4`,
|
||||
UUID ext4 interno `758e2d2e-a427-4797-aad9-39c3a9f17c7e`, mapper `zpool-backup`. All'ispezione
|
||||
era montato in `/mnt/zpool-backup`; il servizio richiede invece che il mapper **non sia montato** prima
|
||||
dell'avvio. Se serve, `systemd-ask-password` chiede interattivamente la passphrase LUKS tramite
|
||||
l'agente di `systemctl start` e la passa direttamente a `cryptsetup`, senza salvarla, esporla negli
|
||||
argomenti o memorizzarla nella cache. Lo script monta il disco privatamente, crea uno snapshot ZFS
|
||||
ricorsivo, copia tutti i dataset in `atlas/snapshots/<timestamp>/` con `rsync --link-dest`, verifica
|
||||
con un dry-run basato sui checksum, aggiorna atomicamente `atlas/latest`, smonta e chiude LUKS. Un
|
||||
errore non sostituisce `latest` né cancella versioni complete precedenti. Borg e USB possono operare
|
||||
contemporaneamente su snapshot distinti, ma la lettura concorrente può ridurre il throughput.
|
||||
|
||||
La copia USB conserva le ACL ma non gli attributi estesi generici, compreso `security.selinux`: la
|
||||
policy della destinazione deve ricreare le etichette dopo un restore. Per un percorso esplicito:
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags restorecon \
|
||||
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
|
||||
```
|
||||
|
||||
Il task accetta solo percorsi sotto la radice del pool Atlas, esegue `restorecon -RFv` solo su quelli
|
||||
indicati ed è altrimenti inattivo; non va lanciato sull'intero pool durante i run ordinari. Le vecchie
|
||||
versioni USB non vengono eliminate automaticamente senza una retention deliberata. Il controllo di
|
||||
capacità include il trasferimento stimato e una riserva libera di 10 GiB. Dopo un backup riuscito,
|
||||
scollegare fisicamente il disco per renderlo davvero offline.
|
||||
|
||||
Validare la configurazione senza avviare il backup e, separatamente, un eventuale relabel pianificato:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags usb_backup,usb_reminder --check --diff
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags restorecon --check \
|
||||
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1
|
||||
```
|
||||
|
||||
Prima dell'avvio manuale smontare in sicurezza `/mnt/zpool-backup`, se ancora montato. Con il mapper
|
||||
chiuso, `sudo systemctl start atlas-usb-backup.service` chiede la passphrase e avvia il backup; né la
|
||||
password LUKS né un keyfile vanno in Ansible. Seguire con
|
||||
`sudo journalctl -fu atlas-usb-backup.service`. **Non esiste un timer di backup USB.** Soltanto
|
||||
`atlas-usb-reminder.timer` è schedulato il primo sabato del mese alle 10:00 `Europe/Rome`: invia un
|
||||
promemoria al notifier 45Drives Houston, senza avviare il backup. Un test manuale ha prodotto una
|
||||
notifica in 45Drives Alerts, **non un'email**; il log conferma l'invio della notifica, non la consegna
|
||||
di posta. Il primo evento pianificato era il 2026-10-03 alle 10:00 CEST. Controllare timer e risultato
|
||||
con `systemctl list-timers atlas-usb-reminder.timer` e in 45Drives Alerts.
|
||||
Per il cutover, arrestare il vecchio Navidrome prima di copiare la sua directory dati, verificare
|
||||
l'ownership dell'account `admin` su Atlas e confermare la presenza del database SQLite copiato prima
|
||||
di impostare `backend_phase1_start_services: true` in `host_vars/atlas.yml`. Conservare i dati sorgente
|
||||
e il container legacy `navidromedb` fermo finche Navidrome su Atlas e una prova di restore non sono
|
||||
stati validati.
|
||||
|
||||
Il primo tentativo USB del 2026-09-23 fallì su `security.selinux` e, dopo l'interruzione, lasciò
|
||||
snapshot e mapper aperti. Applicato il filtro rsync, furono rimossi lo snapshot fallito, il mapper
|
||||
smontato e lo stato failed; non rimase una copia valida di quel tentativo. Un run del 2026-09-24
|
||||
pubblicò una versione verificata ma fallì nella distruzione dello snapshot a causa di mount
|
||||
`.zfs/snapshot` aperti nel namespace host. Dopo la pulizia non forzata, è stato aggiunto un helper
|
||||
`ExecStopPost` mirato e testato con uno snapshot usa-e-getta. Un run successivo del 2026-09-24 ha
|
||||
verificato i checksum, pubblicato la versione ed è terminato con successo: mapper chiuso, nessuno
|
||||
snapshot USB temporaneo e pool sano. Il 2026-09-25 un test di restore indipendente ha aperto il disco
|
||||
in sola lettura, montato ext4 con `ro,noload`, copiato un file di 5.707.945 byte da `atlas/latest` in
|
||||
una directory vuota sotto `/var/tmp` e confrontato contenuto, proprietario, modalità, dimensione,
|
||||
mtime e ACL POSIX. Il test ha rimosso copia e mount temporanei, chiuso LUKS e lasciato il pool sano
|
||||
mentre Borg continuava. È un test su file, non un esercizio completo di disaster recovery.
|
||||
|
||||
Il monitoraggio Atlas è eseguito ogni 30 minuti da `atlas-health-monitor.timer`. Sonde in sola
|
||||
lettura controllano stato/errori del pool e dei vdev, scrub/resilver, SMART dei quattro dischi del
|
||||
pool e dell'NVMe di sistema, temperature dei dischi e CPU, spazio di sistema/pool/snapshot, crescita
|
||||
di `zpool/backup` e quota Hetzner tramite `df -m` via SSH con l'account `borg` e la chiave fissata.
|
||||
La query remota non apre il repository Borg né il suo lock. Gli alert di crescita richiedono una
|
||||
baseline di circa 24 ore. Sono controllati anche attivazione e freschezza dei timer; hook systemd
|
||||
`OnFailure` segnalano errori di snapshot, scrub, Borg, USB, promemoria e monitoraggio. Il monitor non
|
||||
riavvia Borg; avvisa solo se un run supera 14 giorni. Soglie e percorsi stabili dei dischi sono nelle
|
||||
variabili host. Gli avvisi usano 45Drives Houston con deduplicazione; **la consegna email non è stata
|
||||
verificata**. Il controllo live del 2026-09-25 non ha trovato problemi; la notifica di prova è stata
|
||||
inviata e lo Storage Box risultava occupato al 22%. Dimensione dell'archivio Borg e deduplicazione
|
||||
dettagliata richiedono ancora la fine del backup in corso.
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags monitoring --check --diff
|
||||
sudo /usr/local/libexec/atlas-health-monitor --dry-run
|
||||
sudo journalctl -u atlas-health-monitor.service -n 100 --no-pager
|
||||
systemctl list-timers atlas-health-monitor.timer
|
||||
```
|
||||
|
||||
`--dry-run` non invia alert e non modifica lo stato del monitor. Un controllo reale si avvia con
|
||||
`sudo systemctl start atlas-health-monitor.service`, senza avviare servizi di backup. Per una prova
|
||||
etichettata di 45Drives Alerts usare
|
||||
`sudo /usr/local/libexec/atlas-health-monitor --test-notification`.
|
||||
|
||||
### Timer systemd di Atlas
|
||||
|
||||
Tutti i nove timer gestiti sono abilitati. Gli orari sono locali ad Atlas (`Europe/Rome`); Borg e
|
||||
monitoraggio aggiungono il ritardo casuale indicato. Tutti hanno `Persistent=true`: un evento perso
|
||||
viene recuperato quando il timer torna attivo.
|
||||
|
||||
| Timer | Pianificazione (`OnCalendar`) | Azione |
|
||||
| --- | --- | --- |
|
||||
| `atlas-zfs-snapshot-hourly.timer` | `*-*-* *:05:00` — ogni ora al minuto 05 | Snapshot ricorsivo orario e retention |
|
||||
| `atlas-zfs-snapshot-daily.timer` | `*-*-* 00:15:00` — ogni giorno alle 00:15 | Snapshot ricorsivo giornaliero e retention |
|
||||
| `atlas-zfs-snapshot-weekly.timer` | `Sun *-*-* 01:00:00` — domenica alle 01:00 | Snapshot ricorsivo settimanale e retention |
|
||||
| `atlas-zfs-snapshot-monthly.timer` | `*-*-01 02:00:00` — primo giorno del mese alle 02:00 | Snapshot ricorsivo mensile e retention |
|
||||
| `zfs-scrub-monthly@zpool.timer` | `Sun *-*-01..07 03:00:00` — prima domenica alle 03:00 | Scrub ZFS |
|
||||
| `atlas-borg-backup.timer` | `*-*-* 04:30:00` — ogni giorno alle 04:30, più 0–30 min casuali | Backup cifrato offsite |
|
||||
| `atlas-borg-check.timer` | `*-*-15 06:00:00` — giorno 15 alle 06:00, più 0–30 min casuali | Controllo repository Borg |
|
||||
| `atlas-usb-reminder.timer` | `Sat *-*-01..07 10:00:00 Europe/Rome` — primo sabato alle 10:00 | Solo promemoria 45Drives Alerts |
|
||||
| `atlas-health-monitor.timer` | `*:0/30` — ogni mezz'ora, più 0–5 min casuali | Controlli di salute in sola lettura |
|
||||
|
||||
`atlas-usb-backup.service` **non ha timer** e va avviato manualmente. Il timer del fornitore
|
||||
`zfs-scrub-weekly@zpool.timer` è disabilitato a favore dello scrub mensile. Il futuro pull del backup
|
||||
Prometheus non ha ancora un timer, perché non è implementato. Durante un backup Borg attivo,
|
||||
`systemctl list-timers` può mostrare `-` per il prossimo evento senza che il timer sia disabilitato.
|
||||
Per vedere la pianificazione corrente: `systemctl list-timers --all` su Atlas.
|
||||
|
||||
Nextcloud è previsto come servizio temporaneo su Atlas prima di Uranus, ma solo dopo la validazione
|
||||
della protezione dei dati: richiede storage applicativo, database e cache separati, segreti Vault,
|
||||
pubblicazione solo tramite NPM e Aegis, procedure di backup, aggiornamento e migrazione. Non
|
||||
distribuirlo prima di completare la checklist di protezione dei dati.
|
||||
|
||||
La destinazione futura per l'importazione foto iCloud è Atlas, non Aegis. Dopo la validazione dei
|
||||
backup, pianificare una migrazione esplicita di iCloudPD con foto sotto `/zpool/archive/Pictures` e
|
||||
stato applicativo/MFA fuori da `Archive`; testare permessi, SELinux, backup e restore prima del
|
||||
cutover. L'attuale iCloudPD su Aegis e l'export NFS Photobook restano configurati fino
|
||||
all'approvazione e alla verifica di questa migrazione separata. Anche il servizio Atlas sarà
|
||||
temporaneo in attesa di Uranus.
|
||||
|
||||
Il pull dei backup di Prometheus, la valutazione delle dimensioni degli archivi Borg e i test completi
|
||||
di disaster recovery restano da fare. Il backlog prioritizzato è in `AGENTS.md`.
|
||||
Restano da completare retention delle snapshot, topologia Syncthing, validazione WireGuard/firewall,
|
||||
pull di backup da Prometheus, backup cifrati con Borg su una Hetzner Storage Box, backup USB,
|
||||
monitoraggio e test di disaster recovery. Il backlog operativo dettagliato e in `AGENTS.md`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
359
README.md
359
README.md
@@ -63,31 +63,6 @@ nymph is allowed to break
|
||||
|
||||
`ikaros` is the stable personal Fedora/GNOME desktop. `nymph` is the laptop and gets the same shared desktop dotfiles while GNOME itself stays close to the Fedora defaults. The legacy `void` and `desktop` groups are compatibility parents; the main axes are `platform_*`, `role_*`, and `desktop_*`.
|
||||
|
||||
The official ChatGPT desktop RPM is enabled only on `ikaros` and `nymph`. The
|
||||
playbook configures OpenAI's signed RPM repository and imports its pinned RPM
|
||||
signing key before installation; subsequent updates are handled by DNF.
|
||||
|
||||
## Deferred planned node: Cerberus
|
||||
|
||||
`cerberus` is a **postponed** management-plane node, pending the physical setup
|
||||
of the office in the new house. It is not yet an inventory host and no role or
|
||||
playbook targets it.
|
||||
|
||||
The planned hardware is a Lenovo ThinkCentre M700 Tiny (Intel Core i3-6100T,
|
||||
8 GB RAM, and a 256 GB SSD) with native 1 Gbps Ethernet. It will share Ikaros'
|
||||
monitor and peripherals through a multi-input KVM switch, using a passive
|
||||
DisplayPort-to-HDMI cable for its video connection. Fedora Sericea, the
|
||||
immutable Fedora variant with the Sway Wayland compositor, is the intended
|
||||
operating system.
|
||||
|
||||
Cerberus will be an isolated management plane: Ansible will run from a
|
||||
dedicated Toolbox environment to provision the future `uranus` cluster, rather
|
||||
than from Ikaros or an unmanaged host. Its rootless Podman observability stack
|
||||
will run Grafana, Prometheus, and Loki. The local SSD is the hot tier and
|
||||
retains metrics and logs for 30 days; scheduled exports will place older
|
||||
historical data on an NFS-mounted Atlas dataset as the cold tier. The detailed,
|
||||
implementation-gated plan is maintained in `AGENTS.md`.
|
||||
|
||||
## Desktop profiles
|
||||
|
||||
- `ikaros`: stable Fedora Workstation + GNOME desktop.
|
||||
@@ -132,19 +107,21 @@ the Compose stack, update DNS, or perform a cutover.
|
||||
The server profile installs platform-specific packages, Podman and podman-compose, declared systemd
|
||||
services, and firewalld. The manually activated `podman-compose-server` unit contains the existing
|
||||
Nginx Proxy Manager and Gitea services. The desired Compose file no longer includes Navidrome,
|
||||
Syncthing, or the obsolete Navidrome PostgreSQL database; their temporary Atlas deployment is managed
|
||||
by `profile_backend_phase1`. Applying the profile does not stop or remove legacy containers and does
|
||||
not delete `/opt/postgres/data`.
|
||||
Syncthing, or the obsolete Navidrome PostgreSQL database. Navidrome and Syncthing belong to Atlas;
|
||||
official Navidrome uses SQLite instead. Applying the profile does not stop or remove legacy
|
||||
containers and does not delete `/opt/postgres/data`.
|
||||
|
||||
Firewalld enables SSH, Cockpit (`9090/tcp`), HTTP and HTTPS. Nginx Proxy Manager publishes only
|
||||
`80/tcp` and `443/tcp`; its administration interface is bound to `127.0.0.1:81` and can be reached
|
||||
from Ikaros or Nymph with the `npm-tunnel` Bash alias. Nextcloud remains disabled and the profile
|
||||
does not provision any `/srv/nextcloud` directories.
|
||||
|
||||
NPM remains managed only by `profile_server`. Its WireGuard peer is Aegis (`10.0.0.2`), which forwards
|
||||
selected requests to LAN addresses and source-NATs them so no static route is required on the router.
|
||||
Use an Atlas LAN address for any current NAS-backed upstream; when Uranus receives its VIP, add that VIP
|
||||
to Prometheus' Aegis peer `AllowedIPs` and declare the corresponding proxy target separately.
|
||||
The Atlas phase-one work does not change this NPM deployment or its persistent data. Once WireGuard
|
||||
and the Atlas services are active, configure the current NPM proxy hosts with Navidrome upstream
|
||||
`http://10.0.0.2:4533` and Syncthing GUI upstream `http://10.0.0.2:8384`. Only the Syncthing web GUI
|
||||
uses NPM; synchronization traffic remains on explicitly published native ports bound only to the Atlas
|
||||
WireGuard address. Configure both Syncthing authentication and an appropriate NPM access policy before
|
||||
publishing its GUI.
|
||||
|
||||
Server identity comes from `server_username`, `server_user_group`, and `server_user_home` in `ansible/inventory/group_vars/server.yml`. `server_username` defaults to `username`, but it can be overridden, for example:
|
||||
|
||||
@@ -212,19 +189,12 @@ ansible/bootstrap/generate-aegis-ign.sh --write IMAGE DEVICE
|
||||
The controller manages it remotely as `pi@aegis`; unlike local desktop profiles, Aegis is
|
||||
intentionally an SSH inventory target. `profile_aegis` manages rootful Podman Quadlets for AdGuard
|
||||
Home and iCloudPD, persistent data under `/var/lib`, the Podman auto-update timer, LAN-restricted
|
||||
firewalld rules, SSH key-only access for `pi`, the `nfs-utils` and `wireguard-tools` rpm-ostree layers,
|
||||
and `wake-ikaros`. `wireguard_overlay` makes Aegis the internal endpoint and LAN gateway for Prometheus:
|
||||
it enables persistent IPv4 forwarding, installs a scoped WireGuard-to-LAN firewalld policy, and source-NATs
|
||||
forwarded tunnel traffic so the router needs no static route. A new layered package deployment requires a manual reboot; the
|
||||
role reports this condition but never reboots Aegis automatically. Set the host-local
|
||||
`aegis_lan_subnet`, `aegis_adguard_web_port`, and `aegis_network_connection_uuid` values before
|
||||
applying it. The playbook permits
|
||||
firewalld rules, SSH key-only access for `pi`, and `wake-ikaros`. Set the host-local
|
||||
`aegis_lan_subnet` and `aegis_adguard_web_port` values before applying it. The playbook permits
|
||||
AdGuard Home HTTP on port `80`; the initial wizard port `3000` is intentionally unmanaged and must be
|
||||
opened and closed manually during initial setup. The profile disables the local systemd-resolved DNS
|
||||
stub and points `/etc/resolv.conf` to its full resolver data, freeing port 53 for AdGuard. LAN clients
|
||||
may use AdGuard on Aegis, while Aegis itself uses the independent upstream DNS declared by
|
||||
`aegis_host_dns_servers`; this prevents Greenboot from depending on the AdGuard container during
|
||||
startup. Reboot Aegis after changing its NetworkManager DNS profile. Define
|
||||
stub and points `/etc/resolv.conf` to its full resolver data, freeing port 53
|
||||
for AdGuard while retaining DNS learned from the router. Define
|
||||
`vault_aegis_icloudpd_apple_id` in Vault before applying it. iCloudPD still requires interactive MFA
|
||||
initialization after its first deployment.
|
||||
|
||||
@@ -239,21 +209,6 @@ ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit aegis --check --diff --ask-become-pass
|
||||
```
|
||||
|
||||
Apply only the independent host DNS configuration, then reboot Aegis manually:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit aegis --tags dns --ask-become-pass
|
||||
```
|
||||
|
||||
Layer the Aegis NFS and WireGuard client tools independently, then reboot Aegis manually when the role reports
|
||||
that the new deployment is ready:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit aegis --tags nfs --ask-become-pass
|
||||
```
|
||||
|
||||
## NAS
|
||||
|
||||
`atlas` is a Rocky Linux 9 NAS reached through SSH. Normally its pool already exists and the profile
|
||||
@@ -264,9 +219,12 @@ clients use NFSv4 and Windows/WSL clients use SMB; both are restricted to the co
|
||||
|
||||
For the first run, provide `vault_atlas_admin_password_hash`, `vault_atlas_samba_password`, and
|
||||
`vault_atlas_immich_db_password`. Bootstrap the host through its
|
||||
existing administrator. The explicit pool gate is safe to repeat: the role creates the RAIDZ2 pool only when
|
||||
it is absent. Atlas no longer participates in the WireGuard overlay; its old interface is retired manually only after
|
||||
Prometheus and Aegis have completed the replacement handshake.
|
||||
existing administrator:
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas \
|
||||
-e atlas_connection_username=<existing-admin>
|
||||
```
|
||||
|
||||
`vault_atlas_admin_password_hash` must be an `/etc/shadow`-compatible hash, not a clear-text
|
||||
Cockpit password. Subsequent runs use `atlas_admin_username`. Atlas declares storage, sharing, and its
|
||||
@@ -276,11 +234,11 @@ paths, and the Immich database secret are validated. Atlas reads its declared SS
|
||||
separate files below `~/.ssh/authorized_keys.d/`.
|
||||
|
||||
With storage management enabled, Atlas creates the complete dataset hierarchy below the existing or
|
||||
explicitly bootstrapped `zpool`: SMB-shared `archive`, private `services/data` with separate
|
||||
`services/data/navidrome` and `services/data/syncthing` application datasets, `media`, `media/music`,
|
||||
`media/photobook`, and `backup/hosts/prometheus`. Application/archive datasets use `zstd`, while media,
|
||||
Syncthing, and host-backup datasets use `lz4`; `backup` has a `500G` reservation covering its descendants.
|
||||
Atlas enforces targeted SELinux persistently and reports, without initiating, any reboot required to activate it. It assigns its primary LAN interface explicitly to the managed firewalld zone and applies persistent kernel network hardening: redirects and source routes are rejected, martians logged, reverse-path filtering remains loose for WireGuard, and IPv4 forwarding is disabled. SSH permits only the declared administrator using public-key authentication; root login, passwords,
|
||||
explicitly bootstrapped `zpool`: `work`, `archive`, `archive/app_data`, the separate `archive/app_data/navidrome` and
|
||||
`archive/app_data/syncthing` application datasets, `media`, `media/music`, `media/photobook`,
|
||||
`backups`, `backups/services`, and `backup_prometheus`. Application/archive datasets use `zstd`,
|
||||
while media, Syncthing and service-backup datasets use `lz4`; `backups/services` also has a `500G`
|
||||
refreservation. Atlas enforces targeted SELinux persistently and reports, without initiating, any reboot required to activate it. It assigns its primary LAN interface explicitly to the managed firewalld zone and applies persistent kernel network hardening: redirects and source routes are rejected, martians logged, reverse-path filtering remains loose for WireGuard, and IPv4 forwarding is disabled. SSH permits only the declared administrator using public-key authentication; root login, passwords,
|
||||
agent and remote forwarding are disabled, while local forwarding remains available for private administrative tunnels. SMB3 exposes `Archive` only to the configured Vault-backed
|
||||
Samba accounts on encrypted, signed SMB3 over TCP/445 only and admits the configured LAN without host-specific
|
||||
exclusions. NFSv4 exports only `media/photobook` to the configured Aegis IP over TCP/2049, using
|
||||
@@ -292,243 +250,51 @@ and NPM Quadlets share one Podman network. Immich runs as `1100:1100`; Server an
|
||||
and Photobook is mounted read-only at `/external/photobook`. NPM publishes ports `80` and `443`; its
|
||||
administration interface remains restricted to `127.0.0.1:81` for SSH-tunnel access.
|
||||
|
||||
Atlas temporarily hosts rootless Navidrome and Syncthing until Uranus replaces them. They bind only to
|
||||
Atlas' LAN address (`192.168.178.55`); WireGuard remains exclusively between Prometheus (`10.0.0.1`)
|
||||
and Aegis (`10.0.0.2`). Their state is initialized ex novo in `/zpool/services/data/navidrome` and
|
||||
`/zpool/services/data/syncthing`; no source application state is migrated. The music library at
|
||||
`/zpool/media/music` is populated separately.
|
||||
Phase 1 is limited to rootless Navidrome and Syncthing user Quadlets on Atlas. It is enabled in the
|
||||
Atlas host configuration and can be set to `false` only for a deliberate suspension. Official Navidrome `0.63.2` uses its SQLite database below `/data`; it does
|
||||
not support `ND_DATABASE_URL` or an external PostgreSQL backend. The obsolete `navidromedb` service
|
||||
was therefore removed from Prometheus instead of being reproduced on Atlas. The role derives all
|
||||
storage paths from the `zpool` mounted at `/zpool`: music is read-only at
|
||||
`/zpool/media/music`, Navidrome application state and `navidrome.db` are stored at
|
||||
`/zpool/archive/app_data/navidrome`, and Syncthing persists at
|
||||
`/zpool/archive/app_data/syncthing`. `profile_atlas` creates these datasets when
|
||||
`atlas_manage_storage` is enabled; the backend role verifies their exact mountpoints before starting
|
||||
containers. The backend role never creates the pool. The separate `wireguard_overlay` role manages `wg0`
|
||||
between Prometheus (`10.0.0.1`) and Atlas (`10.0.0.2`), generating private keys once
|
||||
on their respective hosts and exchanging only public keys through Ansible. Prometheus alone opens
|
||||
`51820/udp` publicly. Backend ports are admitted only in the WireGuard firewalld zone.
|
||||
|
||||
The separate `wireguard_overlay` role manages `wg0` between Prometheus (`10.0.0.1`) and Aegis
|
||||
(`10.0.0.2`), generating private keys once on their respective hosts and exchanging only public keys
|
||||
through Ansible. Prometheus alone opens `51820/udp`. Aegis forwards only the declared overlay-to-LAN
|
||||
traffic and source-NATs it, so Atlas and future Uranus nodes require neither a VPN interface nor a router
|
||||
static route. Atlas permits Navidrome (`4533/tcp`) and the Syncthing GUI (`8384/tcp`) only from Aegis;
|
||||
Syncthing native ports are limited to the LAN. Configure NPM manually with
|
||||
`http://192.168.178.55:4533` and `http://192.168.178.55:8384` after the services are healthy.
|
||||
Prometheus' peer includes the LAN subnet in `AllowedIPs`; add the Uranus VIP there when it exists.
|
||||
When the WireGuard zone is created, Ansible reloads firewalld and immediately reloads Prometheus'
|
||||
rootful Podman networks so the existing proxy stack retains container DNS and connectivity.
|
||||
`backend_phase1_start_services` stays false during the application-state transfer, so the first real
|
||||
backend run renders the Quadlets without creating an empty Atlas database. After stopping Navidrome
|
||||
on Prometheus, copy the complete `/opt/navidrome/data/` directory into
|
||||
`/zpool/archive/app_data/navidrome/`, preserving `navidrome.db` and any SQLite sidecar files. Then set
|
||||
this variable to true and rerun the role to enable and start Navidrome and Syncthing. The playbook
|
||||
never copies or deletes application data.
|
||||
|
||||
Validate the gateway with:
|
||||
Validate and render the Atlas services with:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff
|
||||
```
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags storage
|
||||
|
||||
The first real WireGuard run must include both peers. If Fedora IoT has just layered `wireguard-tools`,
|
||||
reboot Aegis manually and rerun the command without `--check`; the role then waits for a real peer
|
||||
handshake.
|
||||
|
||||
Atlas declares recursive, systemd-timed ZFS snapshots for the complete pool hierarchy: 24 hourly
|
||||
snapshots at minute 05, 30 daily snapshots at 00:15, 8 weekly snapshots on Sunday at 01:00, and 12
|
||||
monthly snapshots on the first day at 02:00. The retention helper prunes only snapshots carrying its
|
||||
managed `atlas-auto` prefix and never rolls back a dataset. The OpenZFS monthly scrub timer is scheduled
|
||||
for the first Sunday at 03:00; the conflicting weekly scrub timer is disabled explicitly. The first recursive
|
||||
hourly snapshot completed successfully on Atlas; retention pruning and the first scheduled scrub still await
|
||||
live runtime evidence. Validate this layer independently with:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff
|
||||
```
|
||||
ansible-playbook ansible/site.yml --limit prometheus,atlas --tags wireguard
|
||||
|
||||
Atlas also declares an encrypted Borg backup to the dedicated Hetzner Storage Box sub-account
|
||||
`u660064-sub1`. The repository is the sub-account-relative `./borg-data` path and uses the explicitly
|
||||
selected remote Borg 1.4 binary over SSH port 23. The ED25519 server key is pinned; a dedicated client
|
||||
key is generated for the locked, non-login `borg` system account, and its private half never leaves
|
||||
`/etc/atlas-borg`. The account has no sudo or supplementary groups and owns only its SSH identity,
|
||||
passphrase, cache, and Borg state. Borg receives its passphrase through a mode `0600` file rendered from
|
||||
`vault_atlas_borg_passphrase`.
|
||||
|
||||
The daily backup starts at 04:30 with up to 30 minutes of randomized delay. It creates a temporary,
|
||||
recursive ZFS snapshot and reconstructs every dataset below `/zpool` as a read-only bind-mounted tree,
|
||||
so parent and child datasets enter one consistent Borg archive. The wrapper recursively unmounts its
|
||||
private source tree; a narrowly scoped `ExecStopPost` helper removes any remaining host-namespace ZFS
|
||||
snapshot mounts and the named temporary snapshot after the backup process exits. Only the root wrapper
|
||||
performs snapshot and mount operations; it launches the Borg client as `borg` with temporary read-search
|
||||
capability and no ZFS, sudo, or pool-management privileges. Borg retains 30 daily, 8 weekly, and 12
|
||||
monthly archives, then compacts the standard
|
||||
read-write repository. A full metadata and repository check runs as `borg` on the fifteenth day of each
|
||||
month at 06:00. Both operations use a common lock, journal logging, and bounded systemd retries.
|
||||
New backup runs also log the create phase and a compact progress line at most once per minute: an
|
||||
**estimated** percentage, dataset, files processed, and original/compressed/deduplicated bytes. The
|
||||
denominator is the summed ZFS `logicalreferenced` size of the backup's own recursive snapshot, not a
|
||||
Borg-reported total: the estimate can exceed 100% and does not cover retention, compaction, or checks.
|
||||
Progress lines omit individual filenames; warnings may still name affected files.
|
||||
Follow the current run with
|
||||
`sudo journalctl -fu atlas-borg-backup.service` on Atlas; changes to the helper do not alter a run
|
||||
already in progress.
|
||||
|
||||
Initial activation remains explicit:
|
||||
|
||||
1. Add a strong unique `vault_atlas_borg_passphrase` with `ansible-vault edit secrets/vault.yml`.
|
||||
2. Generate and display only the dedicated public key with
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags borg_key`.
|
||||
3. Install that public key in the Hetzner sub-account, then apply with
|
||||
`ansible-playbook ansible/site.yml --limit atlas --tags packages,borg`.
|
||||
4. Copy the ignored `secrets/recovery/atlas-borg-repokey.export` file to genuinely offline storage.
|
||||
The controller-side copy is not an offline backup by itself.
|
||||
|
||||
The role initializes only the missing `repokey` repository and never accepts an unpinned host key or
|
||||
password authentication. It does not start the first backup manually. Validate the rendered state with:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff
|
||||
```
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1 --check --diff
|
||||
|
||||
Atlas runtime activation is complete: the initial backup and repository check succeeded, a full restore
|
||||
to a temporary directory was validated against the live `Archive` tree, the recovery-key export was copied
|
||||
to offline storage, and the temporary snapshot and bind mounts were cleaned up.
|
||||
On 2026-09-25 a separate ZFS restore smoke test copied a small file from an automatic daily
|
||||
`zpool/archive` snapshot to `/var/tmp`, then confirmed matching contents, ownership, mode, mtime and
|
||||
POSIX ACL. The temporary copy and on-demand snapshot mount were removed; Borg kept running. This
|
||||
does not validate a full dataset recovery.
|
||||
|
||||
The offline USB backup is deployed as a manual-only service (`atlas_manage_usb_backup: true`):
|
||||
Ansible never formats, unlocks, mounts, backs up to, or schedules the disk. Atlas' existing USB disk was verified
|
||||
read-only on 2026-09-23 as LUKS UUID `577b3c43-ea37-4611-81a9-39d555cdfbd4`, containing ext4 UUID
|
||||
`758e2d2e-a427-4797-aad9-39c3a9f17c7e` through mapper `zpool-backup`. It was mounted at
|
||||
`/mnt/zpool-backup` at inspection time. The service deliberately requires the verified mapper to be
|
||||
**not mounted** before starting. When necessary, `systemd-ask-password` requests the LUKS passphrase
|
||||
through the `systemctl start` password agent; it is piped directly to `cryptsetup` without saving it,
|
||||
passing it as a command argument, or caching it. The service then mounts the disk privately, takes a recursive ZFS snapshot,
|
||||
copies every dataset to a versioned `atlas/snapshots/<timestamp>/` directory using `rsync --link-dest`,
|
||||
verifies the result with a checksum-based dry run, atomically updates `atlas/latest`, unmounts and closes
|
||||
LUKS. A failed run never replaces `latest` or removes an earlier complete version. Borg and the USB
|
||||
backup may run concurrently from separate snapshots; both reading the same pool can reduce throughput.
|
||||
The USB copy preserves ACLs but not generic extended attributes; `security.selinux` is also intentionally
|
||||
excluded because the target SELinux policy must recreate labels during a restore. Do not restore data into
|
||||
service paths without relabeling. After restoring an explicit dataset path, apply its destination policy with:
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags restorecon \
|
||||
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
|
||||
```
|
||||
|
||||
The task accepts only paths below the Atlas pool mount root, runs `restorecon -RFv` only for the paths
|
||||
provided at invocation, and is otherwise a no-op. It must not be used on the whole pool during routine runs.
|
||||
Old USB versions are not pruned automatically, to avoid deleting the only offline
|
||||
copy without an explicitly chosen retention policy; capacity checks include an estimated transfer size
|
||||
and a 10 GiB free-space reserve. The disk must be physically disconnected after a successful backup
|
||||
to make the copy offline.
|
||||
|
||||
To check the USB backup and reminder configuration without starting a backup, run:
|
||||
|
||||
```bash
|
||||
ANSIBLE_LOCAL_TEMP=/tmp/ansible-local \
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags usb_backup,usb_reminder --check --diff
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags backend_phase1
|
||||
```
|
||||
|
||||
To validate a planned, explicit post-restore relabel operation without changing labels, run:
|
||||
For the cutover, stop the old Navidrome writer before copying its data directory, verify ownership by
|
||||
the Atlas `admin` account and confirm that the copied SQLite database is present before changing
|
||||
`backend_phase1_start_services: true` in `host_vars/atlas.yml`. Keep the source data and the stopped
|
||||
legacy `navidromedb` container until Navidrome on Atlas and a restore test have been validated.
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags restorecon --check \
|
||||
-e '{"atlas_restorecon_paths":["/zpool/archive"]}'
|
||||
```
|
||||
|
||||
Before the first **manual** service start, safely unmount the currently mounted
|
||||
`/mnt/zpool-backup`; never run it on an arbitrary mounted disk. Future starts
|
||||
can begin with the mapper closed: `sudo systemctl start atlas-usb-backup.service` prompts for the
|
||||
passphrase interactively and then performs the backup. Neither the LUKS password nor a key file belongs
|
||||
in Ansible. Inspect the run with
|
||||
`sudo journalctl -fu atlas-usb-backup.service`. There is intentionally no timer. Independently test a
|
||||
read-only mount and restore from `atlas/latest` into an empty temporary directory before marking the
|
||||
USB recovery path complete. Only `atlas-usb-reminder.timer` is enabled, for the first Saturday of each
|
||||
month at 10:00 Europe/Rome. Its warning notification uses the existing 45Drives Houston notifier.
|
||||
A manual test confirmed a notification in 45Drives Alerts, **not** an email. The reminder service log
|
||||
reports notification submission, not email delivery; the role does not depend on SMTP/OAuth settings.
|
||||
The reminder never starts the backup. Check its schedule with
|
||||
`systemctl list-timers atlas-usb-reminder.timer` and the result in 45Drives Alerts.
|
||||
The timer was verified active with its first scheduled run at 2026-10-03 10:00 CEST. No email
|
||||
delivery is claimed.
|
||||
The first manual USB attempt on 2026-09-23 did not complete: rsync was denied while removing
|
||||
`security.selinux` on the USB filesystem, then the interrupted service left its recursive
|
||||
`atlas-usb-20260923T185748Z-2469168` snapshot and the `zpool-backup` LUKS mapper open. The
|
||||
rsync xattr filter was deployed afterward. The incomplete USB directory was absent on inspection;
|
||||
the exact failed snapshot was removed, the verified and unmounted mapper closed, and the service
|
||||
failed state cleared. A final check found no remnant snapshot, mount, mapper, or staging directory.
|
||||
The failed attempt was not a valid backup, and no USB restore had been tested at that point.
|
||||
On 2026-09-24 a later run reported a checksum-verified, published USB version and closed the LUKS
|
||||
mapper, but the service failed while destroying its temporary ZFS snapshot: OpenZFS still had
|
||||
on-demand `.zfs/snapshot` mounts open in the host namespace. Those exact temporary snapshots were
|
||||
unmounted normally and removed; no force or rollback was used. The backup service now records its
|
||||
snapshot name and runs a narrowly scoped `ExecStopPost` cleanup after the private backup process
|
||||
exits. The cleanup helper was tested with a disposable recursive snapshot and an active snapshot
|
||||
mount. A complete run on 2026-09-24 later checksum-verified and published a new USB version; the
|
||||
service ended successfully, the LUKS mapper closed, no temporary USB snapshot remained, and the pool
|
||||
was healthy. On 2026-09-25 an independent restore test opened the configured USB disk read-only, mounted
|
||||
ext4 with `ro,noload`, restored a 5,707,945-byte file from the published `atlas/latest` version to an
|
||||
empty `/var/tmp` directory, and matched its content, owner, mode, size, mtime, and POSIX ACL against
|
||||
the USB source. The test removed its temporary copy and mount, closed the LUKS mapper, and left the
|
||||
pool healthy while Borg continued running. This is a file-level recovery smoke test, not a full dataset
|
||||
or disaster-recovery exercise.
|
||||
|
||||
Atlas health monitoring runs every 30 minutes through `atlas-health-monitor.timer`. Its read-only probes
|
||||
check pool/vdev state and errors, scrub/resilver status, four pool disks and the system NVMe via SMART,
|
||||
disk and CPU temperatures, system/pool/snapshot space, local `zpool/backup` growth, and the Hetzner
|
||||
Storage Box quota via `df -m` over the dedicated `borg` account's pinned-key SSH connection. The remote
|
||||
query never opens the Borg repository or its lock. Growth alerts compare against a roughly 24-hour
|
||||
baseline and therefore begin only after enough samples exist. The monitor also checks
|
||||
maintenance/backup timer activation and freshness; systemd `OnFailure` hooks report snapshot,
|
||||
scrub, Borg, USB, reminder, and monitoring services when they enter the failed state. The ongoing
|
||||
initial Borg run is never restarted by the monitor; only a run exceeding 14 days raises a warning.
|
||||
Thresholds and stable disk paths are declared in Atlas host variables. Alerts use the existing 45Drives
|
||||
Houston notifier and repeated issues are deduplicated; **email delivery is not verified**. The
|
||||
2026-09-25 live probe found no issues and a labelled test notification was submitted. The Storage Box
|
||||
reported 22% used. Detailed Borg archive size and deduplication still require the active run to finish.
|
||||
|
||||
```bash
|
||||
ansible-playbook ansible/site.yml --limit atlas --tags monitoring --check --diff
|
||||
sudo /usr/local/libexec/atlas-health-monitor --dry-run
|
||||
sudo journalctl -u atlas-health-monitor.service -n 100 --no-pager
|
||||
systemctl list-timers atlas-health-monitor.timer
|
||||
```
|
||||
|
||||
`--dry-run` sends no alerts and does not change monitor state. A real check is
|
||||
`sudo systemctl start atlas-health-monitor.service`; do not start the backup services merely to test
|
||||
monitoring. For a labelled 45Drives Alerts delivery test, use
|
||||
`sudo /usr/local/libexec/atlas-health-monitor --test-notification`.
|
||||
|
||||
### Atlas systemd timers
|
||||
|
||||
All nine managed timers below are enabled. Times are local to Atlas (`Europe/Rome`); Borg and monitoring
|
||||
add the indicated randomized delay. Every timer has `Persistent=true`, so a missed calendar run is
|
||||
scheduled after the timer becomes active again.
|
||||
|
||||
| Timer | Schedule (`OnCalendar`) | Action |
|
||||
| --- | --- | --- |
|
||||
| `atlas-zfs-snapshot-hourly.timer` | `*-*-* *:05:00` — every hour at :05 | Recursive hourly snapshot and retention |
|
||||
| `atlas-zfs-snapshot-daily.timer` | `*-*-* 00:15:00` — daily at 00:15 | Recursive daily snapshot and retention |
|
||||
| `atlas-zfs-snapshot-weekly.timer` | `Sun *-*-* 01:00:00` — Sunday at 01:00 | Recursive weekly snapshot and retention |
|
||||
| `atlas-zfs-snapshot-monthly.timer` | `*-*-01 02:00:00` — first day of the month at 02:00 | Recursive monthly snapshot and retention |
|
||||
| `zfs-scrub-monthly@zpool.timer` | `Sun *-*-01..07 03:00:00` — first Sunday at 03:00 | ZFS scrub |
|
||||
| `atlas-borg-backup.timer` | `*-*-* 04:30:00` — daily at 04:30, plus 0–30 min random delay | Encrypted offsite backup |
|
||||
| `atlas-borg-check.timer` | `*-*-15 06:00:00` — 15th of the month at 06:00, plus 0–30 min random delay | Borg repository check |
|
||||
| `atlas-usb-reminder.timer` | `Sat *-*-01..07 10:00:00 Europe/Rome` — first Saturday at 10:00 | 45Drives Alerts reminder only |
|
||||
| `atlas-health-monitor.timer` | `*:0/30` — every half-hour, plus 0–5 min random delay | Read-only health checks |
|
||||
|
||||
`atlas-usb-backup.service` has **no timer**: the encrypted USB backup must be started manually.
|
||||
The vendor's `zfs-scrub-weekly@zpool.timer` is intentionally disabled in favor of the monthly scrub.
|
||||
The future Prometheus backup pull has no timer yet because that workflow is not implemented. While a
|
||||
Borg backup is still running, `systemctl list-timers` may show `-` for its next trigger; this does not
|
||||
mean the timer has been disabled. Inspect the current schedule on Atlas with
|
||||
`systemctl list-timers --all`.
|
||||
|
||||
A temporary Nextcloud deployment on Atlas is also planned before Uranus: it requires separately
|
||||
declared persistent application, database, and cache storage, Vault-backed credentials, NPM-only
|
||||
publishing through Aegis, and defined backup, upgrade, and eventual migration procedures. Do not deploy
|
||||
it before the data-protection checklist is complete.
|
||||
|
||||
The desired future iCloud photo-ingestion host is Atlas, not Aegis. After data-protection validation,
|
||||
plan an explicit iCloudPD migration with photos under `/zpool/archive/Pictures` and application/MFA
|
||||
state outside `Archive`, then test permissions, SELinux, backups and recovery before cutting over.
|
||||
The current Aegis iCloudPD service and Atlas Photobook NFS export remain configured until that
|
||||
separate migration is approved and validated; the eventual Atlas service is temporary until Uranus.
|
||||
|
||||
Prometheus backup pulls, Borg archive-size evaluation, and full disaster-recovery tests remain follow-up
|
||||
work. The prioritized operational backlog is kept in `AGENTS.md`.
|
||||
Snapshot retention, Syncthing topology, WireGuard/firewall validation, Prometheus backup pulls,
|
||||
encrypted Borg backups to a Hetzner Storage Box, USB backup, monitoring, and disaster-recovery tests
|
||||
remain follow-up work. The detailed operational backlog is kept in `AGENTS.md`.
|
||||
|
||||
## How layering works
|
||||
|
||||
@@ -561,10 +327,9 @@ ansible-playbook ansible/site.yml --limit <host> --tags emacs -e emacs_enabled=t
|
||||
## AI coding agents
|
||||
|
||||
The shared npm-managed agents are OpenCode, Claude Code, Codex, Gemini CLI, and
|
||||
GitHub Copilot; IBM Bob is also managed on `deadalus`. Codex Relay is installed
|
||||
only on `ikaros`. Each agent has its own lifecycle flags in
|
||||
`ansible/inventory/group_vars/all.yml`, so one agent can be installed,
|
||||
configured, or removed without affecting the others:
|
||||
GitHub Copilot; IBM Bob is also managed on `deadalus`. Each agent has its own
|
||||
lifecycle flags in `ansible/inventory/group_vars/all.yml`, so one agent can be
|
||||
installed, configured, or removed without affecting the others:
|
||||
|
||||
```yaml
|
||||
ai_agents:
|
||||
@@ -615,8 +380,8 @@ ansible-playbook ansible/site.yml --limit deadalus --tags ai_agents --check --di
|
||||
| `profile_workstation_dev_wsl` | WSL development setup. |
|
||||
| `profile_server` | Server setup. |
|
||||
| `profile_atlas` | Rocky Linux 9 NAS setup. |
|
||||
| `profile_backend_phase1` | Temporary rootless Atlas Navidrome and Syncthing services. |
|
||||
| `wireguard_overlay` | Prometheus/Aegis WireGuard LAN gateway. |
|
||||
| `profile_backend_phase1` | Rootless Navidrome and Syncthing on Atlas. |
|
||||
| `wireguard_overlay` | Prometheus/Atlas WireGuard overlay. |
|
||||
| `profile_aegis` | Fedora IoT always-on LAN node. |
|
||||
| `dotfiles_common` | Shared user dotfiles. |
|
||||
|
||||
@@ -628,8 +393,8 @@ platform_void -> packages_void + services_runit
|
||||
platform_void & graphical_desktop -> profile_desktop_common + profile_desktop_sway + profile_desktop_niri + profile_desktop_host
|
||||
platform_fedora -> packages_fedora + services_systemd
|
||||
platform_rocky -> packages_rocky + services_systemd
|
||||
wireguard_overlay -> wireguard_overlay (after platform_rocky)
|
||||
role_aegis -> profile_aegis
|
||||
wireguard_overlay -> wireguard_overlay (after Aegis profile and platform_rocky)
|
||||
atlas -> profile_atlas
|
||||
role_backend_phase1 -> profile_backend_phase1 (after atlas)
|
||||
rocky_server -> dotfiles_common + profile_server (after platform_rocky)
|
||||
@@ -734,7 +499,7 @@ ansible-playbook ansible/site.yml --list-tags
|
||||
| `sharing` | Atlas NFSv4 and SMB3 configuration. |
|
||||
| `storage` | Atlas child ZFS datasets. |
|
||||
| `tmux` | tmux configuration and plugins. |
|
||||
| `wireguard` | Prometheus/Aegis WireGuard LAN gateway. |
|
||||
| `wireguard` | Prometheus/Atlas WireGuard overlay. |
|
||||
| `wsl` | WSL bootstrap and configuration. |
|
||||
|
||||
## Bootstrapping a new machine
|
||||
|
||||
@@ -93,8 +93,6 @@ common_dotfiles:
|
||||
# AI coding-agent lifecycle controls are independent for every agent.
|
||||
# Installation and removal are mutually exclusive per agent; the common
|
||||
# pre-task enforces this before any changes are made.
|
||||
codex_relay_install_enabled: false
|
||||
|
||||
ai_agents:
|
||||
opencode:
|
||||
npm_package: opencode-ai
|
||||
@@ -111,11 +109,6 @@ ai_agents:
|
||||
install_enabled: true
|
||||
deploy_enabled: true
|
||||
uninstall_enabled: false
|
||||
codex_relay:
|
||||
npm_package: codex-relay
|
||||
install_enabled: "{{ codex_relay_install_enabled }}"
|
||||
deploy_enabled: false
|
||||
uninstall_enabled: false
|
||||
gemini_cli:
|
||||
npm_package: "@google/gemini-cli"
|
||||
install_enabled: true
|
||||
|
||||
@@ -9,7 +9,6 @@ fedora_packages_base:
|
||||
- jq
|
||||
- make
|
||||
- nodejs
|
||||
- openssl
|
||||
- ripgrep
|
||||
|
||||
fedora_manage_docker_repo: true
|
||||
@@ -59,13 +58,3 @@ fedora_dev_packages:
|
||||
|
||||
fedora_flatpak_remote_name: flathub
|
||||
fedora_flatpak_remote_url: https://dl.flathub.org/repo/flathub.flatpakrepo
|
||||
|
||||
chatgpt_desktop_enabled: false
|
||||
chatgpt_desktop_supported_fedora_versions:
|
||||
- "43"
|
||||
- "44"
|
||||
chatgpt_desktop_supported_architectures:
|
||||
- x86_64
|
||||
- aarch64
|
||||
chatgpt_desktop_rpm_gpg_key: RPM-GPG-KEY-chatgpt-3BFA0E4AE8B8CC16A2D9BA684A3B4A566C4660E4.asc
|
||||
chatgpt_desktop_rpm_repository: https://persistent.oaistatic.com/codex-app-prod/linux/rpm/$basearch
|
||||
|
||||
@@ -5,9 +5,6 @@ platform_package_manager: dnf
|
||||
platform_service_manager: systemd
|
||||
|
||||
rocky_openzfs_release_rpm: https://zfsonlinux.org/epel/zfs-release-3-0.el9.noarch.rpm
|
||||
rocky_openzfs_gpg_key_url: >-
|
||||
https://raw.githubusercontent.com/zfsonlinux/zfsonlinux.github.com/master/zfs-release/RPM-GPG-KEY-openzfs-key2
|
||||
rocky_openzfs_gpg_key_fingerprint: 7DC7 299D CF7C 7FD9 CD87 701B A599 FD5E 9DB8 4141
|
||||
rocky_syncthing_version: 2.1.3
|
||||
rocky_syncthing_archive_checksum: sha256:f929eb8e5b72a85543eeeefb2c38f34a68e0c530e70758a2905b78840c76602c
|
||||
rocky_syncthing_archive_url: >-
|
||||
|
||||
@@ -40,3 +40,4 @@ server_firewalld_services:
|
||||
server_firewalld_ports: []
|
||||
server_sshd_service_name: sshd
|
||||
server_compose_selinux_mount_option: Z
|
||||
server_syncthing_enabled: false
|
||||
|
||||
@@ -75,6 +75,21 @@ server_directories:
|
||||
group: root
|
||||
mode: "0755"
|
||||
|
||||
server_syncthing_enabled: true
|
||||
server_syncthing_directories:
|
||||
- path: /opt/syncthing/config
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
- path: /srv/syncthing
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
- path: /srv/syncthing/data
|
||||
owner: "1000"
|
||||
group: "1000"
|
||||
mode: "0755"
|
||||
|
||||
server_sshd_settings:
|
||||
PermitRootLogin: "no"
|
||||
|
||||
|
||||
@@ -3,19 +3,14 @@ enabled_services:
|
||||
- docker
|
||||
|
||||
workstation_dev_wsl_packages:
|
||||
- buildah
|
||||
- helm
|
||||
# Fedora ships the kubectl client binary in the kubernetes-client package.
|
||||
- kubernetes-client
|
||||
- mise
|
||||
- pinentry-curses
|
||||
- python3-pip
|
||||
- tmux
|
||||
|
||||
# Java 11, Java 25 and Maven are managed by Mise on this Fedora WSL profile.
|
||||
# Keep their versions pinned; update them deliberately.
|
||||
# Java 11 and Maven are managed by Mise on this Fedora WSL profile. Keep their
|
||||
# versions pinned; update them deliberately.
|
||||
workstation_mise_java_version: temurin-11.0.31+11
|
||||
workstation_mise_java_25_version: 25.0.2
|
||||
workstation_mise_maven_version: 3.9.16
|
||||
|
||||
workstation_is_wsl: true
|
||||
|
||||
@@ -9,32 +9,6 @@ ansible_ssh_use_tty: false
|
||||
|
||||
aegis_lan_subnet: 192.168.178.0/24
|
||||
aegis_adguard_web_port: 80
|
||||
aegis_network_connection_name: FRITZ!Box 7530 NR
|
||||
aegis_network_connection_uuid: a52fda3d-3eb6-351f-bf04-753edcb76944
|
||||
aegis_host_dns_servers:
|
||||
- 192.168.178.1
|
||||
aegis_host_dns_search_domains:
|
||||
- fritz.box
|
||||
|
||||
# Aegis is the internal WireGuard endpoint and NAT gateway for the LAN.
|
||||
wireguard_overlay_enabled: true
|
||||
wireguard_address: 10.0.0.2/24
|
||||
wireguard_enable_ipv4_forwarding: true
|
||||
wireguard_reload_rootful_podman_networks: true
|
||||
wireguard_forwarding_policies:
|
||||
- name: aegis-wireguard-to-lan
|
||||
ingress_zone: wireguard
|
||||
egress_zone: public
|
||||
source: 10.0.0.0/24
|
||||
destination: "{{ aegis_lan_subnet }}"
|
||||
masquerade: true
|
||||
wireguard_peers:
|
||||
- name: prometheus
|
||||
host: prometheus
|
||||
endpoint: "{{ hostvars['prometheus']['ansible_host'] }}:{{ hostvars['prometheus']['wireguard_listen_port'] }}"
|
||||
allowed_ips:
|
||||
- 10.0.0.0/24
|
||||
persistent_keepalive: 25
|
||||
aegis_ssh_authorized_keys:
|
||||
- name: ikaros
|
||||
key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINrIxXjA3ffPwziKGR5gzc4gAoBehQPlnEMcXF4Wl0ZS ikaros"
|
||||
|
||||
@@ -28,119 +28,38 @@ atlas_manage_firewall: true
|
||||
atlas_firewalld_zone: public
|
||||
atlas_zfs_pool: zpool
|
||||
# Populate only for the first pool bootstrap with four real persistent disk paths.
|
||||
# Confirmed empty 4 TB IronWolf data disks; the NVMe system disk is intentionally excluded.
|
||||
atlas_zpool_disks:
|
||||
- /dev/disk/by-id/ata-ST4000VN006-3CW104_WW6AC1XM
|
||||
- /dev/disk/by-id/ata-ST4000VN006-3CW104_WW6A73T2
|
||||
- /dev/disk/by-id/ata-ST4000VN006-3CW104_WW6A6VJK
|
||||
- /dev/disk/by-id/ata-ST4000VN006-3CW104_WW6AC1LM
|
||||
# Example: /dev/disk/by-id/ata-Seagate_IronWolf_...
|
||||
atlas_zpool_disks: []
|
||||
atlas_create_pool: false
|
||||
atlas_zfs_dataset_work: work
|
||||
atlas_zfs_dataset_archive: archive
|
||||
atlas_zfs_dataset_services: services
|
||||
atlas_zfs_dataset_app_data: services/data
|
||||
atlas_zfs_dataset_navidrome: services/data/navidrome
|
||||
atlas_zfs_dataset_syncthing: services/data/syncthing
|
||||
atlas_zfs_dataset_app_data: archive/app_data
|
||||
atlas_zfs_dataset_navidrome: archive/app_data/navidrome
|
||||
atlas_zfs_dataset_syncthing: archive/app_data/syncthing
|
||||
atlas_zfs_dataset_media: media
|
||||
atlas_zfs_dataset_music: media/music
|
||||
atlas_zfs_dataset_backup: backup
|
||||
atlas_zfs_dataset_host_backups: backup/hosts
|
||||
atlas_zfs_dataset_backup_prometheus: backup/hosts/prometheus
|
||||
atlas_zfs_backup_reservation: 500G
|
||||
atlas_zfs_dataset_backup_prometheus: backup_prometheus
|
||||
atlas_zfs_dataset_photobook: media/photobook
|
||||
atlas_zfs_dataset_backups: backups
|
||||
atlas_zfs_dataset_service_backups: backups/services
|
||||
atlas_mount_root: /zpool
|
||||
atlas_manage_storage: true
|
||||
atlas_manage_zfs_snapshots: true
|
||||
atlas_zfs_snapshot_prefix: atlas-auto
|
||||
atlas_zfs_snapshot_policies:
|
||||
- name: hourly
|
||||
calendar: "*-*-* *:05:00"
|
||||
keep: 24
|
||||
- name: daily
|
||||
calendar: "*-*-* 00:15:00"
|
||||
keep: 30
|
||||
- name: weekly
|
||||
calendar: "Sun *-*-* 01:00:00"
|
||||
keep: 8
|
||||
- name: monthly
|
||||
calendar: "*-*-01 02:00:00"
|
||||
keep: 12
|
||||
atlas_manage_zfs_scrub: true
|
||||
atlas_zfs_scrub_calendar: "Sun *-*-01..07 03:00:00"
|
||||
atlas_manage_borg_backup: true
|
||||
atlas_borg_repository_host: u660064-sub1.your-storagebox.de
|
||||
atlas_borg_repository_user: u660064-sub1
|
||||
atlas_borg_repository_port: 23
|
||||
atlas_borg_repository_path: ./borg-data
|
||||
atlas_borg_remote_path: borg-1.4
|
||||
# Verified against Hetzner's published ED25519 fingerprint on 2026-09-17:
|
||||
# SHA256:XqONwb1S0zuj5A1CDxpOSuD2hnAArV1A3wKY7Z3sdgM
|
||||
atlas_borg_host_key: >-
|
||||
[u660064-sub1.your-storagebox.de]:23 ssh-ed25519
|
||||
AAAAC3NzaC1lZDI1NTE5AAAAIICf9svRenC/PLKIL9nk6K/pxQgoiFC41wTNvoIncOxs
|
||||
atlas_borg_backup_calendar: "*-*-* 04:30:00"
|
||||
atlas_borg_check_calendar: "*-*-15 06:00:00"
|
||||
atlas_borg_randomized_delay: 30m
|
||||
atlas_borg_keep_daily: 30
|
||||
atlas_borg_keep_weekly: 8
|
||||
atlas_borg_keep_monthly: 12
|
||||
atlas_manage_usb_backup: true
|
||||
# Read-only lsblk verification on Atlas, 2026-09-23. Never store the LUKS password here.
|
||||
atlas_usb_backup_luks_uuid: 577b3c43-ea37-4611-81a9-39d555cdfbd4
|
||||
atlas_usb_backup_fs_uuid: 758e2d2e-a427-4797-aad9-39c3a9f17c7e
|
||||
atlas_usb_backup_mapper_name: zpool-backup
|
||||
atlas_manage_usb_reminder: true
|
||||
atlas_usb_reminder_calendar: "Sat *-*-01..07 10:00:00 Europe/Rome"
|
||||
atlas_manage_monitoring: true
|
||||
# Physical pool disks and the system NVMe; the disconnected USB disk is intentionally excluded.
|
||||
atlas_monitor_smart_devices:
|
||||
- { name: pool-1, path: "{{ atlas_zpool_disks[0] }}", warning_c: 50, critical_c: 55 }
|
||||
- { name: pool-2, path: "{{ atlas_zpool_disks[1] }}", warning_c: 50, critical_c: 55 }
|
||||
- { name: pool-3, path: "{{ atlas_zpool_disks[2] }}", warning_c: 50, critical_c: 55 }
|
||||
- { name: pool-4, path: "{{ atlas_zpool_disks[3] }}", warning_c: 50, critical_c: 55 }
|
||||
- name: system-nvme
|
||||
path: /dev/disk/by-id/nvme-Patriot_M.2_P320_256GB_P320ADB26011606111
|
||||
warning_c: 70
|
||||
critical_c: 85
|
||||
atlas_monitor_timers:
|
||||
- { name: atlas-zfs-snapshot-hourly.timer, max_age_hours: 3 }
|
||||
- { name: atlas-zfs-snapshot-daily.timer, max_age_hours: 36 }
|
||||
- { name: atlas-zfs-snapshot-weekly.timer, max_age_hours: 216 }
|
||||
- { name: atlas-zfs-snapshot-monthly.timer, max_age_hours: 960 }
|
||||
- { name: zfs-scrub-monthly@zpool.timer, max_age_hours: 960 }
|
||||
- { name: atlas-borg-backup.timer, max_age_hours: 48 }
|
||||
- { name: atlas-borg-check.timer, max_age_hours: 960 }
|
||||
# The first manual USB reminder is not due until October; activation is checked, not age.
|
||||
- { name: atlas-usb-reminder.timer, max_age_hours: 0 }
|
||||
atlas_monitor_failure_units:
|
||||
- atlas-zfs-snapshot@.service
|
||||
- zfs-scrub@zpool.service
|
||||
- atlas-borg-backup.service
|
||||
- atlas-borg-check.service
|
||||
- atlas-usb-backup.service
|
||||
- atlas-usb-reminder.service
|
||||
- atlas-health-monitor.service
|
||||
atlas_monitor_remote_capacity:
|
||||
user: "{{ atlas_borg_repository_user }}"
|
||||
host: "{{ atlas_borg_repository_host }}"
|
||||
run_as: "{{ atlas_borg_username }}"
|
||||
ssh_wrapper: "{{ atlas_borg_ssh_wrapper_path }}"
|
||||
warning_percent: 80
|
||||
critical_percent: 90
|
||||
growth_warning_gib_day: 500
|
||||
atlas_manage_sharing: true
|
||||
atlas_manage_media_stack: false
|
||||
# Planned after data-protection validation: move iCloudPD photo ingestion from
|
||||
# Aegis to Atlas, with photos under /zpool/archive/Pictures and persistent
|
||||
# application/MFA state outside Archive. Do not deploy or cut over yet.
|
||||
|
||||
# WireGuard is retired on Atlas. These rootless services are a temporary home
|
||||
# until Uranus replaces them.
|
||||
wireguard_overlay_enabled: true
|
||||
wireguard_address: 10.0.0.2/24
|
||||
wireguard_peers:
|
||||
- name: prometheus
|
||||
host: prometheus
|
||||
endpoint: "{{ hostvars['prometheus']['ansible_host'] }}:{{ hostvars['prometheus']['wireguard_listen_port'] }}"
|
||||
allowed_ips:
|
||||
- 10.0.0.0/24
|
||||
persistent_keepalive: 25
|
||||
|
||||
backend_phase1_enabled: true
|
||||
backend_phase1_start_services: true
|
||||
backend_phase1_bind_address: "{{ ansible_host }}"
|
||||
backend_phase1_firewalld_zone: "{{ atlas_firewalld_zone }}"
|
||||
backend_phase1_npm_source_ip: "{{ atlas_aegis_ip }}"
|
||||
backend_phase1_syncthing_native_subnet: "{{ atlas_lan_subnet }}"
|
||||
backend_phase1_start_services: false
|
||||
backend_phase1_wireguard_address: 10.0.0.2
|
||||
|
||||
rocky_manage_openzfs_repo: true
|
||||
rocky_manage_syncthing_binary: false
|
||||
@@ -150,21 +69,13 @@ rocky_podman_packages:
|
||||
|
||||
host_packages:
|
||||
- cockpit
|
||||
- cockpit-podman
|
||||
- cockpit-storaged
|
||||
- realmd
|
||||
- pcp
|
||||
- python3-pcp
|
||||
- cryptsetup
|
||||
- nfs-utils
|
||||
- policycoreutils
|
||||
- policycoreutils-python-utils
|
||||
- python3-libselinux
|
||||
- setroubleshoot-server
|
||||
- samba
|
||||
- samba-client
|
||||
- samba-common-tools
|
||||
- borgbackup
|
||||
- zfs
|
||||
|
||||
atlas_nfs_exports:
|
||||
@@ -198,5 +109,4 @@ atlas_firewalld_rich_rules:
|
||||
host_enabled_services:
|
||||
- sshd
|
||||
- cockpit.socket
|
||||
- pmlogger.service
|
||||
- zfs.target
|
||||
|
||||
@@ -1,9 +1,6 @@
|
||||
---
|
||||
hostname: ikaros
|
||||
|
||||
codex_relay_install_enabled: true
|
||||
chatgpt_desktop_enabled: true
|
||||
|
||||
host_packages:
|
||||
- openssh-clients
|
||||
- openssh-server
|
||||
@@ -232,7 +229,7 @@ gnome_gsettings_settings:
|
||||
- schema: org.gnome.shell
|
||||
key: app-picker-layout
|
||||
value: >-
|
||||
[{'System': <{'position': <0>}>, 'Utilities': <{'position': <1>}>, 'org.fedoraproject.MediaWriter.desktop': <{'position': <2>}>, 'org.gnome.Contacts.desktop': <{'position': <3>}>, 'org.gnome.Calendar.desktop': <{'position': <4>}>, 'org.gnome.Calculator.desktop': <{'position': <5>}>, 'org.gnome.Weather.desktop': <{'position': <6>}>, 'org.gnome.clocks.desktop': <{'position': <7>}>, 'org.gnome.Maps.desktop': <{'position': <8>}>, 'org.gnome.SimpleScan.desktop': <{'position': <9>}>, 'org.gnome.TextEditor.desktop': <{'position': <10>}>, 'org.gnome.Settings.desktop': <{'position': <11>}>, 'org.gnome.Boxes.desktop': <{'position': <12>}>, 'org.gnome.Showtime.desktop': <{'position': <13>}>, 'org.gnome.Snapshot.desktop': <{'position': <14>}>, 'org.gnome.Characters.desktop': <{'position': <15>}>, 'org.gnome.Tour.desktop': <{'position': <16>}>, 'org.gnome.Yelp.desktop': <{'position': <17>}>, 'org.gnome.Software.desktop': <{'position': <18>}>, 'be.alexandervanhee.gradia.desktop': <{'position': <19>}>, 'com.github.jeromerobert.pdfarranger.desktop': <{'position': <20>}>, 'org.onlyoffice.desktopeditors.desktop': <{'position': <21>}>, 'org.gnome.Meld.desktop': <{'position': <22>}>, 'com.mattjakeman.ExtensionManager.desktop': <{'position': <23>}>}, {'org.remmina.Remmina.desktop': <{'position': <0>}>, 'ch.protonmail.protonmail-bridge.desktop': <{'position': <1>}>, 'syncthing-start.desktop': <{'position': <2>}>, 'syncthing-ui.desktop': <{'position': <3>}>, 'emacs.desktop': <{'position': <4>}>, 'emacs-client.desktop': <{'position': <5>}>, 'org.gnome.gitlab.somas.Apostrophe.desktop': <{'position': <6>}>, 'mpv.desktop': <{'position': <7>}>, 'htop.desktop': <{'position': <8>}>, 'com.raspberrypi.rpi-imager.desktop': <{'position': <9>}>}]
|
||||
[{'System': <{'position': <0>}>, 'Utilities': <{'position': <1>}>, 'org.fedoraproject.MediaWriter.desktop': <{'position': <2>}>, 'org.gnome.Contacts.desktop': <{'position': <3>}>, 'org.gnome.Calendar.desktop': <{'position': <4>}>, 'org.gnome.Calculator.desktop': <{'position': <5>}>, 'org.gnome.Weather.desktop': <{'position': <6>}>, 'org.gnome.clocks.desktop': <{'position': <7>}>, 'org.gnome.Maps.desktop': <{'position': <8>}>, 'org.gnome.SimpleScan.desktop': <{'position': <9>}>, 'org.gnome.TextEditor.desktop': <{'position': <10>}>, 'org.gnome.Settings.desktop': <{'position': <11>}>, 'org.gnome.Boxes.desktop': <{'position': <12>}>, 'org.gnome.Showtime.desktop': <{'position': <13>}>, 'org.gnome.Snapshot.desktop': <{'position': <14>}>, 'org.gnome.Characters.desktop': <{'position': <15>}>, 'org.gnome.Tour.desktop': <{'position': <16>}>, 'org.gnome.Yelp.desktop': <{'position': <17>}>, 'org.gnome.Software.desktop': <{'position': <18>}>, 'be.alexandervanhee.gradia.desktop': <{'position': <19>}>, 'com.github.jeromerobert.pdfarranger.desktop': <{'position': <20>}>, 'org.onlyoffice.desktopeditors.desktop': <{'position': <21>}>, 'org.gnome.Meld.desktop': <{'position': <22>}>, 'com.mattjakeman.ExtensionManager.desktop': <{'position': <23>}>}, {'org.remmina.Remmina.desktop': <{'position': <0>}>, 'ch.protonmail.protonmail-bridge.desktop': <{'position': <1>}>, 'syncthing-start.desktop': <{'position': <2>}>, 'syncthing-ui.desktop': <{'position': <3>}>, 'emacs.desktop': <{'position': <4>}>, 'emacs-client.desktop': <{'position': <5>}>, 'org.gnome.gitlab.somas.Apostrophe.desktop': <{'position': <6>}>, 'mpv.desktop': <{'position': <7>}>, 'htop.desktop': <{'position': <8>}>, 'org.mozilla.thunderbird.desktop': <{'position': <9>}>}]
|
||||
- schema: org.gnome.shell
|
||||
key: disabled-extensions
|
||||
value: >-
|
||||
@@ -244,7 +241,7 @@ gnome_gsettings_settings:
|
||||
- schema: org.gnome.shell
|
||||
key: favorite-apps
|
||||
value: >-
|
||||
['org.gnome.Nautilus.desktop', 'org.mozilla.firefox.desktop', 'com.spotify.Client.desktop', 'org.gnome.Evolution.desktop', 'org.telegram.desktop.desktop', 'chatgpt.desktop', 'org.gnome.Ptyxis.desktop', 'virt-manager.desktop']
|
||||
['org.gnome.Nautilus.desktop', 'org.mozilla.firefox.desktop', 'com.spotify.Client.desktop', 'org.gnome.Evolution.desktop', 'org.telegram.desktop.desktop', 'org.gnome.Ptyxis.desktop', 'virt-manager.desktop']
|
||||
- schema: org.gnome.shell
|
||||
key: remember-mount-password
|
||||
value: "true"
|
||||
|
||||
@@ -1,8 +1,6 @@
|
||||
---
|
||||
hostname: nymph
|
||||
|
||||
chatgpt_desktop_enabled: true
|
||||
|
||||
# Fedora Workstation/GNOME laptop target. GNOME is provided by the base Fedora
|
||||
# Workstation install; Ansible deploys shared desktop dotfiles but leaves GNOME
|
||||
# settings unmanaged for now.
|
||||
|
||||
@@ -19,25 +19,16 @@ wireguard_overlay_enabled: true
|
||||
wireguard_address: 10.0.0.1/24
|
||||
wireguard_listen_port: 51820
|
||||
wireguard_enable_ipv4_forwarding: true
|
||||
wireguard_reload_rootful_podman_networks: true
|
||||
wireguard_peers:
|
||||
- name: aegis
|
||||
host: aegis
|
||||
# TODO: Aggiungere VIP di Uranus in futuro
|
||||
- name: atlas
|
||||
host: atlas
|
||||
allowed_ips:
|
||||
- 10.0.0.2/32
|
||||
- 192.168.178.0/24
|
||||
|
||||
host_packages:
|
||||
- cockpit
|
||||
- cockpit-navigator
|
||||
- cockpit-podman
|
||||
- cockpit-storaged
|
||||
- realmd
|
||||
- pcp
|
||||
- python3-pcp
|
||||
- setroubleshoot-server
|
||||
|
||||
host_enabled_services:
|
||||
- cockpit.socket
|
||||
- pmlogger.service
|
||||
|
||||
@@ -99,7 +99,7 @@ all:
|
||||
wireguard_overlay:
|
||||
hosts:
|
||||
prometheus:
|
||||
aegis:
|
||||
atlas:
|
||||
|
||||
role_backend_phase1:
|
||||
hosts:
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
-----BEGIN PGP PUBLIC KEY BLOCK-----
|
||||
|
||||
mQINBGpypFUBEACi1Vvzq9pIpA6lj7chbqELuxJtVuzUzxrasa6ZU0yF4yhq7jf8
|
||||
3YkJRHwbezBKeQyzJ5lkX0EhXS8aXxUhMAm3PFpAlwcInfKzmV7atJwvaxIw6Rmd
|
||||
GYe9fBWKjTN/SmPIjtyxrTznZY97+TfD1AeGZpLaJ8fsnhrC+HkiN2TACiTocgpe
|
||||
hFiP0OWK7mWZeTWnY2scpIYXP1Ro7nQv4KacmY4JacTQ7m/HM0Qej/3olhuEv2Cw
|
||||
lMVWw57/oHhmTllfLDQOogFQyIVqaaR98y/Eu6cAabSfcsqAAZ2A8vfHYD27z28J
|
||||
vLO2PZEJd5ThlnX4Zqv0eIpZdBj//8Sl/MSqTshFZ1NDsRoqwdqw284X5MpnOJ4k
|
||||
4Sc2Se8tJxt/nCeibH3dJ504Fb1X/mnOqhCAQ6pVJz4RB5HRlFPSkxVPyag1v1m/
|
||||
7T4vie+OR4eqFQNz6mudrOoMmeVIfyL5fbe4cOr4fk/FyvEE2xMgkFatPqXn7vM9
|
||||
og+zremPCfwRAFpBPyX74VowFY7llcdaj/w8K5T8PzM14Hb3E4ZKizMluKmTvTq9
|
||||
WE1/eSQJLLQqXD5VmtmdUaC/VyE/1ZlIxcA1LWqvEQ327UXREvX/nHsrkKrl956W
|
||||
jzkiHFUTsD1NJ0dMfs+csOt8Furb5jZj+HsMmCm9jLdfz5b/4WKLPbvxIwARAQAB
|
||||
tBZDb2RleCBMaW51eCBSZXBvc2l0b3J5iQJRBBMBCgA7FiEEO/oOSui4zBai2bpo
|
||||
SjtKVmxGYOQFAmpypFUCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQ
|
||||
SjtKVmxGYORlCQ/9FyikZo8HQcJBP9E/oXVPds/fQnIFB2qJR2z3DrfYEonNt/ev
|
||||
SAySkPPq4/mEOjaI0pFlDDGSaps+FTcJFgoVRTasBIF7JJivvjW9ap8iWEbhhVLe
|
||||
IrFLbMLpUcTRntUx7R4fVMJ/1/cGn+NWZmNwS9ORorzSyCH0IAgCw1Xc3ZrjuMbF
|
||||
VjdToMC1TiXXCEmlYpQakmQ3Ay1cH0FHC2BBNn1MNVkJdPhpZIZCdhaMPHfYFpyo
|
||||
pg8wFvZ5iIcvlbMgyuy8CPJVRWUcYy2dOhEOGnYJnXRPkE3E1hf8YOHNzRlduH89
|
||||
6lT9qcEK2+fpLfrVGoc4zscLZ+Ey+Ko6iQRdVE1j67+wNR3hX8ukue574v1N/xxu
|
||||
i575jumSE19lEj1sH4+P4gFHOtTbF0JhKKzLctbga0IAwTPKhnt3qzj1U5Yj/MZS
|
||||
uEVjrLhdRauOuFBXUclgyVf2w/lE85UUOdlcollsYA6Huq7xDamqf8SslZQGre3E
|
||||
I+lhpqJR1cOwDMUzzcl40uTyhrxXXd/bk4QSlhZbwHR25Pnt+ZMtWavlQWS0eDEV
|
||||
8djuXAURCmx5WOqAFB/TJe1mn5EvyWg4VFzrY/NVNOpzgY5+Xp7J28z7f637r712
|
||||
Eu9j4imVcdPigwS+jf/0f81i2o9b82Y26TN8+EtDLCY841MJ1lrjDrX/dno=
|
||||
=Y+3h
|
||||
-----END PGP PUBLIC KEY BLOCK-----
|
||||
@@ -130,56 +130,6 @@
|
||||
loop_control:
|
||||
label: "{{ item.key }}"
|
||||
|
||||
- name: Validate ChatGPT desktop architecture on Fedora
|
||||
tags: [packages, chatgpt]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- ansible_facts['architecture'] in chatgpt_desktop_supported_architectures
|
||||
- ansible_facts['distribution_major_version'] in chatgpt_desktop_supported_fedora_versions
|
||||
fail_msg: >-
|
||||
ChatGPT desktop supports Fedora {{ chatgpt_desktop_supported_fedora_versions | join(' and ') }}
|
||||
on x86_64 or aarch64; detected Fedora {{ ansible_facts['distribution_major_version'] }}
|
||||
on {{ ansible_facts['architecture'] }}.
|
||||
when: chatgpt_desktop_enabled | bool
|
||||
|
||||
- name: Install ChatGPT desktop RPM signing key
|
||||
tags: [packages, chatgpt]
|
||||
ansible.builtin.copy:
|
||||
src: "{{ chatgpt_desktop_rpm_gpg_key }}"
|
||||
dest: "/etc/pki/rpm-gpg/{{ chatgpt_desktop_rpm_gpg_key }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
when: chatgpt_desktop_enabled | bool
|
||||
|
||||
- name: Import ChatGPT desktop RPM signing key
|
||||
tags: [packages, chatgpt]
|
||||
ansible.builtin.rpm_key:
|
||||
state: present
|
||||
key: "/etc/pki/rpm-gpg/{{ chatgpt_desktop_rpm_gpg_key }}"
|
||||
when: chatgpt_desktop_enabled | bool
|
||||
|
||||
- name: Configure ChatGPT desktop RPM repository
|
||||
tags: [packages, chatgpt]
|
||||
ansible.builtin.yum_repository:
|
||||
name: openai-chatgpt
|
||||
description: ChatGPT
|
||||
baseurl: "{{ chatgpt_desktop_rpm_repository }}"
|
||||
enabled: true
|
||||
gpgcheck: true
|
||||
repo_gpgcheck: true
|
||||
gpgkey: "file:///etc/pki/rpm-gpg/{{ chatgpt_desktop_rpm_gpg_key }}"
|
||||
state: present
|
||||
when: chatgpt_desktop_enabled | bool
|
||||
|
||||
- name: Install ChatGPT desktop on selected Fedora hosts
|
||||
tags: [packages, chatgpt]
|
||||
ansible.builtin.dnf:
|
||||
name: chatgpt
|
||||
state: latest
|
||||
update_cache: true
|
||||
when: chatgpt_desktop_enabled | bool
|
||||
|
||||
- name: Define Fedora Flatpak applications
|
||||
tags: [packages]
|
||||
ansible.builtin.set_fact:
|
||||
|
||||
@@ -27,14 +27,6 @@
|
||||
name: epel-release
|
||||
state: present
|
||||
|
||||
- name: Import official OpenZFS EL9+ signing key
|
||||
tags: [packages, storage]
|
||||
ansible.builtin.rpm_key:
|
||||
state: present
|
||||
key: "{{ rocky_openzfs_gpg_key_url }}"
|
||||
fingerprint: "{{ rocky_openzfs_gpg_key_fingerprint }}"
|
||||
when: rocky_manage_openzfs_repo | bool
|
||||
|
||||
- name: Install official OpenZFS repository package
|
||||
tags: [packages, storage]
|
||||
ansible.builtin.dnf:
|
||||
|
||||
@@ -1,12 +1,5 @@
|
||||
---
|
||||
aegis_hostname: aegis
|
||||
aegis_layered_packages:
|
||||
- nfs-utils
|
||||
- wireguard-tools
|
||||
aegis_network_connection_name: ""
|
||||
aegis_network_connection_uuid: ""
|
||||
aegis_host_dns_servers: []
|
||||
aegis_host_dns_search_domains: []
|
||||
aegis_adguard_image: docker.io/adguard/adguardhome:latest
|
||||
aegis_icloudpd_image: docker.io/boredazfcuk/icloudpd:latest
|
||||
aegis_icloudpd_folder_structure: '{:%Y/%m/%d}'
|
||||
|
||||
@@ -1,18 +1,4 @@
|
||||
---
|
||||
- name: Manage Aegis layered packages
|
||||
tags: [aegis, packages, nfs, wireguard]
|
||||
community.general.rpm_ostree_pkg:
|
||||
name: "{{ aegis_layered_packages }}"
|
||||
state: present
|
||||
register: aegis_layered_packages_result
|
||||
when: aegis_layered_packages | length > 0
|
||||
|
||||
- name: Report reboot required for Aegis layered packages
|
||||
tags: [aegis, packages, nfs, wireguard]
|
||||
ansible.builtin.debug:
|
||||
msg: Reboot Aegis to activate the newly layered packages, then rerun the playbook.
|
||||
when: aegis_layered_packages_result.needs_reboot | default(false)
|
||||
|
||||
- name: Require Aegis iCloudPD Apple ID
|
||||
tags: [aegis, icloudpd]
|
||||
ansible.builtin.assert:
|
||||
@@ -22,86 +8,15 @@
|
||||
no_log: true
|
||||
|
||||
- name: Require completed Aegis network placeholders
|
||||
tags: [aegis, dns, firewall, network, services]
|
||||
tags: [aegis, firewall, services]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- aegis_lan_subnet != 'CHANGEME_LAN_SUBNET'
|
||||
- aegis_firewalld_zone | length > 0
|
||||
- aegis_adguard_web_port | int > 0
|
||||
- aegis_adguard_web_port | int < 65536
|
||||
- aegis_network_connection_name | length > 0
|
||||
- aegis_network_connection_uuid | length > 0
|
||||
- aegis_host_dns_servers | length > 0
|
||||
- ansible_facts["default_ipv4"]["address"] not in aegis_host_dns_servers
|
||||
- aegis_ssh_allowed_users | length > 0
|
||||
fail_msg: >-
|
||||
Define the Aegis LAN subnet, firewalld zone, AdGuard web port, independent host DNS,
|
||||
NetworkManager connection, and SSH users. Aegis must not use its own address as upstream DNS.
|
||||
|
||||
- name: Verify the declared Aegis NetworkManager connection exists
|
||||
tags: [aegis, dns, network, services]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- nmcli
|
||||
- --get-values
|
||||
- connection.id
|
||||
- connection
|
||||
- show
|
||||
- uuid
|
||||
- "{{ aegis_network_connection_uuid }}"
|
||||
register: aegis_network_connection
|
||||
changed_when: false
|
||||
failed_when: >-
|
||||
aegis_network_connection.rc != 0
|
||||
or aegis_network_connection.stdout != aegis_network_connection_name
|
||||
|
||||
- name: Read the current Aegis host DNS configuration
|
||||
tags: [aegis, dns, network, services]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- nmcli
|
||||
- --get-values
|
||||
- ipv4.ignore-auto-dns,ipv4.dns,ipv4.dns-search,ipv6.ignore-auto-dns
|
||||
- connection
|
||||
- show
|
||||
- uuid
|
||||
- "{{ aegis_network_connection_uuid }}"
|
||||
register: aegis_host_dns_current
|
||||
changed_when: false
|
||||
|
||||
- name: Configure independent upstream DNS for the Aegis host
|
||||
tags: [aegis, dns, network, services]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- nmcli
|
||||
- connection
|
||||
- modify
|
||||
- uuid
|
||||
- "{{ aegis_network_connection_uuid }}"
|
||||
- ipv4.ignore-auto-dns
|
||||
- "yes"
|
||||
- ipv4.dns
|
||||
- "{{ aegis_host_dns_servers | join(',') }}"
|
||||
- ipv4.dns-search
|
||||
- "{{ aegis_host_dns_search_domains | join(',') }}"
|
||||
- ipv6.ignore-auto-dns
|
||||
- "yes"
|
||||
register: aegis_host_dns_profile
|
||||
when: >-
|
||||
aegis_host_dns_current.stdout_lines !=
|
||||
[
|
||||
'yes',
|
||||
aegis_host_dns_servers | join(','),
|
||||
aegis_host_dns_search_domains | join(','),
|
||||
'yes'
|
||||
]
|
||||
changed_when: true
|
||||
|
||||
- name: Report Aegis reboot required for host DNS changes
|
||||
tags: [aegis, dns, network, services]
|
||||
ansible.builtin.debug:
|
||||
msg: Reboot Aegis to activate its independent upstream DNS before testing another OS update.
|
||||
when: aegis_host_dns_profile.changed | default(false)
|
||||
fail_msg: Define the Aegis LAN subnet, firewalld zone, AdGuard web port, and SSH users.
|
||||
|
||||
- name: Set Aegis hostname
|
||||
tags: [aegis, services]
|
||||
|
||||
@@ -47,101 +47,31 @@ atlas_hardening_sysctls:
|
||||
net.ipv6.conf.all.accept_source_route: "0"
|
||||
net.ipv6.conf.default.accept_source_route: "0"
|
||||
atlas_zfs_pool: CHANGEME_ZFS_POOL
|
||||
atlas_zfs_dataset_work: work
|
||||
atlas_zfs_dataset_archive: archive
|
||||
atlas_zfs_dataset_services: services
|
||||
atlas_zfs_dataset_app_data: services/data
|
||||
atlas_zfs_dataset_navidrome: services/data/navidrome
|
||||
atlas_zfs_dataset_syncthing: services/data/syncthing
|
||||
atlas_zfs_dataset_app_data: archive/app_data
|
||||
atlas_zfs_dataset_navidrome: archive/app_data/navidrome
|
||||
atlas_zfs_dataset_syncthing: archive/app_data/syncthing
|
||||
atlas_zfs_dataset_media: media
|
||||
atlas_zfs_dataset_music: media/music
|
||||
atlas_zfs_dataset_backup: backup
|
||||
atlas_zfs_dataset_host_backups: backup/hosts
|
||||
atlas_zfs_dataset_backup_prometheus: backup/hosts/prometheus
|
||||
atlas_zfs_backup_reservation: 500G
|
||||
atlas_zfs_dataset_backup_prometheus: backup_prometheus
|
||||
atlas_zfs_dataset_photobook: media/photobook
|
||||
atlas_zfs_dataset_backups: backups
|
||||
atlas_zfs_dataset_service_backups: backups/services
|
||||
atlas_zfs_service_backups_refreservation: 500G
|
||||
atlas_mount_root: /CHANGEME_ATLAS_MOUNT_ROOT
|
||||
|
||||
atlas_manage_zfs_snapshots: false
|
||||
atlas_zfs_snapshot_prefix: atlas-auto
|
||||
atlas_zfs_snapshot_policies: []
|
||||
atlas_manage_zfs_scrub: false
|
||||
atlas_zfs_scrub_calendar: ""
|
||||
|
||||
atlas_manage_borg_backup: false
|
||||
atlas_borg_username: borg
|
||||
atlas_borg_group: borg
|
||||
atlas_borg_home: /var/lib/atlas-borg
|
||||
atlas_borg_repository_host: CHANGEME_BORG_HOST
|
||||
atlas_borg_repository_user: CHANGEME_BORG_USER
|
||||
atlas_borg_repository_port: 23
|
||||
atlas_borg_repository_path: ./borg-data
|
||||
atlas_borg_remote_path: borg-1.4
|
||||
atlas_borg_host_key: ""
|
||||
atlas_borg_ssh_private_key_path: /etc/atlas-borg/id_ed25519
|
||||
atlas_borg_known_hosts_path: /etc/atlas-borg/known_hosts
|
||||
atlas_borg_passphrase_path: /etc/atlas-borg/passphrase
|
||||
atlas_borg_ssh_wrapper_path: /usr/local/libexec/atlas-borg-ssh
|
||||
atlas_borg_passphrase: "{{ vault_atlas_borg_passphrase | default('') }}"
|
||||
atlas_borg_encryption_mode: repokey
|
||||
atlas_borg_archive_prefix: atlas
|
||||
atlas_borg_snapshot_prefix: atlas-borg
|
||||
atlas_borg_compression: auto,zstd,3
|
||||
atlas_borg_backup_calendar: ""
|
||||
atlas_borg_check_calendar: ""
|
||||
atlas_borg_randomized_delay: 30m
|
||||
atlas_borg_keep_daily: 30
|
||||
atlas_borg_keep_weekly: 8
|
||||
atlas_borg_keep_monthly: 12
|
||||
atlas_borg_config_dir: /var/lib/atlas-borg
|
||||
atlas_borg_cache_dir: /var/cache/atlas-borg
|
||||
atlas_borg_lock_path: /var/lib/atlas-borg/backup.lock
|
||||
atlas_borg_recovery_export_path: "{{ playbook_dir }}/../secrets/recovery/atlas-borg-repokey.export"
|
||||
|
||||
# Manual-only offline backup. No USB device is formatted or mounted by Ansible.
|
||||
atlas_manage_usb_backup: false
|
||||
atlas_usb_backup_luks_uuid: ""
|
||||
atlas_usb_backup_fs_uuid: ""
|
||||
atlas_usb_backup_mapper_name: atlas-usb-backup
|
||||
atlas_usb_backup_min_free_bytes: 10737418240
|
||||
atlas_usb_backup_snapshot_prefix: atlas-usb
|
||||
atlas_manage_usb_reminder: false
|
||||
atlas_usb_reminder_calendar: ""
|
||||
atlas_usb_reminder_notifier: /opt/45drives/houston/houston-notify
|
||||
|
||||
# Read-only health probes and 45Drives Alerts; disabled outside Atlas host vars.
|
||||
atlas_manage_monitoring: false
|
||||
atlas_monitor_calendar: "*:0/30"
|
||||
atlas_monitor_notifier: "{{ atlas_usb_reminder_notifier }}"
|
||||
atlas_monitor_smart_devices: []
|
||||
atlas_monitor_timers: []
|
||||
atlas_monitor_failure_units: []
|
||||
atlas_monitor_remote_capacity: {}
|
||||
atlas_monitor_pool_warning_percent: 80
|
||||
atlas_monitor_pool_critical_percent: 90
|
||||
atlas_monitor_root_warning_percent: 80
|
||||
atlas_monitor_root_critical_percent: 90
|
||||
atlas_monitor_snapshot_warning_percent: 10
|
||||
atlas_monitor_snapshot_critical_percent: 20
|
||||
atlas_monitor_snapshot_growth_warning_gib_day: 100
|
||||
atlas_monitor_backup_growth_warning_gib_day: 100
|
||||
atlas_monitor_cpu_warning_c: 85
|
||||
atlas_monitor_cpu_critical_c: 95
|
||||
atlas_monitor_borg_max_runtime_days: 14
|
||||
|
||||
# Explicit post-restore relabeling only; never relabel datasets during ordinary runs.
|
||||
atlas_restorecon_paths: []
|
||||
|
||||
atlas_archive_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_archive }}"
|
||||
atlas_services_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_services }}"
|
||||
atlas_app_data_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_app_data }}"
|
||||
atlas_navidrome_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_navidrome }}"
|
||||
atlas_syncthing_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_syncthing }}"
|
||||
atlas_media_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_media }}"
|
||||
atlas_music_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_music }}"
|
||||
atlas_backup_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_backup }}"
|
||||
atlas_host_backups_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_host_backups }}"
|
||||
atlas_work_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_work }}"
|
||||
atlas_backup_prometheus_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_backup_prometheus }}"
|
||||
atlas_photobook_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_photobook }}"
|
||||
atlas_backups_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_backups }}"
|
||||
atlas_service_backups_mountpoint: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_service_backups }}"
|
||||
|
||||
atlas_45drives_repo_url: https://repo.45drives.com/repofiles/rocky/45drives-enterprise.repo
|
||||
atlas_45drives_repo_file: /etc/yum.repos.d/45drives-enterprise.repo
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Turn Borg's JSON progress stream into bounded, readable journal entries."""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
|
||||
|
||||
def size(value):
|
||||
if not isinstance(value, (int, float)):
|
||||
return "unknown"
|
||||
return f"{value / (1024 ** 3):.2f} GiB"
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--estimated-total-bytes", type=int, required=True)
|
||||
args = parser.parse_args()
|
||||
if args.estimated_total_bytes <= 0:
|
||||
parser.error("estimated total must be positive")
|
||||
|
||||
last_progress = 0.0
|
||||
for line in sys.stdin:
|
||||
try:
|
||||
event = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
print(line.rstrip(), flush=True)
|
||||
continue
|
||||
|
||||
kind = event.get("type")
|
||||
if kind == "archive_progress":
|
||||
now = time.monotonic()
|
||||
if now - last_progress < 60 and not event.get("finished"):
|
||||
continue
|
||||
path = event.get("path") or ""
|
||||
parts = path.split("/")
|
||||
dataset = parts[1] if len(parts) > 1 and parts[0] == "source" else "unknown"
|
||||
original_size = event.get("original_size")
|
||||
if isinstance(original_size, (int, float)) and original_size >= 0:
|
||||
percent = original_size / args.estimated_total_bytes * 100
|
||||
estimated_progress = (
|
||||
f"{percent:.1f}%" if percent < 100 else ">=100% (ZFS estimate exceeded)"
|
||||
)
|
||||
else:
|
||||
estimated_progress = "unknown"
|
||||
print(
|
||||
"Borg create progress: "
|
||||
f"estimated={estimated_progress} dataset={dataset} "
|
||||
f"files={event.get('nfiles', 'unknown')} "
|
||||
f"original={size(original_size)} "
|
||||
f"compressed={size(event.get('compressed_size'))} "
|
||||
f"deduplicated={size(event.get('deduplicated_size'))}",
|
||||
flush=True,
|
||||
)
|
||||
last_progress = now
|
||||
elif kind == "log_message":
|
||||
print(f"Borg {event.get('levelname', 'INFO')}: {event.get('message', '')}", flush=True)
|
||||
elif kind == "progress_message" and event.get("message"):
|
||||
print(f"Borg: {event['message']}", flush=True)
|
||||
@@ -1,396 +0,0 @@
|
||||
#!/usr/bin/python3
|
||||
"""Read-only Atlas health probes with deduplicated 45Drives Alerts."""
|
||||
|
||||
import argparse
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
CONFIG_PATH = Path("/etc/atlas-health-monitor.json")
|
||||
STATE_DIR = Path("/var/lib/atlas-health-monitor")
|
||||
STATE_PATH = STATE_DIR / "state.json"
|
||||
GIB = 1024**3
|
||||
|
||||
|
||||
def run(*argv, timeout=40):
|
||||
return subprocess.run(argv, capture_output=True, text=True, timeout=timeout, check=False)
|
||||
|
||||
|
||||
def issue(issues, key, severity, message):
|
||||
issues[key] = {"severity": severity, "message": message}
|
||||
|
||||
|
||||
def notify(config, event, severity, subject, message):
|
||||
now = datetime.now(timezone.utc)
|
||||
payload = {
|
||||
"timestamp": now.isoformat(timespec="seconds"),
|
||||
"unixtime": int(now.timestamp()),
|
||||
"event": event,
|
||||
"severity": severity,
|
||||
"subject": subject,
|
||||
"email_message": message,
|
||||
}
|
||||
result = run(config["notifier"], json.dumps(payload, ensure_ascii=False), timeout=30)
|
||||
if result.returncode:
|
||||
raise RuntimeError(f"45Drives notifier exited {result.returncode}: {result.stderr.strip()}")
|
||||
|
||||
|
||||
def parse_fields(text):
|
||||
return dict(line.split("=", 1) for line in text.splitlines() if "=" in line)
|
||||
|
||||
|
||||
def systemd_fields(unit, *properties):
|
||||
result = run("systemctl", "show", unit, *(f"-p{item}" for item in properties))
|
||||
if result.returncode:
|
||||
raise RuntimeError(f"systemctl show {unit} exited {result.returncode}")
|
||||
return parse_fields(result.stdout)
|
||||
|
||||
|
||||
def unix_time(text):
|
||||
if not text or text == "n/a":
|
||||
return None
|
||||
result = run("date", "-d", text, "+%s")
|
||||
if result.returncode:
|
||||
raise ValueError(f"Cannot parse systemd timestamp: {text}")
|
||||
return int(result.stdout.strip())
|
||||
|
||||
|
||||
def check_pool(config, issues, measurements):
|
||||
pool = config["pool"]
|
||||
listing = run("zpool", "list", "-H", "-p", "-o", "size,alloc,capacity,health", pool)
|
||||
if listing.returncode:
|
||||
issue(issues, "pool.probe", "critical", f"Cannot query ZFS pool {pool}")
|
||||
return
|
||||
try:
|
||||
size, alloc, capacity, health = listing.stdout.strip().split("\t")
|
||||
size, alloc, capacity = int(size), int(alloc), int(capacity)
|
||||
except (ValueError, TypeError):
|
||||
issue(issues, "pool.probe", "critical", "Invalid ZFS pool capacity response")
|
||||
return
|
||||
measurements.update(pool_size_bytes=size, pool_alloc_bytes=alloc, pool_capacity_percent=capacity)
|
||||
if health != "ONLINE":
|
||||
issue(issues, "pool.health", "critical", f"ZFS pool {pool} state is {health}")
|
||||
if capacity >= config["pool_critical_percent"]:
|
||||
issue(issues, "pool.capacity", "critical", f"ZFS pool {pool} is {capacity}% full")
|
||||
elif capacity >= config["pool_warning_percent"]:
|
||||
issue(issues, "pool.capacity", "warning", f"ZFS pool {pool} is {capacity}% full")
|
||||
|
||||
status = run("zpool", "status", "-P", pool)
|
||||
if status.returncode:
|
||||
issue(issues, "pool.status", "critical", f"Cannot query detailed ZFS status for {pool}")
|
||||
return
|
||||
bad_vdevs = []
|
||||
for line in status.stdout.splitlines():
|
||||
match = re.match(r"^\s*(\S+)\s+(ONLINE|DEGRADED|FAULTED|OFFLINE|UNAVAIL|REMOVED)\s+(\d+)\s+(\d+)\s+(\d+)", line)
|
||||
if match:
|
||||
name, state, reads, writes, checksums = match.groups()
|
||||
if state != "ONLINE" or any(int(value) for value in (reads, writes, checksums)):
|
||||
bad_vdevs.append(f"{name}: {state}, READ={reads}, WRITE={writes}, CKSUM={checksums}")
|
||||
if bad_vdevs:
|
||||
issue(issues, "pool.vdevs", "critical", "ZFS vdev errors: " + "; ".join(bad_vdevs))
|
||||
errors = re.search(r"^errors:\s*(.*)$", status.stdout, re.MULTILINE)
|
||||
if not errors or errors.group(1).strip() != "No known data errors":
|
||||
issue(issues, "pool.data_errors", "critical", "ZFS status reports data errors; inspect zpool status -v")
|
||||
if re.search(r"^\s*scan:\s*resilver in progress", status.stdout, re.MULTILINE | re.IGNORECASE):
|
||||
issue(issues, "pool.resilver", "warning", "ZFS resilver is in progress; inspect zpool status")
|
||||
scan = re.search(r"^\s*scan:\s*(.*)$", status.stdout, re.MULTILINE)
|
||||
if scan and re.search(r"\bwith [1-9][0-9]* errors\b", scan.group(1)):
|
||||
issue(issues, "pool.scan_errors", "critical", f"ZFS scan reported errors: {scan.group(1)}")
|
||||
|
||||
|
||||
def check_capacity(config, issues, measurements):
|
||||
pool = config["pool"]
|
||||
listing = run("zfs", "list", "-H", "-p", "-o", "name,usedbysnapshots", "-r", pool)
|
||||
if listing.returncode:
|
||||
issue(issues, "snapshot.probe", "warning", "Cannot query ZFS snapshot space")
|
||||
else:
|
||||
try:
|
||||
snapshots = sum(int(line.split("\t")[1]) for line in listing.stdout.splitlines())
|
||||
measurements["snapshots_bytes"] = snapshots
|
||||
size = measurements.get("pool_size_bytes")
|
||||
if size:
|
||||
percent = snapshots * 100 // size
|
||||
measurements["snapshots_percent"] = percent
|
||||
if percent >= config["snapshot_critical_percent"]:
|
||||
issue(issues, "snapshot.capacity", "critical", f"Snapshots use {percent}% of pool size")
|
||||
elif percent >= config["snapshot_warning_percent"]:
|
||||
issue(issues, "snapshot.capacity", "warning", f"Snapshots use {percent}% of pool size")
|
||||
except (ValueError, IndexError):
|
||||
issue(issues, "snapshot.probe", "warning", "Invalid ZFS snapshot-space response")
|
||||
backup = run("zfs", "list", "-H", "-p", "-o", "used", config["backup_dataset"])
|
||||
if backup.returncode:
|
||||
issue(issues, "backup.capacity_probe", "warning", "Cannot query local backup dataset space")
|
||||
else:
|
||||
try:
|
||||
measurements["backup_bytes"] = int(backup.stdout.strip())
|
||||
except ValueError:
|
||||
issue(issues, "backup.capacity_probe", "warning", "Invalid local backup space response")
|
||||
|
||||
try:
|
||||
filesystem = os.statvfs("/")
|
||||
total = filesystem.f_blocks * filesystem.f_frsize
|
||||
available = filesystem.f_bavail * filesystem.f_frsize
|
||||
used_percent = (total - available) * 100 // total
|
||||
measurements["root_capacity_percent"] = used_percent
|
||||
if used_percent >= config["root_critical_percent"]:
|
||||
issue(issues, "root.capacity", "critical", f"Atlas system filesystem is {used_percent}% full")
|
||||
elif used_percent >= config["root_warning_percent"]:
|
||||
issue(issues, "root.capacity", "warning", f"Atlas system filesystem is {used_percent}% full")
|
||||
except (OSError, ZeroDivisionError):
|
||||
issue(issues, "root.capacity_probe", "warning", "Cannot query Atlas system filesystem space")
|
||||
|
||||
|
||||
def check_remote_capacity(config, issues, measurements):
|
||||
"""Query only the Storage Box quota; do not open or inspect the Borg repository."""
|
||||
remote = config["remote_capacity"]
|
||||
try:
|
||||
result = run("runuser", "-u", remote["run_as"], "--", remote["ssh_wrapper"],
|
||||
f"{remote['user']}@{remote['host']}", "df", "-m", timeout=65)
|
||||
if result.returncode:
|
||||
raise ValueError(f"SSH df exited {result.returncode}")
|
||||
lines = result.stdout.strip().splitlines()
|
||||
if len(lines) != 2:
|
||||
raise ValueError("Unexpected Storage Box df output")
|
||||
fields = lines[1].split()
|
||||
if len(fields) < 5:
|
||||
raise ValueError("Incomplete Storage Box df output")
|
||||
total_mib, used_mib, available_mib = (int(value) for value in fields[1:4])
|
||||
percent = int(fields[4].rstrip("%"))
|
||||
if total_mib <= 0 or not 0 <= percent <= 100 or available_mib < 0:
|
||||
raise ValueError("Invalid Storage Box quota values")
|
||||
except (OSError, ValueError, subprocess.TimeoutExpired):
|
||||
issue(issues, "remote.capacity_probe", "warning", "Cannot query Hetzner Storage Box quota via pinned-key SSH")
|
||||
return
|
||||
measurements.update(remote_capacity_percent=percent, remote_bytes=used_mib * 1024**2,
|
||||
remote_available_bytes=available_mib * 1024**2)
|
||||
if percent >= remote["critical_percent"]:
|
||||
issue(issues, "remote.capacity", "critical", f"Hetzner Storage Box quota is {percent}% full")
|
||||
elif percent >= remote["warning_percent"]:
|
||||
issue(issues, "remote.capacity", "warning", f"Hetzner Storage Box quota is {percent}% full")
|
||||
|
||||
|
||||
def check_smart(config, issues, measurements):
|
||||
for device in config["smart_devices"]:
|
||||
name, path = device["name"], device["path"]
|
||||
try:
|
||||
result = run("smartctl", "-j", "-a", path, timeout=60)
|
||||
data = json.loads(result.stdout)
|
||||
status = int(data.get("smartctl", {}).get("exit_status", result.returncode))
|
||||
except (subprocess.TimeoutExpired, json.JSONDecodeError, ValueError) as exc:
|
||||
issue(issues, f"smart.{name}.probe", "critical", f"SMART probe failed for {name}: {type(exc).__name__}")
|
||||
continue
|
||||
if status:
|
||||
severity = "critical" if status & 0b00001111 else "warning"
|
||||
issue(issues, f"smart.{name}.status", severity, f"SMART reported exit status {status} for {name}")
|
||||
passed = data.get("smart_status", {}).get("passed")
|
||||
if passed is False:
|
||||
issue(issues, f"smart.{name}.health", "critical", f"SMART self-assessment failed for {name}")
|
||||
elif passed is None:
|
||||
issue(issues, f"smart.{name}.health", "warning", f"SMART self-assessment unavailable for {name}")
|
||||
temperature = data.get("temperature", {}).get("current")
|
||||
if isinstance(temperature, (int, float)):
|
||||
measurements[f"smart_{name}_c"] = temperature
|
||||
if temperature >= device["critical_c"]:
|
||||
issue(issues, f"smart.{name}.temperature", "critical", f"{name} temperature is {temperature} C")
|
||||
elif temperature >= device["warning_c"]:
|
||||
issue(issues, f"smart.{name}.temperature", "warning", f"{name} temperature is {temperature} C")
|
||||
else:
|
||||
issue(issues, f"smart.{name}.temperature", "warning", f"Temperature unavailable for {name}")
|
||||
for attribute in data.get("ata_smart_attributes", {}).get("table", []):
|
||||
attribute_id = attribute.get("id")
|
||||
if attribute_id in (5, 187, 197, 198):
|
||||
raw = attribute.get("raw", {}).get("value", 0)
|
||||
if isinstance(raw, int) and raw > 0:
|
||||
severity = "critical" if attribute_id in (197, 198) else "warning"
|
||||
issue(issues, f"smart.{name}.ata_{attribute_id}", severity,
|
||||
f"{name} SMART attribute {attribute_id} raw count is {raw}")
|
||||
nvme = data.get("nvme_smart_health_information_log", {})
|
||||
if isinstance(nvme, dict):
|
||||
if int(nvme.get("critical_warning", 0)):
|
||||
issue(issues, f"smart.{name}.nvme_warning", "critical", f"{name} NVMe critical warning is nonzero")
|
||||
if int(nvme.get("media_errors", 0)):
|
||||
issue(issues, f"smart.{name}.nvme_media", "critical", f"{name} NVMe media errors are nonzero")
|
||||
|
||||
|
||||
def check_cpu(config, issues, measurements):
|
||||
sensors = []
|
||||
for hwmon in Path("/sys/class/hwmon").glob("hwmon*"):
|
||||
try:
|
||||
if (hwmon / "name").read_text().strip() != "coretemp":
|
||||
continue
|
||||
sensors.extend(int(path.read_text().strip()) / 1000 for path in hwmon.glob("temp*_input"))
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
if not sensors:
|
||||
issue(issues, "cpu.temperature_probe", "warning", "CPU temperature sensors are unavailable")
|
||||
return
|
||||
hottest = max(sensors)
|
||||
measurements["cpu_max_c"] = hottest
|
||||
if hottest >= config["cpu_critical_c"]:
|
||||
issue(issues, "cpu.temperature", "critical", f"CPU temperature is {hottest:g} C")
|
||||
elif hottest >= config["cpu_warning_c"]:
|
||||
issue(issues, "cpu.temperature", "warning", f"CPU temperature is {hottest:g} C")
|
||||
|
||||
|
||||
def check_jobs(config, issues, measurements, now):
|
||||
for timer in config["timers"]:
|
||||
name = timer["name"]
|
||||
try:
|
||||
fields = systemd_fields(name, "ActiveState", "UnitFileState", "LastTriggerUSec", "ActiveEnterTimestamp")
|
||||
if fields.get("ActiveState") != "active" or fields.get("UnitFileState") != "enabled":
|
||||
issue(issues, f"timer.{name}", "critical", f"Timer {name} is not active and enabled")
|
||||
max_age = int(timer["max_age_hours"]) * 3600
|
||||
if max_age:
|
||||
last = unix_time(fields.get("LastTriggerUSec"))
|
||||
if last is None:
|
||||
last = unix_time(fields.get("ActiveEnterTimestamp"))
|
||||
if last is not None and now - last > max_age:
|
||||
issue(issues, f"timer.{name}.stale", "warning",
|
||||
f"Timer {name} has not fired in {int((now-last)/3600)} hours")
|
||||
except (RuntimeError, ValueError, subprocess.TimeoutExpired):
|
||||
issue(issues, f"timer.{name}.probe", "warning", f"Cannot query timer {name}")
|
||||
for unit in config["failure_units"]:
|
||||
if unit.endswith("@.service"):
|
||||
continue
|
||||
try:
|
||||
fields = systemd_fields(unit, "ActiveState", "Result", "ExecMainStartTimestamp")
|
||||
state = fields.get("ActiveState")
|
||||
if state == "failed" or (state == "inactive" and fields.get("Result") not in (None, "", "success")):
|
||||
issue(issues, f"service.{unit}", "critical", f"Service {unit} failed: {fields.get('Result')}")
|
||||
if unit == "atlas-borg-backup.service" and fields.get("ActiveState") == "activating":
|
||||
started = unix_time(fields.get("ExecMainStartTimestamp"))
|
||||
if started is not None and now - started > config["borg_max_runtime_days"] * 86400:
|
||||
issue(issues, "backup.borg_long_running", "warning",
|
||||
"Borg has run longer than its configured limit")
|
||||
except (RuntimeError, ValueError, subprocess.TimeoutExpired):
|
||||
issue(issues, f"service.{unit}.probe", "warning", f"Cannot query service {unit}")
|
||||
|
||||
|
||||
def check_growth(config, issues, measurements, samples, now):
|
||||
previous = [sample for sample in samples if 20 * 3600 <= now - sample.get("time", now) <= 48 * 3600]
|
||||
if previous:
|
||||
baseline = min(previous, key=lambda sample: abs(now - sample["time"] - 86400))
|
||||
days = (now - baseline["time"]) / 86400
|
||||
for name, threshold in (("snapshots", config["snapshot_growth_warning_gib_day"]),
|
||||
("backup", config["backup_growth_warning_gib_day"]),
|
||||
("remote", config["remote_capacity"]["growth_warning_gib_day"])):
|
||||
current, old = measurements.get(f"{name}_bytes"), baseline.get(f"{name}_bytes")
|
||||
if isinstance(current, int) and isinstance(old, int) and days > 0:
|
||||
growth_gib_day = (current - old) / GIB / days
|
||||
measurements[f"{name}_growth_gib_day"] = round(growth_gib_day, 1)
|
||||
if growth_gib_day >= threshold:
|
||||
issue(issues, f"{name}.growth", "warning",
|
||||
f"Local {name} usage grew {growth_gib_day:.1f} GiB/day over {days:.1f} days")
|
||||
|
||||
|
||||
def allowed_failure_unit(config, unit):
|
||||
for allowed in config["failure_units"]:
|
||||
if allowed == unit:
|
||||
return True
|
||||
if allowed.endswith("@.service") and unit.startswith(allowed[:-9] + "@") and unit.endswith(".service"):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def load_state():
|
||||
if not STATE_PATH.exists():
|
||||
return {"active": {}, "samples": []}
|
||||
with STATE_PATH.open(encoding="utf-8") as stream:
|
||||
state = json.load(stream)
|
||||
if not isinstance(state.get("active"), dict) or not isinstance(state.get("samples"), list):
|
||||
raise ValueError("Invalid Atlas monitor state; refusing to overwrite it")
|
||||
return state
|
||||
|
||||
|
||||
def save_state(state):
|
||||
with tempfile.NamedTemporaryFile("w", dir=STATE_DIR, prefix=".state-", delete=False,
|
||||
encoding="utf-8") as stream:
|
||||
path = Path(stream.name)
|
||||
os.chmod(path, 0o600)
|
||||
json.dump(state, stream, sort_keys=True)
|
||||
stream.write("\n")
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.replace(path, STATE_PATH)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--dry-run", action="store_true", help="probe without notifications or state changes")
|
||||
parser.add_argument("--test-notification", action="store_true", help="submit a labelled test alert")
|
||||
parser.add_argument("--job-failed", metavar="UNIT", help="notify about a failed configured service")
|
||||
args = parser.parse_args()
|
||||
with CONFIG_PATH.open(encoding="utf-8") as stream:
|
||||
config = json.load(stream)
|
||||
if args.test_notification:
|
||||
notify(config, "atlas_monitor_test", "warning", "Test monitoraggio Atlas",
|
||||
"Notifica di prova: il monitoraggio Atlas raggiunge 45Drives Alerts. Non conferma l'invio email.")
|
||||
print("Atlas monitor test submitted to 45Drives Alerts; email delivery is not verified.")
|
||||
return 0
|
||||
if args.job_failed:
|
||||
if not allowed_failure_unit(config, args.job_failed):
|
||||
raise ValueError("Unconfigured Atlas failure unit")
|
||||
notify(config, "atlas_job_failed", "critical", f"Job Atlas fallito: {args.job_failed}",
|
||||
f"Il servizio {args.job_failed} e' fallito. Controlla: "
|
||||
f"sudo journalctl -u {args.job_failed} -n 100 --no-pager")
|
||||
print(f"Atlas job failure submitted to 45Drives Alerts: {args.job_failed}")
|
||||
return 0
|
||||
|
||||
now = int(time.time())
|
||||
issues, measurements = {}, {}
|
||||
check_pool(config, issues, measurements)
|
||||
check_capacity(config, issues, measurements)
|
||||
check_remote_capacity(config, issues, measurements)
|
||||
check_smart(config, issues, measurements)
|
||||
check_cpu(config, issues, measurements)
|
||||
check_jobs(config, issues, measurements, now)
|
||||
if args.dry_run:
|
||||
print(json.dumps({"issues": issues, "measurements": measurements}, sort_keys=True))
|
||||
return 0
|
||||
|
||||
STATE_DIR.mkdir(mode=0o700, exist_ok=True)
|
||||
with (STATE_DIR / "monitor.lock").open("w") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
state = load_state()
|
||||
check_growth(config, issues, measurements, state["samples"], now)
|
||||
active, failed_notifications = state["active"], []
|
||||
for key, details in issues.items():
|
||||
old = active.get(key)
|
||||
if old is None or old.get("severity") != details["severity"]:
|
||||
try:
|
||||
notify(config, "atlas_health_issue", details["severity"],
|
||||
f"Atlas: {key}", details["message"])
|
||||
active[key] = details
|
||||
print(f"ALERT {details['severity']} {key}: {details['message']}", flush=True)
|
||||
except (RuntimeError, subprocess.TimeoutExpired) as exc:
|
||||
failed_notifications.append(key)
|
||||
print(f"NOTIFICATION FAILED {key}: {exc}", file=sys.stderr, flush=True)
|
||||
for key in set(active) - set(issues):
|
||||
print(f"RECOVERED {key}", flush=True)
|
||||
del active[key]
|
||||
state["samples"] = [sample for sample in state["samples"] if now - sample.get("time", 0) < 48 * 3600]
|
||||
state["samples"].append({"time": now, **{key: value for key, value in measurements.items()
|
||||
if key in ("snapshots_bytes", "backup_bytes", "remote_bytes")}})
|
||||
save_state(state)
|
||||
print(f"Atlas health: issues={len(issues)} notifications_failed={len(failed_notifications)} "
|
||||
f"pool={measurements.get('pool_capacity_percent', 'unknown')}% "
|
||||
f"remote={measurements.get('remote_capacity_percent', 'unknown')}% "
|
||||
f"snapshots={measurements.get('snapshots_bytes', 'unknown')} bytes "
|
||||
f"backup={measurements.get('backup_bytes', 'unknown')} bytes", flush=True)
|
||||
return 1 if failed_notifications else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
sys.exit(main())
|
||||
except (OSError, RuntimeError, ValueError, subprocess.TimeoutExpired) as error:
|
||||
print(f"Atlas health monitor failed: {error}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -28,18 +28,6 @@
|
||||
name: smb
|
||||
state: restarted
|
||||
|
||||
- name: Restart Atlas Borg timers
|
||||
ansible.builtin.systemd:
|
||||
name: "{{ item }}"
|
||||
state: restarted
|
||||
daemon_reload: true
|
||||
loop:
|
||||
- atlas-borg-backup.timer
|
||||
- atlas-borg-check.timer
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Restart Atlas media Quadlets
|
||||
ansible.builtin.systemd:
|
||||
name: "{{ item }}"
|
||||
|
||||
@@ -1,538 +0,0 @@
|
||||
---
|
||||
- name: Validate Atlas Borg backup configuration
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_manage_storage | bool
|
||||
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
|
||||
- atlas_mount_root.startswith('/')
|
||||
- atlas_borg_username is match('^[a-z_][a-z0-9_-]*$')
|
||||
- atlas_borg_group is match('^[a-z_][a-z0-9_-]*$')
|
||||
- atlas_borg_username not in ['root', atlas_admin_username]
|
||||
- atlas_borg_group != 'wheel'
|
||||
- atlas_borg_home.startswith('/var/lib/')
|
||||
- atlas_borg_repository_host is match('^[A-Za-z0-9.-]+$')
|
||||
- atlas_borg_repository_user is match('^[A-Za-z0-9_-]+$')
|
||||
- atlas_borg_repository_port | int > 0
|
||||
- atlas_borg_repository_port | int < 65536
|
||||
- atlas_borg_repository_path is match('^\./[A-Za-z0-9][A-Za-z0-9._/-]*$')
|
||||
- "'/../' not in ('/' ~ atlas_borg_repository_path ~ '/')"
|
||||
- atlas_borg_remote_path is match('^borg-[0-9]+\.[0-9]+$')
|
||||
- atlas_borg_host_key.startswith(
|
||||
'[' ~ atlas_borg_repository_host ~ ']:' ~ (atlas_borg_repository_port | string) ~ ' ssh-ed25519 '
|
||||
)
|
||||
- atlas_borg_ssh_private_key_path.startswith('/etc/atlas-borg/')
|
||||
- atlas_borg_known_hosts_path.startswith('/etc/atlas-borg/')
|
||||
- atlas_borg_passphrase_path.startswith('/etc/atlas-borg/')
|
||||
- atlas_borg_ssh_wrapper_path.startswith('/usr/local/libexec/')
|
||||
- atlas_borg_encryption_mode == 'repokey'
|
||||
- atlas_borg_archive_prefix is match('^[a-z0-9][a-z0-9_-]*$')
|
||||
- atlas_borg_snapshot_prefix is match('^[a-z0-9][a-z0-9_-]*$')
|
||||
- atlas_borg_keep_daily | int > 0
|
||||
- atlas_borg_keep_weekly | int > 0
|
||||
- atlas_borg_keep_monthly | int > 0
|
||||
fail_msg: >-
|
||||
Atlas Borg needs a safe relative repository path, a pinned ED25519 host
|
||||
key, positive retention counts, and valid dedicated SSH settings.
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Create the Atlas Borg system group
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.group:
|
||||
name: "{{ atlas_borg_group }}"
|
||||
system: true
|
||||
state: present
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Create the least-privilege Atlas Borg account
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.user:
|
||||
name: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
groups: []
|
||||
append: false
|
||||
comment: Atlas Borg backup service
|
||||
home: "{{ atlas_borg_home }}"
|
||||
create_home: false
|
||||
shell: /sbin/nologin
|
||||
password_lock: true
|
||||
system: true
|
||||
state: present
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Read Atlas Borg account group membership
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- id
|
||||
- -nG
|
||||
- "{{ atlas_borg_username }}"
|
||||
register: atlas_borg_account_groups
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Require the Atlas Borg account to have no supplementary groups
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_borg_account_groups.stdout.split() == [atlas_borg_group]
|
||||
fail_msg: >-
|
||||
The Atlas Borg service account must belong only to its private primary
|
||||
group and must never receive wheel or other supplementary membership.
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Validate Atlas Borg systemd calendars
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemd-analyze
|
||||
- calendar
|
||||
- "{{ item }}"
|
||||
loop:
|
||||
- "{{ atlas_borg_backup_calendar }}"
|
||||
- "{{ atlas_borg_check_calendar }}"
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Create Atlas Borg configuration directory
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.file:
|
||||
path: /etc/atlas-borg
|
||||
state: directory
|
||||
owner: root
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0750"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Generate the dedicated Atlas Borg SSH identity
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- ssh-keygen
|
||||
- -q
|
||||
- -t
|
||||
- ed25519
|
||||
- -N
|
||||
- ""
|
||||
- -C
|
||||
- atlas-borg@atlas
|
||||
- -f
|
||||
- "{{ atlas_borg_ssh_private_key_path }}"
|
||||
creates: "{{ atlas_borg_ssh_private_key_path }}"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Protect the Atlas Borg private SSH identity
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.file:
|
||||
path: "{{ atlas_borg_ssh_private_key_path }}"
|
||||
owner: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0600"
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Set permissions on the Atlas Borg public SSH identity
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.file:
|
||||
path: "{{ atlas_borg_ssh_private_key_path }}.pub"
|
||||
owner: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0644"
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Read the dedicated Atlas Borg public SSH identity
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.slurp:
|
||||
src: "{{ atlas_borg_ssh_private_key_path }}.pub"
|
||||
register: atlas_borg_public_key
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Report the public SSH identity to install in the Hetzner sub-account
|
||||
tags: [atlas, storage, backup, borg, borg_key]
|
||||
ansible.builtin.debug:
|
||||
msg: "{{ atlas_borg_public_key.content | b64decode | trim }}"
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Pin the Hetzner Storage Box SSH host key
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.copy:
|
||||
content: "{{ atlas_borg_host_key }}\n"
|
||||
dest: "{{ atlas_borg_known_hosts_path }}"
|
||||
owner: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0600"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Create Atlas Borg state directories
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.file:
|
||||
path: "{{ item }}"
|
||||
state: directory
|
||||
owner: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0700"
|
||||
loop:
|
||||
- "{{ atlas_borg_config_dir }}"
|
||||
- "{{ atlas_borg_cache_dir }}"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Create the shared Atlas Borg operation lock
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.copy:
|
||||
content: ""
|
||||
dest: "{{ atlas_borg_lock_path }}"
|
||||
owner: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0600"
|
||||
force: false
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Require the Atlas Borg encryption passphrase from Vault
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_borg_passphrase | length >= 20
|
||||
fail_msg: >-
|
||||
Define vault_atlas_borg_passphrase with a strong unique value in the
|
||||
encrypted Vault before activating the Borg repository.
|
||||
no_log: true
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install the Atlas Borg passphrase
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.copy:
|
||||
content: "{{ atlas_borg_passphrase }}\n"
|
||||
dest: "{{ atlas_borg_passphrase_path }}"
|
||||
owner: "{{ atlas_borg_username }}"
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0600"
|
||||
diff: false
|
||||
no_log: true
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install the Atlas Borg backup helper
|
||||
tags: [atlas, storage, backup, borg, borg_logging]
|
||||
ansible.builtin.template:
|
||||
src: atlas-borg-backup.sh.j2
|
||||
dest: /usr/local/sbin/atlas-borg-backup
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install the Atlas Borg snapshot cleanup helper
|
||||
tags: [atlas, storage, backup, borg, borg_logging]
|
||||
ansible.builtin.template:
|
||||
src: atlas-borg-snapshot-cleanup.sh.j2
|
||||
dest: /usr/local/sbin/atlas-borg-snapshot-cleanup
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install the Atlas Borg check helper
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.template:
|
||||
src: atlas-borg-check.sh.j2
|
||||
dest: /usr/local/sbin/atlas-borg-check
|
||||
owner: root
|
||||
group: "{{ atlas_borg_group }}"
|
||||
mode: "0750"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Create the local libexec directory for the Atlas Borg SSH wrapper
|
||||
tags: [atlas, storage, backup, borg, borg_logging]
|
||||
ansible.builtin.file:
|
||||
path: "{{ atlas_borg_ssh_wrapper_path | dirname }}"
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install the Atlas Borg progress formatter
|
||||
tags: [atlas, storage, backup, borg, borg_logging]
|
||||
ansible.builtin.copy:
|
||||
src: atlas-borg-progress.py
|
||||
dest: /usr/local/libexec/atlas-borg-progress
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install the capability-dropping Atlas Borg SSH wrapper
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.template:
|
||||
src: atlas-borg-ssh.sh.j2
|
||||
dest: "{{ atlas_borg_ssh_wrapper_path }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Install Atlas Borg systemd units
|
||||
tags: [atlas, storage, backup, borg, borg_logging]
|
||||
ansible.builtin.template:
|
||||
src: "{{ item }}.j2"
|
||||
dest: "/etc/systemd/system/{{ item }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
loop:
|
||||
- atlas-borg-backup.service
|
||||
- atlas-borg-backup.timer
|
||||
- atlas-borg-check.service
|
||||
- atlas-borg-check.timer
|
||||
notify: Restart Atlas Borg timers
|
||||
when: atlas_manage_borg_backup | bool
|
||||
|
||||
- name: Verify dedicated SSH access to the Hetzner Storage Box
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- ssh
|
||||
- -T
|
||||
- -i
|
||||
- "{{ atlas_borg_ssh_private_key_path }}"
|
||||
- -p
|
||||
- "{{ atlas_borg_repository_port | string }}"
|
||||
- -o
|
||||
- BatchMode=yes
|
||||
- -o
|
||||
- IdentitiesOnly=yes
|
||||
- -o
|
||||
- StrictHostKeyChecking=yes
|
||||
- -o
|
||||
- "UserKnownHostsFile={{ atlas_borg_known_hosts_path }}"
|
||||
- "{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}"
|
||||
- pwd
|
||||
register: atlas_borg_ssh_probe
|
||||
become: true
|
||||
become_user: "{{ atlas_borg_username }}"
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Require the dedicated public key on the Hetzner sub-account
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_borg_ssh_probe.rc == 0
|
||||
fail_msg: >-
|
||||
Install the reported Atlas Borg public key in the Hetzner sub-account
|
||||
before rerunning the Borg tasks. Password authentication is never used.
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Probe the remote Atlas Borg repository path
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- ssh
|
||||
- -T
|
||||
- -i
|
||||
- "{{ atlas_borg_ssh_private_key_path }}"
|
||||
- -p
|
||||
- "{{ atlas_borg_repository_port | string }}"
|
||||
- -o
|
||||
- BatchMode=yes
|
||||
- -o
|
||||
- IdentitiesOnly=yes
|
||||
- -o
|
||||
- StrictHostKeyChecking=yes
|
||||
- -o
|
||||
- "UserKnownHostsFile={{ atlas_borg_known_hosts_path }}"
|
||||
- "{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}"
|
||||
- stat
|
||||
- "{{ atlas_borg_repository_path }}"
|
||||
register: atlas_borg_repository_path_probe
|
||||
become: true
|
||||
become_user: "{{ atlas_borg_username }}"
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Probe the Atlas Borg repository
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/borg
|
||||
- --remote-path
|
||||
- "{{ atlas_borg_remote_path }}"
|
||||
- info
|
||||
- >-
|
||||
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
|
||||
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
|
||||
environment:
|
||||
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
|
||||
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
|
||||
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
|
||||
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
|
||||
register: atlas_borg_repository_probe
|
||||
become: true
|
||||
become_user: "{{ atlas_borg_username }}"
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
- atlas_borg_repository_path_probe.rc == 0
|
||||
|
||||
- name: Reject an existing path that is not the configured Borg repository
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_borg_repository_probe.rc == 0
|
||||
fail_msg: >-
|
||||
The remote repository path already exists but Borg could not open it.
|
||||
Refusing to initialize over existing data; verify the path, passphrase,
|
||||
and repository state manually.
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
- atlas_borg_repository_path_probe.rc == 0
|
||||
|
||||
- name: Initialize the encrypted Atlas Borg repository
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/borg
|
||||
- --remote-path
|
||||
- "{{ atlas_borg_remote_path }}"
|
||||
- init
|
||||
- --encryption
|
||||
- "{{ atlas_borg_encryption_mode }}"
|
||||
- >-
|
||||
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
|
||||
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
|
||||
environment:
|
||||
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
|
||||
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
|
||||
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
|
||||
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
|
||||
become: true
|
||||
become_user: "{{ atlas_borg_username }}"
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
- atlas_borg_repository_path_probe.rc != 0
|
||||
|
||||
- name: Verify the encrypted Atlas Borg repository
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/borg
|
||||
- --remote-path
|
||||
- "{{ atlas_borg_remote_path }}"
|
||||
- info
|
||||
- >-
|
||||
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
|
||||
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
|
||||
environment:
|
||||
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
|
||||
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
|
||||
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
|
||||
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
|
||||
become: true
|
||||
become_user: "{{ atlas_borg_username }}"
|
||||
changed_when: false
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Check for the local Atlas Borg recovery-key export
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.stat:
|
||||
path: "{{ atlas_borg_recovery_export_path }}"
|
||||
register: atlas_borg_recovery_export
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Export the Atlas Borg recovery key for offline preservation
|
||||
tags: [atlas, storage, backup, borg]
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
- not atlas_borg_recovery_export.stat.exists
|
||||
no_log: true
|
||||
block:
|
||||
- name: Create the local recovery-material directory
|
||||
ansible.builtin.file:
|
||||
path: "{{ atlas_borg_recovery_export_path | dirname }}"
|
||||
state: directory
|
||||
mode: "0700"
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
|
||||
- name: Export the encrypted Borg repository key on Atlas
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/borg
|
||||
- --remote-path
|
||||
- "{{ atlas_borg_remote_path }}"
|
||||
- key
|
||||
- export
|
||||
- >-
|
||||
ssh://{{ atlas_borg_repository_user }}@{{ atlas_borg_repository_host }}:
|
||||
{{- atlas_borg_repository_port }}/{{ atlas_borg_repository_path }}
|
||||
- "{{ atlas_borg_config_dir }}/atlas-borg-repokey.export"
|
||||
environment:
|
||||
BORG_CACHE_DIR: "{{ atlas_borg_cache_dir }}"
|
||||
BORG_CONFIG_DIR: "{{ atlas_borg_config_dir }}"
|
||||
BORG_PASSCOMMAND: "cat {{ atlas_borg_passphrase_path }}"
|
||||
BORG_RSH: "{{ atlas_borg_ssh_wrapper_path }}"
|
||||
become: true
|
||||
become_user: "{{ atlas_borg_username }}"
|
||||
|
||||
- name: Fetch the encrypted Borg recovery key from Atlas
|
||||
ansible.builtin.fetch:
|
||||
src: "{{ atlas_borg_config_dir }}/atlas-borg-repokey.export"
|
||||
dest: "{{ atlas_borg_recovery_export_path }}"
|
||||
flat: true
|
||||
|
||||
- name: Protect the local Borg recovery-key export
|
||||
ansible.builtin.file:
|
||||
path: "{{ atlas_borg_recovery_export_path }}"
|
||||
mode: "0600"
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
always:
|
||||
- name: Remove the temporary recovery-key export from Atlas
|
||||
ansible.builtin.file:
|
||||
path: "{{ atlas_borg_config_dir }}/atlas-borg-repokey.export"
|
||||
state: absent
|
||||
|
||||
- name: Enable Atlas Borg backup and check timers
|
||||
tags: [atlas, storage, backup, borg]
|
||||
ansible.builtin.systemd:
|
||||
name: "{{ item }}"
|
||||
enabled: true
|
||||
state: started
|
||||
daemon_reload: true
|
||||
loop:
|
||||
- atlas-borg-backup.timer
|
||||
- atlas-borg-check.timer
|
||||
when:
|
||||
- atlas_manage_borg_backup | bool
|
||||
- not ansible_check_mode
|
||||
@@ -14,21 +14,6 @@
|
||||
- name: Import Atlas storage tasks
|
||||
ansible.builtin.import_tasks: storage.yml
|
||||
|
||||
- name: Import Atlas ZFS maintenance tasks
|
||||
ansible.builtin.import_tasks: zfs_maintenance.yml
|
||||
|
||||
- name: Import Atlas Borg backup tasks
|
||||
ansible.builtin.import_tasks: borg_backup.yml
|
||||
|
||||
- name: Import Atlas offline USB backup tasks
|
||||
ansible.builtin.import_tasks: usb_backup.yml
|
||||
|
||||
- name: Import Atlas health monitoring tasks
|
||||
ansible.builtin.import_tasks: monitoring.yml
|
||||
|
||||
- name: Import Atlas post-restore SELinux relabeling tasks
|
||||
ansible.builtin.import_tasks: restorecon.yml
|
||||
|
||||
- name: Import Atlas file sharing tasks
|
||||
ansible.builtin.import_tasks: sharing.yml
|
||||
|
||||
|
||||
@@ -1,201 +0,0 @@
|
||||
---
|
||||
- name: Validate Atlas health monitoring policy
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_manage_storage | bool
|
||||
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
|
||||
- atlas_monitor_calendar | length > 0
|
||||
- atlas_monitor_smart_devices | length > 0
|
||||
- atlas_monitor_timers | length > 0
|
||||
- atlas_monitor_failure_units | length > 0
|
||||
- atlas_monitor_remote_capacity.user == atlas_borg_repository_user
|
||||
- atlas_monitor_remote_capacity.host == atlas_borg_repository_host
|
||||
- atlas_monitor_remote_capacity.run_as == atlas_borg_username
|
||||
- atlas_monitor_remote_capacity.ssh_wrapper == atlas_borg_ssh_wrapper_path
|
||||
- >-
|
||||
0 < atlas_monitor_remote_capacity.warning_percent | int
|
||||
< atlas_monitor_remote_capacity.critical_percent | int < 100
|
||||
- atlas_monitor_remote_capacity.growth_warning_gib_day | int > 0
|
||||
- atlas_monitor_notifier.startswith('/opt/45drives/houston/')
|
||||
- 0 < atlas_monitor_pool_warning_percent | int < atlas_monitor_pool_critical_percent | int < 100
|
||||
- 0 < atlas_monitor_root_warning_percent | int < atlas_monitor_root_critical_percent | int < 100
|
||||
- 0 < atlas_monitor_snapshot_warning_percent | int < atlas_monitor_snapshot_critical_percent | int < 100
|
||||
- atlas_monitor_snapshot_growth_warning_gib_day | int > 0
|
||||
- atlas_monitor_backup_growth_warning_gib_day | int > 0
|
||||
- 0 < atlas_monitor_cpu_warning_c | int < atlas_monitor_cpu_critical_c | int
|
||||
- atlas_monitor_borg_max_runtime_days | int > 0
|
||||
fail_msg: >-
|
||||
Atlas health monitoring needs real devices, job units, a valid calendar,
|
||||
positive ordered thresholds, and the existing Houston notifier.
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Validate monitored Atlas SMART devices
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- item.name is match('^[a-z0-9][a-z0-9_-]*$')
|
||||
- item.path.startswith('/dev/disk/by-id/')
|
||||
- 0 < item.warning_c | int < item.critical_c | int
|
||||
fail_msg: "Every monitored disk needs a stable by-id path and ordered temperature thresholds."
|
||||
loop: "{{ atlas_monitor_smart_devices }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Validate monitored Atlas timer names and age thresholds
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- item.name is match('^[a-zA-Z0-9@_.-]+\\.timer$')
|
||||
- item.max_age_hours | int >= 0
|
||||
loop: "{{ atlas_monitor_timers }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Validate monitored Atlas failure unit names
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- item is match('^[a-zA-Z0-9@_.-]+\\.service$')
|
||||
loop: "{{ atlas_monitor_failure_units }}"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Validate Atlas health monitor calendar
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.command:
|
||||
argv: [systemd-analyze, calendar, "{{ atlas_monitor_calendar }}"]
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Install SMART tooling for Atlas health checks
|
||||
tags: [atlas, monitoring, packages]
|
||||
ansible.builtin.dnf:
|
||||
name: smartmontools
|
||||
state: present
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Inspect the existing 45Drives notifier for monitoring
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.stat:
|
||||
path: "{{ atlas_monitor_notifier }}"
|
||||
register: atlas_monitor_notifier_file
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Require the existing 45Drives notifier for monitoring
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_monitor_notifier_file.stat.executable | default(false)
|
||||
fail_msg: "The existing 45Drives Houston notifier must be executable."
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Create private Atlas health monitor state directory
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.file:
|
||||
path: /var/lib/atlas-health-monitor
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0700"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Install Atlas health monitor configuration
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.template:
|
||||
src: atlas-health-monitor.json.j2
|
||||
dest: /etc/atlas-health-monitor.json
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0600"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Install Atlas health monitor helper
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.copy:
|
||||
src: atlas-health-monitor.py
|
||||
dest: /usr/local/libexec/atlas-health-monitor
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Install Atlas health monitoring units
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.template:
|
||||
src: "{{ item }}.j2"
|
||||
dest: "/etc/systemd/system/{{ item }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
loop:
|
||||
- atlas-health-monitor.service
|
||||
- atlas-health-monitor.timer
|
||||
- atlas-monitor-failure@.service
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Create failure hook directories for monitored Atlas jobs
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.file:
|
||||
path: "/etc/systemd/system/{{ item }}.d"
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
loop: "{{ atlas_monitor_failure_units }}"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Notify 45Drives Alerts when an Atlas job fails
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.template:
|
||||
src: atlas-monitor-failure.conf.j2
|
||||
dest: "/etc/systemd/system/{{ item }}.d/atlas-monitor.conf"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
loop: "{{ atlas_monitor_failure_units }}"
|
||||
when: atlas_manage_monitoring | bool
|
||||
|
||||
- name: Reload systemd after installing Atlas monitoring
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.systemd:
|
||||
daemon_reload: true
|
||||
when:
|
||||
- atlas_manage_monitoring | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Enable the Atlas health monitoring timer
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.systemd:
|
||||
name: atlas-health-monitor.timer
|
||||
enabled: true
|
||||
state: started
|
||||
when:
|
||||
- atlas_manage_monitoring | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Validate the deployed Atlas health monitoring units
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemd-analyze
|
||||
- verify
|
||||
- atlas-health-monitor.service
|
||||
- atlas-health-monitor.timer
|
||||
- atlas-monitor-failure@.service
|
||||
changed_when: false
|
||||
when:
|
||||
- atlas_manage_monitoring | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Probe Atlas health without sending notifications
|
||||
tags: [atlas, monitoring]
|
||||
ansible.builtin.command:
|
||||
argv: [/usr/local/libexec/atlas-health-monitor, --dry-run]
|
||||
register: atlas_monitor_dry_run
|
||||
changed_when: false
|
||||
when:
|
||||
- atlas_manage_monitoring | bool
|
||||
- not ansible_check_mode
|
||||
@@ -16,7 +16,6 @@
|
||||
- name: Inspect declared Atlas pool disks
|
||||
ansible.builtin.stat:
|
||||
path: "{{ item }}"
|
||||
follow: true
|
||||
loop: "{{ atlas_zpool_disks }}"
|
||||
loop_control:
|
||||
label: "{{ item }}"
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
---
|
||||
- name: Validate requested Atlas post-restore relabel paths
|
||||
tags: [atlas, restorecon, recovery]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- item is string
|
||||
- item.startswith(atlas_mount_root ~ '/')
|
||||
- item != atlas_mount_root
|
||||
fail_msg: >-
|
||||
Post-restore relabeling accepts only explicit paths below the Atlas pool
|
||||
mount root. Do not relabel the whole pool during routine provisioning.
|
||||
loop: "{{ atlas_restorecon_paths }}"
|
||||
when: atlas_restorecon_paths | length > 0
|
||||
|
||||
- name: Restore SELinux labels on explicitly restored Atlas paths
|
||||
tags: [atlas, restorecon, recovery]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- restorecon
|
||||
- -RFv
|
||||
- "{{ item }}"
|
||||
register: atlas_restorecon_result
|
||||
changed_when: atlas_restorecon_result.stdout | length > 0
|
||||
loop: "{{ atlas_restorecon_paths }}"
|
||||
when:
|
||||
- atlas_restorecon_paths | length > 0
|
||||
- not ansible_check_mode
|
||||
@@ -56,16 +56,6 @@
|
||||
notify: Reload NFS exports
|
||||
when: atlas_manage_sharing | bool
|
||||
|
||||
- name: Ensure Atlas NFS configuration drop-in directory exists
|
||||
tags: [atlas, sharing]
|
||||
ansible.builtin.file:
|
||||
path: /etc/nfs.conf.d
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: atlas_manage_sharing | bool
|
||||
|
||||
- name: Configure Atlas NFSv4-only service
|
||||
tags: [atlas, sharing]
|
||||
ansible.builtin.template:
|
||||
@@ -224,22 +214,9 @@
|
||||
- atlas_manage_sharing | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Check active Atlas file-sharing services
|
||||
- name: Read Atlas file-sharing service state
|
||||
tags: [atlas, sharing, services, security]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemctl
|
||||
- is-active
|
||||
- --quiet
|
||||
- "{{ item }}"
|
||||
loop:
|
||||
- nfs-server.service
|
||||
- smb.service
|
||||
loop_control:
|
||||
label: "{{ item }}"
|
||||
register: atlas_file_sharing_service_activity
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
ansible.builtin.service_facts:
|
||||
when:
|
||||
- atlas_manage_sharing | bool
|
||||
- not ansible_check_mode
|
||||
@@ -248,7 +225,10 @@
|
||||
tags: [atlas, sharing, services, security]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_file_sharing_service_activity.results | map(attribute='rc') | list == [0, 0]
|
||||
- ansible_facts.services['nfs-server.service'] is defined
|
||||
- ansible_facts.services['nfs-server.service'].state == 'running'
|
||||
- ansible_facts.services['smb.service'] is defined
|
||||
- ansible_facts.services['smb.service'].state == 'running'
|
||||
fail_msg: Atlas NFSv4 or SMB3 did not start after its managed configuration was applied.
|
||||
when:
|
||||
- atlas_manage_sharing | bool
|
||||
|
||||
@@ -3,6 +3,12 @@
|
||||
tags: [atlas, storage]
|
||||
ansible.builtin.set_fact:
|
||||
atlas_zfs_datasets:
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_work }}"
|
||||
mountpoint: "{{ atlas_work_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
mountpoint: "{{ atlas_work_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_archive }}"
|
||||
mountpoint: "{{ atlas_archive_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
@@ -10,13 +16,6 @@
|
||||
properties:
|
||||
compression: zstd
|
||||
mountpoint: "{{ atlas_archive_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_services }}"
|
||||
mountpoint: "{{ atlas_services_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
compression: zstd
|
||||
mountpoint: "{{ atlas_services_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_app_data }}"
|
||||
mountpoint: "{{ atlas_app_data_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
@@ -52,27 +51,11 @@
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_music_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_backup }}"
|
||||
mountpoint: "{{ atlas_backup_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_backup_mountpoint }}"
|
||||
reservation: "{{ atlas_zfs_backup_reservation }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_host_backups }}"
|
||||
mountpoint: "{{ atlas_host_backups_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_host_backups_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_backup_prometheus }}"
|
||||
mountpoint: "{{ atlas_backup_prometheus_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_backup_prometheus_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_photobook }}"
|
||||
mountpoint: "{{ atlas_photobook_mountpoint }}"
|
||||
@@ -81,6 +64,21 @@
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_photobook_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_backups }}"
|
||||
mountpoint: "{{ atlas_backups_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_backups_mountpoint }}"
|
||||
- name: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_service_backups }}"
|
||||
mountpoint: "{{ atlas_service_backups_mountpoint }}"
|
||||
owner: "{{ atlas_admin_username }}"
|
||||
group: "{{ atlas_admin_group }}"
|
||||
properties:
|
||||
compression: lz4
|
||||
mountpoint: "{{ atlas_service_backups_mountpoint }}"
|
||||
refreservation: "{{ atlas_zfs_service_backups_refreservation }}"
|
||||
|
||||
- name: Require completed Atlas storage placeholders
|
||||
tags: [atlas, storage]
|
||||
|
||||
@@ -1,145 +0,0 @@
|
||||
---
|
||||
- name: Validate Atlas offline USB backup configuration
|
||||
tags: [atlas, storage, backup, usb_backup]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_manage_storage | bool
|
||||
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
|
||||
- atlas_mount_root.startswith('/')
|
||||
- atlas_usb_backup_luks_uuid is match('^[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}$')
|
||||
- atlas_usb_backup_fs_uuid is match('^[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}$')
|
||||
- atlas_usb_backup_luks_uuid != atlas_usb_backup_fs_uuid
|
||||
- atlas_usb_backup_mapper_name is match('^[a-z][a-z0-9_-]*$')
|
||||
- atlas_usb_backup_min_free_bytes | int > 0
|
||||
- atlas_usb_backup_snapshot_prefix is match('^[a-z0-9][a-z0-9_-]*$')
|
||||
- atlas_usb_backup_snapshot_prefix != atlas_borg_snapshot_prefix
|
||||
- atlas_usb_backup_snapshot_prefix != atlas_zfs_snapshot_prefix
|
||||
fail_msg: >-
|
||||
The manual Atlas USB backup needs verified LUKS and ext4 UUIDs, a safe
|
||||
mapper name, positive free-space reserve, and a unique snapshot prefix.
|
||||
when: atlas_manage_usb_backup | bool
|
||||
|
||||
- name: Install rsync for the Atlas offline USB backup
|
||||
tags: [atlas, storage, backup, usb_backup]
|
||||
ansible.builtin.dnf:
|
||||
name: rsync
|
||||
state: present
|
||||
when: atlas_manage_usb_backup | bool
|
||||
|
||||
- name: Install the manual Atlas offline USB backup helper
|
||||
tags: [atlas, storage, backup, usb_backup]
|
||||
ansible.builtin.template:
|
||||
src: atlas-usb-backup.sh.j2
|
||||
dest: /usr/local/sbin/atlas-usb-backup
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_usb_backup | bool
|
||||
|
||||
- name: Install the Atlas USB snapshot cleanup helper
|
||||
tags: [atlas, storage, backup, usb_backup]
|
||||
ansible.builtin.template:
|
||||
src: atlas-usb-snapshot-cleanup.sh.j2
|
||||
dest: /usr/local/sbin/atlas-usb-snapshot-cleanup
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_usb_backup | bool
|
||||
|
||||
- name: Install the manual Atlas offline USB backup service
|
||||
tags: [atlas, storage, backup, usb_backup]
|
||||
ansible.builtin.template:
|
||||
src: atlas-usb-backup.service.j2
|
||||
dest: /etc/systemd/system/atlas-usb-backup.service
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
when: atlas_manage_usb_backup | bool
|
||||
|
||||
- name: Reload systemd for the Atlas offline USB backup service
|
||||
tags: [atlas, storage, backup, usb_backup]
|
||||
ansible.builtin.systemd:
|
||||
daemon_reload: true
|
||||
when:
|
||||
- atlas_manage_usb_backup | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Validate the 45Drives Atlas USB reminder configuration
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_manage_usb_backup | bool
|
||||
- atlas_usb_reminder_calendar | length > 0
|
||||
- atlas_usb_reminder_notifier.startswith('/opt/45drives/houston/')
|
||||
fail_msg: >-
|
||||
Enable the manual USB backup and declare a systemd calendar before
|
||||
enabling its 45Drives Alerts reminder.
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Validate the Atlas USB reminder calendar
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemd-analyze
|
||||
- calendar
|
||||
- "{{ atlas_usb_reminder_calendar }}"
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Inspect the existing 45Drives notifier
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.stat:
|
||||
path: "{{ atlas_usb_reminder_notifier }}"
|
||||
register: atlas_usb_reminder_notifier_file
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Require the configured 45Drives notifier for USB reminders
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_usb_reminder_notifier_file.stat.executable | default(false)
|
||||
fail_msg: >-
|
||||
The existing 45Drives Houston notifier must be executable.
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Install the 45Drives Atlas USB reminder helper
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.template:
|
||||
src: atlas-usb-reminder.py.j2
|
||||
dest: /usr/local/libexec/atlas-usb-reminder
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Install the 45Drives Atlas USB reminder service
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.template:
|
||||
src: atlas-usb-reminder.service.j2
|
||||
dest: /etc/systemd/system/atlas-usb-reminder.service
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Install the 45Drives Atlas USB reminder timer
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.template:
|
||||
src: atlas-usb-reminder.timer.j2
|
||||
dest: /etc/systemd/system/atlas-usb-reminder.timer
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
when: atlas_manage_usb_reminder | bool
|
||||
|
||||
- name: Enable only the Atlas USB notification reminder timer
|
||||
tags: [atlas, backup, usb_reminder]
|
||||
ansible.builtin.systemd:
|
||||
name: atlas-usb-reminder.timer
|
||||
enabled: true
|
||||
state: started
|
||||
daemon_reload: true
|
||||
when:
|
||||
- atlas_manage_usb_reminder | bool
|
||||
- not ansible_check_mode
|
||||
@@ -1,175 +0,0 @@
|
||||
---
|
||||
- name: Validate Atlas ZFS snapshot policy
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_manage_storage | bool
|
||||
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
|
||||
- atlas_zfs_snapshot_prefix is match('^[a-z0-9][a-z0-9_-]*$')
|
||||
- atlas_zfs_snapshot_policies | length > 0
|
||||
- >-
|
||||
(atlas_zfs_snapshot_policies | map(attribute='name') | unique | list | length)
|
||||
== (atlas_zfs_snapshot_policies | length)
|
||||
fail_msg: >-
|
||||
Enable Atlas storage and declare a non-empty snapshot policy with a safe
|
||||
prefix and unique policy names before managing automatic snapshots.
|
||||
when: atlas_manage_zfs_snapshots | bool
|
||||
|
||||
- name: Validate Atlas ZFS snapshot policy entries
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- item.name is match('^[a-z][a-z0-9_-]*$')
|
||||
- item.keep | int > 0
|
||||
- item.calendar | length > 0
|
||||
fail_msg: >-
|
||||
Every Atlas snapshot policy needs a safe name, a positive retention
|
||||
count, and a systemd calendar expression.
|
||||
loop: "{{ atlas_zfs_snapshot_policies }}"
|
||||
loop_control:
|
||||
label: "{{ item.name | default('unnamed') }}"
|
||||
when: atlas_manage_zfs_snapshots | bool
|
||||
|
||||
- name: Validate Atlas ZFS snapshot calendars
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemd-analyze
|
||||
- calendar
|
||||
- "{{ item.calendar }}"
|
||||
loop: "{{ atlas_zfs_snapshot_policies }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}: {{ item.calendar }}"
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when: atlas_manage_zfs_snapshots | bool
|
||||
|
||||
- name: Install Atlas ZFS snapshot and retention helper
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.template:
|
||||
src: atlas-zfs-snapshot.sh.j2
|
||||
dest: /usr/local/sbin/atlas-zfs-snapshot
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0750"
|
||||
when: atlas_manage_zfs_snapshots | bool
|
||||
|
||||
- name: Install Atlas ZFS snapshot systemd service
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.template:
|
||||
src: atlas-zfs-snapshot@.service.j2
|
||||
dest: /etc/systemd/system/atlas-zfs-snapshot@.service
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
when: atlas_manage_zfs_snapshots | bool
|
||||
|
||||
- name: Install Atlas ZFS snapshot systemd timers
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.template:
|
||||
src: atlas-zfs-snapshot.timer.j2
|
||||
dest: "/etc/systemd/system/atlas-zfs-snapshot-{{ item.name }}.timer"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
loop: "{{ atlas_zfs_snapshot_policies }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
when: atlas_manage_zfs_snapshots | bool
|
||||
|
||||
- name: Enable Atlas ZFS snapshot systemd timers
|
||||
tags: [atlas, storage, snapshots]
|
||||
ansible.builtin.systemd:
|
||||
name: "atlas-zfs-snapshot-{{ item.name }}.timer"
|
||||
enabled: true
|
||||
state: started
|
||||
daemon_reload: true
|
||||
loop: "{{ atlas_zfs_snapshot_policies }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
when:
|
||||
- atlas_manage_zfs_snapshots | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Validate Atlas ZFS scrub policy
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- atlas_manage_storage | bool
|
||||
- atlas_zfs_pool != 'CHANGEME_ZFS_POOL'
|
||||
- atlas_zfs_scrub_calendar | length > 0
|
||||
fail_msg: >-
|
||||
Enable Atlas storage and declare a systemd calendar expression before
|
||||
managing periodic ZFS scrubs.
|
||||
when: atlas_manage_zfs_scrub | bool
|
||||
|
||||
- name: Validate Atlas ZFS scrub calendar
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemd-analyze
|
||||
- calendar
|
||||
- "{{ atlas_zfs_scrub_calendar }}"
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when: atlas_manage_zfs_scrub | bool
|
||||
|
||||
- name: Require OpenZFS scrub systemd units
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- systemctl
|
||||
- cat
|
||||
- "{{ item }}"
|
||||
loop:
|
||||
- "zfs-scrub@{{ atlas_zfs_pool }}.service"
|
||||
- "zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer"
|
||||
- "zfs-scrub-weekly@{{ atlas_zfs_pool }}.timer"
|
||||
loop_control:
|
||||
label: "{{ item }}"
|
||||
changed_when: false
|
||||
check_mode: false
|
||||
when: atlas_manage_zfs_scrub | bool
|
||||
|
||||
- name: Create Atlas ZFS scrub timer override directory
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.file:
|
||||
path: "/etc/systemd/system/zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer.d"
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: atlas_manage_zfs_scrub | bool
|
||||
|
||||
- name: Configure Atlas ZFS monthly scrub schedule
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.template:
|
||||
src: atlas-zfs-scrub-timer.conf.j2
|
||||
dest: >-
|
||||
/etc/systemd/system/zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer.d/override.conf
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
when: atlas_manage_zfs_scrub | bool
|
||||
|
||||
- name: Disable the conflicting weekly OpenZFS scrub timer
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.systemd:
|
||||
name: "zfs-scrub-weekly@{{ atlas_zfs_pool }}.timer"
|
||||
enabled: false
|
||||
state: stopped
|
||||
daemon_reload: true
|
||||
when:
|
||||
- atlas_manage_zfs_scrub | bool
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Enable the Atlas monthly OpenZFS scrub timer
|
||||
tags: [atlas, storage, scrub]
|
||||
ansible.builtin.systemd:
|
||||
name: "zfs-scrub-monthly@{{ atlas_zfs_pool }}.timer"
|
||||
enabled: true
|
||||
state: started
|
||||
daemon_reload: true
|
||||
when:
|
||||
- atlas_manage_zfs_scrub | bool
|
||||
- not ansible_check_mode
|
||||
@@ -4,12 +4,7 @@ PubkeyAuthentication yes
|
||||
PasswordAuthentication no
|
||||
KbdInteractiveAuthentication no
|
||||
AuthenticationMethods publickey
|
||||
AuthorizedKeysFile {{
|
||||
atlas_admin_ssh_keys
|
||||
| map(attribute='name')
|
||||
| map('regex_replace', '^', '%h/.ssh/authorized_keys.d/')
|
||||
| join(' ')
|
||||
}}
|
||||
AuthorizedKeysFile {% for key in atlas_admin_ssh_keys %}%h/.ssh/authorized_keys.d/{{ key.name }}{% if not loop.last %} {% endif %}{% endfor %}
|
||||
X11Forwarding no
|
||||
AllowTcpForwarding {{ atlas_ssh_allow_tcp_forwarding }}
|
||||
AllowAgentForwarding no
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
[Unit]
|
||||
Description=Back up Atlas ZFS datasets to the encrypted Borg repository
|
||||
Documentation=man:borg-create(1) man:borg-prune(1) man:borg-compact(1)
|
||||
Requires=zfs.target
|
||||
Wants=network-online.target
|
||||
After=zfs.target network-online.target
|
||||
StartLimitIntervalSec=6h
|
||||
StartLimitBurst=3
|
||||
ConditionFileIsExecutable=/usr/local/sbin/atlas-borg-backup
|
||||
ConditionPathExists={{ atlas_borg_passphrase_path }}
|
||||
ConditionPathExists={{ atlas_borg_ssh_private_key_path }}
|
||||
ConditionPathExists={{ atlas_borg_known_hosts_path }}
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/atlas-borg-backup
|
||||
ExecStopPost=+/usr/local/sbin/atlas-borg-snapshot-cleanup
|
||||
User=root
|
||||
Group=root
|
||||
UMask=0077
|
||||
SuccessExitStatus=1
|
||||
Restart=on-failure
|
||||
RestartSec=30m
|
||||
TimeoutStartSec=infinity
|
||||
RuntimeDirectory=atlas-borg
|
||||
RuntimeDirectoryMode=0750
|
||||
Nice=15
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
NoNewPrivileges=true
|
||||
PrivateMounts=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths={{ atlas_borg_cache_dir }} {{ atlas_borg_config_dir }} /run/atlas-borg /run/lock
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -1,212 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
export LC_ALL=C.utf8
|
||||
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
export BORG_CACHE_DIR={{ atlas_borg_cache_dir | quote }}
|
||||
export BORG_CONFIG_DIR={{ atlas_borg_config_dir | quote }}
|
||||
export BORG_PASSCOMMAND={{ ('cat ' ~ atlas_borg_passphrase_path) | quote }}
|
||||
export BORG_RSH={{ atlas_borg_ssh_wrapper_path | quote }}
|
||||
|
||||
readonly pool={{ atlas_zfs_pool | quote }}
|
||||
readonly mount_root={{ atlas_mount_root | quote }}
|
||||
readonly repository={{ ('ssh://' ~ atlas_borg_repository_user ~ '@' ~ atlas_borg_repository_host
|
||||
~ ':' ~ (atlas_borg_repository_port | string) ~ '/' ~ atlas_borg_repository_path) | quote }}
|
||||
readonly remote_path={{ atlas_borg_remote_path | quote }}
|
||||
readonly archive_prefix={{ atlas_borg_archive_prefix | quote }}
|
||||
readonly snapshot_prefix={{ atlas_borg_snapshot_prefix | quote }}
|
||||
readonly compression={{ atlas_borg_compression | quote }}
|
||||
readonly stage=/run/atlas-borg/source
|
||||
readonly snapshot_marker=/run/atlas-borg/snapshot-name
|
||||
readonly borg_user={{ atlas_borg_username | quote }}
|
||||
readonly borg_group={{ atlas_borg_group | quote }}
|
||||
readonly borg_home={{ atlas_borg_home | quote }}
|
||||
readonly borg_lock={{ atlas_borg_lock_path | quote }}
|
||||
readonly progress_filter=/usr/local/libexec/atlas-borg-progress
|
||||
|
||||
snapshot_name=""
|
||||
mounted_targets=()
|
||||
|
||||
# Invoked through the EXIT trap below.
|
||||
# shellcheck disable=SC2329
|
||||
cleanup() {
|
||||
local status=$?
|
||||
local cleanup_status=0
|
||||
local index
|
||||
local source_mount_failed=false
|
||||
trap - EXIT HUP INT TERM
|
||||
set +e
|
||||
|
||||
{% raw %}
|
||||
for ((index = ${#mounted_targets[@]} - 1; index >= 0; index--)); do
|
||||
{% endraw %}
|
||||
if mountpoint -q "${mounted_targets[$index]}" && ! umount -R "${mounted_targets[$index]}"; then
|
||||
printf 'Source snapshot mount cleanup failed: %s\n' "${mounted_targets[$index]}" >&2
|
||||
source_mount_failed=true
|
||||
fi
|
||||
if mountpoint -q "${mounted_targets[$index]}"; then
|
||||
printf 'Source snapshot mount is still active: %s\n' "${mounted_targets[$index]}" >&2
|
||||
source_mount_failed=true
|
||||
else
|
||||
rmdir -- "${mounted_targets[$index]}" 2>/dev/null || true
|
||||
fi
|
||||
done
|
||||
if [[ "$source_mount_failed" == false ]]; then
|
||||
if [[ -d "$stage" ]]; then
|
||||
rmdir -- "$stage" 2>/dev/null || cleanup_status=2
|
||||
fi
|
||||
else
|
||||
cleanup_status=2
|
||||
printf 'Source bind mount cleanup failed; keeping the snapshot for recovery\n' >&2
|
||||
fi
|
||||
|
||||
if ((status == 0 && cleanup_status != 0)); then
|
||||
status=$cleanup_status
|
||||
fi
|
||||
exit "$status"
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
trap 'exit 143' HUP INT TERM
|
||||
|
||||
run_as_borg() {
|
||||
setpriv \
|
||||
--reuid "$borg_user" \
|
||||
--regid "$borg_group" \
|
||||
--clear-groups \
|
||||
--inh-caps=-all,+dac_read_search \
|
||||
--ambient-caps=-all,+dac_read_search \
|
||||
--bounding-set=-all,+dac_read_search \
|
||||
-- env HOME="$borg_home" USER="$borg_user" LOGNAME="$borg_user" "$@"
|
||||
}
|
||||
|
||||
exec 8>"$borg_lock"
|
||||
flock 8
|
||||
exec 9>/run/lock/atlas-zfs-snapshot.lock
|
||||
|
||||
zpool list -H -o name "$pool" >/dev/null
|
||||
rm -rf "$stage"
|
||||
mkdir -p "$stage"
|
||||
chown root:"$borg_group" /run/atlas-borg "$stage"
|
||||
chmod 0750 /run/atlas-borg "$stage"
|
||||
|
||||
flock 9
|
||||
while IFS= read -r stale_snapshot; do
|
||||
stale_suffix="${stale_snapshot#"${pool}@${snapshot_prefix}-"}"
|
||||
if [[ "$stale_suffix" =~ ^[0-9]{8}T[0-9]{6}Z$ ]]; then
|
||||
zfs destroy -r "$stale_snapshot"
|
||||
printf 'Removed stale Borg source snapshot %s\n' "$stale_snapshot"
|
||||
fi
|
||||
done < <(
|
||||
zfs list -H -t snapshot -o name -r "$pool" |
|
||||
grep -E "^${pool}@${snapshot_prefix}-[0-9]{8}T[0-9]{6}Z$" || true
|
||||
)
|
||||
|
||||
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
readonly timestamp
|
||||
snapshot_name="${snapshot_prefix}-${timestamp}"
|
||||
readonly snapshot_name
|
||||
printf '%s\n' "$snapshot_name" >"$snapshot_marker"
|
||||
zfs snapshot -r "${pool}@${snapshot_name}"
|
||||
flock -u 9
|
||||
printf 'Created recursive Borg source snapshot %s@%s\n' "$pool" "$snapshot_name"
|
||||
|
||||
while IFS=$'\t' read -r dataset dataset_mountpoint mounted; do
|
||||
if [[ "$mounted" != yes ]]; then
|
||||
printf 'Dataset %s is not mounted; refusing an incomplete backup\n' "$dataset" >&2
|
||||
exit 65
|
||||
fi
|
||||
if [[ "$dataset_mountpoint" != "$mount_root" && "$dataset_mountpoint" != "$mount_root/"* ]]; then
|
||||
printf 'Dataset %s has unexpected mountpoint %s\n' "$dataset" "$dataset_mountpoint" >&2
|
||||
exit 65
|
||||
fi
|
||||
|
||||
dataset_suffix="${dataset#"$pool"}"
|
||||
source_path="${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}"
|
||||
target_path="${stage}${dataset_suffix}"
|
||||
mkdir -p "$target_path"
|
||||
mount --bind "$source_path" "$target_path"
|
||||
mounted_targets+=("$target_path")
|
||||
mount -o remount,bind,ro "$target_path"
|
||||
done < <(zfs list -H -o name,mountpoint,mounted -s name -r "$pool")
|
||||
|
||||
estimated_source_bytes=0
|
||||
while IFS=$'\t' read -r source_snapshot logical_bytes; do
|
||||
if [[ "$source_snapshot" == *"@${snapshot_name}" ]]; then
|
||||
[[ "$logical_bytes" =~ ^[0-9]+$ ]] || {
|
||||
printf 'Invalid logical size for Borg source snapshot %s\n' "$source_snapshot" >&2
|
||||
exit 74
|
||||
}
|
||||
estimated_source_bytes=$((estimated_source_bytes + logical_bytes))
|
||||
fi
|
||||
done < <(zfs list -H -p -t snapshot -o name,logicalreferenced -r "$pool")
|
||||
((estimated_source_bytes > 0)) || {
|
||||
printf 'Could not estimate the Borg source snapshot size\n' >&2
|
||||
exit 74
|
||||
}
|
||||
printf 'Estimated Borg source logical size: %s bytes (ZFS; progress percentage is approximate)\n' \
|
||||
"$estimated_source_bytes"
|
||||
|
||||
archive="${archive_prefix}-${timestamp}"
|
||||
readonly archive
|
||||
borg_status=0
|
||||
|
||||
printf 'Starting Borg archive %s from snapshot %s@%s\n' "$archive" "$pool" "$snapshot_name"
|
||||
set +e
|
||||
(
|
||||
cd /run/atlas-borg
|
||||
run_as_borg borg --remote-path "$remote_path" --lock-wait 600 --log-json --progress create \
|
||||
--show-rc \
|
||||
--stats \
|
||||
--checkpoint-interval 900 \
|
||||
--compression "$compression" \
|
||||
"${repository}::${archive}" \
|
||||
source 2>&1
|
||||
) | /usr/bin/python3 -u "$progress_filter" --estimated-total-bytes "$estimated_source_bytes"
|
||||
create_pipeline_status=("${PIPESTATUS[@]}")
|
||||
set -e
|
||||
create_status=${create_pipeline_status[0]}
|
||||
if ((create_pipeline_status[1] != 0)); then
|
||||
printf 'Borg progress logging failed with status %s\n' "${create_pipeline_status[1]}" >&2
|
||||
exit 2
|
||||
fi
|
||||
if ((create_status >= 2)); then
|
||||
exit "$create_status"
|
||||
fi
|
||||
borg_status=$create_status
|
||||
|
||||
printf 'Borg archive %s created; applying retention\n' "$archive"
|
||||
set +e
|
||||
run_as_borg borg --remote-path "$remote_path" --lock-wait 600 prune \
|
||||
--show-rc \
|
||||
--list \
|
||||
--glob-archives "${archive_prefix}-*" \
|
||||
--keep-daily {{ atlas_borg_keep_daily | int }} \
|
||||
--keep-weekly {{ atlas_borg_keep_weekly | int }} \
|
||||
--keep-monthly {{ atlas_borg_keep_monthly | int }} \
|
||||
"$repository"
|
||||
prune_status=$?
|
||||
set -e
|
||||
if ((prune_status >= 2)); then
|
||||
exit "$prune_status"
|
||||
fi
|
||||
if ((prune_status > borg_status)); then
|
||||
borg_status=$prune_status
|
||||
fi
|
||||
|
||||
printf 'Borg retention complete; compacting repository\n'
|
||||
set +e
|
||||
run_as_borg borg --remote-path "$remote_path" --lock-wait 600 compact \
|
||||
--show-rc \
|
||||
"$repository"
|
||||
compact_status=$?
|
||||
set -e
|
||||
if ((compact_status >= 2)); then
|
||||
exit "$compact_status"
|
||||
fi
|
||||
if ((compact_status > borg_status)); then
|
||||
borg_status=$compact_status
|
||||
fi
|
||||
|
||||
printf 'Borg backup %s completed with status %s\n' "$archive" "$borg_status"
|
||||
exit "$borg_status"
|
||||
@@ -1,12 +0,0 @@
|
||||
[Unit]
|
||||
Description=Schedule the encrypted Atlas Borg backup
|
||||
|
||||
[Timer]
|
||||
OnCalendar={{ atlas_borg_backup_calendar }}
|
||||
Persistent=true
|
||||
RandomizedDelaySec={{ atlas_borg_randomized_delay }}
|
||||
AccuracySec=1min
|
||||
Unit=atlas-borg-backup.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,32 +0,0 @@
|
||||
[Unit]
|
||||
Description=Check the encrypted Atlas Borg repository
|
||||
Documentation=man:borg-check(1)
|
||||
Wants=network-online.target
|
||||
After=network-online.target atlas-borg-backup.service
|
||||
ConditionFileIsExecutable=/usr/local/sbin/atlas-borg-check
|
||||
ConditionPathExists={{ atlas_borg_passphrase_path }}
|
||||
ConditionPathExists={{ atlas_borg_ssh_private_key_path }}
|
||||
ConditionPathExists={{ atlas_borg_known_hosts_path }}
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/atlas-borg-check
|
||||
User={{ atlas_borg_username }}
|
||||
Group={{ atlas_borg_group }}
|
||||
UMask=0077
|
||||
SuccessExitStatus=1
|
||||
TimeoutStartSec=infinity
|
||||
Nice=15
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths={{ atlas_borg_cache_dir }} {{ atlas_borg_config_dir }}
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
export LC_ALL=C
|
||||
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
export BORG_CACHE_DIR={{ atlas_borg_cache_dir | quote }}
|
||||
export BORG_CONFIG_DIR={{ atlas_borg_config_dir | quote }}
|
||||
export BORG_PASSCOMMAND={{ ('cat ' ~ atlas_borg_passphrase_path) | quote }}
|
||||
export BORG_RSH={{ atlas_borg_ssh_wrapper_path | quote }}
|
||||
|
||||
readonly repository={{ ('ssh://' ~ atlas_borg_repository_user ~ '@' ~ atlas_borg_repository_host
|
||||
~ ':' ~ (atlas_borg_repository_port | string) ~ '/' ~ atlas_borg_repository_path) | quote }}
|
||||
readonly remote_path={{ atlas_borg_remote_path | quote }}
|
||||
readonly archive_prefix={{ atlas_borg_archive_prefix | quote }}
|
||||
readonly borg_lock={{ atlas_borg_lock_path | quote }}
|
||||
|
||||
exec 8>"$borg_lock"
|
||||
flock 8
|
||||
|
||||
exec borg --remote-path "$remote_path" --lock-wait 600 check \
|
||||
--show-rc \
|
||||
--glob-archives "${archive_prefix}-*" \
|
||||
"$repository"
|
||||
@@ -1,12 +0,0 @@
|
||||
[Unit]
|
||||
Description=Schedule checks of the encrypted Atlas Borg repository
|
||||
|
||||
[Timer]
|
||||
OnCalendar={{ atlas_borg_check_calendar }}
|
||||
Persistent=true
|
||||
RandomizedDelaySec={{ atlas_borg_randomized_delay }}
|
||||
AccuracySec=1min
|
||||
Unit=atlas-borg-check.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,51 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
readonly pool={{ atlas_zfs_pool | quote }}
|
||||
readonly mount_root={{ atlas_mount_root | quote }}
|
||||
readonly snapshot_prefix={{ atlas_borg_snapshot_prefix | quote }}
|
||||
readonly marker=/run/atlas-borg/snapshot-name
|
||||
|
||||
[[ -e "$marker" ]] || exit 0
|
||||
[[ -f "$marker" && ! -L "$marker" ]] || {
|
||||
printf 'Unsafe Atlas Borg snapshot marker; leaving snapshots unchanged\n' >&2
|
||||
exit 2
|
||||
}
|
||||
IFS= read -r snapshot_name <"$marker"
|
||||
[[ "$snapshot_name" =~ ^${snapshot_prefix}-[0-9]{8}T[0-9]{6}Z$ ]] || {
|
||||
printf 'Invalid Atlas Borg snapshot marker; leaving snapshots unchanged\n' >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
exec 9>/run/lock/atlas-zfs-snapshot.lock
|
||||
flock 9
|
||||
if zfs list -H -t snapshot -o name "${pool}@${snapshot_name}" >/dev/null 2>&1; then
|
||||
# The private bind mounts are gone, but ZFS may leave its on-demand
|
||||
# .zfs/snapshot mounts in the host namespace until explicitly unmounted.
|
||||
snapshot_mounts=()
|
||||
snapshot_sources=()
|
||||
while IFS=$'\t' read -r dataset dataset_mountpoint; do
|
||||
[[ "$dataset_mountpoint" == "$mount_root" || "$dataset_mountpoint" == "$mount_root/"* ]] || continue
|
||||
snapshot_mounts+=("${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}")
|
||||
snapshot_sources+=("${dataset}@${snapshot_name}")
|
||||
done < <(zfs list -H -o name,mountpoint -s name -r "$pool")
|
||||
|
||||
{% raw %}
|
||||
for ((index = ${#snapshot_mounts[@]} - 1; index >= 0; index--)); do
|
||||
{% endraw %}
|
||||
mounted_source="$(findmnt -rn -M "${snapshot_mounts[$index]}" -o SOURCE || true)"
|
||||
[[ -n "$mounted_source" ]] || continue
|
||||
[[ "$mounted_source" == "${snapshot_sources[$index]}" ]] || {
|
||||
printf 'Unexpected source on Atlas Borg snapshot mount: %s\n' \
|
||||
"${snapshot_mounts[$index]}" >&2
|
||||
exit 2
|
||||
}
|
||||
umount "${snapshot_mounts[$index]}"
|
||||
done
|
||||
|
||||
zfs destroy -r "${pool}@${snapshot_name}"
|
||||
printf 'Removed recursive Atlas Borg source snapshot %s@%s after backup exit\n' \
|
||||
"$pool" "$snapshot_name"
|
||||
fi
|
||||
@@ -1,19 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Borg receives CAP_DAC_READ_SEARCH only for local snapshot traversal. Drop it
|
||||
# before starting the network transport so SSH runs as the plain service user.
|
||||
exec setpriv \
|
||||
--inh-caps=-all \
|
||||
--ambient-caps=-all \
|
||||
-- /usr/bin/ssh \
|
||||
-i {{ atlas_borg_ssh_private_key_path | quote }} \
|
||||
-p {{ atlas_borg_repository_port | int }} \
|
||||
-o BatchMode=yes \
|
||||
-o IdentitiesOnly=yes \
|
||||
-o StrictHostKeyChecking=yes \
|
||||
-o UserKnownHostsFile={{ atlas_borg_known_hosts_path | quote }} \
|
||||
-o ConnectTimeout=30 \
|
||||
-o ServerAliveInterval=60 \
|
||||
-o ServerAliveCountMax=3 \
|
||||
"$@"
|
||||
@@ -1,20 +0,0 @@
|
||||
{
|
||||
"pool": {{ atlas_zfs_pool | to_json }},
|
||||
"backup_dataset": {{ (atlas_zfs_pool ~ '/' ~ atlas_zfs_dataset_backup) | to_json }},
|
||||
"notifier": {{ atlas_monitor_notifier | to_json }},
|
||||
"smart_devices": {{ atlas_monitor_smart_devices | to_json }},
|
||||
"timers": {{ atlas_monitor_timers | to_json }},
|
||||
"failure_units": {{ atlas_monitor_failure_units | to_json }},
|
||||
"remote_capacity": {{ atlas_monitor_remote_capacity | to_json }},
|
||||
"pool_warning_percent": {{ atlas_monitor_pool_warning_percent | int }},
|
||||
"pool_critical_percent": {{ atlas_monitor_pool_critical_percent | int }},
|
||||
"root_warning_percent": {{ atlas_monitor_root_warning_percent | int }},
|
||||
"root_critical_percent": {{ atlas_monitor_root_critical_percent | int }},
|
||||
"snapshot_warning_percent": {{ atlas_monitor_snapshot_warning_percent | int }},
|
||||
"snapshot_critical_percent": {{ atlas_monitor_snapshot_critical_percent | int }},
|
||||
"snapshot_growth_warning_gib_day": {{ atlas_monitor_snapshot_growth_warning_gib_day | int }},
|
||||
"backup_growth_warning_gib_day": {{ atlas_monitor_backup_growth_warning_gib_day | int }},
|
||||
"cpu_warning_c": {{ atlas_monitor_cpu_warning_c | int }},
|
||||
"cpu_critical_c": {{ atlas_monitor_cpu_critical_c | int }},
|
||||
"borg_max_runtime_days": {{ atlas_monitor_borg_max_runtime_days | int }}
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
[Unit]
|
||||
Description=Check Atlas pool, disks, capacity, temperatures and maintenance jobs
|
||||
Wants=houston-dbus.service network-online.target
|
||||
After=zfs.target houston-dbus.service network-online.target
|
||||
ConditionFileIsExecutable=/usr/local/libexec/atlas-health-monitor
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/libexec/atlas-health-monitor
|
||||
User=root
|
||||
Group=root
|
||||
UMask=0077
|
||||
StateDirectory=atlas-health-monitor
|
||||
StateDirectoryMode=0700
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/var/lib/atlas-health-monitor
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -1,11 +0,0 @@
|
||||
[Unit]
|
||||
Description=Schedule Atlas health checks
|
||||
|
||||
[Timer]
|
||||
OnCalendar={{ atlas_monitor_calendar }}
|
||||
Persistent=true
|
||||
RandomizedDelaySec=5min
|
||||
Unit=atlas-health-monitor.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,2 +0,0 @@
|
||||
[Unit]
|
||||
OnFailure=atlas-monitor-failure@%n.service
|
||||
@@ -1,19 +0,0 @@
|
||||
[Unit]
|
||||
Description=Submit a 45Drives Alert for failed Atlas job %I
|
||||
Requires=houston-dbus.service
|
||||
After=houston-dbus.service
|
||||
ConditionFileIsExecutable=/usr/local/libexec/atlas-health-monitor
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/libexec/atlas-health-monitor --job-failed %I
|
||||
User=root
|
||||
Group=root
|
||||
UMask=0077
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
RestrictAddressFamilies=AF_UNIX
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -1,30 +0,0 @@
|
||||
[Unit]
|
||||
Description=Run a manual, UUID-bound offline USB backup of Atlas ZFS datasets
|
||||
Requires=zfs.target
|
||||
After=zfs.target
|
||||
ConditionFileIsExecutable=/usr/local/sbin/atlas-usb-backup
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/atlas-usb-backup
|
||||
ExecStopPost=+/usr/local/sbin/atlas-usb-snapshot-cleanup
|
||||
User=root
|
||||
Group=root
|
||||
UMask=0077
|
||||
TimeoutStartSec=infinity
|
||||
RuntimeDirectory=atlas-usb-backup
|
||||
RuntimeDirectoryMode=0700
|
||||
Nice=15
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
PrivateMounts=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/run/atlas-usb-backup /run/lock
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_UNIX
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -1,242 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
export LC_ALL=C.utf8
|
||||
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
readonly pool={{ atlas_zfs_pool | quote }}
|
||||
readonly mount_root={{ atlas_mount_root | quote }}
|
||||
readonly luks_uuid={{ atlas_usb_backup_luks_uuid | quote }}
|
||||
readonly fs_uuid={{ atlas_usb_backup_fs_uuid | quote }}
|
||||
readonly mapper_name={{ atlas_usb_backup_mapper_name | quote }}
|
||||
readonly snapshot_prefix={{ atlas_usb_backup_snapshot_prefix | quote }}
|
||||
readonly min_free_bytes={{ atlas_usb_backup_min_free_bytes | int }}
|
||||
readonly mapper="/dev/mapper/${mapper_name}"
|
||||
readonly outer="/dev/disk/by-uuid/${luks_uuid}"
|
||||
readonly runtime_dir=/run/atlas-usb-backup
|
||||
readonly snapshot_marker="${runtime_dir}/snapshot-name"
|
||||
readonly source_dir="${runtime_dir}/source"
|
||||
readonly usb_mount="${runtime_dir}/target"
|
||||
readonly backup_root="${usb_mount}/atlas"
|
||||
|
||||
snapshot_name=""
|
||||
mapper_opened_by_script=false
|
||||
usb_mounted=false
|
||||
published=false
|
||||
partial=""
|
||||
mounted_targets=()
|
||||
|
||||
# shellcheck disable=SC2329
|
||||
cleanup() {
|
||||
local status=$?
|
||||
local cleanup_status=0
|
||||
local index
|
||||
local source_mount_failed=false
|
||||
trap - EXIT HUP INT TERM
|
||||
set +e
|
||||
|
||||
if [[ -n "$partial" && "$published" == false && "$usb_mounted" == true ]]; then
|
||||
rm -rf -- "$partial" || cleanup_status=2
|
||||
fi
|
||||
if [[ "$usb_mounted" == true ]]; then
|
||||
umount "$usb_mount" || cleanup_status=2
|
||||
fi
|
||||
|
||||
{% raw %}
|
||||
for ((index = ${#mounted_targets[@]} - 1; index >= 0; index--)); do
|
||||
{% endraw %}
|
||||
if mountpoint -q "${mounted_targets[$index]}" && ! umount -R "${mounted_targets[$index]}"; then
|
||||
printf 'Source snapshot mount cleanup failed: %s\n' "${mounted_targets[$index]}" >&2
|
||||
source_mount_failed=true
|
||||
fi
|
||||
if mountpoint -q "${mounted_targets[$index]}"; then
|
||||
printf 'Source snapshot mount is still active: %s\n' "${mounted_targets[$index]}" >&2
|
||||
source_mount_failed=true
|
||||
else
|
||||
rmdir -- "${mounted_targets[$index]}" 2>/dev/null || true
|
||||
fi
|
||||
done
|
||||
if [[ "$source_mount_failed" == false ]]; then
|
||||
rmdir -- "$source_dir" 2>/dev/null || true
|
||||
else
|
||||
cleanup_status=2
|
||||
printf 'Source bind mount cleanup failed; keeping the snapshot for recovery\n' >&2
|
||||
fi
|
||||
|
||||
if [[ "$usb_mounted" == true || "$mapper_opened_by_script" == true ]] &&
|
||||
! mountpoint -q "$usb_mount" &&
|
||||
! findmnt -rn -S "$mapper" >/dev/null; then
|
||||
cryptsetup close "$mapper_name" || cleanup_status=2
|
||||
fi
|
||||
rmdir -- "$usb_mount" 2>/dev/null || true
|
||||
|
||||
if ((status == 0 && cleanup_status != 0)); then
|
||||
status=$cleanup_status
|
||||
fi
|
||||
exit "$status"
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
trap 'exit 143' HUP INT TERM
|
||||
|
||||
exec 8>/run/lock/atlas-usb-backup.lock
|
||||
flock -n 8 || { printf 'Atlas USB backup is already running\n' >&2; exit 75; }
|
||||
exec 9>/run/lock/atlas-zfs-snapshot.lock
|
||||
|
||||
zpool list -H -o name "$pool" >/dev/null
|
||||
[[ -b "$outer" ]] || { printf 'Configured LUKS UUID is not connected\n' >&2; exit 66; }
|
||||
[[ "$(blkid -s TYPE -o value "$outer")" == crypto_LUKS ]] || {
|
||||
printf 'Configured outer UUID is not a LUKS container\n' >&2
|
||||
exit 65
|
||||
}
|
||||
[[ "$(blkid -s UUID -o value "$outer")" == "$luks_uuid" ]] || exit 65
|
||||
if ! cryptsetup status "$mapper_name" >/dev/null; then
|
||||
printf 'Requesting the LUKS passphrase for the configured USB disk\n'
|
||||
systemd-ask-password -n --no-tty --timeout=300 \
|
||||
--id="atlas-usb-backup:${luks_uuid}" \
|
||||
'Atlas offline USB backup LUKS passphrase:' |
|
||||
cryptsetup open --type luks2 --key-file - "$outer" "$mapper_name"
|
||||
mapper_opened_by_script=true
|
||||
fi
|
||||
backing_device="$(cryptsetup status "$mapper_name" | awk '$1 == "device:" { print $2 }')"
|
||||
[[ -n "$backing_device" && "$(readlink -f "$backing_device")" == "$(readlink -f "$outer")" ]] || {
|
||||
printf 'The unlocked mapper does not belong to the configured LUKS UUID\n' >&2
|
||||
exit 65
|
||||
}
|
||||
[[ "$(blkid -s TYPE -o value "$mapper")" == ext4 ]] || {
|
||||
printf 'The unlocked USB filesystem is not ext4\n' >&2
|
||||
exit 65
|
||||
}
|
||||
[[ "$(blkid -s UUID -o value "$mapper")" == "$fs_uuid" ]] || {
|
||||
printf 'The unlocked USB filesystem UUID does not match\n' >&2
|
||||
exit 65
|
||||
}
|
||||
if findmnt -rn -S "$mapper" >/dev/null; then
|
||||
printf 'The USB filesystem is already mounted elsewhere\n' >&2
|
||||
exit 65
|
||||
fi
|
||||
[[ ! -e "$source_dir" && ! -e "$usb_mount" ]] || {
|
||||
printf 'USB backup staging directories already exist; inspect them manually\n' >&2
|
||||
exit 65
|
||||
}
|
||||
|
||||
mkdir -m 0700 "$usb_mount"
|
||||
mount -t ext4 -o nodev,nosuid,noexec "$mapper" "$usb_mount"
|
||||
usb_mounted=true
|
||||
[[ "$(readlink -f "$(findmnt -nro SOURCE --target "$usb_mount")")" == "$(readlink -f "$mapper")" ]] || {
|
||||
printf 'Mounted USB source does not match the verified mapper\n' >&2
|
||||
exit 65
|
||||
}
|
||||
|
||||
for path in "$backup_root" "$backup_root/snapshots"; do
|
||||
[[ ! -L "$path" ]] || { printf 'Unsafe symlink in USB backup destination\n' >&2; exit 65; }
|
||||
mkdir -p -- "$path"
|
||||
[[ -d "$path" ]] || exit 65
|
||||
chown root:root -- "$path"
|
||||
chmod 0700 -- "$path"
|
||||
done
|
||||
|
||||
free_bytes="$(df -B1 --output=avail "$usb_mount" | tail -n 1 | tr -d ' ')"
|
||||
if ((free_bytes < min_free_bytes)); then
|
||||
printf 'USB free space (%s bytes) is below the required reserve (%s bytes)\n' \
|
||||
"$free_bytes" "$min_free_bytes" >&2
|
||||
exit 73
|
||||
fi
|
||||
|
||||
mkdir -m 0700 "$source_dir"
|
||||
flock 9
|
||||
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
snapshot_name="${snapshot_prefix}-${timestamp}-$$"
|
||||
printf '%s\n' "$snapshot_name" >"$snapshot_marker"
|
||||
zfs snapshot -r "${pool}@${snapshot_name}"
|
||||
flock -u 9
|
||||
printf 'Created recursive USB source snapshot %s@%s\n' "$pool" "$snapshot_name"
|
||||
|
||||
while IFS=$'\t' read -r dataset dataset_mountpoint mounted; do
|
||||
if [[ "$mounted" != yes ]]; then
|
||||
printf 'Dataset %s is not mounted; refusing an incomplete backup\n' "$dataset" >&2
|
||||
exit 65
|
||||
fi
|
||||
if [[ "$dataset_mountpoint" != "$mount_root" && "$dataset_mountpoint" != "$mount_root/"* ]]; then
|
||||
printf 'Dataset %s has unexpected mountpoint %s\n' "$dataset" "$dataset_mountpoint" >&2
|
||||
exit 65
|
||||
fi
|
||||
dataset_suffix="${dataset#"$pool"}"
|
||||
source_path="${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}"
|
||||
target_path="${source_dir}${dataset_suffix}"
|
||||
mkdir -p "$target_path"
|
||||
mount --bind "$source_path" "$target_path"
|
||||
mounted_targets+=("$target_path")
|
||||
mount -o remount,bind,ro "$target_path"
|
||||
done < <(zfs list -H -o name,mountpoint,mounted -s name -r "$pool")
|
||||
|
||||
previous=""
|
||||
if [[ -e "$backup_root/latest" || -L "$backup_root/latest" ]]; then
|
||||
[[ -L "$backup_root/latest" ]] || { printf 'latest is not a symlink\n' >&2; exit 65; }
|
||||
previous="$(readlink -e "$backup_root/latest")"
|
||||
[[ -n "$previous" && "$previous" == "$backup_root/snapshots/"* && -d "$previous" ]] || {
|
||||
printf 'latest does not point to a complete snapshot on the USB disk\n' >&2
|
||||
exit 65
|
||||
}
|
||||
fi
|
||||
|
||||
backup_name="${timestamp}-$$"
|
||||
candidate_partial="${backup_root}/snapshots/.incomplete-${backup_name}"
|
||||
complete="${backup_root}/snapshots/${backup_name}"
|
||||
[[ ! -e "$candidate_partial" && ! -L "$candidate_partial" && ! -e "$complete" && ! -L "$complete" ]] || exit 65
|
||||
mkdir -m 0700 "$candidate_partial"
|
||||
partial="$candidate_partial"
|
||||
|
||||
printf 'Copying the consistent pool tree to USB backup %s\n' "$backup_name"
|
||||
# Preserve POSIX ACLs, ownership, modes, timestamps, hard links, and sparse
|
||||
# files. Do not preserve generic xattrs: Rocky 9's rsync 3.2.7 fails when
|
||||
# combining xattrs with --link-dest, while SELinux labels were intentionally
|
||||
# excluded because restores must relabel for their destination host.
|
||||
rsync_args=(-aHAS --numeric-ids "--info=progress2,stats2")
|
||||
estimate_args=(-aHAS --numeric-ids --dry-run --stats)
|
||||
if [[ -n "$previous" ]]; then
|
||||
rsync_args+=("--link-dest=$previous")
|
||||
estimate_args+=("--link-dest=$previous")
|
||||
fi
|
||||
|
||||
# The rsync dry run estimates changed file bytes after link-dest deduplication.
|
||||
# Metadata and filesystem allocation still require the separate free-space reserve.
|
||||
estimate="$(rsync "${estimate_args[@]}" "${source_dir}/" "${partial}/")"
|
||||
transfer_bytes="$(printf '%s\n' "$estimate" | awk -F: \
|
||||
'/^Total transferred file size:/ { gsub(/[^0-9]/, "", $2); print $2 }')"
|
||||
[[ "$transfer_bytes" =~ ^[0-9]+$ ]] || {
|
||||
printf 'Could not determine the USB transfer size\n' >&2
|
||||
exit 74
|
||||
}
|
||||
if ((free_bytes - transfer_bytes < min_free_bytes)); then
|
||||
printf 'Insufficient USB space: %s bytes free, %s estimated transfer, %s reserved\n' \
|
||||
"$free_bytes" "$transfer_bytes" "$min_free_bytes" >&2
|
||||
exit 73
|
||||
fi
|
||||
|
||||
rsync "${rsync_args[@]}" "${source_dir}/" "${partial}/"
|
||||
|
||||
printf 'Verifying USB backup %s with a checksum-based dry run\n' "$backup_name"
|
||||
verification="${runtime_dir}/verification.out"
|
||||
rsync -aHAS --numeric-ids \
|
||||
--checksum --dry-run --delete --itemize-changes \
|
||||
"${source_dir}/" "${partial}/" >"$verification"
|
||||
if [[ -s "$verification" ]]; then
|
||||
printf 'USB verification found mismatches; refusing to publish the backup\n' >&2
|
||||
exit 74
|
||||
fi
|
||||
|
||||
free_bytes="$(df -B1 --output=avail "$usb_mount" | tail -n 1 | tr -d ' ')"
|
||||
if ((free_bytes < min_free_bytes)); then
|
||||
printf 'USB backup completed below the free-space reserve; refusing to publish it\n' >&2
|
||||
exit 73
|
||||
fi
|
||||
|
||||
mv -- "$partial" "$complete"
|
||||
partial=""
|
||||
ln -s "snapshots/${backup_name}" "${backup_root}/.latest-${backup_name}"
|
||||
mv -Tf -- "${backup_root}/.latest-${backup_name}" "${backup_root}/latest"
|
||||
published=true
|
||||
sync -f "$complete"
|
||||
sync -f "$backup_root"
|
||||
printf 'USB backup %s verified and published; unmounting and closing LUKS\n' "$backup_name"
|
||||
@@ -1,31 +0,0 @@
|
||||
#!/usr/bin/python3
|
||||
"""Submit a manual-backup reminder through Atlas' existing Houston notifier."""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
message = {
|
||||
"timestamp": now.isoformat(timespec="seconds"),
|
||||
"unixtime": int(now.timestamp()),
|
||||
"event": "atlas_usb_backup_reminder",
|
||||
"severity": "warning",
|
||||
"subject": "Promemoria backup USB offline Atlas",
|
||||
"email_message": (
|
||||
"Collega il disco USB di backup ad Atlas ed esegui manualmente il backup offline.\n"
|
||||
"Il promemoria non avvia il backup. Controlla che il disco non sia\n"
|
||||
"montato; poi esegui:\n\n"
|
||||
" sudo systemctl start atlas-usb-backup.service\n\n"
|
||||
"Verifica l'esito con:\n"
|
||||
" sudo journalctl -u atlas-usb-backup.service -n 100 --no-pager\n\n"
|
||||
"Dopo la riuscita, scollega fisicamente il disco."
|
||||
),
|
||||
}
|
||||
|
||||
subprocess.run(
|
||||
[{{ atlas_usb_reminder_notifier | to_json }}, json.dumps(message)],
|
||||
check=True,
|
||||
)
|
||||
print("Atlas USB backup reminder submitted to 45Drives Alerts; email delivery is not verified.", flush=True)
|
||||
@@ -1,22 +0,0 @@
|
||||
[Unit]
|
||||
Description=45Drives Alerts reminder to run the manual Atlas offline USB backup
|
||||
Requires=houston-dbus.service
|
||||
After=houston-dbus.service
|
||||
ConditionFileIsExecutable=/usr/local/libexec/atlas-usb-reminder
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/libexec/atlas-usb-reminder
|
||||
User=root
|
||||
Group=root
|
||||
UMask=0077
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_UNIX
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -1,10 +0,0 @@
|
||||
[Unit]
|
||||
Description=Remind the administrator to run the manual Atlas offline USB backup
|
||||
|
||||
[Timer]
|
||||
OnCalendar={{ atlas_usb_reminder_calendar }}
|
||||
Persistent=true
|
||||
Unit=atlas-usb-reminder.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,51 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
readonly pool={{ atlas_zfs_pool | quote }}
|
||||
readonly mount_root={{ atlas_mount_root | quote }}
|
||||
readonly snapshot_prefix={{ atlas_usb_backup_snapshot_prefix | quote }}
|
||||
readonly marker=/run/atlas-usb-backup/snapshot-name
|
||||
|
||||
[[ -e "$marker" ]] || exit 0
|
||||
[[ -f "$marker" && ! -L "$marker" ]] || {
|
||||
printf 'Unsafe Atlas USB snapshot marker; leaving snapshots unchanged\n' >&2
|
||||
exit 2
|
||||
}
|
||||
IFS= read -r snapshot_name <"$marker"
|
||||
[[ "$snapshot_name" =~ ^${snapshot_prefix}-[0-9]{8}T[0-9]{6}Z-[0-9]+$ ]] || {
|
||||
printf 'Invalid Atlas USB snapshot marker; leaving snapshots unchanged\n' >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
exec 9>/run/lock/atlas-zfs-snapshot.lock
|
||||
flock 9
|
||||
if zfs list -H -t snapshot -o name "${pool}@${snapshot_name}" >/dev/null 2>&1; then
|
||||
# ZFS can leave its on-demand .zfs/snapshot mounts in the host namespace
|
||||
# even after the backup's private bind mounts and process have exited.
|
||||
snapshot_mounts=()
|
||||
snapshot_sources=()
|
||||
while IFS=$'\t' read -r dataset dataset_mountpoint; do
|
||||
[[ "$dataset_mountpoint" == "$mount_root" || "$dataset_mountpoint" == "$mount_root/"* ]] || continue
|
||||
snapshot_mounts+=("${dataset_mountpoint}/.zfs/snapshot/${snapshot_name}")
|
||||
snapshot_sources+=("${dataset}@${snapshot_name}")
|
||||
done < <(zfs list -H -o name,mountpoint -s name -r "$pool")
|
||||
|
||||
{% raw %}
|
||||
for ((index = ${#snapshot_mounts[@]} - 1; index >= 0; index--)); do
|
||||
{% endraw %}
|
||||
mounted_source="$(findmnt -rn -M "${snapshot_mounts[$index]}" -o SOURCE || true)"
|
||||
[[ -n "$mounted_source" ]] || continue
|
||||
[[ "$mounted_source" == "${snapshot_sources[$index]}" ]] || {
|
||||
printf 'Unexpected source on Atlas USB snapshot mount: %s\n' \
|
||||
"${snapshot_mounts[$index]}" >&2
|
||||
exit 2
|
||||
}
|
||||
umount "${snapshot_mounts[$index]}"
|
||||
done
|
||||
|
||||
zfs destroy -r "${pool}@${snapshot_name}"
|
||||
printf 'Removed recursive Atlas USB source snapshot %s@%s after backup exit\n' \
|
||||
"$pool" "$snapshot_name"
|
||||
fi
|
||||
@@ -1,6 +0,0 @@
|
||||
[Timer]
|
||||
OnCalendar=
|
||||
OnCalendar={{ atlas_zfs_scrub_calendar }}
|
||||
Persistent=true
|
||||
RandomizedDelaySec=0
|
||||
AccuracySec=1min
|
||||
@@ -1,72 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
export LC_ALL=C
|
||||
export PATH=/usr/sbin:/usr/bin:/sbin:/bin
|
||||
|
||||
if [[ $# -ne 1 ]]; then
|
||||
printf 'Usage: %s <policy>\n' "$0" >&2
|
||||
exit 64
|
||||
fi
|
||||
|
||||
readonly pool={{ atlas_zfs_pool | quote }}
|
||||
readonly prefix={{ atlas_zfs_snapshot_prefix | quote }}
|
||||
readonly period="$1"
|
||||
|
||||
case "$period" in
|
||||
{% for policy in atlas_zfs_snapshot_policies %}
|
||||
{{ policy.name | quote }})
|
||||
keep={{ policy.keep | int }}
|
||||
;;
|
||||
{% endfor %}
|
||||
*)
|
||||
printf 'Unknown Atlas ZFS snapshot policy: %s\n' "$period" >&2
|
||||
exit 64
|
||||
;;
|
||||
esac
|
||||
readonly keep
|
||||
|
||||
zpool list -H -o name "$pool" >/dev/null
|
||||
|
||||
exec 9>/run/lock/atlas-zfs-snapshot.lock
|
||||
flock 9
|
||||
|
||||
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
readonly timestamp
|
||||
readonly snapshot_prefix="${pool}@${prefix}-${period}-"
|
||||
readonly snapshot="${snapshot_prefix}${timestamp}"
|
||||
|
||||
zfs snapshot -r "$snapshot"
|
||||
printf 'Created recursive ZFS snapshot %s\n' "$snapshot"
|
||||
|
||||
snapshot_listing="$(zfs list -H -t snapshot -o name -s creation -r "$pool")"
|
||||
managed_snapshots=()
|
||||
while IFS= read -r snapshot_name; do
|
||||
if [[ "$snapshot_name" == "$snapshot_prefix"* ]]; then
|
||||
snapshot_suffix="${snapshot_name#"$snapshot_prefix"}"
|
||||
if [[ "$snapshot_suffix" =~ ^[0-9]{8}T[0-9]{6}Z$ ]]; then
|
||||
managed_snapshots+=("$snapshot_name")
|
||||
fi
|
||||
fi
|
||||
done <<< "$snapshot_listing"
|
||||
|
||||
{% raw %}
|
||||
managed_snapshot_count="${#managed_snapshots[@]}"
|
||||
{% endraw %}
|
||||
prune_count=$((managed_snapshot_count - keep))
|
||||
if ((prune_count <= 0)); then
|
||||
printf 'Retaining %d of %d managed %s snapshots\n' \
|
||||
"$managed_snapshot_count" "$keep" "$period"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
for ((index = 0; index < prune_count; index++)); do
|
||||
candidate="${managed_snapshots[$index]}"
|
||||
if [[ "$candidate" != "$snapshot_prefix"* ]]; then
|
||||
printf 'Refusing to destroy unexpected snapshot: %s\n' "$candidate" >&2
|
||||
exit 65
|
||||
fi
|
||||
|
||||
zfs destroy -r "$candidate"
|
||||
printf 'Pruned recursive ZFS snapshot %s\n' "$candidate"
|
||||
done
|
||||
@@ -1,11 +0,0 @@
|
||||
[Unit]
|
||||
Description=Schedule {{ item.name }} ZFS snapshots for {{ atlas_zfs_pool }}
|
||||
|
||||
[Timer]
|
||||
OnCalendar={{ item.calendar }}
|
||||
Persistent=true
|
||||
AccuracySec=1min
|
||||
Unit=atlas-zfs-snapshot@{{ item.name }}.service
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,25 +0,0 @@
|
||||
[Unit]
|
||||
Description=Create and retain %i ZFS snapshots for {{ atlas_zfs_pool }}
|
||||
Documentation=man:zfs-snapshot(8) man:zfs-destroy(8)
|
||||
Requires=zfs.target
|
||||
After=zfs.target
|
||||
ConditionFileIsExecutable=/usr/local/sbin/atlas-zfs-snapshot
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/atlas-zfs-snapshot %i
|
||||
User=root
|
||||
Group=root
|
||||
UMask=0077
|
||||
Nice=10
|
||||
IOSchedulingClass=best-effort
|
||||
IOSchedulingPriority=7
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectHome=true
|
||||
ProtectSystem=strict
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictRealtime=true
|
||||
LockPersonality=true
|
||||
@@ -5,18 +5,14 @@ backend_phase1_username: "{{ atlas_admin_username }}"
|
||||
backend_phase1_user_group: "{{ atlas_admin_group }}"
|
||||
backend_phase1_user_home: "{{ atlas_admin_home }}"
|
||||
backend_phase1_quadlet_dir: "{{ backend_phase1_user_home }}/.config/containers/systemd"
|
||||
backend_phase1_bind_address: CHANGEME_ATLAS_BIND_ADDRESS
|
||||
backend_phase1_firewalld_zone: public
|
||||
backend_phase1_npm_source_ip: CHANGEME_AEGIS_IP
|
||||
backend_phase1_syncthing_native_subnet: CHANGEME_LAN_SUBNET
|
||||
backend_phase1_wireguard_interface: wg0
|
||||
backend_phase1_wireguard_address: CHANGEME_ATLAS_WIREGUARD_ADDRESS
|
||||
backend_phase1_wireguard_firewalld_zone: wireguard
|
||||
backend_phase1_music_dir: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_music }}"
|
||||
backend_phase1_archive_dir: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_archive }}"
|
||||
backend_phase1_syncthing_folder_path: "{{ backend_phase1_archive_dir }}/Org"
|
||||
backend_phase1_app_data_root: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_app_data }}"
|
||||
backend_phase1_navidrome_data_dir: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_navidrome }}"
|
||||
backend_phase1_syncthing_root: "{{ atlas_mount_root }}/{{ atlas_zfs_dataset_syncthing }}"
|
||||
backend_phase1_music_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_music }}"
|
||||
backend_phase1_archive_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_archive }}"
|
||||
backend_phase1_app_data_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_app_data }}"
|
||||
backend_phase1_navidrome_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_navidrome }}"
|
||||
backend_phase1_syncthing_dataset: "{{ atlas_zfs_pool }}/{{ atlas_zfs_dataset_syncthing }}"
|
||||
|
||||
@@ -7,22 +7,17 @@
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- not (atlas_manage_media_stack | bool)
|
||||
- backend_phase1_bind_address != 'CHANGEME_ATLAS_BIND_ADDRESS'
|
||||
- backend_phase1_bind_address is match('^[0-9]{1,3}(\.[0-9]{1,3}){3}$')
|
||||
- backend_phase1_firewalld_zone | length > 0
|
||||
- backend_phase1_npm_source_ip != 'CHANGEME_AEGIS_IP'
|
||||
- backend_phase1_syncthing_native_subnet != 'CHANGEME_LAN_SUBNET'
|
||||
- backend_phase1_wireguard_interface in ansible_facts.interfaces
|
||||
- backend_phase1_wireguard_address != 'CHANGEME_ATLAS_WIREGUARD_ADDRESS'
|
||||
- backend_phase1_music_dir.startswith('/')
|
||||
- backend_phase1_archive_dir.startswith('/')
|
||||
- backend_phase1_syncthing_folder_path.startswith(backend_phase1_archive_dir + '/')
|
||||
- backend_phase1_app_data_root.startswith('/')
|
||||
- backend_phase1_navidrome_data_dir.startswith(backend_phase1_app_data_root + '/')
|
||||
- backend_phase1_syncthing_root.startswith(backend_phase1_app_data_root + '/')
|
||||
fail_msg: >-
|
||||
Disable the rootful media-stack gate and provide the Atlas LAN bind
|
||||
address, firewall sources, and absolute ZFS-backed paths before
|
||||
Disable the rootful media-stack gate and provide the active
|
||||
WireGuard interface/address and absolute ZFS-backed paths before
|
||||
enabling phase one. This role does not manage Prometheus or migrate
|
||||
application data.
|
||||
Navidrome application data.
|
||||
|
||||
- name: Read the rootless service account
|
||||
ansible.builtin.getent:
|
||||
@@ -54,8 +49,6 @@
|
||||
loop:
|
||||
- dataset: "{{ backend_phase1_music_dataset }}"
|
||||
mountpoint: "{{ backend_phase1_music_dir }}"
|
||||
- dataset: "{{ backend_phase1_archive_dataset }}"
|
||||
mountpoint: "{{ backend_phase1_archive_dir }}"
|
||||
- dataset: "{{ backend_phase1_app_data_dataset }}"
|
||||
mountpoint: "{{ backend_phase1_app_data_root }}"
|
||||
- dataset: "{{ backend_phase1_navidrome_dataset }}"
|
||||
@@ -108,8 +101,6 @@
|
||||
mode: "0750"
|
||||
- path: "{{ backend_phase1_syncthing_root }}"
|
||||
mode: "0750"
|
||||
- path: "{{ backend_phase1_syncthing_folder_path }}"
|
||||
mode: "0770"
|
||||
loop_control:
|
||||
label: "{{ item.path }}"
|
||||
|
||||
@@ -141,35 +132,19 @@
|
||||
DBUS_SESSION_BUS_ADDRESS: "unix:path=/run/user/{{ backend_phase1_uid }}/bus"
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Permit NPM access to phase-one web interfaces through Aegis
|
||||
- name: Permit phase-one services only through the WireGuard zone
|
||||
ansible.posix.firewalld:
|
||||
rich_rule: >-
|
||||
rule family="ipv4" source address="{{ backend_phase1_npm_source_ip }}"
|
||||
port port="{{ item }}" protocol="tcp" accept
|
||||
zone: "{{ backend_phase1_firewalld_zone }}"
|
||||
port: "{{ item }}"
|
||||
zone: "{{ backend_phase1_wireguard_firewalld_zone }}"
|
||||
state: enabled
|
||||
permanent: true
|
||||
immediate: true
|
||||
loop:
|
||||
- "{{ backend_phase1_navidrome_port }}"
|
||||
- "{{ backend_phase1_syncthing_gui_port }}"
|
||||
|
||||
- name: Permit native Syncthing traffic from the LAN
|
||||
ansible.posix.firewalld:
|
||||
rich_rule: >-
|
||||
rule family="ipv4" source address="{{ backend_phase1_syncthing_native_subnet }}"
|
||||
port port="{{ item.port }}" protocol="{{ item.protocol }}" accept
|
||||
zone: "{{ backend_phase1_firewalld_zone }}"
|
||||
state: enabled
|
||||
permanent: true
|
||||
immediate: true
|
||||
loop:
|
||||
- port: "{{ backend_phase1_syncthing_transfer_port }}"
|
||||
protocol: tcp
|
||||
- port: "{{ backend_phase1_syncthing_transfer_port }}"
|
||||
protocol: udp
|
||||
- port: "{{ backend_phase1_syncthing_discovery_port }}"
|
||||
protocol: udp
|
||||
- "{{ backend_phase1_navidrome_port }}/tcp"
|
||||
- "{{ backend_phase1_syncthing_gui_port }}/tcp"
|
||||
- "{{ backend_phase1_syncthing_transfer_port }}/tcp"
|
||||
- "{{ backend_phase1_syncthing_transfer_port }}/udp"
|
||||
- "{{ backend_phase1_syncthing_discovery_port }}/udp"
|
||||
|
||||
- name: Start rootless phase-one Quadlets
|
||||
become_user: "{{ backend_phase1_username }}"
|
||||
|
||||
@@ -8,7 +8,7 @@ Image={{ backend_phase1_navidrome_image }}
|
||||
UserNS=keep-id
|
||||
User={{ backend_phase1_uid }}
|
||||
Group={{ backend_phase1_gid }}
|
||||
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_navidrome_port }}:4533
|
||||
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_navidrome_port }}:4533
|
||||
Environment=ND_LOGLEVEL=info
|
||||
Environment=ND_SCANSCHEDULE=1h
|
||||
Environment=ND_SESSIONTIMEOUT=24h
|
||||
|
||||
@@ -9,17 +9,15 @@ HostName=atlas-syncthing
|
||||
UserNS=keep-id
|
||||
User={{ backend_phase1_uid }}
|
||||
Group={{ backend_phase1_gid }}
|
||||
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_gui_port }}:{{ backend_phase1_syncthing_gui_port }}
|
||||
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}
|
||||
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}/udp
|
||||
PublishPort={{ backend_phase1_bind_address }}:{{ backend_phase1_syncthing_discovery_port }}:{{ backend_phase1_syncthing_discovery_port }}/udp
|
||||
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_gui_port }}:{{ backend_phase1_syncthing_gui_port }}
|
||||
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}
|
||||
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_transfer_port }}:{{ backend_phase1_syncthing_transfer_port }}/udp
|
||||
PublishPort={{ backend_phase1_wireguard_address }}:{{ backend_phase1_syncthing_discovery_port }}:{{ backend_phase1_syncthing_discovery_port }}/udp
|
||||
Environment=HOME=/var/syncthing
|
||||
Environment=STHOMEDIR=/var/syncthing/config
|
||||
Environment=STGUIADDRESS=0.0.0.0:{{ backend_phase1_syncthing_gui_port }}
|
||||
Environment=TZ={{ backend_phase1_timezone }}
|
||||
Volume={{ backend_phase1_syncthing_root }}:/var/syncthing:Z
|
||||
# Archive is intentionally shared with Samba; use a shared SELinux container label.
|
||||
Volume={{ backend_phase1_archive_dir }}:/data:z
|
||||
NoNewPrivileges=true
|
||||
DropCapability=all
|
||||
|
||||
|
||||
@@ -20,7 +20,15 @@
|
||||
owner: "{{ item.owner }}"
|
||||
group: "{{ item.group }}"
|
||||
mode: "{{ item.mode }}"
|
||||
loop: "{{ server_directories | default([]) }}"
|
||||
loop: >-
|
||||
{{
|
||||
(server_directories | default([]))
|
||||
+ (
|
||||
server_syncthing_directories | default([])
|
||||
if server_syncthing_enabled | default(true) | bool
|
||||
else []
|
||||
)
|
||||
}}
|
||||
loop_control:
|
||||
label: "{{ item.path }}"
|
||||
|
||||
|
||||
@@ -66,17 +66,6 @@
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Check whether the pinned Java 25 version is installed with Mise
|
||||
tags: [packages, mise, java, wsl]
|
||||
ansible.builtin.command:
|
||||
cmd: "mise where java@{{ workstation_mise_java_25_version }}"
|
||||
become_user: "{{ username }}"
|
||||
environment:
|
||||
HOME: "{{ user_home }}"
|
||||
register: workstation_mise_java_25_where
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Check whether the pinned Maven version is installed with Mise
|
||||
tags: [packages, mise, maven, wsl]
|
||||
ansible.builtin.command:
|
||||
@@ -97,7 +86,6 @@
|
||||
HOME: "{{ user_home }}"
|
||||
when: >-
|
||||
workstation_mise_java_where.rc != 0 or
|
||||
workstation_mise_java_25_where.rc != 0 or
|
||||
workstation_mise_maven_where.rc != 0
|
||||
|
||||
- name: Ensure WSL boot configuration file exists
|
||||
|
||||
@@ -10,15 +10,6 @@ wireguard_mtu: 1420
|
||||
wireguard_firewalld_zone: wireguard
|
||||
wireguard_public_firewalld_zone: public
|
||||
wireguard_enable_ipv4_forwarding: false
|
||||
# Legacy zone-level masquerading; policy-level masquerading is required for inter-zone forwarding.
|
||||
wireguard_enable_masquerade: false
|
||||
# A list of narrowly scoped inter-zone forwarding rules, rendered as persistent
|
||||
# firewalld policies. Each item requires name, ingress_zone, egress_zone,
|
||||
# source, and destination.
|
||||
wireguard_forwarding_policies: []
|
||||
wireguard_reload_rootful_podman_networks: false
|
||||
wireguard_handshake_retries: 12
|
||||
wireguard_handshake_delay: 5
|
||||
wireguard_peers: []
|
||||
wireguard_packages:
|
||||
- wireguard-tools
|
||||
|
||||
@@ -14,44 +14,10 @@
|
||||
Configure this host's WireGuard address and peers, and run the first
|
||||
key bootstrap against every peer in the same play.
|
||||
|
||||
- name: Validate WireGuard forwarding policies
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- item.name is defined
|
||||
- item.ingress_zone is defined
|
||||
- item.egress_zone is defined
|
||||
- item.source is defined
|
||||
- item.destination is defined
|
||||
fail_msg: >-
|
||||
Every WireGuard forwarding policy requires name, ingress_zone,
|
||||
egress_zone, source, and destination.
|
||||
loop: "{{ wireguard_forwarding_policies }}"
|
||||
loop_control:
|
||||
label: "{{ item.name | default('unnamed policy') }}"
|
||||
|
||||
- name: Install WireGuard userspace tools on mutable hosts
|
||||
- name: Install WireGuard userspace tools
|
||||
ansible.builtin.dnf:
|
||||
name: "{{ wireguard_packages }}"
|
||||
state: present
|
||||
when: "'platform_fedora_iot' not in group_names"
|
||||
|
||||
- name: Require WireGuard userspace tools in the booted deployment
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- wg
|
||||
- --version
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: wireguard_userspace_tools
|
||||
|
||||
- name: Require active WireGuard userspace tools
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- wireguard_userspace_tools.rc == 0
|
||||
fail_msg: >-
|
||||
WireGuard userspace tools are not present in the booted deployment.
|
||||
On Fedora IoT, reboot after rpm-ostree layers wireguard-tools, then
|
||||
rerun the WireGuard play.
|
||||
|
||||
- name: Create private WireGuard configuration directory
|
||||
ansible.builtin.file:
|
||||
@@ -158,47 +124,12 @@
|
||||
permanent: true
|
||||
register: wireguard_firewalld_zone_result
|
||||
|
||||
- name: Create the firewalld policy directory
|
||||
ansible.builtin.file:
|
||||
path: /etc/firewalld/policies
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: wireguard_forwarding_policies | length > 0
|
||||
|
||||
- name: Render WireGuard forwarding policies
|
||||
ansible.builtin.template:
|
||||
src: wireguard-forwarding-policy.xml.j2
|
||||
dest: "/etc/firewalld/policies/{{ item.name }}.xml"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
loop: "{{ wireguard_forwarding_policies }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
register: wireguard_forwarding_policy_result
|
||||
|
||||
- name: Reload firewalld after WireGuard firewall changes
|
||||
- name: Reload firewalld after creating the WireGuard zone
|
||||
ansible.builtin.systemd:
|
||||
name: firewalld.service
|
||||
state: reloaded
|
||||
when:
|
||||
- wireguard_firewalld_zone_result is changed or (wireguard_forwarding_policy_result | default({})) is changed
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Restore rootful Podman networking after firewalld reload
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- podman
|
||||
- network
|
||||
- reload
|
||||
- --all
|
||||
register: wireguard_podman_network_reload
|
||||
changed_when: wireguard_podman_network_reload.stdout_lines | length > 0
|
||||
when:
|
||||
- wireguard_firewalld_zone_result is changed or (wireguard_forwarding_policy_result | default({})) is changed
|
||||
- wireguard_reload_rootful_podman_networks | bool
|
||||
- wireguard_firewalld_zone_result is changed
|
||||
- not ansible_check_mode
|
||||
|
||||
- name: Assign the WireGuard interface to its firewalld zone
|
||||
@@ -209,14 +140,6 @@
|
||||
permanent: true
|
||||
immediate: true
|
||||
|
||||
- name: Manage legacy WireGuard zone masquerading
|
||||
ansible.posix.firewalld:
|
||||
zone: "{{ wireguard_firewalld_zone }}"
|
||||
masquerade: true
|
||||
state: "{{ 'enabled' if wireguard_enable_masquerade | bool else 'disabled' }}"
|
||||
permanent: true
|
||||
immediate: true
|
||||
|
||||
- name: Permit this host's public WireGuard listener
|
||||
ansible.posix.firewalld:
|
||||
port: "{{ wireguard_listen_port }}/udp"
|
||||
@@ -233,23 +156,3 @@
|
||||
state: started
|
||||
daemon_reload: true
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Apply pending WireGuard handlers before verification
|
||||
ansible.builtin.meta: flush_handlers
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Wait for every WireGuard peer handshake
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- wg
|
||||
- show
|
||||
- "{{ wireguard_interface }}"
|
||||
- latest-handshakes
|
||||
register: wireguard_latest_handshakes
|
||||
changed_when: false
|
||||
retries: "{{ wireguard_handshake_retries }}"
|
||||
delay: "{{ wireguard_handshake_delay }}"
|
||||
until:
|
||||
- wireguard_latest_handshakes.stdout_lines | length == wireguard_peers | length
|
||||
- wireguard_latest_handshakes.stdout_lines | select('search', '\t0$') | list | length == 0
|
||||
when: not ansible_check_mode
|
||||
|
||||
@@ -11,8 +11,7 @@ ListenPort = {{ wireguard_listen_port }}
|
||||
{% for peer in wireguard_peers %}
|
||||
[Peer]
|
||||
# {{ peer.name }}
|
||||
{% if peer.comment is defined %}# {{ peer.comment }}
|
||||
{% endif %}PublicKey = {{ hostvars[peer.host].wireguard_public_key }}
|
||||
PublicKey = {{ hostvars[peer.host].wireguard_public_key }}
|
||||
AllowedIPs = {{ peer.allowed_ips | join(', ') }}
|
||||
{% if peer.endpoint is defined %}
|
||||
Endpoint = {{ peer.endpoint }}
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<policy target="CONTINUE">
|
||||
<short>WireGuard forwarding: {{ item.name }}</short>
|
||||
<description>Managed WireGuard-to-LAN forwarding policy.</description>
|
||||
{% if item.masquerade | default(false) %}
|
||||
<masquerade/>
|
||||
{% endif %}
|
||||
<ingress-zone name="{{ item.ingress_zone }}"/>
|
||||
<egress-zone name="{{ item.egress_zone }}"/>
|
||||
<rule family="ipv4">
|
||||
<source address="{{ item.source }}"/>
|
||||
<destination address="{{ item.destination }}"/>
|
||||
<accept/>
|
||||
</rule>
|
||||
</policy>
|
||||
@@ -73,13 +73,6 @@
|
||||
- packages_rocky
|
||||
- services_systemd
|
||||
|
||||
- name: Configure Aegis Fedora IoT profile
|
||||
hosts: role_aegis
|
||||
become: true
|
||||
|
||||
roles:
|
||||
- profile_aegis
|
||||
|
||||
- name: Configure WireGuard overlay
|
||||
hosts: wireguard_overlay
|
||||
become: true
|
||||
@@ -87,6 +80,13 @@
|
||||
roles:
|
||||
- wireguard_overlay
|
||||
|
||||
- name: Configure Aegis Fedora IoT profile
|
||||
hosts: role_aegis
|
||||
become: true
|
||||
|
||||
roles:
|
||||
- profile_aegis
|
||||
|
||||
- name: Configure Atlas NAS profile
|
||||
hosts: atlas
|
||||
become: true
|
||||
|
||||
@@ -56,6 +56,24 @@ services:
|
||||
- "3000:3000"
|
||||
- "127.0.0.1:222:22"
|
||||
|
||||
{% if server_syncthing_enabled | default(true) | bool %}
|
||||
syncthing:
|
||||
image: docker.io/syncthing/syncthing:2
|
||||
container_name: syncthing
|
||||
hostname: syncthing
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- "8384"
|
||||
volumes:
|
||||
- "/opt/syncthing/config:/var/syncthing{{ ':' ~ selinux_volume_option if selinux_volume_option else '' }}"
|
||||
- "/srv/syncthing/data:/data{{ ':' ~ selinux_volume_option if selinux_volume_option else '' }}"
|
||||
ports:
|
||||
- "22000:22000/tcp"
|
||||
- "22000:22000/udp"
|
||||
- "21027:21027/udp"
|
||||
networks:
|
||||
- web
|
||||
{% endif %}
|
||||
|
||||
networks:
|
||||
web:
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
[tools]
|
||||
java = ["25.0.2", "temurin-11.0.31+11"]
|
||||
java = "temurin-11.0.31+11"
|
||||
maven = "3.9.16"
|
||||
|
||||
[env]
|
||||
JAVA_HOME = "{{ env.HOME }}/.local/share/mise/installs/java/25.0.2"
|
||||
|
||||
1
secrets/.gitignore
vendored
1
secrets/.gitignore
vendored
@@ -2,4 +2,3 @@
|
||||
.vault_pass*
|
||||
vault.local.yml
|
||||
vault.local.yml*
|
||||
/recovery/
|
||||
|
||||
@@ -1,83 +1,71 @@
|
||||
$ANSIBLE_VAULT;1.1;AES256
|
||||
61353065386233646137323235306631353635663530363237636231316265643562353465323430
|
||||
6165646466623962313835313537633137633766373930380a316335323962616265643136346666
|
||||
63336133336131346336383534356637623831363138323165633262386333363535393365383233
|
||||
6234393835653439370a313963313365373633323464343263383661383336363662633133643232
|
||||
34366634383862363635653034313531623330396639616462343630326162316535643465653532
|
||||
36326534333637376462353561343964633636366331363833313263353133383636623537303663
|
||||
35393032316439336666343161653439643638376134363535656262343963393365623432336433
|
||||
35383934313762313037326430316666363731666231336534326661353034333063643364343230
|
||||
65333739303566366263333565333465613136646237623937393733623438613832393634663463
|
||||
39376131313234333039633735613233373931613232653036663665316636303961653834366339
|
||||
36353730316132316233303964303839363161346564396163336137663134353062363733656430
|
||||
37643339326661653031376265646132623162373562393437373437313732396537383939333666
|
||||
62353036316633306666313461663033303830393765396131643035353730383931646239663935
|
||||
32626461316364386135303761383837613063336466363162323332663764616464373565383231
|
||||
61346463336566346533326535376439643133613762383633396131323632356533636139336365
|
||||
62393838316634623932643034376631333539343965383436613364643962363834346337353334
|
||||
32656439366439313734353963343133333533653839613632323338336131373566613835393536
|
||||
31663433616334373432376531346435336530303936356461303163646463613661643161313661
|
||||
66663866343565616631616338353737356164353562366164383736346131666662623132333466
|
||||
39383865653631373232393433663430643961646265386166333137643966303834363262373636
|
||||
62396434373363353636376133666133663162653265313139313732353639336232333862643036
|
||||
64386231336561396537326139346566306434633934343038663165396665363032383466633662
|
||||
62336163633964363435386630343966333162333730336138333239646631633132663931376462
|
||||
33663139356261313065376636613930353735396131306538306664646135636336643032623131
|
||||
38346264333331353633326535326431626563323036313665643337353563333339646430386564
|
||||
31613435383036313430316366323636663735326336393338353835323861333564363832656462
|
||||
35336435623261326363633033316130393062616339353263643062633331646137376135656365
|
||||
35636139336564346164616235616431326531333433646330386134323932373339646536356464
|
||||
66343533326534326165323564663533653666633035343163633832393361336462343937623165
|
||||
62383931326630363036396333313931393836366439653433623165666166356338653364336534
|
||||
35333936653833386163633738326164386166613561333530633937343230363366333662666539
|
||||
39333361633933663735303438663239303536363433313962643137386533633539326365383765
|
||||
37636538386339333935386132353265353031643662616330316463623661663738353433313830
|
||||
36373963633166333464653338343830373063323536383364393033393235326639613662343737
|
||||
38663362636331343061646465313237313431373433353361353265333766633463353632646536
|
||||
31323231306138323031396630656538363930373439336234343963616334363632653738316465
|
||||
63653938373830336362313238656266613362636634616537653863336132343931616262396130
|
||||
66393239303866656232653832343132366537333537343635666563343639323433383163613335
|
||||
39613533376634316133633430303535306266656333626264343733666335393661666561396633
|
||||
39346265316137326465326635396362333565393133623637633132616232326263663662343137
|
||||
33363733306135363361643031306265363733656362386666306334333035393839636533343363
|
||||
35396638616636633639343930373136376339346162393061393765363837646365383866636131
|
||||
33653465666239393133616232636231333332396138376332393664343364643835306530393238
|
||||
34663237303530303837663535646263393931373531393039356336316561653130356262636562
|
||||
38336362326639653237626634376334666565653036353236313634376364626338646538386536
|
||||
38626636386466373566646166393963643164343536373236396138303532393161363335386638
|
||||
32633032393737626363613463323366366637616361313537356136626661626633613739323338
|
||||
35383963666431343566356562333234663936376562616638636261303466633539376334303331
|
||||
39303834663234663063356233313962326664383839393832303462643636393034383434303465
|
||||
64333635376135326333356435373734643430623736373234643335343130383066326436356664
|
||||
63346663326364343634303930343338336139313864316165366232643537366635653764353763
|
||||
31363863633261643263303433373330366161323166366462336332313135366338393334653764
|
||||
66353733653137663835663731373364613030373334663061313433373861613665363236633130
|
||||
65613965366636343465336533613438373466383737373366653965633437323562643966396431
|
||||
39303033643438633762633263326132663466643438656366363431616237633031333936313831
|
||||
30323930383233313032323638356333626230333764363662313662646536643839353032353462
|
||||
30326166653937353130623133303533343934633565393831623033303234316330353432313266
|
||||
30636536633933376365623665616262663236383731633633346232613366333137396139306363
|
||||
35633336643266326335303261666666653536666630613639376336373237646134306462616537
|
||||
33343561373162666332613634643837343566646161373065366637653135613632353334636363
|
||||
63363232303963646530333366663862323264326536643337323266396566316233613630303637
|
||||
66646366376466373931613734363931316230323063373666653062373364396433633762633762
|
||||
38613933323733653238383935623230383562646563363833653838636165626365646537383639
|
||||
33666535656363393562316336633439636138373365623431393965653765306138646234663938
|
||||
65653133663663393731646337386535333261643932336132396237323930306136643534353930
|
||||
65636438396432623034626561613137336138623265393064383034623863303166356138393564
|
||||
37373164626634653662326234333539663735323464613334616130643937373730363263633366
|
||||
31393937326432386165343338313031376565313866363731643534313233303064373935303538
|
||||
31343832336230393636653432653162336361383963633766343461653466316337353931333363
|
||||
63313137303564336630343937356564643763383764613362366634373362666465626334336539
|
||||
64366533376165306532343461613265366266383862323032333465336161663161376630316465
|
||||
30306562666163646235656664653635366461366435663961623635383437663564356563346462
|
||||
31636234663765623838333237393239373564366262613637363938653463396530613963643837
|
||||
38636634376637366332623035313465393762653865623130336263343663303066366135616639
|
||||
63333964356466613038303263366462346261353030646532366361393965306435613131316463
|
||||
65366266376637323764643239323730366565633335666638666334663635373961303637383861
|
||||
35313431646434656562333937663837393038386361616630626532636339306432353434656165
|
||||
33663261383166386432383465666136376237346565303164363461666663346130346162316338
|
||||
62373061353034316234303835663439396434343738303764376665336239626238386436386234
|
||||
61306166383637366266393730323732386163366261393630336431633862353761343763363665
|
||||
61323039396234393835303633363339373633653334343766653032313230343464326664356566
|
||||
3462623830666664626633373966363866333337383730313066
|
||||
34663237313764396365646239613936373362306636663862373964343131636362333265306564
|
||||
3739343661336433303637343066626263646530663464380a636363303065643130336239663335
|
||||
64323265303964383635366235656163353333393166363635333831343531356339363939333530
|
||||
6266306339323862300a353530623639643836323135363566653161356630316565323233333937
|
||||
34343637383766623036633435346131393935303736613963653762663336383336366666333961
|
||||
39346336356563383464306364373438666638353731616566656164386365373966623230313933
|
||||
61343237373034616163363733326139306665366339383866316666396138373164366363613164
|
||||
34313638373435623439326164396264393838316263633630653935626635333362386132313864
|
||||
34643937653730336637633137623937346131396330356665626564666561343630383562646563
|
||||
65393532656162323931663266323635343437613639373731623631636533326533396662373032
|
||||
61386436613235353638613731313835373137643533363936326566316165636137653438393238
|
||||
37343765356366626666363162663262313663393038323531346461343731396334623339323865
|
||||
65323464323465326364633263346561316364336431323763386138383132643136663630636439
|
||||
66393564343462663933626464623866613563376234663430353837343139346363313539313839
|
||||
65386134653932393464356562346238336535323130373235303964313664363031623137383365
|
||||
39356530666564383334626330613039303135626237393233353362363033653239343634356261
|
||||
35356533643937393834383930336633623932363435646261633837663536356462356535613030
|
||||
63626438393466616238383030643631343661613630623738323933386434636461383136376261
|
||||
39616137336134663130613234633266383434336133353334373063366232663839316536376565
|
||||
62643766316464393135633564373062663335633533626631326534333232323364343133393534
|
||||
63396465363066643261303334363966343935323635383938616539323838393235373730313832
|
||||
37313235316264643932343062353162353534653963336634373264343937653035333735393766
|
||||
32393765643535383535616335613136336566653637343835366538633737313962336531333533
|
||||
32376163613861313139393863646331386133363435373364316134376533316136396533353161
|
||||
39353737633439656466396561623938646438636664373438323866613636653963393938346363
|
||||
63616366313030623464613832336566356339343931643436393138346463356161653039666632
|
||||
63396465306630336634316632663930613637643331363236353363383162333865383536623235
|
||||
33306232333439383135306266656666616135363433636332663865633234653530666435356164
|
||||
34336338343831633966666334643436373162336333313666613766656638656530373663383036
|
||||
39366134326134323435326434383637313334336562386638346333303666396336653231396463
|
||||
37633131393564656265646530373262393231373663303133323566666630366530626439353062
|
||||
37323138656431653235303439653133303831383864376562353363373466396334373063616561
|
||||
34343932383433323331363633643432303633323337656261386339653263393832396539633938
|
||||
36303634336162613366376435366530663632656263363436666532663561636465613736303962
|
||||
35663035303437313933373533336639323563376539303730323933346636626462336530303135
|
||||
65623036326663363130326438376161303463363839653564623462663561346236393262643035
|
||||
63363631663834303734393330353730333234626236616636326236376133633635376435326630
|
||||
33383035363466326139336431353937363236303134633662653066356164656366353038366264
|
||||
63343261393865303163633166663433373362663261343166303932313061393463346662366165
|
||||
31396333633964636265326539396533343932363433636438363137356430663764373835373566
|
||||
63633265323537353666653862623030646661366531646163303261363035356138346538623466
|
||||
63323235393835353864303362356465393739303531653139316338643136346164366630383635
|
||||
64376261376465303134383938346639386563363532313532663532656462663362643339353733
|
||||
61356566613638336639323235393132623032303639393065363238366337333237363863326365
|
||||
32383563313062363166656137626639363138363236383763663262363531646133356434633636
|
||||
39653666626539356538313234636634386438386434303336313337323638383863376163356365
|
||||
32346330306361343337373636376239643538396334303038636664633666306562316435303332
|
||||
33663238336336353237396139633133386439376162616439313862323462613636646262326162
|
||||
62653237636161393938336435333730636237313363663835323932613933373466343163646331
|
||||
37383466386437303433616162323433343761623831323163363331613164376565623633356335
|
||||
61366336393938383036623962353031333735376266643866363562653965306335636531376138
|
||||
64303764343565376561623933663035646163343235646565666465323864323065636462383962
|
||||
66633336313032323333366237366561613734376132346537373136376639623163313837613235
|
||||
37393332323539303338353365373766383537626162613738343534396339613336323437646466
|
||||
61623636303665623262633632653633373931373933366138316565353938656232313961373865
|
||||
66393230663532616532666561666465313361316432376432313034343333636566626564323066
|
||||
34613331633439626233333939303461393239623262376265393132313134653537396165343262
|
||||
39303135366566623037646265616630313638383732323064303237626334383261303230383162
|
||||
66633164353330666436376666316236363164373230633062636430663632393837366630643539
|
||||
33663233306464616165613337633030656661393336613036333036666565356232333966616361
|
||||
62353465643839373266333866353135396262303461666465393736636638636462666136613461
|
||||
31373130663563393833633930323766656530616261323363383333306631653563363734343034
|
||||
64383962346463663861356362346437346630353835383062373433396335343931343462386165
|
||||
33636266306563346431376438333463653165663962346262633037393238393135323536633866
|
||||
65623663323365643233346232613936373562363532373162613533316335376138366432316362
|
||||
30303662363436636263313064363235393632353430386438663833666665316561323431396430
|
||||
64623435323461313763613730643762633535393533333635313435613665383535633161353539
|
||||
37666665313730663366646537363963636631363836353437663963656566353962633935383263
|
||||
38663930373135353336326530396636373938333934326630303138643563633934656566663963
|
||||
6461373661386230356262643732373861366139323261303961
|
||||
|
||||
@@ -3,13 +3,14 @@ vault_duckdns_token: "CHANGEME"
|
||||
vault_personal_full_name: "REPLACE_ME"
|
||||
vault_git_email: "REPLACE_ME"
|
||||
vault_git_signing_key: "REPLACE_ME"
|
||||
vault_icloud_email: "REPLACE_ME"
|
||||
vault_protonmail_email: "REPLACE_ME"
|
||||
vault_icloud_mail_password: "REPLACE_ME"
|
||||
vault_git_work_email: "REPLACE_ME"
|
||||
vault_git_work_gpg: "REPLACE_ME"
|
||||
vault_openai_api_key: "REPLACE_ME"
|
||||
vault_ikaros_authorized_ssh_keys:
|
||||
- "ssh-ed25519 REPLACE_ME"
|
||||
vault_aegis_icloudpd_apple_id: "REPLACE_ME"
|
||||
vault_atlas_admin_password_hash: "REPLACE_WITH_A_SHADOW_COMPATIBLE_HASH"
|
||||
vault_atlas_samba_password: "REPLACE_ME"
|
||||
vault_atlas_immich_db_password: "REPLACE_ME"
|
||||
vault_atlas_borg_passphrase: "REPLACE_WITH_A_STRONG_UNIQUE_PASSPHRASE"
|
||||
|
||||
Reference in New Issue
Block a user