Document deferred Cerberus management node plan

AGENTS.md

@@ -63,6 +63,8 @@ Ansible-driven personal infrastructure repo for Fedora and Void desktops, Fedora
     `ansible-playbook ansible/site.yml --limit atlas --tags snapshots,scrub --check --diff`
   - Atlas encrypted Borg backup:
     `ansible-playbook ansible/site.yml --limit atlas --tags packages,borg --check --diff`
+  - Atlas Borg progress logging only:
+    `ansible-playbook ansible/site.yml --limit atlas --tags borg_logging --check --diff`
   - Prometheus/Aegis WireGuard gateway:
     `ansible-playbook ansible/site.yml --limit prometheus,aegis --tags wireguard --check --diff`
   - DuckDNS config only: `ansible-playbook ansible/site.yml --limit prometheus --tags duckdns --check --diff`
@@ -223,6 +225,41 @@ scheduled retention prune and monthly scrub remain runtime checks.
   after the storage and backup layers are validated; do not make either a dependency of the Atlas
   baseline.
 
+## Cerberus Management Node (Deferred)
+`cerberus` is postponed until the office in the new house is physically set up. It is not an inventory
+host and this section is a design and implementation backlog, not authorization to provision it early.
+
+The planned node is a Lenovo ThinkCentre M700 Tiny with an Intel Core i3-6100T, 8 GB RAM, a 256 GB SSD,
+and native 1 Gbps Ethernet. It will connect to a multi-input KVM switch using a passive DisplayPort-to-HDMI
+cable, sharing the monitor and peripherals with Ikaros. Fedora Sericea (immutable Fedora with the Sway
+Wayland compositor) is the intended OS. Cerberus is an isolated management plane: a dedicated Toolbox
+environment will run Ansible for future `uranus` cluster provisioning. Rootless Podman will host Grafana,
+Prometheus, and Loki. The 256 GB local SSD is the hot tier retaining metrics and logs for 30 days; scheduled,
+validated exports of older historical data will use a dedicated Atlas NFS dataset as cold storage.
+
+### Implementation plan
+- [ ] Confirm the office, KVM switch, passive DisplayPort-to-HDMI path, shared monitor/peripherals, and native
+  1 Gbps Ethernet are physically operational before adding Cerberus to inventory.
+- [ ] Install and update Fedora Sericea with Sway; document the immutable-host lifecycle and keep host changes
+  declarative rather than treating the base OS as a mutable workstation.
+- [ ] Model Cerberus as its own host with independent platform, role, desktop, network, and storage inputs;
+  do not repurpose Ikaros variables or make it a Uranus cluster member.
+- [ ] Provision an isolated Toolbox-based Ansible controller with the required collections and a reproducible
+  project checkout; define its least-privilege SSH access, known-host handling, and Vault workflow without
+  storing secrets in the image or repository.
+- [ ] Define the explicit Uranus provisioning workflow from Cerberus, including inventory boundaries,
+  validation-only runs, and separate approval for any destructive cluster operation.
+- [ ] Design rootless Podman/Quadlet services for Grafana, Prometheus, and Loki, including persistent local
+  state, service ownership, LAN exposure/authentication, resource limits, updates, and backups.
+- [ ] Size and enforce a 30-day local hot-retention policy for metrics and logs on the 256 GB SSD; validate
+  actual disk growth and alert before capacity exhaustion.
+- [ ] Create and validate a dedicated Atlas NFS cold-storage dataset and least-privilege export for Cerberus;
+  do not use a broad existing share or couple it to unrelated Atlas application state.
+- [ ] Implement scheduled, idempotent exports of data older than 30 days to the Atlas NFS cold tier, with
+  locking, capacity checks, integrity verification, retention rules, failure monitoring, and a tested restore.
+- [ ] Validate management-plane recovery: rebuild Cerberus, restore observability history from Atlas, and
+  confirm that Uranus provisioning can resume without depending on unreproducible local state.
+
 ## Coding Agent Notes
 - Shared agent definitions and lifecycle flags live in `ai_agents` in `ansible/inventory/group_vars/all.yml`.
 - Shared agent dotfiles live in `ai_agents_dotfiles`; rendered configs live in `ai_agents_templates`.

README.it.md

@@ -95,6 +95,27 @@ Nota sullo stato attuale del playbook principale:
 - `ansible/site.yml` applica il profilo server Rocky a `prometheus` con DNF, systemd, dotfiles server e firewalld
 - `ansible/site.yml` applica il profilo NAS Rocky su `atlas` tramite SSH remoto
 
+## Nodo pianificato e posticipato: Cerberus
+
+`cerberus` e un nodo di management **posticipato**, in attesa dell'allestimento
+fisico dell'ufficio nella nuova casa. Non e ancora presente nell'inventory e non
+esistono ruoli o playbook che lo prendano come target.
+
+L'hardware previsto e un Lenovo ThinkCentre M700 Tiny (Intel Core i3-6100T,
+8 GB di RAM e SSD da 256 GB) con Ethernet nativa a 1 Gbps. Condividera monitor
+e periferiche di Ikaros tramite uno switch KVM a ingressi multipli, usando un
+cavo passivo DisplayPort-HDMI per il collegamento video. Il sistema operativo
+previsto e Fedora Sericea, la variante Fedora immutabile con compositor Wayland
+Sway.
+
+Cerberus sara un management plane isolato: Ansible verra eseguito in un ambiente
+Toolbox dedicato per il provisioning del futuro cluster `uranus`, anziche da
+Ikaros o da un host non gestito. Lo stack di osservabilita rootless Podman
+eseguira Grafana, Prometheus e Loki. L'SSD locale sara l'hot storage, con
+metriche e log conservati per 30 giorni; esportazioni programmate trasferiranno
+i dati storici piu vecchi su un dataset Atlas montato via NFS come cold storage.
+Il piano di implementazione, con prerequisiti espliciti, e in `AGENTS.md`.
+
 ## Desktop
 
 Target operativi:

README.md

@@ -67,6 +67,27 @@ The official ChatGPT desktop RPM is enabled only on `ikaros` and `nymph`. The
 playbook configures OpenAI's signed RPM repository and imports its pinned RPM
 signing key before installation; subsequent updates are handled by DNF.
 
+## Deferred planned node: Cerberus
+
+`cerberus` is a **postponed** management-plane node, pending the physical setup
+of the office in the new house. It is not yet an inventory host and no role or
+playbook targets it.
+
+The planned hardware is a Lenovo ThinkCentre M700 Tiny (Intel Core i3-6100T,
+8 GB RAM, and a 256 GB SSD) with native 1 Gbps Ethernet. It will share Ikaros'
+monitor and peripherals through a multi-input KVM switch, using a passive
+DisplayPort-to-HDMI cable for its video connection. Fedora Sericea, the
+immutable Fedora variant with the Sway Wayland compositor, is the intended
+operating system.
+
+Cerberus will be an isolated management plane: Ansible will run from a
+dedicated Toolbox environment to provision the future `uranus` cluster, rather
+than from Ikaros or an unmanaged host. Its rootless Podman observability stack
+will run Grafana, Prometheus, and Loki. The local SSD is the hot tier and
+retains metrics and logs for 30 days; scheduled exports will place older
+historical data on an NFS-mounted Atlas dataset as the cold tier. The detailed,
+implementation-gated plan is maintained in `AGENTS.md`.
+
 ## Desktop profiles
 
 - `ikaros`: stable Fedora Workstation + GNOME desktop.
@@ -328,6 +349,12 @@ the Borg client as `borg` with temporary read-search capability and no ZFS, sudo
 privileges. Borg retains 30 daily, 8 weekly, and 12 monthly archives, then compacts the standard
 read-write repository. A full metadata and repository check runs as `borg` on the fifteenth day of each
 month at 06:00. Both operations use a common lock, journal logging, and bounded systemd retries.
+New backup runs also log the create phase and a compact progress line at most once per minute: dataset,
+files processed, and original/compressed/deduplicated bytes. Progress lines omit individual filenames
+and a percentage, since Borg does not know the total in advance; warnings may still name affected files.
+Follow the current run with
+`sudo journalctl -fu atlas-borg-backup.service` on Atlas; changes to the helper do not alter a run
+already in progress.
 
 Initial activation remains explicit:
 

ansible/inventory/host_vars/atlas.yml

@@ -103,6 +103,9 @@ rocky_podman_packages:
 
 host_packages:
   - cockpit
+  - cockpit-podman
+  - cockpit-storaged
+  - cryptsetup
   - nfs-utils
   - policycoreutils
   - policycoreutils-python-utils

ansible/inventory/host_vars/prometheus.yml

@@ -32,6 +32,7 @@ host_packages:
   - cockpit
   - cockpit-navigator
   - cockpit-podman
+  - cockpit-storaged
 
 host_enabled_services:
   - cockpit.socket

ansible/roles/profile_atlas/files/atlas-borg-progress.py

@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+"""Turn Borg's JSON progress stream into bounded, readable journal entries."""
+
+import json
+import sys
+import time
+
+
+def size(value):
+    if not isinstance(value, (int, float)):
+        return "unknown"
+    return f"{value / (1024 ** 3):.2f} GiB"
+
+
+last_progress = 0.0
+for line in sys.stdin:
+    try:
+        event = json.loads(line)
+    except json.JSONDecodeError:
+        print(line.rstrip(), flush=True)
+        continue
+
+    kind = event.get("type")
+    if kind == "archive_progress":
+        now = time.monotonic()
+        if now - last_progress < 60 and not event.get("finished"):
+            continue
+        path = event.get("path") or ""
+        parts = path.split("/")
+        dataset = parts[1] if len(parts) > 1 and parts[0] == "source" else "unknown"
+        print(
+            "Borg create progress: "
+            f"dataset={dataset} files={event.get('nfiles', 'unknown')} "
+            f"original={size(event.get('original_size'))} "
+            f"compressed={size(event.get('compressed_size'))} "
+            f"deduplicated={size(event.get('deduplicated_size'))}",
+            flush=True,
+        )
+        last_progress = now
+    elif kind == "log_message":
+        print(f"Borg {event.get('levelname', 'INFO')}: {event.get('message', '')}", flush=True)
+    elif kind == "progress_message" and event.get("message"):
+        print(f"Borg: {event['message']}", flush=True)

ansible/roles/profile_atlas/tasks/borg_backup.yml

@@ -224,7 +224,7 @@
   when: atlas_manage_borg_backup | bool
 
 - name: Install the Atlas Borg backup helper
-  tags: [atlas, storage, backup, borg]
+  tags: [atlas, storage, backup, borg, borg_logging]
   ansible.builtin.template:
     src: atlas-borg-backup.sh.j2
     dest: /usr/local/sbin/atlas-borg-backup
@@ -244,7 +244,7 @@
   when: atlas_manage_borg_backup | bool
 
 - name: Create the local libexec directory for the Atlas Borg SSH wrapper
-  tags: [atlas, storage, backup, borg]
+  tags: [atlas, storage, backup, borg, borg_logging]
   ansible.builtin.file:
     path: "{{ atlas_borg_ssh_wrapper_path | dirname }}"
     state: directory
@@ -253,6 +253,16 @@
     mode: "0755"
   when: atlas_manage_borg_backup | bool
 
+- name: Install the Atlas Borg progress formatter
+  tags: [atlas, storage, backup, borg, borg_logging]
+  ansible.builtin.copy:
+    src: atlas-borg-progress.py
+    dest: /usr/local/libexec/atlas-borg-progress
+    owner: root
+    group: root
+    mode: "0755"
+  when: atlas_manage_borg_backup | bool
+
 - name: Install the capability-dropping Atlas Borg SSH wrapper
   tags: [atlas, storage, backup, borg]
   ansible.builtin.template:

ansible/roles/profile_atlas/templates/atlas-borg-backup.sh.j2

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 set -Eeuo pipefail
 
-export LC_ALL=C
+export LC_ALL=C.utf8
 export PATH=/usr/sbin:/usr/bin:/sbin:/bin
 export BORG_CACHE_DIR={{ atlas_borg_cache_dir | quote }}
 export BORG_CONFIG_DIR={{ atlas_borg_config_dir | quote }}
@@ -21,6 +21,7 @@ readonly borg_user={{ atlas_borg_username | quote }}
 readonly borg_group={{ atlas_borg_group | quote }}
 readonly borg_home={{ atlas_borg_home | quote }}
 readonly borg_lock={{ atlas_borg_lock_path | quote }}
+readonly progress_filter=/usr/local/libexec/atlas-borg-progress
 
 snapshot_name=""
 snapshot_created=false
@@ -124,24 +125,31 @@ archive="${archive_prefix}-${timestamp}"
 readonly archive
 borg_status=0
 
+printf 'Starting Borg archive %s from snapshot %s@%s\n' "$archive" "$pool" "$snapshot_name"
 set +e
 (
   cd /run/atlas-borg
-  run_as_borg borg --remote-path "$remote_path" --lock-wait 600 create \
+  run_as_borg borg --remote-path "$remote_path" --lock-wait 600 --log-json --progress create \
     --show-rc \
     --stats \
     --checkpoint-interval 900 \
     --compression "$compression" \
     "${repository}::${archive}" \
-    source
+    source 2>&1
-)
+) | /usr/bin/python3 -u "$progress_filter"
-create_status=$?
+create_pipeline_status=("${PIPESTATUS[@]}")
 set -e
+create_status=${create_pipeline_status[0]}
+if ((create_pipeline_status[1] != 0)); then
+  printf 'Borg progress logging failed with status %s\n' "${create_pipeline_status[1]}" >&2
+  exit 2
+fi
 if ((create_status >= 2)); then
   exit "$create_status"
 fi
 borg_status=$create_status
 
+printf 'Borg archive %s created; applying retention\n' "$archive"
 set +e
 run_as_borg borg --remote-path "$remote_path" --lock-wait 600 prune \
   --show-rc \
@@ -160,6 +168,7 @@ if ((prune_status > borg_status)); then
   borg_status=$prune_status
 fi
 
+printf 'Borg retention complete; compacting repository\n'
 set +e
 run_as_borg borg --remote-path "$remote_path" --lock-wait 600 compact \
   --show-rc \
@@ -173,4 +182,5 @@ if ((compact_status > borg_status)); then
   borg_status=$compact_status
 fi
 
+printf 'Borg backup %s completed with status %s\n' "$archive" "$borg_status"
 exit "$borg_status"

secrets/vault.yml.example

@@ -3,14 +3,12 @@ vault_duckdns_token: "CHANGEME"
 vault_personal_full_name: "REPLACE_ME"
 vault_git_email: "REPLACE_ME"
 vault_git_signing_key: "REPLACE_ME"
-vault_icloud_email: "REPLACE_ME"
-vault_protonmail_email: "REPLACE_ME"
 vault_icloud_mail_password: "REPLACE_ME"
 vault_git_work_email: "REPLACE_ME"
 vault_git_work_gpg: "REPLACE_ME"
-vault_openai_api_key: "REPLACE_ME"
 vault_ikaros_authorized_ssh_keys:
   - "ssh-ed25519 REPLACE_ME"
+vault_aegis_icloudpd_apple_id: "REPLACE_ME"
 vault_atlas_admin_password_hash: "REPLACE_WITH_A_SHADOW_COMPATIBLE_HASH"
 vault_atlas_samba_password: "REPLACE_ME"
 vault_atlas_immich_db_password: "REPLACE_ME"