25 Commits
Author SHA1 Message Date
homelab-updaterandClaude Opus 5.5 765a1e0eee docs: TASK_STATE after deploying haiku
ci / check (push) Successful in 1m36s
ci / publish (push) Successful in 1m27s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 00:19:51 +02:00
homelab-updaterandClaude Opus 5.5 b29aec3dee docs: TASK_STATE after adding haiku
ci / check (push) Successful in 1m39s
ci / publish (push) Successful in 1m43s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:57:25 +02:00
homelab-updaterandClaude Opus 5.5 964f2ad815 feat(models): add haiku as a model choice and make it the default
Haiku 5.5 is cheaper and close enough to Sonnet for the release notes
search and assessment, the repair, the migration plan and the
post-upgrade check, so an app that names no model now gets haiku.

- ModelName (haiku|sonnet|opus) is the one vocabulary; modelOr picks the
  default where the old code fell back to sonnet or opus.
- A default "haiku" profile is seeded next to sonnet and opus.
- Migration 26 rebuilds app_config so its CHECKs accept haiku and its
  release notes default is haiku; existing rows are copied verbatim.
- claude CLI 2.1.293: the first release that knows claude-haiku-5-5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:57:25 +02:00
homelab-updaterandClaude Opus 5.5 4b9fa93ed0 docs: TASK_STATE after the post-upgrade check change
ci / check (push) Successful in 1m32s
ci / publish (push) Successful in 1m28s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:59:38 +02:00
homelab-updaterandClaude Opus 5.5 c17b170739 feat(post-upgrade): run the model only when the update may ask for something
ci / check (push) Successful in 1m33s
ci / publish (push) Successful in 1m35s
The post-upgrade model ran after every update: 17 of 18 checks found nothing,
mostly LinuxServer rebuilds of the same app version. And "off" did not turn it
off: the workflow port read a null executor as sonnet.

- A null postUpgradeExecutor is off again: no session, no model.
- New per-app postUpgradeWhen (migration 25): when_needed (default) or always.
- when_needed runs the model for an update the owner approved (release notes
  read), a major one or a migration; otherwise it reads the service's last 200
  log lines in an observe session and calls the model only on errors there.
- ExecutorClient.sessionLogs reads /agent/logs with the session token, like
  sessionCompose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:54:48 +02:00
homelab-updaterandClaude Opus 5.5 bcd144fbdc fix(runs): count a model run once when the terminal node carries its result on
ci / check (push) Successful in 1m35s
ci / publish (push) Successful in 1m33s
A terminal node's result is the record it received, so after post_upgrade the run
listed the same post-upgrade check twice and counted its cost twice (run view, run
list and model costs). Identical calls now count once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:31:30 +02:00
homelab-updaterandClaude Opus 5.5 d05b6c27f3 docs: TASK_STATE after the registry rate limit fix
ci / check (push) Successful in 1m34s
ci / publish (push) Successful in 1m25s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 00:34:18 +02:00
homelab-updaterandClaude Opus 5.5 e060a59b67 fix(registry): wait out a ghcr.io rate limit instead of reporting the version unknown
ci / check (push) Successful in 1m38s
ci / publish (push) Successful in 1m25s
ghcr.io answered 429 to the tag list for jackett, radarr and sonarr, and image_tags
turned that straight into version_unknown with only "skip" left. The registry client
now asks again after 429, honouring Retry-After up to five minutes; the driver waits
30 s, 2 min and 5 min, the server does not wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 00:28:08 +02:00
homelab-updaterandClaude Opus 5.5 8966fec9cf docs: TASK_STATE after the migrate and verify fixes
ci / check (push) Successful in 1m35s
ci / publish (push) Successful in 1m20s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 23:55:43 +02:00
homelab-updaterandClaude Opus 5.5 1b8bfc4446 fix(verify): wait out the old container while the service is recreated
ci / check (push) Successful in 2m31s
ci / publish (push) Successful in 3m9s
For a Git project Arcane recreates the service while verify polls, and
for a moment compose has the old container next to the new one. The
executor refuses to inspect that (multiple_containers), and verify
failed at once, so a healthy update went to model.repair (deluge,
homeassistant, nextcloud, nextcloud-redis). That refusal now counts as
an ordinary failed attempt within the usual 24 x 5 s; any other refusal
still fails at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 23:31:33 +02:00
homelab-updaterandClaude Opus 5.5 d00c8ad4aa fix(migrate): run an approved migration that keeps the compose file
ci / check (push) Successful in 1m35s
ci / publish (push) Successful in 1m29s
A migration plan may leave composeYaml null ("only the image changes"),
but the docker.migrate request required a compose file, so building the
request failed (invalid_privileged_action_request) right after the
backup. Gitea 1.27.3 -> 28.0.0 stopped there. The request now carries
null and the executor keeps the current compose.yaml (read at apply
time, matching the backup) and only pins the new image. The plan's
earlier snapshot is not used: another update of the same project may
have changed the file since.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 21:51:59 +02:00
homelab-updaterandClaude Opus 5.5 2e3d61982f docs: TASK_STATE and phase H status after deploying the login
ci / check (push) Successful in 1m34s
ci / publish (push) Successful in 1m22s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 14:06:22 +02:00
homelab-updaterandClaude Opus 5.5 6cf584894d fix(deploy): start the auth service, and roll back a start that fails part-way
ci / check (push) Successful in 1m36s
ci / publish (push) Successful in 1m22s
Compose writes an inline config only into a writable container, so the
auth service is no longer read_only; it trusts X-Forwarded-* only from
the Docker networks. The first deploy of the login stopped at compose up
and update.sh exited there, leaving the app without its auth backend
and the driver not started: a failed up now goes back to the previous
compose file and image like a start that does not come up healthy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 14:02:01 +02:00
homelab-updaterandClaude Opus 5.5 7131ca9b12 fix(deploy): keep VM103's disk from filling with images and build cache
ci / check (push) Successful in 1m33s
ci / publish (push) Successful in 1m25s
Every CI build and deploy left a 1.2 GB image of this app on the VM and
the build cache kept growing, until the root disk was full and apt in
CI failed. After a good deploy update.sh keeps the running image, the
previous one and the three newest; the publish job prunes build cache
older than two days.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:44:30 +02:00
homelab-updaterandClaude Opus 5.5 4b6b179525 docs: Trilium connected to Arcane Git Sync
ci / check (push) Failing after 6s
ci / publish (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:59:56 +02:00
homelab-updaterandClaude Opus 5.5 a2f289b4fb feat(deploy): Keycloak login in front of the app, open beyond the LAN
ci / check (push) Successful in 2m48s
ci / publish (push) Successful in 2m46s
An auth service (oauth2-proxy v7.15.4, pinned) answers Traefik's
forwardAuth for every request: without a session the browser goes to
Keycloak (realm home) and only the addresses in AUTH_ALLOWED_EMAILS get
in; the LAN-only ipwhitelist goes. Secrets stay in .env on VM103 and
compose refuses without them. update.sh now checks every service of the
compose file after a deploy instead of a fixed list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:31:29 +02:00
homelab-updaterandClaude Opus 5.5 fc68d30734 feat(web): relative times, a go-to palette and dry runs of all updates
ci / check (push) Successful in 1m35s
ci / publish (push) Successful in 1m22s
Lists say "5 min ago" or "in 3 h" (full time on hover, refreshed each
minute). Ctrl+K (or /, or the sidebar's Search) opens a palette that
jumps to pages, apps, recent runs, workflows and functions. The apps
page starts a dry run of every enabled app with an available update,
after one confirmation, and says which were skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:27:24 +02:00
homelab-updaterandClaude Opus 5.5 280d9dc0c9 feat(web): decide from the phone
ci / check (push) Successful in 1m32s
ci / publish (push) Skipped
On a narrow screen the gate's decision buttons stick to the bottom
while the owner reads the gate, larger and two per row; the decision
card gets the window look. A web app manifest, icon and theme color let
the app sit on the phone's home screen and open like an app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:23:14 +02:00
homelab-updaterandClaude Opus 5.5 dbe9abeebf feat: quiet hours and a morning report
ci / check (push) Successful in 1m32s
ci / publish (push) Successful in 1m25s
Owner settings (quiet hours 01:00-07:30, report at 07:30, schedules'
time zone) live in settings, with GET/PUT /api/notification-settings and
a Settings page. In quiet hours the driver holds every push; once a day
from the report time it pushes what waits for the owner, what failed or
is down and what changed since the last report. The overview shows the
same report on top.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:21:17 +02:00
homelab-updaterandClaude Opus 5.5 055e892f92 feat(web): an app's version history and apps that often need care
ci / check (push) Successful in 1m30s
ci / publish (push) Successful in 1m33s
Real runs of an app become its history, newest first: updated from what
to what, not updated (skipped, rolled back) or failed, with a repair by
the model and the release notes verdict and summary of that run. An app
with two or more such runs among its last ten gets a mark on its card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:15:18 +02:00
homelab-updaterandClaude Opus 5.5 3798b8950d feat: what runs on VM103 right now and the age of the VM backup
ci / check (push) Successful in 1m31s
ci / publish (push) Successful in 1m25s
The driver looks once a minute: one docker ps of the compose containers
(a GET through the read-only proxy) and the Proxmox timer's vzdump
record. It keeps the latest look in settings; GET /api/host-status
serves it. The overview shows how many enabled apps run well (which do
not) and how old the newest VM backup is; app cards and the table get a
dot for each app's container.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:12:53 +02:00
homelab-updaterandClaude Opus 5.5 dbb17ea5b9 feat(config): manage Trilium (TriliumNext) with a Sunday schedule
ci / check (push) Successful in 1m40s
ci / publish (push) Successful in 1m27s
Auto stays at patch: on 0.x every minor migrates the database one way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:01:54 +02:00
homelab-updaterandClaude Opus 5.5 12cca5586b docs: TASK_STATE with the deploy button
ci / publish (push) Successful in 1m17s
ci / check (push) Successful in 1m31s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:36:37 +02:00
homelab-updaterandClaude Opus 5.5 136f9d2fb0 feat(deploy): deploy a release by hand from Gitea Actions
ci / check (push) Successful in 1m30s
ci / publish (push) Successful in 1m22s
A deploy workflow, started only by hand (or through the API) and only
from master, pulls the chosen image (release tag, sha or latest) and
runs that image's own update.sh against VM103's compose directory
through the Docker socket. update.sh now copies the database through the
running server first, installs the image's compose.yaml and update.sh
with --install, and restores the previous compose file together with
the previous image when the new version does not come up. CI also
builds release tags v*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:30:19 +02:00
homelab-updaterandClaude Opus 5.5 119dd18d43 docs: TASK_STATE after deploying phase G to VM103
ci / check (push) Successful in 1m29s
ci / publish (push) Successful in 1m18s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:24:46 +02:00
100 changed files with 2669 additions and 182 deletions

No files matched your search

+2
View File
@@ -6,6 +6,8 @@ data
docs
tools
deploy
!deploy/compose.yaml
!deploy/update.sh
.env
.env.*
*.log
+5 -3
View File
@@ -1,5 +1,5 @@
# CI: `npm run check` on every push; on master/foundations also build the image and push it to the Gitea registry.
# It does not deploy: VM103 pulls the image with deploy/update.sh (docs/DEPLOY.md).
# CI: `npm run check` on every push; on master/foundations and on release tags (v…) also build the image and push it to
# the Gitea registry. It does not deploy: that is the deploy workflow, started by hand (docs/DEPLOY.md).
name: ci
on:
@@ -28,7 +28,7 @@ jobs:
publish:
needs: check
if: github.ref_name == 'master' || github.ref_name == 'foundations'
if: github.ref_name == 'master' || github.ref_name == 'foundations' || startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -53,3 +53,5 @@ jobs:
# Push only these refs: the daemon is the VM's, so `--all-tags` would also push the local :rollback tag from update.sh.
for ref in "${refs[@]}"; do docker push "${ref}"; done
docker logout "${REGISTRY}"
# Builds run on the VM's daemon: without this its build cache grows until the disk is full.
docker builder prune -f --filter until=48h >/dev/null 2>&1 || true
+61
View File
@@ -0,0 +1,61 @@
# Deploys one published image to VM103, only when started by hand: Actions → deploy → Run workflow, or the Gitea API.
# The new image's own deploy/update.sh does the work against /opt/homelab/compose/homelab-upgrade, so its guards stay:
# it refuses while a workflow run is in the middle of a step, copies the database first, installs the image's
# compose.yaml, waits for healthy and goes back to the previous image and compose file if the new version does not come up.
name: deploy
on:
workflow_dispatch:
inputs:
version:
description: "Image to deploy: a release tag (v2026.10.1), a full commit sha, or latest (master)"
required: true
default: latest
type: string
force:
description: "Restart even while a workflow run is in the middle of a step, or without a database copy"
required: false
default: false
type: boolean
env:
REGISTRY: git.mariuszpacek.pl
DEPLOY_DIR: /opt/homelab/compose/homelab-upgrade
jobs:
deploy:
# Only the reviewed definition on master may restart production.
if: github.ref_name == 'master'
runs-on: ubuntu-latest
steps:
- name: Check the version
env:
VERSION: ${{ inputs.version }}
run: |
[[ "$VERSION" =~ ^(latest|v[0-9][0-9A-Za-z._-]{0,40}|[0-9a-f]{40})$ ]] || { echo "version must be latest, a v… tag or a full commit sha, not: $VERSION"; exit 2; }
- name: Docker CLI available (talks to the VM daemon through the mounted socket)
run: command -v docker >/dev/null || (apt-get update -qq && apt-get install -y -qq docker.io >/dev/null); docker version --format '{{.Client.Version}} -> {{.Server.Version}}'
- name: Pull the image
env:
VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
image="${REGISTRY}/${GITHUB_REPOSITORY_OWNER}/homelab-upgrade:${VERSION}"
docker pull -q "$image"
echo "revision: $(docker inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$image")"
- name: Deploy
env:
VERSION: ${{ inputs.version }}
FORCE: ${{ inputs.force }}
run: |
set -euo pipefail
image="${REGISTRY}/${GITHUB_REPOSITORY_OWNER}/homelab-upgrade:${VERSION}"
args=("$VERSION" --install)
[ "$FORCE" = true ] && args+=(--force)
# root for the Docker socket and the root-owned compose directory; the same path inside, so compose resolves
# the project (name, .env, paths) exactly as on the VM.
docker run --rm --user 0 --entrypoint /app/deploy/update.sh \
-e DEPLOY_DIR="$DEPLOY_DIR" \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$DEPLOY_DIR:$DEPLOY_DIR" \
"$image" "${args[@]}"
+4 -2
View File
@@ -5,7 +5,7 @@ ARG NODE_IMAGE=node:24-slim@sha256:2fe369e969550cde8e867afc3fe370b260140cab4a23d
ARG DOCKER_CLI_IMAGE=docker:28.5.2-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d
# The claude CLI for the model sidecar, pinned and updated deliberately (docs/DESIGN.md, "Wykonawcy modeli").
ARG CLAUDE_CODE_VERSION=2.1.267
ARG CLAUDE_CODE_VERSION=2.1.293
# The codex CLI for the model sidecar, pinned and updated deliberately (docs/DESIGN.md, "Wykonawcy modeli").
ARG CODEX_VERSION=0.156.1
@@ -48,6 +48,8 @@ COPY --from=codex /opt/codex /opt/codex
COPY bin ./bin
COPY src ./src
COPY config ./config
RUN chmod -R a+rX ./src ./config ./bin
# The deploy workflow runs this image's own update.sh against VM103's compose directory (docs/DEPLOY.md).
COPY deploy/compose.yaml deploy/update.sh ./deploy/
RUN chmod -R a+rX ./src ./config ./bin ./deploy
USER node
CMD ["node", "src/main/server.ts"]
+18 -13
View File
@@ -1,31 +1,36 @@
# Stan zadania — nowoczesny frontend (Faza G)
# Stan zadania — faza H (wygoda na co dzień)
## Cel
Bardziej nowoczesny i graficzny frontend: okna/karty, wykresy z danych, które API już zwraca. Plan i status: `docs/IMPLEMENTATION.md`, „Faza G”.
Aplikacja wygodna na co dzień: stan „tu i teraz”, historia wersji, cisza nocna i poranny raport, decyzja z telefonu, szybka nawigacja, logowanie Keycloak z dostępem spoza LAN. Plan i status: `docs/IMPLEMENTATION.md`, „Faza H”.
## Decyzje
- Sidebar + karty, ciemny motyw domyślny z przełącznikiem (`src/web/theme.ts`).
- ECharts (`src/web/charts/echarts.ts`, osobny chunk), opcje wykresów jako czyste funkcje w `src/web/charts/options.ts`, agregacje w `src/web/stats.ts`; kolory z tokenów CSS, kategorie `--series-1..8`.
- Okna: `src/web/components/dialogs.tsx` (`useConfirm`, `useToast`, `Drawer`); `window.confirm` nie jest już używany.
- Bez zmian w backendzie; lista runów pobierana w całości — przy dużej liczbie runów rozważyć endpoint agregatów.
- Logowanie: oauth2-proxy (usługa `auth`) jako forwardAuth Traefika, Keycloak realm `home`, klient `homelab-upgrade` (założony przez właściciela), wpuszczany tylko `AUTH_ALLOWED_EMAILS`; sekrety w `.env` na VM103 (root 600). Publiczne tylko `manifest.webmanifest` i `icon.svg`.
- Cisza nocna 01:00–07:30, raport o 7:30 (strefa harmonogramów); ustawienia w UI (`#/settings`).
- Wdrożenia tylko przyciskiem `deploy` (Actions albo MCP `dispatch_workflow`), nigdy przez Arcane.
## Zrobione
- G0–G6 na gałęzi `frontend-refresh` (worktree `/root/homelab-upgrade-frontend`), każdy etap osobnym commitem, wypchnięte.
- Graf Cytoscape nie ponawia nieudanego startu canvas przy każdym renderze (wcześniej czyścił listę zastępczą).
- H1–H6 na `master` i na VM103: migawka hosta, historia wersji, raport i cisza nocna, decyzje z telefonu, PWA, `Ctrl+K`, logowanie Keycloak.
- Poprawki po pierwszych nocach: migracja z `composeYaml: null` (executor bierze bieżący compose i przypina obraz; `d00c8ad`, Gitea 28.0.0 zaktualizowana), `verify` czeka na zniknięcie starego kontenera przy przebudowie przez Arcane zamiast od razu wołać `model.repair` (`1b8bfc4`).
- `version_unknown` u jackett/radarr/sonarr (5–6.10) to był HTTP 429 z ghcr.io przy liście tagów, nie wersja: klient rejestru ponawia po 429 (driver 30 s / 2 min / 5 min, `Retry-After` do 5 min; serwer bez czekania), wdrożone `e060a59`.
- Podwójny wpis modelu w runie: węzeł końcowy powtarzał wynik `post_upgrade`; liczony raz (`bcd144f`).
- Krok modelu po aktualizacji (`c17b170`): `postUpgradeExecutor: null` znów wyłącza (port workflow czytał null jako sonnet), nowe `postUpgradeWhen` (`when_needed` domyślnie / `always`, migracja 25). `when_needed`: model tylko przy aktualizacji zatwierdzonej przez właściciela, major, migracji albo błędach w ostatnich 200 liniach logów (sesja observe, bez modelu). Na VM103 wyłączone dla 11 aplikacji (arr, jackett, bazarr, jellyfin, ombi, deluge, orcaslicer, homepage, trilium, nextcloud-redis), reszta sonnet + `when_needed`.
- Haiku 5.5 jako trzeci model (`haiku`): domyślny dla notatek wydania (szukanie + ocena), naprawy, planu migracji i sprawdzenia po aktualizacji, gdy aplikacja nie wybrała modelu; profil `haiku` w seedzie, migracja 26 (przebudowa `app_config` dla CHECK), CLI w obrazie 2.1.293 (pierwsza wersja znająca `claude-haiku-5-5`). Wdrożone `b29aec3` (8.10); na VM103 wszystkie 22 aplikacje przełączone z `sonnet` na `haiku` (wyłączone zostały wyłączone).
- Homelab-compose: whitelist deluge przepuszcza bramę sieci `proxy` (`172.16.0.1/32`), inaczej `verify` dostawał 403.
- Proxmox: pełny dysk 30.09 zawiesił pmxcfs (brak vzdump 1–2.10); po restarcie usług PVE kopie działają. VM103: przycięte obrazy (dysk 89% → 64%).
## Pozostałe
- Przegląd na żywo w przeglądarce (oba motywy, 375 px i desktop), poprawki wyglądu po przeglądzie.
- Merge `frontend-refresh` do `master` i wdrożenie na VM103 — decyzja właściciela.
- Ze strumienia Git/Arcane: sprawdzenie CI dla master i przejście z obrazów hotfix na obraz CI; migracja klientów Influx przed włączeniem aktualizacji.
- Właściciel loguje się raz w przeglądarce i na telefonie (potwierdzenie Keycloak → aplikacja).
- Ze strumienia Git/Arcane: migracja klientów Influx przed włączeniem aktualizacji.
- Do rozważenia: uproszczenie przebiegów (ustawienia per aplikacja zamiast edycji grafu, deklaratywny warunek zamiast funkcji TS), wyłączenie godzinnego sprawdzania aktualizacji obrazów w Arcane (dubluje nasz system), bramka „verify failed” bez opcji „accept”, sprzątanie `# saved as ./compose.yaml` z planów migracji, automatyczny `pg_dump` przed migracją bazy.
## Następny krok
Właściciel ogląda UI (`npm run build:web` + `npm run start:server`, albo `npm run dev:web` z działającym serwerem) i zgłasza poprawki.
W pierwszym runie z modelem sprawdzić, że alias `haiku` w CLI 2.1.293 to Haiku 5.5 (`modelUsage` w odpowiedzi `claude`; próba 8.10 trafiła na limit sesji konta) i czy jakość ocen wystarcza. Dalej obserwować 429 i `when_needed`.
## Walidacja
`npm run check` zielone: 107 plików, 1260+ testów. Testy agregacji i opcji wykresów: `npx vitest run src/web/stats.test.ts src/web/charts`.
`npm run check`: 1320 testów PASS. `deploy/update.sh` sprawdzany na atrapie `docker` (instalacja, kopia bazy, nieudany start i nieudane `up`, przycinanie obrazów).
+2 -1
View File
@@ -18,5 +18,6 @@
{"app": "ombi", "service": "ombi", "imageRepo": "linuxserver/ombi", "tagPattern": "^v[0-9]+\\.[0-9]+\\.[0-9]+-ls[0-9]+$", "auto": "patch", "httpUrl": "https://ombi.mariuszpacek.pl/", "expectStatus": 200},
{"app": "orcaslicer", "service": "orcaslicer", "imageRepo": "linuxserver/orcaslicer", "tagPattern": "^v[0-9]+\\.[0-9]+\\.[0-9]+-ls[0-9]+$", "auto": "minor", "httpUrl": "https://orca.mariuszpacek.pl/", "expectStatus": 401},
{"app": "radarr", "service": "radarr", "imageRepo": "ghcr.io/linuxserver/radarr", "tagPattern": "^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+-ls[0-9]+$", "auto": "patch", "httpUrl": "https://radarr.mariuszpacek.pl/", "expectStatus": 302},
{"app": "sonarr", "service": "sonarr", "imageRepo": "ghcr.io/linuxserver/sonarr", "tagPattern": "^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+-ls[0-9]+$", "auto": "patch", "httpUrl": "https://sonarr.mariuszpacek.pl/", "expectStatus": 302}
{"app": "sonarr", "service": "sonarr", "imageRepo": "ghcr.io/linuxserver/sonarr", "tagPattern": "^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+-ls[0-9]+$", "auto": "patch", "httpUrl": "https://sonarr.mariuszpacek.pl/", "expectStatus": 302},
{"app": "trilium", "service": "trilium", "imageRepo": "triliumnext/trilium", "tagPattern": "^v[0-9]+\\.[0-9]+\\.[0-9]+$", "auto": "patch", "httpUrl": "https://trilium.mariuszpacek.pl/api/health-check", "expectStatus": 200, "notes": "0.x: every minor migrates the database one way"}
]
+15
View File
@@ -0,0 +1,15 @@
# Trilium — VM103
Usługa `trilium`, obraz `triliumnext/trilium` — TriliumNext, utrzymywany fork `zadam/trilium` (oryginał w trybie utrzymania). Domena `trilium.mariuszpacek.pl` (także `notes.mariuszpacek.pl`), port 8080 tylko w sieci `proxy`, bez portu na hoście. Whitelist traefika jak w bazarr (LAN, VPN, `172.16.0.1/32` dla drivera). Aplikacja ma własne logowanie hasłem.
Dane: `/home/node/trilium-data` ← `/srv/appdata/trilium/data` (`document.db` SQLite, `config.ini`, `backup/`, `log/`), właściciel `1000:1000`, tryb `700`. Entrypoint startuje jako root, robi `chown -R node:node /home/node` i uruchamia serwer jako `node` (uid 1000) — nie ustawiaj `user:` w compose.
Obraz ma healthcheck (`/api/health-check`) i etykietę OCI `org.opencontainers.image.version` z tagiem.
Źródła: [wydania](https://github.com/TriliumNext/Trilium/releases), [dokumentacja](https://docs.triliumnotes.org/), [Docker](https://docs.triliumnotes.org/user-guide/setup/server/installation/docker).
Wersje: tagi `vX.Y.Z` (bez `-rc`, `main`, `sha-*`, `latest`). Seria 0.x: każdy minor to wydanie z nowymi funkcjami i zwykle z jednokierunkową migracją bazy — stąd auto `patch`, a minor czeka na decyzję właściciela.
Migracja bazy przy starcie w jednej transakcji; przed nią Trilium sam zapisuje `backup/backup-before-migration.db`. Historia: 2026-09-30 baza przeniesiona z `/fast-vm/docker/trilium/mariusz` na Proxmoksie (Trilium 0.60, dbVersion 214) i zmigrowana na v0.106.0 (dbVersion 240); kopia sprzed tej migracji: `backup/backup-before-migration-v060.db`.
Kopia: codzienny vzdump VM103 obejmuje `/srv/appdata`; workflow kopiuje compose przed zmianą. Dodatkowa kopia bazy nie jest wymagana, bo Trilium robi ją sam przed migracją.
Odbiór: kontener `healthy` na przypiętym obrazie; `/api/health-check` przez HTTPS → 200; w logu `Migration to version N has been successful` (jeśli była migracja) i `All consistency checks passed`; brak `Error` przy starcie. Treść notatek sprawdza właściciel po zalogowaniu.
STOP: log pokazuje nieudaną migrację (`migration failed`, crash przy starcie) albo błędy spójności bazy — nie naprawiaj bazy, opisz i zostaw decyzję właścicielowi.
Powrót: rollback przywraca compose sprzed aktualizacji, ale starsza wersja odmówi otwarcia bazy po migracji. Wtedy trzeba przy zatrzymanym kontenerze przywrócić `document.db` z `backup/backup-before-migration.db` (albo z vzdumpu) — tylko za zgodą właściciela.
+6
View File
@@ -120,6 +120,12 @@
"cron": "0 17 * * 6",
"mode": "real",
"enabled": false
},
{
"workflowId": "update-trilium",
"cron": "0 16 * * 0",
"mode": "real",
"enabled": true
}
]
}
+63 -3
View File
@@ -37,13 +37,15 @@ services:
traefik.http.routers.homelab-upgrade.rule: 'Host(`upgrade.mariuszpacek.pl`)'
traefik.http.routers.homelab-upgrade.entrypoints: "web-secured"
traefik.http.routers.homelab-upgrade.priority: "100"
traefik.http.routers.homelab-upgrade.middlewares: "homelab-upgrade-lan"
traefik.http.routers.homelab-upgrade.middlewares: "homelab-upgrade-login"
traefik.http.routers.homelab-upgrade.tls: "true"
traefik.http.routers.homelab-upgrade.tls.certresolver: "http"
traefik.http.routers.homelab-upgrade.service: "homelab-upgrade"
traefik.http.services.homelab-upgrade.loadbalancer.server.port: "8080"
# No login yet: LAN only.
traefik.http.middlewares.homelab-upgrade-lan.ipwhitelist.sourcerange: "192.168.0.1/24,10.0.0.0/16"
# Every request asks the auth service first; without a session the browser goes to Keycloak (service auth below).
traefik.http.middlewares.homelab-upgrade-login.forwardauth.address: "http://homelab-upgrade-auth:4180/"
traefik.http.middlewares.homelab-upgrade-login.forwardauth.trustForwardHeader: "true"
traefik.http.middlewares.homelab-upgrade-login.forwardauth.authResponseHeaders: "X-Auth-Request-Email,X-Auth-Request-User"
# Tile on dash.mariuszpacek.pl: homepage discovers it through its own Docker connection (my-docker), no edit of its files.
homepage.group: "Infra"
homepage.name: "Upgrade"
@@ -53,6 +55,59 @@ services:
homepage.server: "my-docker"
homepage.container: "homelab-upgrade-server-1"
# Login in front of the app (oauth2-proxy, Traefik forwardAuth with a static upstream): a request without a session
# is sent to Keycloak (realm home), and only the e-mail addresses in AUTH_ALLOWED_EMAILS get in. The client secret,
# the cookie secret and the addresses live in .env on VM103 (docs/DEPLOY.md, "Logowanie"). The session cookie lasts
# 7 days and is refreshed hourly. The manifest and the icon stay public: a phone fetches them without the cookie.
auth:
image: quay.io/oauth2-proxy/oauth2-proxy:v7.15.4@sha256:b1b2021fe8f4004573e8d690dec6c7bb29cc44364572cf8510a05bf3a0ae2ded
restart: unless-stopped
user: "65532:65532"
# Not read_only: compose writes an inline config (auth-emails) only into a writable container filesystem.
cap_drop: [ALL]
security_opt: ["no-new-privileges:true"]
environment:
OAUTH2_PROXY_PROVIDER: keycloak-oidc
OAUTH2_PROXY_OIDC_ISSUER_URL: https://login.mariuszpacek.pl/realms/home
OAUTH2_PROXY_CLIENT_ID: ${AUTH_CLIENT_ID:-homelab-upgrade}
OAUTH2_PROXY_CLIENT_SECRET: ${AUTH_CLIENT_SECRET:?AUTH_CLIENT_SECRET is not set in .env}
OAUTH2_PROXY_COOKIE_SECRET: ${AUTH_COOKIE_SECRET:?AUTH_COOKIE_SECRET is not set in .env}
OAUTH2_PROXY_CODE_CHALLENGE_METHOD: S256
OAUTH2_PROXY_REDIRECT_URL: https://upgrade.mariuszpacek.pl/oauth2/callback
OAUTH2_PROXY_HTTP_ADDRESS: 0.0.0.0:4180
OAUTH2_PROXY_REVERSE_PROXY: "true"
# Only Traefik (on the Docker proxy network) may say who the client is through X-Forwarded-*.
OAUTH2_PROXY_TRUSTED_PROXY_IPS: 172.16.0.0/12
OAUTH2_PROXY_UPSTREAMS: static://202
OAUTH2_PROXY_EMAIL_DOMAINS: "*"
OAUTH2_PROXY_AUTHENTICATED_EMAILS_FILE: /etc/oauth2-proxy/emails
OAUTH2_PROXY_SKIP_PROVIDER_BUTTON: "true"
OAUTH2_PROXY_SKIP_AUTH_ROUTES: "GET=^/manifest\\.webmanifest$$,GET=^/icon\\.svg$$"
OAUTH2_PROXY_COOKIE_NAME: _homelab_upgrade
OAUTH2_PROXY_COOKIE_SECURE: "true"
OAUTH2_PROXY_COOKIE_EXPIRE: 168h
OAUTH2_PROXY_COOKIE_REFRESH: 1h
OAUTH2_PROXY_SET_XAUTHREQUEST: "true"
OAUTH2_PROXY_SHOW_DEBUG_ON_ERROR: "false"
configs:
- source: auth-emails
target: /etc/oauth2-proxy/emails
networks:
proxy:
# A name no other project on the proxy network uses: Traefik's forwardAuth address points at it.
aliases: [homelab-upgrade-auth]
labels:
traefik.enable: "true"
traefik.docker.network: "proxy"
# The sign-in callback and sign-out go straight to the auth service.
traefik.http.routers.homelab-upgrade-auth.rule: 'Host(`upgrade.mariuszpacek.pl`) && PathPrefix(`/oauth2/`)'
traefik.http.routers.homelab-upgrade-auth.entrypoints: "web-secured"
traefik.http.routers.homelab-upgrade-auth.priority: "200"
traefik.http.routers.homelab-upgrade-auth.tls: "true"
traefik.http.routers.homelab-upgrade-auth.tls.certresolver: "http"
traefik.http.routers.homelab-upgrade-auth.service: "homelab-upgrade-auth"
traefik.http.services.homelab-upgrade-auth.loadbalancer.server.port: "4180"
# The driver: runs queued workflow runs one step at a time and pushes to the owner's phone when a run waits for them,
# fails, or a real run ends. It finds the running containers by their compose labels (docker ps through the socket proxy
# below, GET only) and asks the registries what is newer. It reads no compose project file and changes nothing itself:
@@ -205,6 +260,11 @@ services:
cap_drop: [ALL]
security_opt: ["no-new-privileges:true"]
configs:
# One address per line; nobody else gets in, whatever else Keycloak's realm holds.
auth-emails:
content: ${AUTH_ALLOWED_EMAILS:?AUTH_ALLOWED_EMAILS is not set in .env}
networks:
proxy:
name: proxy
+92 -9
View File
@@ -3,25 +3,37 @@
# Lives next to compose.yaml on VM103 (/opt/homelab/compose/homelab-upgrade). See docs/DEPLOY.md.
#
# ./update.sh deploy the tag from .env (TAG=..., default "latest")
# ./update.sh <tag> deploy <tag> (a branch name or a commit sha) and remember it in .env
# ./update.sh --force skip the "a workflow run is in the middle of a step" guard
# ./update.sh <tag> deploy <tag> (a release tag, a branch name or a commit sha) and remember it in .env
# ./update.sh --force skip the "a workflow run is in the middle of a step" guard and a failed database copy
# ./update.sh --install first put this script's compose.yaml and update.sh into DEPLOY_DIR: the deploy workflow
# runs the copy inside the new image (/app/deploy) against the VM's directory
set -euo pipefail
cd "$(dirname "$(readlink -f "$0")")"
script_dir="$(dirname "$(readlink -f "$0")")"
cd "${DEPLOY_DIR:-$script_dir}"
image_repo=git.mariuszpacek.pl/mariusz.pacek/homelab-upgrade
healthy_timeout=${HEALTHY_TIMEOUT:-90}
backups_kept=${BACKUPS_KEPT:-5}
images_kept=${IMAGES_KEPT:-3}
tag_arg=""
force=0
install=0
for arg in "$@"; do
case "$arg" in
--force) force=1 ;;
--install) install=1 ;;
-*) echo "unknown option: $arg" >&2; exit 2 ;;
*) tag_arg="$arg" ;;
esac
done
if ((install == 1)) && [[ "$(pwd -P)" == "$(readlink -f "$script_dir")" ]]; then
echo "--install needs DEPLOY_DIR set to another directory: a script cannot replace itself while it runs" >&2
exit 2
fi
env_tag() {
[[ -f .env ]] && sed -n 's/^TAG=//p' .env | tail -n 1 || true
}
@@ -42,6 +54,59 @@ step_in_progress() {
esac
}
# A consistent online copy of the database through SQLite's backup API, taken by the running server (it owns /data);
# the newest $backups_kept copies stay.
backup_database() {
docker compose exec -T server node -e "
const { DatabaseSync, backup } = require('node:sqlite');
const { readdirSync, rmSync } = require('node:fs');
const prefix = 'queue.sqlite.before-deploy-';
const name = prefix + new Date().toISOString().replace(/[-:]/g, '').replace(/[.][0-9]+Z\$/, 'Z');
const db = new DatabaseSync('/data/queue.sqlite', { readOnly: true });
backup(db, '/data/' + name).then(() => {
db.close();
for (const old of readdirSync('/data').filter((file) => file.startsWith(prefix)).sort().reverse().slice($backups_kept)) rmSync('/data/' + old);
console.log('database copy: /data/' + name);
}, (error) => { console.error(error.message); process.exit(1); });
"
}
# The new compose.yaml and update.sh come from the image being deployed; the old compose file is kept, so a version
# that does not come up goes back together with the compose file it ran with.
install_files() {
cp -p compose.yaml compose.yaml.previous
install -m 600 "$script_dir/compose.yaml" compose.yaml
install -m 755 "$script_dir/update.sh" update.sh
if cmp -s compose.yaml.previous compose.yaml; then
echo "compose.yaml unchanged"
else
echo "compose.yaml changed:"
diff -u compose.yaml.previous compose.yaml || true
fi
}
restore_compose() {
if ((install == 1)) && [[ -f compose.yaml.previous ]]; then
echo "restoring the previous compose.yaml" >&2
cp -p compose.yaml.previous compose.yaml
fi
}
# Each CI build and each deploy leaves an image of this app on the VM (over a gigabyte each), and a full disk stops
# every app on it. Kept: the one running, the previous one (to go back) and the newest $images_kept; a failure here
# never fails the deploy.
prune_images() {
local running id count=0
running=$(docker inspect --format '{{.Image}}' "$(service_container server)" 2>/dev/null || true)
# `docker images` lists the newest first; an image with several tags appears once per tag.
for id in $(docker images --no-trunc --format '{{.ID}}' "$image_repo" | awk '!seen[$0]++'); do
count=$((count + 1))
[[ "$id" == "$running" || "$id" == "$previous_image" ]] && continue
((count <= images_kept)) && continue
docker image rm -f "$id" >/dev/null 2>&1 || true
done
}
wait_until_up() {
local deadline=$((SECONDS + healthy_timeout)) status
while ((SECONDS < deadline)); do
@@ -50,9 +115,10 @@ wait_until_up() {
sleep 3
done
[[ "$status" == healthy ]] || return 1
sleep 5 # the driver and the socket proxy have no healthcheck: they must still be running, and not restarting, after a few seconds
sleep 5 # services without a healthcheck must still be running, and not restarting, after a few seconds
local service
for service in driver docker-proxy executor model-claude model-codex function-runtime; do
for service in $(docker compose config --services); do
[[ "$service" == server ]] && continue
[[ "$(docker inspect --format '{{.State.Status}} {{.RestartCount}}' "$(service_container "$service")" 2>/dev/null)" == "running 0" ]] || return 1
done
# The executor has a healthcheck of its own; give it the rest of the time to pass it.
@@ -71,25 +137,42 @@ if ((force == 0)) && [[ -n "$(service_container server)" ]] && step_in_progress;
exit 3
fi
if [[ -n "$(service_container server)" ]] && backup_database; then
:
elif ((force == 0)); then
echo "could not copy the database (is the server running?); fix it or use --force" >&2
exit 5
else
echo "warning: no database copy (--force)" >&2
fi
previous_image=$(docker inspect --format '{{.Image}}' "$(service_container server)" 2>/dev/null || true)
echo "pulling $image_repo:$tag"
TAG="$tag" docker compose pull
TAG="$tag" docker compose up -d --remove-orphans
((install == 1)) && install_files
if wait_until_up; then
echo "pulling $image_repo:$tag"
if ! TAG="$tag" docker compose pull; then
echo "could not pull $image_repo:$tag; nothing was restarted" >&2
restore_compose
exit 1
fi
# A start that fails part-way (compose refuses a service, a container does not start) goes back like one that does
# not come up healthy.
if TAG="$tag" docker compose up -d --remove-orphans && wait_until_up; then
# Only the TAG line changes: .env also holds HA_TOKEN and other settings.
if [[ -n "$tag_arg" ]]; then
touch .env
{ grep -v '^TAG=' .env || true; printf 'TAG=%s\n' "$tag"; } > .env.new && chmod --reference=.env .env.new && mv .env.new .env
fi
revision=$(docker inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$(service_container server)")
prune_images
echo "ok: $image_repo:$tag revision ${revision:-unknown}"
exit 0
fi
echo "new version did not come up healthy" >&2
docker compose logs --tail 30 >&2 || true
restore_compose
if [[ -n "$previous_image" ]]; then
echo "rolling back to $previous_image" >&2
docker tag "$previous_image" "$image_repo:rollback"
+48 -13
View File
@@ -5,7 +5,7 @@ Stan na 2026-09-19: na VM103 działa łańcuch prawdziwej aktualizacji przez sid
| | |
|---|---|
| Adres | https://upgrade.mariuszpacek.pl (traefik, `web-secured`, certyfikat `http`, wildcard DNS na 192.168.0.139) |
| Dostęp | tylko LAN: middleware `ipwhitelist` `192.168.0.1/24,10.0.0.0/16`; **appka nie ma logowania** |
| Dostęp | z LAN i z internetu, **tylko po zalogowaniu Keycloakiem** (realm `home`, usługa `auth` = oauth2-proxy jako forwardAuth Traefika; sekcja „Logowanie”) |
| Manifest | `/opt/homelab/compose/homelab-upgrade/compose.yaml` (kopia `deploy/compose.yaml`) |
| Obraz | `git.mariuszpacek.pl/mariusz.pacek/homelab-upgrade:<tag>` z rejestru Gitei, budowany przez Gitea Actions (etykieta `org.opencontainers.image.revision` = sha commita) |
| Dashboard | kafelek „Upgrade” w grupie Infra na https://dash.mariuszpacek.pl z etykiet `homepage.*` na `server` (homepage czyta je przez swoje połączenie z Dockerem; jego pliki nie są zmieniane) |
@@ -21,20 +21,55 @@ Port 8080 na hoście VM jest zajęty przez inną usługę, dlatego appka nie pub
## Aktualizacja
Obieg: `git push` → Gitea Actions (`.gitea/workflows/ci.yaml`): `npm run check`, a na `master` i `foundations` także build i push obrazu do rejestru Gitei → na VM103 jedna komenda:
Obieg: `git push` → Gitea Actions (`.gitea/workflows/ci.yaml`): `npm run check`, a na `master`, `foundations` i tagach wydań `v…` także build i push obrazu do rejestru Gitei → wdrożenie **tylko ręcznie**, przyciskiem albo przez API. Arcane nie zarządza tym projektem.
Wydanie: tag `v<rok>.<miesiąc>.<numer>` (np. `v2026.10.1`) na commicie z `master`, najlepiej jako release w Gitei; CI buduje obraz `:v2026.10.1` (i `:<sha>`). Stopka pokazuje wtedy `v2026.10.1 · build N`.
Wdrożenie: Gitea → repo → Actions → **deploy** → *Run workflow* (gałąź `master`), pole `version`: tag wydania, pełny sha albo `latest` (ostatni `master`); `force` tylko świadomie. To samo przez API/MCP: `dispatch_workflow` z `workflow_id: deploy.yaml`, `ref: master`, `inputs: { version, force }`; wynik i log w runie.
Workflow (`.gitea/workflows/deploy.yaml`) działa tylko z definicji na `master`: sprawdza format wersji, pobiera obraz i uruchamia **skrypt z tego obrazu** (`/app/deploy/update.sh --install`) na katalogu `/opt/homelab/compose/homelab-upgrade` przez socket Dockera. `update.sh`:
1. odmawia (kod 3), gdy instancja silnika jest w stanie `running` (run na bramce lub w kolejce restart przeżywa, przerwany węzeł wznawia się w UI przyciskiem „Retry the step”);
2. robi spójną kopię bazy przez działający serwer (`/data/queue.sqlite.before-deploy-<czas>`, zostaje 5 najnowszych); bez kopii odmawia (kod 5);
3. z `--install` wgrywa `compose.yaml` i `update.sh` z obrazu (poprzedni compose zostaje jako `compose.yaml.previous`, diff w logu);
4. `docker compose pull` (gdy obrazu nie ma: przywraca compose i kończy bez restartu) i `up -d`;
5. czeka do 90 s na `healthy` serwera i executora oraz na stabilne pozostałe usługi; gdy nowa wersja nie wstaje, wraca do poprzedniego obrazu (lokalny tag `:rollback`) i poprzedniego `compose.yaml`, nie zmienia `.env` i kończy się kodem 1.
`--force` pomija krok 1 i pozwala na brak kopii z kroku 2. Wycofanie to ponowne wdrożenie starszej wersji tym samym przyciskiem. Stare obrazy zostają na VM.
Awaryjnie, gdy Gitea lub runner nie działa, ten sam skrypt ręcznie na VM103:
```bash
cd /opt/homelab/compose/homelab-upgrade
./update.sh # tag z .env (domyślnie latest)
./update.sh foundations # inny tag (gałąź albo pełny sha) i zapamiętanie go w .env
./update.sh --force # pomija blokadę „trwa krok / węzeł runu”
./update.sh v2026.10.1 # inna wersja (tag, gałąź albo pełny sha) i zapamiętanie jej w .env
./update.sh --force # pomija blokadę „trwa krok / węzeł runu” i brak kopii bazy
```
`update.sh` (kopia `deploy/update.sh`): odmawia restartu, gdy driver ma krok starego łańcucha w stanie `running` albo instancja nowego silnika jest w stanie `running` (kod 3; run czekający na bramce lub w kolejce restart przeżywa, a przerwany węzeł wznawia się w UI przyciskiem „Retry the step”); robi `docker compose pull` i `up -d`; czeka do 90 s na `healthy` serwera i sprawdza, że driver i `docker-proxy` działają i się nie restartowały. Gdy nowa wersja nie wstaje, wraca do poprzedniego obrazu (lokalny tag `:rollback`), nie zmienia `.env` i kończy się kodem 1. Stare obrazy zostają na VM, więc wycofanie do dowolnego commita to `./update.sh <sha>`.
Ręczny `update.sh` nie wgrywa `compose.yaml`; po zmianie `deploy/compose.yaml` wdrażaj przyciskiem albo skopiuj plik.
CI niczego nie wdraża: deploy jest świadomym krokiem na VM (na razie bez timera, bo restart w trakcie kroku byłby groźny dopiero po dodaniu prawdziwych handlerów).
Przy zmianie schematu bazy nowa wersja migruje bazę przy starcie (migracje są transakcyjne; nowszej bazy niż kod odmawia otwarcia). Wycofanie do starszego obrazu po zmianie schematu nie zadziała, bo starszy kod odmówi otwarcia nowszej bazy: wtedy przywraca się kopię `queue.sqlite.before-deploy-*` z kroku 2.
Przy zmianie schematu bazy nowa wersja migruje bazę przy starcie (migracje są transakcyjne; nowszej bazy niż kod odmawia otwarcia). Wycofanie do starszego obrazu po zmianie schematu nie zadziała, bo starszy kod odmówi otwarcia nowszej bazy: przed takim wdrożeniem zrób kopię `queue.sqlite*`.
### Logowanie (Keycloak)
Usługa `auth` (oauth2-proxy, `deploy/compose.yaml`) stoi przed aplikacją: Traefik pyta ją o każde zapytanie (forwardAuth); bez sesji przeglądarka idzie do `https://login.mariuszpacek.pl/realms/home`, po powrocie wpuszczane są tylko adresy z `AUTH_ALLOWED_EMAILS`. Sesja trwa 7 dni (odświeżana co godzinę), wylogowanie: `https://upgrade.mariuszpacek.pl/oauth2/sign_out`. Publiczne zostają tylko `manifest.webmanifest` i `icon.svg`.
Jednorazowo w Keycloaku (realm `home`):
1. Clients → Create client: typ OpenID Connect, Client ID `homelab-upgrade`; Client authentication **on**, Standard flow **on**, Direct access grants off.
2. Valid redirect URIs `https://upgrade.mariuszpacek.pl/oauth2/callback`, Web origins `https://upgrade.mariuszpacek.pl`.
3. Client scopes → `homelab-upgrade-dedicated` → Add mapper → By configuration → **Audience**: Included Client Audience `homelab-upgrade`, Add to access token on (oauth2-proxy sprawdza audience).
4. Credentials → skopiuj Client secret.
W `/opt/homelab/compose/homelab-upgrade/.env` na VM103 (root, 600):
```bash
AUTH_CLIENT_SECRET=<client secret z Keycloaka>
AUTH_COOKIE_SECRET=<openssl rand -base64 32 | tr -- '+/' '-_'>
AUTH_ALLOWED_EMAILS=<adres e-mail konta w Keycloaku>
```
Bez tych trzech wartości `docker compose` odmawia (wdrożenie kończy się bez restartu, `update.sh` przywraca poprzedni `compose.yaml`).
### Jednorazowa konfiguracja
@@ -43,7 +78,7 @@ Przy zmianie schematu bazy nowa wersja migruje bazę przy starcie (migracje są
3. **Sekret repo** `REGISTRY_TOKEN` (repo → Settings → Actions → Secrets): token `write:package`.
4. **Runner** (`deploy/runner/`, na VM103 w `/opt/homelab/compose/gitea-runner/`; dane w `/srv/appdata/gitea-runner/data`). Token rejestracyjny: repo → Settings → Actions → Runners → Create new runner (token na poziomie repo zamiast całej instancji). Runner ma `docker.sock`, czyli praktycznie root na VM: rejestruj go tylko dla tego repo.
5. **`docker login`** na VM103: pomijamy, bo pull jest anonimowy. Gdyby pakiet stał się prywatny: `docker login git.mariuszpacek.pl -u mariusz.pacek` tokenem `read:package`.
6. **Przełączenie z lokalnego obrazu**: skopiuj `deploy/compose.yaml` i `deploy/update.sh` do `/opt/homelab/compose/homelab-upgrade/`, po pierwszym zielonym buildzie na `foundations` uruchom `./update.sh foundations`. Kopie trzeba odświeżyć ręcznie, gdy te dwa pliki się zmienią.
6. **Przełączenie z lokalnego obrazu**: skopiuj `deploy/compose.yaml` i `deploy/update.sh` do `/opt/homelab/compose/homelab-upgrade/`, po pierwszym zielonym buildzie uruchom `./update.sh <sha>`. Później oba pliki wgrywa workflow deploy.
Ograniczenia, które warto znać: job dostaje repo z `http://192.168.0.139:13080` (port Gitei na VM), bo whitelist traefika nie obejmuje sieci jobów; `docker push` idzie z demona VM, więc źródłowy adres to 192.168.0.139 z whitelisty Gitei.
@@ -63,7 +98,7 @@ Notatki wydań nie są ustawieniem: są w **skillu** projektu (Apps → aplikacj
Jedno ustawienie w Apps → aplikacja, domyślnie wyłączone:
- **Post-upgrade check executor**: `sonnet` / `opus` = po udanym `verify` model sprawdza, czy aktualizacja wymaga czynności poinstalacyjnych (czyta ocenę notatek, skill i stan usługi; co robić po aktualizacji, można dopisać w skillu). Model wykona tylko to, co dopuszcza `config/agent-exec.json` (dokładne komendy, bez wildcardów); reszta trafia do właściciela jako STOP `migration_required`. Dziś lista ma tylko komendy Nextclouda: kontrola odbiorcza ze skilla (tylko odczyt: `php occ status` z `--output=json` i bez, `php occ config:app:get core lastcron`, `php occ app:list`, `pidof crond` w usłudze `cron`) oraz to, co po upgrade brakuje w bazie (`php occ db:add-missing-indices`, `db:add-missing-columns`, `db:add-missing-primary-keys`; użytkownik `www-data`, usługa `app`). Model widzi listę przez `hl exec-rules` i nie zgaduje. Dla pozostałych aplikacji ten krok tylko wypisze, co zrobić ręcznie.
- **Post-upgrade check executor**: `haiku` (domyślny) / `sonnet` / `opus` = po udanym `verify` model sprawdza, czy aktualizacja wymaga czynności poinstalacyjnych (czyta ocenę notatek, skill i stan usługi; co robić po aktualizacji, można dopisać w skillu). Model wykona tylko to, co dopuszcza `config/agent-exec.json` (dokładne komendy, bez wildcardów); reszta trafia do właściciela jako STOP `migration_required`. Dziś lista ma tylko komendy Nextclouda: kontrola odbiorcza ze skilla (tylko odczyt: `php occ status` z `--output=json` i bez, `php occ config:app:get core lastcron`, `php occ app:list`, `pidof crond` w usłudze `cron`) oraz to, co po upgrade brakuje w bazie (`php occ db:add-missing-indices`, `db:add-missing-columns`, `db:add-missing-primary-keys`; użytkownik `www-data`, usługa `app`). Model widzi listę przez `hl exec-rules` i nie zgaduje. Dla pozostałych aplikacji ten krok tylko wypisze, co zrobić ręcznie.
Schemat bazy przechodzi na v10 (v8 dodał `post_upgrade_executor`, v9 zdejmuje niepotrzebną `release_notes_source`, v10 dodaje niepodłączony jeszcze katalog i instancje nowego silnika workflow); starszy obraz jej nie otworzy, więc przed wdrożeniem kopia `queue.sqlite*`.
@@ -156,13 +191,13 @@ Ręczny powrót do dowolnej kopii: STOP w kolejce → rollback, albo z kroku z `
## Arcane i repo compose
Repozytorium `ssh://git@git.mariuszpacek.pl:2244/mariusz.pacek/homelab-compose.git` jest prywatne, na `master`, zawiera 19 katalogów projektów z samym `compose.yaml`, archiwum `to_delete/dockge` oraz `.gitignore`; pliki `.env` i sekrety nie są śledzone. Główny checkout VM103: `/srv/appdata/homelab-upgrade/git/repo`. Dziesięć compose używających `env_file` w formacie `raw` skonwertowano przez `tools/compose-env-compat.py`; pełny wynik `docker compose config` porównano ściśle, wartości lokalne pozostały bez zmian.
Repozytorium `ssh://git@git.mariuszpacek.pl:2244/mariusz.pacek/homelab-compose.git` jest prywatne, na `master`, zawiera 20 katalogów projektów z samym `compose.yaml`, archiwum `to_delete/dockge` oraz `.gitignore`; pliki `.env` i sekrety nie są śledzone. Główny checkout VM103: `/srv/appdata/homelab-upgrade/git/repo`. Dziesięć compose używających `env_file` w formacie `raw` skonwertowano przez `tools/compose-env-compat.py`; pełny wynik `docker compose config` porównano ściśle, wartości lokalne pozostały bez zmian.
Arcane v2.13.1 działa pod `https://arcane.mariuszpacek.pl`. Projekty używają Git Sync w trybie **Deploy** (w UI: Pull), ścieżka `<projekt>/compose.yaml`, `syncDirectory=false`, interwał 5 minut. Autosync jest włączony dla wszystkich 18 projektów; stosy Arcane i `homelab-upgrade` są wyłączone, a `upgrade-lab` działa bezpośrednio. Wszystkie manifesty projektów zawierają etykietę `com.getarcaneapp.arcane.updater: "false"`. Arcane sam wykonuje redeploy projektu po zmianie compose, więc executor w trybie git nie uruchamia `pull/up`.
Arcane v2.13.1 działa pod `https://arcane.mariuszpacek.pl`. Projekty używają Git Sync w trybie **Deploy** (w UI: Pull), ścieżka `<projekt>/compose.yaml`, `syncDirectory=false`, interwał 5 minut. Autosync jest włączony dla wszystkich 19 projektów; stosy Arcane i `homelab-upgrade` są wyłączone, a `upgrade-lab` działa bezpośrednio. Wszystkie manifesty projektów zawierają etykietę `com.getarcaneapp.arcane.updater: "false"`. Arcane sam wykonuje redeploy projektu po zmianie compose, więc executor w trybie git nie uruchamia `pull/up`.
Początkowe synchronizacje 18 projektów zachowały SHA plików env i nie zmieniły startów kontenerów; Arcane może utworzyć pusty `.env`. Źródła nie wymagały restartu podczas importu. Test Jackett przez API executora potwierdził ścieżkę komentarz → commit `47c2b7f` → push → udany sync Arcane i HTTP 301; rollback executora wykonał `git revert`, przywrócił pierwotny SHA i zakończył się powodzeniem.
Początkowe synchronizacje 18 projektów (i Trilium, dołączonego 2026-09-30) zachowały SHA plików env i nie zmieniły startów kontenerów; Arcane może utworzyć pusty `.env`. Źródła nie wymagały restartu podczas importu. Test Jackett przez API executora potwierdził ścieżkę komentarz → commit `47c2b7f` → push → udany sync Arcane i HTTP 301; rollback executora wykonał `git revert`, przywrócił pierwotny SHA i zakończył się powodzeniem.
`config/workflow-schedules.json` rozkłada starty w strefie `Europe/Warsaw` (czas polski): poniedziałek Bazarr 19:00, Jackett 21:00; wtorek Radarr 19:00, Sonarr 21:00; środa Deluge 19:00, Ombi 21:00; czwartek Homepage 19:00, OrcaSlicer 21:00; piątek Gitea 18:00, baza 19:00, Keycloak 20:00, baza 22:00; sobota Home Assistant 18:00, Nextcloud 20:00, baza 22:00, Redis 23:00; niedziela Grafana 18:00, Jellyfin 20:00. InfluxDB i Dockge są wyłączone.
`config/workflow-schedules.json` rozkłada starty w strefie `Europe/Warsaw` (czas polski): poniedziałek Bazarr 19:00, Jackett 21:00; wtorek Radarr 19:00, Sonarr 21:00; środa Deluge 19:00, Ombi 21:00; czwartek Homepage 19:00, OrcaSlicer 21:00; piątek Gitea 18:00, baza 19:00, Keycloak 20:00, baza 22:00; sobota Home Assistant 18:00, Nextcloud 20:00, baza 22:00, Redis 23:00; niedziela Trilium 16:00, Grafana 18:00, Jellyfin 20:00. InfluxDB i Dockge są wyłączone.
Konfiguracja prywatna `/srv/appdata/homelab-upgrade/git-config/deployments.json` jest montowana jako `/git-config` tylko do odczytu, a `GIT_DEPLOYMENTS_FILE=/git-config/deployments.json`. Schemat v1: `repoPath=/git/repo`, `branch=master`, `sshKeyPath=/git-ssh/id_ed25519`, `knownHostsPath=/git-ssh/known_hosts`; projekty mają tryb `direct` albo `git`, `webhookUrl` i opcjonalny `redeployWebhookUrl`. Executor ma checkout do zapisu, katalog `git-ssh` tylko do odczytu oraz sieci `executor` i `default`; git i OpenSSH są częścią obrazu. Klucz executora z prawem zapisu znajduje się w `/srv/appdata/homelab-upgrade/git-ssh/id_ed25519`, klucz Arcane tylko do odczytu w `/srv/appdata/arcane/git-ssh/id_ed25519`. `known_hosts` przypina trzy publiczne algorytmy Gitei porównane z plikiem `.pub` kontenera. Nie umieszczaj tokenów ani webhook URL zawierających sekrety w dokumentacji, repo ani logach.
+1 -1
View File
@@ -95,7 +95,7 @@ Projekt compose może mieć kilka usług z własnymi obrazami (grafana + influxd
## Model naprawia (decyzja właściciela, 2026-09-19 wieczór)
„Skrypt robi upgrade, dopóki wszystko przechodzi; kiedy STOP albo kontener nie wstał, wchodzi model, patrzy i naprawia.”
- Krok `repair`: gdy `verify` nie przeszedł, a aplikacja ma „Migration executor” (sonnet/opus), `verify` sam kolejkuje `repair` (jedna próba; drugi nieudany `verify` = STOP dla właściciela). Na dowolnym STOP właściciel może wybrać „Repair (model)”.
- Krok `repair`: gdy `verify` nie przeszedł, a aplikacja ma „Migration executor” (haiku/sonnet/opus; bez wyboru haiku), `verify` sam kolejkuje `repair` (jedna próba; drugi nieudany `verify` = STOP dla właściciela). Na dowolnym STOP właściciel może wybrać „Repair (model)”.
- Kontener `model`: niezmodyfikowany `claude` CLI (przypięta wersja w obrazie) zalogowany kontem właściciela (`CLAUDE_CONFIG_DIR=/claude`). Bez `docker.sock`, bez plików projektów i bazy.
- Model działa tylko przez narzędzie `hl` w sesji executora otwartej na jeden projekt: `ps`, `logs`, `compose` (sekrety jako `<secret:KLUCZ>`), `write`, `pull`, `up`, `restart`, `exec-rules` (lista komend, które `exec` w ogóle przyjmie: model patrzy, zamiast zgadywać), `exec` w kontenerach tego projektu. Egzekwowane w kodzie (`domain/agent-guard.ts`): zapis compose sprawdzany listą zakazów z „Granic” względem manifestu z otwarcia sesji; wyjścia maskowane wartościami sekretów projektu; hook `PreToolUse` przepuszcza w Bash tylko `hl`, a Read/Edit/Write tylko w katalogu sesji; trasy drivera w executorze i `/run` modelu odpowiadają tylko adresowi drivera. Każde wywołanie jest w logu sesji i w wyniku kroku.
- Wyjątek za zgodą właściciela (2026-09-23): na bramce po nieudanej naprawie przycisk „Repair with full access” uruchamia jedną ponowną naprawę w sesji `full`. W niej, tylko w tym projekcie, `exec` przyjmuje dowolną komendę i użytkownika, `run` uruchamia jednorazowy kontener usługi z jej wolumenami (np. `chown` katalogu danych przy crash-loopie), a `write` sprawdza tylko poprawność compose, bez listy zakazów. Maskowanie sekretów, log sesji, kopia i rollback działają dalej. To jedna próba na run: po niej zostaje rollback albo skip.
+13
View File
@@ -169,3 +169,16 @@ Decyzje: sidebar + karty, ciemny motyw domyślny z przełącznikiem; wykresy w E
- **G5** strona runu: Gantt kroków (klik zaznacza węzeł), historia w oknie.
- **G6** Models: koszt dziennie wg modelu, udział modeli, wyniki wg rodzaju pytania; edycja profilu w panelu bocznym.
- **G7** porządki stylów, dokumentacja. Przegląd na żywo w przeglądarce jeszcze nie zrobiony (brak przeglądarki na hoście).
### Faza H — wygoda na co dzień (zatwierdzona przez właściciela 2026-09-30)
Decyzje: logowanie Keycloak dla wszystkich (oauth2-proxy jako forwardAuth Traefika) i dostęp spoza LAN; cisza nocna 01:00–07:30 (wieczorne awarie pushują od razu), w ciszy nic nie dzwoni, bramki i błędy trafiają do porannego raportu o 7:30; godziny zmienia się w UI. Etapy wdrażane przyciskiem `deploy` bez osobnej zgody.
- **H1** stan „tu i teraz”: driver co minutę zbiera `docker ps` kontenerów compose (przez proxy tylko do odczytu) i rekord vzdump, zapisuje migawkę w `settings`; Overview pokazuje zdrowie aplikacji i wiek kopii VM.
- **H2** historia wersji aplikacji (z runów: z czego na co, którym runem, naprawy i rollbacki) i oznaczenie aplikacji, które często wymagają uwagi.
- **H3** ustawienia powiadomień w UI, cisza nocna i poranny raport (push + karta „Ostatnia noc” na Overview).
- **H4** decyzja z telefonu: strona runu z bramką na górze i przyciskami przyklejonymi do dołu ekranu; push otwiera ją bezpośrednio.
- **H5** czasy względne, szybkie wyszukiwanie `Ctrl+K`, dry run wszystkich aplikacji z dostępną aktualizacją.
- **H6** logowanie Keycloak i zdjęcie ograniczenia do LAN.
Status 2026-09-30: H1–H6 wdrożone na VM103 (ostatnio `6cf5848`). Przy pierwszym wdrożeniu H6 compose odrzucił inline config w usłudze `read_only`, a `update.sh` przerwał się bez wycofania (ok. 12 min awarii, wycofane przyciskiem do `fc68d30`); poprawione: `auth` bez `read_only`, nieudane `up -d` wraca do poprzedniego obrazu i compose. Tego samego dnia dysk VM103 zapełnił się obrazami aplikacji i cache buildów CI: `update.sh` zostawia teraz uruchomiony, poprzedni i 3 najnowsze obrazy, CI czyści cache starszy niż 2 dni.
+35 -3
View File
@@ -89,6 +89,38 @@ describe('component-operation-executor: verify', () => {
expect(calls).toBe(4);
});
it('retries while compose still has the old container next to the new one, then passes', async () => {
const clock = new FakeClock(NOW);
const executor = new FakeExecutorClient();
let calls = 0;
executor.on('inspect', () => {
calls += 1;
return calls <= 2
? { outcome: 'refused', reason: 'multiple_containers', detail: 'service "jackett" of "jackett" has 2 containers' }
: { outcome: 'done', result: inspectResult({ state: 'running', health: 'healthy', image: PINNED }) };
});
const app = createComponentOperationExecutor({ clock, executor, backupVmid: 103 });
const outcome = await settle(app.execute('verify', 1, verifyInput()), clock);
expect(outcome).toMatchObject({ event: 'healthy', result: { healthy: true, attempts: 3 } });
});
it('gives up on a service that keeps two containers after the usual attempts, and at once on any other refusal', async () => {
const clock = new FakeClock(NOW);
const executor = new FakeExecutorClient();
executor.on('inspect', { outcome: 'refused', reason: 'multiple_containers', detail: 'service "jackett" of "jackett" has 2 containers' });
const app = createComponentOperationExecutor({ clock, executor, backupVmid: 103 });
const outcome = await settle(app.execute('verify', 1, verifyInput()), clock);
expect(outcome).toMatchObject({ event: 'unhealthy', result: { healthy: false, reason: 'inspect_unavailable', attempts: 24 } });
const other = new FakeExecutorClient();
other.on('inspect', { outcome: 'refused', reason: 'unknown_service', detail: 'no service' });
const once = await settle(createComponentOperationExecutor({ clock, executor: other, backupVmid: 103 }).execute('verify', 1, verifyInput()), clock);
expect(once).toMatchObject({ event: 'unhealthy', result: { healthy: false, reason: 'inspect_unavailable', detail: 'no service' } });
expect(other.calls).toHaveLength(1);
});
it('gives up after 24 unhealthy attempts, 5s apart, once the container is no longer starting', async () => {
const clock = new FakeClock(NOW);
const executor = new FakeExecutorClient();
@@ -249,17 +281,17 @@ describe('component operation executor: migration_plan', () => {
.on('sessionCompose', { outcome: 'done', result: { text: 'services:\n influxdb:\n image: influxdb:1.8\n' } })
.on('closeSession', { outcome: 'done', result: { project: 'grafana', service: 'influxdb', manifest_changed: false, manifest_sha256: 'a'.repeat(64), log: [{ at: NOW, op: 'compose', ok: true, detail: 'read' }] } });
it('has opus look at the app in an observe session and hands over the plan with its digest', async () => {
it('has haiku look at the app in an observe session and hands over the plan with its digest', async () => {
const model = new FakeModelExecutor().reply({ output: plan, costUsd: 0.4 });
const executor = observer();
const done = await createComponentOperationExecutor({ backupVmid: 103, clock: new FakeClock(NOW), skills, model, executor }).execute('migration_plan', 1, context);
expect(done).toMatchObject({
event: 'migration_needed',
result: { plan: { needed: true, inputs: [{ name: 'ADMIN_PASSWORD' }] }, model: 'opus', costUsd: 0.4, actions: ['compose: read'], composeBefore: 'services:\n influxdb:\n image: influxdb:1.8\n' },
result: { plan: { needed: true, inputs: [{ name: 'ADMIN_PASSWORD' }] }, model: 'haiku', costUsd: 0.4, actions: ['compose: read'], composeBefore: 'services:\n influxdb:\n image: influxdb:1.8\n' },
});
expect((done.result as { planDigest: string }).planDigest).toMatch(/^[0-9a-f]{64}$/);
expect(model.requests[0]).toMatchObject({ executor: 'opus', profile: 'observe', sessionToken: 'observe-token' });
expect(model.requests[0]).toMatchObject({ executor: 'haiku', profile: 'observe', sessionToken: 'observe-token' });
expect(model.requests[0]?.prompt).toContain('Before update: create an admin user');
expect(executor.calls.map((call) => call.op)).toEqual(['openSession', 'sessionCompose', 'closeSession']);
expect(executor.calls[0]?.request).toEqual({ project: 'grafana', service: 'influxdb', mode: 'observe' });
+10 -5
View File
@@ -4,7 +4,7 @@ import { resolveImageRef } from './ops/image-resolve.ts';
import { backupVerdict, HostBackupRecord } from '../domain/backup-check.ts';
import { isWithinAuto } from '../domain/version.ts';
import { describeError } from '../domain/mask.ts';
import { nextVerifyAction, VERIFY_INTERVAL_MS, verifyChecks } from '../domain/verify.ts';
import { nextVerifyAction, VERIFY_ATTEMPTS, VERIFY_INTERVAL_MS, verifyChecks } from '../domain/verify.ts';
import type { VerifyObservation } from '../domain/verify.ts';
import type { Clock, ExecutorClient, ModelExecutor, PrivilegedActionRun, RegistryClient, SkillStore } from '../domain/ports.ts';
import type { SessionLogEntry } from '../domain/executor-api.ts';
@@ -15,7 +15,7 @@ import { buildMigrationPlanPrompt, MIGRATION_PLAN_SCHEMA, MigrationPlan, migrati
import { ModelError } from '../domain/ports.ts';
import type { GroupMember } from './ops/group.ts';
import type { ComponentOperationId } from '../domain/workflow-component-operation.ts';
import { toJson } from '../domain/steps.ts';
import { modelOr, toJson } from '../domain/steps.ts';
import type { JsonValue } from '../domain/steps.ts';
import { validateImage } from '../domain/compose-edit.ts';
import { splitImage } from '../domain/image-ref.ts';
@@ -229,7 +229,12 @@ export function createComponentOperationExecutor(deps: ComponentOperationExecuto
for (let attempt = 1; ; ) {
const inspected = await executor.inspect({ project, service }, signal);
if (inspected.outcome !== 'done') {
return { event: 'unhealthy', result: { healthy: false, reason: 'inspect_unavailable', detail: inspected.detail } };
// While compose (or Arcane, for a Git project) recreates the service, the old container still sits next to the new one.
const transient = inspected.outcome === 'refused' && inspected.reason === 'multiple_containers' && attempt < VERIFY_ATTEMPTS;
if (!transient) return { event: 'unhealthy', result: { healthy: false, reason: 'inspect_unavailable', detail: inspected.detail, attempts: attempt } };
attempt += 1;
await deps.clock.sleep(VERIFY_INTERVAL_MS, signal);
continue;
}
const { container } = inspected.result;
const http = httpUrl === null || expectStatus === null ? null : await probe(httpUrl, expectStatus);
@@ -280,7 +285,7 @@ async function releaseNotes(deps: ComponentOperationExecutorDeps, record: Record
versionChange: firstString(record['change'], selection['change'], 'unknown'),
auto: firstString(record['auto'], 'none'),
promptHint: typeof record['promptHint'] === 'string' ? record['promptHint'] : null,
model: record['releaseNotesExecutor'] === 'opus' ? 'opus' : 'sonnet',
model: modelOr(record['releaseNotesExecutor']),
group: async () => (groupMembers === undefined ? [] : [...await groupMembers(app, signal)]),
}, signal);
@@ -316,7 +321,7 @@ async function migrationPlan(deps: ComponentOperationExecutorDeps, record: Recor
const project = asString(record['project']);
const service = asString(record['service']);
const app = firstString(record['component'], record['app'], project);
const executorName = record['migrateExecutor'] === 'sonnet' ? 'sonnet' : 'opus';
const executorName = modelOr(record['migrateExecutor']);
const nextSteps = asRecord(notes['report'])['nextSteps'];
const prompt = buildMigrationPlanPrompt({
app, project, service,
+43
View File
@@ -0,0 +1,43 @@
import { describe, expect, it } from 'vitest';
import { isQuiet, sendDigestIfDue } from './digest.ts';
import { createSqliteNotificationSettingsStore } from '../infra/sqlite/notification-settings-store.ts';
import { openDatabase } from '../infra/sqlite/open.ts';
import { FakeClock } from '../testing/fake-clock.ts';
import { RecordingNotifier } from '../testing/fakes.ts';
// 05:30 UTC is 07:30 in Warsaw (summer time).
function setup(start: string) {
const clock = new FakeClock(start);
const settings = createSqliteNotificationSettingsStore(openDatabase(':memory:'));
const notifier = new RecordingNotifier();
const failed = { app: 'bazarr', workflowId: 'update-bazarr', status: 'failed', mode: 'real' as const, current: '1', candidate: '2', running: '1', outcome: 'Verify failed', step: null, updatedAt: '2026-09-30T01:00:00.000Z' };
return { clock, settings, notifier, deps: { clock, timeZone: 'Europe/Warsaw', settings, notifier, runs: () => [failed], down: () => ['grafana'] } };
}
describe('sendDigestIfDue', () => {
it('sends the report once a day from 07:30 on, with what happened since the last one', async () => {
const { clock, notifier, settings, deps } = setup('2026-09-30T05:29:00.000Z');
expect(await sendDigestIfDue(deps)).toBe(false);
await clock.advance(60_000);
expect(await sendDigestIfDue(deps)).toBe(true);
expect(await sendDigestIfDue(deps)).toBe(false);
expect(notifier.sent).toEqual([
{ runId: 'digest-2026-09-30', title: 'Night report: 2 need you', detail: 'Failed: bazarr (Verify failed)\nDown now: grafana', link: '#/overview' },
]);
expect(settings.lastDigest()).toEqual({ day: '2026-09-30', at: '2026-09-30T05:30:00.000Z' });
});
it('tries again next time when the push fails', async () => {
const { notifier, settings, deps } = setup('2026-09-30T06:00:00.000Z');
notifier.failWith = new Error('HA down');
await expect(sendDigestIfDue(deps)).rejects.toThrow('HA down');
expect(settings.lastDigest()).toBeNull();
});
});
describe('isQuiet', () => {
it('follows the owner’s quiet hours in the schedules’ time zone', () => {
expect(isQuiet(setup('2026-09-29T23:30:00.000Z').deps)).toBe(true); // 01:30 in Warsaw
expect(isQuiet(setup('2026-09-29T20:30:00.000Z').deps)).toBe(false); // 22:30 in Warsaw
});
});
+51
View File
@@ -0,0 +1,51 @@
import { zonedMinute } from '../domain/cron.ts';
import { digestDue, inQuietHours, nightDigest } from '../domain/notification-settings.ts';
import type { DigestRun } from '../domain/notification-settings.ts';
import type { Clock, Notifier, NotificationSettingsStore } from '../domain/ports.ts';
export interface DigestDeps {
readonly clock: Clock;
/** The schedules' time zone: quiet hours and the report time are wall-clock there. */
readonly timeZone: string;
readonly settings: NotificationSettingsStore;
readonly runs: () => readonly DigestRun[];
/** Enabled apps whose container is down right now (the driver's latest look). */
readonly down: () => readonly string[];
readonly notifier: Notifier;
}
const DAY_MS = 86_400_000;
/** True while the owner's quiet hours hold pushes back. */
export function isQuiet(deps: Pick<DigestDeps, 'clock' | 'timeZone' | 'settings'>): boolean {
return inQuietHours(deps.settings.get(), zonedMinute(deps.clock.now(), deps.timeZone));
}
/** Sends the morning report when it is due; true when it went out. A failed push is not recorded, so it is tried again. */
export async function sendDigestIfDue(deps: DigestDeps): Promise<boolean> {
const now = deps.clock.now();
const at = zonedMinute(now, deps.timeZone);
const last = deps.settings.lastDigest();
if (!digestDue(deps.settings.get(), at, last?.day ?? null)) return false;
const day = at.key.slice(0, 10);
const digest = nightDigest({ runs: deps.runs(), since: last?.at ?? new Date(now.getTime() - DAY_MS).toISOString(), down: deps.down() });
await deps.notifier.notify({ runId: `digest-${day}`, title: digest.title, detail: digest.detail, link: '#/overview' });
deps.settings.recordDigest(day, now.toISOString());
return true;
}
/** Checks once per `intervalMs` whether the report is due, until `signal` aborts. */
export async function runDigestLoop(deps: DigestDeps & { readonly intervalMs: number; readonly signal: AbortSignal; readonly onError: (cause: unknown) => void }): Promise<void> {
while (!deps.signal.aborted) {
try {
await sendDigestIfDue(deps);
} catch (cause) {
deps.onError(cause);
}
try {
await deps.clock.sleep(deps.intervalMs, deps.signal);
} catch {
return; // stopped
}
}
}
+12
View File
@@ -716,6 +716,18 @@ services:
expect(JSON.stringify(result)).not.toContain('old-db-secret');
});
it('without a new compose file pins the image in the current one, secrets untouched', async () => {
const { context, request } = await prepared();
const result = await context.executor.migrate({ ...request, compose_yaml: null, env: {} });
const written = WITH_SECRET.replace(`image: ${OLD}`, `image: ${NEW}`);
expect(written).not.toBe(WITH_SECRET);
expect(result).toMatchObject({ outcome: 'done', result: { state: 'applied', image: NEW, applied_manifest_sha256: sha(written), env_keys: [] } });
expect(readFileSync(context.manifest, 'utf8')).toBe(written);
expect(readFileSync(join(context.dir, '.env'), 'utf8')).toBe(ENV);
});
it('refuses an unknown secret, an unfilled input, a changed manifest, and leaves everything as it was', async () => {
const { context, request, envCandidate } = await prepared();
const cases = [
+4 -1
View File
@@ -588,7 +588,10 @@ export function createComposeExecutor(deps: ComposeExecutorDeps): ComposeExecuto
return refused('manifest_changed', `compose.yaml of "${project}" is ${short(current)}, not ${short(request.expect_manifest_sha256)} as planned`);
}
// The plan was written against the masked file (hl compose): its <secret:KEY> are this file's values.
const unmasked = unmaskComposeSecrets(request.compose_yaml, maskComposeSecrets(manifest.value.toString('utf8')).originals);
// Without a new file the migration keeps the current one (read just now, so it matches the backup) and only pins the image.
const unmasked = request.compose_yaml === null
? { ok: true as const, text: manifest.value.toString('utf8') }
: unmaskComposeSecrets(request.compose_yaml, maskComposeSecrets(manifest.value.toString('utf8')).originals);
if (!unmasked.ok) return refused('invalid_manifest', `the migration's compose.yaml has an ${unmasked.reason}`);
if (/<input:[A-Za-z0-9_]+>/.test(unmasked.text)) return refused('invalid_manifest', "the migration's compose.yaml still has an <input:NAME> the owner did not fill");
const pinned = rewriteImage(unmasked.text, service, image);
+58
View File
@@ -0,0 +1,58 @@
import { describe, expect, it } from 'vitest';
import { collectHostStatus, runHostStatusLoop } from './host-status.ts';
import type { HostStatus } from '../domain/host-status.ts';
import { FakeCommandRunner } from '../testing/fakes.ts';
import { FakeClock } from '../testing/fake-clock.ts';
const PS = ['docker', 'ps', '-a', '--filter', 'label=com.docker.compose.project', '--format', '{{json .}}'].join(' ');
const APPS = [{ app: 'jellyfin', project: 'jellyfin', service: 'jellyfin', enabled: true }];
const running = JSON.stringify({ Labels: 'com.docker.compose.project=jellyfin,com.docker.compose.service=jellyfin', State: 'running', Status: 'Up 1 hour (healthy)' });
function deps(runner: FakeCommandRunner, clock: FakeClock, backup: string | null) {
return { runner, clock, apps: () => APPS, readBackup: () => Promise.resolve(backup), backupVmid: 103 };
}
describe('collectHostStatus', () => {
it('lists every app with its container and the newest VM backup', async () => {
const clock = new FakeClock('2026-09-30T06:00:00.000Z');
const runner = new FakeCommandRunner().on(PS, { exitCode: 0, stdout: running, stderr: '' });
const record = JSON.stringify({ vmid: 103, storage: 'pbs', latest: { volid: 'pbs:backup/vm/103/2026-09-30T00:00:00Z', ctime: Date.parse('2026-09-30T00:00:00Z') / 1000 }, checked_at: '2026-09-30T05:00:00Z' });
expect(await collectHostStatus(deps(runner, clock, record))).toEqual({
checkedAt: '2026-09-30T06:00:00.000Z',
apps: [{ app: 'jellyfin', enabled: true, state: 'running', health: 'healthy' }],
backup: { ok: true, createdAt: '2026-09-30T00:00:00.000Z', ageHours: 6, volid: 'pbs:backup/vm/103/2026-09-30T00:00:00Z' },
error: null,
});
});
it('says why the containers are unknown and keeps a bad backup record as a finding', async () => {
const clock = new FakeClock();
const runner = new FakeCommandRunner().on(PS, { exitCode: 1, stdout: '', stderr: 'proxy down' });
const status = await collectHostStatus(deps(runner, clock, '{"vmid": "x"}'));
expect(status.apps).toEqual([]);
expect(status.error).toContain('docker ps');
expect(status.backup?.ok).toBe(false);
});
it('has no backup entry without a record', async () => {
const runner = new FakeCommandRunner().on(PS, { exitCode: 0, stdout: '', stderr: '' });
expect((await collectHostStatus(deps(runner, new FakeClock(), null))).backup).toBeNull();
});
});
describe('runHostStatusLoop', () => {
it('stores a fresh status every interval until stopped', async () => {
const clock = new FakeClock();
const runner = new FakeCommandRunner().on(PS, { exitCode: 0, stdout: running, stderr: '' });
const stored: HostStatus[] = [];
const stop = new AbortController();
const loop = runHostStatusLoop({ ...deps(runner, clock, null), store: { put: (status) => stored.push(status) }, intervalMs: 60_000, signal: stop.signal, onError: () => undefined });
await new Promise((resolve) => setTimeout(resolve, 0));
expect(stored).toHaveLength(1);
await clock.advance(60_000);
await new Promise((resolve) => setTimeout(resolve, 0));
expect(stored).toHaveLength(2);
stop.abort();
await loop;
});
});
+66
View File
@@ -0,0 +1,66 @@
import { backupVerdict, HostBackupRecord } from '../domain/backup-check.ts';
import { appHealth, parseLabeledDockerPs } from '../domain/host-status.ts';
import type { BackupStatus, HostStatus } from '../domain/host-status.ts';
import type { Clock, CommandRunner } from '../domain/ports.ts';
import { describeError } from '../domain/mask.ts';
export interface HostStatusDeps {
/** Commands see only a read-only socket proxy: `docker ps` is a GET. */
readonly runner: CommandRunner;
readonly clock: Clock;
readonly apps: () => readonly { readonly app: string; readonly project: string; readonly service: string; readonly enabled: boolean }[];
/** The Proxmox timer's record of the newest vzdump; null when there is none. */
readonly readBackup: () => Promise<string | null>;
readonly backupVmid: number;
}
const PS_ARGS = ['ps', '-a', '--filter', 'label=com.docker.compose.project', '--format', '{{json .}}'];
async function backupStatus(deps: HostStatusDeps): Promise<BackupStatus | null> {
try {
const raw = await deps.readBackup();
if (raw === null) return null;
const record = HostBackupRecord.safeParse(JSON.parse(raw));
if (!record.success) return { ok: false, detail: 'the host backup record is not readable' };
const verdict = backupVerdict(record.data, { vmid: deps.backupVmid, now: deps.clock.now() });
return verdict.ok ? { ok: true, createdAt: verdict.result.created_at, ageHours: verdict.result.age_hours, volid: verdict.result.volid } : { ok: false, detail: verdict.detail };
} catch (cause) {
return { ok: false, detail: `the host backup record is not readable: ${describeError(cause)}` };
}
}
/** One look at VM103: the containers of every configured app and the newest VM backup. Never throws. */
export async function collectHostStatus(deps: HostStatusDeps, signal?: AbortSignal): Promise<HostStatus> {
const checkedAt = deps.clock.now().toISOString();
const backup = await backupStatus(deps);
try {
const ps = await deps.runner.run({ command: 'docker', args: PS_ARGS, ...(signal === undefined ? {} : { signal }) });
if (ps.exitCode !== 0) return { checkedAt, apps: [], backup, error: `docker ps exited with ${String(ps.exitCode)}: ${ps.stderr.trim().slice(0, 200)}` };
return { checkedAt, apps: appHealth(parseLabeledDockerPs(ps.stdout), deps.apps()), backup, error: null };
} catch (cause) {
return { checkedAt, apps: [], backup, error: `docker ps failed: ${describeError(cause)}` };
}
}
/** Looks again every `intervalMs` and stores what it saw, until `signal` aborts. */
export async function runHostStatusLoop(
deps: HostStatusDeps & {
readonly store: { put(status: HostStatus): void };
readonly intervalMs: number;
readonly signal: AbortSignal;
readonly onError: (cause: unknown) => void;
},
): Promise<void> {
while (!deps.signal.aborted) {
try {
deps.store.put(await collectHostStatus(deps, deps.signal));
} catch (cause) {
deps.onError(cause);
}
try {
await deps.clock.sleep(deps.intervalMs, deps.signal);
} catch {
return; // stopped
}
}
}
+5 -5
View File
@@ -7,7 +7,7 @@ const PINNED = `lscr.io/linuxserver/jackett:v0.24.2644-ls36@sha256:${'e'.repeat(
const config = (over: Partial<AppConfig> = {}): AppConfig => ({
app: 'jackett', project: 'jackett', enabled: true, position: 1, service: 'jackett', imageRepo: 'lscr.io/linuxserver/jackett',
tagPattern: '^v.*$', auto: 'patch', httpUrl: 'http://jackett:9117', expectStatus: 200, releaseNotesExecutor: 'sonnet',
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, notes: null, apply: true, ...over,
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, postUpgradeWhen: 'when_needed', notes: null, apply: true, ...over,
});
function prompts(app: AppConfig) {
@@ -24,16 +24,16 @@ describe('model action prompts', () => {
it('asks for the post-upgrade check what the old chain asks, with the versions, the group and the skill', async () => {
const asked = await prompts(config()).postUpgrade(target);
expect(asked.executor).toBe('sonnet');
expect(asked.executor).toBe('haiku');
expect(asked.prompt).toContain('v0.24.2627-ls34');
expect(asked.prompt).toContain('v0.24.2644-ls36');
expect(asked.prompt).toContain('flaresolverr');
expect(asked.prompt).toContain('Jackett keeps its indexers in /config.');
});
it('uses the models the app chose, opus by default for a repair', async () => {
expect((await prompts(config()).repair({ ...target, reason: 'verify_failed' })).executor).toBe('opus');
expect((await prompts(config({ postUpgradeExecutor: 'opus', migrateExecutor: 'sonnet' })).postUpgrade(target)).executor).toBe('opus');
it('uses the models the app chose, haiku by default for a repair', async () => {
expect((await prompts(config()).repair({ ...target, reason: 'verify_failed' })).executor).toBe('haiku');
expect((await prompts(config({ postUpgradeExecutor: 'opus', postUpgradeWhen: 'when_needed', migrateExecutor: 'sonnet' })).postUpgrade(target)).executor).toBe('opus');
const repair = await prompts(config({ migrateExecutor: 'sonnet' })).repair({ ...target, previousImage: null, reason: 'verify_failed: unhealthy' });
expect(repair.executor).toBe('sonnet');
expect(repair.prompt).toContain('verify_failed: unhealthy');
+5 -4
View File
@@ -2,6 +2,7 @@ import { splitImage } from '../domain/image-ref.ts';
import { buildPostUpgradePrompt } from '../domain/post-upgrade.ts';
import { buildRepairPrompt } from '../domain/repair.ts';
import type { AppConfigStore, ModelActionPrompts, SkillStore } from '../domain/ports.ts';
import { modelOr } from '../domain/steps.ts';
import type { AppConfig } from '../domain/steps.ts';
import type { GroupMember } from './ops/group.ts';
@@ -15,8 +16,8 @@ const tagOf = (image: string | null): string => (image === null ? 'unknown' : sp
/**
* The workflow engine's model actions ask what the old chain asks (domain/post-upgrade.ts, domain/repair.ts), with the
* app's compose group and skill, and the model the app chose: postUpgradeExecutor (sonnet by default) for the check
* after an update, migrateExecutor (opus by default) for a repair.
* app's compose group and skill, and the model the app chose: postUpgradeExecutor for the check after an update,
* migrateExecutor for a repair (each haiku by default).
*/
export function createModelActionPrompts(deps: ModelActionPromptDeps): ModelActionPrompts {
const configOf = (component: string): AppConfig => {
@@ -41,7 +42,7 @@ export function createModelActionPrompts(deps: ModelActionPromptDeps): ModelActi
update: { current: tagOf(request.previousImage), candidate: tagOf(request.image), pinned: request.image },
group, skill, migration: request.migration,
});
return { executor: config.postUpgradeExecutor === 'opus' ? 'opus' : 'sonnet', prompt };
return { executor: modelOr(config.postUpgradeExecutor), prompt };
},
async repair(request, signal) {
const { config, group, skill } = await facts(request.component, signal);
@@ -50,7 +51,7 @@ export function createModelActionPrompts(deps: ModelActionPromptDeps): ModelActi
update: { current: tagOf(request.previousImage), candidate: tagOf(request.image), pinned: request.image },
httpUrl: config.httpUrl, expectStatus: config.expectStatus, group, skill, fullAccess: request.fullAccess === true,
});
return { executor: config.migrateExecutor === 'sonnet' ? 'sonnet' : 'opus', prompt };
return { executor: modelOr(config.migrateExecutor), prompt };
},
};
}
+2 -1
View File
@@ -6,6 +6,7 @@ import { STRUCTURED_OUTPUT } from '../../domain/agent-guard.ts';
import { describeError, truncate } from '../../domain/mask.ts';
import type { CommandRunner, ModelFailure } from '../../domain/ports.ts';
import type { ModelSidecarRunResult } from '../../domain/model-api.ts';
import type { ModelName } from '../../domain/steps.ts';
// Runs the unmodified `claude` CLI, logged in with the owner's own account (docs/DESIGN.md, "Wykonawcy modeli"): print mode,
// JSON answer checked against a schema, a fresh empty working directory (no CLAUDE.md, no settings of anyone else), and for
@@ -30,7 +31,7 @@ export interface ClaudeDeps {
}
export interface ClaudeRunRequest {
readonly model: 'sonnet' | 'opus';
readonly model: ModelName;
readonly profile: 'readonly' | 'observe' | 'tools';
readonly prompt: string;
readonly outputSchema: unknown;
+1 -1
View File
@@ -88,7 +88,7 @@ describe('privileged action dispatcher', () => {
const current = {
app: 'jackett', project: 'jackett', enabled: true, position: 1, service: 'jackett', imageRepo: 'linuxserver/jackett',
tagPattern: '.*', auto: 'patch' as const, httpUrl: null, expectStatus: 200, releaseNotesExecutor: 'sonnet' as const,
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, notes: null, apply: false,
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, postUpgradeWhen: 'when_needed' as const, notes: null, apply: false,
};
const liveConfig = { get: () => current, list: () => [current], upsert: () => undefined };
const backupPolicy = PrivilegedActionPolicy.parse({
+1 -1
View File
@@ -22,7 +22,7 @@ const config = (over: Partial<AppConfig> = {}): AppConfig => ({
releaseNotesExecutor: 'sonnet',
migrateExecutor: null,
promptHint: null,
postUpgradeExecutor: null,
postUpgradeExecutor: null, postUpgradeWhen: 'when_needed',
notes: null,
apply: true,
...over,
+4 -3
View File
@@ -4,6 +4,7 @@ import { ModelError } from '../domain/ports.ts';
import type { ModelExecutor, SkillStore } from '../domain/ports.ts';
import { buildReleaseNotesPrompt, RELEASE_NOTES_ANSWER_SCHEMA, ReleaseNotesAnswer } from '../domain/release-notes.ts';
import type { ReleaseNotesContext } from '../domain/release-notes.ts';
import type { ModelName } from '../domain/steps.ts';
// Before an update a model finds the release notes itself (web search and fetch, the owner's list of trusted sources as
// a preference) and lays its assessment in front of the owner (requirements, breaking changes, what to do before and
@@ -14,8 +15,8 @@ export interface ReleaseNotesRead {
}
export type ReleaseNotesAssessment =
| { readonly kind: 'failed'; readonly why: string; readonly notes: ReleaseNotesRead; readonly model: 'sonnet' | 'opus'; readonly costUsd: number }
| { readonly kind: 'assessed'; readonly report: AgentReport; readonly notes: ReleaseNotesRead; readonly model: 'sonnet' | 'opus'; readonly costUsd: number };
| { readonly kind: 'failed'; readonly why: string; readonly notes: ReleaseNotesRead; readonly model: ModelName; readonly costUsd: number }
| { readonly kind: 'assessed'; readonly report: AgentReport; readonly notes: ReleaseNotesRead; readonly model: ModelName; readonly costUsd: number };
export interface ReleaseNotesQuestion {
readonly app: string;
@@ -27,7 +28,7 @@ export interface ReleaseNotesQuestion {
readonly versionChange: string;
readonly auto: string;
readonly promptHint: string | null;
readonly model: 'sonnet' | 'opus';
readonly model: ModelName;
readonly group: () => Promise<ReleaseNotesContext['group']>;
}
+7
View File
@@ -82,4 +82,11 @@ describe('notifyRun', () => {
expect(await notifyRun(deps, 'nope')).toBe(false);
expect(notifier.sent).toHaveLength(1);
});
it('holds the push in quiet hours', async () => {
const notifier = new RecordingNotifier();
const deps = { get: () => instance('failed'), report: () => report({}), notifier, quiet: () => true };
expect(await notifyRun(deps, 'run-1')).toBe(false);
expect(notifier.sent).toHaveLength(0);
});
});
+9 -3
View File
@@ -61,15 +61,21 @@ export function runNotification(instance: WorkflowInstance, report: RunReport):
}
}
/** Pushes the run's news, if any; true when a push went out. A failed push is the caller's to log. */
/** Pushes the run's news, if any, outside quiet hours; true when a push went out. A failed push is the caller's to log. */
export async function notifyRun(
deps: { readonly get: (instanceId: string) => WorkflowInstance | null; readonly report: (instance: WorkflowInstance) => RunReport; readonly notifier: Notifier },
deps: {
readonly get: (instanceId: string) => WorkflowInstance | null;
readonly report: (instance: WorkflowInstance) => RunReport;
readonly notifier: Notifier;
/** True in the owner's quiet hours: nothing rings, the morning report covers the run (app/digest.ts). */
readonly quiet?: () => boolean;
},
instanceId: string,
): Promise<boolean> {
const instance = deps.get(instanceId);
if (instance === null) return false;
const notification = runNotification(instance, deps.report(instance));
if (notification === null) return false;
if (notification === null || deps.quiet?.() === true) return false;
await deps.notifier.notify(notification);
return true;
}
+1 -1
View File
@@ -44,7 +44,7 @@ async function setup() {
appConfigStore.upsert({
app: 'jackett', project: 'jackett', enabled: true, position: 1, service: 'jackett', imageRepo: 'linuxserver/jackett',
tagPattern: 'version-*', auto: 'patch', httpUrl: null, expectStatus: 200, releaseNotesExecutor: 'script',
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, notes: null, apply: true,
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, postUpgradeWhen: 'when_needed', notes: null, apply: true,
});
skillStore.upsert({ project: 'jackett', content: 'Jackett torrent proxy', updatedAt: now });
await ensureAppWorkflows({ workflowCatalog, functionCatalog, functionRuntime: passingRuntime, extensions, newId: () => crypto.randomUUID(), now: () => now }, ['jackett']);
+8 -6
View File
@@ -8,7 +8,8 @@ import type {
WorkflowInstanceStore,
} from '../domain/ports.ts';
import type { WorkflowInstance } from '../domain/workflow.ts';
import type { AppConfig, JsonValue } from '../domain/steps.ts';
import { modelOr } from '../domain/steps.ts';
import type { AppConfig, JsonValue, ModelName } from '../domain/steps.ts';
import { buildComponentUpdateSnapshot, buildSystemSnapshot } from '../domain/workflow-snapshot-builder.ts';
import { appWorkflowId } from './workflow-seed.ts';
@@ -123,13 +124,13 @@ export function startSystemRun(deps: WorkflowRunDeps, workflowId: string): Start
* project and service), so only what the context cannot say is here: apply's dry-run flag and the model profiles.
*/
function realRunRequests(deps: WorkflowRunDeps, config: AppConfig): Record<string, JsonValue> {
const profile = (model: 'sonnet' | 'opus') => {
const profile = (model: ModelName) => {
const latest = deps.modelProfileStore.getLatestVersion(model);
return latest === null ? null : { profileId: latest.profileId, version: latest.version };
};
// The same models the old chain uses: the app's choice, else sonnet checks after an update and opus repairs.
const postUpgrade = config.postUpgradeExecutor === 'opus' ? 'opus' : 'sonnet';
const repair = config.migrateExecutor === 'sonnet' ? 'sonnet' : 'opus';
// The app's choice, else the default model.
const postUpgrade = modelOr(config.postUpgradeExecutor);
const repair = modelOr(config.migrateExecutor);
return {
backupRequest: { project: config.project },
applyDryRunRequest: { dryRun: true },
@@ -139,7 +140,8 @@ function realRunRequests(deps: WorkflowRunDeps, config: AppConfig): Record<strin
repairFullRequest: { reason: 'owner_requested', access: 'full' },
rollbackRequest: {},
migrateRequest: {},
postUpgradeRequest: {},
// Without a model chosen the check is off (domain/steps.ts); with one, the app says when it runs.
postUpgradeRequest: { when: config.postUpgradeExecutor === null ? 'off' : config.postUpgradeWhen },
postUpgradeProfile: profile(postUpgrade),
repairProfile: profile(repair),
};
+24 -6
View File
@@ -104,13 +104,14 @@ const MIGRATION_PLAN = {
stepsBefore: [], stepsAfter: ['hl ps'], rollback: 'Restore the backup.',
};
function fakeOperations(tags: 'candidate_within_auto' | 'candidate_above_auto' = 'candidate_within_auto', migration: 'none' | 'needed' = 'none'): ComponentOperationRunner {
function fakeOperations(tags: 'candidate_within_auto' | 'candidate_above_auto' = 'candidate_within_auto', migration: 'none' | 'needed' | 'image_only' = 'none'): ComponentOperationRunner {
const plan = migration === 'image_only' ? { ...MIGRATION_PLAN, composeYaml: null, inputs: [] } : MIGRATION_PLAN;
const results: Record<string, { event: string; result: JsonValue }> = {
image_version: { event: 'version_detected', result: { event: 'version_detected', detectedVersion: '0.22.0', imageVersionState: 'detected' } },
image_tags: { event: tags, result: { event: tags, current: '0.22.0', candidate: 'version-v0.22.1', change: tags === 'candidate_within_auto' ? 'patch' : 'major' } },
release_notes: { event: 'no_source', result: { sources: [] } },
migration_plan: migration === 'needed'
? { event: 'migration_needed', result: { plan: MIGRATION_PLAN, planDigest: 'd'.repeat(64), summary: MIGRATION_PLAN.summary, model: 'opus', costUsd: 0.3, actions: [] } }
migration_plan: migration !== 'none'
? { event: 'migration_needed', result: { plan, planDigest: 'd'.repeat(64), summary: plan.summary, model: 'opus', costUsd: 0.3, actions: [] } }
: { event: 'no_migration', result: { plan: null, summary: 'Nothing to migrate.' } },
image_resolve: {
event: 'resolved',
@@ -146,7 +147,7 @@ async function setup(
appConfigStore.upsert({
app: 'jackett', project: 'jackett', enabled: true, position: 1, service: 'jackett', imageRepo: 'linuxserver/jackett',
tagPattern: 'version-*', auto: 'patch', httpUrl: null, expectStatus: 200, releaseNotesExecutor: 'script',
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, notes: null, apply: true,
migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, postUpgradeWhen: 'when_needed', notes: null, apply: true,
});
skillStore.upsert({ project: 'jackett', content: 'Jackett torrent proxy', updatedAt: now });
// The real sandbox: the template functions run in bwrap exactly as on VM103.
@@ -253,7 +254,7 @@ describe('workflow runner', () => {
{ image: PINNED, backupId: BACKUP_ID, expectedManifestSha256: MANIFEST, dryRun: true },
{ image: PINNED, backupId: BACKUP_ID, expectedManifestSha256: MANIFEST, dryRun: false },
]);
expect(docker.requests.find((request) => request.action === 'model.postUpgrade')).toMatchObject({ image: PINNED, backupId: BACKUP_ID, profile: { profileId: 'sonnet' } });
expect(docker.requests.find((request) => request.action === 'model.postUpgrade')).toMatchObject({ image: PINNED, backupId: BACKUP_ID, profile: { profileId: 'haiku' }, when: 'off', reasons: [] });
});
it('rolls the app back when the owner chooses rollback after the model asked for action', async () => {
@@ -321,10 +322,27 @@ describe('workflow runner', () => {
action: 'docker.migrate', version: 1, component: 'jackett', project: 'jackett', service: 'jackett', image: PINNED, backupId: BACKUP_ID,
expectedManifestSha256: MANIFEST, composeYaml: MIGRATION_PLAN.composeYaml, planDigest: 'd'.repeat(64), inputs: [{ name: 'ADMIN_PASSWORD', target: 'env' }],
});
expect(docker.requests.find((request) => request.action === 'model.postUpgrade')).toMatchObject({ migration: { summary: MIGRATION_PLAN.summary, stepsAfter: ['hl ps'] } });
expect(docker.requests.find((request) => request.action === 'model.postUpgrade')).toMatchObject({
migration: { summary: MIGRATION_PLAN.summary, stepsAfter: ['hl ps'] }, reasons: ['approved_update', 'major', 'migration'],
});
expect(instanceStore.getReplayStatus('run-1').status).toBe('ok');
});
it('runs an approved migration that keeps the compose file: the executor only pins the new image', async () => {
const { runner, instanceStore, docker, runDeps } = await setup(fakeOperations('candidate_above_auto', 'image_only'));
startWorkflowRun(runDeps, 'jackett', 'real');
await runUntilIdle(runner);
const instance = instanceStore.get('run-1')!;
expect(instanceStore.submitOwnerDecision({
decision: { schemaVersion: 1, decisionId: crypto.randomUUID(), signal: 'owner-decision.v1', instanceId: 'run-1', waitNodeId: 'gate_migration', action: 'approve', note: null },
expectedLifecycleSeq: instance.projection.seq, expectedExecutionSeq: instanceStore.getReplayStatus('run-1').replaySeq, now,
}).ok).toBe(true);
expect(await runUntilIdle(runner)).toContain('apply_migration');
expect(docker.requests.find((request) => request.action === 'docker.migrate')).toMatchObject({ composeYaml: null, inputs: [] });
expect(instanceStore.get('run-1')?.projection.status).toBe('succeeded');
});
it('goes the usual way above the auto threshold when the model finds no migration', async () => {
const { runner, runDeps } = await setup(fakeOperations('candidate_above_auto'));
startShadowRun(runDeps, 'jackett');
+3 -3
View File
@@ -60,10 +60,10 @@ describe('ensureUpdatePolicy', () => {
});
describe('ensureDefaultModelProfiles', () => {
it('adds read-only Claude profiles for the sonnet and opus executors once', () => {
it('adds read-only Claude profiles for the haiku, sonnet and opus executors once', () => {
const store = createSqliteModelProfileStore(openDatabase(':memory:'));
expect(ensureDefaultModelProfiles(store, now)).toEqual(['sonnet', 'opus']);
expect(ensureDefaultModelProfiles(store, now)).toEqual(['haiku', 'sonnet', 'opus']);
expect(ensureDefaultModelProfiles(store, now)).toEqual([]);
expect(store.getLatestVersion('opus')?.document).toMatchObject({ provider: 'claude', model: 'opus', mode: 'readonly', timeoutMs: 1_800_000 });
});
@@ -74,7 +74,7 @@ describe('ensureDefaultModelProfiles', () => {
store.publishVersion({ profileId: 'sonnet', expectedVersion: 1, document: first('sonnet'), now });
store.publishVersion({ profileId: 'opus', expectedVersion: 1, document: first('opus', { provider: 'codex' }), now });
expect(ensureDefaultModelProfiles(store, now)).toEqual(['sonnet']);
expect(ensureDefaultModelProfiles(store, now)).toEqual(['haiku', 'sonnet']);
expect(store.getLatestVersion('sonnet')).toMatchObject({ version: 2, document: { timeoutMs: 1_800_000 } });
expect(store.getLatestVersion('opus')).toMatchObject({ version: 1, document: { provider: 'codex', timeoutMs: 300_000 } });
});
+5 -4
View File
@@ -12,6 +12,7 @@ import {
} from '../domain/workflow-update.ts';
import { createWorkflowCatalogApp, createWorkflowGraphValidator } from './workflow-catalog.ts';
import { createWorkflowFunctionCatalogApp } from './workflow-function-catalog.ts';
import { ModelName } from '../domain/steps.ts';
/** Each app gets its own update workflow, first published as a copy of the component-update template. */
export { appWorkflowId };
@@ -28,7 +29,7 @@ export function ensureUpdatePolicy(store: PrivilegedActionStore, now: string): b
}
/** A default profile's first document; one still exactly like it was never changed by the owner. */
const FIRST_DEFAULT_PROFILE = (model: 'sonnet' | 'opus'): ModelProfileDocumentV1 => ({
const FIRST_DEFAULT_PROFILE = (model: ModelName): ModelProfileDocumentV1 => ({
schemaVersion: 1, provider: 'claude', model, mode: 'readonly', timeoutMs: 300_000, maxCostUsd: null, tools: [], systemPrompt: null, context: null,
});
@@ -36,17 +37,17 @@ const FIRST_DEFAULT_PROFILE = (model: 'sonnet' | 'opus'): ModelProfileDocumentV1
* The default profile: the model sidecar's own limit (30 min, MODEL_TIMEOUT_MS), since a repair session takes long; the
* dispatcher stops a run at its profile's timeout.
*/
const DEFAULT_PROFILE = (model: 'sonnet' | 'opus'): ModelProfileDocumentV1 => ({ ...FIRST_DEFAULT_PROFILE(model), timeoutMs: 30 * 60_000 });
const DEFAULT_PROFILE = (model: ModelName): ModelProfileDocumentV1 => ({ ...FIRST_DEFAULT_PROFILE(model), timeoutMs: 30 * 60_000 });
/**
* App configs name 'sonnet' and 'opus' as executors; a model request without a pinned profile uses the profile of that
* App configs name 'haiku', 'sonnet' and 'opus' as executors; a model request without a pinned profile uses the profile of that
* name, and a workflow snapshot pins one for each. Adds the missing ones as Claude profiles, and moves a profile the owner
* never changed from its first document to the current default; the owner edits them on the Models page (the provider
* picks the Claude or Codex sidecar).
*/
export function ensureDefaultModelProfiles(store: ModelProfileStore, now: string): readonly string[] {
const added: string[] = [];
for (const model of ['sonnet', 'opus'] as const) {
for (const model of ModelName.options) {
const latest = store.getLatestVersion(model);
if (latest !== null && !isDeepStrictEqual(latest.document, FIRST_DEFAULT_PROFILE(model))) continue;
const result = store.publishVersion({ profileId: model, expectedVersion: (latest?.version ?? 0) + 1, document: DEFAULT_PROFILE(model), now });
+1 -1
View File
@@ -16,7 +16,7 @@ const bazarr: AppConfig = {
releaseNotesExecutor: 'sonnet',
migrateExecutor: null,
promptHint: null,
postUpgradeExecutor: null,
postUpgradeExecutor: null, postUpgradeWhen: 'when_needed',
notes: null,
apply: false,
};
+5 -2
View File
@@ -1,5 +1,5 @@
import { z } from 'zod';
import { AutoLevel, Executor } from './steps.ts';
import { AutoLevel, Executor, ModelName, PostUpgradeWhen } from './steps.ts';
import type { AppConfig } from './steps.ts';
import { compileTagPattern } from './tags.ts';
@@ -15,7 +15,7 @@ export function countWords(text: string): number {
}
/** Model steps run on a model; `script` is not a choice for them. */
const ModelChoice = z.enum(['sonnet', 'opus']);
const ModelChoice = ModelName;
const cleared = (value: unknown) => (typeof value === 'string' && value.trim() === '' ? null : value);
@@ -41,6 +41,7 @@ export const AppConfigPatch = z
migrateExecutor: ModelChoice.nullable(),
promptHint: z.preprocess(cleared, z.string().trim().max(500).nullable()),
postUpgradeExecutor: ModelChoice.nullable(),
postUpgradeWhen: PostUpgradeWhen,
notes: z.preprocess(
cleared,
z
@@ -61,6 +62,8 @@ export const AppConfigValue = z.strictObject({
service: z.string().min(1), imageRepo: z.string().min(1), tagPattern: z.string(), auto: AutoLevel,
httpUrl: z.string().nullable(), expectStatus: z.number().int(), releaseNotesExecutor: Executor,
migrateExecutor: Executor.nullable(), promptHint: z.string().nullable(), postUpgradeExecutor: Executor.nullable(),
// Revisions from before 2026-10-07 have none: the same default as the app_config column (migration 25).
postUpgradeWhen: PostUpgradeWhen.default('when_needed'),
notes: z.string().nullable(), apply: z.boolean(),
});
+2 -1
View File
@@ -88,7 +88,8 @@ export const MigrateRequest = z.strictObject({
image: PinnedImage,
expect_manifest_sha256: Sha256,
backup_id: BackupId,
compose_yaml: z.string().min(1).max(65_536),
/** null: the plan keeps the current compose.yaml; only the service's image changes. */
compose_yaml: z.string().min(1).max(65_536).nullable(),
env: z.record(z.string().regex(/^[A-Z][A-Z0-9_]{0,63}$/), z.string().max(4096).regex(/^[^\n\r]*$/, 'one line')),
});
export type MigrateRequest = z.infer<typeof MigrateRequest>;
+64
View File
@@ -0,0 +1,64 @@
import { describe, expect, it } from 'vitest';
import { appHealth, downApps, parseLabeledDockerPs, worstHealth } from './host-status.ts';
import type { LabeledContainer } from './host-status.ts';
const row = (project: string, service: string, state: string, status: string) =>
JSON.stringify({ Labels: `com.docker.compose.project=${project},traefik.rule=Host(\`a,b\`),com.docker.compose.service=${service}`, State: state, Status: status, Image: 'img' });
describe('parseLabeledDockerPs', () => {
it('reads the compose project and service from the labels, and health from the status', () => {
const rows = parseLabeledDockerPs([row('grafana', 'grafana', 'running', 'Up 2 days (healthy)'), row('grafana', 'influxdb', 'restarting', 'Restarting (1) 3 seconds ago')].join('\n'));
expect(rows).toEqual([
{ project: 'grafana', service: 'grafana', state: 'running', health: 'healthy' },
{ project: 'grafana', service: 'influxdb', state: 'restarting', health: null },
]);
});
it('skips containers that compose did not start', () => {
expect(parseLabeledDockerPs(JSON.stringify({ Labels: 'other=1', State: 'running', Status: 'Up' }))).toEqual([]);
});
});
describe('appHealth', () => {
const apps = [
{ app: 'grafana', project: 'grafana', service: 'grafana', enabled: true },
{ app: 'grafana-influxdb', project: 'grafana', service: 'influxdb', enabled: false },
{ app: 'jellyfin', project: 'jellyfin', service: 'jellyfin', enabled: true },
{ app: 'ombi', project: 'ombi', service: 'ombi', enabled: true },
];
it('gives every app the state of its service: missing when no container, restarting before running, the worst health', () => {
const rows: LabeledContainer[] = [
{ project: 'grafana', service: 'grafana', state: 'running', health: 'healthy' },
{ project: 'grafana', service: 'influxdb', state: 'exited', health: null },
{ project: 'ombi', service: 'ombi', state: 'running', health: 'healthy' },
{ project: 'ombi', service: 'ombi', state: 'restarting', health: null },
];
expect(appHealth(rows, apps)).toEqual([
{ app: 'grafana', enabled: true, state: 'running', health: 'healthy' },
{ app: 'grafana-influxdb', enabled: false, state: 'exited', health: null },
{ app: 'jellyfin', enabled: true, state: 'missing', health: null },
{ app: 'ombi', enabled: true, state: 'restarting', health: 'healthy' },
]);
});
});
describe('downApps', () => {
it('names enabled apps that do not run or run unhealthy', () => {
const apps = [
{ app: 'a', enabled: true, state: 'running', health: null },
{ app: 'b', enabled: true, state: 'running', health: 'unhealthy' as const },
{ app: 'c', enabled: true, state: 'missing', health: null },
{ app: 'd', enabled: false, state: 'exited', health: null },
];
expect(downApps({ apps })).toEqual(['b', 'c']);
});
});
describe('worstHealth', () => {
it('orders unhealthy over starting over healthy over no check', () => {
expect(worstHealth([null, 'healthy', 'starting'])).toBe('starting');
expect(worstHealth(['healthy', 'unhealthy'])).toBe('unhealthy');
expect(worstHealth([null])).toBeNull();
});
});
+93
View File
@@ -0,0 +1,93 @@
import { z } from 'zod';
import { healthFromStatus } from './compose.ts';
// What runs on VM103 right now, as the driver last saw it: the containers of every configured app and the age of the
// newest VM backup. The driver writes it, the server shows it; nothing decides on it.
export const ContainerHealth = z.enum(['healthy', 'unhealthy', 'starting']);
export type ContainerHealth = z.infer<typeof ContainerHealth>;
export const AppHealth = z.object({
app: z.string(),
enabled: z.boolean(),
/** Docker's state of the service's container (`running`, `restarting`, `exited`…), or `missing` when there is none. */
state: z.string(),
health: ContainerHealth.nullable(),
});
export type AppHealth = z.infer<typeof AppHealth>;
export const BackupStatus = z.discriminatedUnion('ok', [
z.object({ ok: z.literal(true), createdAt: z.string(), ageHours: z.number(), volid: z.string() }),
z.object({ ok: z.literal(false), detail: z.string() }),
]);
export type BackupStatus = z.infer<typeof BackupStatus>;
export const HostStatus = z.object({
checkedAt: z.string(),
apps: z.array(AppHealth),
/** Null when the driver has no backup record to read. */
backup: BackupStatus.nullable(),
/** Why the containers could not be listed; the apps list is then empty. */
error: z.string().nullable(),
});
export type HostStatus = z.infer<typeof HostStatus>;
export interface LabeledContainer {
readonly project: string;
readonly service: string;
readonly state: string;
readonly health: ContainerHealth | null;
}
// Label values may contain commas (traefik rules), so each compose label is matched by its full name at an entry start.
const PROJECT_LABEL = /(?:^|,)com\.docker\.compose\.project=([^,]*)/;
const SERVICE_LABEL = /(?:^|,)com\.docker\.compose\.service=([^,]*)/;
const PsRow = z.object({ Labels: z.string().default(''), State: z.string().default('unknown'), Status: z.string().default('') });
/** Rows of `docker ps -a --format '{{json .}}'`, one JSON object per line; containers without compose labels are left out. */
export function parseLabeledDockerPs(raw: string): LabeledContainer[] {
const out: LabeledContainer[] = [];
for (const line of raw.split('\n')) {
if (line.trim() === '') continue;
const row = PsRow.parse(JSON.parse(line));
const project = PROJECT_LABEL.exec(row.Labels)?.[1];
const service = SERVICE_LABEL.exec(row.Labels)?.[1];
if (project === undefined || service === undefined) continue;
out.push({ project, service, state: row.State, health: healthFromStatus(row.Status) });
}
return out;
}
const HEALTH_RANK: Readonly<Record<ContainerHealth, number>> = { healthy: 1, starting: 2, unhealthy: 3 };
export function worstHealth(values: readonly (ContainerHealth | null)[]): ContainerHealth | null {
let worst: ContainerHealth | null = null;
for (const value of values) {
if (value !== null && (worst === null || HEALTH_RANK[value] > HEALTH_RANK[worst])) worst = value;
}
return worst;
}
// A restarting container outweighs a running one of the same service; otherwise the first container's state stands.
function worstState(states: readonly string[]): string {
if (states.length === 0) return 'missing';
if (states.includes('restarting')) return 'restarting';
return states.every((state) => state === 'running') ? 'running' : (states.find((state) => state !== 'running') ?? 'unknown');
}
/** Enabled apps whose container is not running, or runs unhealthy. */
export function downApps(status: Pick<HostStatus, 'apps'>): string[] {
return status.apps.filter((app) => app.enabled && (app.state !== 'running' || app.health === 'unhealthy')).map((app) => app.app);
}
/** Every configured app with the state of its compose service's containers, in the apps' order. */
export function appHealth(
containers: readonly LabeledContainer[],
apps: readonly { readonly app: string; readonly project: string; readonly service: string; readonly enabled: boolean }[],
): AppHealth[] {
return apps.map((app) => {
const own = containers.filter((container) => container.project === app.project && container.service === app.service);
return { app: app.app, enabled: app.enabled, state: worstState(own.map((container) => container.state)), health: worstHealth(own.map((container) => container.health)) };
});
}
+2 -1
View File
@@ -1,5 +1,6 @@
import { z } from 'zod';
import type { ModelFailure } from './ports.ts';
import { ModelName } from './steps.ts';
// The model sidecar's API: the driver asks it to run one prompt (readonly, or with the hl tool in a repair session).
@@ -16,7 +17,7 @@ export type ModelSidecarRunResult =
| { readonly ok: false; readonly kind: ModelFailure; readonly detail: string; readonly costUsd?: number | null };
export const ModelRunRequest = z.strictObject({
executor: z.enum(['sonnet', 'opus']),
executor: ModelName,
profile: z.enum(['readonly', 'observe', 'tools']),
prompt: z.string().min(1).max(400_000),
output_schema: z.record(z.string(), z.unknown()),
+68
View File
@@ -0,0 +1,68 @@
import { describe, expect, it } from 'vitest';
import { DEFAULT_NOTIFICATION_SETTINGS, digestDue, inQuietHours, nightDigest } from './notification-settings.ts';
const at = (time: string, day = '2026-09-30') => ({ hour: Number(time.slice(0, 2)), minute: Number(time.slice(3)), key: `${day}T${time}` });
describe('inQuietHours', () => {
it('holds pushes from 01:00 to 07:30 by default, and not on the evening before', () => {
const s = DEFAULT_NOTIFICATION_SETTINGS;
expect(inQuietHours(s, at('00:59'))).toBe(false);
expect(inQuietHours(s, at('01:00'))).toBe(true);
expect(inQuietHours(s, at('07:29'))).toBe(true);
expect(inQuietHours(s, at('07:30'))).toBe(false);
expect(inQuietHours(s, at('22:00'))).toBe(false);
});
it('wraps over midnight and is off when both ends are equal', () => {
const late = { ...DEFAULT_NOTIFICATION_SETTINGS, quietFrom: '22:00', quietUntil: '06:00' };
expect(inQuietHours(late, at('23:15'))).toBe(true);
expect(inQuietHours(late, at('05:59'))).toBe(true);
expect(inQuietHours(late, at('12:00'))).toBe(false);
expect(inQuietHours({ ...late, quietUntil: '22:00' }, at('23:00'))).toBe(false);
});
});
describe('digestDue', () => {
it('is due once a day from the report time on', () => {
const s = DEFAULT_NOTIFICATION_SETTINGS;
expect(digestDue(s, at('07:29'), null)).toBe(false);
expect(digestDue(s, at('07:30'), null)).toBe(true);
expect(digestDue(s, at('09:00'), '2026-09-29')).toBe(true);
expect(digestDue(s, at('09:00'), '2026-09-30')).toBe(false);
expect(digestDue({ ...s, digestEnabled: false }, at('09:00'), null)).toBe(false);
});
});
describe('nightDigest', () => {
const run = (overrides: Record<string, unknown>) => ({
app: 'sonarr', workflowId: 'update-sonarr', status: 'succeeded', mode: 'real' as const, current: '4.0.1', candidate: '4.0.2', running: '4.0.2', outcome: 'Updated', step: null, updatedAt: '2026-09-30T02:00:00.000Z',
...overrides,
});
it('lists what needs the owner first, then what changed since the last report', () => {
const digest = nightDigest({
since: '2026-09-29T05:30:00.000Z',
runs: [
run({}),
run({ app: 'radarr', running: '5.1', current: '5.1', candidate: '5.2', outcome: 'Rolled back' }),
run({ app: 'bazarr', status: 'failed', outcome: 'Verify failed', running: '1.0', current: '1.0' }),
run({ app: 'jellyfin', status: 'waiting', step: 'Owner decides', updatedAt: '2026-09-28T00:00:00.000Z' }),
run({ app: 'ombi', mode: 'shadow' as const }),
run({ app: 'old', updatedAt: '2026-09-20T00:00:00.000Z' }),
],
down: ['grafana'],
});
expect(digest.title).toBe('Night report: 3 need you');
expect(digest.detail.split('\n')).toEqual([
'Waiting for you: jellyfin (Owner decides)',
'Failed: bazarr (Verify failed)',
'Down now: grafana',
'Updated: sonarr 4.0.1 → 4.0.2',
'Not updated: radarr (Rolled back)',
]);
});
it('says all is well when nothing happened', () => {
expect(nightDigest({ since: '2026-09-29T05:30:00.000Z', runs: [], down: [] })).toEqual({ title: 'Night report: all good', detail: 'No real update, nothing failed, nothing waits for you.', needsYou: 0 });
});
});
+87
View File
@@ -0,0 +1,87 @@
import { z } from 'zod';
// When the owner's phone may ring and what the morning report says. Quiet hours hold every push (the report covers what
// happened in them); the evening before is not quiet, so a failure then still rings. Times are wall-clock in the
// schedules' time zone.
const Time = z.string().regex(/^([01]\d|2[0-3]):[0-5]\d$/, 'must be HH:MM');
export const NotificationSettings = z.object({
quietFrom: Time,
quietUntil: Time,
digestAt: Time,
digestEnabled: z.boolean(),
});
export type NotificationSettings = z.infer<typeof NotificationSettings>;
export const DEFAULT_NOTIFICATION_SETTINGS: NotificationSettings = { quietFrom: '01:00', quietUntil: '07:30', digestAt: '07:30', digestEnabled: true };
interface WallClock {
readonly hour: number;
readonly minute: number;
/** `YYYY-MM-DDTHH:MM` (domain/cron.ts zonedMinute): its day names the report. */
readonly key: string;
}
const minutes = (time: string): number => Number(time.slice(0, 2)) * 60 + Number(time.slice(3, 5));
export function inQuietHours(settings: NotificationSettings, at: WallClock): boolean {
const from = minutes(settings.quietFrom);
const until = minutes(settings.quietUntil);
const now = at.hour * 60 + at.minute;
if (from === until) return false;
return from < until ? now >= from && now < until : now >= from || now < until;
}
/** The report goes out once a day, at its time or at the first look after it (the driver may have been down). */
export function digestDue(settings: NotificationSettings, at: WallClock, lastDay: string | null): boolean {
return settings.digestEnabled && at.hour * 60 + at.minute >= minutes(settings.digestAt) && lastDay !== at.key.slice(0, 10);
}
export interface DigestRun {
readonly app: string | null;
readonly workflowId: string;
readonly status: string;
readonly mode: 'shadow' | 'real';
readonly current: string | null;
readonly candidate: string | null;
readonly running: string | null;
readonly outcome: string | null;
readonly step: string | null;
readonly updatedAt: string;
}
export interface Digest {
readonly title: string;
readonly detail: string;
/** Runs waiting at a gate, failures and apps down: what the owner should look at. */
readonly needsYou: number;
}
/**
* The morning report: what waits for the owner and what is broken first, then what changed since the last report.
* A run still waiting counts whenever it started; the rest only when it ended after `since`. Dry runs change nothing.
*/
export function nightDigest(input: { readonly runs: readonly DigestRun[]; readonly since: string; readonly down: readonly string[] }): Digest {
const since = Date.parse(input.since);
const name = (run: DigestRun) => run.app ?? run.workflowId;
const recent = input.runs.filter((run) => Date.parse(run.updatedAt) > since);
const waiting = input.runs.filter((run) => run.status === 'waiting').map((run) => `${name(run)} (${run.step ?? 'decision'})`);
const failed = recent.filter((run) => run.status === 'failed').map((run) => `${name(run)} (${run.outcome ?? 'failed'})`);
const real = recent.filter((run) => run.mode === 'real' && run.status === 'succeeded');
const updated = real.filter((run) => run.running !== null && run.running !== run.current).map((run) => `${name(run)} ${run.current ?? '?'} → ${run.running ?? '?'}`);
const kept = real.filter((run) => run.running === null || run.running === run.current).map((run) => `${name(run)} (${run.outcome ?? 'not updated'})`);
const lines = [
waiting.length > 0 ? `Waiting for you: ${waiting.join(', ')}` : null,
failed.length > 0 ? `Failed: ${failed.join(', ')}` : null,
input.down.length > 0 ? `Down now: ${input.down.join(', ')}` : null,
updated.length > 0 ? `Updated: ${updated.join(', ')}` : null,
kept.length > 0 ? `Not updated: ${kept.join(', ')}` : null,
].filter((line): line is string => line !== null);
const needsYou = waiting.length + failed.length + input.down.length;
return {
title: needsYou > 0 ? `Night report: ${String(needsYou)} need you` : 'Night report: all good',
detail: lines.length > 0 ? lines.join('\n') : 'No real update, nothing failed, nothing waits for you.',
needsYou,
};
}
+1 -1
View File
@@ -16,7 +16,7 @@ const config = (app: string, position: number, enabled = true): AppConfig => ({
releaseNotesExecutor: 'sonnet',
migrateExecutor: null,
promptHint: null,
postUpgradeExecutor: null,
postUpgradeExecutor: null, postUpgradeWhen: 'when_needed',
notes: null,
apply: false,
});
+22 -3
View File
@@ -18,6 +18,7 @@ import type {
SessionOpenRequest,
SessionOpenResult,
SessionLogResult,
AgentLogsRequest,
AgentTextResult,
SessionLogEntry,
SessionSummary,
@@ -25,7 +26,9 @@ import type {
import type { PreflightReport } from './preflight.ts';
import type { Skill, SkillVersion } from './skill.ts';
import type { AppConfigRevision } from './app-config.ts';
import type { AppConfig, Executor, JsonValue } from './steps.ts';
import type { AppConfig, Executor, JsonValue, ModelName } from './steps.ts';
import type { HostStatus } from './host-status.ts';
import type { NotificationSettings } from './notification-settings.ts';
import type { AppendableWorkflowInstanceEventInput, PublishedWorkflowVersion, WorkflowDraft, WorkflowDocument, WorkflowInstance, WorkflowInstanceCreateInput, WorkflowInstanceEvent, WorkflowInstanceStatus, WorkflowValidation } from './workflow.ts';
import type { ExecutionCommand, NodeAttempt, ReplayStatus, WorkflowExecutionProjection } from './workflow-execution.ts';
import type { ExecutionFinalizeOutcome } from './workflow-execution-ledger.ts';
@@ -401,6 +404,8 @@ export interface ExecutorClient {
sessionLog(request: SessionCloseRequest, signal?: AbortSignal): Promise<ExecutorCall<SessionLogResult>>;
/** The project's compose.yaml as the session sees it (secrets masked as `<secret:NAME>`). */
sessionCompose(request: SessionCloseRequest, signal?: AbortSignal): Promise<ExecutorCall<AgentTextResult>>;
/** A service's last log lines as the session reads them (secrets masked), like the model's `hl logs`. */
sessionLogs(request: SessionCloseRequest & AgentLogsRequest, signal?: AbortSignal): Promise<ExecutorCall<AgentTextResult>>;
}
export interface ModelRequest {
@@ -433,6 +438,20 @@ export interface ModelResult {
export type ModelFailure = 'rate_limited' | 'auth_failed' | 'invalid_output' | 'unavailable' | 'busy';
/** The owner's trusted release notes sources, one list for all apps (http(s) pages or `github:owner/repo`). */
/** The driver's latest look at VM103 (domain/host-status.ts); null before its first look. */
export interface HostStatusStore {
get(): HostStatus | null;
put(status: HostStatus): void;
}
/** The owner's quiet hours and morning report (domain/notification-settings.ts), and when the last report went out. */
export interface NotificationSettingsStore {
get(): NotificationSettings;
put(settings: NotificationSettings, now: string): void;
lastDigest(): { readonly day: string; readonly at: string } | null;
recordDigest(day: string, at: string): void;
}
export interface ReleaseNotesSourcesStore {
get(): string[];
put(sources: readonly string[], now: string): void;
@@ -460,14 +479,14 @@ export interface ModelActionPrompts {
readonly migration?: { readonly summary: string; readonly stepsAfter: readonly string[] };
},
signal?: AbortSignal,
): Promise<{ readonly executor: 'sonnet' | 'opus'; readonly prompt: string }>;
): Promise<{ readonly executor: ModelName; readonly prompt: string }>;
repair(
request: {
readonly component: string; readonly project: string; readonly service: string; readonly image: string; readonly reason: string; readonly previousImage: string | null;
readonly fullAccess?: boolean;
},
signal?: AbortSignal,
): Promise<{ readonly executor: 'sonnet' | 'opus'; readonly prompt: string }>;
): Promise<{ readonly executor: ModelName; readonly prompt: string }>;
}
/** Runs a model (the `claude` CLI logged in with the owner's account, in the model sidecar). Throws ModelError. */
+33 -1
View File
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest';
import { buildPostUpgradePrompt, POST_UPGRADE_ANSWER_SCHEMA, PostUpgradeAnswer } from './post-upgrade.ts';
import { buildPostUpgradePrompt, POST_UPGRADE_ANSWER_SCHEMA, PostUpgradeAnswer, suspiciousLogLines } from './post-upgrade.ts';
const context = {
app: 'nextcloud',
@@ -58,3 +58,35 @@ describe('PostUpgradeAnswer', () => {
expect(POST_UPGRADE_ANSWER_SCHEMA).toMatchObject({ type: 'object', required: ['outcome', 'summary', 'actions'] });
});
});
describe('suspiciousLogLines', () => {
it('finds errors, fatal ones, panics, exceptions and failed migrations', () => {
const logs = [
'[migrations] started',
'[migrations] no migrations found',
'2026-10-07 09:27:01 INFO Starting Sonarr',
'2026-10-07 09:27:02 ERROR Unable to connect to indexer',
'level=fatal msg="database is locked"',
'panic: runtime error: index out of range',
'System.NullReferenceException: Object reference not set',
'Traceback (most recent call last):',
'Migration failed: column already exists',
].join('\n');
expect(suspiciousLogLines(logs)).toEqual([
'2026-10-07 09:27:02 ERROR Unable to connect to indexer',
'level=fatal msg="database is locked"',
'panic: runtime error: index out of range',
'System.NullReferenceException: Object reference not set',
'Traceback (most recent call last):',
'Migration failed: column already exists',
]);
});
it('a clean start, and lines that only say there are no errors, find nothing', () => {
expect(suspiciousLogLines('[migrations] no migrations found\nStarted in 2.1 s, 0 errors\nno errors found\n')).toEqual([]);
});
it('keeps at most the first lines it is asked for', () => {
expect(suspiciousLogLines(Array.from({ length: 30 }, (_, index) => `ERROR ${String(index)}`).join('\n'), 3)).toEqual(['ERROR 0', 'ERROR 1', 'ERROR 2']);
});
});
+14 -1
View File
@@ -2,7 +2,20 @@ import { z } from 'zod';
// The step after a successful update (owner, 2026-09-20): some apps ask for actions once the new version runs (Nextcloud: missing
// database indices, occ repair). A model looks at what the release notes analysis and the app's skill say, does what the tool
// allows and lists the rest for the owner. Per app, off by default: then the chain stays scripts only.
// allows and lists the rest for the owner. Per app, off by default: then the chain stays scripts only. When on, by default only
// when the update may ask for something (owner, 2026-10-07): one the owner approved, a major one, a migration, or errors in the logs.
/** Why a post-upgrade check runs the model. */
export const PostUpgradeReason = z.enum(['approved_update', 'major', 'migration', 'log_errors']);
export type PostUpgradeReason = z.infer<typeof PostUpgradeReason>;
const SUSPICIOUS = /\b(error|fatal|panic|traceback|critical)\b|exception\b|migrations? failed/i;
const NO_ERRORS = /\b(no|0) errors?\b/i;
/** The log lines after a start that look like trouble, the first `max` of them. */
export function suspiciousLogLines(logs: string, max = 10): string[] {
return logs.split('\n').map((line) => line.trim()).filter((line) => SUSPICIOUS.test(line) && !NO_ERRORS.test(line)).slice(0, max);
}
export const PostUpgradeAnswer = z.object({
outcome: z
+13 -2
View File
@@ -1,4 +1,6 @@
import { z } from 'zod';
import { PostUpgradeReason } from './post-upgrade.ts';
import { ModelName } from './steps.ts';
/**
* The executor is intentionally a closed vocabulary. Adding an action is a
@@ -128,7 +130,8 @@ export const DockerMigrateRequest = z.strictObject({
image: PinnedImage,
backupId: BackupId,
expectedManifestSha256: Sha256,
composeYaml: z.string().min(1).max(65_536),
/** null when the plan keeps the compose file as it is apart from the image. */
composeYaml: z.string().min(1).max(65_536).nullable(),
planDigest: Sha256,
inputs: z.array(z.strictObject({ name: z.string().regex(/^[A-Z][A-Z0-9_]{0,63}$/), target: z.enum(['env', 'compose']) })).max(20),
});
@@ -182,6 +185,10 @@ export const ModelPostUpgradeRequest = z.strictObject({
profile: ProfileReference,
/** After an approved migration (docker.migrate): its summary and the hl steps its plan runs once the app is up. */
migration: z.strictObject({ summary: z.string().max(2000), stepsAfter: z.array(z.string().min(1).max(500)).max(30) }).optional(),
/** The app's setting: off = no model, when_needed = only with a reason (or errors in the logs). Older requests have none: always. */
when: z.enum(['off', 'when_needed', 'always']).optional(),
/** What in this run asks for a look: an update the owner approved, a major one, a migration. */
reasons: z.array(PostUpgradeReason).max(4).optional(),
});
export type ModelPostUpgradeRequest = z.infer<typeof ModelPostUpgradeRequest>;
@@ -322,7 +329,7 @@ const ModelActionReport = z.strictObject({
outcome: z.string().min(1).max(64),
summary: z.string().max(4000),
actions: z.array(z.string().max(500)).max(50),
model: z.enum(['sonnet', 'opus']),
model: ModelName,
costUsd: z.number().nonnegative(),
});
@@ -354,6 +361,10 @@ export const ModelPostUpgradeResult = z.strictObject({
manifestChanged: z.boolean(),
manifestSha256: Sha256,
report: ModelActionReport.optional(),
/** Why the model ran; absent on results from before 2026-10-07. */
reasons: z.array(PostUpgradeReason).max(4).optional(),
/** The model did not run: the app has it off, or nothing in the run or the logs asked for it. */
skipped: z.enum(['off', 'not_needed']).optional(),
}).strict();
export type ModelPostUpgradeResult = z.infer<typeof ModelPostUpgradeResult>;
+16 -1
View File
@@ -3,10 +3,24 @@ import { z } from 'zod';
// Shared vocabulary: model executors, auto levels, reason codes, JSON and an app's configuration. The zod schema is the
// single source of the type. (The file is named after the retired step queue it started with.)
export const Executor = z.enum(['script', 'sonnet', 'opus']);
/** The Claude models a model step runs on; the profile of the same name picks the sidecar (Models page). */
export const ModelName = z.enum(['haiku', 'sonnet', 'opus']);
export type ModelName = z.infer<typeof ModelName>;
/** The model of a model step whose app chose none (owner, 2026-10-07: haiku is cheap and good enough). */
export const DEFAULT_MODEL: ModelName = 'haiku';
/** The app's model choice, or the default when it names none (null, `script`, anything else). */
export const modelOr = (choice: unknown): ModelName => {
const parsed = ModelName.safeParse(choice);
return parsed.success ? parsed.data : DEFAULT_MODEL;
};
export const Executor = z.enum(['script', ...ModelName.options]);
export type Executor = z.infer<typeof Executor>;
export const AutoLevel = z.enum(['none', 'patch', 'minor', 'major']);
/** When an app's post-upgrade check runs its model: only when the update may ask for something, or after every update. */
export const PostUpgradeWhen = z.enum(['when_needed', 'always']);
export type PostUpgradeWhen = z.infer<typeof PostUpgradeWhen>;
export type AutoLevel = z.infer<typeof AutoLevel>;
/** Why an operation stopped or failed (app/ops); unknown cause = `unexpected`. */
@@ -59,6 +73,7 @@ export interface AppConfig {
readonly promptHint: string | null;
/** null = after an update only scripts check it; a model here checks what the update asks to be done afterwards. */
readonly postUpgradeExecutor: Executor | null;
readonly postUpgradeWhen: PostUpgradeWhen;
readonly notes: string | null;
/** Real updates through the executor (backup, apply, verify). Off = only dry runs. */
readonly apply: boolean;
+1 -1
View File
@@ -21,7 +21,7 @@ export const PRIVILEGED_ACTION_DESCRIPTIONS: Readonly<Record<PrivilegedActionId,
'docker.diskUsage': { title: 'Check the disk', kind: 'read', summary: 'Executor reads the free space of the Docker filesystem and what images and build cache could give back.' },
'docker.imagePrune': { title: 'Remove unused images', kind: 'change', summary: 'Executor removes Docker images no container uses (and the build cache when asked); volumes are never touched.' },
'model.repair': { title: 'Model repairs the app', kind: 'model', summary: 'A model reads logs and the compose file and may change them to make the app healthy again.' },
'model.postUpgrade': { title: 'Model checks the update', kind: 'model', summary: 'A model checks the updated app (hl ps, hl logs), may restart it, and asks the owner if something needs a decision.' },
'model.postUpgrade': { title: 'Model checks the update', kind: 'model', summary: 'A model checks the updated app (hl ps, hl logs), may restart it, and asks the owner if something needs a decision. Off per app; by default only for an approved or major update, a migration, or errors in the logs.' },
};
export const COMPONENT_OPERATION_DESCRIPTIONS: Readonly<Record<ComponentOperationId, BlockDescription>> = {
+14
View File
@@ -27,4 +27,18 @@ describe('workflowModelCalls', () => {
expect(calls[2]).toMatchObject({ instanceId: 'run-1', workflowId: 'update-radarr', summary: 'Clean start.', actions: ['hl ps'], at: '2026-09-22T21:35:00.000Z' });
expect(totalCostUsd(calls)).toBeCloseTo(0.82);
});
it('counts a model run once when a later node carries its result on, as the terminal node does', () => {
const postUpgrade = { action: 'model.postUpgrade', state: 'completed', report: { outcome: 'nothing_to_do', summary: 'Clean start.', actions: ['hl ps'], model: 'sonnet', costUsd: 0.2 } };
const notes = { verdict: 'attention', summary: 'A migration runs.', model: 'opus', costUsd: 0.12 };
const calls = workflowModelCalls({ id: 'run-1', workflowId: 'update-radarr' }, [
attempt(1, 'release_notes', notes),
attempt(2, 'gate', notes),
attempt(3, 'post_upgrade', postUpgrade),
attempt(4, 'terminal_updated', postUpgrade),
]);
expect(calls.map((call) => call.nodeId)).toEqual(['release_notes', 'post_upgrade']);
expect(totalCostUsd(calls)).toBeCloseTo(0.32);
});
});
+32 -23
View File
@@ -22,34 +22,43 @@ const cost = (value: unknown): number => (typeof value === 'number' && Number.is
/**
* The model runs a workflow run made, read from its node attempts: a model action's result carries `report`
* (app/privileged-action-executor-adapter), the release notes operation's result carries `verdict` and `model`.
* A node that passes the record on (a terminal node's result is its input) repeats that result; the same call counts once.
*/
export function workflowModelCalls(instance: { readonly id: string; readonly workflowId: string }, attempts: readonly NodeAttempt[]): WorkflowModelCall[] {
return attempts.flatMap((attempt): WorkflowModelCall[] => {
const result = record(attempt.result);
const at = attempt.finishedAt ?? attempt.startedAt;
const base = { instanceId: instance.id, workflowId: instance.workflowId, nodeId: attempt.nodeId, at };
const action = text(result['action']);
if (action === 'model.postUpgrade' || action === 'model.repair') {
const report = record(result['report']);
if (Object.keys(report).length === 0) return [];
return [{
...base, kind: action === 'model.repair' ? 'repair' : 'postUpgrade', model: text(report['model']),
outcome: text(report['outcome']) ?? 'unknown', summary: text(report['summary']) ?? '',
actions: Array.isArray(report['actions']) ? report['actions'].filter((item): item is string => typeof item === 'string') : [],
costUsd: cost(report['costUsd']),
}];
}
// Only a release notes result that reached a model names one; "no source" and unreadable notes cost nothing.
if ('verdict' in result && text(result['model']) !== null) {
return [{
...base, kind: 'releaseNotes', model: text(result['model']), outcome: text(result['verdict']) ?? 'unknown',
summary: text(result['summary']) ?? '', actions: [], costUsd: cost(result['costUsd']),
}];
}
return [];
const seen = new Set<string>();
return attempts.flatMap((attempt) => modelCallOf(instance, attempt)).filter(({ nodeId: _nodeId, at: _at, ...call }) => {
const key = JSON.stringify(call);
if (seen.has(key)) return false;
seen.add(key);
return true;
});
}
function modelCallOf(instance: { readonly id: string; readonly workflowId: string }, attempt: NodeAttempt): WorkflowModelCall[] {
const result = record(attempt.result);
const at = attempt.finishedAt ?? attempt.startedAt;
const base = { instanceId: instance.id, workflowId: instance.workflowId, nodeId: attempt.nodeId, at };
const action = text(result['action']);
if (action === 'model.postUpgrade' || action === 'model.repair') {
const report = record(result['report']);
if (Object.keys(report).length === 0) return [];
return [{
...base, kind: action === 'model.repair' ? 'repair' : 'postUpgrade', model: text(report['model']),
outcome: text(report['outcome']) ?? 'unknown', summary: text(report['summary']) ?? '',
actions: Array.isArray(report['actions']) ? report['actions'].filter((item): item is string => typeof item === 'string') : [],
costUsd: cost(report['costUsd']),
}];
}
// Only a release notes result that reached a model names one; "no source" and unreadable notes cost nothing.
if ('verdict' in result && text(result['model']) !== null) {
return [{
...base, kind: 'releaseNotes', model: text(result['model']), outcome: text(result['verdict']) ?? 'unknown',
summary: text(result['summary']) ?? '', actions: [], costUsd: cost(result['costUsd']),
}];
}
return [];
}
export function totalCostUsd(calls: readonly WorkflowModelCall[]): number {
return calls.reduce((sum, call) => sum + call.costUsd, 0);
}
@@ -7,6 +7,7 @@ import {
PrivilegedActionVersion,
} from './privileged-action.ts';
import type { PrivilegedActionPolicy as PrivilegedActionPolicyType, PrivilegedActionResult as ActionResult } from './privileged-action.ts';
import type { PostUpgradeReason } from './post-upgrade.ts';
import { privilegedActionIdentityMatches } from './privileged-action-invocation.ts';
import type { JsonValue } from './steps.ts';
import { bindWorkflowResult, bindingInput, DataBindingV1 } from './workflow-data-binding.ts';
@@ -83,6 +84,15 @@ export function resultEvent(result: ActionResult): string {
export function createPrivilegedActionExtensionRegistry(policy?: unknown): WorkflowExtensionRegistry {
const parsedPolicy = policy === undefined ? null : PrivilegedActionPolicy.safeParse(policy);
const policyValue = parsedPolicy === null ? undefined : parsedPolicy.success ? parsedPolicy.data : null;
/** What in the run asks for a post-upgrade look: release notes are read only for an update above the app's auto level. */
function postUpgradeReasons(context: JsonValue, migrated: boolean): PostUpgradeReason[] {
const reasons: PostUpgradeReason[] = [];
if (isRecord(jsonPointer(context, '/releaseNotes'))) reasons.push('approved_update');
if (jsonPointer(context, '/imageSelection/change') === 'major') reasons.push('major');
if (migrated) reasons.push('migration');
return reasons;
}
function deriveMergedSelected(
parsedConfig: PrivilegedActionNodeConfig,
context: JsonValue,
@@ -135,7 +145,7 @@ function deriveMergedSelected(
const plan = jsonPointer(context, '/migrationPlan/plan');
const digest = jsonPointer(context, '/migrationPlan/planDigest');
if (isRecord(plan)) {
if (!('composeYaml' in result) && typeof plan.composeYaml === 'string') result['composeYaml'] = plan.composeYaml;
if (!('composeYaml' in result) && (typeof plan.composeYaml === 'string' || plan.composeYaml === null)) result['composeYaml'] = plan.composeYaml;
if (!('inputs' in result) && Array.isArray(plan.inputs)) {
result['inputs'] = plan.inputs.filter(isRecord).map((input) => ({ name: input.name ?? null, target: input.target ?? null }));
}
@@ -151,6 +161,7 @@ function deriveMergedSelected(
result['migration'] = { summary: plan.summary, stepsAfter: plan.stepsAfter };
}
}
if (action === 'model.postUpgrade' && !('reasons' in result)) result['reasons'] = postUpgradeReasons(context, 'migration' in result);
// Profile & Reason for model actions
if (action === 'model.repair' || action === 'model.postUpgrade') {
+8 -1
View File
@@ -1,6 +1,8 @@
import { HostStatus } from '../domain/host-status.ts';
import { NotificationSettings } from '../domain/notification-settings.ts';
import { z } from 'zod';
import { SkillContent } from '../domain/skill.ts';
import { AutoLevel, Executor, Json } from '../domain/steps.ts';
import { AutoLevel, Executor, Json, PostUpgradeWhen } from '../domain/steps.ts';
import {
WorkflowDraft,
PublishedWorkflowVersion,
@@ -63,6 +65,7 @@ export const AppConfigResponse = z.object({
migrateExecutor: Executor.nullable(),
promptHint: z.string().nullable(),
postUpgradeExecutor: Executor.nullable(),
postUpgradeWhen: PostUpgradeWhen,
notes: z.string().nullable(),
apply: z.boolean(),
});
@@ -383,3 +386,7 @@ export type ModelProfilePublishRequest = z.infer<typeof ModelProfilePublishReque
/** GET/PUT /api/release-notes-sources: the owner's trusted release notes sources for all apps. */
export const ReleaseNotesSourcesResponse = z.object({ sources: z.array(z.string()) });
/** GET /api/host-status: the driver's latest look at VM103, null before its first. */
export const HostStatusResponse = z.object({ status: HostStatus.nullable() });
/** GET/PUT /api/notification-settings: quiet hours and the morning report, in the schedules' time zone. */
export const NotificationSettingsResponse = z.object({ settings: NotificationSettings, timeZone: z.string(), lastDigestAt: z.string().nullable() });
+21 -1
View File
@@ -24,7 +24,7 @@ import type {
WorkflowRunInputStore,
ModelActivityStore,
WorkflowScheduleStore,
ReleaseNotesSourcesStore,
HostStatusStore, NotificationSettingsStore, ReleaseNotesSourcesStore,
} from '../domain/ports.ts';
import { compileTagPattern, tagShapePattern } from '../domain/tags.ts';
import { buildAppOverview } from '../domain/overview.ts';
@@ -71,6 +71,7 @@ import {
import { buildWorkflowTimeline, decisionContext } from '../domain/workflow-timeline.ts';
import { saveSchedule, scheduleNextRuns } from '../app/workflow-schedules.ts';
import type { MigrationDecisionContext } from '../domain/workflow-timeline.ts';
import { NotificationSettings } from '../domain/notification-settings.ts';
import { totalCostUsd, workflowModelCalls } from '../domain/workflow-model-calls.ts';
import { createWorkflowExtensions } from '../domain/workflow-extensions.ts';
@@ -99,6 +100,10 @@ export interface ServerDeps {
readonly modelProfiles: ModelProfileStore;
/** The owner's trusted release notes sources for all apps. */
readonly releaseNotesSources: ReleaseNotesSourcesStore;
/** The driver's latest look at VM103's containers and backup; unset: the overview shows none. */
readonly hostStatus?: Pick<HostStatusStore, 'get'>;
/** Quiet hours and the morning report; the driver reads them. */
readonly notificationSettings: NotificationSettingsStore;
/** What the owner typed into a migration form. */
readonly workflowRunInputs: WorkflowRunInputStore;
/** The models' executor session logs per node attempt (written by the driver); unset: the run page shows none. */
@@ -913,6 +918,21 @@ export async function buildServer(deps: ServerDeps): Promise<FastifyInstance> {
});
app.get('/api/release-notes-sources', (_request, reply) => reply.send({ sources: deps.releaseNotesSources.get() }));
app.get('/api/host-status', (_request, reply) => reply.send({ status: deps.hostStatus?.get() ?? null }));
const notificationSettings = () => ({
settings: deps.notificationSettings.get(),
timeZone: deps.schedules.timeZone,
lastDigestAt: deps.notificationSettings.lastDigest()?.at ?? null,
});
app.get('/api/notification-settings', (_request, reply) => reply.send(notificationSettings()));
app.put('/api/notification-settings', (request, reply) => {
const body = NotificationSettings.strict().safeParse(request.body);
if (!body.success) return fail(reply, 400, 'invalid_request', issues(body.error));
deps.notificationSettings.put(body.data, clock.now().toISOString());
hub.notify();
return reply.send(notificationSettings());
});
app.put('/api/release-notes-sources', (request, reply) => {
const body = z.strictObject({ sources: ReleaseNotesSources }).safeParse(request.body);
+13
View File
@@ -95,6 +95,19 @@ describe('HTTP executor client failures', () => {
expect(seen).toEqual([{ url: 'http://executor:8081/agent/compose', authorization: `Bearer ${'a'.repeat(32)}` }]);
});
it('reads a service’s logs with the session token, like the model’s hl logs', async () => {
const seen: { url: string; authorization: string | null; body: unknown }[] = [];
const client = createHttpExecutorClient({
baseUrl: 'http://executor:8081',
fetch: (input, init) => {
seen.push({ url: input instanceof Request ? input.url : input.toString(), authorization: new Headers(init?.headers).get('authorization'), body: JSON.parse(init?.body as string) });
return answering({ outcome: 'done', result: { text: 'started\n' } })();
},
});
expect(await client.sessionLogs({ token: 'a'.repeat(32), service: 'sonarr', tail: 200 })).toEqual({ outcome: 'done', result: { text: 'started\n' } });
expect(seen).toEqual([{ url: 'http://executor:8081/agent/logs', authorization: `Bearer ${'a'.repeat(32)}`, body: { service: 'sonarr', tail: 200 } }]);
});
it('an unreachable executor is unavailable, naming where it looked', async () => {
const client = createHttpExecutorClient({ baseUrl: 'http://executor:8081', fetch: () => Promise.reject(new TypeError('fetch failed', { cause: { code: 'ECONNREFUSED' } })) });
expect(await client.inspect({ project: 'jackett', service: 'jackett' })).toEqual({ outcome: 'unavailable', detail: 'executor unreachable at http://executor:8081/inspect: fetch failed (ECONNREFUSED)' });
+1
View File
@@ -74,5 +74,6 @@ export function createHttpExecutorClient(options: HttpExecutorClientOptions): Ex
closeSession: (request, signal) => post('/session/close', request, SessionCloseResponse, signal),
sessionLog: (request, signal) => post('/session/log', request, SessionLogResponse, signal),
sessionCompose: ({ token }, signal) => post('/agent/compose', {}, AgentTextResponse, signal, token),
sessionLogs: ({ token, ...request }, signal) => post('/agent/logs', request, AgentTextResponse, signal, token),
};
}
+41
View File
@@ -137,6 +137,47 @@ describe('createHttpRegistryClient', () => {
});
});
describe('HTTP 429', () => {
const tooMany = (headers: Record<string, string> = {}) => new Response('', { status: 429, headers });
it('waits and asks again, so a short rate limit does not fail the lookup', async () => {
const waits: number[] = [];
const { fetchFn, calls } = fakeRegistry({ [HUB]: [tooMany(), tooMany(), json({ tags: ['1'] })] });
const client = createHttpRegistryClient({ fetch: fetchFn, rateLimitWaitsMs: [5_000, 30_000], sleep: (ms) => (waits.push(ms), Promise.resolve()) });
expect(await client.listTags('nextcloud')).toEqual(['1']);
expect(waits).toEqual([5_000, 30_000]);
expect(calls).toHaveLength(3);
});
it('waits as long as Retry-After asks when that is longer, up to five minutes', async () => {
const waits: number[] = [];
const { fetchFn } = fakeRegistry({ [HUB]: [tooMany({ 'retry-after': '20' }), tooMany({ 'retry-after': '3600' }), json({ tags: [] })] });
await createHttpRegistryClient({ fetch: fetchFn, rateLimitWaitsMs: [5_000, 30_000], sleep: (ms) => (waits.push(ms), Promise.resolve()) }).listTags('nextcloud');
expect(waits).toEqual([20_000, 300_000]);
});
it('does not ask again unless pauses are given', async () => {
const { fetchFn, calls } = fakeRegistry({ [HUB]: tooMany() });
await createHttpRegistryClient({ fetch: fetchFn }).listTags('nextcloud').catch(() => undefined);
expect(calls).toHaveLength(1);
});
it('gives up as rate_limited once the pauses are used', async () => {
const { fetchFn, calls } = fakeRegistry({ [HUB]: tooMany() });
const error = await createHttpRegistryClient({ fetch: fetchFn, rateLimitWaitsMs: [1] }).listTags('nextcloud').catch((cause: unknown) => cause);
expect((error as RegistryError).kind).toBe('rate_limited');
expect(calls).toHaveLength(2);
});
it('an abort during the pause is rethrown as it is', async () => {
const controller = new AbortController();
const { fetchFn } = fakeRegistry({ [HUB]: tooMany() });
const running = createHttpRegistryClient({ fetch: fetchFn, rateLimitWaitsMs: [5_000] }).listTags('nextcloud', controller.signal);
setTimeout(() => controller.abort(new Error('step time limit')), 20);
await expect(running).rejects.toThrow('step time limit');
});
});
describe('time', () => {
const hanging = (_input: string | URL | Request, init?: RequestInit) =>
new Promise<Response>((_resolve, reject) => init?.signal?.addEventListener('abort', () => reject(init.signal?.reason as Error)));
+26 -1
View File
@@ -1,4 +1,5 @@
import { createHash } from 'node:crypto';
import { setTimeout as delay } from 'node:timers/promises';
import { z } from 'zod';
import { findPlatformEntry, parseImageRef, platformNames } from '../domain/image-ref.ts';
import type { IndexEntry } from '../domain/image-ref.ts';
@@ -14,11 +15,16 @@ export interface HttpRegistryOptions {
readonly maxPages?: number;
/** Largest response body we read (tag pages, manifests, image configs). */
readonly maxBodyBytes?: number;
/** Pauses before asking again after HTTP 429, one retry per entry; none by default. ghcr.io answers 429 now and then and is fine later. */
readonly rateLimitWaitsMs?: readonly number[];
readonly sleep?: (ms: number, signal: AbortSignal | undefined) => Promise<void>;
}
const DEFAULT_TIMEOUT_MS = 30_000;
const DEFAULT_MAX_PAGES = 50;
const DEFAULT_MAX_BODY_BYTES = 4 * 1024 * 1024;
/** A longer `Retry-After` than this is not waited out. */
const MAX_RATE_LIMIT_WAIT_MS = 300_000;
const MANIFEST_ACCEPT = [
'application/vnd.oci.image.index.v1+json',
@@ -58,6 +64,8 @@ export function createHttpRegistryClient(options: HttpRegistryOptions = {}): Reg
const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const maxPages = options.maxPages ?? DEFAULT_MAX_PAGES;
const maxBodyBytes = options.maxBodyBytes ?? DEFAULT_MAX_BODY_BYTES;
const rateLimitWaitsMs = options.rateLimitWaitsMs ?? [];
const sleep = options.sleep ?? ((ms: number, signal: AbortSignal | undefined) => delay(ms, undefined, signal === undefined ? {} : { signal }));
/** One repository on one registry: the token is fetched once and reused by every request. */
function openSession(location: RepoLocation, signal: AbortSignal | undefined) {
@@ -65,8 +73,25 @@ export function createHttpRegistryClient(options: HttpRegistryOptions = {}): Reg
const what = `${location.registry}/${location.path}`;
let token: string | null = null;
/** One request with the caller's abort and our own time limit. A caller abort is rethrown untouched. */
/** `request`, asked again after HTTP 429 as long as `rateLimitWaitsMs` has pauses left; the last 429 is returned. */
async function request(url: string, authorization: string | null, accept?: string): Promise<Response> {
for (let attempt = 0; ; attempt++) {
const response = await requestOnce(url, authorization, accept);
const wait = rateLimitWaitsMs[attempt];
if (response.status !== 429 || wait === undefined) return response;
const retryAfterMs = Number(response.headers.get('retry-after')) * 1000;
const pause = Number.isFinite(retryAfterMs) && retryAfterMs > wait ? Math.min(retryAfterMs, MAX_RATE_LIMIT_WAIT_MS) : wait;
void response.body?.cancel().catch(() => undefined);
try {
await sleep(pause, signal);
} catch (cause) {
throw signal?.aborted === true ? (signal.reason as Error) : cause;
}
}
}
/** One request with the caller's abort and our own time limit. A caller abort is rethrown untouched. */
async function requestOnce(url: string, authorization: string | null, accept?: string): Promise<Response> {
const timeout = AbortSignal.timeout(timeoutMs);
const combined = signal === undefined ? timeout : AbortSignal.any([signal, timeout]);
const headers: Record<string, string> = {};
+1 -1
View File
@@ -16,7 +16,7 @@ export interface ModelDispatcherOptions {
/**
* Sends each model request to the sidecar of its profile's provider (Claude, Codex). The profile is the pinned
* `profileRef` of a run's snapshot, or else the latest profile named like the request's executor (`sonnet`, `opus`: the
* `profileRef` of a run's snapshot, or else the latest profile named like the request's executor (`haiku`, `sonnet`, `opus`: the
* app's configuration names the model by profile id). There is no default provider and no fallback: an unknown
* profile or a provider without a sidecar is an error, and one provider's failure never moves the request to another.
* The profile's timeout and cost cap apply to the run.
@@ -82,6 +82,58 @@ describe('executor-backed privileged action adapter', () => {
}
});
describe('when the post-upgrade model runs', () => {
const nothingToDo = () => new FakeModelExecutor().reply({ output: { outcome: 'nothing_to_do', summary: 'Nothing to do.', actions: [] }, costUsd: 0.2 });
const withLogs = (text: string) => session().on('sessionLogs', { outcome: 'done', result: { text } });
it('off: no session and no model, the hash comes from the service', async () => {
const client = new FakeExecutorClient()
.on('inspect', { outcome: 'done', result: { project: 'jackett', service: 'jackett', manifest_sha256: 'd'.repeat(64), env_sha256: null, manifest_image: PINNED, container: null } });
const model = nothingToDo();
const result = await createExecutorBackedPrivilegedActionExecutor({ executor: client, modelExecutor: model, prompts: prompts() }).execute({ ...postUpgrade, when: 'off', reasons: ['major'] });
expect(result).toEqual({ ...target, action: 'model.postUpgrade', version: 1, state: 'completed', manifestChanged: false, manifestSha256: 'd'.repeat(64), reasons: ['major'], skipped: 'off' });
expect(model.requests).toEqual([]);
expect(client.calls.map((call) => call.op)).toEqual(['inspect']);
});
it('when needed, with nothing in the run asking and clean logs: reads the logs and stops there', async () => {
const client = withLogs('[migrations] no migrations found\nStarted.\n');
const model = nothingToDo();
const result = await createExecutorBackedPrivilegedActionExecutor({ executor: client, modelExecutor: model, prompts: prompts() }).execute({ ...postUpgrade, when: 'when_needed', reasons: [] });
expect(result).toMatchObject({ state: 'completed', manifestChanged: false, manifestSha256: 'c'.repeat(64), reasons: [], skipped: 'not_needed' });
expect(result).not.toHaveProperty('report');
expect(model.requests).toEqual([]);
expect(client.calls.map((call) => [call.op, call.request])).toEqual([
['openSession', { project: 'jackett', service: 'jackett', mode: 'observe' }],
['sessionLogs', { token: 'tok', service: 'jackett', tail: 200 }],
['closeSession', { token: 'tok' }],
]);
});
it('when needed, errors in the logs (or logs it cannot read) bring the model in', async () => {
for (const client of [withLogs('Started.\nERROR Unable to open database\n'), session().on('sessionLogs', { outcome: 'failed', detail: 'no container', manifest_replaced: false })]) {
const model = nothingToDo();
const result = await createExecutorBackedPrivilegedActionExecutor({ executor: client, modelExecutor: model, prompts: prompts() }).execute({ ...postUpgrade, when: 'when_needed', reasons: [] });
expect(result).toMatchObject({ state: 'completed', reasons: ['log_errors'], report: { outcome: 'nothing_to_do', costUsd: 0.2 } });
expect(result).not.toHaveProperty('skipped');
expect(model.requests).toHaveLength(1);
}
});
it('when needed, a reason from the run brings the model in without reading the logs first', async () => {
const client = session();
const model = nothingToDo();
const result = await createExecutorBackedPrivilegedActionExecutor({ executor: client, modelExecutor: model, prompts: prompts() }).execute({ ...postUpgrade, when: 'when_needed', reasons: ['approved_update', 'major'] });
expect(result).toMatchObject({ state: 'completed', reasons: ['approved_update', 'major'], report: { outcome: 'nothing_to_do' } });
expect(model.requests).toHaveLength(1);
expect(client.calls.map((call) => call.op)).not.toContain('sessionLogs');
});
});
it('takes the manifest hash from the service when the session gave no summary', async () => {
const client = session()
.on('closeSession', { outcome: 'failed', detail: 'gone', manifest_replaced: false })
@@ -169,6 +221,18 @@ describe('executor-backed privileged action adapter', () => {
expect(store.stored).toEqual({});
});
it('sends no compose file when the plan keeps the current one', async () => {
const client = new FakeExecutorClient().on('migrate', {
outcome: 'done',
result: { project: 'grafana', service: 'influxdb', state: 'applied', image: PINNED, manifest_sha256: 'a'.repeat(64), applied_manifest_sha256: 'b'.repeat(64), env_keys: [] },
});
const adapter = createExecutorBackedPrivilegedActionExecutor({ executor: client, runInputs: inputs({}) });
await adapter.execute({ ...request, composeYaml: null, inputs: [] }, undefined, { instanceId: 'run-1', nodeAttemptOrdinal: 4 });
expect(client.calls[0]?.request).toMatchObject({ compose_yaml: null, env: {} });
});
it('refuses to run with a value missing, and keeps the values when the executor refused', async () => {
const client = new FakeExecutorClient().on('migrate', { outcome: 'refused', reason: 'invalid_manifest', detail: 'bad' });
await expect(createExecutorBackedPrivilegedActionExecutor({ executor: client, runInputs: inputs({ ADMIN_USER: 'admin' }) }).execute(request, undefined, { instanceId: 'run-1', nodeAttemptOrdinal: 4 }))
@@ -6,7 +6,8 @@ import type { SessionSummary } from '../domain/executor-api.ts';
import { watchModelSession } from '../app/model-activity.ts';
import type { ModelActivitySink } from '../app/model-activity.ts';
import { REPAIR_ANSWER_SCHEMA, RepairAnswer } from '../domain/repair.ts';
import { POST_UPGRADE_ANSWER_SCHEMA, PostUpgradeAnswer } from '../domain/post-upgrade.ts';
import { POST_UPGRADE_ANSWER_SCHEMA, PostUpgradeAnswer, suspiciousLogLines } from '../domain/post-upgrade.ts';
import type { PostUpgradeReason } from '../domain/post-upgrade.ts';
/** Drops closing tags the model sometimes leaves at the end of its summary (`…</summary>\n</invoke>`). */
function cleanSummary(summary: string): string {
@@ -15,6 +16,8 @@ function cleanSummary(summary: string): string {
/** The session log operations a repair result may list (ModelRepairResult.actions). */
const SESSION_OPERATIONS: ReadonlySet<string> = new Set(['ps', 'logs', 'compose', 'write', 'pull', 'up', 'restart', 'exec-rules', 'exec', 'run']);
/** How many of the service's last log lines a post-upgrade check scans before it decides whether a model looks. */
const POST_UPGRADE_LOG_LINES = 200;
/** The closed driver-side mapping. It owns no command construction and only the executor client can reach Docker. */
export function createExecutorBackedPrivilegedActionExecutor(deps: {
@@ -84,6 +87,31 @@ export function createExecutorBackedPrivilegedActionExecutor(deps: {
return { output, costUsd, failure, model, manifestChanged: summary?.manifest_changed ?? false, manifestSha256, actions };
}
/**
* Whether a post-upgrade check needs its model, and why. When needed only: a reason from the run, else the service's
* last log lines read in an observe session (no model); errors there, or logs it cannot read, bring the model in.
*/
async function postUpgradeNeed(
request: Extract<PrivilegedActionRequest, { readonly action: 'model.postUpgrade' }>,
signal?: AbortSignal,
): Promise<{ readonly run: true; readonly reasons: readonly PostUpgradeReason[] } | { readonly run: false; readonly skipped: 'off' | 'not_needed'; readonly manifestSha256: string | null }> {
const reasons = request.reasons ?? [];
if (request.when === 'off') return { run: false, skipped: 'off', manifestSha256: null };
if (request.when !== 'when_needed' || reasons.length > 0) return { run: true, reasons };
const session = completed(request.action, await deps.executor.openSession({ project: request.project, service: request.service, mode: 'observe' }, signal));
const close = () => deps.executor.closeSession({ token: session.token }, signal);
let logs: ExecutorCall<{ readonly text: string }>;
try {
logs = await deps.executor.sessionLogs({ token: session.token, service: request.service, tail: POST_UPGRADE_LOG_LINES }, signal);
} catch (cause) {
await close().catch(() => undefined);
throw cause;
}
const closed = await close();
if (logs.outcome !== 'done' || suspiciousLogLines(logs.result.text).length > 0) return { run: true, reasons: ['log_errors'] };
return { run: false, skipped: 'not_needed', manifestSha256: closed.outcome === 'done' ? closed.result.manifest_sha256 : null };
}
return {
async execute(request, signal, run) {
const parsed = PrivilegedActionRequest.parse(request);
@@ -141,9 +169,10 @@ export function createExecutorBackedPrivilegedActionExecutor(deps: {
const values = deps.runInputs.values(run.instanceId);
const missing = parsed.inputs.filter((input) => values[input.name] === undefined).map((input) => input.name);
if (missing.length > 0) throw new Error(`docker.migrate is missing the owner's values for ${missing.join(', ')}`);
const composeYaml = parsed.inputs
const plannedCompose = parsed.composeYaml;
const composeYaml = plannedCompose === null ? null : parsed.inputs
.filter((input) => input.target === 'compose')
.reduce((text, input) => text.replaceAll(`<input:${input.name}>`, values[input.name] ?? ''), parsed.composeYaml);
.reduce((text, input) => text.replaceAll(`<input:${input.name}>`, values[input.name] ?? ''), plannedCompose);
const env = Object.fromEntries(parsed.inputs.filter((input) => input.target === 'env').map((input) => [input.name, values[input.name] ?? '']));
const answer = await deps.executor.migrate({
project: parsed.project, service: parsed.service, image: parsed.image, expect_manifest_sha256: parsed.expectedManifestSha256,
@@ -195,6 +224,17 @@ export function createExecutorBackedPrivilegedActionExecutor(deps: {
break;
}
case 'model.postUpgrade': {
const need = await postUpgradeNeed(parsed, signal);
if (!need.run) {
const manifestSha256 = need.manifestSha256
?? completed(parsed.action, await deps.executor.inspect({ project: parsed.project, service: parsed.service }, signal)).manifest_sha256;
result = {
action: parsed.action, version: 1, component: parsed.component, project: parsed.project, service: parsed.service,
image: parsed.image, backupId: parsed.backupId, state: 'completed', manifestChanged: false, manifestSha256,
reasons: parsed.reasons ?? [], skipped: need.skipped,
};
break;
}
const asked = await ask(parsed, 'postUpgrade', signal, run);
const answer = PostUpgradeAnswer.safeParse(asked.output);
const report = answer.success
@@ -208,6 +248,7 @@ export function createExecutorBackedPrivilegedActionExecutor(deps: {
state: done ? 'completed' : 'action_required',
manifestChanged: asked.manifestChanged, manifestSha256: asked.manifestSha256,
report: { ...report, model: asked.model, costUsd: asked.costUsd },
reasons: need.reasons,
};
break;
}
+4 -3
View File
@@ -21,16 +21,16 @@ export function createSqliteAppConfigStore(db: DatabaseSync): VersionedAppConfig
const previousConfig = previous === undefined ? null : parseAppConfig(previous);
db.prepare(
`INSERT INTO app_config (app, enabled, position, service, image_repo, tag_pattern, auto, http_url, expect_status,
release_notes_executor, migrate_executor, prompt_hint, post_upgrade_executor, notes, apply, project)
release_notes_executor, migrate_executor, prompt_hint, post_upgrade_executor, post_upgrade_when, notes, apply, project)
VALUES (:app, :enabled, :position, :service, :imageRepo, :tagPattern, :auto, :httpUrl, :expectStatus,
:releaseNotesExecutor, :migrateExecutor, :promptHint, :postUpgradeExecutor, :notes, :apply, :project)
:releaseNotesExecutor, :migrateExecutor, :promptHint, :postUpgradeExecutor, :postUpgradeWhen, :notes, :apply, :project)
ON CONFLICT (app) DO UPDATE SET
enabled = excluded.enabled, position = excluded.position, service = excluded.service,
image_repo = excluded.image_repo, tag_pattern = excluded.tag_pattern, auto = excluded.auto,
http_url = excluded.http_url, expect_status = excluded.expect_status,
release_notes_executor = excluded.release_notes_executor, migrate_executor = excluded.migrate_executor,
prompt_hint = excluded.prompt_hint,
post_upgrade_executor = excluded.post_upgrade_executor, notes = excluded.notes, apply = excluded.apply, project = excluded.project`,
post_upgrade_executor = excluded.post_upgrade_executor, post_upgrade_when = excluded.post_upgrade_when, notes = excluded.notes, apply = excluded.apply, project = excluded.project`,
).run({
app: config.app,
enabled: config.enabled ? 1 : 0,
@@ -45,6 +45,7 @@ export function createSqliteAppConfigStore(db: DatabaseSync): VersionedAppConfig
migrateExecutor: config.migrateExecutor,
promptHint: config.promptHint,
postUpgradeExecutor: config.postUpgradeExecutor,
postUpgradeWhen: config.postUpgradeWhen,
notes: config.notes,
apply: config.apply ? 1 : 0,
project: config.project,
@@ -0,0 +1,15 @@
import { describe, expect, it } from 'vitest';
import { openDatabase } from './open.ts';
import { createSqliteHostStatusStore } from './host-status-store.ts';
describe('host status store', () => {
it('keeps only the latest look', () => {
const store = createSqliteHostStatusStore(openDatabase(':memory:'));
expect(store.get()).toBeNull();
const first = { checkedAt: '2026-09-30T06:00:00.000Z', apps: [], backup: null, error: 'docker ps failed' };
const second = { checkedAt: '2026-09-30T06:01:00.000Z', apps: [{ app: 'a', enabled: true, state: 'running', health: null }], backup: { ok: false as const, detail: 'none' }, error: null };
store.put(first);
store.put(second);
expect(store.get()).toEqual(second);
});
});
+20
View File
@@ -0,0 +1,20 @@
import type { DatabaseSync } from 'node:sqlite';
import { HostStatus } from '../../domain/host-status.ts';
import type { HostStatusStore } from '../../domain/ports.ts';
const KEY = 'hostStatus';
export function createSqliteHostStatusStore(db: DatabaseSync): HostStatusStore {
return {
get() {
const row = db.prepare('SELECT value FROM settings WHERE key = :key').get({ key: KEY });
// A value an older or newer driver wrote in another shape shows as "not looked yet" until the next look.
return row === undefined ? null : (HostStatus.safeParse(JSON.parse(String(row['value']))).data ?? null);
},
put(status) {
db.prepare(`INSERT INTO settings (key, value, updated_at) VALUES (:key, :value, :now)
ON CONFLICT (key) DO UPDATE SET value = excluded.value, updated_at = excluded.updated_at`)
.run({ key: KEY, value: JSON.stringify(HostStatus.parse(status)), now: status.checkedAt });
},
};
}
+14 -1
View File
@@ -20,7 +20,7 @@ describe('migrate', () => {
]);
expect(columns(db, 'app_config')).toEqual([
'app', 'enabled', 'position', 'service', 'image_repo', 'tag_pattern', 'auto', 'http_url', 'expect_status',
'release_notes_executor', 'migrate_executor', 'prompt_hint', 'notes', 'apply', 'project', 'post_upgrade_executor',
'release_notes_executor', 'migrate_executor', 'prompt_hint', 'notes', 'apply', 'project', 'post_upgrade_executor', 'post_upgrade_when',
]);
expect(columns(db, 'workflows')).toEqual(['id', 'name', 'created_at']);
expect(columns(db, 'workflow_drafts')).toEqual(['workflow_id', 'revision', 'document', 'validation_revision', 'validation_document', 'diagnostics']);
@@ -120,6 +120,19 @@ describe('migrate', () => {
expect(db.prepare('PRAGMA user_version').get()).toEqual({ user_version: 19 });
});
it('v26 accepts haiku as an app model and keeps every app config verbatim', () => {
const db = new DatabaseSync(':memory:');
migrate(db, MIGRATIONS.slice(0, 25));
db.exec(`INSERT INTO app_config (app, service, image_repo, release_notes_executor, migrate_executor, post_upgrade_executor, post_upgrade_when, notes, apply, project)
VALUES ('bazarr', 'bazarr', 'lscr.io/bazarr', 'opus', 'sonnet', NULL, 'always', 'n', 1, 'arr')`);
const before = db.prepare('SELECT * FROM app_config').all();
migrate(db, MIGRATIONS.slice(0, 26));
expect(db.prepare('SELECT * FROM app_config').all()).toEqual(before);
db.exec("INSERT INTO app_config (app, service, image_repo, migrate_executor, post_upgrade_executor) VALUES ('new', 'new', 'r', 'haiku', 'haiku')");
expect(db.prepare("SELECT release_notes_executor FROM app_config WHERE app = 'new'").get()).toEqual({ release_notes_executor: 'haiku' });
expect(() => db.exec("UPDATE app_config SET release_notes_executor = 'gpt' WHERE app = 'new'")).toThrow(/CHECK/);
});
it('v7 adds a nullable progress to steps already there', () => {
const db = new DatabaseSync(':memory:');
migrate(db, MIGRATIONS.slice(0, 6));
+40
View File
@@ -802,6 +802,46 @@ export const MIGRATIONS: readonly Migration[] = [
) STRICT;
`,
},
{
// When an app's post-upgrade check runs its model (owner, 2026-10-07): by default only when the update may ask for something.
version: 25,
sql: `
ALTER TABLE app_config ADD COLUMN post_upgrade_when TEXT NOT NULL DEFAULT 'when_needed' CHECK (post_upgrade_when IN ('when_needed', 'always'));
`,
},
{
// Haiku as a model choice and the default release notes model (owner, 2026-10-07). SQLite cannot change a CHECK in
// place, so the table is rebuilt; the retired steps table keeps its old one, nothing writes it any more.
version: 26,
sql: `
ALTER TABLE app_config RENAME TO app_config_v25;
CREATE TABLE app_config (
app TEXT PRIMARY KEY,
enabled INTEGER NOT NULL DEFAULT 1 CHECK (enabled IN (0, 1)),
position INTEGER NOT NULL DEFAULT 0,
service TEXT NOT NULL,
image_repo TEXT NOT NULL,
tag_pattern TEXT NOT NULL DEFAULT '.',
auto TEXT NOT NULL DEFAULT 'patch' CHECK (auto IN ('none', 'patch', 'minor', 'major')),
http_url TEXT,
expect_status INTEGER NOT NULL DEFAULT 200,
release_notes_executor TEXT NOT NULL DEFAULT 'haiku' CHECK (release_notes_executor IN ('script', 'haiku', 'sonnet', 'opus')),
migrate_executor TEXT CHECK (migrate_executor IN ('script', 'haiku', 'sonnet', 'opus')),
prompt_hint TEXT,
notes TEXT,
apply INTEGER NOT NULL DEFAULT 0 CHECK (apply IN (0, 1)),
project TEXT NOT NULL DEFAULT '',
post_upgrade_executor TEXT CHECK (post_upgrade_executor IN ('script', 'haiku', 'sonnet', 'opus')),
post_upgrade_when TEXT NOT NULL DEFAULT 'when_needed' CHECK (post_upgrade_when IN ('when_needed', 'always'))
) STRICT;
INSERT INTO app_config (app, enabled, position, service, image_repo, tag_pattern, auto, http_url, expect_status,
release_notes_executor, migrate_executor, prompt_hint, notes, apply, project, post_upgrade_executor, post_upgrade_when)
SELECT app, enabled, position, service, image_repo, tag_pattern, auto, http_url, expect_status,
release_notes_executor, migrate_executor, prompt_hint, notes, apply, project, post_upgrade_executor, post_upgrade_when
FROM app_config_v25;
DROP TABLE app_config_v25;
`,
},
];
function schemaVersion(db: DatabaseSync): number {
@@ -0,0 +1,17 @@
import { describe, expect, it } from 'vitest';
import { openDatabase } from './open.ts';
import { createSqliteNotificationSettingsStore } from './notification-settings-store.ts';
import { DEFAULT_NOTIFICATION_SETTINGS } from '../../domain/notification-settings.ts';
describe('notification settings store', () => {
it('starts from the defaults, keeps the owner’s settings and the last report', () => {
const store = createSqliteNotificationSettingsStore(openDatabase(':memory:'));
expect(store.get()).toEqual(DEFAULT_NOTIFICATION_SETTINGS);
expect(store.lastDigest()).toBeNull();
store.put({ ...DEFAULT_NOTIFICATION_SETTINGS, quietFrom: '00:30' }, '2026-09-30T08:00:00.000Z');
store.recordDigest('2026-09-30', '2026-09-30T05:30:00.000Z');
expect(store.get().quietFrom).toBe('00:30');
expect(store.lastDigest()).toEqual({ day: '2026-09-30', at: '2026-09-30T05:30:00.000Z' });
expect(() => store.put({ ...DEFAULT_NOTIFICATION_SETTINGS, digestAt: '7:30' }, '2026-09-30T08:00:00.000Z')).toThrow();
});
});
@@ -0,0 +1,27 @@
import type { DatabaseSync } from 'node:sqlite';
import { z } from 'zod';
import { DEFAULT_NOTIFICATION_SETTINGS, NotificationSettings } from '../../domain/notification-settings.ts';
import type { NotificationSettingsStore } from '../../domain/ports.ts';
const SETTINGS = 'notificationSettings';
const LAST_DIGEST = 'lastDigest';
const LastDigest = z.object({ day: z.string(), at: z.string() });
export function createSqliteNotificationSettingsStore(db: DatabaseSync): NotificationSettingsStore {
const read = (key: string): unknown => {
const row = db.prepare('SELECT value FROM settings WHERE key = :key').get({ key });
return row === undefined ? undefined : JSON.parse(String(row['value']));
};
const write = (key: string, value: unknown, now: string) => {
db.prepare(`INSERT INTO settings (key, value, updated_at) VALUES (:key, :value, :now)
ON CONFLICT (key) DO UPDATE SET value = excluded.value, updated_at = excluded.updated_at`)
.run({ key, value: JSON.stringify(value), now });
};
return {
// Until the owner saves their own, the defaults apply.
get: () => NotificationSettings.safeParse(read(SETTINGS)).data ?? DEFAULT_NOTIFICATION_SETTINGS,
put: (settings, now) => write(SETTINGS, NotificationSettings.parse(settings), now),
lastDigest: () => LastDigest.safeParse(read(LAST_DIGEST)).data ?? null,
recordDigest: (day, at) => write(LAST_DIGEST, { day, at }, at),
};
}
+3 -1
View File
@@ -1,5 +1,5 @@
import { z } from 'zod';
import { AutoLevel, Executor } from '../../domain/steps.ts';
import { AutoLevel, Executor, PostUpgradeWhen } from '../../domain/steps.ts';
import type { AppConfig, JsonValue } from '../../domain/steps.ts';
// Rows are validated where they leave the database, so a corrupted or foreign row fails loudly here
@@ -20,6 +20,7 @@ const AppConfigRow = z
migrate_executor: Executor.nullable(),
prompt_hint: z.string().nullable(),
post_upgrade_executor: Executor.nullable(),
post_upgrade_when: PostUpgradeWhen,
notes: z.string().nullable(),
apply: z.number(),
project: z.string().min(1),
@@ -40,6 +41,7 @@ const AppConfigRow = z
migrateExecutor: row.migrate_executor,
promptHint: row.prompt_hint,
postUpgradeExecutor: row.post_upgrade_executor,
postUpgradeWhen: row.post_upgrade_when,
notes: row.notes,
apply: row.apply === 1,
}),
@@ -119,7 +119,7 @@ async function setupEnvironment() {
releaseNotesExecutor: 'sonnet',
migrateExecutor: null,
promptHint: null,
postUpgradeExecutor: null,
postUpgradeExecutor: null, postUpgradeWhen: 'when_needed',
notes: null,
apply: true,
});
+1 -1
View File
@@ -8,7 +8,7 @@ const one = { app: 'bazarr', service: 'bazarr', imageRepo: 'linuxserver/bazarr',
describe('parseAppsFile', () => {
it('fills the defaults: enabled, models off, real updates off, positions from the file order', () => {
const [first, second] = parseAppsFile([one, { ...one, app: 'sonarr', service: 'sonarr', enabled: false, notes: 'x' }]);
expect(first).toEqual({ ...one, enabled: true, position: 1, releaseNotesExecutor: 'sonnet', migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, notes: null, apply: false, project: 'bazarr' });
expect(first).toEqual({ ...one, enabled: true, position: 1, releaseNotesExecutor: 'haiku', migrateExecutor: null, promptHint: null, postUpgradeExecutor: null, postUpgradeWhen: 'when_needed', notes: null, apply: false, project: 'bazarr' });
expect(second).toMatchObject({ app: 'sonarr', enabled: false, position: 2, notes: 'x' });
});
+3 -2
View File
@@ -1,6 +1,6 @@
import { z } from 'zod';
import { parseProjectName } from '../domain/compose.ts';
import { AutoLevel, Executor } from '../domain/steps.ts';
import { AutoLevel, DEFAULT_MODEL, Executor, PostUpgradeWhen } from '../domain/steps.ts';
import type { AppConfig } from '../domain/steps.ts';
import { compileTagPattern } from '../domain/tags.ts';
@@ -19,10 +19,11 @@ const AppEntry = z.strictObject({
auto: AutoLevel,
httpUrl: z.string().nullable().default(null),
expectStatus: z.number().int().min(100).max(599).default(200),
releaseNotesExecutor: Executor.default('sonnet'),
releaseNotesExecutor: Executor.default(DEFAULT_MODEL),
migrateExecutor: Executor.nullable().default(null),
promptHint: z.string().nullable().default(null),
postUpgradeExecutor: Executor.nullable().default(null),
postUpgradeWhen: PostUpgradeWhen.default('when_needed'),
notes: z.string().nullable().default(null),
/** Real updates stay off until the owner turns them on in the app. */
apply: z.boolean().default(false),
+41 -2
View File
@@ -14,6 +14,11 @@ import { openDatabase } from '../infra/sqlite/open.ts';
import { createSqliteAppConfigStore } from '../infra/sqlite/app-config-store.ts';
import { createSqliteSkillStore } from '../infra/sqlite/skill-store.ts';
import { createSqliteReleaseNotesSourcesStore } from '../infra/sqlite/release-notes-sources-store.ts';
import { createSqliteHostStatusStore } from '../infra/sqlite/host-status-store.ts';
import { runHostStatusLoop } from '../app/host-status.ts';
import { isQuiet, runDigestLoop } from '../app/digest.ts';
import { downApps } from '../domain/host-status.ts';
import { createSqliteNotificationSettingsStore } from '../infra/sqlite/notification-settings-store.ts';
import { systemClock } from '../infra/system-clock.ts';
import { createComponentOperationExecutor } from '../app/component-operation-executor.ts';
import { groupMembers } from '../app/ops/group.ts';
@@ -37,6 +42,8 @@ import { missingSkills, readSkillFiles } from './skills-file.ts';
import { createWorkflowExtensions } from '../domain/workflow-extensions.ts';
const logger = createConsoleLogger();
/** ghcr.io answers 429 now and then; the schedule is spread out, so a run can wait minutes for the tag list. */
const REGISTRY_RATE_LIMIT_WAITS_MS = [30_000, 120_000, 300_000];
let config;
try {
@@ -113,7 +120,7 @@ const runner = createWorkflowRunner({
functionCatalog,
functionRuntime: createFunctionRuntimeClient({ baseUrl: config.functionRuntimeUrl }),
componentOperations: createComponentOperationExecutor({
registry: createHttpRegistryClient(),
registry: createHttpRegistryClient({ rateLimitWaitsMs: REGISTRY_RATE_LIMIT_WAITS_MS }),
hostBackup,
backupVmid: config.backupVmid,
clock: systemClock,
@@ -147,7 +154,38 @@ const runner = createWorkflowRunner({
runInputs,
});
const notifyDeps = { get: (id: string) => instanceStore.get(id), report: (instance: WorkflowInstance) => reportRun(instanceStore, instance), notifier };
const notificationSettings = createSqliteNotificationSettingsStore(db);
const hostStatusStore = createSqliteHostStatusStore(db);
const quietDeps = { clock: systemClock, timeZone: config.scheduleTimeZone, settings: notificationSettings };
// In the owner's quiet hours nothing rings; the morning report covers those runs.
const notifyDeps = { get: (id: string) => instanceStore.get(id), report: (instance: WorkflowInstance) => reportRun(instanceStore, instance), notifier, quiet: () => isQuiet(quietDeps) };
// The morning report: what waits for the owner, what failed or is down, what changed since the last report.
const digestLoop = runDigestLoop({
...quietDeps,
notifier,
runs: () => instanceStore.listInstances({}).map((instance) => ({ ...reportRun(instanceStore, instance), workflowId: instance.workflowId, status: instance.projection.status, updatedAt: instance.updatedAt })),
down: () => {
const status = hostStatusStore.get();
return status === null ? [] : downApps(status);
},
intervalMs: 60_000,
signal: shutdown.signal,
onError: (cause) => logger.error('digest.failed', { error: cause instanceof Error ? cause.message : String(cause) }),
});
// Once a minute: what runs on VM103 and how old the newest VM backup is, for the overview (read-only, through the proxy).
const hostStatusLoop = runHostStatusLoop({
runner: processRunner,
clock: systemClock,
apps: () => appConfig.list(),
readBackup: hostBackup.read,
backupVmid: config.backupVmid,
store: hostStatusStore,
intervalMs: 60_000,
signal: shutdown.signal,
onError: (cause) => logger.error('host_status.failed', { error: cause instanceof Error ? cause.message : String(cause) }),
});
await runWorkflowLoop(runner, {
signal: shutdown.signal,
@@ -164,6 +202,7 @@ await runWorkflowLoop(runner, {
if (result.kind === 'finalized' || result.kind === 'cancelled') removeFinishedInputs(runInputs);
},
});
await Promise.all([hostStatusLoop, digestLoop]);
db.close();
/** Never lets a cleanup failure stop the loop: the inputs are only sensitive, not load-bearing. */
+4
View File
@@ -6,6 +6,8 @@ import { buildServer } from '../http/server.ts';
import { dataVersion, openDatabase } from '../infra/sqlite/open.ts';
import { createSqliteAppConfigStore } from '../infra/sqlite/app-config-store.ts';
import { createSqliteSkillStore } from '../infra/sqlite/skill-store.ts';
import { createSqliteNotificationSettingsStore } from '../infra/sqlite/notification-settings-store.ts';
import { createSqliteHostStatusStore } from '../infra/sqlite/host-status-store.ts';
import { createSqliteReleaseNotesSourcesStore } from '../infra/sqlite/release-notes-sources-store.ts';
import { createSqliteWorkflowCatalogStore } from '../infra/sqlite/workflow-catalog-store.ts';
import { createSqliteWorkflowLayoutStore } from '../infra/sqlite/workflow-layout-store.ts';
@@ -76,6 +78,8 @@ const app = await buildServer({
privilegedActionStore,
modelProfiles,
releaseNotesSources: createSqliteReleaseNotesSourcesStore(db),
hostStatus: createSqliteHostStatusStore(db),
notificationSettings: createSqliteNotificationSettingsStore(db),
workflowRunInputs: createSqliteWorkflowRunInputStore(db),
modelActivity: createSqliteModelActivityStore(db),
startWorkflowRun: (name, mode) => startWorkflowRun(runDeps, name, mode),
+6
View File
@@ -17,6 +17,7 @@ import type {
SessionOpenRequest,
SessionOpenResult,
SessionLogResult,
AgentLogsRequest,
AgentTextResult,
SessionSummary,
} from '../domain/executor-api.ts';
@@ -135,6 +136,7 @@ interface ExecutorOps {
closeSession: [SessionCloseRequest, SessionSummary];
sessionLog: [SessionCloseRequest, SessionLogResult];
sessionCompose: [SessionCloseRequest, AgentTextResult];
sessionLogs: [SessionCloseRequest & AgentLogsRequest, AgentTextResult];
}
type ExecutorOp = keyof ExecutorOps;
type ExecutorScript<K extends ExecutorOp> = (request: ExecutorOps[K][0]) => ExecutorCall<ExecutorOps[K][1]>;
@@ -204,4 +206,8 @@ export class FakeExecutorClient implements ExecutorClient {
sessionCompose(request: SessionCloseRequest) {
return this.#call('sessionCompose', request);
}
sessionLogs(request: SessionCloseRequest & AgentLogsRequest) {
return this.#call('sessionLogs', request);
}
}
+4
View File
@@ -5,6 +5,8 @@ import { createIsolatedFunctionRuntime } from '../infra/runtime/isolated-functio
import { openDatabase } from '../infra/sqlite/open.ts';
import { createSqliteAppConfigStore } from '../infra/sqlite/app-config-store.ts';
import { createSqliteModelProfileStore } from '../infra/sqlite/model-profile-store.ts';
import { createSqliteNotificationSettingsStore } from '../infra/sqlite/notification-settings-store.ts';
import { createSqliteHostStatusStore } from '../infra/sqlite/host-status-store.ts';
import { createSqliteReleaseNotesSourcesStore } from '../infra/sqlite/release-notes-sources-store.ts';
import { createSqlitePrivilegedActionStore } from '../infra/sqlite/privileged-action-store.ts';
import { createSqliteSkillStore } from '../infra/sqlite/skill-store.ts';
@@ -34,6 +36,8 @@ export function sqliteServerDeps(db: DatabaseSync = openDatabase(':memory:')) {
privilegedActionStore: createSqlitePrivilegedActionStore(db),
modelProfiles: createSqliteModelProfileStore(db),
releaseNotesSources: createSqliteReleaseNotesSourcesStore(db),
hostStatus: createSqliteHostStatusStore(db),
notificationSettings: createSqliteNotificationSettingsStore(db),
workflowRunInputs: createSqliteWorkflowRunInputStore(db),
modelActivity: createSqliteModelActivityStore(db),
schedules: { store: createSqliteWorkflowScheduleStore(db), timeZone: 'Europe/Warsaw' },
+1 -1
View File
@@ -25,7 +25,7 @@ export function seedTestApps(appConfig: AppConfigStore): void {
releaseNotesExecutor: 'sonnet',
migrateExecutor: null,
promptHint: null,
postUpgradeExecutor: null,
postUpgradeExecutor: null, postUpgradeWhen: 'when_needed',
notes: null,
apply: false,
});
+7
View File
@@ -18,11 +18,15 @@ import {
ModelCallsResponse,
ModelProfileListResponse,
ReleaseNotesSourcesResponse,
HostStatusResponse,
NotificationSettingsResponse,
ModelProfileSummaryResponse,
TagPreviewResponse,
} from '../http/contract.ts';
import type { AppConfigPatch, ModelProfilePublishRequest } from '../http/contract.ts';
type NotificationSettingsResponse = z.infer<typeof NotificationSettingsResponse>;
export class ApiError extends Error {
status: number;
@@ -115,6 +119,9 @@ export const api = {
modelCalls: (limit = 50) => call(ModelCallsResponse, `/api/model-calls?limit=${String(limit)}`),
modelProfiles: () => call(ModelProfileListResponse, '/api/model-profiles'),
releaseNotesSources: () => call(ReleaseNotesSourcesResponse, '/api/release-notes-sources'),
hostStatus: () => call(HostStatusResponse, '/api/host-status'),
notificationSettings: () => call(NotificationSettingsResponse, '/api/notification-settings'),
saveNotificationSettings: (settings: NotificationSettingsResponse['settings']) => call(NotificationSettingsResponse, '/api/notification-settings', settings, 'PUT'),
saveReleaseNotesSources: (sources: readonly string[]) => call(ReleaseNotesSourcesResponse, '/api/release-notes-sources', { sources }, 'PUT'),
modelProfileVersions: (id: string) => call(z.array(ModelProfileSummaryResponse), `/api/model-profiles/${encodeURIComponent(id)}/versions`),
modelProfileVersion: (id: string, version: number) => call(ModelProfileSummaryResponse, `/api/model-profiles/${encodeURIComponent(id)}/versions/${String(version)}`),
+65 -1
View File
@@ -35,6 +35,7 @@ class FakeEventSource {
let server: FastifyInstance;
let appConfig: AppConfigStore;
let schedules: ReturnType<typeof sqliteServerDeps>['schedules'];
let hostStatus: ReturnType<typeof sqliteServerDeps>['hostStatus'];
let requests: { method: string; url: string; body?: unknown }[];
let container: HTMLElement;
@@ -42,7 +43,7 @@ beforeEach(async () => {
FakeEventSource.instances = [];
requests = [];
const deps = sqliteServerDeps();
({ appConfig, schedules } = deps);
({ appConfig, schedules, hostStatus } = deps);
seedTestApps(appConfig);
const registry = new FakeRegistryClient({ 'demo/bazarr': ['1.0.0', '1.1.0', '1.2.0'] });
server = await buildServer({ ...deps, registry, build: { branch: 'foundations', number: 6, revision: 'de5ee92a1b2c3d4e5f60718293a4b5c6d7e8f901' } });
@@ -205,6 +206,18 @@ describe('app configuration', () => {
await eventually(() => expect((requests.filter((r) => r.method === 'POST').at(-1)!.body as Record<string, unknown>)['postUpgradeExecutor']).toBeNull());
});
it('sets when the post-upgrade model runs, once a model is chosen', async () => {
await open();
expect(form()!.querySelector('[name="postUpgradeWhen"]')).toBeNull();
await choose(field('postUpgradeExecutor'), 'sonnet');
expect(field<HTMLSelectElement>('postUpgradeWhen').value).toBe('when_needed');
await choose(field('postUpgradeWhen'), 'always');
button(form()!, 'Save').click();
await eventually(() => expect(form()!.textContent).toContain('Saved'));
expect(requests.find((r) => r.method === 'POST')!.body).toEqual({ postUpgradeExecutor: 'sonnet', postUpgradeWhen: 'always' });
expect((await server.inject('/api/apps/bazarr/config')).json()).toMatchObject({ postUpgradeWhen: 'always' });
});
it('checks the change in the browser before sending it', async () => {
await open();
await type(field('tagPattern'), '(?P<v>x)');
@@ -327,6 +340,57 @@ describe('other views', () => {
await eventually(() => expect(container.querySelectorAll('tbody tr')).toHaveLength(5));
});
it('shows on the overview which apps are down and how old the VM backup is', async () => {
const now = Date.now();
hostStatus.put({
checkedAt: new Date(now).toISOString(),
apps: [
{ app: 'grafana', enabled: true, state: 'exited', health: null },
{ app: 'bazarr', enabled: true, state: 'running', health: 'healthy' },
],
backup: { ok: true, createdAt: new Date(now - 5 * 3_600_000).toISOString(), ageHours: 5, volid: 'pbs:vm/103' },
error: null,
});
window.location.hash = '#/overview';
await eventually(() => expect(text()).toContain('1/2'));
expect(text()).toContain('grafana exited');
expect(text()).toContain('5 h ago');
window.location.hash = '#/apps';
await eventually(() => expect(container.querySelector('.health-down')?.getAttribute('title')).toBe('down: exited'));
});
it('saves the quiet hours and shows the morning report on the overview', async () => {
window.location.hash = '#/settings';
const from = await eventually(() => {
const found = container.querySelectorAll<HTMLInputElement>('input[type="time"]')[0];
if (found === undefined) throw new Error('no time field');
return found;
});
expect(from.value).toBe('01:00');
await type(from, '00:30');
button(container, 'Save').click();
await eventually(() => expect(text()).toContain('Notification settings saved.'));
expect((await server.inject('/api/notification-settings')).json()).toMatchObject({ settings: { quietFrom: '00:30', quietUntil: '07:30' }, timeZone: 'Europe/Warsaw' });
window.location.hash = '#/overview';
await eventually(() => expect(text()).toContain('All good'));
expect(text()).toContain('No real update, nothing failed, nothing waits for you.');
});
it('jumps to an app from the palette opened with Ctrl+K', async () => {
window.dispatchEvent(new KeyboardEvent('keydown', { key: 'k', ctrlKey: true }));
const input = await eventually(() => {
const found = container.querySelector<HTMLInputElement>('.palette input');
if (found === null) throw new Error('no palette');
return found;
});
await eventually(() => expect(container.querySelector('.palette-list')?.textContent).toContain('bazarr'));
await type(input, 'bazarr app');
input.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true }));
await eventually(() => expect(window.location.hash).toBe('#/apps/bazarr'));
expect(container.querySelector('.palette')).toBeNull();
});
it('says so for an unknown page', async () => {
window.location.hash = '#/nowhere';
await eventually(() => expect(text()).toContain('Page not found'));
+14 -2
View File
@@ -10,6 +10,7 @@ import { hrefFor, parseRoute } from './route.ts';
import { useTheme } from './theme.ts';
import { Icon } from './components/ui.tsx';
import { DialogProvider } from './components/dialogs.tsx';
import { CommandPalette, useCommandPalette } from './components/command-palette.tsx';
import type { IconName } from './components/ui.tsx';
import type { Route } from './route.ts';
import { AppDetailView, AppsView } from './views/apps.tsx';
@@ -21,6 +22,7 @@ import { FunctionEditorView } from './views/function-editor.tsx';
import { WorkflowInstancesView } from './views/instances.tsx';
import { InstanceTimelineView } from './views/instance-timeline.tsx';
import { ModelsView } from './views/models.tsx';
import { SettingsView } from './views/settings.tsx';
function useRoute(): Route {
const [route, setRoute] = useState(() => parseRoute(window.location.hash));
@@ -35,7 +37,7 @@ function useRoute(): Route {
const LIVE_LABEL: Record<LiveStatus, string> = { connecting: 'connecting…', live: 'live', offline: 'offline — retrying' };
function Sidebar({ route, status }: { route: Route; status: LiveStatus }) {
function Sidebar({ route, status, onSearch }: { route: Route; status: LiveStatus; onSearch: () => void }) {
const gates = useResource(() => api.workflowInstances({ status: 'waiting' }), []);
const { theme, toggle } = useTheme();
const waiting = gates.data?.length ?? 0;
@@ -63,6 +65,11 @@ function Sidebar({ route, status }: { route: Route; status: LiveStatus }) {
<span class="brand-mark" aria-hidden="true" />
Homelab upgrade
</span>
<button type="button" class="search-button" onClick={onSearch} title="Search (Ctrl+K)">
<Icon name="search" size={16} />
<span class="nav-label">Search</span>
<kbd>Ctrl K</kbd>
</button>
<nav>
{link('overview', 'Overview', 'overview', { view: 'overview' }, waiting)}
{link('apps', 'Apps', 'apps', { view: 'apps' })}
@@ -70,6 +77,7 @@ function Sidebar({ route, status }: { route: Route; status: LiveStatus }) {
{link('workflows', 'Workflows', 'workflows', { view: 'workflows' })}
{link('functions', 'Functions', 'functions', { view: 'functions' })}
{link('models', 'Models', 'models', { view: 'models' })}
{link('settings', 'Settings', 'settings', { view: 'settings' })}
</nav>
<div class="sidebar-foot">
<span class={`live live-${status}`} title="Live updates from the server">
@@ -109,6 +117,8 @@ function View({ route }: { route: Route }): ComponentChildren {
return <FunctionEditorView functionId={route.functionId} version={route.version} />;
case 'models':
return <ModelsView />;
case 'settings':
return <SettingsView />;
case 'not_found':
return (
<section>
@@ -147,11 +157,12 @@ function Footer() {
export function App() {
const route = useRoute();
const { revision, status, refresh } = useLive();
const [paletteOpen, setPaletteOpen] = useCommandPalette();
return (
<LiveContext.Provider value={{ revision, refresh }}>
<DialogProvider>
<div class="shell">
<Sidebar route={route} status={status} />
<Sidebar route={route} status={status} onSearch={() => setPaletteOpen(true)} />
<div class="content">
<main>
<View route={route} />
@@ -159,6 +170,7 @@ export function App() {
<Footer />
</div>
</div>
{paletteOpen ? <CommandPalette onClose={() => setPaletteOpen(false)} /> : null}
</DialogProvider>
</LiveContext.Provider>
);
+12
View File
@@ -1,5 +1,7 @@
import type { ComponentChildren } from 'preact';
import { runModeLabel, runStatusLabel } from './labels.ts';
import { formatInstant, formatRelative } from './format.ts';
import { useNow } from './hooks.ts';
/** A workflow run's status in words, colored by status. */
export function RunBadge({ status }: { status: string }) {
@@ -26,3 +28,13 @@ export function ErrorNote({ message }: { message: string }) {
export function Empty({ children }: { children: ComponentChildren }) {
return <p class="empty">{children}</p>;
}
/** An instant as "5 min ago" or "in 3 h", with the full time on hover. */
export function When({ iso }: { readonly iso: string | null }) {
const now = useNow();
return (
<time dateTime={iso ?? undefined} title={formatInstant(iso)}>
{formatRelative(iso, now)}
</time>
);
}
@@ -0,0 +1,16 @@
import { describe, expect, it } from 'vitest';
import { filterItems } from './command-palette.tsx';
describe('filterItems', () => {
const items = [
{ label: 'Overview', hint: 'page', href: '#/overview', icon: 'overview' as const },
{ label: 'bazarr', hint: 'app · bazarr', href: '#/apps/bazarr', icon: 'apps' as const },
{ label: 'bazarr: Failed', hint: 'run · Verify failed', href: '#/runs/r1', icon: 'runs' as const },
];
it('keeps items with every word of the query, case aside, in their order', () => {
expect(filterItems(items, 'BAZ').map((item) => item.href)).toEqual(['#/apps/bazarr', '#/runs/r1']);
expect(filterItems(items, 'baz run').map((item) => item.href)).toEqual(['#/runs/r1']);
expect(filterItems(items, ' ')).toHaveLength(3);
});
});
+146
View File
@@ -0,0 +1,146 @@
import { useEffect, useLayoutEffect, useRef, useState } from 'preact/hooks';
import { api } from '../api.ts';
import { runStatusLabel } from '../labels.ts';
import { hrefFor } from '../route.ts';
import { Icon } from './ui.tsx';
import type { IconName } from './ui.tsx';
export interface PaletteItem {
readonly label: string;
readonly hint: string;
readonly href: string;
readonly icon: IconName;
}
const PAGES: readonly PaletteItem[] = [
{ label: 'Overview', hint: 'page', href: hrefFor({ view: 'overview' }), icon: 'overview' },
{ label: 'Apps', hint: 'page', href: hrefFor({ view: 'apps' }), icon: 'apps' },
{ label: 'Runs', hint: 'page', href: hrefFor({ view: 'runs' }), icon: 'runs' },
{ label: 'Workflows', hint: 'page', href: hrefFor({ view: 'workflows' }), icon: 'workflows' },
{ label: 'Functions', hint: 'page', href: hrefFor({ view: 'functions' }), icon: 'functions' },
{ label: 'Models', hint: 'page', href: hrefFor({ view: 'models' }), icon: 'models' },
{ label: 'Settings', hint: 'page', href: hrefFor({ view: 'settings' }), icon: 'settings' },
];
const RECENT_RUNS = 15;
const SHOWN = 12;
/** Every word of the query must appear in the item's label or hint; the order of the items stays. */
export function filterItems(items: readonly PaletteItem[], query: string): PaletteItem[] {
const words = query.toLowerCase().split(/\s+/).filter((word) => word !== '');
return items.filter((item) => {
const text = `${item.label} ${item.hint}`.toLowerCase();
return words.every((word) => text.includes(word));
});
}
/** Everything the palette can jump to; a list that fails to load is left out rather than failing the palette. */
async function loadItems(): Promise<PaletteItem[]> {
const [apps, runs, workflows, functions] = await Promise.allSettled([api.apps(), api.workflowInstances(), api.workflows(), api.functions()]);
return [
...PAGES,
...(apps.status === 'fulfilled' ? apps.value.map((app) => ({ label: app.app, hint: `app · ${app.project}`, href: hrefFor({ view: 'app', app: app.app }), icon: 'apps' as const })) : []),
...(runs.status === 'fulfilled'
? runs.value.slice(0, RECENT_RUNS).map((run) => ({
label: `${run.app ?? run.workflowId}: ${runStatusLabel(run.status)}`,
hint: `run · ${run.outcome ?? run.step ?? run.id}`,
href: hrefFor({ view: 'run', runId: run.id }),
icon: 'runs' as const,
}))
: []),
...(workflows.status === 'fulfilled' ? workflows.value.map((wf) => ({ label: wf.workflowId, hint: `workflow · ${wf.name}`, href: hrefFor({ view: 'workflow', workflowId: wf.workflowId }), icon: 'workflows' as const })) : []),
...(functions.status === 'fulfilled' ? functions.value.map((fn) => ({ label: fn.functionId, hint: `function · ${fn.name}`, href: hrefFor({ view: 'function', functionId: fn.functionId }), icon: 'functions' as const })) : []),
];
}
/** Opens on Ctrl+K (⌘K) anywhere, or `/` outside a text field; the sidebar's search button opens it too. */
export function useCommandPalette(): [boolean, (open: boolean) => void] {
const [open, setOpen] = useState(false);
useLayoutEffect(() => {
const onKey = (event: KeyboardEvent) => {
const typing = event.target instanceof HTMLElement && (event.target.isContentEditable || ['INPUT', 'TEXTAREA', 'SELECT'].includes(event.target.tagName));
if ((event.ctrlKey || event.metaKey) && event.key.toLowerCase() === 'k') {
event.preventDefault();
setOpen((current) => !current);
} else if (event.key === '/' && !typing) {
event.preventDefault();
setOpen(true);
}
};
window.addEventListener('keydown', onKey);
return () => window.removeEventListener('keydown', onKey);
}, []);
return [open, setOpen];
}
export function CommandPalette({ onClose }: { readonly onClose: () => void }) {
const [items, setItems] = useState<readonly PaletteItem[]>(PAGES);
const [query, setQuery] = useState('');
const [active, setActive] = useState(0);
const input = useRef<HTMLInputElement>(null);
useLayoutEffect(() => input.current?.focus(), []);
useEffect(() => {
let current = true;
void loadItems().then((loaded) => {
if (current) setItems(loaded);
});
return () => {
current = false;
};
}, []);
const shown = filterItems(items, query).slice(0, SHOWN);
const go = (item: PaletteItem | undefined) => {
if (item === undefined) return;
window.location.hash = item.href;
onClose();
};
const onKey = (event: KeyboardEvent) => {
if (event.key === 'ArrowDown') {
event.preventDefault();
setActive((index) => Math.min(index + 1, shown.length - 1));
} else if (event.key === 'ArrowUp') {
event.preventDefault();
setActive((index) => Math.max(index - 1, 0));
} else if (event.key === 'Enter') {
event.preventDefault();
go(shown[active]);
} else if (event.key === 'Escape') {
onClose();
}
};
return (
<div class="modal-backdrop palette-backdrop" onClick={(event) => event.target === event.currentTarget && onClose()}>
<div class="palette" role="dialog" aria-modal="true" aria-label="Go to">
<div class="palette-input">
<Icon name="search" />
<input
ref={input}
type="text"
placeholder="Go to an app, a run, a workflow or a page…"
value={query}
onInput={(event) => {
setQuery((event.target as HTMLInputElement).value);
setActive(0);
}}
onKeyDown={onKey}
aria-label="Search"
/>
<kbd>Esc</kbd>
</div>
{shown.length === 0 ? (
<p class="palette-empty">Nothing matches.</p>
) : (
<ul class="palette-list" role="listbox">
{shown.map((item, index) => (
<li key={item.href} role="option" aria-selected={index === active} class={index === active ? 'active' : undefined} onMouseEnter={() => setActive(index)} onClick={() => go(item)}>
<Icon name={item.icon} size={16} />
<span class="palette-label">{item.label}</span>
<span class="palette-hint">{item.hint}</span>
</li>
))}
</ul>
)}
</div>
</div>
);
}
+3
View File
@@ -19,6 +19,9 @@ const ICONS = {
upgrade: 'M12 19V5M5 12l7-7 7 7',
grid: 'M3 3h7v7H3zM14 3h7v7h-7zM14 14h7v7h-7zM3 14h7v7H3z',
list: 'M8 6h13M8 12h13M8 18h13M3 6h.01M3 12h.01M3 18h.01',
settings: 'M4 21v-7M4 10V3M12 21v-9M12 8V3M20 21v-5M20 12V3M1 14h6M9 8h6M17 16h6',
bell: 'M18 8a6 6 0 0 0-12 0c0 7-3 9-3 9h18s-3-2-3-9M13.7 21a2 2 0 0 1-3.4 0',
search: 'M11 19a8 8 0 1 0 0-16 8 8 0 0 0 0 16zM21 21l-4.3-4.3',
} as const;
export type IconName = keyof typeof ICONS;
+21 -1
View File
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest';
import { formatBuild, formatCost, formatDuration, formatInstant, humanizeCode, nodeRunStates, runVersions, traversedEdges } from './format.ts';
import { formatBuild, formatCost, formatDuration, formatInstant, formatRelative, humanizeCode, nodeRunStates, runVersions, traversedEdges } from './format.ts';
describe('formatInstant', () => {
it('shows date and time in the given zone', () => {
@@ -122,3 +122,23 @@ describe('traversedEdges', () => {
expect(traversedEdges({ a: 'done', b: 'done', c: 'current' }, edges)).toEqual([true, true, false, false]);
});
});
describe('formatRelative', () => {
const now = Date.parse('2026-09-30T12:00:00.000Z');
it.each([
['2026-09-30T11:59:40.000Z', 'just now'],
['2026-09-30T11:55:00.000Z', '5 min ago'],
['2026-09-30T09:00:00.000Z', '3 h ago'],
['2026-09-29T10:00:00.000Z', 'yesterday'],
['2026-09-27T12:00:00.000Z', '3 days ago'],
['2026-09-30T14:30:00.000Z', 'in 3 h'],
['2026-10-01T12:00:00.000Z', 'tomorrow'],
['2026-09-10T12:00:00.000Z', '2026-09-10'],
])('%s -> %s', (iso, text) => {
expect(formatRelative(iso, now, 'UTC')).toBe(text);
});
it('shows a dash for nothing', () => {
expect(formatRelative(null, now)).toBe('—');
});
});
+21
View File
@@ -100,3 +100,24 @@ export function runVersions(run: { readonly running: string | null; readonly can
if (run === null) return { running: null, available: null };
return { running: run.running, available: run.candidate !== null && run.candidate !== run.running ? run.candidate : null };
}
/**
* `5 min ago`, `in 3 h`, `yesterday`: how far an instant is from now, for lists read at a glance. Past a week it is the
* date (`2026-09-18`). The full time belongs in the element's title.
*/
export function formatRelative(iso: string | null, now: number, timeZone?: string): string {
if (iso === null) return DASH;
const time = Date.parse(iso);
if (Number.isNaN(time)) return DASH;
const diff = time - now;
const minutes = Math.round(Math.abs(diff) / 60_000);
const say = (amount: string) => (diff < 0 ? `${amount} ago` : `in ${amount}`);
if (minutes < 1) return diff < 0 ? 'just now' : 'in a moment';
if (minutes < 60) return say(`${String(minutes)} min`);
const hours = Math.round(minutes / 60);
if (hours < 24) return say(`${String(hours)} h`);
const days = Math.round(hours / 24);
if (days === 1) return diff < 0 ? 'yesterday' : 'tomorrow';
if (days < 7) return say(`${String(days)} days`);
return new Intl.DateTimeFormat('sv-SE', { dateStyle: 'short', ...(timeZone === undefined ? {} : { timeZone }) }).format(time);
}
+10
View File
@@ -38,3 +38,13 @@ export function useResource<T>(load: () => Promise<T>, deps: readonly unknown[])
return state;
}
/** The current time, refreshed every `everyMs`, so relative times ("5 min ago") keep up without a reload. */
export function useNow(everyMs = 60_000): number {
const [now, setNow] = useState(() => Date.now());
useEffect(() => {
const timer = setInterval(() => setNow(Date.now()), everyMs);
return () => clearInterval(timer);
}, [everyMs]);
return now;
}
+8 -1
View File
@@ -2,8 +2,15 @@
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
<title>Homelab upgrade</title>
<meta name="theme-color" content="#0d1117" />
<meta name="mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<link rel="icon" href="/icon.svg" type="image/svg+xml" />
<link rel="apple-touch-icon" href="/icon.svg" />
<link rel="manifest" href="/manifest.webmanifest" />
</head>
<body>
<div id="app"></div>
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64"><defs><linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#7c9cff"/><stop offset="1" stop-color="#2dd4bf"/></linearGradient></defs><rect width="64" height="64" rx="14" fill="#0d1117"/><rect x="8" y="8" width="48" height="48" rx="12" fill="url(#g)"/><path d="M32 44V20M22 30l10-10 10 10" fill="none" stroke="#0d1117" stroke-width="5" stroke-linecap="round" stroke-linejoin="round"/></svg>

After

Width:  |  Height:  |  Size: 476 B

+9
View File
@@ -0,0 +1,9 @@
{
"name": "Homelab upgrade",
"short_name": "Upgrade",
"start_url": "./#/overview",
"display": "standalone",
"background_color": "#0d1117",
"theme_color": "#0d1117",
"icons": [{ "src": "icon.svg", "sizes": "any", "type": "image/svg+xml", "purpose": "any" }]
}
+2
View File
@@ -23,6 +23,8 @@ describe('parseRoute', () => {
['#/instances', { view: 'runs' }],
['#/instances/inst-123', { view: 'run', runId: 'inst-123' }],
['#/models', { view: 'models' }],
['#/settings', { view: 'settings' }],
['#/settings/x', { view: 'not_found' }],
['#/nope', { view: 'not_found' }],
['#/runs/a/b', { view: 'not_found' }],
])('%j -> %j', (hash, route) => {
+4
View File
@@ -11,6 +11,7 @@ export type Route =
| { readonly view: 'function'; readonly functionId: string }
| { readonly view: 'function_version'; readonly functionId: string; readonly version: number }
| { readonly view: 'models' }
| { readonly view: 'settings' }
| { readonly view: 'not_found' };
const NOT_FOUND: Route = { view: 'not_found' };
@@ -58,6 +59,7 @@ export function parseRoute(hash: string): Route {
if (head === 'models' && arg === undefined) {
return { view: 'models' };
}
if (head === 'settings' && arg === undefined) return { view: 'settings' };
return NOT_FOUND;
}
@@ -87,5 +89,7 @@ export function hrefFor(route: Exclude<Route, { view: 'not_found' }>): string {
return `#/functions/${encodeURIComponent(route.functionId)}/v/${String(route.version)}`;
case 'models':
return '#/models';
case 'settings':
return '#/settings';
}
}
+59 -1
View File
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest';
import { appsWithUpdate, costPerDay, ganttBars, lastDays, recentAppRuns, runDurationMs, runDurations, runsPerDay, successRate } from './stats.ts';
import { appCondition, appsWithUpdate, costPerDay, ganttBars, hostSummary, lastDays, recentAppRuns, runDurationMs, runDurations, runsPerDay, successRate, troubleCount, versionHistory } from './stats.ts';
import type { WorkflowInstanceSummaryResponse, WorkflowModelCallResponse } from '../http/contract.ts';
const NOW = Date.parse('2026-09-30T12:00:00.000Z');
@@ -187,3 +187,61 @@ describe('ganttBars', () => {
]);
});
});
describe('appCondition and hostSummary', () => {
const app = (name: string, state: string, health: string | null, enabled = true) => ({ app: name, enabled, state, health });
it('calls an app down when its container restarts, stopped, is missing or unhealthy; a disabled one is off', () => {
expect(appCondition(app('a', 'running', null))).toBe('ok');
expect(appCondition(app('a', 'running', 'healthy'))).toBe('ok');
expect(appCondition(app('a', 'running', 'starting'))).toBe('starting');
expect(appCondition(app('a', 'running', 'unhealthy'))).toBe('down');
expect(appCondition(app('a', 'restarting', null))).toBe('down');
expect(appCondition(app('a', 'missing', null))).toBe('down');
expect(appCondition(app('a', 'exited', null, false))).toBe('off');
});
it('counts enabled apps that run well and lists the others, down first', () => {
const summary = hostSummary(
{
checkedAt: '2026-09-30T11:57:30.000Z',
apps: [app('ombi', 'running', 'starting'), app('jellyfin', 'running', 'healthy'), app('bazarr', 'exited', null), app('old', 'exited', null, false)],
},
NOW,
);
expect(summary).toEqual({
up: 1,
total: 3,
problems: [
{ app: 'bazarr', condition: 'down', state: 'exited' },
{ app: 'ombi', condition: 'starting', state: 'running' },
],
ageMinutes: 2,
});
});
});
describe('versionHistory', () => {
it('keeps real ended runs as updates, runs that left the app as it was, and failures, with repairs and release notes', () => {
const runs = [
run({ id: 'dry', mode: 'shadow', status: 'succeeded', current: '1.2', candidate: '1.3' }),
run({ id: 'live', mode: 'real', status: 'running', current: '1.2', candidate: '1.3' }),
run({ id: 'up', mode: 'real', status: 'succeeded', current: '1.1', candidate: '1.2', running: '1.2', outcome: 'Updated' }),
run({ id: 'back', mode: 'real', status: 'succeeded', current: '1.1', candidate: '1.2', running: '1.1', outcome: 'Rolled back' }),
run({ id: 'bad', mode: 'real', status: 'failed', current: '1.0', candidate: '1.1', running: '1.0', outcome: 'Verify failed' }),
];
const calls = [
call({ instanceId: 'up', kind: 'releaseNotes', outcome: 'clear', summary: 'Bug fixes.' }),
call({ instanceId: 'up', kind: 'repair', outcome: 'fixed', summary: '' }),
];
const history = versionHistory(runs, calls);
expect(history.map((entry) => [entry.runId, entry.kind, entry.from, entry.to, entry.repaired])).toEqual([
['up', 'updated', '1.1', '1.2', true],
['back', 'kept', '1.1', '1.2', false],
['bad', 'failed', '1.0', '1.1', false],
]);
expect(history[0]?.releaseNotes).toEqual({ verdict: 'clear', summary: 'Bug fixes.' });
expect(troubleCount(history)).toBe(3);
expect(troubleCount(history, 1)).toBe(1);
});
});
+77
View File
@@ -164,3 +164,80 @@ export function ganttBars(items: readonly Pick<WorkflowTimelineItem, 'type' | 'n
}
return bars;
}
export type AppCondition = 'ok' | 'starting' | 'down' | 'off';
/** How an app is doing right now, from its container: down covers restarting, stopped, missing and unhealthy. */
export function appCondition(health: { readonly enabled: boolean; readonly state: string; readonly health: string | null }): AppCondition {
if (health.state === 'running') return health.health === 'unhealthy' ? 'down' : health.health === 'starting' ? 'starting' : 'ok';
return health.enabled ? 'down' : 'off';
}
export interface HostSummary {
/** Enabled apps whose container runs well, of all enabled apps. */
readonly up: number;
readonly total: number;
/** Enabled apps that are down or still starting, worst first. */
readonly problems: readonly { readonly app: string; readonly condition: AppCondition; readonly state: string }[];
/** Minutes since the driver last looked: an old look means the driver is not looking. */
readonly ageMinutes: number;
}
export function hostSummary(status: { readonly checkedAt: string; readonly apps: readonly { readonly app: string; readonly enabled: boolean; readonly state: string; readonly health: string | null }[] }, now: number): HostSummary {
const enabled = status.apps.filter((app) => app.enabled);
const problems = enabled
.map((app) => ({ app: app.app, condition: appCondition(app), state: app.health === 'unhealthy' ? 'unhealthy' : app.state }))
.filter((app) => app.condition !== 'ok')
.sort((a, b) => (a.condition === b.condition ? a.app.localeCompare(b.app) : a.condition === 'down' ? -1 : 1));
return { up: enabled.length - problems.length, total: enabled.length, problems, ageMinutes: Math.max(0, Math.floor((now - Date.parse(status.checkedAt)) / 60_000)) };
}
export type HistoryKind = 'updated' | 'kept' | 'failed';
export interface HistoryEntry {
readonly runId: string;
readonly at: string;
readonly kind: HistoryKind;
readonly from: string | null;
readonly to: string | null;
readonly outcome: string | null;
/** The model repaired the app during this run. */
readonly repaired: boolean;
/** What the model read in the release notes before the update, when it was asked. */
readonly releaseNotes: { readonly verdict: string; readonly summary: string } | null;
}
type HistoryCall = Pick<WorkflowModelCallResponse, 'instanceId' | 'kind' | 'outcome' | 'summary'>;
/**
* An app's real runs as its version history, newest first: an update (from → to), a run that ended but left the app as
* it was (skipped, rolled back), or a failure. Dry runs and runs in progress change nothing and are left out.
*/
export function versionHistory(runs: readonly WorkflowInstanceSummaryResponse[], calls: readonly HistoryCall[]): HistoryEntry[] {
const out: HistoryEntry[] = [];
for (const run of runs) {
if (run.mode !== 'real' || !ENDED.has(run.status)) continue;
const kind: HistoryKind = run.status === 'failed' ? 'failed' : run.status === 'succeeded' && run.running !== null && run.running !== run.current ? 'updated' : 'kept';
const own = calls.filter((call) => call.instanceId === run.id);
const notes = own.find((call) => call.kind === 'releaseNotes');
out.push({
runId: run.id,
at: run.updatedAt,
kind,
from: run.current,
to: kind === 'updated' ? run.running : run.candidate,
outcome: run.outcome,
repaired: own.some((call) => call.kind === 'repair'),
releaseNotes: notes === undefined ? null : { verdict: notes.outcome, summary: notes.summary },
});
}
return out;
}
/** How many of the app's last `window` real runs did not simply update it: failed, left as it was, or needed a repair. */
export function troubleCount(history: readonly HistoryEntry[], window = 10): number {
return history.slice(0, window).filter((entry) => entry.kind !== 'updated' || entry.repaired).length;
}
/** Two troubled runs among the last ten make an app worth a closer look. */
export const TROUBLE_THRESHOLD = 2;
+295 -4
View File
@@ -1140,12 +1140,17 @@ h2 {
.decision-card-interactive {
background: var(--surface);
border: 2px solid var(--stop);
border-radius: var(--radius);
border: 1px solid color-mix(in srgb, var(--stop) 55%, var(--line));
border-radius: var(--radius-lg);
box-shadow: var(--shadow-md), 0 0 0 3px var(--stop-bg);
padding: 16px;
margin-bottom: 20px;
}
.decision-card-interactive h3 {
margin-top: 0;
}
.timeline-header {
display: flex;
flex-wrap: wrap;
@@ -1162,9 +1167,31 @@ h2 {
}
@media (max-width: 719px) {
/* On a phone the decision stays one tap away while the owner reads the gate: the buttons stick to the bottom. */
.decision-actions {
position: sticky;
bottom: 0;
z-index: 5;
margin: 12px -16px -16px;
padding: 10px 16px calc(10px + env(safe-area-inset-bottom));
background: color-mix(in srgb, var(--surface) 92%, transparent);
backdrop-filter: blur(8px);
border-top: 1px solid var(--line);
border-radius: 0 0 var(--radius-lg) var(--radius-lg);
}
.decision-actions button {
flex: 1 1 100%;
min-height: 44px;
flex: 1 1 40%;
min-height: 48px;
font-size: 1rem;
}
.timeline-header h1 {
font-size: 1.3rem;
}
.overview-waiting .btn-link {
padding: 8px 12px;
}
.graph-canvas {
@@ -2023,3 +2050,267 @@ a.button-link.primary {
text-align: center;
color: var(--muted);
}
/* how an app's container runs right now */
.health-dot {
display: inline-block;
width: 9px;
height: 9px;
border-radius: 50%;
vertical-align: middle;
background: var(--queued);
}
.health-ok { background: var(--ok); box-shadow: 0 0 0 3px var(--ok-bg); }
.health-starting { background: var(--running); box-shadow: 0 0 0 3px var(--running-bg); }
.health-down { background: var(--error); box-shadow: 0 0 0 3px var(--error-bg); animation: pulse 1.6s ease-in-out infinite; }
.health-off { background: var(--queued); opacity: 0.6; }
@keyframes pulse {
50% { box-shadow: 0 0 0 6px transparent; }
}
/* an app's version history */
.history {
margin: 0;
padding: 0;
list-style: none;
}
.history li {
position: relative;
display: flex;
gap: 12px;
padding-bottom: 14px;
}
.history li:not(:last-child)::before {
content: '';
position: absolute;
top: 14px;
bottom: 0;
left: 5px;
width: 2px;
background: var(--line);
}
.history-mark {
flex: none;
width: 12px;
height: 12px;
margin-top: 5px;
border-radius: 50%;
background: var(--queued);
}
.history-updated .history-mark { background: var(--ok); }
.history-kept .history-mark { background: var(--stop); }
.history-failed .history-mark { background: var(--error); }
.history-body {
flex: 1;
min-width: 0;
}
.history-head {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 6px 10px;
}
.history-when {
margin-left: auto;
font-size: 0.85em;
}
.history-body p,
.history-body details {
margin: 4px 0 0;
font-size: 0.9em;
}
/* the report on top of the overview */
.report .panel-body {
padding-top: 10px;
padding-bottom: 10px;
}
.report-lines {
display: grid;
gap: 4px;
margin: 0;
padding-left: 18px;
}
/* settings */
.settings-form {
display: grid;
gap: 16px;
max-width: 560px;
}
.settings-form fieldset {
margin: 0;
padding: 12px 16px;
border: 1px solid var(--line);
border-radius: var(--radius);
}
.settings-form legend {
padding: 0 6px;
font-weight: 600;
}
.settings-form p {
margin: 0 0 8px;
}
.settings-row {
display: flex;
flex-wrap: wrap;
align-items: end;
gap: 12px 20px;
}
.settings-row label {
display: grid;
gap: 4px;
}
.settings-row label.check {
display: flex;
align-items: center;
gap: 8px;
}
input[type='time'] {
padding: 5px 8px;
border: 1px solid var(--line);
border-radius: var(--radius);
background: var(--surface);
font: inherit;
color: inherit;
}
/* go-to palette (Ctrl+K) */
.search-button {
display: flex;
align-items: center;
gap: 10px;
padding: 7px 10px;
color: var(--muted);
background: var(--surface-2);
text-align: left;
}
.search-button .nav-label {
flex: 1;
}
kbd {
padding: 1px 6px;
border: 1px solid var(--line);
border-bottom-width: 2px;
border-radius: 5px;
font: 0.75rem var(--mono);
color: var(--muted);
background: var(--surface);
}
.palette-backdrop {
align-items: flex-start;
padding-top: 12vh;
}
.palette {
width: min(620px, calc(100% - 32px));
background: var(--surface);
border: 1px solid var(--line);
border-radius: var(--radius-lg);
box-shadow: var(--shadow-lg);
overflow: hidden;
animation: pop-in 0.15s ease-out;
}
.palette-input {
display: flex;
align-items: center;
gap: 10px;
padding: 12px 16px;
border-bottom: 1px solid var(--line);
color: var(--muted);
}
.palette-input input {
flex: 1;
padding: 4px 0;
border: 0;
outline: none;
background: transparent;
font-size: 1.05rem;
}
.palette-list {
max-height: 50vh;
margin: 0;
padding: 6px;
overflow-y: auto;
list-style: none;
}
.palette-list li {
display: flex;
align-items: center;
gap: 10px;
padding: 8px 10px;
border-radius: var(--radius);
cursor: pointer;
}
.palette-list li.active {
background: var(--accent-soft);
}
.palette-list li.active .icon {
color: var(--accent);
}
.palette-label {
font-weight: 600;
white-space: nowrap;
}
.palette-hint {
overflow: hidden;
color: var(--muted);
font-size: 0.85em;
text-overflow: ellipsis;
white-space: nowrap;
}
.palette-empty {
margin: 0;
padding: 16px;
color: var(--muted);
text-align: center;
}
@media (max-width: 719px) {
.search-button kbd {
display: none;
}
}
.page-actions {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 10px;
}
.page-actions > button {
display: inline-flex;
align-items: center;
gap: 6px;
}
+7 -4
View File
@@ -17,6 +17,7 @@ interface Draft {
releaseNotesExecutor: string;
migrateExecutor: string;
postUpgradeExecutor: string;
postUpgradeWhen: string;
promptHint: string;
notes: string;
}
@@ -31,6 +32,7 @@ const toDraft = (config: AppConfigView): Draft => ({
releaseNotesExecutor: config.releaseNotesExecutor,
migrateExecutor: config.migrateExecutor ?? 'off',
postUpgradeExecutor: config.postUpgradeExecutor ?? 'off',
postUpgradeWhen: config.postUpgradeWhen,
promptHint: config.promptHint ?? '',
notes: config.notes ?? '',
});
@@ -102,7 +104,7 @@ function ConfigForm({ initial }: { initial: AppConfigView }) {
<input type="text" name={key} value={draft[key]} disabled={pending} onInput={(event) => set(key, event.currentTarget.value)} />
</label>
);
const select = (key: 'auto' | 'releaseNotesExecutor' | 'migrateExecutor' | 'postUpgradeExecutor', label: string, options: readonly string[]) => (
const select = (key: 'auto' | 'releaseNotesExecutor' | 'migrateExecutor' | 'postUpgradeExecutor' | 'postUpgradeWhen', label: string, options: readonly string[]) => (
<label>
{label}
<select name={key} value={draft[key]} disabled={pending} onChange={(event) => set(key, event.currentTarget.value)}>
@@ -153,9 +155,10 @@ function ConfigForm({ initial }: { initial: AppConfigView }) {
</div>
{text('httpUrl', 'Check URL', 'empty = none')}
{text('expectStatus', 'Expected HTTP status')}
{select('releaseNotesExecutor', 'Release notes executor', ['sonnet', 'opus'])}
{select('migrateExecutor', 'Repair executor', ['off', 'sonnet', 'opus'])}
{select('postUpgradeExecutor', 'Post-upgrade check executor', ['off', 'sonnet', 'opus'])}
{select('releaseNotesExecutor', 'Release notes executor', ['haiku', 'sonnet', 'opus'])}
{select('migrateExecutor', 'Repair executor', ['off', 'haiku', 'sonnet', 'opus'])}
{select('postUpgradeExecutor', 'Post-upgrade check executor', ['off', 'haiku', 'sonnet', 'opus'])}
{draft.postUpgradeExecutor === 'off' ? null : select('postUpgradeWhen', 'Post-upgrade check runs', ['when_needed', 'always'])}
{text('promptHint', 'Prompt hint')}
{text('notes', 'Notes', 'up to 10 words')}
<div class="actions">
+140 -10
View File
@@ -1,19 +1,23 @@
import { useState } from 'preact/hooks';
import { Empty, ErrorNote, Loading, ModeTag, RunBadge } from '../components.tsx';
import { useContext, useState } from 'preact/hooks';
import { useConfirm, useToast } from '../components/dialogs.tsx';
import { Empty, ErrorNote, Loading, ModeTag, RunBadge, When } from '../components.tsx';
import { Card, Icon } from '../components/ui.tsx';
import { Chart } from '../components/chart.tsx';
import { durationOption } from '../charts/options.ts';
import { runStatusLabel } from '../labels.ts';
import { recentAppRuns, runDurations } from '../stats.ts';
import { appCondition, appsWithUpdate, recentAppRuns, runDurations, troubleCount, TROUBLE_THRESHOLD, versionHistory } from '../stats.ts';
import type { AppCondition, HistoryKind } from '../stats.ts';
import { api } from '../api.ts';
import { formatCost, formatInstant, runVersions } from '../format.ts';
import { useResource } from '../hooks.ts';
import { LiveContext, useResource } from '../hooks.ts';
import { hrefFor } from '../route.ts';
import { AppConfigPanel } from './app-config.tsx';
import { SkillPanel } from './skill.tsx';
import { RunButton, ScheduleCell } from './workflows.tsx';
import { appWorkflowId } from '../../domain/workflow-run-summary.ts';
import type { AppOverview, WorkflowInstanceSummaryResponse } from '../../http/contract.ts';
import type { AppOverview, WorkflowInstanceSummaryResponse, WorkflowModelCallResponse } from '../../http/contract.ts';
type HistoryCall = Pick<WorkflowModelCallResponse, 'instanceId' | 'kind' | 'outcome' | 'summary'>;
/** A workflow run's status and where it is or ended (terminal_updated, terminal_dry_run, a gate…). */
function RunStatus({ run }: { readonly run: WorkflowInstanceSummaryResponse }) {
@@ -53,10 +57,32 @@ function useLayout(): [AppsLayout, (layout: AppsLayout) => void] {
return [layout, choose];
}
type AppHealthData = NonNullable<Awaited<ReturnType<typeof api.hostStatus>>['status']>['apps'][number];
interface AppsData {
readonly apps: readonly AppOverview[];
readonly runs: readonly WorkflowInstanceSummaryResponse[] | null;
readonly nextRun: (app: string) => string | null;
/** The app's container as the driver last saw it; null when unknown. */
readonly health: (app: string) => AppHealthData | null;
/** Real runs among the app's last ten that failed, left it as it was or needed a repair. */
readonly trouble: (app: string) => number;
}
const CONDITION_LABEL: Readonly<Record<AppCondition, string>> = { ok: 'running', starting: 'starting', down: 'down', off: 'off' };
/** A dot for how the app's container runs right now. */
function HealthDot({ health }: { readonly health: AppHealthData | null }) {
if (health === null) return null;
const condition = appCondition(health);
const title = `${CONDITION_LABEL[condition]}: ${health.state}${health.health === null ? '' : `, ${health.health}`}`;
return (
<>
<span class={`health-dot health-${condition}`} title={title}>
<span class="sr-only">{title}</span>
</span>{' '}
</>
);
}
/** The last runs of an app as dots, oldest on the left, each a link to its run. */
@@ -104,7 +130,9 @@ function AppCard({ group, data }: { readonly group: readonly AppOverview[]; read
<header class="app-card-head">
<span class="app-avatar" aria-hidden="true">{first.app.slice(0, 2)}</span>
<div class="app-card-title">
<a href={hrefFor({ view: 'app', app: first.app })}>{first.app}</a>
<span>
<HealthDot health={data.health(first.app)} /><a href={hrefFor({ view: 'app', app: first.app })}>{first.app}</a>
</span>
<span class="muted">
{first.enabled ? `auto: ${first.auto ?? '—'}` : first.configured ? 'disabled' : 'not configured'}
</span>
@@ -112,9 +140,16 @@ function AppCard({ group, data }: { readonly group: readonly AppOverview[]; read
<ModeTag mode={first.apply ? 'real' : 'shadow'} />
</header>
<Versions run={run} />
{data.trouble(first.app) >= TROUBLE_THRESHOLD ? (
<span class="badge badge-warning" title="Failed, not updated or repaired by the model in several of the last ten real runs">
often needs care
</span>
) : null}
{members.map((member) => (
<div key={member.app} class="app-member">
<a href={hrefFor({ view: 'app', app: member.app })}>└ {member.service ?? member.app}</a>
<span>
<HealthDot health={data.health(member.app)} /><a href={hrefFor({ view: 'app', app: member.app })}>└ {member.service ?? member.app}</a>
</span>
<Versions run={latest(member.app)} />
</div>
))}
@@ -124,7 +159,7 @@ function AppCard({ group, data }: { readonly group: readonly AppOverview[]; read
</div>
<footer class="app-card-foot">
<span class="muted">
<Icon name="clock" size={14} /> {formatInstant(data.nextRun(first.app))}
<Icon name="clock" size={14} /> <When iso={data.nextRun(first.app)} />
</span>
{first.enabled ? <RunButton app={first.app} mode="shadow" /> : null}
</footer>
@@ -179,6 +214,7 @@ function AppsTable({ data }: { readonly data: AppsData }) {
return (
<tr key={entry.app} class={[entry.enabled ? '' : 'row-off', member ? 'row-member' : ''].join(' ').trim()}>
<td>
<HealthDot health={data.health(entry.app)} />
<a href={hrefFor({ view: 'app', app: entry.app })}>{member ? `└ ${entry.service ?? entry.app}` : entry.app}</a>
{entry.enabled ? null : <span class="muted"> · {entry.configured ? 'disabled' : 'not configured'}</span>}
</td>
@@ -199,11 +235,55 @@ function AppsTable({ data }: { readonly data: AppsData }) {
);
}
/**
* Starts a dry run of every enabled app whose latest run found a newer image, one after another. A dry run changes
* nothing; an app with a run still active is skipped and counted.
*/
function DryRunUpdates({ apps, runs }: { readonly apps: readonly AppOverview[]; readonly runs: readonly WorkflowInstanceSummaryResponse[] }) {
const confirm = useConfirm();
const toast = useToast();
const { refresh } = useContext(LiveContext);
const [busy, setBusy] = useState(false);
const targets = appsWithUpdate(apps, runs);
if (targets.length === 0) return null;
const start = async () => {
const ok = await confirm({
title: `Dry run ${String(targets.length)} ${targets.length === 1 ? 'app' : 'apps'} with an update?`,
message: `${targets.join(', ')}. A dry run checks the new version and the release notes and changes nothing.`,
confirmLabel: 'Start dry runs',
});
if (!ok) return;
setBusy(true);
let started = 0;
const skipped: string[] = [];
for (const app of targets) {
try {
await api.startWorkflowRun(app, 'shadow');
started += 1;
} catch {
skipped.push(app);
}
}
setBusy(false);
refresh();
toast(skipped.length === 0 ? `Started ${String(started)} dry runs.` : `Started ${String(started)} dry runs; skipped ${skipped.join(', ')} (a run is still active).`, skipped.length === 0 ? 'ok' : 'error');
};
return (
<button type="button" disabled={busy} onClick={() => void start()}>
<Icon name="upgrade" size={16} /> {busy ? 'Starting…' : `Dry run ${String(targets.length)} with updates`}
</button>
);
}
export function AppsView() {
const { data, error } = useResource(() => api.apps(), []);
const runs = useResource(() => api.workflowInstances(), []);
const schedules = useResource(() => api.workflowSchedules(), []);
const host = useResource(() => api.hostStatus(), []);
const [layout, setLayout] = useLayout();
const health = (app: string) => host.data?.status?.apps.find((entry) => entry.app === app) ?? null;
const calls = useResource(() => api.modelCalls(500), []);
const trouble = (app: string) => troubleCount(versionHistory((runs.data ?? []).filter((run) => run.app === app), calls.data?.calls ?? []));
const nextRun = (app: string) => schedules.data?.find((entry) => entry.workflowId === appWorkflowId(app))?.next[0] ?? null;
return (
<section>
@@ -212,6 +292,8 @@ export function AppsView() {
<h1>Apps</h1>
<p>What each app runs, what is available and how its updates went.</p>
</div>
<div class="page-actions">
{data === null || runs.data === null ? null : <DryRunUpdates apps={data} runs={runs.data} />}
<div class="segmented" role="group" aria-label="Layout">
<button type="button" aria-pressed={layout === 'cards'} onClick={() => setLayout('cards')}>
<Icon name="grid" size={16} /> Cards
@@ -220,6 +302,7 @@ export function AppsView() {
<Icon name="list" size={16} /> Table
</button>
</div>
</div>
</div>
{error === null ? null : <ErrorNote message={error} />}
{data === null ? (
@@ -227,9 +310,9 @@ export function AppsView() {
) : data.length === 0 ? (
<Empty>No apps configured yet.</Empty>
) : layout === 'cards' ? (
<AppCards data={{ apps: data, runs: runs.data, nextRun }} />
<AppCards data={{ apps: data, runs: runs.data, nextRun, health, trouble }} />
) : (
<AppsTable data={{ apps: data, runs: runs.data, nextRun }} />
<AppsTable data={{ apps: data, runs: runs.data, nextRun, health, trouble }} />
)}
</section>
);
@@ -237,6 +320,51 @@ export function AppsView() {
const DURATION_RUNS = 20;
const HISTORY_LABEL: Readonly<Record<HistoryKind, string>> = { updated: 'Updated', kept: 'Not updated', failed: 'Failed' };
const HISTORY_SHOWN = 15;
/** The app's real runs as a version history: what it ran before and after, and what the release notes said. */
function VersionHistory({ runs, calls }: { readonly runs: readonly WorkflowInstanceSummaryResponse[]; readonly calls: readonly HistoryCall[] }) {
const history = versionHistory(runs, calls);
const trouble = troubleCount(history);
return (
<Card title="Version history" icon="upgrade" actions={trouble >= TROUBLE_THRESHOLD ? <span class="badge badge-warning">{trouble} of the last {Math.min(history.length, 10)} real runs needed care</span> : undefined}>
{history.length === 0 ? (
<p class="muted">No real update yet: dry runs do not change the app.</p>
) : (
<ol class="history">
{history.slice(0, HISTORY_SHOWN).map((entry) => (
<li key={entry.runId} class={`history-${entry.kind}`}>
<span class="history-mark" aria-hidden="true" />
<div class="history-body">
<div class="history-head">
<strong>{HISTORY_LABEL[entry.kind]}</strong>
<span class="versions">
<code class="version">{entry.from ?? '?'}</code>
<Icon name="arrow" size={14} />
<code class={`version${entry.kind === 'updated' ? ' version-new' : ''}`}>{entry.to ?? '?'}</code>
</span>
{entry.repaired ? <span class="badge badge-warning">repaired by the model</span> : null}
<a class="muted history-when" href={hrefFor({ view: 'run', runId: entry.runId })}>
<When iso={entry.at} />
</a>
</div>
{entry.kind === 'updated' || entry.outcome === null ? null : <p class="muted">{entry.outcome}</p>}
{entry.releaseNotes === null ? null : (
<details>
<summary>Release notes: {entry.releaseNotes.verdict}</summary>
<p>{entry.releaseNotes.summary}</p>
</details>
)}
</div>
</li>
))}
</ol>
)}
</Card>
);
}
/** How long the app's last runs took: a run that suddenly takes much longer is worth a look. */
function DurationChart({ runs }: { readonly runs: readonly WorkflowInstanceSummaryResponse[] }) {
const durations = runDurations(runs, DURATION_RUNS);
@@ -262,6 +390,7 @@ export function AppDetailView({ app }: { app: string }) {
const runs = useResource(() => api.workflowInstances({ app }), [app]);
const schedules = useResource(() => api.workflowSchedules(), []);
const schedule = schedules.data?.find((entry) => entry.workflowId === appWorkflowId(app));
const calls = useResource(() => api.modelCalls(500), []);
return (
<section>
<p class="crumbs">
@@ -319,6 +448,7 @@ export function AppDetailView({ app }: { app: string }) {
</>
)}
</Card>
{runs.data === null ? null : <VersionHistory runs={runs.data} calls={calls.data?.calls ?? []} />}
<Card title="Configuration" icon="workflows">
<AppConfigPanel app={app} />
</Card>
+1 -1
View File
@@ -176,7 +176,7 @@ export function ModelsView() {
</button>
</div>
<p class="muted">
Apps pick a model as <code>sonnet</code> or <code>opus</code>: the profile of that name decides which sidecar runs it
Apps pick a model as <code>haiku</code>, <code>sonnet</code> or <code>opus</code>: the profile of that name decides which sidecar runs it
(Provider: Claude or Codex), how long it may take and what it may cost. A run keeps the profile version it started with.
</p>
+79 -7
View File
@@ -1,12 +1,14 @@
import { api } from '../api.ts';
import { Empty, ErrorNote, Loading, ModeTag, RunBadge } from '../components.tsx';
import { Empty, ErrorNote, Loading, ModeTag, RunBadge, When } from '../components.tsx';
import { formatInstant } from '../format.ts';
import { useResource } from '../hooks.ts';
import { hrefFor } from '../route.ts';
import { Card, StatTile } from '../components/ui.tsx';
import { Chart } from '../components/chart.tsx';
import { activityOption, formatUsd, scheduleOption, sparklineOption } from '../charts/options.ts';
import { appsWithUpdate, costPerDay, RUN_BUCKETS, runsPerDay, successRate } from '../stats.ts';
import { appsWithUpdate, costPerDay, hostSummary, RUN_BUCKETS, runsPerDay, successRate } from '../stats.ts';
import { downApps } from '../../domain/host-status.ts';
import { nightDigest } from '../../domain/notification-settings.ts';
import type { SuccessRate } from '../stats.ts';
import type { AppOverview, WorkflowInstanceSummaryResponse, WorkflowScheduleResponse } from '../../http/contract.ts';
@@ -32,7 +34,7 @@ function Waiting({ runs }: { readonly runs: readonly WorkflowInstanceSummaryResp
<strong><RunName run={run} /></strong>
<span>{run.step ?? 'Owner decides'}</span>
<ModeTag mode={run.mode} />
<span class="muted">since {formatInstant(run.updatedAt)}</span>
<span class="muted">since <When iso={run.updatedAt} /></span>
<OpenRun run={run} label="Decide →" />
</li>
))}
@@ -67,7 +69,7 @@ function Recent({ runs }: { readonly runs: readonly WorkflowInstanceSummaryRespo
<RunBadge status={run.status} />
<span>{run.outcome ?? ''}</span>
<ModeTag mode={run.mode} />
<span class="muted">{formatInstant(run.updatedAt)}</span>
<span class="muted"><When iso={run.updatedAt} /></span>
<OpenRun run={run} label="Open" />
</li>
))}
@@ -85,7 +87,7 @@ function Scheduled({ schedules }: { readonly schedules: readonly WorkflowSchedul
{upcoming.map((schedule) => (
<li key={schedule.workflowId} class="overview-item">
<strong><a href={hrefFor({ view: 'workflow', workflowId: schedule.workflowId })}>{schedule.workflowId}</a></strong>
<span>{formatInstant(schedule.next[0] ?? null)}</span>
<span><When iso={schedule.next[0] ?? null} /></span>
<ModeTag mode={schedule.mode} />
{schedule.lastResult === null ? null : <span class="muted">last: {schedule.lastResult}</span>}
</li>
@@ -117,7 +119,49 @@ function RateTrend({ rate }: { readonly rate: SuccessRate }) {
);
}
function Stats({ runs, apps, calls }: { readonly runs: readonly WorkflowInstanceSummaryResponse[]; readonly apps: readonly AppOverview[] | null; readonly calls: ModelCallsResponse | null }) {
type HostStatusData = Awaited<ReturnType<typeof api.hostStatus>>['status'];
/** Minutes after which the driver's look counts as old: it looks once a minute. */
const STALE_MINUTES = 5;
function AppsUp({ status }: { readonly status: HostStatusData | undefined }) {
if (status === undefined) return <StatTile label="Apps running" value="…" icon="apps" />;
if (status === null) return <StatTile label="Apps running" value="—" icon="apps" hint="the driver has not looked yet" />;
if (status.error !== null) return <StatTile label="Apps running" value="?" icon="apps" tone="error" hint={status.error} />;
const summary = hostSummary(status, Date.now());
const stale = summary.ageMinutes >= STALE_MINUTES;
const hint = stale
? `last look ${String(summary.ageMinutes)} min ago`
: summary.problems.length === 0
? 'all up'
: summary.problems.slice(0, 3).map((item) => `${item.app} ${item.state}`).join(', ') + (summary.problems.length > 3 ? '…' : '');
const down = summary.problems.some((item) => item.condition === 'down');
return (
<StatTile
label="Apps running"
value={`${String(summary.up)}/${String(summary.total)}`}
icon="apps"
tone={stale ? 'stop' : down ? 'error' : summary.problems.length > 0 ? 'running' : 'ok'}
hint={hint}
href={hrefFor({ view: 'apps' })}
/>
);
}
function hoursAgo(hours: number): string {
return hours < 1 ? 'under 1 h ago' : hours < 48 ? `${String(Math.round(hours))} h ago` : `${String(Math.round(hours / 24))} days ago`;
}
function VmBackup({ status }: { readonly status: HostStatusData | undefined }) {
const backup = status?.backup;
if (backup === undefined) return <StatTile label="VM backup" value="…" icon="clock" />;
if (backup === null) return <StatTile label="VM backup" value="—" icon="clock" hint="no record from the Proxmox timer" />;
if (!backup.ok) return <StatTile label="VM backup" value="!" icon="clock" tone="error" hint={backup.detail} />;
// A nightly backup: older than a day means last night's did not happen; real updates refuse past 48 h.
return <StatTile label="VM backup" value={hoursAgo(backup.ageHours)} icon="clock" tone={backup.ageHours > 26 ? 'stop' : 'ok'} hint={formatInstant(backup.createdAt)} />;
}
function Stats({ runs, apps, calls, host }: { readonly runs: readonly WorkflowInstanceSummaryResponse[]; readonly apps: readonly AppOverview[] | null; readonly calls: ModelCallsResponse | null; readonly host: HostStatusData | undefined }) {
const now = Date.now();
const waiting = runs.filter((run) => run.status === 'waiting').length;
const live = runs.filter((run) => LIVE.has(run.status)).length;
@@ -143,6 +187,8 @@ function Stats({ runs, apps, calls }: { readonly runs: readonly WorkflowInstance
hint={updates === null || updates.length === 0 ? 'every app is current' : updates.slice(0, 3).join(', ') + (updates.length > 3 ? '…' : '')}
href={hrefFor({ view: 'apps' })}
/>
<AppsUp status={host} />
<VmBackup status={host} />
</div>
);
}
@@ -179,12 +225,37 @@ function Upcoming({ schedules }: { readonly schedules: readonly WorkflowSchedule
);
}
const DAY_MS = 24 * 60 * 60 * 1000;
/** The morning report as a card: what needs the owner and what changed since the last report went out. */
function SinceLastReport({ runs, host, lastDigestAt }: { readonly runs: readonly WorkflowInstanceSummaryResponse[]; readonly host: HostStatusData | undefined; readonly lastDigestAt: string | null }) {
const since = lastDigestAt ?? new Date(Date.now() - DAY_MS).toISOString();
const digest = nightDigest({ runs, since, down: host === undefined || host === null ? [] : downApps(host) });
return (
<Card
title={digest.needsYou > 0 ? `${String(digest.needsYou)} ${digest.needsYou === 1 ? 'thing needs' : 'things need'} you` : 'All good'}
icon={digest.needsYou > 0 ? 'alert' : 'check'}
tone={digest.needsYou > 0 ? 'attention' : undefined}
actions={<span class="muted">since {lastDigestAt === null ? 'yesterday' : `the report of ${formatInstant(lastDigestAt)}`}</span>}
class="report"
>
<ul class="report-lines">
{digest.detail.split('\n').map((line) => (
<li key={line}>{line}</li>
))}
</ul>
</Card>
);
}
/** The home page: what waits for the owner, how runs went lately, what they cost and what runs next. */
export function OverviewView() {
const runs = useResource(() => api.workflowInstances(), []);
const schedules = useResource(() => api.workflowSchedules(), []);
const apps = useResource(() => api.apps(), []);
const calls = useResource(() => api.modelCalls(500), []);
const host = useResource(() => api.hostStatus(), []);
const notifications = useResource(() => api.notificationSettings(), []);
const all = runs.data ?? [];
const since = Date.now() - RECENT_DAYS * 24 * 60 * 60 * 1000;
const recent = all.filter((run) => ENDED.has(run.status) && Date.parse(run.updatedAt) >= since).slice(0, RECENT_LIMIT);
@@ -202,7 +273,8 @@ export function OverviewView() {
runs.error === null ? <Loading /> : null
) : (
<>
<Stats runs={all} apps={apps.data} calls={calls.data} />
<SinceLastReport runs={all} host={host.data?.status ?? undefined} lastDigestAt={notifications.data?.lastDigestAt ?? null} />
<Stats runs={all} apps={apps.data} calls={calls.data} host={host.data === null ? (host.error === null ? undefined : null) : host.data.status} />
<div class="panel-grid">
<Card title="Needs your decision" icon="alert" tone={waiting.length > 0 ? 'attention' : undefined}>
<Waiting runs={waiting} />
+88
View File
@@ -0,0 +1,88 @@
import { useState } from 'preact/hooks';
import { api } from '../api.ts';
import { ErrorNote, Loading } from '../components.tsx';
import { Card } from '../components/ui.tsx';
import { useToast } from '../components/dialogs.tsx';
import { formatInstant } from '../format.ts';
import { useResource } from '../hooks.ts';
type Settings = Awaited<ReturnType<typeof api.notificationSettings>>['settings'];
function TimeField({ label, value, onChange }: { readonly label: string; readonly value: string; readonly onChange: (value: string) => void }) {
return (
<label>
{label}
<input type="time" value={value} required onInput={(event) => onChange((event.target as HTMLInputElement).value)} />
</label>
);
}
/** When the phone may ring, and when the morning report goes out. Remounted (by key) when the saved settings change. */
function NotificationsForm({ initial, timeZone, lastDigestAt }: { readonly initial: Settings; readonly timeZone: string; readonly lastDigestAt: string | null }) {
const [settings, setSettings] = useState(initial);
const [saving, setSaving] = useState(false);
const [error, setError] = useState<string | null>(null);
const toast = useToast();
const set = (patch: Partial<Settings>) => setSettings((current) => ({ ...current, ...patch }));
const save = async (event: Event) => {
event.preventDefault();
setSaving(true);
setError(null);
try {
await api.saveNotificationSettings(settings);
toast('Notification settings saved.');
} catch (cause) {
setError(cause instanceof Error ? cause.message : 'Cannot save the settings');
} finally {
setSaving(false);
}
};
return (
<form class="settings-form" onSubmit={(event) => void save(event)}>
<fieldset>
<legend>Quiet hours</legend>
<p class="muted">Nothing rings in these hours: gates and failures wait for the morning report. A failure in the evening before still rings.</p>
<div class="settings-row">
<TimeField label="From" value={settings.quietFrom} onChange={(quietFrom) => set({ quietFrom })} />
<TimeField label="Until" value={settings.quietUntil} onChange={(quietUntil) => set({ quietUntil })} />
</div>
</fieldset>
<fieldset>
<legend>Morning report</legend>
<p class="muted">One push a day: what waits for you, what failed or is down, what was updated since the last report.</p>
<div class="settings-row">
<label class="check">
<input type="checkbox" checked={settings.digestEnabled} onChange={(event) => set({ digestEnabled: (event.target as HTMLInputElement).checked })} /> Send the report
</label>
<TimeField label="At" value={settings.digestAt} onChange={(digestAt) => set({ digestAt })} />
</div>
<p class="muted">Last report: {lastDigestAt === null ? 'none yet' : formatInstant(lastDigestAt)}</p>
</fieldset>
<p class="muted">Times are in {timeZone}, like the schedules.</p>
{error === null ? null : <ErrorNote message={error} />}
<div class="actions">
<button type="submit" class="primary" disabled={saving}>
{saving ? 'Saving…' : 'Save'}
</button>
</div>
</form>
);
}
export function SettingsView() {
const { data, error } = useResource(() => api.notificationSettings(), []);
return (
<section>
<div class="page-head">
<div>
<h1>Settings</h1>
<p>How and when the app tells you what happened.</p>
</div>
</div>
<Card title="Notifications" icon="bell">
{error === null ? null : <ErrorNote message={error} />}
{data === null ? (error === null ? <Loading /> : null) : <NotificationsForm key={JSON.stringify(data.settings)} initial={data.settings} timeZone={data.timeZone} lastDigestAt={data.lastDigestAt} />}
</Card>
</section>
);
}