From 74c6445e0ecfad62fc4b804e742cbcce7f02118c Mon Sep 17 00:00:00 2001 From: Carlos Arias Date: Fri, 24 Jul 2026 07:12:29 -0400 Subject: [PATCH] docs(.memory): QA agent reference + Web Designer self-logging/QA-loop update --- .memory/qa-agent.md | 132 +++++++++++++++++++++++++++++++++++++++++ .memory/webdesigner.md | 22 ++++++- 2 files changed, 153 insertions(+), 1 deletion(-) create mode 100644 .memory/qa-agent.md diff --git a/.memory/qa-agent.md b/.memory/qa-agent.md new file mode 100644 index 0000000..60614a9 --- /dev/null +++ b/.memory/qa-agent.md @@ -0,0 +1,132 @@ +# The QA agent — automated site testing with a self-healing loop + +**Status:** Implemented · **Date:** 2026-07-24 +**Related:** [webdesigner.md](webdesigner.md) · [../.claude/skills/qa/SKILL.md](../.claude/skills/qa/SKILL.md) · [../agents/console/SETUP.md](../agents/console/SETUP.md) · [../agents/console/PLAN.md](../agents/console/PLAN.md) + +--- + +## What it is + +The QA agent tests the live site (carlosarias.co) — links, images, the contact form, the +API, SEO/meta, and accessibility — records findings in the admin, and **auto-fixes the safe +ones** by feeding the [Web Designer](webdesigner.md) queue. It runs on a **heartbeat** (hourly) +and on demand from `/admin/qa`. + +**The honest architecture.** The confined agent (`claude -p`, no Bash/WebFetch) *cannot make +HTTP requests or drive a browser*, and no headless browser is usable by the runner. Since the +site is **static** (Astro → baked HTML), that's fine: a **deterministic Node crawler inside +the console runner** does the actual testing (Node 22 global `fetch`, zero deps), findings are +stored via **PHP CLI + DB** (the runner has no DB driver), and a **thin LLM step** writes a +plain-English summary using the `qa` skill. The crawl is the source of truth; the LLM only +triages. + +## The closed loop + +``` +heartbeat (hourly) ──► runQa (crawl) ──► cja_qa_findings + │ + ├─ safe findings (alt, internal links) + │ └─ qa-autofix.php → cja_tasks (queued) + │ └─ Web Designer fixes → publishes + │ └─ self-logs to /changelog + │ └─ next QA run: green + └─ judgment calls (external links, etc.) + └─ shown in /admin/qa with an + instructions box → you dispatch a fix +``` + +The site went 14 warnings → 0 through this loop (sitemap, résumé portrait alt, blog cover alt +auto-fixed; GitHub link removed by hand). + +## What the crawler checks + +| check_type | Verifies | Severity | +|---|---|---| +| `page` | every route returns 200 + HTML | error | +| `link` | every internal `` resolves | error | +| `image` | every ``/og:image/icon loads | error | +| `external` | off-site links reachable | warning (bot-blocking is common) | +| `form` | `/api/contact/submit` alive (honeypot probe → 200) + rejects bad input (→ 422) | error/warning | +| `health` | `/api/health` → 200 & `db:"connected"` (read at `data.db` — API wraps in `{ok,data,error}`) | error | +| `seo` | each page has `` + description; canonical host = carlosarias.co; no duplicate titles | warning (error on wrong canonical) | +| `sitemap` | every route is in `sitemap.xml` | warning | +| `a11y` | images have alt — **empty/bare `alt` is flagged too** (valid only for decorative images) | warning | + +Route set = static list + DB slugs (`qa-routes.php`) + sitemap + discovered internal links. +HTML parsed with regex (clean static HTML). Each finding carries a templated `fix_hint`. + +## Auto-fix (safe findings) + +- Runs after every QA pass when `QA_AUTOFIX` ≠ `0` (default **on**). `qa-autofix.php` queues a + Web Designer `cja_tasks` row for each **open** finding in a **safe category** — + `['a11y', 'link']` (see `SAFE` in that file) — then the runner kicks `drainTasks()`. +- **Dedup:** skips a finding if a same-title task is queued/running *or was created in the last + 6 hours* — so hourly heartbeats never spam or loop on the same issue. +- Judgment calls (`external`, `image`, `seo`, `sitemap`, `form`, `health`, `page`) are **not** + auto-fixed — they wait in `/admin/qa` for a human decision. + +## Report + instructions (the human path) + +`/admin/qa` lists runs (counts, summary) and the latest run's findings grouped by severity. +Each open finding has: +- **Create fix task** → queues a Web Designer fix (uses the `fix_hint`). +- an **instructions input** → type how to handle it ("remove it", "use https://…", "reword") + and that becomes the Web Designer's brief instead of the default hint. This is where + decisions like a wrong external URL get made — in the admin, not in chat. +- **Ignore** → mark the finding handled. + +## Heartbeat + +In-process timer in `server.mjs` (the runner is always up, so no cron needed): +- `QA_HEARTBEAT_MIN` (default **60**, `0` disables) — set in the systemd unit. +- One pulse ~90s after each restart; then every N minutes. +- **Triage is gated** to save tokens: a scheduled run with 0 errors gets a cheap templated + summary (no LLM call); only manual runs or runs with errors get an LLM summary. + +## Storage & pruning + +- `cja_qa_runs` (status, trigger manual/scheduled, summary, counts JSON, timestamps) +- `cja_qa_findings` (run_id, check_type, severity, url, detail, fix_hint, status + open/fix_queued/ignored) — migration `api/db/migrations/013_create_cja_qa.sql`. +- `qa-finish.php` prunes to the **50 most recent runs** (hourly runs accumulate). + +## Files + +| File | Role | +|---|---| +| `agents/console/server.mjs` | `runQa`, `qaTriage`, `runQaFlow`, `POST /devconsole/qa/run`, heartbeat, `QA_AUTOFIX`/`QA_HEARTBEAT_MIN` | +| `api/db/migrations/013_create_cja_qa.sql` | runs + findings tables | +| `api/cli/qa-start.php` | open a run → run_id | +| `api/cli/qa-finish.php` | store findings (temp JSON file) + prune | +| `api/cli/qa-routes.php` | DB-driven slugs for the crawl | +| `api/cli/qa-autofix.php` | queue Web Designer fixes for safe findings (6h dedup) | +| `api/public/controllers/adminqa.php` | runs / findings / fix (optional instructions) / ignore | +| `app/src/pages/admin/qa.astro` | report page + instructions box + "Run QA now" | +| `.claude/skills/qa/SKILL.md` | what QA checks, severity model, gotchas | + +## Operating it + +- **Run now:** the "Run QA now" button, or `curl -s -X POST http://127.0.0.1:3011/devconsole/qa/run`. +- **Change cadence / disable heartbeat:** `QA_HEARTBEAT_MIN` in the unit (`0` = off), then + `systemctl daemon-reload && systemctl restart astroagent-console.service`. +- **Disable auto-fix:** `QA_AUTOFIX=0` in the unit env. +- **View:** `/admin/qa`. + +## Safety & gotchas + +- **Read-only against the live site** — no git, no build, no writes. The two contact-form + probes write nothing (the honeypot `company` field is filled on purpose; the validation + probe is rejected with 422). Verified: a full run leaves `cja_contact` untouched. +- **Fixes only ever go through the Web Designer queue** — build-gated, scoped commit, + git-revertable, self-logged. Auto-fix ≠ unsafe; a bad fix is one `git revert` away. +- **External links = warnings, never errors** (403/429/timeouts are usually bot-blocking; only + a clear 404/410/DNS failure is flagged). +- **Empty alt IS a finding.** `alt=""`/bare `alt` is valid only for *decorative* images; a + content image (photo, cover, screenshot) needs descriptive alt. (An earlier version wrongly + treated empty alt as a pass — corrected.) +- **Health check reads `data.db`** — the API wraps every response in `{ok, data, error}`. +- **Site-coupling:** `QA_BASE`, the static route list, and `carlosarias.co` are hard-coded in + `runQa` — same site-agnosticism gap noted for the Web Designer; parameterise from config for + a clean clone. +- **Timezone:** the server (and thus the runner's timestamps) is `America/New_York` as of + 2026-07-24 (was UTC). New Web Designer changelog stamps are Eastern. diff --git a/.memory/webdesigner.md b/.memory/webdesigner.md index c42a23f..32c1b0a 100644 --- a/.memory/webdesigner.md +++ b/.memory/webdesigner.md @@ -1,7 +1,7 @@ # The Web Designer — a core Astroagent console update **Status:** Implemented · **Date:** 2026-07-24 -**Related:** [../agents/console/PLAN.md](../agents/console/PLAN.md) · [../agents/console/SETUP.md](../agents/console/SETUP.md) · [../brand/BRAND.md](../brand/BRAND.md) · [../.claude/skills/brand/SKILL.md](../.claude/skills/brand/SKILL.md) +**Related:** [qa-agent.md](qa-agent.md) · [../agents/console/PLAN.md](../agents/console/PLAN.md) · [../agents/console/SETUP.md](../agents/console/SETUP.md) · [../brand/BRAND.md](../brand/BRAND.md) · [../.claude/skills/brand/SKILL.md](../.claude/skills/brand/SKILL.md) · [../.claude/skills/changelog/SKILL.md](../.claude/skills/changelog/SKILL.md) --- @@ -28,6 +28,26 @@ It coexists with two sibling surfaces built earlier in the same arc: The Web Designer is the *freeform, queued* path for everything else. +## Added after first write (2026-07-24) + +Two capabilities were added to the Web Designer after this doc's first version: + +- **Self-logs every change to the changelog.** `buildDesignPrompt` instructs the agent, after + finishing, to add one entry to `api/cli/seed-changelog.php` via the `changelog` skill, + attributed to **`Website Designer Agent`**, with the current timestamp (injected by the + runner via `nowStamp()` — a headless agent can't read the clock). Because the confined agent + has no shell, **the runner reseeds** `cja_changelog` (`php api/cli/seed-changelog.php`) when + that file changed, before the build, so the entry goes live. Changelog entries carry a + `by`/`actor` field (migration 012): agents name themselves; human/CLI edits default to + `Carlos Arias`. +- **Fed by the QA agent.** QA findings become Web Designer tasks — auto-queued for safe + categories, or dispatched from the `/admin/qa` instructions box. A QA-origin task is just a + normal `cja_tasks` row, so it fixes, publishes, and self-logs like any other. See + [qa-agent.md](qa-agent.md) for the closed loop. + +**Timezone note:** `nowStamp()` uses the runner's system timezone, set to `America/New_York` +on 2026-07-24 (was UTC, which mis-stamped early agent entries). + ## Architecture ```