Carlos Arias
2c969c0753
Adopt the agents/ architecture proven on medellin.co (reference impl): - Move the content engine to a top-level agents/ dir: orchestrators, prompts, config, run.sh, admin console, shared libs. All content-pipeline literals repointed (config paths, scripts, admin, LLM-facing prompts/image.md string, configure.mjs, new-site.sh, astroagent tokenFile, .gitignore runtime block). - Every script carries a parseable @agent-manifest header: name, title, class (content|operational|runtime|plumbing), trigger, model, prompts, skills (MCP), tools, reads/writes tables. 5 content agents + 3 plumbing scripts. - New agents/catalog.mjs generates the catalog from the headers: agents/AGENTS.md (human, grouped by class) + agents/agents.json (machine manifest — a clone diffs it against a source to find missing tools/tables/MCP before running). configure.mjs regenerates the catalog on every identity stamp. No DB table, no watcher. - config.json gains paths.stateDir/newsDir; publish-tick, write-daily, and news-radar read them instead of hardcoding. - Full cut: content-pipeline/ deleted (the seed has no live crons, so no hybrid period needed). Docs updated (AGENTS.md structure + pipeline section, README paths). Clones migrating from content-pipeline/: see medellin.co's .memory/handoffs/agents-directory-migration.md for the cutover playbook (one cron set active at a time; migrate drafts/state after repointing cron). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FMQeUnUrAeexcZ7P2Hxa6G
134 lines
7.8 KiB
Markdown
134 lines
7.8 KiB
Markdown
# Comiida Content Pipeline
|
|
|
|
An agentic pipeline that plans a 3-month editorial calendar and drafts one SEO/EEAT
|
|
restaurant article per day (English, for Medellín expats/tourists) with an AI cover image,
|
|
queued for human approval before going live on the Astro site.
|
|
|
|
Engine: **headless Claude Code** (`claude -p`) + **Higgsfield** image MCP (authenticated at
|
|
the claude.ai level — reachable headless, confirmed). See the design at
|
|
`/root/.claude/plans/init-elegant-giraffe.md`.
|
|
|
|
## Layout
|
|
- `config.json` — site, author, models, image, editorial mix/word-counts.
|
|
- `.env` — optional secrets (git-ignored). None required today (Higgsfield auth is global).
|
|
- `prompts/` — system prompts: `research.system.md`, `writer.system.md`, `image.md`.
|
|
- `scripts/` — `research.mjs`, `write-daily.mjs`, `list-drafts.mjs`, `approve.mjs`, `lib/`.
|
|
- `calendar.json` — the generated editorial calendar (created by `research.mjs`).
|
|
- `drafts/<slug>/` — pending drafts (`index.mdx`, `cover.jpg`, `sources.json`).
|
|
- `run.sh` — cron-safe wrapper (sets PATH, loads `.env`).
|
|
|
|
## Setup (one-time)
|
|
1. Add the real author to `app/src/lib/blog-data.js` and set `author.slug`/`name` in
|
|
`config.json` to match (EEAT requires a real byline).
|
|
2. Image generation needs no key — it uses the Higgsfield claude.ai MCP. Check credits with
|
|
the `balance` tool if generations start failing.
|
|
|
|
## Usage
|
|
```bash
|
|
# Plan ~3 months (writes calendar.json). Default days = config.editorial.calendarDays.
|
|
./run.sh research.mjs [days]
|
|
|
|
# Draft the next due article into drafts/<slug>/ (auto-runs the SEO audit after).
|
|
./run.sh write-daily.mjs [slug]
|
|
|
|
# SEO Specialist audit (EEAT / spam-policy / on-page / AEO / readability).
|
|
# Writes <postDir>/seo-review.json. Default audits the draft; --published audits the live post.
|
|
./run.sh seo-review.mjs <slug> [--published]
|
|
|
|
# Auto-revise a failing draft until it passes the SEO gate (or maxReviseAttempts).
|
|
./run.sh revise.mjs <slug>
|
|
|
|
# News radar — discover timely Medellín food news/events → news-queue.json.
|
|
# (also runs automatically inside write-daily, once per day)
|
|
./run.sh news-radar.mjs
|
|
|
|
# Research — top up the evergreen backlog (additive merge into calendar.json).
|
|
# (also runs automatically inside write-daily when the planned backlog is low)
|
|
./run.sh research.mjs [count]
|
|
|
|
# Review the queue, then publish one manually (SEO-gated; --force overrides).
|
|
node scripts/list-drafts.mjs
|
|
./run.sh approve.mjs <slug> [--force] # moves into the blog, builds, goes live
|
|
|
|
# Randomized auto-publisher (cron). Publishes ONE SEO-passing draft per day at a random time.
|
|
./run.sh publish-tick.mjs [--now] # --now ignores the time gate (for testing)
|
|
```
|
|
|
|
## Scheduling (managed in the aaPanel Cron UI; server clock is UTC)
|
|
Only **two** jobs are scheduled — research is now demand-driven (see below), not a cron.
|
|
```
|
|
0 11 * * * run.sh write-daily.mjs # daily 11:00 UTC (06:00 Bogotá) — news scan + refill + draft + SEO audit
|
|
*/15 * * * * run.sh publish-tick.mjs # every 15m — publish 1 passing draft in the morning (random timestamp)
|
|
```
|
|
(aaPanel runs these in UTC; `publish-tick` computes its Bogotá floor time internally.)
|
|
|
|
## Evergreen backlog is demand-driven (no monthly cron)
|
|
`research.mjs` is **additive** — it proposes evergreen topics and merges new ones into
|
|
`calendar.json` (deduped, dates auto-assigned). `write-daily` **auto-refills** when the
|
|
`planned` backlog drops below `editorial.refillThreshold` (guarded once/day), so the calendar
|
|
stays stocked without thinking in "months." Run manually any time: `./run.sh research.mjs [count]`.
|
|
|
|
## Concurrency safety
|
|
All `calendar.json` writes go through `lib/calendar.mjs` — an exclusive **lockfile** plus
|
|
atomic read-modify-write (`updateEntry`/`addEntry`/`mergeEntries`). This prevents the
|
|
lost-update race where a long `write-daily` run could clobber concurrent `publish-tick` /
|
|
research writes.
|
|
|
|
## Auto-publish: morning go-live, randomized timestamp (`config.json` → `publish`)
|
|
Posts go **live in the morning** (not a random hour all day), but each is stamped with a
|
|
**random earlier-today timestamp** so published times aren't a fixed-minute fingerprint:
|
|
- **06:00** `write-daily.mjs` drafts the day's topic, runs the SEO audit, and auto-revises.
|
|
- **`publish-tick.mjs`** (every 15 min) publishes **one** SEO-passing draft on the first tick
|
|
after the **`publish.notBefore`** floor (default `07:00` Bogotá — a buffer so the morning's
|
|
fresh/news article is the one that goes live). The post's `date` is set to a random time
|
|
between midnight and the publish moment (`randomTimestampTodaySoFar` — varied but never
|
|
future). News drafts fast-track ahead of evergreen. One publish/day; idempotent; SEO-failing
|
|
drafts wait for manual review. State in `state/publish-YYYY-MM-DD.json`.
|
|
- Set `publish.auto=false` to disable auto-publish and use manual `approve.mjs`.
|
|
|
|
## Admin dashboard (read-only)
|
|
A small Node service (`admin/server.mjs`, no deps) shows the week's scheduled topics, the
|
|
draft queue (with cover + preview), and the full calendar.
|
|
- Served at **`https://comiida.com/admin?key=<ADMIN_TOKEN>`** (Apache proxies `/admin` →
|
|
`127.0.0.1:3010`; token is in `.env`). A valid `?key=` sets a 30-day cookie.
|
|
- Kept alive by **supervisor**: program `comiidaAdmin`
|
|
(`/www/server/panel/plugin/supervisor/profile/comiidaAdmin.ini`).
|
|
Manage with: `supervisorctl -c /etc/supervisor/supervisord.conf {status|restart|stop} comiidaAdmin:comiidaAdmin_00`.
|
|
- Read-only by design — approving still happens via `./run.sh approve.mjs <slug>`.
|
|
|
|
## Daily news radar (timely news/events)
|
|
`scripts/news-radar.mjs` + `prompts/news-radar.system.md` discover **time-sensitive** Medellín
|
|
food items (openings/closings, festivals/events, awards, press) via WebSearch → `news-queue.json`
|
|
(deduped vs published + calendar; stale fresh items pruned past `news.recencyDays`).
|
|
- Runs **automatically at the start of `write-daily.mjs`**, guarded once/day via
|
|
`state/news-YYYY-MM-DD.json` (so the hourly test cron doesn't re-scan).
|
|
- **News-first selection:** the writer prefers the freshest queue item (by `freshnessScore`),
|
|
injects it into `calendar.json` as a `news:true` entry, and drafts it; on a quiet day it falls
|
|
back to the next evergreen calendar topic.
|
|
- **Fast-track publishing:** `publish-tick.mjs` releases `news:true` drafts before evergreen
|
|
ones (still 1/day, still SEO-gated, still randomized timestamp) — so timely pieces go live the
|
|
same day.
|
|
- Config: `config.json → news { enabled, recencyDays, maxPerScan }`.
|
|
|
|
## Auto-revise loop
|
|
When a fresh draft fails the SEO gate, `write-daily` automatically runs `scripts/revise.mjs`:
|
|
it feeds the audit's `blocking` + `topFixes` back to the writer (`prompts/reviser.system.md`),
|
|
which edits the draft in place (finding real sources via WebSearch for uncited claims — never
|
|
fabricating), then **re-audits**. Repeats until `pass` or `seo.maxReviseAttempts` (config).
|
|
Config: `seo.autoRevise`, `seo.maxReviseAttempts`. Verified: a draft at fail/82 → pass/90 in one pass.
|
|
|
|
## SEO Specialist agent
|
|
`scripts/seo-review.mjs` + `prompts/seo-review.system.md` audit an article across five
|
|
weighted dimensions — EEAT (30), spam-policy compliance (20), on-page SEO (25), AEO/answer-
|
|
engine (15), readability (10) — and write `seo-review.json` with per-dimension scores, a
|
|
verdict (`pass`/`revise`/`fail`), blocking issues, and a ranked fix list.
|
|
- Runs **automatically** at the end of `write-daily.mjs` (verdict also stored on the calendar
|
|
entry).
|
|
- **Gates `approve.mjs`**: approval is blocked when `verdict==="fail"` or `overall <
|
|
seo.minScore` (config; default 85). Override with `approve.mjs <slug> --force`.
|
|
- Verdict + scores + fixes show in the `/admin` dashboard (draft cards and the draft preview).
|
|
- Reviewer model: `models.reviewer` in `config.json`.
|
|
|
|
## EEAT guardrails (enforced in prompts)
|
|
Data-driven only; every claim cited; no fabricated first-person dining; real author byline;
|
|
honest "AI-generated illustration" image credit.
|