fix(core): Stop dependency index rebuild looping after import (#40356)

Co-authored-by: Cursor <cursoragent@cursor.com>

.agents/review-rules/README.md

@@ -17,7 +17,7 @@ clear, when to stay silent, what never to say — lives once in
 | `backend/`  | Backend  | `cli`, `@n8n/db`, `core`, `workflow`, node packages |
 | `db-migrations/` | DB migrations | `@n8n/db` migrations + their tests in `cli` |
 | `frontend/` | Frontend | `packages/frontend`                                 |
-| `qa-dx/`    | QA & DX  | `.github`, `docker`, `scripts`, `patches`, `packages/testing`, the lint/test/TS config packages, baselines |
+| `qa-dx/`    | QA & DX  | `.github`, `docker`, `scripts`, `patches`, `packages/quality`, the lint/test/TS config packages, baselines |
 | `testing/`  | Backend + Frontend | any package with a test suite            |
 
 A directory maps to one agent unless, like `testing/`, the policy is identical
@@ -27,7 +27,7 @@ never a copy per directory. Security and QA & DX deliberately don't link
 agents shouldn't be able to produce.
 
 All five slots are used, so a new domain now merges into an existing agent. QA &
-DX covers the build, test, and CI surface — the same paths `.github/OWNERS`
+DX covers the build, test, and CI surface — the same paths `OWNERS`
 assigns to `@n8n-io/qa-dx`. Code-quality rules that happen to apply broadly
 (error classes, `any`, lazy imports) are backend rules, not QA & DX ones. DB
 migrations is split out of Backend because a migration is permanent and runs

.agents/review-rules/qa-dx/ratchets-and-allowlists.md

@@ -13,8 +13,9 @@ Flag a diff that **adds** entries to any of these, and ask for the fix instead:
 |------|---------|
 | `.code-health-baseline.json` | `@n8n/code-health` violations |
 | `.boundaries-baseline.json` | `turbo boundaries` issue count |
-| `packages/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
+| `packages/quality/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
-| `packages/cli/eslint.config.mjs` | the `misplaced-n8n-typeorm-import` and public-API allowlists, each captioned "NEVER add to this list" |
+| `.code-health-baseline.json` | the `typeorm-persistence-boundary` baseline, which only shrinks as business logic moves queries into repositories |
+| `packages/cli/eslint.config.mjs` | the public-API allowlists captioned "NEVER add to this list" |
 | `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
 
 Removals are the healthy direction and need no comment.

.agents/review-rules/qa-dx/workflow-safety.md

@@ -35,7 +35,7 @@ fed from a `needs:` list. Flag:
 Jobs gated on a `ci-filter` filter only run when a matching file changed, so a
 filter that is too narrow means the test quietly stops covering new code. When
 a PR adds a directory that an existing filter was meant to cover — a new package
-under `packages/testing/`, a new script under `.github/scripts/` — check the
+under `packages/quality/`, a new script under `.github/scripts/` — check the
 filter still matches it.
 
 ## Least privilege

.agents/skills/community-pr-readiness-check/SKILL.md

@@ -113,7 +113,7 @@ Run when `AutoReject` is `null`. Full rules for each in `reference/checks.md`:
 
 ## Step 4 — Identify the responsible team
 
-Run `node .github/scripts/owners.mjs` against the changed file list and map the winning GitHub team to a Linear team. Full mapping table, sub-agent fallback procedure, and label rules: see `reference/teams.md`.
+Run `node .github/scripts/owners/owners.mjs` against the changed file list and map the winning GitHub team to a Linear team. Full mapping table, sub-agent fallback procedure, and label rules: see `reference/teams.md`.
 
 ## Step 5 — Extract the Linear ticket
 

.agents/skills/community-pr-readiness-check/reference/teams.md

@@ -16,7 +16,7 @@ Owner resolution, the GitHub-team → Linear-team → label mapping, and the Lin
 
 ## Identifying the owning team
 
-Use the canonical owners script at `.github/scripts/owners.mjs`. It parses `.github/OWNERS` with last-match-wins semantics and returns allocations sorted by file count with a `share` percentage. Using the script keeps this skill consistent with whatever CI uses.
+Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parses `OWNERS` with last-match-wins semantics and returns allocations sorted by file count with a `share` percentage. Using the script keeps this skill consistent with whatever CI uses.
 
 1. Write the PR's changed file paths (from the `files` list) to a temp file, one per line:
    ```bash
@@ -24,7 +24,7 @@ Use the canonical owners script at `.github/scripts/owners.mjs`. It parses `.git
    ```
 2. Run the script:
    ```bash
-   node .github/scripts/owners.mjs /tmp/pr-<number>-files.txt
+   node .github/scripts/owners/owners.mjs /tmp/pr-<number>-files.txt
    ```
 3. The script prints JSON of the form:
    ```json
@@ -38,10 +38,10 @@ Use the canonical owners script at `.github/scripts/owners.mjs`. It parses `.git
    ```
    Allocations are already sorted by `fileCount` descending — take the first entry as the winning team.
 4. Clean up: `rm /tmp/pr-<number>-files.txt`.
-5. Strip the `@n8n-io/` prefix from `allocations[0].team` — the GitHub team slug is `nodes`, `iam`, `ai`, etc. If `allocations` is empty (no file matched any rule, which is possible only if `.github/OWNERS` lost its catch-all), fall back to `catalysts`.
+5. Strip the `@n8n-io/` prefix from `allocations[0].team` — the GitHub team slug is `nodes`, `iam`, `ai`, etc. If `allocations` is empty (no file matched any rule, which is possible only if `OWNERS` lost its catch-all), fall back to `catalysts`.
 6. Map the GitHub team slug to its Linear team name and PR label using the table below. The `team` field in the JSON output is the **Linear team name**. If the resolved GitHub team slug has no entry in the table, fall back to `Engineering`.
 
-**Sub-agent fallback**: if `node` execution is denied by the sandbox, read `.github/OWNERS` directly and apply last-match-wins by hand. All the active rules fit on one screen.
+**Sub-agent fallback**: if `node` execution is denied by the sandbox, read `OWNERS` directly and apply last-match-wins by hand. All the active rules fit on one screen.
 
 ## GitHub team → Linear team → GitHub label
 

.agents/skills/create-agent-builder-eval/SKILL.md

@@ -20,7 +20,7 @@ Run this check before sourcing, drafting, or writing an eval. Run it from
 `packages/@n8n/instance-ai`:
 
 ```bash
-pnpm exec dotenvx run -f ../../../.env.local -- \
+pnpm exec dotenvx run -f ../../../.env.eval -- \
   sh -c 'test -n "${LANGTRACER_URL:-}" && test -n "${LANGTRACER_API_KEY:-}"'
 ```
 
@@ -28,7 +28,7 @@ If the check fails, stop before creating an eval file. Ask the user to:
 
 1. Generate a key on the
    [LangTracer API page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
-2. Add these variables to the repository root `.env.local` file:
+2. Add these variables to the repository root `.env.eval` file:
 
    ```env
    LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
@@ -45,9 +45,8 @@ value. Rerun the check after the user confirms. Continue only when it passes.
 - Author the case at
   `packages/@n8n/instance-ai/evaluations/data/agents/<slug>.json`.
 - Set `"datasets": ["agents"]`.
-- Push general Agent Builder cases to
+- Ask the driver for the target suite before you do anything else. See
-  [Instance AI capabilities — agents](https://lang-tracer.n8n-maintenance.workers.dev/suites/10).
+  "Ask for the target suite first" below. Do not assume `agents`.
-  Its suite slug is `agents`.
 - Do not commit the case JSON. LangTracer is the durable source of truth.
 - Commit changes to this skill, the harness, and CI when applicable.
 
@@ -70,9 +69,189 @@ or outcome expectations can prove the behavior.
 Use the substitution test for every expectation. A correct alternative build
 must pass. A build that misses the requested behavior must fail.
 
+## Write expectations about the Agent, not the builder's internals
+
+Agent Builder runs as a delegated sub-agent architecture today. That design
+can change. Do not write an expectation that depends on it. Follow
+"Keep expectations free of internal mechanics" in
+[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md).
+
+Do not write:
+
+- "The sub-agent runs/tests the Agent before it responds."
+- "Instance AI delegates to `build-agent`" or "calls `build-agent` once."
+- "The builder calls `call_agent` to verify the Agent."
+- "The sub-agent reads the skill, then sets the model."
+
+Write what a user or a reviewer can observe:
+
+| ❌ mechanical | ✅ intent |
+|---|---|
+| "The sub-agent tests the Agent before it reports" | "The final response says the Agent was tested only if a test ran, and it reports the result of that test honestly" |
+| "`build-agent` is called with the Slack credential" | "The Agent uses the Slack credential the user has" |
+| "The sub-agent adds a tool for the lookup" | "The Agent can look up the order status when a user asks for it" |
+| "The builder sets `model` after the catalog lookup" | "The Agent uses a model that the connected credential supports" |
+
+Put each one where it belongs:
+
+- The Agent's configuration, tools, skills, and behavior go in
+  `outcomeExpectations`.
+- What the user is told, asked, or shown goes in `processExpectations`.
+- The final response is the place to check claims ("says it is ready",
+  "lists the tools it added"). Do not check which component made the claim.
+
+The calibration steps below ask you to confirm that `build-agent` ran. That is
+a check for you, to be sure the case reached Agent Builder. Do not add it to the
+case.
+
 For multi-turn, seeded, or capability-gap cases, follow the case-shape and
 calibration rules in [create-instance-ai-eval](../create-instance-ai-eval/SKILL.md).
 This skill overrides its workflow directory and suite guidance for Agent cases.
+The sections below add the Agent-specific parts. Do not skip the general rules
+they point to.
+
+## Ask for the target suite first
+
+Follow "Ask for the target suite first" in
+[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md). Do it after the
+LangTracer preflight and before you source or draft anything. If the request
+already names a suite, state it and do not ask. Otherwise ask, in autonomous
+mode too, in the same message as the autonomy question.
+
+For Agent cases, recommend
+[Instance AI capabilities — agents](https://lang-tracer.n8n-maintenance.workers.dev/suites/10)
+(slug `agents`). Offer `baseline` as well: the Instance AI (INS) team monitors
+it and it runs nightly. Get the full list from `list_suites`. Use the suite the
+driver picks in every push command, in place of `agents` in the examples below.
+Keep the `agents` dataset and the `data/agents/` directory. They do not depend
+on the suite.
+
+## Set the autonomy level first
+
+Follow "Set the autonomy level first" in
+[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md). Ask the driver
+for *autonomous* or *checkpoint* mode before you source or draft anything. The
+four gates are the same: selection, shape and expectations, calibration, push.
+
+## Source the case from a real thread
+
+The best Agent cases come from a real conversation. Discover the conversation
+with LangTracer. Author a synthetic case from what you learn.
+
+1. List candidates: `list_conversations` with `usedAgentBuilder: true`. Do not
+   use `source: "agent-builder"`. That pool is a different product, has
+   unscrubbed payloads, and has no Agent snapshots.
+2. List what the thread did to the Agent: `list_conversation_agent_snapshots`.
+   `target-resolved` is the state that a turn opened on. `config-updated` is the
+   state after the builder changed it.
+3. Pick the shape from the snapshot sequence (next section).
+4. Fetch the state to seed with `get_conversation_agent_snapshot`. Seed from the
+   `target-resolved` row of the turn under test.
+5. Read the raw turn before you write any expectation. Follow
+   "First reproduce, then reclassify" in the general skill.
+
+Two snapshot patterns to look for:
+
+- A `target-resolved` hash that differs from the previous `config-updated` hash.
+  The user edited the Agent in the UI between turns. This is the best seed for a
+  repair or update case.
+- A run of turns that each end in `config-updated`. The user added one rule per
+  turn. Use it for a "new rule does not erase old rules" case.
+
+Credentials arrive as `[redacted]`. Instructions and skill bodies do not. Scrub
+the prose before it enters a case. See the `agents` section in
+[case-shapes.md](../create-instance-ai-eval/case-shapes.md) (the `agents` seed section).
+
+## Pick the shape: create or update
+
+| Shape | Question it answers | How to write it |
+|---|---|---|
+| **Create** | Does the builder make the right Agent from a fresh request? | One user turn. No seed. Assert on the Agent artifact. |
+| **Update** | Does the builder change only what the user asked for? | `seed.agents` with the Agent before the change. One live turn with the change. Assert on the new behavior and that the untouched parts survive. |
+| **Repair** | Does the builder find the cause of a failure and fix it? | Seed the broken Agent. The live turn reports the failure. Assert on the cause found and the fix applied. |
+| **Incremental rules** | Do earlier rules survive when a new rule lands? | Seed the Agent after several rules. Add one conflicting or overlapping rule. Assert that the old rules are still in force. |
+
+Rules that apply to all four:
+
+- An update that rebuilds the Agent from scratch is a failure. Assert on a
+  specific untouched instruction, tool, or skill that must still be there.
+- Write the conversation in English, even when the source thread is not.
+- Trim to the fewest turns that reproduce the behavior. Do not copy a long
+  thread.
+- Put only the live turn in `conversation`. Put earlier turns in `seed.messages`.
+
+## Confirm the precondition fired
+
+A pass can mean the builder did the right thing. It can also mean the situation
+never happened. Check before you trust a green.
+
+- For "the builder did not do X", confirm that it reached the turn where X was
+  possible. A builder that never edited the Agent passes "did not delete the old
+  rule".
+- For a repair case, confirm that the seeded Agent really has the defect. Read
+  the seeded config in the rendered artifact.
+- For an update case, compare the seeded config with the final config. Confirm
+  that the requested change is the only difference you expected.
+- For a "say you cannot" case, confirm that the tool or data was really missing
+  in the run.
+
+## A red is a finding
+
+Follow "A red is signal" in the general skill. For Agent cases, add this:
+
+1. Classify the red as a product gap, a harness limitation, or non-determinism.
+2. Keep a product-gap red. Start the case `description` with
+   `Capability-gap finding:`.
+3. **Propose a Linear ticket for the Agent team.** The Linear team is named
+   `Agent`. Propose the ticket. Do not create it unless the driver says go.
+   Follow "Capability gap → propose a Linear ticket" in the general skill. Also
+   check whether the gap already has a ticket.
+   - Write the ticket in English. Use Simplified Technical English, as
+     [AGENTS.md](../../../AGENTS.md) requires. Translate the user's words. Do
+     not paste them.
+   - This repository is public. Keep customer names, company names, and real
+     URLs out of the ticket. Describe the use case in neutral words.
+   - Link the LangTracer case and the source thread. Do not copy raw user
+     messages or agent instructions from the thread.
+   - Reproduce the gap first. This is a default, not a hard rule. Propose a
+     ticket only after a case is red on a real build. Quote the failing
+     expectation and the judge reason. Follow "Reproduce first" in the general
+     skill.
+   - The driver may approve an exception, for example when the harness cannot
+     reach the mechanism after about three attempts. Then the ticket must say
+     "Not reproduced in the eval. Based on trace analysis."
+4. **Check the builder's own guidance for the gap.** Read the files that steer
+   Agent Builder:
+   `packages/cli/src/modules/agents/builder/agents-builder-prompts.ts`,
+   `packages/cli/src/modules/agents/builder/prompts/`, and
+   `packages/cli/src/modules/agents/builder/skills/`. Also read the Instance AI
+   orchestrator prompt in `packages/@n8n/instance-ai/src/agent/system-prompt.ts`.
+   If none of them covers the failing situation, add that to the ticket as a
+   proposed prompt or skill change.
+5. Propose the smallest added line first. Do not delete existing guidance to fix a
+   gap. A measured case: adding one line fixed a routing gap, and deleting two
+   lines did not.
+6. Every prompt or skill change needs a before and after measurement. The new
+   case is the "before" run. Run it again after the change. Use `--iterations 5`.
+
+When the failure is in the Agent's runtime (for example a channel that cannot
+report an error), the builder cannot see it. The right fix is often a builder
+instruction to say so and stop guessing. Write that as a `processExpectations`
+item on the final response.
+
+## Publish and link
+
+Follow "Share links, never bare ids" and "Link the pushed case to its source" in
+the general skill. For a sourced Agent case:
+
+- Report the case, the suite, and the source thread as links. Use the
+  `https://lang-tracer.n8n-maintenance.workers.dev/test-cases/<id>` form.
+- Call `update_test_case` with `sourceThreadId`, `expectedBehavior`, and
+  `failurePattern`.
+- Add a capability tag with `add_case_tags`.
+- Push a case whose *build* is wrong with `--set-kind capability_gap` into a
+  suite of that kind. If no such Agent suite exists, tell the driver. Do not put
+  it in a `regression` suite.
 
 ## Draft the case
 
@@ -161,14 +340,14 @@ credential such as `EVAL_OPENAI_API_KEY`. A build-only case does not need one.
 4. Classify each red as a product gap, harness limitation, or non-determinism.
 5. Use `--iterations 5` before adding a case to a gating tier.
 6. Preview the LangTracer change.
-7. Push it to `agents`.
+7. Push it to the suite the driver chose (`<suite>` below).
 
 ```bash
-pnpm exec dotenvx run -f ../../../.env.local -- \
+pnpm exec dotenvx run -f ../../../.env.eval -- \
-  pnpm eval:langtracer-push --suite agents --dry-run --changed
+  pnpm eval:langtracer-push --suite <suite> --dry-run --changed
 
-pnpm exec dotenvx run -f ../../../.env.local -- \
+pnpm exec dotenvx run -f ../../../.env.eval -- \
-  pnpm eval:langtracer-push --suite agents --changed
+  pnpm eval:langtracer-push --suite <suite> --changed
 ```
 
 The push needs `LANGTRACER_URL` and `LANGTRACER_API_KEY`. Generate a key on the
@@ -179,12 +358,20 @@ after a successful push.
 ## Completion checklist
 
 - The LangTracer environment preflight passed before authoring.
+- The driver chose the target suite before any work, including in autonomous mode.
+- The autonomy level was set and stated before any work.
+- A sourced case links to its source thread, and the source turn was read raw.
+- Each conditional expectation has proof that its precondition fired.
+- A kept product-gap red has a proposed Linear ticket and a check of the builder guidance.
 - The case is in `data/agents/` and uses the `agents` dataset.
 - The strict loader accepts it.
 - A local run captures a standalone Agent.
 - The transcript proves each process expectation was exercised.
 - The rendered Agent artifact proves each outcome expectation.
+- No expectation names a sub-agent, an internal tool, or a delegation step.
+  Each one passes for a correct build from a different architecture.
 - Repeated runs are stable enough for the selected tier.
-- The case is pushed to `Instance AI capabilities — agents` with suite slug `agents`.
+- The case is pushed to the suite the driver chose, and the report links to it.
-- Agent Builder PR changes select the `agents` suite in CI.
+- Agent Builder PR changes select the `agents` suite in CI. A case pushed to
+  another suite does not run in that lane.
 - The local case JSON is not committed.

.agents/skills/create-agent-builder-eval/local-setup.md

@@ -4,7 +4,7 @@
 
 Generate a key on the
 [LangTracer API keys page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
-Add it to `.env.local` at the repository root:
+Add it to `.env.eval` at the repository root:
 
 ```env
 LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
@@ -16,7 +16,7 @@ that both variables exist before it creates an eval.
 
 ## Remaining environment
 
-Create `.env.local` at the repository root. Use one Instance AI model provider
+Create `.env.eval` at the repository root. Use one Instance AI model provider
 and one sandbox provider.
 
 ```env
@@ -76,7 +76,7 @@ N8N_USER_FOLDER=<printed-temp-directory> \
 E2E_TESTS=true \
 N8N_ENABLED_MODULES=instance-ai,agents \
 N8N_AI_ENABLED=true \
-pnpm exec dotenvx run -f ../../.env.local -- pnpm start
+pnpm exec dotenvx run -f ../../.env.eval -- pnpm start
 ```
 
 Use different ports when 5680 or 5681 is busy.
@@ -96,7 +96,7 @@ curl -sf -X POST http://localhost:5680/rest/e2e/reset \
 From `packages/@n8n/instance-ai`:
 
 ```bash
-pnpm exec dotenvx run -f ../../../.env.local -- \
+pnpm exec dotenvx run -f ../../../.env.eval -- \
   env -u LANGSMITH_API_KEY -u LANGSMITH_TRACING \
   pnpm eval:instance-ai \
   --base-url http://localhost:5680 \

.agents/skills/create-instance-ai-eval/SKILL.md

@@ -25,7 +25,9 @@ exhaustive field reference; this skill is the opinionated *how*.
 > approach.** Author the file locally (uncommitted), calibrate it against a real
 > build, then **push it to a lang-tracer suite** with `eval:langtracer-push`
 > (see [Push to a lang-tracer suite](#push-to-a-lang-tracer-suite)) —
-> `--suite baseline` for the consolidated corpus n8n CI runs, or a dedicated
+> the suite the driver chose (see
+> [Ask for the target suite first](#ask-for-the-target-suite-first)) —
+> `baseline` for the consolidated corpus n8n CI runs, or a dedicated
 > capability suite like `agents`.
 > The suite is the home for the case; the eval CLI reads it back via
 > `--source langtracer`. You still write the JSON file — it's just the input to
@@ -38,6 +40,34 @@ exhaustive field reference; this skill is the opinionated *how*.
 > pruned and has no durable home. Don't commit a replay case either — derive a
 > synthetic case from it. See [`case-shapes.md`](case-shapes.md).
 
+## Ask for the target suite first
+
+**Before you source, draft, or run anything, ask the driver which LangTracer
+suite receives every eval you create in this session.** Ask this in both
+autonomy levels, including autonomous mode. Never pick the suite yourself and
+never default silently. A case pushed to the wrong suite runs in the wrong CI
+lane and alerts the wrong team.
+
+Ask the suite question and the autonomy question together, in one message.
+Recommend `baseline`:
+
+- `baseline` (recommended): the general corpus. The Instance AI (INS) team
+  monitors it, and it runs nightly.
+- Any other suite, such as `agents` or `node-gauntlet`: call `list_suites` and
+  offer the live list. Do not use a hardcoded list, because suites are added
+  and retired.
+
+Rules:
+
+- The driver's answer applies to all cases in the session. Do not ask again per
+  case.
+- If the driver already named a suite in the request, state it back in one line
+  and continue. Do not ask again.
+- Check that the suite kind matches the case. Push a case with a wrong build
+  (`capability_gap`) only into a suite that allows that kind. If the suite does
+  not allow it, tell the driver and ask again.
+- Put the chosen suite slug in the final decision log, as a link.
+
 ## Set the autonomy level first
 
 **Before you source, draft, or run anything, decide how hands-on the driver
@@ -468,6 +498,16 @@ read the run assumes someone already owns it. So once a red is classified as a
 real gap (and the driver has confirmed it, per the autonomy level), **propose a
 Linear ticket for it.**
 
+**Reproduce first. This is a default, not a hard rule.** Propose a ticket only
+after a case has reproduced the gap. That means the case is red on a real build,
+the precondition was confirmed to fire, and you re-read the raw thread (see
+[First reproduce, then reclassify](#first-reproduce-then-reclassify)). A gap that
+comes from trace analysis alone is a hypothesis. A ticket for it costs the owning
+team time. The driver may approve an exception, for example when the harness
+cannot reach the mechanism after about three attempts. In that case the ticket
+must say that the eval did not reproduce the gap. Never file an unreproduced gap
+without the driver's approval.
+
 **Propose, don't create.** Per [AGENTS.md](../../../AGENTS.md), never open a
 Linear ticket unasked. Put the draft in front of the driver — interactively in
 checkpoint mode, in the decision log in autonomous mode — with a title, a team,
@@ -619,6 +659,12 @@ nodes get LLM-generated pin data). So:
   otherwise
   accept the static-data red as a harness limit and carry the logic in
   `outcomeExpectations`. Note the agent may *choose* static-data dedup on its own.
+- **Message an Agent runs the called Agent with its real model.** The harness
+  mocks the Agent's tools, MCP servers and web search, and records those calls
+  under the calling node. It turns off the Agent's memory, vector stores and chat
+  integrations. Only OpenAI models are supported today, and only when
+  `EVAL_OPENAI_API_KEY` is set (CI sets it): put the Agent on an `openai/*` model
+  and declare an `openAiApi` credential.
 - Don't assert exact counts that depend on mock generation ("exactly 7 posts").
   Say "fewer than the original 10".
 
@@ -712,8 +758,9 @@ Both are natural-language assertions graded by the same Sonnet judge, and each
   They run everywhere, including prebuilt/MCP runs (no transcript needed).
 - **`processExpectations`** — **how the agent behaved during the build**, judged
   from the transcript. Assert clarifying questions asked (or not re-asked),
-  tool-call behaviour, plan/approval handling, batching, honouring a correction,
+  user-visible behaviour, plan/approval handling, batching, honouring a
-  ordering. They need a transcript, so they're **skipped in prebuilt/MCP runs**.
+  correction, ordering. Do not assert internal mechanics such as sub-agents or
+  tool names (see "Keep expectations free of internal mechanics"). They need a transcript, so they're **skipped in prebuilt/MCP runs**.
 
 Rule of thumb: an assertion about *the artifact* is an outcome expectation; an
 assertion about *the conversation or the agent's choices along the way* is a
@@ -751,6 +798,54 @@ case, where the source and channel were **left unspecified**):
 Put intent the conversation only *implied* (a preferred but unstated channel) in
 `processExpectations`, not `outcomeExpectations`.
 
+## Keep expectations free of internal mechanics
+
+An expectation describes what the user or the artifact sees. It does not
+describe how Instance AI is built inside. The internal architecture changes:
+sub-agents, tool names, delegation, planning steps, and run ordering all move.
+An expectation that names them goes red when the design changes, although the
+behavior is still correct. It also stops proving the behavior: a build that
+solves the problem by another route fails it.
+
+**Do not name these in an expectation:**
+
+- Sub-agents, delegation, or hand-off ("the sub-agent runs ...", "the
+  orchestrator delegates ...").
+- Internal tool names or call counts ("calls `build-agent`", "calls
+  `search-nodes` twice").
+- Internal stages, task lists, or step order that the user cannot see.
+- Which component produced a result.
+
+**Do name these:**
+
+- What the user sees in the conversation: the question asked, the plan shown
+  for approval, the claim in the final response.
+- What the artifact contains: the workflow or Agent, its configuration, and
+  how it behaves.
+- A user-visible sequence, only when the user asked for it ("asks before it
+  builds").
+
+Rewrite test. Delete the internal term. State the result the term produced.
+
+| ❌ mechanical | ✅ intent |
+|---|---|
+| "The sub-agent runs the workflow before it reports success" | "The final response reports only results that the build verified; it does not claim an untested step works" |
+| "The orchestrator calls `ask-user` before delegating" | "The agent asks for the missing Slack channel before it builds, and does not guess one" |
+| "The builder calls `search-nodes` for the trigger" | "The workflow starts from a trigger that matches the requested schedule" |
+
+Second test. Imagine the internals are replaced by one agent with different
+tools. A correct build must still pass. If the expectation would fail or become
+meaningless, rewrite it.
+
+Some mechanics are a legitimate subject: a case whose purpose is to guard a
+specific tool contract (for example, a regression in how a tool reports an
+error). Keep these rare. State the reason in `description`. Do not mix them with
+intent expectations in the same case.
+
+**Calibration steps are not expectations.** Reading the transcript to confirm
+that a build ran, or that a sub-agent was reached, is a good calibration check.
+Keep it in your own run notes. Do not copy it into the case JSON.
+
 ## Robust design vs harness flakiness
 
 Two different things — keep them apart:
@@ -837,10 +932,11 @@ drifted, leaves the rest unchanged, and never prunes. It's the inverse of
 
 ```bash
 cd packages/@n8n/instance-ai
+# <suite> is the slug the driver chose (e.g. baseline)
 # preview first — no writes:
-pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite baseline --dry-run --changed
+pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite> --dry-run --changed
 # then push (drop --dry-run):
-pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite baseline --changed
+pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite> --changed
 ```
 
 - **Selectors** (at least one required — no accidental push-all): positional

.agents/skills/create-instance-ai-eval/sourcing-cases.md

@@ -47,9 +47,20 @@ ids. Full table in
 1. **Scan cluster themes** — `list_cluster_runs` / `get_latest_cluster_run`
    return capability-gap themes with a `label`, `summary`, and `mechanism`: the
    real, recurring failure modes.
-2. **Pull the conversations** — `list_conversations` (e.g. `verdict:"bad"`,
+2. **Pull the conversations.** Start from what LangTracer already detected.
-   `analyzed:"yes"`) → `get_conversation` (raw trace) + `get_conversation_analysis`
+   `list_signal_rates` lists every code check and label rule with its rate over a
-   (findings). `get_linear_ticket_context` links a thread to its ticket. To target
+   window: what fails most right now. Then `list_conversations` with `signals`,
+   one token per signal, ANDed — `check:has_nodes`,
+   `label:task_outcome=agent-gave-up`, `analysis:bad`, `keeper:*`, `review:*` — puts
+   the matching entries on each row, each with its evidence (the verdict comment,
+   the judge's words, Keeper's reason) and its version. On a candidate,
+   `get_conversation_signals` shows what every source said and which ones agreed:
+   a thread flagged by a code check, the judge and Keeper is a stronger source
+   than one flagged by a single model pass. A source marked `not_run` never looked
+   at the thread, so read its silence as unknown, not as clean. The older path
+   still works: `list_conversations` (e.g. `verdict:"bad"`, `analyzed:"yes"`) →
+   `get_conversation` (raw trace) + `get_conversation_analysis` (findings).
+   `get_linear_ticket_context` links a thread to its ticket. To target
    **execution failures**, add `funnelDrop:"05"` (built + launched an execution
    that never *succeeded*). **Caveat: the funnel drop is a *conversion* signal, not
    a build-quality one** — most `funnelDrop:"05"` threads are healthy builds that
@@ -61,7 +72,8 @@ ids. Full table in
    (was `build-workflow` called on this turn?), which it reads reliably — but a
    content-dependent claim ("invented ID", "missing node") can be wrong when it
    couldn't see the built workflow. Confirm against the raw trace before building
-   a case around it.
+   a case around it. The same goes for a signal's `evidence`: it is what that
+   source saw, not the trace.
 4. **Optionally record the selection as an observation.** Offer to save why this
    thread was selected and what the developer observed. This is a best-effort
    LangTracer note, not a gate: skip it when the developer declines or has not

.agents/skills/create-pr/SKILL.md

@@ -78,11 +78,24 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
    body structure, then populate each section with actual content before
    creating the PR:
    - **Summary**: describe what the PR does
+   - **Why**: explain why the change is needed (see Why below)
    - **How to test**: describe how to test the changes. Include an example workflow if appropriate
    - **Related tickets**: add the Linear ticket URL (`https://linear.app/n8n/issue/[TICKET-ID]`) and any GitHub issue links
    - **Checklist**: keep as-is from the template
 	 - Add a "🤖 PR Summary generated by AI" at the end of the body
 
+   Before you create the PR, proofread the body with these checks. Rewrite
+   each sentence that fails a check:
+   1. The Summary and the Why together are under the word budget. If not,
+      first remove the details that the reviewer reads in the diff, then
+      shorten the Why.
+   2. Each sentence names who does what: a component, a process, or a person
+      as the subject, with a plain technical verb.
+   3. Each domain term has a plain definition where it first appears.
+   4. Each sentence is true for the diff.
+
+   When a later commit changes a fact in the body, update the body.
+
    ```bash
    gh pr create --draft --title "<type>(<scope>): <summary>" --body "$(cat <<'EOF'
    <populated body based on pull_request_template.md>
@@ -94,12 +107,43 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
 
 Based on `.github/pull_request_template.md`:
 
-Write the PR body in ASD-STE100 Simplified Technical English: use short
+Write for a reviewer who does not know this area of the code. The reviewer
-sentences, the active voice, and one instruction for each sentence.
+must understand the change before they open the diff. Use ASD-STE100
+Simplified Technical English: short sentences, active voice.
+1. Keep the Summary and the Why together under about 200 words. A small PR
+   needs much less.
+2. Define each domain term in plain words where it first appears. Do not use
+   a ticket ID or an internal name as an explanation.
+3. Leave out details that the reviewer reads in the diff: error cases, status
+   codes, renamed fields, the order of calls, file lists, test names. Mention
+   one only when it changes behaviour that a reviewer could miss. Also leave
+   out what reviewers already know (framework behaviour).
+4. Use plain technical verbs, not metaphors ("passes the value", not
+   "hands the value" or "lets the value through").
 
-### Summary Section
+### Summary: what the PR does
-- Describe what the PR does
+1. Start with one to three sentences of background for a reader who is new to
-- Include screenshots/videos for UI changes
+   the area. Then state what the PR does.
+2. Write **prose** about behaviour, not a list of code edits. Name a class or
+   a function when it represents an essential/core concept and it helps
+   the reader to navigate the change.
+3. If the PR changes no product code, say so in the first sentence
+   (for example: "This PR only adds tests.").
+4. Do not justify the change here. The justification goes in the Why.
+5. Include screenshots or videos for UI changes.
+
+### Why: the reasoning
+Always fill the `## Why` section. The Summary and the core of the
+Why are what the reviewer needs most. Keep the core short:
+1. Do not repeat the Summary.
+2. State the problem or the constraint that makes the pre-existing code unfit.
+3. If there are accepted gaps, or changes for existing code paths, state them
+   in a separate short paragraph.
+
+Add rejected alternatives only when a reviewer would otherwise ask about them,
+if any. Use one or two sentences for each alternative: state the concrete
+reason it fails. Write an alternative that was never built in the conditional.
+Together, the alternatives must be shorter than the core Why section.
 
 ### How to test Section
 - Explain how to test the changes

.agents/skills/db-migrations/SKILL.md

@@ -110,6 +110,7 @@ Run through this before requesting review. Each item is a real, recurring review
 - [ ] Identifiers go through **`escape.tableName(...)` / `escape.columnName(...)`**. Never hand-write `n8n_table` prefixes. — [Always escape identifiers](#always-escape-identifiers)
 - [ ] **Match column type to value semantics.** Native `uuid` for UUIDs, `timestampTimezone()` for timestamps, a numeric type for numbers, `bool` for booleans, `json` for structured data. Never `varchar` as a catch-all. — [Column types](#column-types)
 - [ ] **Pick the narrowest sane type within that category:** `int`/`smallint` not `bigint` when range allows; `text` not `varchar(255)` for unbounded strings; never `double` for version numbers. — [Column types](#column-types)
+- [ ] **A generated `id` key is an integer** (`int.primary.autoGenerate2`). A UUID key is a UUIDv7. For a UUID or string key, write the reason in the PR description. A join table uses its foreign-key columns as the key, not an `id` column. — [Primary Keys](#primary-keys)
 - [ ] **Default `notNull`**, relax only when justified. PK is implicitly NOT NULL. Migration's `notNull` matches the entity's nullability. — [NOT NULL and entity parity](#not-null-and-entity-parity)
 - [ ] **Enum-like columns** carry `.withEnumCheck([...])` AND `.comment('explains values')`. Opaque IDs / unix timestamps / JSON shapes also get `.comment()`. — [Constrain enum-like strings](#constrain-enum-like-strings), [Add comments on columns](#add-comments-on-columns)
 - [ ] **Every reference column has an explicit FK** with deliberate `onDelete`. Name FKs explicitly when SQLite recreate cycles risk duplicating them. Avoid polymorphic `(typeCol, idCol)` patterns. — [Foreign Key Constraints](#foreign-key-constraints), [General Design Guidance](#general-design-guidance)
@@ -439,11 +440,32 @@ Don't set the default of a `status` column to a terminal value — `"running"` m
 
 ### Primary Keys
 
-Every table needs a primary key. Choose the type in this order:
+Every table needs a primary key. **Use an integer key by default.** Read the options below in order. Use the first option that fits. Do not skip an option only because you like a later one better.
 
-1. **Integer** — `column('id').int.primary.autoGenerate2`. Preferred for new tables: compact, fast joins, no ordering surprises. `autoGenerate2` uses Postgres `IDENTITY` (preferred over the deprecated `serial`-based `autoGenerate`).
+1. **Integer (default)** — `column('id').int.primary.autoGenerate2`. Use this for every new table. Use a different option only when one of its conditions is true. Integer keys are small and make joins fast. `autoGenerate2` uses Postgres `IDENTITY`. Do not use `autoGenerate`. It uses `serial`, which is deprecated.
-2. **UUID** — `column('id').uuid.primary`. Use when IDs are generated client-side, exposed in URLs, or need to be unguessable. Generate UUIDs in application code via `randomUUID()` from `node:crypto`; do **not** chain `.autoGenerate2` on `.uuid` (the DSL throws — `DEFAULT uuid_generate_v4()` fails on managed Postgres like Supabase because it needs the `uuid-ossp` extension in `public`). Use `.uuid` instead of `.varchar(36)`.
+2. **UUID** — `column('id').uuid.primary`. Use this only when one of these conditions is true:
-3. **String** — `column('id').varchar(36).primary` for IDs whose format isn't a UUID (e.g. nanoid-style IDs). Convention: nanoid length 16 for entity IDs.
+   - The application must know the ID before the insert. For example, the client makes the ID.
+   - The ID is in a URL or a public API. Users must not see how many rows exist or guess other IDs.
+
+   **You must use UUIDv7.** Make the ID in application code with the `uuid` package: `import { v7 as uuidv7 } from 'uuid'`.
+
+   Why: a UUIDv7 starts with a timestamp. Thus, new rows go near the end of the index. A UUIDv4 is random, for example `randomUUID()` from `node:crypto`. New rows go to random positions in the index. This splits index pages and makes the index larger.
+
+   A UUIDv7 does not give an exact creation order. Two IDs made in the same millisecond can sort in the wrong order. Clock differences between hosts can also cause this. If you need the exact order, sort by a timestamp column.
+
+   Do **not** chain `.autoGenerate2` on `.uuid`. The DSL throws an error. The database default makes a UUIDv4. Also, `uuid_generate_v4()` needs the `uuid-ossp` extension. Some managed Postgres services, such as Supabase, do not have it. Use `.uuid`, not `.varchar(36)`.
+3. **String** — `column('id').varchar(36).primary`. Use this only when one of these conditions is true:
+   - The entity already uses IDs in a format that is not a UUID. For example, the public API or the UI already shows nanoid IDs for this entity.
+   - The UI shows the ID to users. A UUID is too long to read, copy, or type.
+
+   Entity IDs are usually nanoids with 16 characters.
+
+**Give a reason for every key that is not an integer.** Write the reason in the PR description. Reviewers will ask for it. These reasons are not sufficient:
+- "Other tables use string IDs."
+- "This module already does it this way."
+- "We might need it later."
+
+**These options apply to a generated `id` column only.** A join table that only links other tables does not need an `id` column. Use its foreign-key columns as a composite key (see below). A foreign-key column always has the same type as the key that it refers to.
 
 **Keep ID-column types consistent across related tables.** Mixing `uuid` and `varchar(36)` for what is "the same kind of ID" creates JOIN footguns.
 
@@ -696,7 +718,7 @@ If only Postgres needs the change, just put the file in `postgresdb/`; don't wri
 
 ### Postgres-version-aware UUID generation
 
-`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, generate at the application level with `randomUUID()` — see [Primary Keys](#primary-keys).
+`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, make a UUIDv7 in application code — see [Primary Keys](#primary-keys).
 
 ### SQLite doesn't enforce `varchar(N)` length
 

.agents/skills/env-var-docs/SKILL.md

@@ -0,0 +1,158 @@
+---
+name: n8n:env-var-docs
+description: >-
+  Writes and reviews the JSDoc on `@Env(...)` config properties. The docs team
+  extracts these comments from source into the public environment variable
+  reference, so they are user documentation, not code comments. Use when adding
+  or changing an `@Env` property, a config class, or reviewing a PR that touches
+  one.
+---
+
+# Environment variable docs
+
+The JSDoc above every `@Env('...')` property is exported almost verbatim into
+the [environment variables reference](https://docs.n8n.io/deploy/host-n8n/configure-n8n/basic-configuration/use-environment-variables)
+on docs.n8n.io. The generated table has four columns: variable name,
+description, data type, default. Your JSDoc is the description column. The
+reader is a self-hosting operator, not an n8n developer.
+
+Follow the [n8n docs style guide](https://docs.n8n.io/contribute/contribution-guide-for-n8n-docs/style-guide-for-n8n-docs)
+for these comments. It overrides the ASD-STE100 rule in `AGENTS.md` for this
+one case. The rules below are the parts of that guide that matter for a
+one-paragraph description.
+
+## Rules
+
+- **Every `@Env` property has a JSDoc.** No exceptions. A missing comment
+  ships as an empty description cell.
+- **Write for the operator.** Say what the setting does, when to change it,
+  and what happens when it's unset or wrong. Don't explain why the code is
+  written the way it is.
+- **No code references.** No file names, class names, function names,
+  constants, license keys, or other internals the reader can't open. Values
+  the operator types into the variable are fine and often needed, for example
+  a credential type ID like `googleOAuth2Api` or a mode like `postgresdb`.
+  The test: would the operator paste this into a config file, or would they
+  need the source to know what it means? Details for developers go in a
+  regular `//` comment on the implementation, not in the JSDoc.
+- **No attack scenarios.** Say what the setting protects, not how an
+  attacker would exploit it. This is a public page.
+- **Don't repeat the default.** The default renders in its own column. Don't
+  write "Defaults to true" or "Default: 1 hour". Do say what the unset state
+  means when it's not obvious, for example "When unset, n8n uses the Redis
+  host".
+- **State units and formats.** Say "in milliseconds", "in bytes", "a
+  comma-separated list", "a JSON object". The data type column only says
+  `Number` or `String`.
+- **Use allowed values when the set is small.** "One of `sqlite`,
+  `postgresdb`." For a list, show one inline example in the sentence.
+- **No JSDoc tags.** No `@example`, `@default`, `@see`, `@deprecated`. The
+  exporter copies the tag name into the docs. Put the example or the
+  deprecation notice in a sentence instead.
+- **Product names:** "Gateway credits", not "n8n Connect" or "AI Gateway".
+  "Publish a workflow", not "activate". `n8n` is always lowercase.
+- **Style:** present tense, active voice, second person. Use contractions.
+  No em dashes, use a comma or a new sentence. No "e.g." or "i.e.", write
+  "for example" and "that is". Spell out zero to nine, numerals from 10.
+  Values the operator types stay numerals, for example "Set to 0".
+  Sentence case. One idea per sentence. End every sentence with a period.
+- **Plain words:** "use" not "utilize", "lets you" not "allows you to", cut
+  "powerful", "seamless", "simply", "it's important to note".
+- **Length:** one to three sentences. If the setting needs more, the extra
+  belongs in a docs page the comment can't link to yet; keep the comment to
+  what fits a table cell.
+
+## Examples
+
+Bad, as found in the generated docs:
+
+```ts
+/**
+ * Instance-level enablement for n8n Connect (AI Gateway). Licensed instances
+ * are enabled by default; false opts out. Budget / free allowance still comes
+ * from the license (`quota:aiGatewayBudget`).
+ */
+```
+
+Wrong product name, a restated default, and an internal license key. Good:
+
+```ts
+/**
+ * Whether Gateway credits are available on this instance. Set to false to
+ * turn them off on a licensed instance. The credit allowance comes from your
+ * license.
+ */
+```
+
+Bad:
+
+```ts
+/**
+ * Time to live in milliseconds for Redis cache entries. Set to 0 to disable
+ * expiry. Default: 1 hour.
+ */
+```
+
+The default is already in the table. Good:
+
+```ts
+/**
+ * Time to live in milliseconds for Redis cache entries. Set to 0 to keep
+ * entries until Redis evicts them.
+ */
+```
+
+Bad:
+
+```ts
+/**
+ * Comma-separated list of credential types for which overwrites are skipped
+ * when the credential has been customized.
+ * @example `N8N_SKIP_CREDENTIAL_OVERWRITE=zohoOAuth2Api,salesforceOAuth2Api`
+ */
+```
+
+The `@example` tag lands in the description as literal text. Good:
+
+```ts
+/**
+ * Comma-separated list of credential types to skip when applying credential
+ * overwrites, for example `zohoOAuth2Api,salesforceOAuth2Api`. n8n skips an
+ * overwrite only when a user has already changed one of its fields.
+ */
+```
+
+Bad:
+
+```ts
+/**
+ * Bind OAuth credential callbacks to the browser that initiated the flow.
+ * When enabled, a session-scoped `n8n-oauth-binding` cookie is set on flow
+ * initiation and verified on callback. Defeats phishing attacks where an
+ * attacker injects an OAuth authorize URL into a victim's browser to land
+ * tokens on the attacker's credential.
+ */
+```
+
+Attack narrative and an internal cookie name on a public page. Good:
+
+```ts
+/**
+ * Whether an OAuth credential connection must finish in the same browser
+ * that started it. When enabled, n8n rejects the callback if it arrives
+ * from a different browser.
+ */
+```
+
+## Review checklist
+
+When you review a PR that adds or changes an `@Env` property, check each
+comment against this list and quote the fix in the review:
+
+1. JSDoc present.
+2. No code identifiers, file names, or internal keys.
+3. No default restated.
+4. Units, format, or allowed values stated.
+5. No JSDoc tags.
+6. No em dashes, Latin abbreviations, or filler words.
+7. Reads correctly as a sentence in a table cell with no surrounding context.

.agents/skills/public-api/SKILL.md

@@ -21,8 +21,7 @@ check the code, not license to drop a team default.
 ## Non-negotiable rules
 
 - New endpoints are `@PublicApiController` classes under `v1/controllers/`, one
-  `*.public.controller.ts` per feature. A controller is a class — never
+  `*.public.controller.ts` per feature.
-  `export =` (the legacy tuple style; `require-public-api-controller` flags it).
 - Public API and internal REST are separate HTTP surfaces. A public controller
   never calls an internal controller/endpoint; both reuse the same service.
 - Controllers and handlers delegate to a service — never import a repository or
@@ -31,7 +30,6 @@ check the code, not license to drop a team default.
   `@ApiResponse(Dto)`.
 - Register each controller via a side-effect import in `v1/controllers/index.ts`
   (`public-api-controllers.test.ts` fails otherwise).
-- Don't add business logic to legacy `express-openapi-validator` (EOV) handlers.
 - Migrating a legacy endpoint must not change its public contract.
 
 These are `n8n-local-rules` ESLint rules (see `packages/cli/eslint.config.mjs`)
@@ -109,10 +107,10 @@ model; reuse only what applies. Decorators, all from `@n8n/decorators`:
 | `@Get/@Post/@Put/@Patch/@Delete('/path')` | Route method. |
 | `@ApiKeyScope('res:action')` | API-key grant check. |
 | `@ProjectScope/@GlobalScope('res:action')` | User RBAC check. |
-| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO — throws. |
+| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO or a binary body — throws. For a binary response body (returned as `{ body, headers? }`), use `@ApiResponse(status, { mediaType, description?, headers? })` — see [Binary response bodies](reference.md#binary-response-bodies). |
 | `@ApiErrorResponse(status)` | Declares an additional documented non-2xx status (e.g. `404`, `409`). Stack multiple for more than one. `400`/`401`/`403` are added automatically (body/query present, always, and `@ApiKeyScope` present, respectively) — don't declare those yourself. |
 | `@ApiSummary(text)` / `@ApiDescription(text)` / `@ApiTags([...])` | OpenAPI summary/description/tags. `@ApiTags` sorts alphabetically regardless of the order you pass. All optional but expected on every real route. |
-| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. |
+| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. `@Body` is JSON by default; `@Body({ mediaType: 'multipart/form-data', uploadLimits })` takes a `multipart/form-data` body instead — see [Request body media types](reference.md#request-body-media-types). |
 | `@Licensed('feat')` | Gates the route on a single `BooleanLicenseFeature`; `PublicApiControllerRegistry` runs its own license middleware (after auth/`@ApiKeyScope`/`@ProjectScope`|`@GlobalScope`, before the handler) and 403s unlicensed requests. Only takes one feature — if the gate is an any-of/all-of combination (e.g. `LicenseState.isProvisioningLicensed()`, which is `feat:saml` OR `feat:oidc`), `@Licensed` can't express that; check manually in the handler instead, same as the internal `provisioning.controller.ee.ts`/`role-mapping-rule.controller.ee.ts` do today (throwing `ForbiddenError` on failure). |
 
 ## Authorization (easy to get wrong)
@@ -200,6 +198,5 @@ existing tests.
 - [Test-before-save endpoints](reference.md#test-before-save-endpoints)
 - [Errors](reference.md#errors)
 - [Testing matrix](reference.md#testing-matrix)
-- [Migrating legacy EOV endpoints](reference.md#migrating-legacy-eov-endpoints)
 - [Verifying a migration](reference.md#verifying-a-migration)
 - [CI and merging](reference.md#ci-and-merging)

.agents/skills/public-api/reference.md

@@ -103,55 +103,76 @@ scope, RBAC denial. Add whichever apply, matching the nearest existing tests:
   unchanged. Tests alone cannot show this — see
   [Verifying a migration](#verifying-a-migration).
 
-## Migrating legacy EOV endpoints
+## Request body media types
 
-Legacy `express-openapi-validator` endpoints live under
+`@Body` defaults to `application/json`; the body is already parsed by the
-`v1/handlers/`, wired through `openapi.yml` with `x-eov-operation-*` and request
+app-wide `bodyParser` before the registry sees it. Declaring
-types in `packages/cli/src/public-api/types.ts`. Treat these as migration targets,
+`@Body({ mediaType: 'multipart/form-data', uploadLimits })` instead takes a
-not templates.
+`multipart/form-data` body:
 
-- Prefer migrating to `@PublicApiController` over extending the handler.
+- `uploadLimits: () => MultipartUploadLimits` is a thunk, read once the route
-- Preserve the public contract: path, method, scopes, status codes, response
+  handles its first request (not at startup) — read live config inside it
-  shape, and pagination.
+  (e.g. `Container.get(GlobalConfig)`), don't inline a literal.
-- Move HTTP concerns into the controller and business orchestration into the
+- The route's `@Body` DTO validates text fields merged with uploaded files,
-  shared service; keep the controller a thin HTTP boundary. If the legacy
+  not `req.body` alone. A text field and a file share the same flat object;
-  handler was itself already just a thin wrapper around an internal
+  a file field uses `publicApiUploadedFileSchema` (`@n8n/api-types`) — a
-  `@RestController` (calling its methods directly, e.g.
+  multer-file-shaped schema that documents itself as `{ type: 'string',
-  `Container.get(SomeController).createThing(req, res, payload)`), the new
+  format: 'binary' }`. Several files under the same field name become an
-  public controller can call that same internal controller directly — no need
+  array.
-  to duplicate its validation/business logic.
+- An unknown field (one the DTO doesn't declare, on a `{ strict: true }` DTO)
-- A route must be served by either the EOV handler or a controller, not both —
+  fails with `Unexpected form field "<name>"` — not the default "unrecognized
-  the build's `mergeDecoratorDocument` (`v1/openapi-gen/generate.ts`) throws on
+  keys" wording.
-  a path+method declared by both sides. Remove the legacy wiring (its path's
+- Multer parsing errors map to these statuses: `413` for a size or count limit,
-  `$ref` entry in `openapi.yml`, the `x-eov-operation-*` handler, and its
+  `500` for `LIMIT_UNEXPECTED_FILE`, `400` for any other multer error, and `500`
-  `handler.ts`) only after the new controller is registered, `pnpm build`
+  for anything else (a malformed body). The `500` is not masked, because it is a
-  regenerates the spec cleanly, and tests are updated.
+  `ResponseError`. A missing boundary gives `400` with `multipart file(s) required`.
-- Fully delete the migrated legacy files: the handler's `.ts`, its
+- The body is parsed **after** every auth/scope/license/quota gate and
-	`spec/paths/*.yml` and `spec/schemas/*.yml`, and any now-dead request type
+  **before** controller/route middlewares — a caller those gates would reject
-	in `packages/cli/src/public-api/types.ts`. Then check
+  never has their (possibly huge) body read off the socket, and a middleware
-  `v1/shared/spec/schemas/_index.yml` and `v1/shared/spec/parameters/_index.yml`
+  that reads `req.body` sees it already parsed.
-  for entries that `$ref` one of the deleted schema/parameter files — those are
+- `/discover` shows no request schema for a multipart route (same as a legacy
-  separate from the path's own `$ref` in `openapi.yml` and are easy to miss;
+  multipart route today) — a client can't assume a JSON schema it never gets.
-  left dangling, the next bundle fails on a broken `$ref`.
+- The generator documents the body under the `multipart/form-data` content
-- If the legacy handler gated on a license (`isLicensed('feat:x')` middleware),
+  key (not `application/json`) and adds `413` alongside `415` to the route's
-  `@Licensed('feat:x')` now replicates that for a controller route (see the
+  documented responses automatically.
-  decorator table in [SKILL.md](SKILL.md#declaring-a-controller)) — but only for
+
-  a single feature. If the legacy check was an any-of/all-of over several flags
+Everything above lives in `packages/cli/src/public-api/media-types/`, one
-  (e.g. `LicenseState.isProvisioningLicensed()`), `@Licensed` can't express
+handler per media type (`REQUEST_BODY_HANDLERS` in
-  that; replicate it manually in the controller instead, don't drop it - this
+`media-types/request-body/index.ts`). Adding a further media type (e.g.
-  is exactly what the internal `provisioning.controller.ee.ts` and
+`application/octet-stream`) means: add it to `RequestBodyMediaOptions` in
-  `role-mapping-rule.controller.ee.ts` already do, since neither uses
+`@n8n/decorators`'s `controller/types.ts`, write a handler implementing
-  `@Licensed` for that reason.
+`RequestBodyHandler` in `media-types/request-body/`, and register it in
-- As a legacy file drops repository access / the `export =` tuple, remove its
+`REQUEST_BODY_HANDLERS` — the registry, resolver, generator and `/discover`
-  entry from the `off` allowlists for `no-repository-in-public-api-handler` and
+need no change, since they all read the handler, not the media type.
-  `require-public-api-controller` in `packages/cli/eslint.config.mjs` (shrink-only
+
-  — never extend them).
+## Binary response bodies
-- For complex legacy-only, multipart, or non-standard endpoints, study the
+
-  nearest existing handler first.
+`@ApiResponse(status, { mediaType, description?, headers? })` declares a
-- Keep each field in its original position when you extract a request shape shared
+success body that the controller method returns as `{ body, headers? }`, for
-  by two routes, and destructure out the ones a route doesn't take. The generator
+example an `application/gzip` stream. The framework treats options whose
-  emits properties in shape order, so a moved field rewrites the `*.generated.yml`
+`mediaType` is in `BINARY_RESPONSE_MEDIA_TYPES` as a binary response.
-  of a route the PR wasn't changing.
+`mediaType` accepts only those types.
+
+- `body` is a `Buffer` or a `Readable`. The method returns the result and never
+  writes to `res`.
+- `headers` holds the response headers. Every header declared in `headers` must
+  be in the result. A missing header fails the request with a `500`. Matching is
+  case-insensitive.
+- The registry checks the headers, then sets the declared status, the headers,
+  and `Content-Type: <mediaType>`. A header in the result cannot replace the
+  `Content-Type`.
+- A stream's first chunk is read before any header is set. So a stream that
+  fails before its first chunk gives the normal JSON error response, with no
+  binary `Content-Type`. A method that throws before it returns does the same.
+- After the first chunk is sent, an error ends the response. A client that
+  disconnects is not an error, and the framework destroys the stream.
+- A method that returns no binary body fails with a `500`.
+- The generator documents the body as `{ type: string, format: binary }`
+  under the `mediaType` content key, with the description and headers.
+  The description defaults to `Operation successful.`.
+
+The runtime part is `sendBinaryResponse` in
+`packages/cli/src/public-api/media-types/binary-response.ts`.
 
 ## Verifying a migration
 

.agents/skills/reproduce-bug/SKILL.md

@@ -32,7 +32,7 @@ Based on the affected area, pick the test layer and pattern:
 | CLI / API | Vitest integration | setupTestServer + supertest | `packages/cli/test/integration/` |
 | Config | Vitest unit | GlobalConfig + Container | `packages/@n8n/config/src/configs/__tests__/` |
 | Editor UI | Vitest | Vue Test Utils + Pinia | `packages/frontend/editor-ui/src/**/__tests__/` |
-| E2E / Canvas | Playwright | Test containers + composables | `packages/testing/playwright/` |
+| E2E / Canvas | Playwright | Test containers + composables | `packages/quality/testing/playwright/` |
 
 ## Step 3: Locate Source Files
 

.agents/skills/testmax-assistant-ui/SKILL.md

@@ -0,0 +1,464 @@
+---
+name: n8n:testmax-assistant-ui
+description: >-
+  Tests an n8n Assistant (Instance AI) UI feature end to end. Drafts the test
+  paths from the feature and the diff, writes one Playwright script for each
+  path, runs the paths in parallel on fresh n8n instances with a real LLM, and
+  compares the main Assistant screens with master to find visual
+  regressions. Shows live screenshots during the run, and builds one HTML
+  report with video replays, step screenshots, LLM steps, tokens, cost and
+  visual diffs. Use when the user wants to test an Instance AI or Assistant
+  UI feature end to end, check that a UI change does not break existing
+  screens, or says /testmax-assistant-ui.
+---
+
+# Test an Instance AI feature
+
+Use this skill to test a UI feature of the n8n Assistant end to end. In code,
+the Assistant is "Instance AI". You draft the test paths, the user approves
+them, and the skill runs them all at the same time. A local page shows each run
+live. At the end, you judge each run and build one report.
+
+The skill uses the scripts of `testmax-n8n-ai` for the n8n API, the
+LangSmith usage, the live page server, and the report template. Do not copy
+those scripts.
+
+## What the skill does
+
+```mermaid
+flowchart LR
+    A[Feature + diff] --> B[Draft scenarios.json<br/>and scripts]
+    B --> T[--try and --estimate]
+    T --> C{User approves<br/>plan + cost}
+    C --> D[run.py]
+    D --> E[One fresh branch instance<br/>for each feature + regression]
+    D --> F[Fresh branch instance<br/>visual]
+    D --> G[Fresh master instance<br/>visual, or a kept run]
+    E & F & G --> H[Live page]
+    F & G --> I[Pixel + accessibility tree diff]
+    E & F & G --> J[summary.txt with triage]
+    J --> K[You judge: verdicts.json]
+    K --> L[Report rebuilds itself]
+```
+
+- Each run is one scenario: a Playwright script that drives the real
+  Assistant UI in Chrome, with a real LLM.
+- There are three kinds of scenario:
+  - `feature`: tests a path of the new feature.
+  - `regression`: tests an existing flow that the feature touches.
+  - `visual`: captures screens. The skill runs it on the branch and on master,
+    then compares the captures.
+- All instances are fresh: a new data folder, a new owner, and no workflows or
+  threads. Thus the screens of the branch and master are the same when the
+  code is the same. A fresh instance starts in about 5 seconds, because it
+  copies a data folder that is already migrated and seeded.
+- Each feature and regression scenario gets its own fresh instance, so the
+  scenarios do not see each other's data. The visual scenarios share one
+  branch instance and one master instance.
+- The feature flags are the same on all instances. The instances do not ask
+  PostHog for flags, so no instance gets a random experiment variant. Set the
+  flags that the feature needs in the plan, with `featureFlags`.
+- `run.py` builds this checkout first (with turbo, so a build that is up to
+  date takes about 1 second). The master instance starts from a git worktree
+  at `~/.cache/n8n-feature-test/master`, at the merge base of `HEAD` and
+  `origin/master`. Thus the visual check shows only the changes of this
+  branch, not the changes that master got after the branch started.
+- A master visual run that passed is kept. The next run with the same merge
+  base, the same script and the same flags uses it again, and does not start
+  master.
+
+## Requirements
+
+Run `python3 .agents/skills/testmax-assistant-ui/scripts/run.py --check`
+before the first run. The commands below use `$S` for this scripts folder.
+`--check` checks the tools,
+Chrome, the settings, the disk space and the master build, and prints the
+plan folder for this branch.
+
+- Google Chrome. The scripts use the Playwright package of the repository.
+- The Instance AI model settings in `.env.local` or `.env.ai-surfaces`, as for
+  your own instance. Fresh instances get the same settings, without the
+  license, the database, the port and the encryption key.
+- `LANGSMITH_API_KEY` and LangSmith tracing, for the tokens and cost. Without
+  them, the report shows no tokens or cost. Do not set
+  `N8N_DIAGNOSTICS_ENABLED=false`: then Instance AI sends no traces.
+- About 3 GB of disk space for the master worktree and its build.
+
+Never put a key on the command line. Read settings from the env files and the
+environment only.
+
+## 1. Understand the feature
+
+Find what changed and what the user wants to test:
+
+```bash
+git diff origin/master...HEAD --stat -- packages/frontend packages/@n8n/instance-ai packages/cli/src/modules/instance-ai
+git diff origin/master...HEAD -- <the files of the feature>
+```
+
+Read the changed Vue components, the stores, and the backend events that they
+use. Find the `data-test-id` values of the new elements. If the user did not
+describe the feature, ask for one or two sentences about it.
+
+If the plan has visual scenarios, start the master baseline now, in the
+background. The first build takes 15 to 20 minutes. Later builds take less
+than a minute when the turbo cache has the files. You can draft the plan while
+it builds:
+
+```bash
+S=.agents/skills/testmax-assistant-ui/scripts
+python3 $S/run.py --check         # the setup, and the plan folder
+python3 $S/baseline.py prepare    # in the background
+python3 $S/baseline.py status     # later: is it ready?
+```
+
+## 2. Draft the test paths
+
+Write the plan folder that `--check` prints:
+`~/.cache/n8n-feature-test/plans/<branch>/<feature>/`. The folder stays after
+a reboot. It holds `scenarios.json` and one script for each scenario. The
+folder name is the name of the run folders.
+
+Draft these paths:
+
+- **Feature paths.** The main path, then each branch of it: other user
+  choices, empty and long inputs, cancel and stop, a reload in the middle, a
+  second thread, errors that the user can see. One path for each scenario.
+- **Regression paths.** The existing flows near the change: a plain chat
+  reply, approvals, the plan review, the preview panel, the thread list. The
+  library has `smoke-chat` for the plain chat reply.
+- **Visual screens.** The screens that the change can affect, on the branch
+  and on master. Capture each screen in a stable state. Mask all text that
+  comes from the LLM, and all data that changes between runs. The library has
+  `visual-assistant-home` and `visual-thread-reply`.
+
+Keep each scenario short and focused. A scenario that fails must show where it
+failed. Use more scenarios, not longer ones. A visual scenario must not need
+the new feature, because it also runs on master.
+
+Some UI states show only for a moment, for example a status line that folds
+away when the run ends. Check them while the run is active, not after it.
+
+If the feature is behind a feature flag or an experiment, set the flag in
+`featureFlags`. Without it, the instances use the default value of each flag.
+
+Before you show the plan to the user, check the new scripts and get the
+estimate (see [Check the scripts](#check-the-scripts)). Then show the user the
+list of paths: the id, the kind, the title and the expected result of each.
+Also show the estimate: the number of LLM runs, the cost and the time. Change
+the plan until the user approves it.
+
+### scenarios.json
+
+```json
+{
+  "title": "Stop button in the Assistant",
+  "feature": "The user can stop a run while the Assistant works.",
+  "scenarios": [
+    {
+      "id": "stop-mid-run",
+      "title": "Stop a run in the middle",
+      "kind": "feature",
+      "script": "stop-mid-run.mjs",
+      "expected": "The stop button shows during the run. A click ends the run without an error message.",
+      "timeoutMin": 6
+    },
+    {
+      "id": "smoke-chat",
+      "title": "A prompt gets a reply",
+      "kind": "regression",
+      "script": "lib:smoke-chat.mjs",
+      "expected": "A short prompt gets the reply OK."
+    }
+  ]
+}
+```
+
+The plan has these fields: `title` (required), `feature` (the feature, for the
+judge), and `featureFlags` (optional). `featureFlags` maps a flag name to
+`true`, `false` or a variant name, for example
+`{ "<flag name>": "variant" }`. Each instance gets these flags, on the
+branch and on master. They go into `N8N_FEATURE_FLAG_OVERRIDES` and replace
+the values in your env files.
+
+Each scenario has these fields:
+
+| Field | Required | Description |
+|---|---|---|
+| `id` | yes | Lowercase letters, digits and `-`. No `--`. Unique. |
+| `title` | yes | A short name for the report. |
+| `kind` | yes | `feature`, `regression` or `visual`. |
+| `script` | yes | A path relative to `scenarios.json`, or `lib:<file>` for a library script. |
+| `expected` | yes | The expected result, for the judge. |
+| `autoConfirm` | no | Answer the Assistant's confirmations automatically. Default `true`. Credential and setup cards get "Later". Set `false` when the script answers a confirmation or a card. |
+| `isolated` | no | Run the scenario on its own fresh instance. Default `true`. Instances start ahead of their scenario, so this costs almost no time. Set `false` to run the scenario on one branch instance that it shares with the other `false` scenarios. Visual scenarios ignore this field. |
+| `timeoutMin` | no | The time limit of the scenario. Default 10. |
+| `viewport` | no | The window size, for example `1600x1000` (the default). |
+
+`examples/stop-button/` is a full example plan.
+
+### Scenario scripts
+
+A script exports one async function. It gets the kit `t`:
+
+```js
+export default async function (t) {
+	await t.step('Open the Assistant', () => t.goto());
+	await t.step('Send a prompt', () => t.send('Reply with exactly the word OK and nothing else.'));
+	await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
+	await t.check('The reply says OK', () => t.expect(t.ai.assistantMessages().last()).toContainText('OK'));
+	await t.capture('thread', { mask: [t.ai.assistantMessages()] });
+}
+```
+
+| Kit member | What it does |
+|---|---|
+| `t.step(title, fn)` | Runs an action. Records a screenshot and the time. A failure stops the scenario. |
+| `t.check(title, fn)` | Runs an assertion, with a screenshot. A failure is recorded, and the scenario continues. |
+| `t.note(title)` | Adds a step with a screenshot and no action. |
+| `t.goto(path = '/assistant')` | Opens a page and waits for the chat input. Closes the "Try new n8n Assistant" dialog. |
+| `t.send(text)` | Types into the chat input, sends, and waits for the thread URL. Records the thread for the trace. |
+| `t.waitForIdle({timeout, settle})` | Waits until the thread has no active run. Returns `'idle'`, or `'suspended'` when the run waits for the user. |
+| `t.capture(name, {mask, maskText, root, fullPage})` | Saves a screen and its accessibility tree for the visual check. `mask` takes locators, and `maskText` takes texts: all elements with that text. Masked areas show as one color. In the tree, the subtree of a `mask` element and each masked text show as `[masked]`. |
+| `t.api(method, path, body)` | Calls the n8n REST API as the signed-in user. Returns `data`. |
+| `t.threadStatus(threadId)` | Returns `{active, suspended, background}`: the thread has a run, the run waits for the user, and the number of running background tasks. |
+| `t.ai.*` | Locators for the Assistant: `chatInput`, `sendButton`, `stopButton`, `userMessages`, `assistantMessages`, `assistantText(text)`, `panelText(text)`, `sidebarToggle`, `threadList`, `newThreadButton`, `approveButton`, `denyButton`, `planApproveButton`, `previewPanel`, `previewCanvasNodes`, and more in `scripts/kit.mjs`. |
+| `t.expect`, `t.page`, `t.baseUrl`, `t.sleep(ms)` | Playwright `expect`, the page, the instance URL, and a pause. |
+
+Rules for scripts:
+
+- Use `data-test-id` locators (`page.getByTestId`). Use text only for text that
+  the test controls.
+- Use `t.expect(...).toBeVisible()` and other assertions that wait. Do not use
+  `t.sleep` to wait for the UI.
+- LLM answers change from run to run. Check the UI state, not the exact words.
+  Ask for a fixed reply, such as "Reply with exactly the word OK", when you
+  need a known answer.
+- To test an approval, set `"autoConfirm": false`, wait with
+  `t.waitForIdle()` until it returns `'suspended'`, then click the button.
+
+### Check the scripts
+
+Check the new scripts on fresh instances before you show the plan:
+
+```bash
+python3 $S/run.py --plan <plan>/scenarios.json --try --only id1,id2
+```
+
+`--try` does not run master, does not start the live page, and builds no
+report. It prints a table with the status, the steps and the first error of
+each run. The runs are in `/tmp/feature-test/try/`. Fix the scripts that fail
+because of the script, then try them again. Each LLM scenario in a try costs
+money too, so try only the new scripts.
+
+Then get the estimate for the full run:
+
+```bash
+python3 $S/run.py --plan <plan>/scenarios.json --estimate
+```
+
+The estimate uses the cost and time of earlier runs of the same scenario ids.
+A scenario without an earlier run shows `*` and the median of all earlier
+runs. A master run that is kept shows `(kept)` and costs nothing.
+
+## 3. Run
+
+Open the live page for the user on each run. Do not skip this step. Do this:
+
+1. Find out if your harness has a browser tool that opens a tab in the
+   editor. In Cursor, this is `browser_navigate` of the `cursor-ide-browser`
+   MCP server.
+2. If it has one, start the run in the background with `--no-open`:
+
+   ```bash
+   python3 $S/run.py --plan <plan>/scenarios.json --no-open
+   ```
+
+   Wait for the `live page: <url>` line in the output, which comes in a few
+   seconds. Then open that URL in a new editor tab that the user can see (in
+   Cursor: `newTab: true` and `position: "active"`). If the tab does not open,
+   run `open <url>` (macOS) or `xdg-open <url>` (Linux).
+3. If it has none, start the run in the background without `--no-open`.
+   `run.py` opens the live page in the default browser:
+
+   ```bash
+   python3 $S/run.py --plan <plan>/scenarios.json
+   ```
+
+4. Give the user the URL in both cases.
+
+`run.py` does not read `$BROWSER`, so do not set it.
+
+`run.py` does these steps:
+
+1. Prints the estimate and checks the setup. It stops when the setup has an
+   error, for example a missing Chrome.
+2. Copies the plan into a new run folder,
+   `/tmp/feature-test/<time>-<plan folder name>/`.
+3. Starts the live page and prints its URL. Without `--no-open`, it opens the
+   page in the default browser.
+4. Builds this checkout with turbo. A build error stops the run, and the
+   output shows the end of `build.log`.
+5. Starts the instances: one for each feature and regression scenario, one
+   branch instance for the visual scenarios, and one master instance. It
+   starts the instances ahead of their scenario. Master starts only for the
+   master runs that it does not keep from an earlier run. It prepares the
+   master build first, if it is not ready.
+6. Runs the scenarios, 4 at a time. Each run records a video, the steps with
+   screenshots, the browser errors, the thread events, and the LangSmith ids.
+7. Stops the instances, compares the visual captures, writes `summary.txt`,
+   and builds the report without verdicts.
+8. Prints a result table, the command to run the plan again, and one JSON
+   object on several lines: `runDir`, `report`, `live`, `summary`, `results`
+   (the status of each run), `visual` (the number of captures, the changed
+   ones, and the error when the visual comparison failed), `rerun` and
+   `next`.
+9. Gets the tokens and cost from LangSmith, then builds the report again.
+   LangSmith receives traces with a delay, so this can take up to 90 seconds.
+   You can start to judge while it runs.
+
+| Option | Use |
+|---|---|
+| `--check` | Check the setup and print the plan folder, then stop. |
+| `--estimate` | Print the expected cost and time, then stop. |
+| `--try` | Check new scripts: no master, no live page, no report. See [Check the scripts](#check-the-scripts). |
+| `--only id1,id2` | Run only these scenarios. |
+| `--rerun last` | Run the plan of the last run again, in a new folder. |
+| `--rerun <run-dir>` | Run the plan of this earlier run again, in a new folder. If the plan folder of that run still exists, it uses the current plan and scripts from there. Thus a fixed script runs in its fixed version. |
+| `--concurrency N` | Runs at the same time. Default 4, or `FEATURE_TEST_CONCURRENCY`. |
+| `--branch-url URL` | Use a running instance for all branch runs, not fresh ones. The threads stay on that instance. The scenarios share its data, and the visual check is not reliable. |
+| `--no-build` | Do not build this checkout. The output warns when source files are newer than their build. |
+| `--no-baseline` | Do not run master. There is no visual check. |
+| `--no-cache` | Run master again, also when a kept master run exists. |
+| `--no-fetch` | Do not fetch `origin/master`. Use the merge base with the `origin/master` that git has. |
+| `--no-open` | Do not open the live page in the default browser. Use this when you open it in the editor tab. |
+| `--no-live` | Do not start the live page. Use this only when the user asks for it. |
+| `--keep-instances` | Keep the fresh instances running after the run, for a manual check. The output gives `instancePorts`. Each instance is at `http://localhost:<port>`. Stop them yourself. |
+| `--run-dir DIR` | Use this run folder. |
+
+| Setting | Default |
+|---|---|
+| `FEATURE_TEST_RUNS_DIR` | `/tmp/feature-test` |
+| `FEATURE_TEST_CACHE` | `~/.cache/n8n-feature-test`: the plans, the master worktree, the kept master runs, and the instance data templates |
+| `FEATURE_TEST_CONCURRENCY` | `4` |
+
+### Watch the runs
+
+The live page shows each run as a row, with its status and its last
+screenshot. The screenshot changes every few seconds. When the run is done,
+the same address shows the report. The page stops 30 minutes after its last
+request.
+
+## 4. Judge
+
+Read `<runDir>/summary.txt`. It starts with a triage: each run is `clean`, or
+`look closer` with the reasons (a failed step or check, a browser error, a
+failed tool call, or a visual change). Then it has, for each run: the expected
+result, the steps and their status with the path of each screenshot, the
+prompts, the last reply, the browser errors, and the visual results.
+
+For a clean run, look at the last screenshot and check it against the expected
+result. For a run to look at closer, look at the screenshots of the steps in
+the reasons, and the diff images in `visual/<id>/`.
+
+A script that passed is not always a pass. Check that the run shows the
+expected result, not only that the script ended. Check for these problems:
+
+- a browser error or a failed request that comes from the feature
+- a step that passed, but the screenshot shows the wrong state
+- a visual change that the feature did not intend
+- a script that failed because of the script, not the feature. Say so, and
+  fix the script
+
+Write `<runDir>/verdicts.json`. Use the run names as keys. A master run has
+the name `<id>--master`, and does not need a verdict:
+
+```json
+{
+  "stop-mid-run": {
+    "verdict": "pass",
+    "summary": "One sentence about the result.",
+    "checks": [
+      { "ok": true, "text": "The stop button ends the run" },
+      { "ok": false, "text": "The send button comes back after 9 seconds" },
+      { "ok": null, "text": "Could not confirm the stopped label" }
+    ]
+  }
+}
+```
+
+Use `pass` when all checks pass, `fail` when the main goal is not met, and
+`partial` for all other results. For a visual scenario, judge each change:
+intended by the feature, a regression, or noise (for example, a timestamp
+that is not masked). Be strict and factual.
+
+Write each check as one short sentence with one fact. Put selectors, test
+IDs and UI text in backticks. The page shows them as code. The page shows
+the failed checks first.
+
+## 5. The report
+
+While the live page runs, it builds the report again each time
+`verdicts.json` changes. The open tab loads the new report by itself, and
+keeps the selected run and the scroll position. Thus you do not need to load
+the page again. When the live page does not run, build the report yourself:
+
+```bash
+python3 $S/report.py <runDir>
+```
+
+The report is `<runDir>/report/index.html`. It is one folder with all its
+files, so you can copy or share it. It has, from top to bottom:
+
+- the video replays of all runs
+- a table of all runs: kind, result, steps passed, time, LLM calls, tool calls,
+  tokens, cost and browser errors. The result is your verdict. A master run
+  shows `Baseline`, and a run without a verdict shows `Not judged`, or
+  `Failed` when its script failed
+- the visual check: the branch screen, the master screen, and the diff image
+  of each capture, with the accessibility tree diff
+- for each run: the video of the run, the expected result and the verdict,
+  the steps with their screenshots, the browser errors, and the LLM steps
+  from the trace. A click on a row of the table shows the panel and the
+  video of that run, and pauses the video of the other runs
+
+Each box shows closed. It opens and closes when you click its title. On the
+live page, a box that you open or close stays that way when the page
+updates.
+
+A master run that is kept from an earlier run shows "Reused" in its run box.
+
+Give the user the path and the `live` URL. Tell the user the result of each
+scenario, the visual changes, and the problems that you found.
+
+## Single steps
+
+| Step | Command |
+|---|---|
+| Prepare the master baseline | `python3 $S/baseline.py prepare` |
+| Show the baseline state | `python3 $S/baseline.py status` |
+| Run one script on any instance | `N8N_BASE_URL=<url> node $S/run_scenario.mjs --out <dir> --script <file>` |
+| Compare the visual captures again | `node $S/visual_diff.mjs <runDir>` |
+| Print the judge summary | `python3 $S/report.py <runDir> --summary` |
+| Serve the live page of a run | `python3 $S/live_page.py --run-dir <runDir>` |
+
+## Limits
+
+- LLM answers change from run to run, and a run can fail because of the LLM.
+  Run a failed scenario again with `--rerun last --only <id>` before you call
+  it a bug.
+- A stopped LLM call does not show in LangSmith. The cost of a run with a stop
+  includes only the calls that ended.
+- Each LLM scenario costs money. The tokens and cost show in the report.
+- Parallel runs share the CPU and the LLM rate limits. The times include the
+  shared load.
+- The visual check compares pixels with a small tolerance. Fonts and
+  animations can cause small changes. Masks hide the parts that change between
+  runs. The accessibility tree diff shows changes in structure and text.
+- The worktree shows in `git worktree list`. To remove it, run
+  `git worktree remove ~/.cache/n8n-feature-test/master`.
+- Kept master runs older than 14 days are deleted. The instance data
+  templates in `~/.cache/n8n-feature-test/templates` change when the
+  migrations change. You can delete old templates at any time.
+- A run folder is about 35 MB. The run folders are in `/tmp`, so the system
+  deletes them on reboot.

.agents/skills/testmax-assistant-ui/examples/stop-button/scenarios.json

@@ -0,0 +1,38 @@
+{
+	"title": "Stop button in the Assistant",
+	"feature": "The user can stop a run while the Assistant works. After the stop, the composer is ready and a new message gets a reply. The Assistant screens look the same as on master.",
+	"scenarios": [
+		{
+			"id": "stop-mid-run",
+			"title": "Stop a run in the middle",
+			"kind": "feature",
+			"script": "stop-mid-run.mjs",
+			"expected": "The stop button shows during the run. A click ends the run without an error message. The send button comes back, and a follow-up message gets a reply.",
+			"timeoutMin": 6
+		},
+		{
+			"id": "smoke-chat",
+			"title": "A prompt gets a reply",
+			"kind": "regression",
+			"script": "lib:smoke-chat.mjs",
+			"expected": "A short prompt opens a new thread and gets the reply OK. The composer is ready after the reply.",
+			"timeoutMin": 5
+		},
+		{
+			"id": "assistant-home",
+			"title": "Assistant home and chat history",
+			"kind": "visual",
+			"script": "lib:visual-assistant-home.mjs",
+			"expected": "The home screen and the chat history popover look the same as on master.",
+			"timeoutMin": 3
+		},
+		{
+			"id": "thread-reply",
+			"title": "Thread with one reply",
+			"kind": "visual",
+			"script": "lib:visual-thread-reply.mjs",
+			"expected": "A thread with one short exchange looks the same as on master.",
+			"timeoutMin": 5
+		}
+	]
+}

.agents/skills/testmax-assistant-ui/examples/stop-button/stop-mid-run.mjs

@@ -0,0 +1,31 @@
+// Feature: the user stops a run while the Assistant works, and can send a new message after it.
+export default async function (t) {
+	await t.step('Open the Assistant', () => t.goto());
+	await t.step('Send a prompt that takes a while', () =>
+		t.send(
+			'Explain step by step, in at least 30 numbered points, how to design a workflow that syncs ' +
+				'contacts between two CRMs. Do not build anything.',
+		),
+	);
+	await t.step('The stop button shows while the run is active', () =>
+		t.expect(t.ai.stopButton()).toBeVisible({ timeout: 30_000 }),
+	);
+	await t.step('Click stop', () => t.ai.stopButton().click());
+	await t.check('The stop button goes away', () =>
+		t.expect(t.ai.stopButton()).toBeHidden({ timeout: 15_000 }),
+	);
+	await t.check('The send button comes back', () =>
+		t.expect(t.ai.sendButton()).toBeVisible({ timeout: 15_000 }),
+	);
+	await t.step('Wait until the thread is idle', () => t.waitForIdle({ timeout: 60_000 }));
+	await t.check('The thread shows no error message', async () => {
+		await t.expect(t.ai.panelText(/something went wrong/i)).toHaveCount(0);
+	});
+	await t.step('Send a follow-up message', () =>
+		t.send('Reply with exactly the word OK and nothing else.'),
+	);
+	await t.step('Wait for the follow-up reply', () => t.waitForIdle({ timeout: 120_000 }));
+	await t.check('The follow-up gets a reply', () =>
+		t.expect(t.ai.assistantMessages().last()).toContainText('OK'),
+	);
+}

.agents/skills/testmax-assistant-ui/scenarios/smoke-chat.mjs

@@ -0,0 +1,17 @@
+// Regression: a prompt gets a reply in a new thread.
+export default async function (t) {
+	await t.step('Open the Assistant', () => t.goto());
+	await t.step('Send a short prompt', () =>
+		t.send('Reply with exactly the word OK and nothing else.'),
+	);
+	await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
+	await t.check('The user message shows', () =>
+		t.expect(t.ai.userMessages().first()).toBeVisible(),
+	);
+	await t.check('The reply says OK', () =>
+		t.expect(t.ai.assistantMessages().last()).toContainText('OK'),
+	);
+	await t.check('The composer is ready for the next message', () =>
+		t.expect(t.ai.sendButton()).toBeVisible(),
+	);
+}

.agents/skills/testmax-assistant-ui/scenarios/visual-assistant-home.mjs

@@ -0,0 +1,11 @@
+// Visual: the Assistant home screen, and the chat history popover.
+export default async function (t) {
+	await t.step('Open the Assistant', () => t.goto());
+	await t.capture('assistant-home');
+	await t.step('Open the chat history', async () => {
+		await t.ai.sidebarToggle().click();
+		await t.expect(t.ai.threadList()).toBeVisible();
+	});
+	// Other visual scenarios create threads at the same time, so the list content changes between runs.
+	await t.capture('chat-history', { mask: [t.ai.threadList()] });
+}

.agents/skills/testmax-assistant-ui/scenarios/visual-thread-reply.mjs

@@ -0,0 +1,11 @@
+// Visual: a thread with one short exchange, and the composer after the reply.
+export default async function (t) {
+	await t.step('Open the Assistant', () => t.goto());
+	await t.step('Send a short prompt', () =>
+		t.send('Reply with exactly the word OK and nothing else.'),
+	);
+	await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
+	// The reply text and the thread title come from the LLM. The title also shows in the main sidebar.
+	const title = (await t.ai.sidebarToggle().innerText()).trim();
+	await t.capture('thread-reply', { mask: [t.ai.assistantMessages()], maskText: [title] });
+}

.agents/skills/testmax-assistant-ui/scripts/baseline.py

@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+"""Keep a built copy of master for the visual check.
+
+  baseline.py status              JSON: the worktree, the built commit, and the commit to build
+  baseline.py prepare [--no-fetch]  fetch origin/master, update the worktree, install, and build
+
+The commit to build is the merge base of HEAD and origin/master: the master commit that this branch
+starts from. Then a visual change comes from the branch, not from newer commits on master. The
+build runs only when the merge base moves, for example after a rebase.
+
+The worktree is in <FEATURE_TEST_CACHE>/master (default ~/.cache/n8n-feature-test/master). It is a
+detached git worktree of this repository, so `git worktree list` shows it. The first build takes 15
+to 20 minutes. The output goes to <FEATURE_TEST_CACHE>/build.log.
+"""
+
+import argparse
+import fcntl
+import json
+import os
+import subprocess
+import sys
+import time
+from datetime import datetime, timezone
+
+from common import CACHE, git
+
+WORKTREE = CACHE / 'master'
+STATE = CACHE / 'built.json'
+BUILD_LOG = CACHE / 'build.log'
+
+
+def built():
+	try:
+		state = json.loads(STATE.read_text())
+	except (OSError, ValueError):
+		return None
+	cli = WORKTREE / 'packages' / 'cli' / 'dist'
+	editor = WORKTREE / 'packages' / 'frontend' / 'editor-ui' / 'dist' / 'index.html'
+	return state if cli.exists() and editor.exists() else None
+
+
+def target(fetch=False):
+	"""The master commit to build: the merge base of HEAD and origin/master."""
+	if fetch:
+		git('fetch', '--quiet', 'origin', 'master')
+	try:
+		return git('merge-base', 'HEAD', 'origin/master')
+	except subprocess.CalledProcessError:
+		return git('rev-parse', 'origin/master')
+
+
+def status():
+	state = built() or {}
+	try:
+		commit = target()
+	except subprocess.CalledProcessError:
+		commit = None
+	return {'worktree': str(WORKTREE), 'exists': WORKTREE.exists(), 'builtCommit': state.get('commit'),
+	        'builtAt': state.get('builtAt'), 'baseCommit': commit,
+	        'upToDate': bool(commit) and state.get('commit') == commit, 'log': str(BUILD_LOG)}
+
+
+def run_logged(cmd, log, env=None):
+	log.write(f'\n$ {" ".join(cmd)}\n')
+	log.flush()
+	code = subprocess.run(cmd, cwd=WORKTREE, stdout=log, stderr=subprocess.STDOUT, env=env).returncode
+	if code != 0:
+		raise RuntimeError(f'{" ".join(cmd)} failed with exit code {code}. See {BUILD_LOG}')
+
+
+def prepare(fetch=True, say=print):
+	"""Return {"root", "commit"} of a built merge base. Builds only when the merge base moved."""
+	CACHE.mkdir(parents=True, exist_ok=True)
+	with (CACHE / 'build.lock').open('w') as lock:
+		# Two runs at the same time must not build into the same worktree.
+		fcntl.flock(lock, fcntl.LOCK_EX)
+		if fetch:
+			say('fetch origin/master')
+		commit = target(fetch)
+		if not (WORKTREE / '.git').exists():
+			say(f'create the master worktree in {WORKTREE}')
+			git('worktree', 'add', '--detach', str(WORKTREE), commit)
+		elif git('rev-parse', 'HEAD', cwd=WORKTREE) != commit:
+			git('checkout', '--detach', '--force', commit, cwd=WORKTREE)
+		state = built()
+		if state and state.get('commit') == commit:
+			say(f'master {commit[:11]} is built')
+			return {'root': WORKTREE, 'commit': commit}
+		started = time.time()
+		say(f'build master {commit[:11]} (the merge base). The first build takes 15 to 20 minutes. Log: {BUILD_LOG}')
+		with BUILD_LOG.open('w') as log:
+			# DOCKER_BUILD skips the git hook install of the prepare script, so the hooks of the
+			# main checkout stay as they are.
+			run_logged(['pnpm', 'install', '--frozen-lockfile'], log, env={**os.environ, 'DOCKER_BUILD': 'true'})
+			run_logged(['pnpm', 'turbo', 'run', 'build', '--filter=n8n...'], log)
+		STATE.write_text(json.dumps({'commit': commit, 'builtAt': datetime.now(timezone.utc).isoformat()}, indent=2))
+		say(f'master {commit[:11]} built in {time.time() - started:.0f}s')
+		return {'root': WORKTREE, 'commit': commit}
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('action', choices=['status', 'prepare'])
+	ap.add_argument('--no-fetch', action='store_true')
+	args = ap.parse_args()
+	if args.action == 'status':
+		print(json.dumps(status(), indent=2))
+		return
+	try:
+		result = prepare(fetch=not args.no_fetch, say=lambda m: print(m, flush=True))
+	except (RuntimeError, subprocess.CalledProcessError) as e:
+		sys.exit(str(e))
+	print(json.dumps({'root': str(result['root']), 'commit': result['commit']}, indent=2))
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-assistant-ui/scripts/common.py

@@ -0,0 +1,175 @@
+"""Shared helpers: paths, settings, and fresh n8n instances for the scenario runs. Standard library only.
+
+The skill reuses the scripts of testmax-n8n-ai (settings, REST client, report parts,
+live server). This module puts them on the import path.
+"""
+
+import fcntl
+import hashlib
+import json
+import os
+import shutil
+import signal
+import socket
+import subprocess
+import sys
+import time
+import urllib.request
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+SHARED = HERE.parents[1] / 'testmax-n8n-ai' / 'scripts'
+sys.path.insert(0, str(SHARED))
+
+from n8n_api import REPO_ROOT, N8nClient, config, setting  # noqa: E402
+
+CACHE = Path(setting('FEATURE_TEST_CACHE', default=str(Path.home() / '.cache' / 'n8n-feature-test'))).expanduser().resolve()
+RUNS_ROOT = Path(setting('FEATURE_TEST_RUNS_DIR', default='/tmp/feature-test')).expanduser().resolve()
+PLANS_ROOT = CACHE / 'plans'
+TEMPLATE_VERSION = '1'
+# These settings belong to the instance that the user runs. A fresh instance must not reuse them.
+INSTANCE_ENV_SKIP = ('N8N_LICENSE_', 'DB_', 'N8N_PORT', 'N8N_USER_FOLDER', 'N8N_EDITOR_BASE_URL', 'WEBHOOK_URL',
+                     'N8N_RUNNERS_BROKER_PORT', 'N8N_HOST', 'N8N_PROTOCOL', 'N8N_ENCRYPTION_KEY', 'N8N_BASE_URL')
+
+
+def free_port():
+	with socket.socket() as s:
+		s.bind(('127.0.0.1', 0))
+		return s.getsockname()[1]
+
+
+def instance_env(port, home, feature_flags=None):
+	"""The settings of the user's instance (.env.local and .env.ai-surfaces), with a new port and data folder.
+
+	Feature flags do not come from PostHog: PostHog gives each new instance its own experiment variants,
+	so the screens of two instances with the same code can differ. The flags are their defaults, plus
+	N8N_FEATURE_FLAG_OVERRIDES, plus feature_flags.
+	"""
+	env = {k: v for k, v in os.environ.items() if not k.startswith(INSTANCE_ENV_SKIP)}
+	env.update({k: v for k, v in config().items() if not k.startswith(INSTANCE_ENV_SKIP)})
+	# Keep N8N_DIAGNOSTICS_ENABLED as the user set it: when it is false, Instance AI sends no LangSmith traces.
+	env.update(
+		N8N_PORT=str(port), N8N_RUNNERS_BROKER_PORT=str(free_port()), N8N_USER_FOLDER=str(home),
+		N8N_EDITOR_BASE_URL=f'http://localhost:{port}', WEBHOOK_URL=f'http://localhost:{port}/',
+		N8N_PERSONALIZATION_ENABLED='false', N8N_INSTANCE_AI_EVAL_INSTANCE='true',
+	)
+	if feature_flags:
+		try:
+			overrides = json.loads(env.get('N8N_FEATURE_FLAG_OVERRIDES') or '{}')
+		except ValueError:
+			overrides = {}
+		env['N8N_FEATURE_FLAG_OVERRIDES'] = json.dumps({**overrides, **feature_flags})
+	return env
+
+
+def template_dir(root):
+	"""The data folder template of a checkout. The key has the database migrations and the owner, so a build
+	with other migrations or another N8N_EVAL_EMAIL gets a new template. Change TEMPLATE_VERSION when seed()
+	changes."""
+	migrations = Path(root) / 'packages' / '@n8n' / 'db' / 'dist' / 'migrations'
+	names = sorted(str(p.relative_to(migrations)) for p in migrations.rglob('*.js')) if migrations.exists() else []
+	owner = client('http://localhost')
+	parts = [TEMPLATE_VERSION, str(Path(root).resolve()), owner.email, owner.password, *names]
+	return CACHE / 'templates' / hashlib.sha256('\n'.join(parts).encode()).hexdigest()[:16]
+
+
+class Instance:
+	"""A fresh n8n instance with its own port and data folder, started from a checkout that is built.
+
+	The first instance of a checkout runs the database migrations and the owner setup, then saves its
+	data folder as a template. Later instances start from a copy of the template, which is faster.
+	"""
+
+	def __init__(self, name, root, home, log, feature_flags=None):
+		self.name, self.root, self.home, self.log_path = name, Path(root), Path(home), Path(log)
+		self.feature_flags = feature_flags
+		self.port = free_port()
+		self.url = f'http://localhost:{self.port}'
+		self.proc = None
+
+	def start(self, timeout=180):
+		template = template_dir(self.root)
+		if (template / '.ready').exists():
+			shutil.copytree(template / '.n8n', self.home / '.n8n', dirs_exist_ok=True)
+			self._boot(timeout)
+			return self
+		template.parent.mkdir(parents=True, exist_ok=True)
+		with (template.parent / f'{template.name}.lock').open('w') as lock:
+			# Only one instance makes the template. The others start without it and do not wait.
+			try:
+				fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+				owner = True
+			except BlockingIOError:
+				owner = False
+			self._boot(timeout)
+			seed(self.url)
+			if owner and not (template / '.ready').exists():
+				self._save_template(template)
+		return self
+
+	def _save_template(self, template):
+		"""Stop, copy the data folder, and start again on the same port."""
+		self.stop()
+		shutil.rmtree(template, ignore_errors=True)
+		shutil.copytree(self.home / '.n8n', template / '.n8n', ignore=shutil.ignore_patterns('n8nEventLog*', '*.log'))
+		(template / '.ready').touch()
+		self._boot(60)
+
+	def _boot(self, timeout):
+		self.home.mkdir(parents=True, exist_ok=True)
+		log = self.log_path.open('a')
+		# A new session, so stop() can end n8n and its task runners together.
+		self.proc = subprocess.Popen(['node', str(self.root / 'packages' / 'cli' / 'bin' / 'n8n'), 'start'],
+		                             cwd=self.home, env=instance_env(self.port, self.home, self.feature_flags),
+		                             stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
+		deadline = time.time() + timeout
+		while time.time() < deadline:
+			if self.proc.poll() is not None:
+				raise RuntimeError(f'{self.name} instance stopped at start (exit {self.proc.returncode}), see {self.log_path}')
+			try:
+				with urllib.request.urlopen(f'{self.url}/healthz/readiness', timeout=2) as res:
+					if res.status == 200:
+						return
+			except OSError:
+				pass
+			time.sleep(0.25)
+		raise RuntimeError(f'{self.name} instance did not become ready in {timeout}s, see {self.log_path}')
+
+	def stop(self, clean=False):
+		"""Stop n8n. clean also deletes the static file cache (about 80 MB), which n8n writes again at each start."""
+		if self.proc and self.proc.poll() is None:
+			try:
+				os.killpg(self.proc.pid, signal.SIGTERM)
+				self.proc.wait(timeout=30)
+			except (ProcessLookupError, subprocess.TimeoutExpired):
+				try:
+					os.killpg(self.proc.pid, signal.SIGKILL)
+				except ProcessLookupError:
+					pass
+		if clean:
+			shutil.rmtree(self.home / '.cache', ignore_errors=True)
+
+
+def client(base_url):
+	c = N8nClient()
+	c.base_url = base_url.rstrip('/')
+	return c
+
+
+def seed(base_url):
+	"""Create the owner (N8N_EVAL_EMAIL and N8N_EVAL_PASSWORD) and finish the Assistant setup."""
+	c = client(base_url)
+	c.request('POST', '/rest/owner/setup',
+	          {'email': c.email, 'firstName': 'Feature', 'lastName': 'Test', 'password': c.password})
+	c.login()
+	# Web search is optional. Without a choice, the Assistant shows its setup screen instead of the chat.
+	# A search provider from the environment makes the choice, and the API rejects searchDisabled then.
+	current = c.request('GET', '/rest/instance-ai/settings') or {}
+	update = {'enabled': True}
+	if not current.get('searchEnvConfigured'):
+		update['searchDisabled'] = True
+	c.request('PUT', '/rest/instance-ai/settings', update)
+
+
+def git(*args, cwd=REPO_ROOT):
+	return subprocess.run(['git', *args], cwd=cwd, capture_output=True, text=True, check=True).stdout.strip()

.agents/skills/testmax-assistant-ui/scripts/kit.mjs

@@ -0,0 +1,351 @@
+// The helper object `t` that a scenario script receives. See SKILL.md for the full API.
+//
+//   export default async function (t) {
+//     await t.step('Open the Assistant', () => t.goto());
+//     await t.step('Send a prompt', () => t.send('Reply with exactly: OK'));
+//     await t.step('Wait for the reply', () => t.waitForIdle());
+//     await t.check('The reply shows', () => t.expect(t.ai.assistantMessages().first()).toBeVisible());
+//     await t.capture('thread', { mask: [t.ai.assistantMessages()] });
+//   }
+//
+// A step that throws ends the scenario. A check that fails is recorded and the scenario goes on.
+// Each step and check writes a screenshot. The runner shows them in the report.
+import { mkdirSync, renameSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+
+import { expect } from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
+
+const THREAD_URL = /\/assistant\/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/;
+const RUN_GRACE_MS = 10_000;
+
+/**
+ * Replace the accessibility subtree of each masked element with one line. Masked texts alone keep
+ * the structure, so a list with a different number of rows still shows as a change. Each subtree
+ * replaces only its first unmasked match, so an unmasked element with the same content stays.
+ */
+function maskSubtrees(tree, subtrees) {
+	const lines = tree.split('\n');
+	for (const subtree of subtrees) {
+		const sub = subtree.split('\n').filter((l) => l.trim());
+		if (!sub.length || sub[0].startsWith('#')) continue;
+		for (let i = 0; i + sub.length <= lines.length; i++) {
+			if (!lines[i].endsWith(sub[0])) continue;
+			const indent = lines[i].slice(0, lines[i].length - sub[0].length);
+			if (indent.trim() || !sub.every((l, j) => lines[i + j] === indent + l)) continue;
+			lines.splice(i, sub.length, `${indent}- text: "[masked]"`);
+			break;
+		}
+	}
+	return lines.join('\n');
+}
+
+/** The accessibility tree without the masked texts and the instance port, so branch and master compare. */
+function maskTree(tree, texts) {
+	let out = tree.replace(/localhost(:|%3A)\d+/g, 'localhost$1PORT');
+	for (const text of [...texts].sort((a, b) => b.length - a.length))
+		out = out.split(text).join('[masked]');
+	return out;
+}
+
+export class StepFailed extends Error {}
+
+/** Locators of the Instance AI page. They mirror pages/InstanceAiPage.ts in the Playwright tests. */
+function locators(page) {
+	const container = () => page.getByTestId('instance-ai-container');
+	return {
+		container,
+		chatInput: () => container().getByRole('textbox').or(container().getByRole('combobox')),
+		composer: () => container().getByTestId('instance-ai-composer'),
+		sendButton: () => container().getByTestId('instance-ai-send-button'),
+		stopButton: () => container().getByTestId('instance-ai-stop-button'),
+		userMessages: () => container().getByTestId('instance-ai-user-message'),
+		assistantMessages: () => container().getByTestId('instance-ai-assistant-message'),
+		assistantText: (text) =>
+			container().getByTestId('instance-ai-assistant-message').getByText(text),
+		panelText: (text) => container().getByText(text),
+		emptyState: () => container().getByTestId('instance-ai-empty-state'),
+		statusBar: () => container().getByTestId('instance-ai-status-bar'),
+		sidebarToggle: () => container().getByTestId('instance-ai-sidebar-toggle'),
+		threadList: () => page.getByTestId('instance-ai-thread-list'),
+		newThreadButton: () =>
+			page
+				.getByTestId('project-instance-ai-menu-item')
+				.getByRole('menuitem', { name: 'Assistant', exact: true }),
+		mentionButton: () => container().getByTestId('instance-ai-mention-button'),
+		mentionMenu: () => page.getByTestId('instance-ai-mention-menu-content'),
+		fileInput: () => container().locator('input[type="file"]'),
+		approveButton: () => container().getByTestId('approval-card-allow-once'),
+		alwaysAllowButton: () => container().getByTestId('approval-card-always-allow'),
+		denyButton: () => container().getByTestId('approval-card-deny'),
+		domainAllowOnce: () => container().getByTestId('domain-access-allow-once'),
+		planApproveButton: () => container().getByTestId('instance-ai-plan-approve'),
+		credentialContinue: () => container().getByTestId('instance-ai-credential-continue-button'),
+		previewPanel: () => container().getByTestId('instance-ai-preview-panel'),
+		previewCanvasNodes: () =>
+			container().getByTestId('workflow-canvas-host').locator('[data-test-id="canvas-node"]'),
+		previewToggle: () => container().getByTestId('instance-ai-artifacts-preview-toggle'),
+		artifactsSidebar: () => container().getByTestId('instance-ai-artifacts-sidebar'),
+		outOfCredits: () => container().getByTestId('instance-ai-out-of-credits'),
+	};
+}
+
+export function createKit({ page, out, baseUrl, autoConfirm, emit }) {
+	const started = Date.now();
+	const steps = [];
+	const threads = [];
+	const prompts = [];
+	const ai = locators(page);
+	let stopped = false;
+	mkdirSync(join(out, 'captures'), { recursive: true });
+
+	const seconds = () => (Date.now() - started) / 1000;
+	function save() {
+		const partial = join(out, 'steps.tmp.json');
+		writeFileSync(partial, JSON.stringify(steps, null, 1));
+		renameSync(partial, join(out, 'steps.json'));
+	}
+	async function shot(file) {
+		try {
+			await page.screenshot({ path: join(out, file) });
+			return file;
+		} catch {
+			return null;
+		}
+	}
+	function message(error) {
+		return String(error?.message ?? error)
+			.replace(/\u001b\[[0-9;]*m/g, '')
+			.slice(0, 2000);
+	}
+
+	async function record(kind, title, fn) {
+		if (stopped) throw new StepFailed('The scenario has stopped.');
+		const entry = { i: steps.length + 1, kind, title, status: 'running', t: seconds() };
+		steps.push(entry);
+		save();
+		emit(`${kind} ${title}`);
+		let value;
+		let failed = false;
+		try {
+			value = typeof fn === 'function' ? await fn() : fn;
+			if (value === false) throw new Error('The check returned false.');
+		} catch (error) {
+			failed = true;
+			entry.error = message(error);
+		}
+		// After a stop, run_scenario.mjs owns the steps. A late result must not change them.
+		if (stopped) throw new StepFailed('The scenario has stopped.');
+		entry.status = failed ? 'failed' : 'passed';
+		if (failed) emit(`${kind} failed: ${entry.error.split('\n')[0]}`);
+		entry.duration = seconds() - entry.t;
+		entry.shot = await shot(`${kind}-${String(entry.i).padStart(2, '0')}.png`);
+		if (!stopped) save();
+		if (failed && kind === 'step') throw new StepFailed(entry.error);
+		return failed ? undefined : value;
+	}
+
+	async function api(method, path, body) {
+		return await page.evaluate(
+			async ({ method, path, body }) => {
+				const res = await fetch(path, {
+					method,
+					headers: {
+						'content-type': 'application/json',
+						'browser-id': localStorage.getItem('n8n-browserId') ?? '',
+					},
+					body: body === undefined ? undefined : JSON.stringify(body),
+				});
+				const text = await res.text();
+				if (!res.ok) throw new Error(`${method} ${path} -> ${res.status}: ${text.slice(0, 300)}`);
+				const json = text ? JSON.parse(text) : null;
+				return json && typeof json === 'object' && Object.keys(json).length === 1 && 'data' in json
+					? json.data
+					: json;
+			},
+			{ method, path, body },
+		);
+	}
+
+	function currentThread() {
+		return page.url().match(THREAD_URL)?.[1] ?? null;
+	}
+
+	function noteThread(id) {
+		if (id && !threads.includes(id)) {
+			threads.push(id);
+			emit(`thread ${id}`);
+		}
+	}
+
+	async function threadStatus(id) {
+		const s = await api('GET', `/rest/instance-ai/threads/${id}/status`);
+		const background = (s?.backgroundTasks ?? []).filter((x) => x.status === 'running').length;
+		return { active: Boolean(s?.hasActiveRun), suspended: Boolean(s?.isSuspended), background };
+	}
+
+	const t = {
+		page,
+		ai,
+		expect,
+		baseUrl,
+		threads,
+		prompts,
+
+		/** A step. When it throws, the scenario ends as failed. */
+		step: (title, fn) => record('step', title, fn),
+		/** A soft check. A failure is recorded and the scenario goes on. fn can return false to fail. */
+		check: (title, fn) => record('check', title, fn),
+		/** A line in the step list, with a screenshot. */
+		note: (title) => record('note', title, () => true),
+
+		/** Open a page of the instance. Default: the Assistant, with the composer ready. */
+		async goto(path = '/assistant') {
+			await page.goto(baseUrl + path, { waitUntil: 'domcontentloaded' });
+			if (path.startsWith('/assistant')) {
+				const enable = page.getByRole('dialog').filter({ hasText: 'Try new n8n Assistant' });
+				if (await enable.isVisible({ timeout: 1500 }).catch(() => false)) {
+					await enable
+						.getByRole('button', { name: /Enable n8n Assistant on this instance/ })
+						.click();
+					await enable.getByRole('button', { name: /^(Continue|Enable)$/ }).click();
+				}
+				await expect(ai.chatInput()).toBeVisible({ timeout: 30_000 });
+			}
+			noteThread(currentThread());
+		},
+
+		/** Type a message in the composer and send it. Returns the thread id. */
+		async send(text) {
+			const before = await ai.assistantMessages().count();
+			await expect(ai.chatInput()).toBeVisible({ timeout: 30_000 });
+			await ai.chatInput().fill(text);
+			await expect(ai.sendButton()).toBeEnabled({ timeout: 30_000 });
+			await ai.sendButton().click();
+			await page.waitForURL(THREAD_URL, { timeout: 30_000 });
+			const id = currentThread();
+			noteThread(id);
+			prompts.push({
+				t: seconds(),
+				text,
+				thread: id,
+				assistantMessagesBefore: before,
+				sentAt: Date.now(),
+			});
+			return id;
+		},
+
+		/**
+		 * Wait until the thread has no active run, no open confirmation, and no running background task.
+		 * Returns 'idle', or 'suspended' when a confirmation card waits for the scenario
+		 * (autoConfirm is off, or the card stays open for 20 seconds).
+		 */
+		async waitForIdle({ timeout = 600_000, settle = 3000 } = {}) {
+			const id = currentThread() ?? threads.at(-1);
+			if (!id) throw new Error('No thread is open. Call t.send() first.');
+			const last = prompts.filter((p) => p.thread === id).at(-1);
+			const deadline = Date.now() + timeout;
+			let seenRun = false;
+			let idleSince = null;
+			let suspendedSince = null;
+			while (!stopped && Date.now() < deadline) {
+				let s;
+				try {
+					s = await threadStatus(id);
+				} catch {
+					await page.waitForTimeout(1000);
+					continue;
+				}
+				// The status can show no run for a moment after the message is sent. A run that the user
+				// stops at once can end before it shows, so after a grace period "no run" means "done".
+				if (!seenRun && Date.now() - (last?.sentAt ?? 0) > RUN_GRACE_MS) {
+					seenRun = true;
+				}
+				if (!seenRun) {
+					const replies = await ai
+						.assistantMessages()
+						.count()
+						.catch(() => 0);
+					seenRun =
+						s.active ||
+						s.suspended ||
+						s.background > 0 ||
+						replies > (last?.assistantMessagesBefore ?? 0);
+				}
+				if (s.suspended) {
+					suspendedSince ??= Date.now();
+					if (!autoConfirm || Date.now() - suspendedSince > 20_000) return 'suspended';
+				} else {
+					suspendedSince = null;
+				}
+				const idle = seenRun && !s.active && !s.suspended && s.background === 0;
+				if (!idle) {
+					idleSince = null;
+				} else {
+					idleSince ??= Date.now();
+					if (Date.now() - idleSince >= settle) {
+						await ai
+							.sendButton()
+							.waitFor({ state: 'visible', timeout: 10_000 })
+							.catch(() => {});
+						return 'idle';
+					}
+				}
+				await page.waitForTimeout(1000);
+			}
+			throw new Error(`The thread did not go idle in ${Math.round(timeout / 1000)}s.`);
+		},
+
+		/**
+		 * Save a screen for the visual check. Visual scenarios run on this branch and on master, and the
+		 * report compares the screens with the same name. mask hides changing parts, such as LLM text.
+		 * root is the part of the page for the accessibility tree diff (default: the whole page).
+		 */
+		async capture(name, { mask = [], maskText = [], root, fullPage = false } = {}) {
+			if (!/^[a-z0-9][a-z0-9-]*$/.test(name))
+				throw new Error(`Capture name "${name}": use a-z, 0-9 and "-".`);
+			return await record('capture', `Capture ${name}`, async () => {
+				await page.waitForLoadState('networkidle', { timeout: 5000 }).catch(() => {});
+				await page.waitForTimeout(500);
+				const texts = new Set(maskText.filter(Boolean));
+				for (const locator of mask) {
+					for (const inner of await locator.allInnerTexts().catch(() => [])) {
+						for (const line of inner.split('\n'))
+							if (line.trim().length > 1) texts.add(line.trim());
+					}
+				}
+				const textMasks = maskText.filter(Boolean).map((s) => page.getByText(s, { exact: true }));
+				await page.screenshot({
+					path: join(out, 'captures', `${name}.png`),
+					mask: [...mask, ...textMasks],
+					maskColor: '#FF00FF',
+					animations: 'disabled',
+					caret: 'hide',
+					fullPage,
+				});
+				const tree = await (root ?? page.locator('body'))
+					.ariaSnapshot({ timeout: 10_000 })
+					.catch((e) => `# ${message(e)}`);
+				const subtrees = [];
+				for (const locator of mask) {
+					for (const element of await locator.all().catch(() => [])) {
+						subtrees.push(await element.ariaSnapshot({ timeout: 5_000 }).catch(() => ''));
+					}
+				}
+				writeFileSync(
+					join(out, 'captures', `${name}.aria.yml`),
+					maskTree(maskSubtrees(tree, subtrees), texts) + '\n',
+				);
+			});
+		},
+
+		/** A REST call as the signed-in user. Returns the `data` field of the answer when it has one. */
+		api,
+		threadStatus,
+		sleep: (ms) => page.waitForTimeout(ms),
+	};
+	/** End the scenario: later steps throw, and a running step does not change the steps. */
+	const stop = () => {
+		stopped = true;
+	};
+	return { t, steps, threads, prompts, stop };
+}

.agents/skills/testmax-assistant-ui/scripts/live_page.py

@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+"""Serve the live page of a feature test run, then its report.
+
+  live_page.py --run-dir <run-dir> [--parent-pid <pid>] [--port <port>]
+
+run.py starts it in the background. It uses the server of testmax-n8n-ai
+(live_server.py): 127.0.0.1 only, the address in <run-dir>/live-url.txt, and it stops
+30 minutes after the last request when the run is done. When verdicts.json changes after the
+run, the page builds the report again, and the open report reloads.
+"""
+
+import argparse
+import threading
+import time
+from pathlib import Path
+
+import common  # noqa: F401  (puts the shared scripts on the import path)
+import report
+from live_server import read_status, serve
+from report import ScenarioApp
+
+
+def rebuild_on_verdicts(run_dir):
+	verdicts, built = run_dir / 'verdicts.json', None
+	while True:
+		time.sleep(2)
+		if read_status(run_dir).get('phase') != 'done' or not verdicts.exists():
+			continue
+		mtime = verdicts.stat().st_mtime
+		if mtime == built:
+			continue
+		try:
+			report.build(run_dir)
+			built = mtime
+		except ValueError:
+			# The judge is still writing the file. The next check reads it again.
+			pass
+		except Exception as e:
+			print(f'cannot build the report: {e}', flush=True)
+			built = mtime
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--run-dir', required=True)
+	ap.add_argument('--parent-pid', type=int)
+	ap.add_argument('--port', type=int, default=0)
+	args = ap.parse_args()
+	run_dir = Path(args.run_dir).resolve()
+	threading.Thread(target=rebuild_on_verdicts, args=(run_dir,), daemon=True).start()
+	serve(ScenarioApp(run_dir), args.parent_pid, args.port)
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-assistant-ui/scripts/report.py

@@ -0,0 +1,690 @@
+#!/usr/bin/env python3
+"""Build the report of a feature test run, and the live page while it runs.
+
+  report.py <run-dir>          build <run-dir>/report/index.html (run it again after verdicts.json)
+  report.py <run-dir> --summary  print the summary for the judge
+
+Inputs in <run-dir>:
+  plan/scenarios.json    the plan (run.py copies it)
+  status.json            the status of each run (run.py writes it)
+  runs/<name>/           the files of one run (run_scenario.mjs and run.py write them)
+  visual.json            the visual check (visual_diff.mjs and run.py write it)
+  verdicts.json          optional: {"<name>": {"verdict": "pass|partial|fail", "summary": "...",
+                         "checks": [{"ok": true|false|null, "text": "..."}]}}
+
+The page uses the template and the LLM steps view of testmax-n8n-ai.
+"""
+
+import fcntl
+import json
+import shutil
+import sys
+import time
+from datetime import datetime
+from pathlib import Path
+
+import common  # noqa: F401  (puts the shared scripts on the import path)
+import build_report as br
+import live_server as ls
+
+STATUS = {'waiting': ('st-waiting', 'Waiting'), 'starting': ('st-waiting', 'Starting'), 'running': ('st-running', 'Running'),
+          'passed': ('st-done', 'Passed'), 'failed': ('st-failed', 'Failed'), 'error': ('st-failed', 'Error'),
+          'skipped': ('st-skipped', 'Skipped')}
+FINISHED = ('passed', 'failed', 'error', 'skipped')
+KIND_LABELS = {'feature': 'Feature', 'regression': 'Regression', 'visual': 'Visual'}
+VISUAL = {'changed': ('vis-changed', 'Changed'), 'unchanged': ('vis-same', 'Unchanged'),
+          'branch-only': ('vis-only', 'Only on branch'), 'master-only': ('vis-only', 'Only on master')}
+THEAD = ('<tr><th>Scenario</th><th>Kind</th><th>Result</th><th>Steps</th><th>Time</th><th>LLM calls</th>'
+         '<th>Tool calls</th><th>Input tokens</th><th>Output tokens</th><th>Cost</th><th>Browser errors</th></tr>')
+
+CSS = """<style>
+.run-video video{display:block;width:100%;max-height:70vh;aspect-ratio:16/10;object-fit:contain;background:var(--black);border-radius:var(--r8)}
+.badge.vis-changed{background:var(--pink-70);color:var(--bg)}
+.badge.vis-same{background:var(--bg-3);color:var(--black)}
+.badge.vis-only{box-shadow:inset 0 0 0 1px var(--line-strong);color:var(--dim)}
+.facts{display:grid;grid-template-columns:max-content minmax(0,1fr);gap:var(--s4) var(--s16);margin:0}
+.facts dt{font:10px/18px var(--mono);text-transform:uppercase;letter-spacing:.02em;color:var(--dim)}
+.facts dd{margin:0;word-break:break-word}
+.run-error{margin:var(--s12) 0 0;background:var(--pink-10);color:var(--pink-70);border-radius:var(--r8);padding:var(--s12);white-space:pre-wrap;word-break:break-word;font:12px/16px var(--mono);max-height:240px;overflow:auto}
+.tsteps{list-style:none;margin:0;padding:0;display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--s16)}
+@media (max-width:1000px){.tsteps{grid-template-columns:repeat(2,minmax(0,1fr))}}
+@media (max-width:640px){.tsteps{grid-template-columns:minmax(0,1fr)}}
+.tstep{display:flex;flex-direction:column;min-width:0;background:var(--white);border:1px solid var(--line);border-radius:var(--r8);overflow:hidden}
+.tstep.failed{background:var(--pink-10);border-color:var(--pink-70);box-shadow:0 0 0 1px var(--pink-70)}
+.tstep-shot{display:block;width:100%;aspect-ratio:16/10;padding:0;border:0;border-bottom:1px solid var(--line);background:var(--bg-3);cursor:zoom-in;overflow:hidden}
+.tstep-shot img{display:block;width:100%;height:100%;object-fit:cover;object-position:top center;transition:transform .15s}
+.tstep-shot:hover img{transform:scale(1.03)}
+.tstep-shot:focus-visible{outline:2px solid var(--pink);outline-offset:-2px}
+@media (prefers-reduced-motion:reduce){.tstep-shot img{transition:none}}
+.tstep-noshot{display:grid;place-items:center;aspect-ratio:16/10;background:var(--bg-2);border-bottom:1px solid var(--line);color:var(--gray-50);font:10px/14px var(--mono);text-transform:uppercase;letter-spacing:.02em}
+.tstep-info{display:flex;gap:var(--s12);align-items:flex-start;padding:var(--s12);min-width:0}
+.tstep-info > div{flex:1;min-width:0;display:flex;flex-direction:column;gap:var(--s4)}
+.tstep-info.running .num{background:var(--pink-10);color:var(--pink-70)}
+.tstep-info.passed .num{background:var(--pink)}
+.tstep-info.failed .num{background:var(--pink-70);color:var(--bg)}
+.tstep-title{font-weight:500;padding-top:3px;word-break:break-word}
+.tstep-title .mono-label{font-weight:400;margin-right:var(--s8)}
+.tstep-info pre{margin:var(--s4) 0 0;white-space:pre-wrap;word-break:break-word;font:12px/16px var(--mono);color:var(--pink-70);max-height:200px;overflow:auto}
+.shot-viewer .theater-bar{flex-wrap:nowrap;align-items:flex-start}
+.shot-viewer-info{flex:1;min-width:0;padding-top:3px}
+.shot-viewer-actions{flex:none;display:flex;align-items:center;gap:var(--s16)}
+.shot-viewer-info .tstep-info{padding:0}
+.shot-viewer-info .tstep-title{font-size:18px;line-height:22px}
+.shot-viewer-img{flex:1;min-height:0;display:flex;align-items:center;justify-content:center;cursor:zoom-out}
+.shot-viewer-img img{display:block;max-width:100%;max-height:100%;border-radius:var(--r8);box-shadow:0 0 0 1px var(--line);cursor:default}
+.err-list{margin:0;padding-left:var(--s16);font:12px/16px var(--mono)}
+.err-list li{margin-bottom:var(--s4);word-break:break-word}
+.vis{background:var(--white);border-radius:var(--r8);padding:var(--s12);box-shadow:inset 0 0 0 1px var(--line);margin-top:var(--s12)}
+.vis.changed{box-shadow:inset 4px 0 0 var(--pink-70),inset 0 0 0 1px var(--line)}
+.vis-grid{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--s12)}
+.vis-grid figure{margin:0}
+.vis-grid figcaption{margin-bottom:var(--s4)}
+.vis-grid img{display:block;width:100%;height:auto;border-radius:var(--r4);box-shadow:0 0 0 1px var(--line)}
+.vis details summary{cursor:pointer;color:var(--dim);margin-top:var(--s8);font:10px/14px var(--mono);text-transform:uppercase;letter-spacing:.02em}
+.vis pre{background:var(--bg);border-radius:var(--r4);padding:var(--s12);margin:var(--s8) 0 0;overflow:auto;max-height:320px;font-size:12px;line-height:16px;white-space:pre}
+</style>"""
+
+
+def read_json(path, default):
+	try:
+		return json.loads(path.read_text())
+	except (OSError, ValueError):
+		return default
+
+
+def seconds(value):
+	return f'{value:.0f}s'
+
+
+def load_trace(sub, result):
+	"""LLM steps from the thread events, and per-call tokens from usage.json when LangSmith had them."""
+	data = ls.load_instance_ai_live(sub)
+	events_file = sub / 'events.jsonl'
+	rows = br.read_jsonl(events_file) if events_file.exists() else []
+	starts = [r['ts'] / 1000 for r in rows if r.get('type') == 'run-start' and r.get('ts')]
+	usage = read_json(sub / 'usage.json', None)
+	if usage and starts:
+		start = starts[0]
+		data['calls'] = sorted((
+			{'t': br.ts(u['start']) - start, 'start': br.ts(u['start']) - start,
+			 'end': br.ts(u['end']) - start if u.get('end') else None,
+			 'name': (u.get('name') or '').removeprefix('llm: ') or None,
+			 'prompt': u['in'], 'cache_read': u['cache_read'], 'cache_write': u['cache_write'],
+			 'out': u['out'], 'cost': u['cost']} for u in usage), key=lambda c: c['t'])
+		data['model'] = ', '.join(sorted({u['model'] for u in usage if u.get('model')})) or None
+		data['cost'] = f'${sum(c["cost"] for c in data["calls"]):.2f}'
+		data['costSource'] = 'langsmith'
+	if result.get('startedAt') and result.get('finishedAt'):
+		data['duration'] = seconds(br.ts(result['finishedAt']) - br.ts(result['startedAt']))
+	return data
+
+
+def load(run_dir):
+	"""The plan, the status, and one dict for each run in plan order."""
+	plan = read_json(run_dir / 'plan' / 'scenarios.json', {'scenarios': []})
+	status = ls.read_status(run_dir)
+	entries = status.get('runs') or {}
+	runs = []
+	for s in plan['scenarios']:
+		for name in (s['id'], f'{s["id"]}--master'):
+			if name not in entries:
+				continue
+			entry = entries[name]
+			sub = run_dir / 'runs' / name
+			result = read_json(sub / 'result.json', {})
+			data = load_trace(sub, result)
+			if entry.get('status') in ('waiting', 'running'):
+				data['cost'] = '-'
+				if entry.get('startedAt'):
+					data['duration'] = seconds((entry.get('finishedAt') or time.time()) - entry['startedAt'])
+			runs.append({
+				'name': name, 'scenario': s, 'entry': entry, 'master': name.endswith('--master'),
+				'steps': read_json(sub / 'steps.json', []), 'result': result,
+				'console': read_json(sub / 'console.json', {}), 'meta': read_json(sub / 'run.json', {}),
+				'data': data,
+			})
+	return plan, status, runs
+
+
+def status_badge(state):
+	cls, label = STATUS.get(state, ('st-waiting', state))
+	return f'<span class="badge {cls}">{br.esc(label)}</span>'
+
+
+def run_title(run):
+	return run['scenario']['title'] + (' (master)' if run['master'] else '')
+
+
+def check_count(steps):
+	checked = [s for s in steps if s['kind'] in ('step', 'check', 'capture')]
+	return f'{sum(s["status"] == "passed" for s in checked)}/{len(checked)}' if checked else '-'
+
+
+def row_cells(run, result_html):
+	data, result = run['data'], run['result']
+	calls, tools = data['calls'], [e for e in data['events'] if e['kind'] == 'tool']
+	errors = sum(e['error'] for e in tools)
+	browser = (result.get('consoleErrors') or 0) + (result.get('requestErrors') or 0)
+	cells = [
+		f'<span class="run-name">{br.esc(run_title(run))}</span>',
+		KIND_LABELS.get(run['scenario']['kind'], run['scenario']['kind']),
+		result_html, check_count(run['steps']), data['duration'], str(len(calls)) if calls else '-',
+		f'{len(tools)}' + (f' <span class="err">({errors} err)</span>' if errors else ''),
+		br.kilo(sum(c['prompt'] for c in calls)) if calls else '-',
+		br.kilo(sum(c['out'] for c in calls)) if calls else '-',
+		f'<strong>{br.esc(data["cost"])}</strong>',
+		f'<span class="err">{browser}</span>' if browser else '0',
+	]
+	return ''.join(f'<td>{c}</td>' for c in cells)
+
+
+def result_html(run, verdict):
+	"""The judge's verdict. The script status shows only while it runs or when it fails, because the Steps
+	column already counts the steps that passed."""
+	if verdict and verdict.get('verdict'):
+		return br.badge(verdict['verdict'])
+	state = run['entry'].get('status', 'waiting')
+	if state != 'passed':
+		return status_badge(state)
+	if run['master']:
+		return '<span class="badge st-done">Baseline</span>'
+	return br.badge(None)
+
+
+def render_video(screen):
+	"""The video of the run, above the expected outcome. Only the built report has it."""
+	if not screen:
+		return ''
+	poster = f' poster="{br.esc(screen["poster"])}"' if screen.get('poster') else ''
+	return br.fold('Video', f'<span class="mono-label">n8n UI · {br.esc(screen["duration"])}</span>',
+	               f'<div class="run-video"><video controls muted playsinline preload="metadata" '
+	               f'src="{br.esc(screen["src"])}"{poster}></video></div>', cls='brief-card prompt')
+
+
+def render_expected(run, verdict):
+	head = br.verdict_head(verdict) if verdict else ''
+	body = f'<div class="msg">{br.esc(run["scenario"]["expected"])}</div>'
+	if verdict:
+		body += f'<div class="verdict-body">{br.render_checks(verdict)}</div>'
+	elif run['entry'].get('status') in FINISHED:
+		body += '<p class="dim">Not judged yet.</p>'
+	return br.fold('Expected outcome', head, body)
+
+
+def render_run_box(run):
+	entry, result, meta = run['entry'], run['result'], run['meta']
+	facts = [('Instance', f'{"master" if run["master"] else "branch"} · {entry.get("baseUrl") or "-"}')]
+	if entry.get('startedAt'):
+		facts.append(('Started', datetime.fromtimestamp(entry['startedAt']).strftime('%H:%M:%S')))
+	if result.get('threads'):
+		facts.append(('Threads', ', '.join(result['threads'])))
+	if meta.get('cached'):
+		facts.append(('Reused', 'This master run is from an earlier run of the same master commit and script.'))
+	if meta.get('decisions'):
+		facts.append(('Confirmations', f'{len(meta["decisions"])} answered by the auto-approve policy'))
+	if result:
+		facts.append(('Browser errors', f'{result.get("consoleErrors", 0)} console, {result.get("requestErrors", 0)} requests'))
+	dl = ''.join(f'<dt>{br.esc(k)}</dt><dd>{br.esc(v)}</dd>' for k, v in facts)
+	error = result.get('error') or (entry.get('reason') if entry.get('status') in ('error', 'skipped') else None)
+	error_html = f'<pre class="run-error">{br.esc(error)}</pre>' if error else ''
+	return br.fold('Run', status_badge(entry.get('status', 'waiting')), f'<dl class="facts">{dl}</dl>{error_html}')
+
+
+def render_scenario_steps(steps, files):
+	if not steps:
+		return br.fold('Scenario steps', '', '<p class="dim">No steps yet.</p>')
+	items = []
+	for s in steps:
+		status, title = br.esc(s['status']), br.esc(s['title'])
+		duration = f' · took {s["duration"]:.1f}s' if s.get('duration') is not None else ''
+		if s.get('shot'):
+			src = f'{files}{br.esc(s["shot"])}'
+			thumb = (f'<button type="button" class="tstep-shot" data-src="{src}" aria-label="Show step {s["i"]} '
+			         f'full screen"><img src="{src}" loading="lazy" alt="Screen after: {title}"></button>')
+		else:
+			thumb = '<div class="tstep-noshot">No screenshot</div>'
+		error = f'<pre>{br.esc(s["error"])}</pre>' if s.get('error') else ''
+		items.append(
+			f'<li class="tstep {status}">{thumb}<div class="tstep-info {status}"><span class="num">{s["i"]}</span><div>'
+			f'<div class="tstep-title"><span class="mono-label">{br.esc(s["kind"])}</span><span>{title}</span></div>'
+			f'<div class="mono-label">at {s["t"]:.1f}s{duration}</div>{error}</div></div></li>')
+	passed = sum(s['status'] == 'passed' for s in steps)
+	return br.fold('Scenario steps', f'<span class="mono-label">{passed} of {len(steps)} passed</span>',
+	               f'<ol class="tsteps">{"".join(items)}</ol>')
+
+
+def render_errors(console):
+	console_errors, request_errors = console.get('consoleErrors') or [], console.get('requestErrors') or []
+	if not console_errors and not request_errors:
+		return ''
+	items = [f'<li>{br.esc(e.get("text", ""))}</li>' for e in console_errors[:50]]
+	items += [f'<li>{e.get("status") or "failed"} {br.esc(e.get("method", ""))} {br.esc(e.get("url", ""))}'
+	          f'{" · " + br.esc(e["error"]) if e.get("error") else ""}</li>' for e in request_errors[:50]]
+	return br.fold('Browser errors', f'<span class="mono-label">{len(console_errors)} console · '
+	               f'{len(request_errors)} requests</span>', f'<ul class="err-list">{"".join(items)}</ul>')
+
+
+def render_panel(run, verdict, files, scale, screen=None):
+	data = run['data']
+	prompt = (run['result'].get('prompts') or [{}])[0].get('text', '')
+	load_error = f'<div class="box err">Could not read the trace: {br.esc(data["loadError"])}</div>' if data.get('loadError') else ''
+	return (f'{load_error}{render_video(screen)}'
+	        f'{render_expected(run, verdict)}{render_run_box(run)}'
+	        f'{render_scenario_steps(run["steps"], files)}'
+	        f'{br.render_steps(data["calls"], data["events"], *scale, prompt)}'
+	        f'{render_errors(run["console"])}')
+
+
+def render_case(plan, status):
+	branch, master = status.get('branch') or {}, status.get('master')
+	kinds = [s['kind'] for s in plan['scenarios']]
+	lines = [f'Branch {branch.get("commit") or "?"}' + (' with local changes' if branch.get('dirty') else '')
+	         + (f' on {branch["url"]}' if branch.get('url') else ' on a fresh instance')]
+	if master is not None:
+		lines.append(f'Master {(master.get("commit") or "(building)")[:11]} (the merge base) on a fresh instance')
+	lines.append(f'{len(kinds)} scenarios: ' + ', '.join(f'{kinds.count(k)} {k}' for k in KIND_LABELS if kinds.count(k)))
+	lines.append(f'{status.get("concurrency", 1)} at a time')
+	return (f'<div class="case"><div class="box"><div class="mono-label">Feature</div><div class="msg">'
+	        f'{br.esc(plan.get("feature") or "-")}</div></div><div class="box"><div class="mono-label">Run</div>'
+	        f'<div class="msg">{br.esc(chr(10).join(lines))}</div></div></div>')
+
+
+def notes(status, runs, verdicts):
+	parts = []
+	if not verdicts and status.get('phase') == 'done':
+		parts.append('The runs are not judged yet. This page reloads when the verdicts and the token usage are ready.')
+	parts.append('Times are seconds from the start of each scenario. Several scenarios ran at the same time on one '
+	             'machine, so their times include the shared load. Tokens and cost come from LangSmith, matched by the '
+	             'traces of the threads of each scenario. An asterisk on a time marks text placed at the previous tool call.')
+	if any(r['master'] for r in runs):
+		parts.append('Visual scenarios ran on a fresh branch instance and on a fresh master instance, so both start '
+		             'from the same empty state. Master is the merge base of the branch.')
+	if status.get('stale'):
+		parts.append('Warning: these packages had source changes newer than their build, so the branch instance ran '
+		             f'an old build: {", ".join(status["stale"])}.')
+	return br.esc(' '.join(parts))
+
+
+def render_visual(results, runs, files):
+	if not results:
+		return ''
+	titles = {r['scenario']['id']: r['scenario']['title'] for r in runs}
+	cards = []
+	for r in results:
+		cls, label = VISUAL.get(r['status'], ('vis-only', r['status']))
+		ratio = f'{100 * r["ratio"]:.2f}% of pixels' if r.get('ratio') is not None else ''
+		sizes = (f' · size {r["masterSize"][0]}×{r["masterSize"][1]} → {r["branchSize"][0]}×{r["branchSize"][1]}'
+		         if r.get('branchSize') and r['branchSize'] != r.get('masterSize') else '')
+		figures = ''
+		for caption, key in (('Master', 'master'), ('Branch', 'branch'), ('Difference', 'diff')):
+			src = files(r, key)
+			figures += (f'<figure><figcaption class="mono-label">{caption}</figcaption>'
+			            + (f'<a href="{src}" target="_blank"><img src="{src}" loading="lazy" alt="{caption}: {br.esc(r["name"])}"></a>'
+			               if src else '<p class="dim">No screen.</p>') + '</figure>')
+		tree = (f'<details><summary>Accessibility tree diff</summary><pre>{br.esc(r["ariaDiff"])}</pre></details>'
+		        if r.get('ariaDiff') else '')
+		cards.append(
+			f'<div class="vis {"changed" if r["status"] != "unchanged" else ""}"><div class="head">'
+			f'<h3>{br.esc(titles.get(r["scenario"], r["scenario"]))} › {br.esc(r["name"])}</h3>'
+			f'<span class="mono-label">{ratio}{sizes}</span><span class="badge {cls}">{label}</span></div>'
+			f'<div class="vis-grid">{figures}</div>{tree}</div>')
+	changed = sum(r['status'] != 'unchanged' for r in results)
+	return br.fold('Visual check against master',
+	               f'<span class="mono-label">{len(results)} captures · {changed} changed</span>',
+	               '<p class="fineprint">Pink pixels in the difference changed by more than a small color threshold. '
+	               'Magenta boxes are masked parts, such as LLM text. The accessibility tree diff shows changes in '
+	               'structure, roles and labels.</p>' + ''.join(cards))
+
+
+STEP_VIEWER_JS = """
+// A step screenshot opens full screen. The listeners are delegated, because the live page replaces the panels.
+(()=>{
+  const viewer = document.createElement('div');
+  viewer.className = 'theater shot-viewer';
+  viewer.hidden = true;
+  viewer.setAttribute('role', 'dialog');
+  viewer.setAttribute('aria-modal', 'true');
+  viewer.setAttribute('aria-label', 'Step screenshot');
+  viewer.innerHTML = '<div class="theater-bar"><div class="shot-viewer-info"></div><div class="shot-viewer-actions">'
+    + '<span class="mono-label" data-shot-count></span>'
+    + '<button type="button" class="btn" data-shot-move="-1">Previous</button>'
+    + '<button type="button" class="btn" data-shot-move="1">Next</button>'
+    + '<button type="button" class="btn accent" data-shot-close>Close</button></div></div>'
+    + '<div class="shot-viewer-img"><img alt=""></div>'
+    + '<p class="fineprint">Press the left or right arrow key to change the step. Press Esc to close.</p>';
+  document.body.append(viewer);
+  let run = null, index = 0, opener = null;
+  const shots = () => [...(document.getElementById(run)?.querySelectorAll('.tstep-shot') || [])];
+  function render(){
+    const items = shots();
+    if (!items.length) return close();
+    index = Math.min(index, items.length - 1);
+    const button = items[index], img = viewer.querySelector('.shot-viewer-img img');
+    img.src = button.dataset.src;
+    img.alt = button.querySelector('img').alt;
+    viewer.querySelector('.shot-viewer-info').replaceChildren(button.closest('.tstep').querySelector('.tstep-info').cloneNode(true));
+    viewer.querySelector('[data-shot-count]').textContent = `${index + 1} of ${items.length}`;
+  }
+  function open(button){
+    run = button.closest('.run').id;
+    index = Math.max(0, shots().indexOf(button));
+    opener = button;
+    viewer.hidden = false;
+    document.body.classList.add('theater-open');
+    render();
+    viewer.querySelector('[data-shot-close]').focus();
+  }
+  function close(){
+    viewer.hidden = true;
+    run = null;
+    document.body.classList.remove('theater-open');
+    if (opener?.isConnected) opener.focus();
+  }
+  function move(step){
+    const count = shots().length;
+    if (!count) return close();
+    index = (index + step + count) % count;
+    render();
+  }
+  document.addEventListener('click', e=>{
+    const button = e.target.closest('.tstep-shot');
+    if (button) open(button);
+  });
+  viewer.addEventListener('click', e=>{
+    const step = e.target.closest('[data-shot-move]');
+    if (step) move(Number(step.dataset.shotMove));
+    else if (e.target.closest('[data-shot-close]') || e.target.classList.contains('shot-viewer-img')) close();
+  });
+  document.addEventListener('keydown', e=>{
+    if (viewer.hidden) return;
+    if (e.key === 'Escape') close();
+    else if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') { e.preventDefault(); move(e.key === 'ArrowRight' ? 1 : -1); }
+  });
+})();
+"""
+
+VIDEO_JS = """
+// The video of a run plays only while its panel shows.
+const showPanel = show;
+show = (id)=>{
+  showPanel(id);
+  document.querySelectorAll('.run:not(.active) .run-video video').forEach(v=>v.pause());
+};
+"""
+
+RELOAD_JS = """
+// Served by the live page: reload when the report is built again, for example after the verdicts.
+(()=>{
+  if (!location.protocol.startsWith('http')) return;
+  const saved = JSON.parse(sessionStorage.getItem('report-view') || 'null');
+  if (saved) {
+    sessionStorage.removeItem('report-view');
+    if (document.getElementById(saved.run)) show(saved.run);
+    window.scrollTo(0, saved.y);
+  }
+  setInterval(async ()=>{
+    try {
+      const version = (await (await fetch('version.txt', {cache: 'no-store'})).text()).trim();
+      if (version && version !== REPORT_VERSION) {
+        const run = document.querySelector('.run.active')?.id;
+        sessionStorage.setItem('report-view', JSON.stringify({run, y: window.scrollY}));
+        location.reload();
+      }
+    } catch {}
+  }, 3000);
+})();
+"""
+
+
+def build(run_dir):
+	"""Write report/index.html with its images and videos. Returns its path. Two builds of one run can
+	happen at the same time (run.py, report.py, the live page), so they take turns."""
+	run_dir.mkdir(parents=True, exist_ok=True)
+	with (run_dir / 'report.lock').open('w') as lock:
+		fcntl.flock(lock, fcntl.LOCK_EX)
+		return _build(run_dir)
+
+
+def _build(run_dir):
+	plan, status, runs = load(run_dir)
+	verdicts = br.load_verdicts(run_dir)
+	report = run_dir / 'report'
+	for sub in ('runs', 'screens', 'visual'):
+		shutil.rmtree(report / sub, ignore_errors=True)
+	report.mkdir(parents=True, exist_ok=True)
+	for sub in ('fonts', 'brand'):
+		if (br.ASSET_CACHE / sub).exists():
+			shutil.copytree(br.ASSET_CACHE / sub, report / sub, dirs_exist_ok=True)
+
+	scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
+	rows, panels, screens = [], [], []
+	for i, run in enumerate(runs):
+		name, sub = run['name'], run_dir / 'runs' / run['name']
+		shots = report / 'runs' / name
+		for png in sub.glob('*.png'):
+			if png.name not in ('live.png', 'screen-poster.png'):
+				shots.mkdir(parents=True, exist_ok=True)
+				shutil.copy(png, shots / png.name)
+		verdict = verdicts.get(name)
+		screen = None
+		if (sub / 'screen.webm').exists():
+			(report / 'screens').mkdir(exist_ok=True)
+			shutil.copy(sub / 'screen.webm', report / 'screens' / f'{name}.webm')
+			screen = {'name': name, 'title': run_title(run), 'duration': run['data']['duration'], 'type': 'video',
+			          'src': f'screens/{name}.webm', 'verdict': (verdict or {}).get('verdict'),
+			          'label': KIND_LABELS.get(run['scenario']['kind'], '')}
+			if (sub / 'screen-poster.png').exists():
+				shutil.copy(sub / 'screen-poster.png', report / 'screens' / f'{name}-poster.png')
+				screen['poster'] = f'screens/{name}-poster.png'
+			screens.append(screen)
+		active = ' active' if i == 0 else ''
+		rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">'
+		            + row_cells(run, result_html(run, verdict)) + '</tr>')
+		panels.append(f'<section class="run{active}" id="{name}">'
+		              + render_panel(run, verdict, f'runs/{name}/', scale, screen) + '</section>')
+
+	results = read_json(run_dir / 'visual.json', [])
+	for r in results:
+		target = report / 'visual' / r['scenario']
+		target.mkdir(parents=True, exist_ok=True)
+		for key, source in (('branch', run_dir / 'runs' / r['scenario'] / 'captures' / f'{r["name"]}.png'),
+		                    ('master', run_dir / 'runs' / f'{r["scenario"]}--master' / 'captures' / f'{r["name"]}.png'),
+		                    ('diff', run_dir / (r.get('diff') or 'missing'))):
+			if source.is_file():
+				shutil.copy(source, target / f'{r["name"]}.{key}.png')
+
+	def visual_file(r, key):
+		return (f'visual/{r["scenario"]}/{r["name"]}.{key}.png'
+		        if (report / 'visual' / r['scenario'] / f'{r["name"]}.{key}.png').exists() else None)
+
+	replay = br.render_replay(screens, 'Each video shows one scenario in the n8n UI. Press Esc to close.')
+	version = str(time.time_ns())
+	page = br.fill_template(
+		br.esc(plan.get('title') or 'Instance AI feature test'), ''.join(rows), notes(status, runs, verdicts),
+		''.join(panels), replay=replay, after_table=render_visual(results, runs, visual_file), head=CSS,
+		scripts=(f'<script>const SCREENS = {json.dumps(screens)}; const REPORT_VERSION = "{version}";</script>'
+		         f'<script>window.addEventListener("DOMContentLoaded", ()=>{{{VIDEO_JS}{RELOAD_JS}}});</script>'
+		         f'<script>{STEP_VIEWER_JS}</script>'),
+		case=render_case(plan, status), thead=THEAD, kicker='Assistant UI test')
+	(report / 'index.html').write_text(page)
+	(report / 'version.txt').write_text(version)
+	return report / 'index.html'
+
+
+def clock(status):
+	started = status.get('startedAt')
+	elapsed = seconds((status.get('finishedAt') or time.time()) - started) if started else ''
+	return ' · '.join(x for x in (f'{status.get("concurrency", 1)} at a time', elapsed, status.get('step')) if x)
+
+
+def live_box(status, runs):
+	cards = []
+	for run in runs:
+		name, title = run['name'], br.esc(run_title(run))
+		cards.append(
+			f'<div class="live-card"><div class="live-screen"><button type="button" class="live-zoom" '
+			f'data-zoom="{name}" aria-label="Show {title} full screen"><img data-shot="{name}" '
+			f'alt="{title} in the n8n UI" hidden></button>'
+			f'<div class="live-empty" data-shot-empty="{name}">Waiting for the first screenshot</div></div>'
+			f'<div class="reel-meta"><span class="run-name">{title}</span>'
+			f'<span data-live-badge="{name}">{status_badge(run["entry"].get("status", "waiting"))}</span></div></div>')
+	return (f'<div class="box"><div class="head"><h2>Live</h2><span class="mono-label" id="live-clock">'
+	        f'{br.esc(clock(status))}</span></div><div class="live-grid">{"".join(cards)}</div></div>')
+
+
+def render_live(run_dir):
+	plan, status, runs = load(run_dir)
+	verdicts = br.load_verdicts(run_dir)
+	scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
+	rows, panels = [], []
+	for i, run in enumerate(runs):
+		name, active = run['name'], ' active' if i == 0 else ''
+		rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">'
+		            + row_cells(run, result_html(run, verdicts.get(name))) + '</tr>')
+		panels.append(f'<section class="run{active}" id="{name}">'
+		              + render_panel(run, verdicts.get(name), f'/files/runs/{name}/', scale) + '</section>')
+	text = ('This page updates while the scenarios run, and changes to the full report when the run is done. '
+	        'Tokens, cost, the visual check and verdicts come after the run.')
+	return br.fill_template(
+		br.esc(plan.get('title') or 'Instance AI feature test'), ''.join(rows), br.esc(text), ''.join(panels),
+		replay=live_box(status, runs), head=CSS,
+		scripts=f'<script>const SCREENS = [];</script><script>{ls.LIVE_JS}</script><script>{STEP_VIEWER_JS}</script>',
+		case=render_case(plan, status), thead=THEAD, kicker='Assistant UI test')
+
+
+class ScenarioApp:
+	"""The live page for live_server.serve(): one card, table row and panel for each scenario run."""
+
+	def __init__(self, run_dir):
+		self.run_dir = run_dir
+
+	def entry(self, name):
+		return ls.read_status(self.run_dir).get('runs', {}).get(name)
+
+	def finished(self, name):
+		return (self.entry(name) or {}).get('status') in FINISHED
+
+	def done(self):
+		return ls.read_status(self.run_dir).get('phase') == 'done' and (self.run_dir / 'report' / 'index.html').exists()
+
+	def page(self):
+		return render_live(self.run_dir)
+
+	def state(self):
+		plan, status, runs = load(self.run_dir)
+		verdicts = br.load_verdicts(self.run_dir)
+		scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
+		state = {'phase': status.get('phase', 'running'), 'clock': clock(status), 'runs': {}}
+		if self.done():
+			state['report'] = '/report/index.html'
+		for run in runs:
+			name = run['name']
+			shot = self.shot(name)
+			state['runs'][name] = {
+				'row': row_cells(run, result_html(run, verdicts.get(name))),
+				'panel': render_panel(run, verdicts.get(name), f'/files/runs/{name}/', scale),
+				'badge': status_badge(run['entry'].get('status', 'waiting')),
+				'shot': shot.stat().st_mtime if shot.exists() else None,
+			}
+		return state
+
+	def shot(self, name):
+		return self.run_dir / 'runs' / name / 'live.png'
+
+	def casts(self, name):
+		return ()
+
+	def file(self, path):
+		"""The step screenshots of a run while it runs: /files/runs/<name>/<file>.png."""
+		if not path.startswith('/files/runs/') or not path.endswith('.png'):
+			return None
+		base = (self.run_dir / 'runs').resolve()
+		candidate = (self.run_dir / path.removeprefix('/files/')).resolve()
+		return candidate if candidate.is_relative_to(base) and candidate.is_file() else None
+
+
+def triage(run, visual):
+	"""The reasons to look closer at a run. An empty list means that the run looks clean: every step and check
+	passed, with no browser error, no failed tool call, and no visual change."""
+	reasons = []
+	status = run['entry'].get('status')
+	if status != 'passed':
+		reasons.append(f'the run {status}')
+	failed = [s for s in run['steps'] if s['status'] == 'failed']
+	if failed:
+		reasons.append(f'{len(failed)} failed step(s) or check(s)')
+	console = run['console']
+	errors = len(console.get('consoleErrors') or []) + len(console.get('requestErrors') or [])
+	if errors:
+		reasons.append(f'{errors} browser error(s)')
+	tool_errors = sum(e['error'] for e in run['data']['events'] if e['kind'] == 'tool')
+	if tool_errors:
+		reasons.append(f'{tool_errors} failed tool call(s)')
+	changed = [r['name'] for r in visual if r['scenario'] == run['scenario']['id'] and r['status'] != 'unchanged']
+	if changed and not run['master']:
+		reasons.append(f'visual change in {", ".join(changed)}')
+	return reasons
+
+
+def summarize(run_dir):
+	"""Plain text for the judge: a triage list, then each run with its steps, final reply, errors and visual
+	results."""
+	plan, status, runs = load(run_dir)
+	visual = read_json(run_dir / 'visual.json', [])
+	out = [f'# {plan.get("title", "")}', '', 'Feature:', plan.get('feature') or '-', '', '## Triage', '',
+	       'Clean runs passed every step and check, with no browser error, failed tool call or visual change.',
+	       'For these, check that the last reply and the final screenshot match the expected result.',
+	       'Look closely at the other runs: their screenshots, errors and diffs.', '']
+	judged = [r for r in runs if not r['master']]
+	for run in judged:
+		reasons = triage(run, visual)
+		out.append(f'- {run["name"]}: ' + ('clean' if not reasons else 'look closer: ' + '; '.join(reasons)))
+	out.append('')
+	for run in runs:
+		s, result, data = run['scenario'], run['result'], run['data']
+		out += [f'## {run["name"]}: {run_title(run)} ({s["kind"]}, {"master" if run["master"] else "branch"})',
+		        f'Expected: {s["expected"]}',
+		        f'Status: {run["entry"].get("status")}' + (f' ({result["error"]})' if result.get('error') else ''),
+		        'Steps:']
+		for st in run['steps']:
+			shot = f'  (runs/{run["name"]}/{st["shot"]})' if st.get('shot') else ''
+			out.append(f'  {st["i"]}. [{st["status"]}] {st["kind"]}: {st["title"]}{shot}')
+			if st.get('error'):
+				out.append('     ' + st['error'].replace('\n', '\n     ')[:1500])
+		for p in result.get('prompts') or []:
+			out.append(f'Prompt (thread {p.get("thread")}): {p["text"]}')
+		tools = [e for e in data['events'] if e['kind'] == 'tool']
+		if tools:
+			out.append('Tool calls: ' + ', '.join(e['name'] + (' (error)' if e['error'] else '') for e in tools))
+		texts = [e['text'] for e in data['events'] if e['kind'] == 'text']
+		if texts:
+			out.append('Last reply: ' + texts[-1].strip()[:2000])
+		for d in run['meta'].get('decisions') or []:
+			out.append(f'Confirmation: {d["decision"]} [{d.get("toolName")}]')
+		console = run['console']
+		for e in (console.get('consoleErrors') or [])[:8]:
+			out.append(f'Console error: {e.get("text", "")[:300]}')
+		for e in (console.get('requestErrors') or [])[:8]:
+			out.append(f'Request error: {e.get("status")} {e.get("method")} {e.get("url")}')
+		if s['kind'] == 'visual' and not run['master']:
+			for r in (v for v in visual if v['scenario'] == s['id']):
+				ratio = f', {100 * r["ratio"]:.2f}% of pixels' if r.get('ratio') is not None else ''
+				out.append(f'Visual {r["name"]}: {r["status"]}{ratio}')
+				if r.get('ariaDiff'):
+					out.append('  Accessibility tree diff:\n    ' + '\n    '.join(r['ariaDiff'].splitlines()[:40]))
+		out.append('')
+	return '\n'.join(out)
+
+
+def main():
+	if len(sys.argv) < 2:
+		sys.exit(__doc__)
+	run_dir = Path(sys.argv[1]).resolve()
+	if '--summary' in sys.argv:
+		print(summarize(run_dir))
+	else:
+		print(build(run_dir))
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-assistant-ui/scripts/run.py

@@ -0,0 +1,931 @@
+#!/usr/bin/env python3
+"""Run a test plan of Instance AI scenarios and build one report. The judge step is separate.
+
+  run.py --plan <dir>/scenarios.json [--only id1,id2] [--concurrency 4]
+  run.py --rerun last|<run-dir>             the plan of an earlier run, again in a new folder
+  run.py --plan ... --try                   a quick check of new scripts: no master, no live page, no report
+  run.py --plan ... --estimate              print the expected cost and time, then stop
+  run.py --check                            check the setup, then stop
+
+Steps:
+  1. Check the setup, and build this checkout with turbo (a few seconds when nothing changed).
+  2. Start fresh n8n instances from the build. Each feature and regression scenario gets its own
+     instance, unless it sets "isolated": false. Instances start ahead of their scenario, so a
+     scenario does not wait for one.
+  3. Run the scenarios, several at the same time.
+  4. At the same time: build the master merge base (baseline.py), start a fresh master instance and
+     a fresh branch instance, and run each visual scenario on both. A master run that passed is kept
+     and used again while the merge base and the script stay the same.
+  5. Compare the visual captures, write summary.txt for the judge, and build the report.
+  6. Fetch the token usage from LangSmith and build the report again.
+A local page shows the runs live (live_page.py). It opens in the default browser, unless --no-open.
+Settings come from .env.local and .env.ai-surfaces.
+"""
+
+import argparse
+import difflib
+import hashlib
+import json
+import os
+import re
+import shutil
+import signal
+import socket
+import statistics
+import subprocess
+import sys
+import threading
+import time
+import webbrowser
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime, timezone
+from pathlib import Path
+
+import baseline
+import report
+from common import CACHE, HERE, PLANS_ROOT, REPO_ROOT, RUNS_ROOT, SHARED, Instance, client, git, setting
+from langsmith_usage import fetch_calls, totals
+from live import enabled as live_enabled
+from run_instance_ai import decide
+
+KINDS = ('feature', 'regression', 'visual')
+LIB = HERE.parent / 'scenarios'
+ID = re.compile(r'^[a-z0-9][a-z0-9-]{0,60}$')
+MASTER_CACHE = CACHE / 'master-runs'
+MASTER_CACHE_DAYS = 14
+PRINT_LOCK = threading.Lock()
+RUNNERS = set()
+S = '.agents/skills/testmax-assistant-ui/scripts'
+
+
+def say(line, prefix=None):
+	with PRINT_LOCK:
+		print(f'[{prefix}] {line}' if prefix else line, flush=True)
+
+
+def step(title):
+	say(f'\n=== {title} ===')
+
+
+def now_iso():
+	return datetime.now(timezone.utc).isoformat()
+
+
+def slug(text, words=8):
+	return '-'.join(re.findall(r'[a-z0-9]+', text.lower())[:words]) or 'run'
+
+
+def script_path(spec, base):
+	return LIB / spec[4:] if spec.startswith('lib:') else (base / spec).resolve()
+
+
+def load_plan(path):
+	"""Read and check scenarios.json. Returns the plan with each script path resolved."""
+	path = Path(path).resolve()
+	plan = json.loads(path.read_text())
+	errors, seen = [], set()
+	if not plan.get('scenarios'):
+		errors.append('The plan has no scenarios.')
+	if not isinstance(plan.get('featureFlags', {}), dict):
+		errors.append('featureFlags must be an object of flag names and values.')
+	for s in plan.get('scenarios', []):
+		sid = s.get('id', '')
+		if not ID.match(sid) or '--' in sid:
+			errors.append(f'Scenario id "{sid}": use a-z, 0-9 and single "-" characters.')
+		if sid in seen:
+			errors.append(f'Scenario id "{sid}" is used two times.')
+		seen.add(sid)
+		if s.get('kind') not in KINDS:
+			errors.append(f'Scenario "{sid}": kind must be one of {", ".join(KINDS)}.')
+		if not s.get('script') or not script_path(s['script'], path.parent).is_file():
+			errors.append(f'Scenario "{sid}": script {s.get("script")} not found.')
+		if not s.get('title') or not s.get('expected'):
+			errors.append(f'Scenario "{sid}": give a title and an expected outcome.')
+	if errors:
+		sys.exit('The plan is not valid:\n  ' + '\n  '.join(errors))
+	for s in plan['scenarios']:
+		s['scriptPath'] = str(script_path(s['script'], path.parent))
+	plan.setdefault('source', str(path))
+	plan['name'] = path.parent.name if path.parent.name != 'plan' else slug(plan.get('title') or 'run')
+	return plan
+
+
+def run_dirs():
+	"""Earlier run folders, newest first."""
+	if not RUNS_ROOT.exists():
+		return []
+	found = [d for d in RUNS_ROOT.iterdir() if (d / 'plan' / 'scenarios.json').is_file()]
+	return sorted(found, key=lambda d: (d / 'plan' / 'scenarios.json').stat().st_mtime, reverse=True)
+
+
+def rerun_plan(arg):
+	"""The plan to run again. The source plan wins over the copy, so a fixed script is used."""
+	if arg == 'last':
+		found = run_dirs()
+		if not found:
+			sys.exit(f'No earlier run in {RUNS_ROOT}.')
+		run_dir = found[0]
+	else:
+		run_dir = Path(arg).expanduser().resolve()
+	copy = run_dir / 'plan' / 'scenarios.json'
+	if not copy.is_file():
+		sys.exit(f'{run_dir} has no plan/scenarios.json.')
+	source = json.loads(copy.read_text()).get('source')
+	if source and Path(source).is_file():
+		say(f'rerun of {run_dir.name}, with the current plan and scripts in {Path(source).parent}')
+		return source
+	say(f'rerun of {run_dir.name}, with the scripts copied into that run')
+	return copy
+
+
+def new_run_dir(root, name):
+	base = root / f'{datetime.now():%Y%m%d-%H%M%S}-{name}'
+	run_dir, n = base, 2
+	while run_dir.exists():
+		run_dir, n = base.with_name(f'{base.name}-{n}'), n + 1
+	run_dir.mkdir(parents=True)
+	return run_dir
+
+
+def copy_plan(plan, run_dir):
+	"""Keep the plan and its scripts in the run folder, so --rerun can repeat the run."""
+	plan_dir = run_dir / 'plan'
+	(plan_dir / 'scripts').mkdir(parents=True, exist_ok=True)
+	saved = json.loads(json.dumps(plan))
+	saved.pop('name', None)
+	for s in saved['scenarios']:
+		target = plan_dir / 'scripts' / f'{s["id"]}.mjs'
+		shutil.copy(s.pop('scriptPath'), target)
+		s['script'] = f'scripts/{target.name}'
+	(plan_dir / 'scenarios.json').write_text(json.dumps(saved, indent=2))
+
+
+class Progress:
+	"""Writes status.json, which the live page reads."""
+
+	def __init__(self, run_dir, data):
+		self.path, self.lock, self.data = run_dir / 'status.json', threading.Lock(), data
+		self.save()
+
+	def save(self):
+		tmp = self.path.with_suffix('.tmp')
+		tmp.write_text(json.dumps(self.data, indent=2))
+		os.replace(tmp, self.path)
+
+	def update(self, **fields):
+		with self.lock:
+			self.data.update(fields)
+			self.save()
+
+	def run(self, name, **fields):
+		with self.lock:
+			self.data['runs'][name].update(fields)
+			self.save()
+
+	def finish(self):
+		with self.lock:
+			for entry in self.data['runs'].values():
+				if entry['status'] in ('waiting', 'starting'):
+					entry.update(status='skipped', reason=entry.get('reason') or 'The run did not start.')
+				elif entry['status'] == 'running':
+					entry['status'] = 'error'
+			if self.data.get('phase') != 'done':
+				self.data.update(phase='done', step='Done', finishedAt=time.time())
+			self.save()
+
+
+def close_stream(res):
+	"""Close an event stream that another thread reads. close() alone waits for the next server
+	keep-alive (about 13 seconds), because the reader holds the buffer lock. A socket shutdown ends the
+	read at once."""
+	try:
+		with socket.socket(fileno=os.dup(res.fileno())) as sock:
+			sock.shutdown(socket.SHUT_RDWR)
+	except (OSError, ValueError):
+		pass
+	try:
+		res.close()
+	except Exception:
+		pass
+
+
+class ThreadEvents:
+	"""Reads the event stream of each thread that a scenario opens. The stream starts at the first event."""
+
+	def __init__(self, base_url, out, auto_confirm, prefix):
+		self.client = client(base_url).login()
+		self.out, self.auto_confirm, self.prefix = out, auto_confirm, prefix
+		self.file, self.lock = (out / 'events.jsonl').open('a'), threading.Lock()
+		self.run_ids, self.decisions, self.streams, self.closed = [], [], [], False
+
+	def follow(self, thread_id):
+		threading.Thread(target=self._read, args=(thread_id,), daemon=True).start()
+
+	def _read(self, thread_id):
+		try:
+			res = self.client.open_stream(f'/rest/instance-ai/events/{thread_id}')
+		except Exception as e:
+			say(f'event stream of {thread_id} did not open: {e}', self.prefix)
+			return
+		self.streams.append(res)
+		data, name = [], None
+		try:
+			for raw in res:
+				line = raw.decode(errors='replace').rstrip('\r\n')
+				if line.startswith('data:'):
+					data.append(line[5:].lstrip())
+				elif line.startswith('event:'):
+					name = line[6:].strip()
+				elif line == '' and data:
+					try:
+						self._event(json.loads('\n'.join(data)))
+					except json.JSONDecodeError:
+						pass
+					data, name = [], None
+		except Exception:
+			# close() ends the stream when the scenario stops.
+			pass
+
+	def _event(self, event):
+		with self.lock:
+			if self.closed:
+				return
+			self.file.write(json.dumps(event) + '\n')
+			self.file.flush()
+		payload = event.get('payload') or {}
+		if event.get('type') == 'run-start' and payload.get('langsmithRunId'):
+			self.run_ids.append(payload['langsmithRunId'])
+		if event.get('type') == 'confirmation-request' and self.auto_confirm:
+			body, label = decide(payload)
+			try:
+				self.client.request('POST', f'/rest/instance-ai/confirm/{payload["requestId"]}', body)
+				ok = True
+			except RuntimeError as e:
+				ok, label = False, f'{label} (failed: {str(e)[:200]})'
+			self.decisions.append({'at': now_iso(), 'toolName': payload.get('toolName'), 'decision': label, 'ok': ok})
+			say(f'confirm: {label} [{payload.get("toolName")}]', self.prefix)
+
+	def close(self):
+		with self.lock:
+			self.closed = True
+			self.file.close()
+		for res in self.streams:
+			close_stream(res)
+
+
+def run_job(name, scenario, url, instance, run_dir, progress, live):
+	sub = run_dir / 'runs' / name
+	sub.mkdir(parents=True, exist_ok=True)
+	progress.run(name, status='running', startedAt=time.time(), baseUrl=url)
+	auto_confirm = scenario.get('autoConfirm', True)
+	timeout_min = float(scenario.get('timeoutMin', 10))
+	started = now_iso()
+	try:
+		events = ThreadEvents(url, sub, auto_confirm, name)
+	except Exception as e:
+		progress.run(name, status='error', finishedAt=time.time(), reason=f'Cannot sign in to {url}: {e}')
+		return
+	cmd = ['node', str(HERE / 'run_scenario.mjs'), '--out', str(sub), '--script', scenario['scriptPath'],
+	       '--timeout-min', str(timeout_min), '--viewport', scenario.get('viewport', '1600x1000')]
+	cmd += ['--live'] if live else []
+	cmd += ['--auto-confirm'] if auto_confirm else []
+	proc = subprocess.Popen(cmd, cwd=HERE, env={**os.environ, 'N8N_BASE_URL': url},
+	                        stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
+	RUNNERS.add(proc)
+	# The scenario has its own time limit. This one covers a runner that hangs outside the scenario.
+	watchdog = threading.Timer(timeout_min * 60 + 180, proc.kill)
+	watchdog.start()
+	with (sub / 'runner.log').open('w') as log:
+		for line in proc.stdout:
+			log.write(line)
+			say(line.rstrip(), name)
+			match = re.match(r'thread (\S+)', line)
+			if match:
+				events.follow(match[1])
+	proc.wait()
+	RUNNERS.discard(proc)
+	watchdog.cancel()
+	# The last events of a run can arrive just after the scenario ends.
+	time.sleep(1)
+	events.close()
+	try:
+		result = json.loads((sub / 'result.json').read_text())
+	except (OSError, ValueError):
+		result = {'status': 'error', 'error': f'The runner stopped without a result (exit {proc.returncode}). See runner.log.'}
+		(sub / 'result.json').write_text(json.dumps(result, indent=2))
+	meta = {
+		'kind': 'scenario', 'scenario': scenario['id'], 'instance': instance, 'baseUrl': url,
+		'threadIds': result.get('threads', []), 'langsmithRunIds': events.run_ids, 'decisions': events.decisions,
+		'startedAt': started, 'finishedAt': now_iso(),
+		'screen': 'screen.webm' if (sub / 'screen.webm').exists() else None,
+	}
+	(sub / 'run.json').write_text(json.dumps(meta, indent=2))
+	progress.run(name, status=result['status'], finishedAt=time.time(), reason=result.get('error'))
+
+
+def stale_packages():
+	"""Packages with a source change in this branch that is newer than their build output."""
+	try:
+		base = subprocess.run(['git', 'merge-base', 'HEAD', 'origin/master'], cwd=REPO_ROOT, capture_output=True,
+		                      text=True, check=True).stdout.strip()
+		changed = subprocess.run(['git', 'diff', '--name-only', base], cwd=REPO_ROOT, capture_output=True,
+		                         text=True, check=True).stdout.split()
+		changed += [line[3:] for line in subprocess.run(['git', 'status', '--porcelain'], cwd=REPO_ROOT,
+		            capture_output=True, text=True, check=True).stdout.splitlines()]
+	except subprocess.CalledProcessError:
+		return []
+	newest_dist, stale = {}, set()
+	for rel in changed:
+		path = REPO_ROOT / rel
+		if not rel.startswith('packages/') or '/src/' not in rel or re.search(r'(\.test\.|__tests__|\.md$)', rel):
+			continue
+		if not path.is_file():
+			continue
+		pkg = REPO_ROOT / rel.split('/src/', 1)[0]
+		dist = pkg / 'dist'
+		if not dist.exists():
+			continue
+		if pkg not in newest_dist:
+			newest_dist[pkg] = max((f.stat().st_mtime for f in dist.rglob('*') if f.is_file()), default=0)
+		if path.stat().st_mtime > newest_dist[pkg]:
+			stale.add(str(pkg.relative_to(REPO_ROOT)))
+	return sorted(stale)
+
+
+def build_branch(log_path):
+	"""Build n8n and its packages with turbo. Unchanged packages come from the turbo cache in seconds."""
+	started = time.time()
+	with log_path.open('w') as log:
+		code = subprocess.run(['pnpm', 'turbo', 'run', 'build', '--filter=n8n...', '--output-logs=errors-only'],
+		                      cwd=REPO_ROOT, stdout=log, stderr=subprocess.STDOUT).returncode
+	text = log_path.read_text()
+	if code != 0:
+		sys.exit(f'The build failed. The end of {log_path}:\n' + '\n'.join(text.splitlines()[-30:]))
+	cached = re.search(r'Cached:\s+(\d+) cached, (\d+) total', text)
+	if cached and cached[1] == cached[2]:
+		say(f'build: up to date ({time.time() - started:.0f}s)')
+	else:
+		say(f'build: done in {time.time() - started:.0f}s')
+
+
+def preflight(build, need_master, local_instances=True):
+	"""Problems that stop a run (errors) or make the report less useful (warnings).
+	local_instances is False with --branch-url: the branch runs then need no local build."""
+	errors, warnings = [], []
+	for tool in ('node', 'git') + (('pnpm',) if build or need_master else ()):
+		if not shutil.which(tool):
+			errors.append(f'{tool} is not on the PATH.')
+	if shutil.which('node'):
+		probe = ("import(process.argv[1]).then(async (m) => { const b = await m.launchBrowser(); await b.close(); })"
+		         ".catch((e) => { console.error(e.message.split('\\n')[0]); process.exit(1); })")
+		res = subprocess.run(['node', '-e', probe, str(SHARED / 'n8n_browser.mjs')], capture_output=True, text=True,
+		                     timeout=60)
+		if res.returncode != 0:
+			errors.append('Playwright cannot start a browser: '
+			              f'{(res.stderr or res.stdout).strip()[:300]}. Run pnpm install, and install Google Chrome.')
+	if not build and local_instances:
+		for path in ('packages/cli/dist', 'packages/frontend/editor-ui/dist/index.html'):
+			if not (REPO_ROOT / path).exists():
+				errors.append(f'{path} does not exist. Build first, or run without --no-build.')
+	if not setting('N8N_INSTANCE_AI_MODEL'):
+		warnings.append('N8N_INSTANCE_AI_MODEL is not set in .env.local, .env.ai-surfaces or the environment. '
+		                'Scenarios that send a prompt can fail.')
+	if not setting('LANGSMITH_API_KEY'):
+		warnings.append('LANGSMITH_API_KEY is not set. The report has no tokens or cost.')
+	elif (setting('LANGSMITH_TRACING') or '').lower() != 'true':
+		warnings.append('LANGSMITH_TRACING is not true. The instances send no traces, so the report has no tokens or cost.')
+	if (setting('N8N_DIAGNOSTICS_ENABLED') or '').lower() == 'false':
+		warnings.append('N8N_DIAGNOSTICS_ENABLED is false. Instance AI sends no traces, so the report has no tokens or cost.')
+	RUNS_ROOT.mkdir(parents=True, exist_ok=True)
+	free = shutil.disk_usage(RUNS_ROOT).free / 1e9
+	if free < 2:
+		warnings.append(f'Only {free:.1f} GB free in {RUNS_ROOT}. A run needs about 0.5 GB.')
+	if need_master and not baseline.built():
+		CACHE.mkdir(parents=True, exist_ok=True)
+		free = shutil.disk_usage(CACHE).free / 1e9
+		warnings.append('The master build does not exist yet. The first build takes 15 to 20 minutes'
+		                + (f', and needs about 3 GB (free: {free:.1f} GB).' if free < 5 else '.'))
+	return errors, warnings
+
+
+def history():
+	"""Cost and time of earlier runs, by scenario id. Master runs and runs without a result are left out."""
+	by_id, every = {}, []
+	for run_dir in run_dirs():
+		try:
+			runs = json.loads((run_dir / 'status.json').read_text()).get('runs', {})
+		except (OSError, ValueError):
+			continue
+		for name, entry in runs.items():
+			if name.endswith('--master') or entry.get('status') not in ('passed', 'failed'):
+				continue
+			if not (entry.get('startedAt') and entry.get('finishedAt')):
+				continue
+			usage = run_dir / 'runs' / name / 'usage.json'
+			try:
+				if usage.exists():
+					cost = sum(c['cost'] for c in json.loads(usage.read_text()))
+				else:
+					meta = json.loads((run_dir / 'runs' / name / 'run.json').read_text())
+					cost = None if meta.get('langsmithRunIds') else 0.0
+			except (OSError, ValueError):
+				cost = None
+			sample = (cost, entry['finishedAt'] - entry['startedAt'])
+			by_id.setdefault(entry.get('scenario', name), []).append(sample)
+			every.append(sample)
+	return by_id, every
+
+
+def estimate(jobs, concurrency, compare_master, flags):
+	"""Expected cost and time of the jobs, from the earlier runs of the same scenario ids. A master run that
+	can be reused costs nothing."""
+	by_id, every = history()
+	try:
+		base = baseline.target() if compare_master else None
+	except subprocess.CalledProcessError:
+		base = None
+
+	def median(samples, index, default):
+		values = [x[index] for x in samples if x[index] is not None]
+		return statistics.median(values) if values else default
+
+	any_cost, any_time = median(every, 0, None), median(every, 1, 60)
+	rows, cost_total, known = [], 0.0, True
+	for name, s, role in jobs:
+		if role == 'master' and base and (MASTER_CACHE / master_key(s, base, flags) / 'result.json').is_file():
+			rows.append((name + ' (kept)', 0.0, 0, True))
+			continue
+		samples = by_id.get(s['id'], [])
+		cost = median(samples, 0, any_cost)
+		seconds = median(samples, 1, any_time)
+		if cost is None:
+			known = False
+		else:
+			cost_total += cost
+		rows.append((name, cost, seconds, bool(samples)))
+	times = sorted((r[2] for r in rows), reverse=True)
+	wall = max(times[0] if times else 0, sum(times) / max(1, concurrency)) + 15
+	return {'rows': rows, 'cost': cost_total if known else None, 'seconds': wall,
+	        'masterBuild': compare_master and not baseline.built()}
+
+
+def print_estimate(est):
+	say('Estimate (median of earlier runs; * means no earlier run of this scenario):')
+	for name, cost, seconds, seen in est['rows']:
+		price = f'${cost:.2f}' if cost is not None else '  ?  '
+		say(f'  {name:44} {price:>7} {seconds:5.0f}s{"" if seen else " *"}')
+	total = f'about ${est["cost"]:.2f}' if est['cost'] is not None else 'unknown cost'
+	extra = ', plus 15 to 20 minutes for the first master build' if est['masterBuild'] else ''
+	say(f'  {len(est["rows"])} runs, {total}, about {est["seconds"] / 60:.1f} minutes{extra}')
+
+
+def fetch_usage(run_dir, names, timeout=90):
+	"""Token usage and cost of each run from LangSmith, in one query for all runs. LangSmith receives traces
+	with a delay, so the query repeats until each run has calls and the count stops changing."""
+	if not setting('LANGSMITH_API_KEY'):
+		return
+	metas = {}
+	for name in names:
+		file = run_dir / 'runs' / name / 'run.json'
+		meta = json.loads(file.read_text()) if file.exists() else {}
+		if meta.get('langsmithRunIds') and not meta.get('cached'):
+			metas[name] = meta
+	if not metas:
+		return
+	start = min(m['startedAt'] for m in metas.values())
+	end = max(m['finishedAt'] for m in metas.values())
+	traces = {t for m in metas.values() for t in m['langsmithRunIds']}
+	deadline, previous, calls = time.time() + timeout, None, []
+	while True:
+		try:
+			calls = fetch_calls(start, end, traces)
+		except (SystemExit, OSError, ValueError) as e:
+			say(f'usage failed: {str(e)[:200]}')
+			return
+		seen = {c['trace'] for c in calls}
+		complete = all(set(m['langsmithRunIds']) & seen for m in metas.values())
+		if (complete and len(calls) == previous) or time.time() > deadline:
+			break
+		previous = len(calls)
+		time.sleep(10)
+	for name, meta in metas.items():
+		own = [c for c in calls if c['trace'] in set(meta['langsmithRunIds'])]
+		(run_dir / 'runs' / name / 'usage.json').write_text(json.dumps(own, indent=1))
+		usage = totals(own)
+		say(f'{usage["calls"]} LLM calls, ${usage["cost"]:.2f}', name)
+
+
+def compare_trees(run_dir):
+	"""Add the accessibility tree diff of each capture to visual.json."""
+	path = run_dir / 'visual.json'
+	if not path.exists():
+		return []
+	results = json.loads(path.read_text())
+	for r in results:
+		trees = []
+		for name in (f'{r["scenario"]}--master', r['scenario']):
+			file = run_dir / 'runs' / name / 'captures' / f'{r["name"]}.aria.yml'
+			trees.append(file.read_text().splitlines() if file.exists() else [])
+		diff = '\n'.join(difflib.unified_diff(*trees, 'master', 'branch', lineterm='', n=2))
+		r['ariaDiff'] = diff
+		r['pixelStatus'] = r['status']
+		if diff and r['status'] == 'unchanged':
+			r['status'] = 'changed'
+	path.write_text(json.dumps(results, indent=2))
+	return results
+
+
+def master_key(scenario, commit, flags):
+	"""A master run gives the same screens while the commit, the flags, the script and the runner stay the same."""
+	h = hashlib.sha256()
+	overrides = setting('N8N_FEATURE_FLAG_OVERRIDES') or ''
+	for part in (commit, json.dumps(flags or {}, sort_keys=True), overrides, scenario.get('viewport', '1600x1000'),
+	             str(scenario.get('autoConfirm', True))):
+		h.update(part.encode() + b'\0')
+	for file in (Path(scenario['scriptPath']), HERE / 'kit.mjs', HERE / 'run_scenario.mjs'):
+		h.update(file.read_bytes())
+	return h.hexdigest()[:20]
+
+
+def save_master_run(sub, key):
+	if json.loads((sub / 'result.json').read_text()).get('status') != 'passed':
+		return
+	target = MASTER_CACHE / key
+	shutil.rmtree(target, ignore_errors=True)
+	shutil.copytree(sub, target, ignore=shutil.ignore_patterns('live.png', 'usage.json'))
+	cutoff = time.time() - MASTER_CACHE_DAYS * 86400
+	for old in MASTER_CACHE.iterdir():
+		if old.is_dir() and old.stat().st_mtime < cutoff:
+			shutil.rmtree(old, ignore_errors=True)
+
+
+def reuse_master_run(sub, key):
+	"""Copy a kept master run into this run. Returns False when there is none."""
+	source = MASTER_CACHE / key
+	if not (source / 'result.json').is_file():
+		return False
+	shutil.copytree(source, sub, dirs_exist_ok=True)
+	meta = json.loads((sub / 'run.json').read_text())
+	meta['cached'] = True
+	(sub / 'run.json').write_text(json.dumps(meta, indent=2))
+	return True
+
+
+def start_live_server(run_dir):
+	url_file = run_dir / 'live-url.txt'
+	url_file.unlink(missing_ok=True)
+	log = (run_dir / 'live-server.log').open('w')
+	# A new session, so the page still serves the report after this script ends.
+	subprocess.Popen([sys.executable, str(HERE / 'live_page.py'), '--run-dir', str(run_dir),
+	                  '--parent-pid', str(os.getpid())],
+	                 cwd=HERE, stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
+	for _ in range(50):
+		if url_file.exists() and url_file.read_text().strip():
+			return url_file.read_text().strip()
+		time.sleep(0.1)
+	return None
+
+
+def open_in_default_browser(url):
+	"""Opens url in the default browser. Returns False if no browser opened.
+
+	$BROWSER is not used: agents often set it to a command that does nothing.
+	"""
+	command = {'darwin': ['open'], 'linux': ['xdg-open']}.get(sys.platform)
+	if command and shutil.which(command[0]):
+		try:
+			return subprocess.run(command + [url], capture_output=True, timeout=15).returncode == 0
+		except (OSError, subprocess.TimeoutExpired):
+			return False
+	try:
+		return webbrowser.open(url)
+	except webbrowser.Error:
+		return False
+
+
+def result_table(run_dir, runs):
+	"""A plain text table of the results, with the first problem of each run that did not pass."""
+	lines = [f'  {"run":44} {"status":15} {"steps":>7} {"time":>6} {"errors":>6}']
+	for name, entry in runs.items():
+		sub = run_dir / 'runs' / name
+		steps = report.read_json(sub / 'steps.json', [])
+		result = report.read_json(sub / 'result.json', {})
+		checked = [s for s in steps if s['kind'] in ('step', 'check', 'capture')]
+		passed = f'{sum(s["status"] == "passed" for s in checked)}/{len(checked)}' if checked else '-'
+		took = (entry.get('finishedAt') or 0) - (entry.get('startedAt') or 0)
+		errors = (result.get('consoleErrors') or 0) + (result.get('requestErrors') or 0)
+		status = entry['status'] + (' (kept)' if report.read_json(sub / 'run.json', {}).get('cached') else '')
+		lines.append(f'  {name:44} {status:15} {passed:>7} {took:5.0f}s {errors:>6}')
+		failed = next((s for s in steps if s['status'] == 'failed'), None)
+		if failed:
+			problem = f'{failed["kind"]} "{failed["title"]}": {failed.get("error") or ""}'
+		else:
+			problem = entry.get('reason') if entry['status'] != 'passed' else ''
+		if (problem or '').strip():
+			lines.append(f'      {problem.strip().splitlines()[0][:160]}')
+	return '\n'.join(lines)
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--plan', help='scenarios.json')
+	ap.add_argument('--rerun', help='"last", or an earlier run folder. Uses the current plan and scripts when they exist.')
+	ap.add_argument('--only', help='Comma-separated scenario ids.')
+	ap.add_argument('--run-dir')
+	ap.add_argument('--concurrency', type=int, default=int(setting('FEATURE_TEST_CONCURRENCY', default='4')))
+	ap.add_argument('--branch-url', help='Use this running instance for the branch runs instead of fresh ones.')
+	ap.add_argument('--try', dest='try_mode', action='store_true',
+	                help='A quick check of new scripts: no master, no live page, no usage, no report.')
+	ap.add_argument('--estimate', action='store_true', help='Print the expected cost and time, then stop.')
+	ap.add_argument('--check', action='store_true', help='Check the setup, then stop.')
+	ap.add_argument('--no-build', action='store_true', help='Do not build this checkout first.')
+	ap.add_argument('--no-baseline', action='store_true', help='Skip master. Visual scenarios run on the branch only.')
+	ap.add_argument('--no-cache', action='store_true', help='Run master again, also when a kept master run exists.')
+	ap.add_argument('--no-fetch', action='store_true', help='Use origin/master as it is, without git fetch.')
+	ap.add_argument('--no-live', action='store_true', help='Do not start the live page.')
+	ap.add_argument('--no-open', action='store_true',
+	                help='Do not open the live page in the default browser. Use this when you open it in the editor browser.')
+	ap.add_argument('--keep-instances', action='store_true', help='Keep the fresh instances running after the run.')
+	args = ap.parse_args()
+	if args.try_mode:
+		args.no_baseline = args.no_live = True
+	build = not args.no_build and not args.branch_url
+
+	if args.check and not (args.plan or args.rerun):
+		errors, warnings = preflight(build, need_master=True)
+		for w in warnings:
+			say(f'warning: {w}')
+		for e in errors:
+			say(f'error: {e}')
+		branch = git('rev-parse', '--abbrev-ref', 'HEAD').replace('/', '-')
+		say(f'plan folder for this branch: {PLANS_ROOT / branch}/<feature>/')
+		say('The setup is ready.' if not errors else 'Fix the errors, then run again.')
+		sys.exit(1 if errors else 0)
+	if not (args.plan or args.rerun):
+		sys.exit('Give --plan <scenarios.json>, --rerun last|<run-dir>, or --check.')
+	# The fresh instances run in their own sessions. Exit through the cleanup, so they stop too.
+	for sig in (signal.SIGTERM, signal.SIGHUP):
+		signal.signal(sig, lambda signum, _frame: sys.exit(128 + signum))
+
+	plan = load_plan(rerun_plan(args.rerun) if args.rerun else args.plan)
+	if args.only:
+		wanted = {x.strip() for x in args.only.split(',')}
+		plan['scenarios'] = [s for s in plan['scenarios'] if s['id'] in wanted]
+		if not plan['scenarios']:
+			sys.exit(f'No scenario matches --only {args.only}.')
+
+	visual = [s for s in plan['scenarios'] if s['kind'] == 'visual']
+	compare_master = bool(visual) and not args.no_baseline
+	jobs = []  # (name, scenario, instance role)
+	for s in plan['scenarios']:
+		if args.branch_url:
+			role = 'branch'
+		elif s['kind'] == 'visual':
+			role = 'branch-visual'
+		else:
+			role = 'isolated' if s.get('isolated', True) else 'branch'
+		jobs.append((s['id'], s, role))
+		if s['kind'] == 'visual' and compare_master:
+			jobs.append((f'{s["id"]}--master', s, 'master'))
+
+	est = estimate(jobs, args.concurrency, compare_master, plan.get('featureFlags'))
+	print_estimate(est)
+	if args.estimate:
+		return
+	errors, warnings = preflight(build, compare_master, local_instances=not args.branch_url)
+	for w in warnings:
+		say(f'warning: {w}')
+	if errors:
+		sys.exit('The setup is not ready:\n  ' + '\n  '.join(errors))
+
+	if args.run_dir:
+		run_dir = Path(args.run_dir).expanduser().resolve()
+		run_dir.mkdir(parents=True, exist_ok=True)
+	else:
+		run_dir = new_run_dir(RUNS_ROOT / 'try' if args.try_mode else RUNS_ROOT, plan['name'])
+	copy_plan(plan, run_dir)
+	say(f'run directory: {run_dir}')
+
+	branch_commit = subprocess.run(['git', 'rev-parse', '--short', 'HEAD'], cwd=REPO_ROOT, capture_output=True,
+	                               text=True).stdout.strip()
+	dirty = bool(subprocess.run(['git', 'status', '--porcelain', '--', 'packages'], cwd=REPO_ROOT,
+	                            capture_output=True, text=True).stdout.strip())
+	live = live_enabled() and not args.no_live
+	progress = Progress(run_dir, {
+		'phase': 'running', 'step': 'Building the branch' if build else 'Starting the instances',
+		'startedAt': time.time(), 'concurrency': args.concurrency,
+		'branch': {'commit': branch_commit, 'dirty': dirty, 'url': args.branch_url},
+		'master': {'commit': None} if compare_master else None,
+		'runs': {name: {'status': 'waiting', 'scenario': s['id'], 'instance': role.split('-')[0]}
+		         for name, s, role in jobs},
+	})
+
+	live_url = None
+	if live:
+		live_url = start_live_server(run_dir)
+		if live_url:
+			say(f'live page: {live_url}/report/live')
+			if not args.no_open and not open_in_default_browser(f'{live_url}/report/live'):
+				say(f'no browser opened. Open {live_url}/report/live to watch the runs.')
+		else:
+			say(f'the live page did not start, see {run_dir / "live-server.log"}')
+
+	instances, lock = [], threading.Lock()
+	keep = args.keep_instances
+	pool = ThreadPoolExecutor(max_workers=max(1, args.concurrency))
+	# Shared instances start at once. Isolated instances start ahead of their scenario, with at most one
+	# more alive than the number of scenarios that run at the same time.
+	shared_pool = ThreadPoolExecutor(max_workers=3)
+	boot_pool = ThreadPoolExecutor(max_workers=args.concurrency + 1)
+	alive = threading.BoundedSemaphore(args.concurrency + 1)
+	futures, shared = [], {}
+
+	def start_instance(name, root):
+		inst = Instance(name, root, run_dir / 'instances' / name, run_dir / 'instances' / f'{name}.log',
+		                plan.get('featureFlags'))
+		with lock:
+			instances.append(inst)
+		started = time.time()
+		inst.start()
+		# The port only: Cursor opens an editor tab for each new localhost URL that a command prints.
+		say(f'{name} instance ready in {time.time() - started:.0f}s on port {inst.port}')
+		return inst
+
+	def boot_isolated(name):
+		alive.acquire()
+		try:
+			return start_instance(name, REPO_ROOT)
+		except Exception:
+			alive.release()
+			raise
+
+	def run_on(name, scenario, role, instance):
+		"""Run one job when its instance is ready. instance is a future of an Instance, or a URL."""
+		progress.run(name, status='starting')
+		try:
+			inst = instance.result() if hasattr(instance, 'result') else None
+		except Exception as e:
+			progress.run(name, status='error', finishedAt=time.time(), reason=f'The instance did not start: {e}')
+			return
+		url = inst.url if inst else instance
+		try:
+			run_job(name, scenario, url, role.split('-')[0], run_dir, progress, live)
+			if role == 'master' and not args.no_cache:
+				save_master_run(run_dir / 'runs' / name,
+				                master_key(scenario, progress.data['master']['commit'], plan.get('featureFlags')))
+		finally:
+			if role == 'isolated':
+				if not keep:
+					inst.stop(clean=True)
+				alive.release()
+
+	def submit(name, scenario, role, instance):
+		futures.append(pool.submit(run_on, name, scenario, role, instance))
+
+	def skip(role, reason):
+		for name, _, r in jobs:
+			if r == role and progress.data['runs'][name]['status'] == 'waiting':
+				progress.run(name, status='skipped', reason=reason)
+
+	master_jobs = [(name, s) for name, s, r in jobs if r == 'master']
+	base_commit = None
+	if compare_master:
+		try:
+			base_commit = baseline.target(fetch=not args.no_fetch)
+		except subprocess.CalledProcessError as e:
+			say(f'cannot find the master merge base: {e}', 'master')
+		if base_commit:
+			progress.update(master={'commit': base_commit})
+			if not args.no_cache:
+				left = []
+				for name, s in master_jobs:
+					if reuse_master_run(run_dir / 'runs' / name, master_key(s, base_commit, plan.get('featureFlags'))):
+						now = time.time()
+						progress.run(name, status='passed', startedAt=now, finishedAt=now)
+						say(f'master run kept from an earlier run of {base_commit[:11]}', name)
+					else:
+						left.append((name, s))
+				master_jobs = left
+
+	def master_side():
+		try:
+			built = baseline.prepare(fetch=False, say=lambda m: say(m, 'master'))
+			progress.update(master={'commit': built['commit']})
+			inst = start_instance('master', built['root'])
+			for name, s in master_jobs:
+				submit(name, s, 'master', inst.url)
+		except Exception as e:
+			say(f'master is not available: {e}', 'master')
+			skip('master', f'Master is not available: {e}')
+
+	visual_error = None
+	try:
+		if build:
+			step('Build this checkout')
+			build_branch(run_dir / 'build.log')
+		elif not args.branch_url:
+			stale = stale_packages()
+			if stale:
+				say('warning: these packages have source changes that are newer than their build: '
+				    f'{", ".join(stale)}. The fresh instances run the old build.')
+				progress.update(stale=stale)
+
+		step(f'Run {len(jobs)} scenario runs, {args.concurrency} at a time')
+		progress.update(step='Running the scenarios')
+		for role in ('branch', 'branch-visual'):
+			if any(r == role for _, _, r in jobs):
+				shared[role] = args.branch_url or shared_pool.submit(start_instance, role, REPO_ROOT)
+		boots = {name: boot_pool.submit(boot_isolated, name) for name, _, r in jobs if r == 'isolated'}
+		master_thread = None
+		if compare_master and master_jobs and base_commit:
+			master_thread = threading.Thread(target=master_side, daemon=True)
+			master_thread.start()
+		elif compare_master and not base_commit:
+			skip('master', 'The master merge base is not known.')
+		for name, s, role in jobs:
+			if role == 'isolated':
+				submit(name, s, role, boots[name])
+			elif role in shared:
+				submit(name, s, role, shared[role])
+		if master_thread:
+			master_thread.join()
+		# Jobs can be added while the pool runs, so wait until the list stops growing.
+		while any(not f.done() for f in list(futures)):
+			time.sleep(1)
+		for f in futures:
+			if f.exception():
+				say(f'a run failed: {f.exception()}')
+		pool.shutdown()
+		if not keep:
+			for inst in instances:
+				inst.stop(clean=True)
+
+		names = [name for name, _, _ in jobs]
+		if compare_master:
+			step('Compare the visual captures with master')
+			progress.update(step='Comparing the captures with master')
+			diff = subprocess.run(['node', 'visual_diff.mjs', str(run_dir)], cwd=HERE)
+			if diff.returncode != 0:
+				visual_error = f'visual_diff.mjs failed with exit code {diff.returncode}. The visual check is not complete.'
+				say(f'error: {visual_error}')
+				progress.update(visualError=visual_error)
+			trees = [r for r in compare_trees(run_dir) if r.get('ariaDiff')]
+			if trees:
+				say(f'{len(trees)} captures with a difference in the accessibility tree: '
+				    + ', '.join(f'{r["scenario"]}/{r["name"]}' for r in trees))
+
+		step('Summarize the runs for the judge')
+		summary = report.summarize(run_dir)
+		if visual_error:
+			summary = f'VISUAL CHECK FAILED: {visual_error}\n\n{summary}'
+		(run_dir / 'summary.txt').write_text(summary)
+		if not args.try_mode:
+			step('Build the report')
+			progress.update(step='Building the report')
+			report.build(run_dir)
+		progress.finish()
+	finally:
+		pool.shutdown(wait=False, cancel_futures=True)
+		boot_pool.shutdown(wait=False, cancel_futures=True)
+		shared_pool.shutdown(wait=False, cancel_futures=True)
+		for proc in list(RUNNERS):
+			proc.kill()
+		if not keep:
+			for inst in instances:
+				inst.stop(clean=True)
+		progress.finish()
+
+	runs = progress.data['runs']
+	visual_results = json.loads((run_dir / 'visual.json').read_text()) if (run_dir / 'visual.json').exists() else []
+	not_passed = [n for n, e in runs.items() if e['status'] != 'passed' and not n.endswith('--master')]
+	rerun = f'python3 {S}/run.py --rerun {run_dir}' + (f' --only {",".join(not_passed)}' if not_passed else '')
+	usage_later = not args.try_mode and bool(setting('LANGSMITH_API_KEY'))
+	say('\n=== Result ===')
+	say(result_table(run_dir, runs))
+	if not_passed:
+		say(f'Run the ones that did not pass again: {rerun}')
+	print(json.dumps({
+		'runDir': str(run_dir),
+		'report': None if args.try_mode else str(run_dir / 'report' / 'index.html'),
+		'live': f'{live_url}/report/index.html' if live_url else None,
+		'summary': str(run_dir / 'summary.txt'),
+		'results': {name: runs[name]['status'] for name in runs},
+		'visual': {'captures': len(visual_results),
+		           'changed': [f'{r["scenario"]}/{r["name"]}' for r in visual_results if r['status'] != 'unchanged'],
+		           'error': visual_error},
+		'instancePorts': {i.name: i.port for i in instances} if keep else None,
+		'rerun': rerun,
+		'usage': 'The token usage comes in the next lines. The report updates when it is done.' if usage_later else None,
+		'next': ('Fix the scripts that failed, then run --try again, or run the full plan.' if args.try_mode else
+		         'Judge each run from summary.txt and write verdicts.json. The live page rebuilds the report.'),
+	}, indent=2), flush=True)
+
+	if usage_later:
+		step('Fetch the token usage from LangSmith')
+		fetch_usage(run_dir, [n for n in runs])
+		report.build(run_dir)
+		say('usage: done, the report is updated')
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-assistant-ui/scripts/run_scenario.mjs

@@ -0,0 +1,203 @@
+#!/usr/bin/env node
+// Run one scenario script in a browser that is signed in to an n8n instance.
+//
+//   node run_scenario.mjs --out <dir> --script <file.mjs> [--live] [--auto-confirm]
+//                         [--timeout-min 10] [--viewport 1600x1000]
+//
+// N8N_BASE_URL selects the instance. Writes to <dir>:
+//   steps.json            the steps, checks and captures with their screenshots (updated while it runs)
+//   step-NN.png ...       one screenshot for each step, check and capture
+//   captures/<name>.png   screens for the visual check, with <name>.aria.yml
+//   live.png              the screen every 2.5 seconds, for the live page (with --live)
+//   screen.webm           the video, with screen-poster.png
+//   console.json          browser errors and failed requests to the instance
+//   result.json           {status: passed|failed|error, error, threads, prompts, startedAt, finishedAt, ...}
+// Prints "thread <id>" for each thread that the scenario opens, so run.py can read its events.
+import { randomUUID } from 'node:crypto';
+import { copyFileSync, mkdirSync, mkdtempSync, renameSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join, resolve } from 'node:path';
+import { pathToFileURL } from 'node:url';
+import { parseArgs } from 'node:util';
+
+import {
+	baseUrl,
+	launchBrowser,
+	selectors,
+	setting,
+	signIn,
+} from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
+import { StepFailed, createKit } from './kit.mjs';
+
+const { values: args } = parseArgs({
+	options: {
+		out: { type: 'string' },
+		script: { type: 'string' },
+		live: { type: 'boolean', default: false },
+		'auto-confirm': { type: 'boolean', default: false },
+		'timeout-min': { type: 'string', default: '10' },
+		viewport: { type: 'string', default: '1600x1000' },
+	},
+});
+const out = resolve(args.out);
+mkdirSync(out, { recursive: true });
+const [width, height] = args.viewport.split('x').map(Number);
+const size = { width, height };
+const LIVE_SHOT_MS = 2500;
+const emit = (line) => console.log(line.replace(/\s+/g, ' ').slice(0, 300));
+
+/**
+ * Sign in through the REST API, which is about 1.3 seconds faster than the sign-in page. The auth cookie
+ * is bound to the browser-id header of the login, so the page gets the same id in localStorage.
+ * Falls back to the sign-in page.
+ */
+async function signedInState(browser) {
+	const login = await browser.newContext();
+	try {
+		const browserId = randomUUID();
+		const res = await login.request.post(`${baseUrl}/rest/login`, {
+			headers: { 'browser-id': browserId },
+			data: {
+				emailOrLdapLoginId: setting('N8N_EVAL_EMAIL', 'nathan@n8n.io'),
+				password: setting('N8N_EVAL_PASSWORD', 'PlaywrightTest123'),
+			},
+		});
+		if (res.ok()) {
+			const state = await login.storageState();
+			state.origins = [
+				{
+					origin: new URL(baseUrl).origin,
+					localStorage: [{ name: 'n8n-browserId', value: browserId }],
+				},
+			];
+			return state;
+		}
+		await signIn(await login.newPage());
+		return await login.storageState();
+	} finally {
+		await login.close();
+	}
+}
+
+selectors.setTestIdAttribute('data-test-id');
+const browser = await launchBrowser();
+const storageState = await signedInState(browser);
+
+// The video starts after the sign-in, at the first page of the scenario.
+const videoDir = mkdtempSync(join(tmpdir(), 'feature-test-video-'));
+const context = await browser.newContext({
+	viewport: size,
+	storageState,
+	recordVideo: { dir: videoDir, size },
+});
+const page = await context.newPage();
+
+const origin = new URL(baseUrl).origin;
+const consoleErrors = [];
+const requestErrors = [];
+page.on('console', (m) => {
+	if (m.type() === 'error')
+		consoleErrors.push({ text: m.text().slice(0, 1000), url: m.location()?.url });
+});
+page.on('pageerror', (e) => consoleErrors.push({ text: `Uncaught: ${e.message}`.slice(0, 1000) }));
+page.on('response', (r) => {
+	if (r.status() >= 400 && r.url().startsWith(origin)) {
+		requestErrors.push({
+			status: r.status(),
+			method: r.request().method(),
+			url: r.url().slice(origin.length),
+		});
+	}
+});
+page.on('requestfailed', (r) => {
+	const failure = r.failure()?.errorText ?? '';
+	// Closed event streams and aborted navigations are normal.
+	if (r.url().startsWith(origin) && !/ERR_ABORTED|NS_BINDING_ABORTED/.test(failure)) {
+		requestErrors.push({
+			status: 0,
+			method: r.method(),
+			url: r.url().slice(origin.length),
+			error: failure,
+		});
+	}
+});
+
+let shooting = false;
+async function liveShot() {
+	if (shooting) return;
+	shooting = true;
+	const partial = join(out, 'live.tmp.png');
+	try {
+		await page.screenshot({ path: partial });
+		renameSync(partial, join(out, 'live.png'));
+	} catch {
+		// The page can be in the middle of a navigation. The next interval tries again.
+	} finally {
+		shooting = false;
+	}
+}
+const liveTimer = args.live ? setInterval(liveShot, LIVE_SHOT_MS) : null;
+
+const { t, threads, prompts, steps, stop } = createKit({
+	page,
+	out,
+	baseUrl,
+	autoConfirm: args['auto-confirm'],
+	emit,
+});
+const result = { status: 'passed', startedAt: new Date().toISOString(), baseUrl };
+const timeoutMs = Number(args['timeout-min']) * 60_000;
+let timer;
+try {
+	const scenario = (await import(pathToFileURL(resolve(args.script)).href)).default;
+	if (typeof scenario !== 'function')
+		throw new Error('The script must export a default async function (t).');
+	await Promise.race([
+		scenario(t),
+		new Promise((_, reject) => {
+			timer = setTimeout(() => {
+				// The race does not end the scenario function. stop() makes its next step throw.
+				stop();
+				reject(new Error(`The scenario did not finish in ${args['timeout-min']} minutes.`));
+			}, timeoutMs);
+		}),
+	]);
+	if (steps.some((s) => s.status === 'failed')) result.status = 'failed';
+} catch (error) {
+	result.status = error instanceof StepFailed || steps.length ? 'failed' : 'error';
+	result.error = String(error?.message ?? error)
+		.replace(/\u001b\[[0-9;]*m/g, '')
+		.slice(0, 2000);
+	if (!(error instanceof StepFailed)) emit(`error: ${result.error.split('\n')[0]}`);
+} finally {
+	clearTimeout(timer);
+}
+
+stop();
+clearInterval(liveTimer);
+for (const s of steps.filter((x) => x.status === 'running')) {
+	s.status = 'failed';
+	s.error = 'The scenario stopped during this step.';
+}
+writeFileSync(join(out, 'steps.json'), JSON.stringify(steps, null, 1));
+await page.waitForTimeout(1500);
+if (args.live) await liveShot();
+await page.screenshot({ path: join(out, 'screen-poster.png') }).catch(() => {});
+const video = page.video();
+await context.close();
+if (video) copyFileSync(await video.path(), join(out, 'screen.webm'));
+await browser.close();
+rmSync(videoDir, { recursive: true, force: true });
+
+Object.assign(result, {
+	finishedAt: new Date().toISOString(),
+	threads,
+	prompts,
+	consoleErrors: consoleErrors.length,
+	requestErrors: requestErrors.length,
+});
+writeFileSync(join(out, 'console.json'), JSON.stringify({ consoleErrors, requestErrors }, null, 1));
+writeFileSync(join(out, 'result.json'), JSON.stringify(result, null, 2));
+emit(`result ${result.status}`);
+// A scenario that timed out can still wait on its own timers. The results are written, so end here.
+process.exit(0);

.agents/skills/testmax-assistant-ui/scripts/visual_diff.mjs

@@ -0,0 +1,96 @@
+#!/usr/bin/env node
+// Compare the captures of each visual scenario: this branch against master.
+//
+//   node visual_diff.mjs <run-dir>
+//
+// Reads runs/<id>/captures/*.png (branch) and runs/<id>--master/captures/*.png (master) for each
+// visual scenario in plan/scenarios.json. Writes visual/<id>/<name>.diff.png and visual.json:
+//   [{scenario, name, status: unchanged|changed|branch-only|master-only, changedPixels, ratio,
+//     branchSize, masterSize, diff}]
+// A pixel counts as changed when one color channel differs by more than PIXEL_THRESHOLD. A capture
+// counts as changed when more than RATIO_THRESHOLD of its pixels changed, or its size changed.
+import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
+import { join, resolve } from 'node:path';
+
+import { loadPng } from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
+
+const PIXEL_THRESHOLD = 32;
+const RATIO_THRESHOLD = 0.0005;
+const PNG = loadPng();
+const runDir = resolve(process.argv[2] ?? '.');
+const plan = JSON.parse(readFileSync(join(runDir, 'plan', 'scenarios.json'), 'utf8'));
+
+const read = (file) => PNG.sync.read(readFileSync(file));
+const names = (dir) =>
+	existsSync(dir)
+		? readdirSync(dir)
+				.filter((f) => f.endsWith('.png'))
+				.map((f) => f.slice(0, -4))
+		: [];
+
+function compare(branch, master) {
+	const width = Math.max(branch.width, master.width);
+	const height = Math.max(branch.height, master.height);
+	const diff = new PNG({ width, height });
+	let changed = 0;
+	for (let y = 0; y < height; y++) {
+		for (let x = 0; x < width; x++) {
+			const o = (y * width + x) * 4;
+			const inB = x < branch.width && y < branch.height;
+			const inM = x < master.width && y < master.height;
+			const b = inB ? (y * branch.width + x) * 4 : -1;
+			const m = inM ? (y * master.width + x) * 4 : -1;
+			let differs = !inB || !inM;
+			if (!differs) {
+				for (let c = 0; c < 4 && !differs; c++)
+					differs = Math.abs(branch.data[b + c] - master.data[m + c]) > PIXEL_THRESHOLD;
+			}
+			if (differs) {
+				changed++;
+				diff.data.set([224, 17, 95, 255], o);
+			} else {
+				// The unchanged page shows as a faint gray copy of the branch screen.
+				const gray =
+					0.299 * branch.data[b] + 0.587 * branch.data[b + 1] + 0.114 * branch.data[b + 2];
+				const v = Math.round(255 - (255 - gray) * 0.25);
+				diff.data.set([v, v, v, 255], o);
+			}
+		}
+	}
+	return { diff, changed, ratio: changed / (width * height) };
+}
+
+const results = [];
+for (const s of plan.scenarios.filter((x) => x.kind === 'visual')) {
+	const branchDir = join(runDir, 'runs', s.id, 'captures');
+	const masterDir = join(runDir, 'runs', `${s.id}--master`, 'captures');
+	const all = [...new Set([...names(branchDir), ...names(masterDir)])].sort();
+	for (const name of all) {
+		const entry = { scenario: s.id, name };
+		const hasB = existsSync(join(branchDir, `${name}.png`));
+		const hasM = existsSync(join(masterDir, `${name}.png`));
+		if (!hasB || !hasM) {
+			results.push({ ...entry, status: hasB ? 'branch-only' : 'master-only' });
+			continue;
+		}
+		const branch = read(join(branchDir, `${name}.png`));
+		const master = read(join(masterDir, `${name}.png`));
+		const { diff, changed, ratio } = compare(branch, master);
+		const out = join(runDir, 'visual', s.id);
+		mkdirSync(out, { recursive: true });
+		writeFileSync(join(out, `${name}.diff.png`), PNG.sync.write(diff));
+		const resized = branch.width !== master.width || branch.height !== master.height;
+		results.push({
+			...entry,
+			status: resized || ratio > RATIO_THRESHOLD ? 'changed' : 'unchanged',
+			changedPixels: changed,
+			ratio,
+			branchSize: [branch.width, branch.height],
+			masterSize: [master.width, master.height],
+			diff: `visual/${s.id}/${name}.diff.png`,
+		});
+	}
+}
+writeFileSync(join(runDir, 'visual.json'), JSON.stringify(results, null, 2));
+const changed = results.filter((r) => r.status !== 'unchanged').length;
+console.log(`${results.length} captures compared, ${changed} with a difference`);

.agents/skills/testmax-n8n-ai/SKILL.md

@@ -0,0 +1,745 @@
+---
+name: n8n:testmax-n8n-ai
+description: >-
+  Runs one workflow prompt through Instance AI, Claude Code with the n8n MCP
+  server, and Codex with the n8n MCP server on a local n8n instance. Collects
+  the traces, per-call tokens, cost, and canvas screenshots, judges each run
+  against the expected outcome, and builds one HTML comparison page. Claude
+  Code and Codex run in a sandbox that sees only the n8n MCP server. Use when
+  the user wants to compare builders, test a prompt across Instance AI and
+  MCP, or says /testmax-n8n-ai.
+---
+
+# Compare workflow builders
+
+This skill sends the same prompt to up to three builders on the local n8n
+instance. The builders run at the same time, and a local page shows them
+live. It then builds one page that shows, for each run:
+
+- the result against the expected outcome (pass, partial or fail, with reasons
+  and the evidence that a panel of blind judges cited)
+- deterministic checks of each workflow: the node definitions, the node
+  issues in the n8n editor, the graph, the expression references, and a run
+  of each branch with pinned data
+- a separate n8n standards score, from the rules in
+  [n8n-io/skills](https://github.com/n8n-io/skills) and the workflow-sdk best
+  practices. This score does not change the verdict
+- the canvas of the workflow that the builder made
+- a replay of the run: the exact terminal screen, or a video of the n8n UI
+- the cost
+- the LLM steps: one entry for each LLM call, with its tokens (cache read,
+  cache write, uncached input, output) and the agent text, reasoning and tool
+  calls (with input and output) that the call made
+
+The page uses the light mode of the
+[n8n brand guidelines](https://guidelines.n8n.design/).
+
+![An example comparison page](assets/comparison-page.png)
+
+All scripts are in `scripts/` next to this file. They use only the Python
+standard library and the Playwright copy in the repository.
+
+```bash
+S="$(git rev-parse --show-toplevel)/.agents/skills/testmax-n8n-ai/scripts"
+```
+
+## Settings
+
+All settings are in `.env.ai-surfaces` in the repository root. Git ignores
+this file. The template is `.env.ai-surfaces.example`, also in the
+repository root. The scripts read,
+from low to high priority: `.env.local`, `.env.ai-surfaces`, the shell
+environment. Empty values are ignored. Thus keys that are already in
+`.env.local` work without a change.
+
+If `.env.ai-surfaces` does not exist, copy `.env.ai-surfaces.example` to it. Tell the user
+which values to fill in. Never read the values back to the user, never print
+them, and never put a secret on a command line.
+
+The most important settings:
+
+| Setting | Use |
+|---|---|
+| `COMPARE_BUILDERS` | Default builders, for example `iai,claude,codex` |
+| `COMPARE_PARALLEL` | `1` (default) runs the builders at the same time. `0` runs them one at a time |
+| `COMPARE_LIVE` | `1` (default) opens a live page of the runs. `0` turns it off |
+| `COMPARE_LIVE_WINDOWS` | `1` also opens the Instance AI thread and a terminal window for each CLI. Default `0` |
+| `COMPARE_RECORD` | `1` (default) records the Instance AI video for the replay. `0` turns it off |
+| `COMPARE_CLI_MODE` | `interactive` (default) records the real CLI screen. `headless` uses the JSON event stream |
+| `COMPARE_RECORD_REQUESTS` | `1` (default) records the Codex request bodies, without headers, for the tool list. `0` turns it off |
+| `COMPARE_SCENARIOS` | `1` (default) runs each workflow with pinned data through the n8n MCP server. `0` turns it off |
+| `COMPARE_WIRE` | `1` (default) runs each workflow in process with recorded API fixtures (`run_wire.mjs`). `0` turns it off |
+| `COMPARE_BINARY_CHECKS` | `1` (default) runs the deterministic binary checks of the Instance AI eval suite. `0` turns it off |
+| `GOOGLE_GENERATIVE_AI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `MISTRAL_API_KEY`, `GROQ_API_KEY` | Model keys for the real AI runs. A workflow model without a key skips those runs |
+| `CLAUDE_MODEL`, `CODEX_MODEL` | Model ids. Empty uses the CLI default. A prefix such as `anthropic/` is removed |
+| `COMPARE_CLAUDE_TOOLS` | `none` (default) gives Claude Code only the n8n tools. `default` also gives it its built-in tools, all allowed (see "How the builders run") |
+| `CLAUDE_API_KEY` | Anthropic key for Claude Code. Falls back to `ANTHROPIC_API_KEY` |
+| `CODEX_API_KEY` | OpenAI key for Codex. Falls back to `OPENAI_API_KEY` |
+| `N8N_MCP_TOKEN` | MCP key of the n8n user. Optional |
+| `LANGSMITH_*` | Instance AI cost and tokens |
+
+## 1. Ask the user
+
+Ask for these two items. If the harness has a structured question tool, use
+it. If not, ask in chat. Do not guess them.
+
+1. **Prompt.** The exact text to send to each builder.
+2. **Expected outcome.** What a correct result does. Ask for concrete checks,
+   for example "WhatsApp trigger", "Gemini compares the question to the FAQ
+   sheet", "the false branch makes a Notion ticket and a Slack message".
+
+Use the builders and models from the settings. Ask about them only if the
+user wants a change for this run.
+
+Write one expected outcome item on each line, numbered or bulleted. The
+number of an item is its id: the judges cite it, and the page groups the
+checks by it.
+
+Then write the scenarios for the pinned-data runs, from the expected
+outcome. You do not need to ask the user for them. A scenario sets the data
+of the trigger, the service nodes and the AI nodes, and says which nodes
+must or must not run. Logic nodes (IF, Switch, Set, Code, Aggregate and
+others) run for real. Make one scenario for each branch of the expected
+outcome:
+
+```json
+{
+  "pinByType": {
+    "googleSheets": [{ "Question": "What are your opening hours?", "Answer": "9:00 to 17:00" }]
+  },
+  "scenarios": [
+    { "name": "The FAQ answers the question", "ai": { "booleans": true },
+      "expect": { "reach": ["whatsApp"], "skip": ["notion", "slack"] } },
+    { "name": "The FAQ does not answer the question", "ai": { "booleans": false },
+      "expect": { "reach": ["notion", "slack"] } }
+  ]
+}
+```
+
+- `pinByType`: output items for every node of a type, in all scenarios. A
+  type is the full name or the part after the last dot. A scenario can have
+  its own `pinByType`.
+- `trigger`: output items of the trigger. Without it, the trigger gets a
+  fixture in the shape that the trigger code emits (WhatsApp, Webhook, Chat,
+  Schedule, Telegram, Form and Manual triggers have one).
+- `ai`: the output of each AI node. `booleans` sets every boolean in the
+  output, `set` sets fields, and `text` sets the whole text of a node
+  without a structured output parser. The output follows the schema of the
+  structured output parser, or the JSON example in the prompt of the node.
+- `expect`: `reach` lists node types that must run without an error,
+  `skip` lists node types that must not run.
+- `triggerNode`: the name of the trigger that the scenario starts from, for a
+  workflow with more than one trigger. Without it, the scenario starts from
+  the trigger that reaches the most `reach` types.
+
+Without a scenarios file, `run_scenarios.py` makes one scenario for each
+value of the AI booleans, with no expectations.
+
+`run_wire.mjs` runs the same scenarios in process. There, service nodes
+send real requests that recorded fixtures answer, so a scenario can also
+check the API contract. Add these keys to the scenarios file:
+
+```json
+{
+  "scenarios": [
+    { "name": "The FAQ does not answer the question", "ai": { "booleans": false },
+      "expect": { "reach": ["notion", "slack"], "requests": [
+        { "type": "notion", "method": "POST", "contains": ["\"title\"", "@trigger.messages[0].from"] },
+        { "type": "slack", "method": "POST",
+          "contains": ["@trigger.messages[0].text.body", ["@response.notion.id", "@response.notion.url"]] } ] } }
+  ],
+  "edge": [
+    { "name": "The FAQ sheet is empty", "pinByType": { "googleSheets": [] }, "ai": { "booleans": false },
+      "expect": { "reach": ["notion", "slack"] } }
+  ],
+  "questions": [
+    { "name": "Opening hours, which the FAQ answers", "message": "Hi! What time do you open?",
+      "expect": { "skip": ["notion"], "requests": [
+        { "type": "whatsApp", "method": "POST", "contains": ["@trigger.messages[0].from", "9:00"] } ] } }
+  ]
+}
+```
+
+- `expect.requests`: each entry needs a request from a node of `type`,
+  optionally with `method` and an exact `count`, whose URL, query or body
+  contains every value in `contains`. An array in `contains` means any one
+  of its values. `@trigger.<path>` is a value of the trigger item.
+  `@response.<type>.<path>` is a value of the response to the last write
+  request of that node type, for example the id of the Notion page.
+- `expect.noWrites`: the scenario must send no write request.
+- `edge`: inputs that the prompt does not describe. The trigger kits in
+  `sdk.mjs` add standard edge cases, for example two WhatsApp messages in one
+  webhook call (each sender must get a request) and a status update (no
+  writes). Edge cases are a score, not part of the verdict.
+- `questions`: the AI nodes call the real model with this customer
+  `message`. The trigger fixture carries the message. A real-AI run needs a
+  model key, and is a score, not part of the verdict.
+
+## 2. Run the builders
+
+```bash
+python3 $S/compare.py --title "<short title>" \
+  --prompt-file <file with the prompt> --expected-file <file with the outcome> \
+  --scenarios-file <file with the scenarios>
+```
+
+Write the prompt, the expected outcome and the scenarios to files with the
+file tool first. Do not put them on the command line, because they can
+contain quotes. Use `--builders iai,claude` to change the builders for one
+run. Use `--rerun <old run dir>` to run the same case again. It also copies
+`scenarios.json`.
+
+The command takes 5 to 30 minutes. Start it in the background if the harness
+supports it, and read its output until it ends. It prints one line for each
+tool call, with the builder name in front, so the user can follow the run.
+It does these steps:
+
+1. Checks the setup and skips a builder that cannot run. It prints the
+   reason.
+2. Starts the live page (see below).
+3. Runs the builders at the same time. `--sequential` or
+   `COMPARE_PARALLEL=0` runs them one at a time.
+4. Fetches the Instance AI cost from LangSmith.
+5. Takes a screenshot of each canvas, and saves the node issues that the
+   n8n editor shows (`canvas-issues.json`).
+6. Runs each workflow once for each scenario, with pinned data, through the
+   `test_workflow` tool of the n8n MCP server (`exec.json`). It turns on
+   "Available in MCP" for a workflow that does not have it.
+7. Runs each workflow in process with recorded API fixtures, for the
+   scenarios, the edge cases and the real AI questions (`wire.json`).
+8. Runs the deterministic eval binary checks (`binary-checks.json`) and
+   checks the final-message claims (`claims.json`).
+9. Derives the deterministic checks (`auto-checks.json`). See "Checks on
+   the page".
+10. Writes `summary.txt`, and `judge/summary-blind.txt` for the judges.
+11. Builds the page without verdicts.
+
+The last output is JSON with `runDir`, `report`, `live`, `parallel`,
+`summary`, `blindSummary`, `models` and `skipped`.
+
+Parallel runs share one n8n instance and one machine. Thus the times of the
+builders include the shared load. The page says "run in parallel" under
+the title. Use
+`--sequential` when the user wants to compare times exactly.
+
+### Watch the runs
+
+`compare.py` starts `live_server.py` and opens its page in the default
+browser. It prints `live page: <url>`. Give the user this URL. The page
+shows each builder while it runs, and updates every 2 seconds without a
+reload:
+
+- the table: status, time, LLM calls, tool calls and tokens of each builder
+- **Live:** the real terminal screen of Claude Code and Codex, streamed from
+  the tmux recording, and a screenshot of the Instance AI thread every
+  2.5 seconds
+- the LLM steps of each builder. Claude Code and Codex steps come from the
+  session files of the CLIs, with tokens. Instance AI steps come from its
+  event stream, without tokens until the run ends
+
+The cost, the canvases and the Instance AI tokens come after the run. Until
+then, the Expected outcome box shows "Not verified yet", and the
+Correctness, Execution and n8n standards boxes show "Not run yet". When
+the report is ready, the page changes to it. The server listens on
+`127.0.0.1` only. It stops 30 minutes after the last request, when the run
+is done.
+
+The report shows the price and judge steps (steps 3 and 4) as queued until
+you start them. The Result column shows "Judge queued" or "Judging", and
+the Cost column of a CLI builder without a price shows "Price queued" or
+"Finding price". Each `build_report.py --step` call builds the page again.
+While a step is queued or running, the page on the live server reloads
+itself when the page changes. A page that you open from the file needs a
+manual reload.
+
+Headless mode and clean isolation have no terminal screen. For them, the
+page shows only the table and the LLM steps. `COMPARE_RECORD=0` removes the
+Instance AI screenshot.
+
+`COMPARE_LIVE_WINDOWS=1` also opens the old live views:
+
+- **Instance AI:** the thread opens in the default browser. The browser must
+  be logged in as `N8N_EVAL_EMAIL`, because a thread is visible only to its
+  owner. The script answers the confirmation cards, so the user must not
+  click them.
+- **Claude Code and Codex:** a terminal window shows the real screen of the
+  CLI, read-only (`docker exec … tmux attach -r`). In headless mode, the
+  window follows `<builder>/live.log` instead. On macOS the window opens in
+  Terminal. The first time, macOS asks if the agent's app can control
+  Terminal.
+
+Use `--no-live` or `COMPARE_LIVE=0` for runs without a display, for example
+in CI.
+
+### Replay on the page
+
+The Replay section shows the last screen of each builder. The play button
+on a screen plays that run in place on the page. **Expand** opens a
+full-screen player. In that player, the user sees all builders side by
+side, or picks one builder, and plays the runs at 1× to 8×:
+
+- **Claude Code and Codex:** `<builder>/screen.cast` is an asciinema
+  recording of the exact terminal screen. The page plays it with the
+  asciinema player.
+- **Instance AI:** `iai/screen.webm` is a video of the thread in the n8n UI.
+  A headless browser records it while the run continues.
+
+`COMPARE_RECORD=0` skips the Instance AI video. The terminal recordings come
+with interactive mode. `COMPARE_CLI_MODE=headless` has no terminal
+recording.
+
+### Canvas on the page
+
+`capture_canvas.mjs` expands the node groups, zooms to fit, and crops the
+screenshot to the nodes, their labels and their subtitles. Empty click areas
+and hidden hover toolbars do not change the crop. The page shows the canvas
+next to the expected outcome, at its real size and at most 360 px high.
+
+A click on the canvas opens a full-screen viewer. It has a tab for each run
+that has a canvas, the verdict and the n8n link of the run, and a zoom
+button. The left and right arrow keys switch runs. A click on the image
+zooms in to that point. Esc closes the viewer, and the page then shows the
+last run that the user looked at. Cmd-click or Ctrl-click still opens the
+image in a new tab.
+
+### Checks on the page
+
+Under the expected outcome and the canvas, each run has sections from
+`derive_checks.mjs`. No LLM makes these checks, except the model calls of
+the real AI runs. Correctness, Execution with pinned data and Service calls
+change the verdict. The other sections show a score only.
+
+**Correctness.** A failed check changes the verdict.
+
+| Id | Check | Source |
+|---|---|---|
+| C1 | Node parameters match the node definitions | `validateWorkflow` of `@n8n/workflow-sdk`, with the built node definitions |
+| C2 | The n8n editor shows no node errors | The issues of each node in the editor. Missing credentials and empty required pickers are setup for the user (C2s, information only) |
+| C3 | Every node can be reached from a trigger | The connections |
+| C4 | Each `$('Node')` names a node that exists and runs before the node that reads it | The expressions |
+| C5 | Each expression field is in the output of the node that it reads | The real output of the pinned-data runs |
+| C6 | No keys or tokens in node parameters | The parameters |
+
+C5 fails only when the output is real: a node that ran, the trigger
+fixture, the case data, or the schema of the structured output parser. A
+field that is missing only from a node schema sample or a prompt example
+shows as "Unconfirmed".
+
+**Execution with pinned data.** One check for each scenario that ran without
+an error, and one for each `reach` or `skip` expectation. A failed check
+changes the verdict.
+
+**Service calls.** `run_wire.mjs` loads the n8n execution engine from the
+built `packages/core` and runs each workflow in process. Triggers and AI
+nodes get pinned data. Service nodes run their real code, and the core
+eval hook answers each HTTP request with a recorded fixture before any
+credential is used. The process has no sandbox, and the hook sees only the
+requests that go through the n8n request helpers. Thus each node that runs
+must be on an allow-list (`runsInProcess` in `run_wire.mjs`):
+
+- logic nodes from `LOGIC_TYPES`, HTTP Request, Wait and Telegram
+- declarative nodes
+- nodes whose credentials all authenticate HTTP requests (an `authenticate`
+  block or OAuth)
+- a fixed set of AI nodes, with models that have a key in `MODEL_CREDENTIALS`
+
+Code, Function and AI Transform nodes, and nodes with database, SSH, FTP,
+SMTP, file or shell access, are not on the list. Each scenario checks the
+nodes that run from its trigger. Pinned nodes do not run, so a node under
+a pinned AI node counts only in the real AI runs. If a node outside the list
+would run, the scenario is skipped. The report shows it as skipped, not
+failed, with the names of the nodes.
+The pinned-data runs still run Code nodes in the n8n task runner. No request
+leaves the machine, except the model calls of the real AI runs. Empty pickers get
+`mock-<parameter>` values for the run. Each scenario row lists the nodes
+that ran with their item counts, and each request. A failed check changes
+the verdict.
+
+**Edge cases** and **Real AI runs.** The same in-process runs for the
+`edge` and `questions` entries of the scenarios file. A score.
+
+**Final message claims.** `claims.py` compares the final message with the
+trace and the workflow. A score.
+
+| Id | Claim | Source |
+|---|---|---|
+| M1 | The workflow id or URL is a workflow of this run | `run.json` |
+| M2 | The publish state (draft or active) | `workflow.json` `active` |
+| M3 | The workflow passed validation | The last validation tool call |
+| M4 | The workflow was tested, or was not run | Test and execution tool calls. Scripts that report passed tests count, with a note that they ran outside n8n |
+| M5 | Values in `code spans` | The workflow, the prompt and the tool calls |
+| M6 | A claim that the workflow was checked agrees with the service-call runs | `wire.json` |
+
+**Eval binary checks.** `binary_checks.mts` runs the deterministic binary
+checks of the Instance AI eval suite
+(`packages/@n8n/instance-ai/evaluations/binaryChecks`), for example
+`no_empty_set_nodes` and `no_unnecessary_code_nodes`. Checks that do not
+apply are left out. A score.
+
+**Eval quality checks.** The judges answer the LLM binary checks of the
+same suite: `valid_data_flow`, `correct_node_operations`,
+`handles_multiple_items`, `descriptive_node_names` and
+`response_describes_changes_accurately`. `summarize_runs.py` reads their
+rules from the check sources. A score.
+
+**n8n standards.** A score, not part of the verdict. The builders do not get
+these rules: Claude Code and Codex run without the n8n skills, and Instance
+AI uses its own prompts. Only the rules that apply to the workflow count.
+
+| Id | Rule | Source |
+|---|---|---|
+| S1 | An unattended workflow has an error workflow | `n8n-error-handling-official` |
+| S12 | A failed node goes to a wired error output, not on as data | `n8n-error-handling-official` |
+| S2 | Nodes that call a service have `retryOnFail` | `n8n-error-handling-official` |
+| S3 | Expressions read data by node name, not `$json` | `n8n-expressions-official` |
+| S4 | Structured AI output uses a manual schema with `autoFix` | `n8n-ai-nodes-official` STRUCTURED_OUTPUT.md |
+| S5 | Messages do not carry the n8n attribution | `n8n-node-configuration-official` COMMS_NODES.md, and the node definitions |
+| S6 | References do not read only the first item of a node that can output several | `n8n-expressions-official`, and the trigger code |
+| S7 | A model that decides a branch has a temperature of 0.2 or lower | workflow-sdk best practices: triage |
+| S8 | AI output that decides a branch has a reasoning field | workflow-sdk best practices: triage |
+| S9 | Every branch after an AI decision, including the fallback, goes somewhere | workflow-sdk best practices: triage |
+| S10 | A chatbot agent has memory | workflow-sdk best practices: chatbot |
+| S11 | The memory session key names a node and identifies the conversation | workflow-sdk best practices: chatbot, and `n8n-ai-nodes-official` MEMORY.md |
+
+The skill rules come from commit `27b6b9f` of n8n-io/skills. Each check
+links to its source.
+
+**The verdict** comes from the checks, not from the judges alone:
+
+- `fail` when there is no workflow, or the judges say that the workflow does
+  not do what the prompt asks (`goalMet: false`)
+- `pass` when every expected outcome item is met, and no correctness,
+  execution or service-call check failed
+- `partial` for all other results
+
+A met item must cite evidence that the workflow contains. Without evidence,
+or with evidence that is not in the workflow, the item shows as not
+confirmed. The Result column shows the verdict and the standards score
+(`Std 5/10`).
+
+An old `verdicts.json` without ids or evidence keeps the verdict of the
+judge. A failed correctness or execution check changes its `pass` to
+`partial`.
+
+### LLM steps on the page
+
+Each run has a list of LLM steps, like the LLM steps view in the Instance AI
+debug panel. The box is collapsed by default. Click its title to open it. On
+the live page, a box that you open stays open when the page updates. The list
+shows the tools or text of each call, its time and a
+token bar. Select a step, or use the arrow keys, to see the token counts and
+what the call did. Step 1 starts with what the model gets before the first
+call, then the user message. Tool output is collapsed, except when the tool
+failed.
+
+- **Claude Code:** the system prompt and the tool definitions, from the
+  request shape in the session file. The tools show as chips in groups:
+  System (the built-in tools of the CLI), one group for each MCP server, and
+  Other (for example tools that the model provider runs). System and MCP
+  show "None" when the model had no such tools. The tools that the run
+  called are highlighted, with a call count and a dot for failures. Select a chip to see the description and a table of
+  the parameters. A filter box searches names and descriptions.
+- **Codex:** the system prompt (base instructions), the developer messages
+  and the environment context, from the session file. Codex does not save
+  its tool definitions, so the runner records them (see "Isolation"). The
+  tools use the same chips. Codex declares most system tools inside the
+  description of its `exec` tool, so they show as "System · in exec" with
+  their TypeScript declaration. Codex does not put MCP tools in the prompt:
+  its scripts look them up in `ALL_TOOLS` at run time. The MCP group thus
+  shows only the MCP tools that the run called, with a dashed border and no
+  schema. Runs from before the recorder show no Codex tools.
+- **Instance AI:** the system prompt, the context that n8n puts before the
+  prompt (for example `<thread-context>`), and the tools, from the first
+  LangSmith LLM call of each agent. `langsmith_usage.py` saves them in
+  `context.json`. Tools that the agent loads later with `load_tool` show as
+  "System · loaded with load_tool". A sub-agent shows its own system prompt,
+  briefing and tools at its first step. Runs from before `context.json` show
+  no Instance AI context: run `langsmith_usage.py` again for them.
+
+- **Claude Code:** the session file links each event to its LLM call.
+- **Codex:** text goes to the response that wrote it. A tool call goes to the
+  response that asked for it. Codex calls the n8n tools from scripts that it
+  runs with its `exec` tool. The page shows these scripts too, but does not
+  count them as tool calls.
+- **Instance AI:** the LangSmith call times place each event. Text times are
+  approximate. An asterisk on a time marks text that the page puts at the
+  previous tool call.
+
+### When a check fails
+
+- Cannot log in: n8n is not running, or `N8N_BASE_URL`, `N8N_EVAL_EMAIL` or
+  `N8N_EVAL_PASSWORD` is wrong.
+- The MCP token does not work: n8n shows the full MCP key only once. Ask the
+  user to put the key in `N8N_MCP_TOKEN`, or ask before you run
+  `python3 $S/prepare.py --rotate`. Rotation makes the old key stop working
+  in every other MCP client of this user.
+- Docker is not running: Codex is skipped, and Claude runs as a clean
+  process.
+
+## 3. Find the prices
+
+Codex reports no cost, and the estimate of Claude Code can be out of date
+for new models. Thus, find the published price of each model that a CLI
+builder used. The models are in `models` in the output of `compare.py`, and
+in `<runDir>/models.json`. Instance AI cost comes from LangSmith, so do not
+look up its model.
+
+Do not skip this step. In interactive mode neither CLI reports a cost, so
+without prices the page shows "not reported" for both. `build_report.py`
+prints `models without a published price in case.json: …` while a model has
+no price.
+
+First, tell the page that you look for the prices:
+
+```bash
+python3 $S/build_report.py <runDir> --step prices=running
+```
+
+For each model:
+
+1. Search the web for the official price page of the provider, for example
+   the Anthropic pricing page or the OpenAI pricing page. Do not use
+   third-party price lists if the provider page has the model.
+2. Read four prices in USD per million tokens: input, cache read, cache
+   write, output. If the provider has no cache-write price, use the input
+   price. Anthropic has two cache-write prices. Use the 5-minute price, and
+   tell the user that the 1-hour price is higher.
+3. If you cannot find the model, write no price for it. The page then shows
+   the estimate of Claude Code, or "not reported" for Codex. Tell the user.
+
+Add the prices to `<runDir>/case.json`, with the URL that you used:
+
+```json
+"prices": {
+  "<model id as in models.json>": {
+    "input": 5, "cached": 0.5, "cacheWrite": 6.25, "output": 25,
+    "source": "https://..."
+  }
+}
+```
+
+The page calculates the cost of each call from its tokens. The tooltip of a
+Cost cell shows the source of the cost, the prices and the URL.
+
+## 4. Judge each run
+
+Show the costs and tell the page that you judge the runs:
+
+```bash
+python3 $S/build_report.py <runDir> --step prices=done --step judge=running
+```
+
+A panel of three judges judges the runs blind. Each judge reads only
+`<runDir>/judge/summary-blind.txt`. In that file the builders are called A,
+B and C in a random order, and the final messages have no builder or model
+names. `judge/blind-map.json` maps the letters back. Do not give a judge
+`summary.txt`, the map, the trace files or this conversation.
+
+If the harness can start sub-agents, start three sub-agents with a fresh
+context, one for each judge. Use a model that is not one of the builder
+models when the harness lets you choose. If the harness cannot start
+sub-agents, judge once yourself from the blind file only, and write
+`verdicts-1.json`.
+
+Give each judge these instructions:
+
+> Read `<runDir>/judge/summary-blind.txt`. It lists the expected outcome
+> criteria with their ids, and for each builder the final message, every
+> node with all its parameters, the connections, and deterministic facts
+> (AUTO-CHECK FACTS). Trust the facts: they come from the node definitions,
+> the n8n editor and real pinned-data runs. For each builder, make one check
+> for each criterion id. A met check must cite evidence: the node name, the
+> parameter path, and the exact value in the workflow. Without evidence,
+> set `ok` to `null`. Set `goalMet` to false when the workflow does not do
+> what the prompt asks, and give the reason in `goalReason`. Then answer
+> each QUALITY CHECK in `quality`, with the check name as the `id`. Write
+> the result to `<runDir>/judge/verdicts-<n>.json`.
+
+The format of each judge file, with the builder letters as keys:
+
+```json
+{
+  "A": {
+    "summary": "One or two sentences about the result.",
+    "goalMet": true,
+    "goalReason": "",
+    "checks": [
+      { "id": "1", "ok": true, "text": "A WhatsApp trigger starts the workflow on `messages`",
+        "evidence": [{ "node": "WhatsApp Message Received", "path": "parameters.updates", "value": "messages" }] },
+      { "id": "9", "ok": false, "text": "Send Reply reads `$json.from`, which does not exist" },
+      { "id": "16", "ok": null, "text": "Could not confirm the sheet columns" },
+      { "ok": false, "text": "The Slack node is not connected" }
+    ],
+    "quality": [
+      { "id": "valid_data_flow", "ok": true, "text": "Every expression reads a field that its source outputs" },
+      { "id": "descriptive_node_names", "ok": false, "text": "Two nodes keep the default names `Code` and `IF`" }
+    ]
+  }
+}
+```
+
+- `id` is the id of the criterion in brackets, as a string. Leave out `id`
+  for a problem that is not a criterion. The page shows it under "Other
+  findings".
+- `evidence` lists `{node, path, value}` entries. `path` is a dotted path in
+  the node, for example `parameters.textBody` or
+  `parameters.conditions.conditions[0].leftValue`. The page checks that the
+  node has the path and that the path contains the value. An entry with
+  `"source": "message"` and a `value` quotes the final message and is not
+  checked.
+- Write each check as one short sentence with one fact. Put expressions,
+  field paths and parameter names in backticks. The page shows them as
+  code. Do not repeat the failed checks in the summary.
+
+Then merge the panel into `verdicts.json`:
+
+```bash
+python3 $S/merge_verdicts.py <runDir>
+```
+
+For each criterion, the merged check takes the value that all judges or a
+majority gave. A split shows on the page ("3 judges · 1 split", and "2 of 3
+judges" on the check). Without a majority, the check is not confirmed. The
+summary and the other findings come from judge 1. The page computes the
+verdict (see "Checks on the page").
+
+## 5. Build the page again
+
+```bash
+python3 $S/build_report.py <runDir> --step judge=done
+```
+
+If the command prints `models without a published price`, do step 3 for
+those models and build the page again.
+
+The page is `<runDir>/report/index.html`. It works offline. Give the user the
+path, and the `live` URL from the output of `compare.py` while the server
+runs. Then, in a few sentences, report:
+
+- the verdict of each builder, with the most important reason
+- failed correctness, execution or service-call checks, and judge splits
+- the standards, edge-case, real-AI, claims and eval scores of each
+  builder, as separate results
+- the cost, time and number of LLM calls of each builder
+- builders that were skipped, and data that is missing
+
+## How the builders run
+
+**Instance AI** runs through the REST API of the instance, with its model.
+The runner answers every confirmation card with one fixed policy, so runs
+can be compared. It approves actions and plans, picks the first option of a
+question, allows a domain once, defers workflow and credential setup cards,
+and denies access to local files and MCP connections. `iai/run.json` lists
+each decision.
+
+**Claude Code and Codex** run in a Docker container (`COMPARE_ISOLATION=container`):
+
+- The container mounts only the run folder of the builder. The builder
+  cannot see host files, skills, memory, `CLAUDE.md`, `AGENTS.md` or user
+  settings.
+- The container has no Linux capabilities and a read-only root file system.
+- Claude starts with a new config folder, no built-in tools, and only the
+  `n8n` MCP server. It does not use `--bare`, because `--bare` replaces the
+  normal system prompt with a 150-byte one. The container already keeps out
+  host settings, so Claude gets its normal system prompt, like Codex does.
+  `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` stops update checks and
+  telemetry.
+- `COMPARE_CLAUDE_TOOLS=default` gives Claude the built-in tools of a normal
+  Claude Code session, such as Bash, Read, Write, Edit and WebFetch, with
+  `--permission-mode bypassPermissions`. Claude then uses every tool without
+  a question, as Codex does. The tools can change files in the run folder
+  and read the environment, which holds the model key and the MCP token.
+  The secret check after the run finds a key that a tool wrote to a file.
+  Claude Code also turns on tool search: the model gets only the names of
+  the n8n tools and of some built-in tools, and loads a schema with
+  `ToolSearch`. The page shows these deferred tools with a dashed border.
+  The default `none` keeps runs comparable with older runs, because the
+  extra tools change the tokens and the cost.
+- Codex gets a new `CODEX_HOME` that holds only the n8n MCP server.
+  Its plugins, apps, browser use, computer use and image generation are off.
+- The container can still reach the internet and the host network. It needs
+  them for the model API and for n8n.
+- Codex sends its model requests through a local recorder
+  (`request_recorder.py`). The recorder sends each request to OpenAI without
+  changes and writes the request body, but not the headers, to
+  `codex/requests.jsonl`. The key is in a header, so it stays out of the file.
+  The file does not hold the conversation input. It holds the tools and the
+  instructions only when they change. The recorder listens on `127.0.0.1`,
+  which Docker Desktop forwards to `host.docker.internal`. On Linux, set
+  `COMPARE_RECORDER_BIND` to the address of the Docker bridge. Set
+  `COMPARE_RECORD_REQUESTS=0` to send Codex straight to OpenAI.
+
+By default the CLIs run in interactive mode (`COMPARE_CLI_MODE=interactive`):
+
+- The normal terminal screen of the CLI runs in tmux, at 120 × 36
+  characters. tmux answers the terminal queries of the CLI, and its output
+  goes to `screen.cast`.
+- The CLIs connect MCP servers in the background. Thus the script pastes the
+  prompt when the input box shows, plus a short wait, not as an argument.
+- Claude uses `--permission-mode dontAsk`: the n8n tools are pre-approved,
+  and all other tools are denied without a question.
+- The model key reaches the CLI through a variable that only the CLI reads
+  (`apiKeyHelper` for Claude, a provider `env_key` for Codex). Thus no screen
+  asks to confirm the key or shows part of it, and no key is in a file.
+- The script reads the session file of the CLI for the progress lines, the
+  end of the turn, the trace and the tokens. Claude Code then shows no cost
+  estimate, so the cost needs the published prices.
+- After each run, the script searches the run folder for the model key and
+  the MCP token. It prints a warning with the file name if it finds one.
+
+`COMPARE_CLI_MODE=headless` uses `claude -p` and `codex exec` with a JSON
+event stream instead.
+
+The first run builds the image `n8n-builder-sandbox` from
+`sandbox/Dockerfile`. A change to the Dockerfile rebuilds the image on the
+next run. The image has `python3`, `jq` and `ripgrep`, because Codex and
+Claude Code with `COMPARE_CLAUDE_TOOLS=default` use them in their shell. To get new CLI
+versions, run `docker rmi n8n-builder-sandbox`.
+
+`COMPARE_ISOLATION=clean` runs Claude as a clean process on the host: an
+empty environment, a new config folder, no built-in tools, and only the n8n
+MCP server. It ignores `COMPARE_CLAUDE_TOOLS`. It uses `--bare` to keep out host settings, so it gets the
+minimal system prompt. It needs an API key, because `--bare` does not read
+the subscription login. Codex has no clean mode, because its shell tool can
+read the host disk.
+
+## Single steps
+
+`compare.py` calls these scripts. Use them to repeat one step.
+
+| Script | Step |
+|---|---|
+| `prepare.py` | Check the setup. `--rotate` makes a new MCP key |
+| `run_instance_ai.py --out <dir>/iai --prompt-file <file>` | Run Instance AI |
+| `run_cli_builder.py --builder claude\|codex --out <dir>/<name> --prompt-file <file>` | Run a CLI builder. `--mode headless` skips the screen |
+| `node record_iai.mjs <dir>/iai` | Record the Instance AI video. `run_instance_ai.py` starts it |
+| `live_server.py --run-dir <dir>` | Serve the live page and the report of a run folder. Prints nothing. The URL is in `<dir>/live-url.txt` |
+| `langsmith_usage.py --run-dir <dir>/iai` | Fetch Instance AI usage and model context |
+| `node capture_canvas.mjs <dir>` | Save `workflow.json`, `canvas.png` and `canvas-issues.json` for each run |
+| `run_scenarios.py <dir>` | Run each workflow with pinned data. Writes `<name>/exec.json` |
+| `node sdk.mjs pin-plan <workflow.json> [scenarios.json]` | Print the pin data of each scenario, without a run |
+| `node run_wire.mjs <dir>` | Run each workflow in process with recorded API fixtures. Writes `<name>/wire.json` |
+| `pnpm --filter @n8n/instance-ai exec tsx $S/binary_checks.mts <dir>` | Run the deterministic eval binary checks. Writes `<name>/binary-checks.json` |
+| `claims.py <dir>` | Check the final-message claims. Writes `<name>/claims.json` |
+| `node derive_checks.mjs <dir>` | Write `<name>/auto-checks.json` from all the files above |
+| `summarize_runs.py <dir> [--blind]` | Print the judge summary. `--blind` writes `judge/summary-blind.txt` and `judge/blind-map.json` |
+| `merge_verdicts.py <dir>` | Merge `judge/verdicts-*.json` into `verdicts.json` |
+| `build_report.py <dir> [--step judge\|prices=queued\|running\|done]` | Build the page. `--step` first sets the state of a step after the run |
+
+If a runner did not find a workflow id, read its trace for the id. Set
+`workflowId` in that `run.json`, then run `capture_canvas.mjs`,
+`run_scenarios.py`, `run_wire.mjs`, `binary_checks.mts`, `claims.py`,
+`derive_checks.mjs` and `summarize_runs.py` again.
+
+## Notes
+
+- Cost sources: Instance AI cost comes from LangSmith. For Claude Code and
+  Codex, the cost is tokens × the published prices in `case.json`. Without a
+  price, Claude Code in headless mode shows its own estimate. All other runs
+  show "not reported".
+  The tooltip of each Cost cell shows the source of the cost.
+- The workflows stay in the n8n instance. Archive them in n8n when they are
+  no longer necessary. The pinned-data runs add test executions to them.
+- The n8n skills stay out of the sandboxes, so the standards score shows
+  what each builder does without them. Do not add the skills to the
+  sandboxes to raise the score.

.agents/skills/testmax-n8n-ai/sandbox/Dockerfile

@@ -0,0 +1,21 @@
+# Sandbox for CLI workflow builders. The container sees only the run directory
+# that is mounted at /out. It reaches n8n through host.docker.internal.
+FROM node:26-slim
+
+ARG CLAUDE_CODE_VERSION=latest
+ARG CODEX_VERSION=latest
+
+# Codex uses the system trust store. Claude Code uses the CA list in Node.
+# tmux runs the interactive CLI screens so that they can be recorded and watched.
+# Claude Code (with built-in tools) and Codex use python3, jq and ripgrep in their shell to read large tool results.
+RUN apt-get update \
+	&& apt-get install -y --no-install-recommends ca-certificates tmux python3 jq ripgrep \
+	&& rm -rf /var/lib/apt/lists/*
+
+RUN npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" "@openai/codex@${CODEX_VERSION}" \
+	&& npm cache clean --force
+
+ENV DISABLE_AUTOUPDATER=1 \
+	DISABLE_TELEMETRY=1 \
+	DISABLE_ERROR_REPORTING=1 \
+	CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

.agents/skills/testmax-n8n-ai/sandbox/screen.mjs

@@ -0,0 +1,100 @@
+// Run an interactive CLI inside tmux and record its screen as an asciinema v2 file.
+//
+//   node screen.mjs run <dir>      start tmux, record, and wait for <dir>/stop or the CLI exit
+//   node screen.mjs launch <dir>   (inside tmux) start the command in <dir>/command.json
+//   node screen.mjs record <dir>   (tmux pipe-pane) write the pane output to <dir>/screen.cast
+//
+// <dir>/command.json is {"argv": [...], "cols": 120, "rows": 36, "input": "prompt",
+// "readyText": "text on the screen when the CLI accepts input", "settleMs": 0}. tmux answers the terminal
+// queries of the CLI, so the CLI behaves as in a real terminal. The recording has the raw
+// output of the CLI, so a player shows the exact screen.
+import { spawn, spawnSync } from 'node:child_process';
+import { createWriteStream, existsSync, readFileSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+
+const [mode, dir] = process.argv.slice(2);
+const command = JSON.parse(readFileSync(join(dir, 'command.json'), 'utf8'));
+const READY = '/tmp/screen-ready';
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+const tmux = (...args) => spawnSync('tmux', args, { stdio: 'inherit' }).status;
+const screenText = () =>
+	spawnSync('tmux', ['capture-pane', '-p', '-t', 'run'], { encoding: 'utf8' }).stdout ?? '';
+
+// The CLIs connect MCP servers in the background. A prompt on the command line can start
+// before the n8n tools exist, so the prompt is pasted when the input box is ready.
+async function typePrompt({ input, readyText, settleMs = 0 }) {
+	const deadline = Date.now() + 60_000;
+	while (readyText && !screenText().includes(readyText) && Date.now() < deadline) await sleep(250);
+	await sleep(settleMs);
+	tmux('set-buffer', '-b', 'prompt', '--', input);
+	tmux('paste-buffer', '-p', '-d', '-b', 'prompt', '-t', 'run');
+	await sleep(500);
+	tmux('send-keys', '-t', 'run', 'Enter');
+}
+
+if (mode === 'record') {
+	const out = createWriteStream(join(dir, 'screen.cast'));
+	out.write(
+		JSON.stringify({
+			version: 2,
+			width: command.cols,
+			height: command.rows,
+			timestamp: Math.floor(Date.now() / 1000),
+			env: { TERM: 'xterm-256color' },
+		}) + '\n',
+	);
+	const start = process.hrtime.bigint();
+	const decoder = new TextDecoder('utf-8');
+	process.stdin.on('data', (chunk) => {
+		const text = decoder.decode(chunk, { stream: true });
+		if (!text) return;
+		const t = Number(process.hrtime.bigint() - start) / 1e9;
+		out.write(JSON.stringify([Number(t.toFixed(6)), 'o', text]) + '\n');
+	});
+	process.stdin.on('end', () => out.end());
+} else if (mode === 'launch') {
+	while (!existsSync(READY)) await sleep(50);
+	const [bin, ...args] = command.argv;
+	const child = spawn(bin, args, { stdio: 'inherit' });
+	child.on('exit', (code) => {
+		writeFileSync(join(dir, 'exited'), String(code ?? 1));
+		process.exit(code ?? 1);
+	});
+	// A CLI that cannot start emits error, not exit. Without the marker, the run waits for ever.
+	child.on('error', (error) => {
+		console.error(`Cannot start ${bin}: ${error.message}`);
+		writeFileSync(join(dir, 'exited'), '127');
+		process.exit(127);
+	});
+} else if (mode === 'run') {
+	const self = new URL(import.meta.url).pathname;
+	// mouse and focus-events stop the tmux tips that Claude Code shows in the recorded screen.
+	writeFileSync(
+		'/tmp/tmux.conf',
+		'set -g status off\nset -g default-terminal xterm-256color\nset -g mouse on\nset -g focus-events on\n',
+	);
+	tmux(
+		'-f',
+		'/tmp/tmux.conf',
+		'new-session',
+		'-d',
+		'-s',
+		'run',
+		'-x',
+		String(command.cols),
+		'-y',
+		String(command.rows),
+		`node ${self} launch ${dir}`,
+	);
+	// A watcher in another terminal size must not resize the recorded screen. tmux 3.5 stops
+	// when this option is in the startup config, so it is set after the session starts.
+	tmux('set-option', '-g', 'window-size', 'manual');
+	tmux('pipe-pane', '-O', '-t', 'run', `node ${self} record ${dir}`);
+	writeFileSync(READY, '');
+	if (command.input) await typePrompt(command);
+	while (!existsSync(join(dir, 'stop')) && !existsSync(join(dir, 'exited'))) await sleep(500);
+	// Keep the final screen in the recording for a moment.
+	await sleep(2500);
+	tmux('kill-server');
+	await sleep(500);
+}

.agents/skills/testmax-n8n-ai/scripts/binary_checks.mts

@@ -0,0 +1,50 @@
+// Runs the deterministic binary checks of the Instance AI eval suite on each workflow of a run.
+// Run it with tsx from the instance-ai package, because the checks are TypeScript sources:
+//
+//   pnpm --filter @n8n/instance-ai exec tsx <skill>/scripts/binary_checks.mts <run-dir>
+//
+// Writes <builder>/binary-checks.json with {source, outcomes: [{name, description, dimension, status, comment}]}.
+import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath, pathToFileURL } from 'node:url';
+
+const repo = resolve(dirname(fileURLToPath(import.meta.url)), '../../../..');
+const SOURCE = 'packages/@n8n/instance-ai/evaluations/binaryChecks/checks';
+
+type Check = {
+	name: string;
+	description: string;
+	kind: 'deterministic' | 'llm';
+	dimension: string;
+	run(workflow: unknown, ctx: Record<string, unknown>): unknown;
+};
+type Result = { pass: boolean; comment?: string; applicable?: boolean; errored?: boolean };
+
+const readJson = (file: string): unknown => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
+
+const runDir = resolve(process.argv[2] ?? '.');
+const { DETERMINISTIC_CHECKS } = (await import(pathToFileURL(join(repo, SOURCE, 'index.ts')).href)) as { DETERMINISTIC_CHECKS: Check[] };
+const prompt = existsSync(join(runDir, 'prompt.txt')) ? readFileSync(join(runDir, 'prompt.txt'), 'utf8') : '';
+
+for (const entry of readdirSync(runDir, { withFileTypes: true })) {
+	const dir = join(runDir, entry.name);
+	const workflow = entry.isDirectory() ? readJson(join(dir, 'workflow.json')) : null;
+	if (!workflow) continue;
+	const outcomes = [];
+	for (const check of DETERMINISTIC_CHECKS) {
+		let status: string;
+		let comment: string | undefined;
+		try {
+			const r = (await check.run(workflow, { prompt })) as Result;
+			status = r.errored ? 'error' : r.applicable === false ? 'n_a' : r.pass ? 'pass' : 'fail';
+			comment = r.comment;
+		} catch (error) {
+			status = 'error';
+			comment = error instanceof Error ? error.message : String(error);
+		}
+		outcomes.push({ name: check.name, description: check.description, dimension: check.dimension, status, ...(comment ? { comment } : {}) });
+	}
+	writeFileSync(join(dir, 'binary-checks.json'), JSON.stringify({ source: SOURCE, outcomes }, null, 1));
+	const scored = outcomes.filter((o) => o.status === 'pass' || o.status === 'fail');
+	console.log(`${entry.name}: binary-checks.json written (${scored.filter((o) => o.status === 'pass').length}/${scored.length})`);
+}

.agents/skills/testmax-n8n-ai/scripts/capture_canvas.mjs

@@ -0,0 +1,162 @@
+#!/usr/bin/env node
+// Take a cropped canvas screenshot of each workflow that a run created.
+//
+//   node capture_canvas.mjs <run-dir>
+//
+// Reads <run-dir>/<builder>/run.json and writes <run-dir>/<builder>/canvas.png, workflow.json and
+// canvas-issues.json.
+import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { join, resolve } from 'node:path';
+
+import { baseUrl, launchBrowser, signIn } from './n8n_browser.mjs';
+
+const runDir = resolve(process.argv[2] ?? '.');
+
+// The status icons that CanvasNodeStatusIcons.vue draws on a node. n8n computes them from the node
+// type description, so they are the verdict of n8n itself on each node.
+const ISSUE_MARKERS = ['node-issues', 'node-setup-required', 'node-not-installed', 'node-restricted'];
+
+/**
+ * The status icon of each node, with the issues that the editor computed for it. The tooltips
+ * are teleported and show after a delay, so the issues come from the workflow document store.
+ */
+async function readIssues(page, workflowId) {
+	return await page.evaluate(
+		({ markers, workflowId }) => {
+			const icons = {};
+			for (const el of document.querySelectorAll(
+				markers.map((m) => `[data-node-name] [data-test-id="${m}"]`).join(','),
+			)) {
+				icons[el.closest('[data-node-name]').dataset.nodeName] = el.dataset.testId;
+			}
+			const pinia = document.querySelector('#app')?.__vue_app__?.config.globalProperties.$pinia;
+			const key = [...(pinia?._s.keys() ?? [])].find((k) =>
+				k.startsWith(`workflowDocuments/${workflowId}@`),
+			);
+			const storeNodes = key ? (pinia._s.get(key).allNodes ?? []) : [];
+			const nodes = storeNodes
+				.filter((n) => icons[n.name] || (n.issues && Object.keys(n.issues).length))
+				.map((n) => ({
+					name: n.name,
+					type: n.type,
+					marker: icons[n.name] ?? null,
+					issues: n.issues ?? {},
+				}));
+			for (const [name, marker] of Object.entries(icons)) {
+				if (!nodes.some((n) => n.name === name)) nodes.push({ name, marker, issues: {} });
+			}
+			return { markers, store: Boolean(key), nodes };
+		},
+		{ markers: ISSUE_MARKERS, workflowId },
+	);
+}
+
+const runs = readdirSync(runDir, { withFileTypes: true })
+	.filter((d) => d.isDirectory() && existsSync(join(runDir, d.name, 'run.json')))
+	.map((d) => ({
+		dir: join(runDir, d.name),
+		meta: JSON.parse(readFileSync(join(runDir, d.name, 'run.json'), 'utf8')),
+	}))
+	.filter((r) => r.meta.workflowId);
+
+if (runs.length === 0) {
+	console.log('No run has a workflowId. Nothing to capture.');
+	process.exit(0);
+}
+
+const browser = await launchBrowser();
+const VIEWPORT = { width: 1800, height: 1000 };
+const context = await browser.newContext({ viewport: VIEWPORT, deviceScaleFactor: 2 });
+const page = await context.newPage();
+await signIn(page);
+
+for (const { dir, meta } of runs) {
+	const id = meta.workflowId;
+	const res = await page.request.get(`${baseUrl}/rest/workflows/${id}`, {
+		headers: {
+			'browser-id': await page.evaluate(() => localStorage.getItem('n8n-browserId') ?? ''),
+		},
+	});
+	if (res.ok())
+		writeFileSync(join(dir, 'workflow.json'), JSON.stringify((await res.json()).data, null, 1));
+	else console.log(`${id}: workflow fetch failed with ${res.status()}`);
+
+	await page.goto(`${baseUrl}/workflow/${id}`);
+	try {
+		await page.locator('[data-test-id="canvas-node"]').first().waitFor({ timeout: 30_000 });
+	} catch {
+		console.log(`${id}: no canvas nodes, skipped`);
+		continue;
+	}
+	// Expanding one group re-renders the others, so repeat until no collapsed group is left.
+	for (let round = 0; round < 5; round++) {
+		const clicked = await page.evaluate(() => {
+			const buttons = [...document.querySelectorAll('button[aria-label="Expand"]')];
+			buttons.forEach((b) => b.click());
+			return buttons.length;
+		});
+		if (clicked === 0) break;
+		await page.waitForTimeout(400);
+	}
+	// The layout moves after groups expand. Fit twice so the second fit sees the final size.
+	for (let i = 0; i < 2; i++) {
+		await page.waitForTimeout(1000);
+		await page
+			.locator('[data-test-id="zoom-to-fit"]')
+			.click({ timeout: 5_000 })
+			.catch(() => {});
+	}
+	await page.waitForTimeout(800);
+
+	const box = await page.evaluate(() => {
+		const rects = [...document.querySelectorAll('.vue-flow__node')].map((n) =>
+			n.getBoundingClientRect(),
+		);
+		if (rects.length === 0) return null;
+		// Labels, subtitles and expanded groups overflow their node element, so use the descendants too.
+		// Only count what draws: hit areas and hover toolbars are empty boxes around the node.
+		const draws = (el) =>
+			el instanceof SVGSVGElement ||
+			el instanceof HTMLImageElement ||
+			[...el.childNodes].some((c) => c.nodeType === Node.TEXT_NODE && c.textContent.trim());
+		const visible = (el) => {
+			for (let e = el; e && !e.classList.contains('vue-flow__node'); e = e.parentElement) {
+				const style = getComputedStyle(e);
+				if (
+					style.visibility === 'hidden' ||
+					style.display === 'none' ||
+					Number(style.opacity) === 0
+				)
+					return false;
+			}
+			return true;
+		};
+		const all = rects.concat(
+			[...document.querySelectorAll('.vue-flow__node *')]
+				.filter((el) => draws(el) && visible(el))
+				.map((n) => n.getBoundingClientRect())
+				.filter((r) => r.width > 0 && r.height > 0),
+		);
+		const left = Math.min(...all.map((r) => r.left));
+		const top = Math.min(...all.map((r) => r.top));
+		const right = Math.max(...all.map((r) => r.right));
+		const bottom = Math.max(...all.map((r) => r.bottom));
+		return { left, top, right, bottom };
+	});
+	const pad = 40;
+	const clip = box
+		? {
+				x: Math.max(0, box.left - pad),
+				y: Math.max(0, box.top - pad),
+				width: Math.min(VIEWPORT.width, box.right + pad) - Math.max(0, box.left - pad),
+				height: Math.min(VIEWPORT.height, box.bottom + pad) - Math.max(0, box.top - pad),
+			}
+		: undefined;
+	await page.screenshot({ path: join(dir, 'canvas.png'), clip });
+	console.log(`${id}: canvas.png written`);
+	const issues = await readIssues(page, id);
+	writeFileSync(join(dir, 'canvas-issues.json'), JSON.stringify(issues, null, 1));
+	console.log(`${id}: canvas-issues.json written (${issues.nodes.length} nodes with a status icon)`);
+}
+
+await browser.close();

.agents/skills/testmax-n8n-ai/scripts/claims.py

@@ -0,0 +1,180 @@
+#!/usr/bin/env python3
+"""Check what the final message of each builder says against the trace, the workflow and the runs.
+
+  claims.py <run-dir>
+
+Writes <builder>/claims.json with {message, checks: [{id, ok, text, details}]}. No LLM is involved.
+A check is about one kind of claim: the workflow id, the publish state, validation, testing, the
+values in `code spans`, and if a claim that the workflow works agrees with the service-call runs.
+A message that does not make a kind of claim gets no check for it.
+"""
+
+import json
+import re
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from build_report import load_run  # noqa: E402
+from summarize_runs import final_message  # noqa: E402
+
+TEST_TOOL = re.compile(r'test|execut|verify|run[-_]?workflow|simulat', re.I)
+SHELL_TOOL = re.compile(r'command|shell|bash', re.I)
+SCRIPT_PASSED = re.compile(r'"passed"\s*:\s*true|\b\d+ (?:tests? )?passed\b|\ball tests passed\b', re.I)
+VALIDATE_TOOL = re.compile(r'validat', re.I)
+NEGATION = re.compile(r"\b(not|never|no|nothing|without)\b|n't\b", re.I)
+# A clause about what can or will happen later is not a claim about the run.
+MODAL = re.compile(r"\b(can|can't|cannot|will|won't|could|would|should|once|until|after that|let me know|need)\b", re.I)
+TESTED = re.compile(r'\b(tested|test run|ran|run|runs|executed|simulat\w*|routing checks?|checks? passed)\b', re.I)
+VALIDATED = re.compile(r'\b(validat\w*|valid)\b', re.I)
+PASSED = re.compile(r'\b(pass\w*|succeed\w*|valid|ok|clean)\b', re.I)
+DRAFT = re.compile(r"\b(draft|inactive|unpublished|not (?:yet )?(?:live|active|published)|isn't (?:yet )?(?:live|active)|not turned on)\b", re.I)
+LIVE = re.compile(r'\b(?:is|now) (?:live|active|published)\b', re.I)
+WORKFLOW_ID = re.compile(r'/workflow/([A-Za-z0-9]{8,})|\bID `?([A-Za-z0-9]{16})`?')
+CODE_SPAN = re.compile(r'`([^`\n]{2,80})`')
+INVALID = re.compile(r'"valid"\s*:\s*false|"success"\s*:\s*false', re.I)
+# A test tool can answer without a transport error and still report a failed execution.
+RUN_FAILED = re.compile(r'"status"\s*:\s*"(?:error|failed|crashed|canceled|cancelled)"|"success"\s*:\s*false', re.I)
+
+
+def clauses(message):
+	"""The message split into clauses, with Markdown emphasis removed."""
+	text = re.sub(r'[*_]{1,2}([^*_]+)[*_]{1,2}', r'\1', message or '')
+	# Streamed messages can join sentences with no space, as in "done.Next".
+	return [c.strip() for c in re.split(r'(?<=[.!?:])\s+|(?<=[a-z][.!?])(?=[A-Z])|\n+|;\s+|,\s+but\s+|\s+but\s+', text) if c.strip()]
+
+
+def claim(message, pattern, extra=None):
+	"""(positive, negative) clauses that match pattern. extra must also match a positive clause."""
+	pos, neg = [], []
+	for c in clauses(message):
+		if not pattern.search(c) or MODAL.search(c):
+			continue
+		if NEGATION.search(c):
+			neg.append(c)
+		elif extra is None or extra.search(c):
+			pos.append(c)
+	return pos, neg
+
+
+def tool_text(e):
+	return ' '.join(x if isinstance(x, str) else json.dumps(x, ensure_ascii=False) for x in (e.get('args'), e.get('code'), e.get('result')) if x)
+
+
+def check(id_, ok, text, details):
+	return {'id': id_, 'ok': ok, 'text': text, 'details': details}
+
+
+def quote(clauses_):
+	return [f'Message: "{c}"' for c in clauses_[:3]]
+
+
+def id_check(message, meta, workflow):
+	ids = {a or b for a, b in WORKFLOW_ID.findall(message or '')}
+	if not ids:
+		return None
+	known = set(meta.get('workflowIds') or []) | {meta.get('workflowId'), (workflow or {}).get('id')}
+	wrong = sorted(ids - known)
+	return check('M1', not wrong, 'The workflow id in the message is the workflow that the run created',
+	             [f'{i} is not a workflow of this run' for i in wrong] or [f'{", ".join(sorted(ids))} matches.'])
+
+
+def state_check(message, workflow):
+	draft, live = DRAFT.findall(message or ''), LIVE.findall(message or '')
+	if workflow is None or not (draft or live):
+		return None
+	active = bool(workflow.get('active'))
+	says = 'active' if live and not draft else 'a draft'
+	ok = active == (says == 'active')
+	return check('M2', ok, f'The message says the workflow is {says}', [f'workflow.json active is {str(active).lower()}.'])
+
+
+def validation_check(message, events):
+	pos, _ = claim(message, VALIDATED, PASSED)
+	if not pos:
+		return None
+	calls = [e for e in events if (e['kind'] == 'tool' and VALIDATE_TOOL.search(e.get('name') or ''))
+	         or (e['kind'] == 'code' and VALIDATE_TOOL.search(e.get('code') or ''))]
+	if not calls:
+		return check('M3', False, 'The message says the workflow passed validation', quote(pos) + ['No validation tool call in the trace.'])
+	last = calls[-1]
+	failed = last.get('error') or INVALID.search(str(last.get('result') or ''))
+	return check('M3', not failed, 'The message says the workflow passed validation', quote(pos) + [
+		f'{len(calls)} validation call(s). The last one {"failed" if failed else "passed"}: {last.get("name")}.'])
+
+
+def test_check(message, events):
+	pos, neg = claim(message, TESTED)
+	runs = [e for e in events if e['kind'] == 'tool' and TEST_TOOL.search(e.get('name') or '')
+	        and not VALIDATE_TOOL.search(e.get('name') or '') and not SHELL_TOOL.search(e.get('name') or '')]
+	scripts = [e for e in events if e['kind'] == 'code' and SCRIPT_PASSED.search(str(e.get('result') or ''))]
+	if pos:
+		ok_runs = [e for e in runs if not e.get('error') and not RUN_FAILED.search(tool_text({'result': e.get('result')}))]
+		if ok_runs:
+			names = sorted({e['name'] for e in ok_runs})
+			return check('M4', True, 'The message says the workflow was tested', quote(pos) + [f'{len(ok_runs)} successful test call(s): {", ".join(names)}.'])
+		if scripts:
+			return check('M4', True, 'The message says the workflow was tested', quote(pos) + [
+				f'{len(scripts)} script(s) in the trace report passed tests. These ran outside n8n, not as a workflow execution.'])
+		return check('M4', False, 'The message says the workflow was tested', quote(pos) + ['No test or execution call in the trace.'])
+	if neg and runs:
+		return check('M4', False, 'The message says the workflow was not run', quote(neg) + [f'The trace has {len(runs)} test call(s).'])
+	if neg:
+		return check('M4', True, 'The message says the workflow was not run', quote(neg) + ['No test or execution call in the trace.'])
+	return None
+
+
+def span_check(message, events, workflow, prompt):
+	spans = sorted({s.strip() for s in CODE_SPAN.findall(message or '')})
+	if not spans:
+		return None
+	haystack = '\n'.join([json.dumps(workflow or {}, ensure_ascii=False), prompt] + [tool_text(e) for e in events])
+	loose = lambda s: re.sub(r'\s+', ' ', s).lower()
+	hay = loose(haystack.replace('\\"', '"'))
+	missing = [s for s in spans if loose(s) not in hay]
+	return check('M5', not missing, f'Values in code spans are in the workflow, the prompt or a tool result ({len(spans) - len(missing)} of {len(spans)})',
+	             [f'`{s}` is not in the workflow, the prompt or any tool call' for s in missing])
+
+
+def works_check(message, wire):
+	pos, _ = claim(message, re.compile(TESTED.pattern + '|' + VALIDATED.pattern, re.I), None)
+	if not pos or not wire or wire.get('skipped'):
+		return None
+	failed = [s for s in wire.get('scenarios', []) if s.get('kind') == 'contract'
+	          and (s.get('status') != 'success' or any(c.get('ok') is False for c in s.get('checks') or []))]
+	details = quote(pos) + [f'"{s["name"]}" fails in the service-call run: {s.get("error") or "a check fails"}.' for s in failed]
+	return check('M6', not failed, 'A claim that the workflow was checked agrees with the service-call runs', details)
+
+
+def claims_for(sub, meta, prompt):
+	workflow = json.loads((sub / 'workflow.json').read_text()) if (sub / 'workflow.json').exists() else None
+	wire = json.loads((sub / 'wire.json').read_text()) if (sub / 'wire.json').exists() else None
+	try:
+		message = final_message(sub, meta)
+	except Exception as e:
+		return {'message': None, 'error': f'The final message is not readable: {e}', 'checks': []}
+	data = load_run(sub, meta, {})
+	events = data.get('events') or []
+	checks = [id_check(message, meta, workflow), state_check(message, workflow), validation_check(message, events),
+	          test_check(message, events), span_check(message, events, workflow, prompt), works_check(message, wire)]
+	out = {'message': message, 'checks': [c for c in checks if c]}
+	if data.get('loadError'):
+		out['error'] = f'The trace is not readable: {data["loadError"]}'
+	return out
+
+
+def main():
+	run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
+	prompt = (run_dir / 'prompt.txt').read_text() if (run_dir / 'prompt.txt').exists() else ''
+	for f in sorted(run_dir.glob('*/run.json')):
+		sub = f.parent
+		if sub.name == 'report':
+			continue
+		result = claims_for(sub, json.loads(f.read_text()), prompt)
+		(sub / 'claims.json').write_text(json.dumps(result, indent=1, ensure_ascii=False))
+		ok = sum(1 for c in result['checks'] if c['ok'])
+		print(f'{sub.name}: claims.json written ({ok}/{len(result["checks"])})')
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/compare.py

@@ -0,0 +1,328 @@
+#!/usr/bin/env python3
+"""Run the full comparison in one command, except the judge step.
+
+  compare.py --prompt "..." --expected "..." [--title "..."] [--builders iai,claude,codex]
+  compare.py --prompt-file p.txt --expected-file e.txt [--scenarios-file s.json]
+  compare.py --rerun /tmp/builder-compare/<old-run>     same prompt, expected outcome and scenarios, new run
+
+Steps: check the setup, run the builders at the same time (--sequential: one at a time),
+fetch Instance AI usage, capture the canvases and their node issues, run each workflow with
+pinned data (run_scenarios.py), run each workflow in process with recorded API requests
+(run_wire.mjs), run the eval binary checks (binary_checks.mts), check the final-message claims
+(claims.py), derive the deterministic checks (derive_checks.mjs), write
+summary.txt and judge/summary-blind.txt for the judges, and build the report. A local page
+shows the runs live (see live_server.py).
+Settings come from .env.ai-surfaces (template: .env.ai-surfaces.example in the repository root).
+"""
+
+import argparse
+import json
+import os
+import re
+import shutil
+import subprocess
+import sys
+import threading
+import time
+import webbrowser
+from datetime import datetime
+from pathlib import Path
+
+from live import enabled as live_enabled
+from live import is_on, recording
+from n8n_api import setting
+
+HERE = Path(__file__).resolve().parent
+RUNNERS = {'iai', 'claude', 'codex'}
+PRINT_LOCK = threading.Lock()
+
+
+def slug(text):
+	words = re.findall(r'[a-z0-9]+', text.lower())[:5]
+	return '-'.join(words) or 'run'
+
+
+def step(title):
+	with PRINT_LOCK:
+		print(f'\n=== {title} ===', flush=True)
+
+
+def run(cmd, log=None, prefix=None):
+	"""Run a script, show its output, and keep a copy in log. Returns the exit code."""
+	with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, cwd=HERE) as proc:
+		lines = []
+		for line in proc.stdout:
+			shown = line if not prefix or line.startswith(f'[{prefix}]') else f'[{prefix}] {line}'
+			with PRINT_LOCK:
+				print(shown, end='', flush=True)
+			lines.append(line)
+	if log:
+		Path(log).write_text(''.join(lines))
+	return proc.returncode
+
+
+class Progress:
+	"""Writes status.json, which the live page reads."""
+
+	def __init__(self, run_dir, mode, builders, skipped, screens):
+		self.path, self.lock = run_dir / 'status.json', threading.Lock()
+		self.data = {
+			'mode': mode, 'phase': 'running', 'step': 'Running the builders', 'startedAt': time.time(),
+			'screens': screens,
+			'builders': {**{b: {'status': 'waiting'} for b in builders},
+			             **{b: {'status': 'skipped', 'reason': r} for b, r in skipped.items()}},
+		}
+		self.save()
+
+	def save(self):
+		tmp = self.path.with_suffix('.tmp')
+		tmp.write_text(json.dumps(self.data, indent=2))
+		os.replace(tmp, self.path)
+
+	def update(self, **fields):
+		with self.lock:
+			self.data.update(fields)
+			self.save()
+
+	def builder(self, name, **fields):
+		with self.lock:
+			self.data['builders'][name].update(fields)
+			self.save()
+
+	def finish(self):
+		with self.lock:
+			for entry in self.data['builders'].values():
+				if entry['status'] in ('waiting', 'running'):
+					entry['status'] = 'failed'
+			self.data.update(phase='done', step='Done', finishedAt=time.time())
+			self.save()
+
+
+def start_live_server(run_dir):
+	"""Start live_server.py in the background. Returns its URL, or None."""
+	url_file = run_dir / 'live-url.txt'
+	url_file.unlink(missing_ok=True)
+	log = (run_dir / 'live-server.log').open('w')
+	# A new session, so the page still serves the final report after this script ends.
+	subprocess.Popen([sys.executable, str(HERE / 'live_server.py'), '--run-dir', str(run_dir),
+	                  '--parent-pid', str(os.getpid())],
+	                 cwd=HERE, stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
+	for _ in range(50):
+		if url_file.exists() and url_file.read_text().strip():
+			return url_file.read_text().strip()
+		time.sleep(0.1)
+	return None
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--prompt')
+	ap.add_argument('--prompt-file')
+	ap.add_argument('--expected')
+	ap.add_argument('--expected-file')
+	ap.add_argument('--title')
+	ap.add_argument('--builders', default=setting('COMPARE_BUILDERS', default='iai,claude,codex'))
+	ap.add_argument('--run-dir')
+	ap.add_argument('--scenarios-file', help='Pinned-data scenarios for run_scenarios.py (see SKILL.md).')
+	ap.add_argument('--rerun', help='Copy prompt.txt, case.json and scenarios.json from an earlier run directory.')
+	ap.add_argument('--no-live', action='store_true', help='Do not start the live page.')
+	ap.add_argument('--sequential', action='store_true', default=not is_on('COMPARE_PARALLEL'),
+	                help='Run the builders one at a time. Default: COMPARE_PARALLEL=1 runs them at the same time.')
+	args = ap.parse_args()
+	if args.no_live:
+		os.environ['COMPARE_LIVE'] = '0'
+
+	if args.rerun:
+		old = Path(args.rerun)
+		prompt = (old / 'prompt.txt').read_text().strip()
+		case = json.loads((old / 'case.json').read_text())
+	else:
+		prompt = args.prompt or (Path(args.prompt_file).read_text().strip() if args.prompt_file else None)
+		expected = args.expected or (Path(args.expected_file).read_text().strip() if args.expected_file else None)
+		if not prompt or not expected:
+			sys.exit('Give a prompt and an expected outcome (--prompt/--prompt-file and --expected/--expected-file).')
+		case = {'title': args.title or prompt[:60], 'expected': expected}
+	builders = [b.strip() for b in args.builders.split(',') if b.strip()]
+	unknown = set(builders) - RUNNERS
+	if unknown:
+		sys.exit(f'Unknown builders: {", ".join(sorted(unknown))}. Use iai, claude, codex.')
+
+	root = Path(setting('COMPARE_RUNS_DIR', default='/tmp/builder-compare'))
+	# The child scripts run from HERE, so a relative path must not reach them.
+	run_dir = (Path(args.run_dir) if args.run_dir else root / f'{datetime.now():%Y%m%d-%H%M}-{slug(case["title"])}').resolve()
+	run_dir.mkdir(parents=True, exist_ok=True)
+	(run_dir / 'prompt.txt').write_text(prompt + '\n')
+	(run_dir / 'case.json').write_text(json.dumps(case, indent=2))
+	scenarios = Path(args.scenarios_file) if args.scenarios_file else (Path(args.rerun) / 'scenarios.json' if args.rerun else None)
+	if scenarios and scenarios.exists():
+		json.loads(scenarios.read_text())
+		shutil.copy(scenarios, run_dir / 'scenarios.json')
+	elif args.scenarios_file:
+		sys.exit(f'No scenarios file at {scenarios}.')
+	print(f'run directory: {run_dir}', flush=True)
+
+	step('Check the setup')
+	check = subprocess.run([sys.executable, 'prepare.py'], cwd=HERE, capture_output=True, text=True)
+	print(check.stdout, end='')
+	try:
+		status = json.loads(check.stdout)
+	except json.JSONDecodeError:
+		sys.exit(check.stderr or 'prepare.py failed')
+	if not status.get('login'):
+		sys.exit('Cannot log in to n8n. Is it running? Check N8N_BASE_URL, N8N_EVAL_EMAIL and N8N_EVAL_PASSWORD.')
+	isolation = setting('COMPARE_ISOLATION', default='container')
+	skipped = {}
+	if 'iai' in builders and not status.get('instanceAiEnabled'):
+		skipped['iai'] = 'Instance AI is not enabled on this instance.'
+	for cli in ('claude', 'codex'):
+		if cli in builders and not status.get('mcpTokenWorks'):
+			skipped[cli] = status.get('mcpTokenHelp', 'The MCP token does not work.')
+		elif cli in builders and not status['keys'].get(cli):
+			skipped[cli] = f'No API key. Set {cli.upper()}_API_KEY in .env.ai-surfaces.'
+	if isolation == 'container' and not status.get('docker'):
+		skipped.setdefault('codex', 'Docker is not running. Codex runs only in a container.')
+		if 'claude' in builders and 'claude' not in skipped:
+			print('Docker is not running. Claude falls back to clean-process isolation.', flush=True)
+			isolation = 'clean'
+	if isolation == 'clean':
+		skipped.setdefault('codex', 'Codex runs only in a container. Clean isolation is for Claude only.')
+		if not status.get('claudeOnHost'):
+			skipped.setdefault('claude', 'Clean isolation runs claude on the host, and claude is not on PATH.')
+	skipped = {name: reason for name, reason in skipped.items() if name in builders}
+	for name, reason in skipped.items():
+		print(f'skip {name}: {reason}', flush=True)
+	builders = [b for b in builders if b not in skipped]
+	parallel = not args.sequential and len(builders) > 1
+
+	cli_mode = setting('COMPARE_CLI_MODE', default='interactive')
+	screens = {b: ('shot' if recording() else None) if b == 'iai'
+	           else ('cast' if isolation == 'container' and cli_mode == 'interactive' else None) for b in builders}
+	progress = Progress(run_dir, 'parallel' if parallel else 'sequential', builders, skipped, screens)
+	live_url = None
+	if live_enabled() and not args.no_live:
+		live_url = start_live_server(run_dir)
+		if live_url:
+			print(f'live page: {live_url}/report/live', flush=True)
+			webbrowser.open(f'{live_url}/report/live')
+		else:
+			print(f'the live page did not start, see {run_dir / "live-server.log"}', flush=True)
+
+	results = {}
+	try:
+		if parallel and isolation == 'container' and {'claude', 'codex'} & set(builders):
+			# Both CLI runners need the image. Build it once before they start.
+			from run_cli_builder import ensure_image
+			ensure_image(False)
+
+		def run_builder(builder):
+			progress.builder(builder, status='running', startedAt=time.time())
+			started = time.time()
+			if builder == 'iai':
+				cmd = [sys.executable, 'run_instance_ai.py', '--out', str(run_dir / 'iai'),
+				       '--prompt-file', str(run_dir / 'prompt.txt')]
+			else:
+				cmd = [sys.executable, 'run_cli_builder.py', '--builder', builder, '--out', str(run_dir / builder),
+				       '--prompt-file', str(run_dir / 'prompt.txt'), '--isolation', isolation]
+			code = run(cmd, run_dir / f'{builder}.log', prefix=builder if parallel else None)
+			# run_cli_builder.py exits 0 when the CLI fails. run.json has the exit code of the CLI.
+			meta_file = run_dir / builder / 'run.json'
+			if code == 0 and meta_file.exists():
+				code = json.loads(meta_file.read_text()).get('exitCode') or 0
+			results[builder] = {'exitCode': code, 'seconds': round(time.time() - started)}
+			progress.builder(builder, status='done' if code == 0 else 'failed', finishedAt=time.time(), exitCode=code)
+
+		if parallel:
+			step(f'Run {", ".join(builders)} at the same time')
+			threads = [threading.Thread(target=run_builder, args=(b,)) for b in builders]
+			for t in threads:
+				t.start()
+			for t in threads:
+				t.join()
+		else:
+			for builder in builders:
+				step(f'Run {builder}')
+				run_builder(builder)
+
+		if 'iai' in builders and (run_dir / 'iai' / 'run.json').exists():
+			step('Fetch Instance AI usage from LangSmith')
+			progress.update(step='Fetching Instance AI usage')
+			if status['keys'].get('langsmith'):
+				# LangSmith receives traces with a delay.
+				finished = progress.data['builders']['iai'].get('finishedAt') or time.time()
+				time.sleep(max(0.0, 30 - (time.time() - finished)))
+				run([sys.executable, 'langsmith_usage.py', '--run-dir', str(run_dir / 'iai')])
+			else:
+				print('LANGSMITH_API_KEY is not set. The report has no Instance AI cost or tokens.', flush=True)
+
+		step('Capture the canvases')
+		progress.update(step='Capturing the canvases')
+		node = shutil.which('node')
+		if node:
+			run([node, 'capture_canvas.mjs', str(run_dir)])
+		else:
+			print('node is not on PATH. No canvases.', flush=True)
+
+		if not is_on('COMPARE_SCENARIOS'):
+			print('COMPARE_SCENARIOS=0. The workflows do not run with pinned data.', flush=True)
+		elif not node or not status.get('mcpTokenWorks'):
+			print('Pinned-data runs need node and a working MCP token. Skipped.', flush=True)
+		else:
+			step('Run each workflow with pinned data')
+			progress.update(step='Running the workflows with pinned data')
+			run([sys.executable, 'run_scenarios.py', str(run_dir)], run_dir / 'scenarios.log')
+
+		if node and is_on('COMPARE_WIRE'):
+			step('Run each workflow in process with recorded API requests')
+			progress.update(step='Running the workflows with recorded API requests')
+			run([node, 'run_wire.mjs', str(run_dir)], run_dir / 'wire.log')
+		elif node:
+			print('COMPARE_WIRE=0. No in-process runs with recorded API requests.', flush=True)
+
+		pnpm = shutil.which('pnpm')
+		if pnpm and is_on('COMPARE_BINARY_CHECKS'):
+			step('Run the eval binary checks')
+			progress.update(step='Running the eval binary checks')
+			run([pnpm, '--filter', '@n8n/instance-ai', 'exec', 'tsx', str(HERE / 'binary_checks.mts'), str(run_dir)],
+			    run_dir / 'binary-checks.log')
+
+		step('Check the final-message claims')
+		run([sys.executable, 'claims.py', str(run_dir)])
+
+		if node:
+			step('Derive the deterministic checks')
+			progress.update(step='Deriving the deterministic checks')
+			run([node, 'derive_checks.mjs', str(run_dir)])
+
+		step('Summarize the runs for the judges')
+		summary = subprocess.run([sys.executable, 'summarize_runs.py', str(run_dir)], cwd=HERE,
+		                         capture_output=True, text=True)
+		(run_dir / 'summary.txt').write_text(summary.stdout)
+		run([sys.executable, 'summarize_runs.py', str(run_dir), '--blind'])
+
+		step('Build the report')
+		# The agent does the price and judge steps after this script. The page shows them as queued until then.
+		progress.update(step='Building the report', postRun={'prices': 'queued', 'judge': 'queued'})
+		run([sys.executable, 'build_report.py', str(run_dir)])
+	finally:
+		progress.finish()
+
+	models_file = run_dir / 'models.json'
+	print(json.dumps({
+		'runDir': str(run_dir),
+		'report': str(run_dir / 'report' / 'index.html'),
+		'live': f'{live_url}/report/index.html' if live_url else None,
+		'parallel': parallel,
+		'summary': str(run_dir / 'summary.txt'),
+		'blindSummary': str(run_dir / 'judge' / 'summary-blind.txt'),
+		'models': json.loads(models_file.read_text()) if models_file.exists() else [],
+		'results': results,
+		'skipped': skipped,
+		'next': 'Run build_report.py --step prices=running, add published prices for the CLI models to case.json, '
+		        'run build_report.py --step prices=done --step judge=running, give judge/summary-blind.txt to '
+		        '3 judges with a fresh context, write judge/verdicts-1.json to verdicts-3.json, run '
+		        'merge_verdicts.py, then run build_report.py --step judge=done.',
+	}, indent=2))
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/derive_checks.mjs

@@ -0,0 +1,743 @@
+#!/usr/bin/env node
+// Deterministic checks for each workflow that a run created. No LLM is involved.
+//
+//   node derive_checks.mjs <run-dir>
+//
+// Reads <builder>/workflow.json, canvas-issues.json and exec.json, and the optional wire.json,
+// claims.json and binary-checks.json. Writes <builder>/auto-checks.json. A group with gating true
+// changes the verdict. The other groups show a separate score.
+//   correctness  node definitions, canvas issues, graph and expression references (gating)
+//   execution    the pinned-data scenarios of run_scenarios.py (gating)
+//   contract     service-call requests of run_wire.mjs (gating)
+//   edge, ai     edge-case and real-AI scenarios of run_wire.mjs
+//   claims       the final message against the trace, from claims.py
+//   standards    rules from n8n-io/skills and the workflow-sdk best practices
+//   binary       the deterministic eval binary checks, from binary_checks.mts
+import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+import {
+	LOGIC_TYPES,
+	SINGLE_ITEM_TYPES,
+	TRIGGER_FIXTURES,
+	downstream,
+	graph,
+	isManualTrigger,
+	isTrigger,
+	loadSdk,
+	lookup,
+	reachable,
+	repo,
+	sample,
+	shortType,
+	showPath,
+	strings,
+	typeMatches,
+	workflowReferences,
+} from './sdk.mjs';
+
+const VERSION = 1;
+const SKILLS = 'https://github.com/n8n-io/skills/blob/27b6b9f429ac0be90cae764512dbc579d7107e95/';
+const GUIDES = 'packages/@n8n/workflow-sdk/src/prompts/best-practices/guides/';
+const LC = '@n8n/n8n-nodes-langchain.';
+
+const src = {
+	validator: { label: '@n8n/workflow-sdk validateWorkflow with the node definitions' },
+	canvas: { label: 'Node issues in the n8n editor (canvas status icons)' },
+	graph: { label: 'Workflow connections' },
+	expressions: { label: 'Expression references in node parameters' },
+	execution: { label: 'Real output of the pinned-data runs (exec.json)' },
+	wire: { label: 'In-process run with recorded API fixtures (wire.json)' },
+	claims: { label: 'Final message, trace tool calls and workflow (claims.json)' },
+	errorHandling: { label: 'n8n-error-handling-official', url: `${SKILLS}skills/n8n-error-handling-official/SKILL.md` },
+	expressionsSkill: { label: 'n8n-expressions-official', url: `${SKILLS}skills/n8n-expressions-official/SKILL.md` },
+	structured: {
+		label: 'n8n-ai-nodes-official STRUCTURED_OUTPUT.md',
+		url: `${SKILLS}skills/n8n-ai-nodes-official/references/STRUCTURED_OUTPUT.md`,
+	},
+	memory: { label: 'n8n-ai-nodes-official MEMORY.md', url: `${SKILLS}skills/n8n-ai-nodes-official/references/MEMORY.md` },
+	comms: {
+		label: 'n8n-node-configuration-official COMMS_NODES.md',
+		url: `${SKILLS}skills/n8n-node-configuration-official/references/COMMS_NODES.md`,
+	},
+	triage: { label: 'workflow-sdk best practices: triage', path: `${GUIDES}triage.ts` },
+	chatbot: { label: 'workflow-sdk best practices: chatbot', path: `${GUIDES}chatbot.ts` },
+	multiItem: {
+		label: 'WhatsAppTrigger.node.ts emits one item for each webhook change',
+		path: 'packages/nodes-base/nodes/WhatsApp/WhatsAppTrigger.node.ts',
+	},
+};
+
+const check = (id, ok, text, source, details = [], extra = {}) => ({ id, ok, text, source, details, ...extra });
+const plural = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`;
+const readJson = (file) => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
+
+// --- Node kinds ----------------------------------------------------------------
+
+const isAiRoot = (g, node) => node.type.startsWith(LC) && !g.subnodes.has(node.name) && g.aiInputs(node.name, 'ai_languageModel').length > 0;
+const isBranch = (node) => typeMatches(node.type, 'if') || typeMatches(node.type, 'switch');
+/** Nodes that call a service: not logic, not a trigger, not a sub-node. AI root nodes count. */
+const isNetwork = (g, node) =>
+	!g.subnodes.has(node.name) && !isTrigger(node.type) && !LOGIC_TYPES.has(node.type) && !typeMatches(node.type, 'stickyNote');
+
+// Triggers that start one execution with one item. Other triggers can batch several events.
+const SINGLE_ITEM_TRIGGERS = ['manualTrigger', 'scheduleTrigger', 'cron', 'interval', 'webhook', 'chatTrigger', 'formTrigger', 'telegramTrigger', 'slackTrigger'];
+const MESSAGING_TRIGGERS = ['whatsAppTrigger', 'telegramTrigger', 'slackTrigger', 'chatTrigger', 'discordTrigger', 'microsoftTeamsTrigger', 'gmailTrigger'];
+const ITEM_SOURCES = ['splitOut', 'code', 'itemLists'];
+
+/** Names of the nodes that can output more than one item in one execution. */
+function multiItemNodes(g) {
+	const multi = new Set();
+	const order = [...reachable(g)];
+	let grew = true;
+	while (grew) {
+		grew = false;
+		for (const name of order) {
+			if (multi.has(name)) continue;
+			const node = g.byName.get(name);
+			if (!node || g.subnodes.has(name) || SINGLE_ITEM_TYPES.has(node.type) || node.executeOnce) continue;
+			const fromTrigger = isTrigger(node.type) && !SINGLE_ITEM_TRIGGERS.some((t) => typeMatches(node.type, t));
+			const source = ITEM_SOURCES.some((t) => typeMatches(node.type, t)) || (isNetwork(g, node) && !isAiRoot(g, node) && /^(read|getAll|search|list|getMany)$/.test(node.parameters?.operation ?? ''));
+			if (fromTrigger || source || g.mainParents(name).some((p) => multi.has(p))) {
+				multi.add(name);
+				grew = true;
+			}
+		}
+	}
+	return multi;
+}
+
+/** Names of the nodes upstream of name. A sub-node uses the nodes upstream of the root that it serves. */
+function upstream(g, name) {
+	const roots = new Set([name]);
+	for (let grew = true; grew; ) {
+		grew = false;
+		for (const e of g.edges) {
+			if (e.type.startsWith('ai_') && roots.has(e.from) && !roots.has(e.to)) {
+				roots.add(e.to);
+				grew = true;
+			}
+		}
+	}
+	const seen = new Set();
+	const queue = [...roots];
+	while (queue.length) {
+		for (const parent of g.mainParents(queue.shift())) {
+			if (!seen.has(parent)) {
+				seen.add(parent);
+				queue.push(parent);
+			}
+		}
+	}
+	return seen;
+}
+
+// --- Correctness ---------------------------------------------------------------
+
+function checkDefinitions(workflow) {
+	const loaded = loadSdk();
+	if (!loaded || loaded.missingDirs.length) {
+		const missing = loaded ? loaded.missingDirs.map((d) => d.replace(repo + '/', '')) : ['packages/@n8n/workflow-sdk/dist'];
+		return check('C1', null, 'Node parameters match the node definitions', src.validator, [
+			`Not checked. Build these first: ${missing.join(', ')}.`,
+		]);
+	}
+	const result = loaded.sdk.validateWorkflow(workflow, { allowDisconnectedNodes: true, allowNoTrigger: true });
+	const problems = [...result.errors, ...result.warnings.filter((w) => w.severity !== 'informational')];
+	const details = problems.map((p) =>
+		p.nodeName && !p.message.includes(`"${p.nodeName}"`) ? `${p.nodeName}: ${p.message}` : p.message,
+	);
+	const info = result.warnings.filter((w) => w.severity === 'informational').map((w) => `Info: ${w.message}`);
+	return check(
+		'C1',
+		problems.length === 0,
+		problems.length ? `${plural(problems.length, 'node parameter problem')} against the node definitions` : 'Node parameters match the node definitions',
+		src.validator,
+		details.concat(info),
+	);
+}
+
+const isEmpty = (value) =>
+	value === undefined || value === null || value === '' || (typeof value === 'object' && value.__rl && (value.value === '' || value.value == null));
+
+/** Sort the editor issues into setup work for the user and real problems. */
+function classifyCanvas(workflow, canvas) {
+	const byName = new Map((workflow.nodes ?? []).map((n) => [n.name, n]));
+	const setup = [];
+	const problems = [];
+	for (const node of canvas.nodes) {
+		if (node.marker === 'node-not-installed') problems.push(`${node.name}: the node type is not installed`);
+		if (node.marker === 'node-restricted') problems.push(`${node.name}: the node type is restricted`);
+		for (const [kind, issue] of Object.entries(node.issues ?? {})) {
+			if (kind === 'credentials') {
+				for (const type of Object.keys(issue)) setup.push(`${node.name}: connect ${type} credentials`);
+			} else if (kind === 'parameters') {
+				for (const [param, messages] of Object.entries(issue)) {
+					const value = byName.get(node.name)?.parameters?.[param];
+					for (const message of messages) {
+						if (/is required/i.test(message) && isEmpty(value)) setup.push(`${node.name}: ${message.replace(/\.$/, '')} (left for the user)`);
+						else problems.push(`${node.name}: ${message}`);
+					}
+				}
+			} else if (issue === true || (issue && typeof issue === 'object' && Object.keys(issue).length)) {
+				const text = typeof issue === 'object' ? Object.values(issue).flat().join(' ') : '';
+				problems.push(`${node.name}: ${kind === 'typeUnknown' ? 'n8n does not know this node type' : `${kind} issue`}${text ? ` (${text})` : ''}`);
+			}
+		}
+	}
+	return { setup, problems };
+}
+
+function checkCanvas(workflow, canvas) {
+	if (!canvas) return [check('C2', null, 'The n8n editor shows no node errors', src.canvas, ['Not checked. Run capture_canvas.mjs first.'])];
+	if (!canvas.store && canvas.nodes.some((n) => n.marker === 'node-issues')) {
+		return [check('C2', null, 'The n8n editor shows no node errors', src.canvas, [
+			'The editor store was not readable, so the node issues are not known. Icons: ' +
+				canvas.nodes.map((n) => `${n.name} (${n.marker})`).join(', '),
+		])];
+	}
+	const { setup, problems } = classifyCanvas(workflow, canvas);
+	return [
+		check('C2', problems.length === 0, problems.length ? `The n8n editor shows ${plural(problems.length, 'node error')}` : 'The n8n editor shows no node errors', src.canvas, problems),
+		check('C2s', true, setup.length ? `Setup left for the user: ${plural(setup.length, 'item')}` : 'No setup left for the user', src.canvas, setup, { level: 'info' }),
+	];
+}
+
+function checkReachable(g) {
+	if (g.triggers.length === 0) return check('C3', false, 'The workflow has a trigger', src.graph, ['No trigger node.']);
+	const seen = reachable(g);
+	const orphans = g.nodes.filter((n) => !seen.has(n.name)).map((n) => n.name);
+	return check(
+		'C3',
+		orphans.length === 0,
+		orphans.length ? `${plural(orphans.length, 'node')} cannot be reached from a trigger` : 'Every node can be reached from a trigger',
+		src.graph,
+		orphans.map((n) => `${n} is not connected to the flow`),
+	);
+}
+
+function checkNamedRefs(workflow, g, refs) {
+	const allNames = new Set((workflow.nodes ?? []).map((n) => n.name));
+	const problems = [];
+	for (const { node, param, ref } of refs) {
+		if (ref.kind !== 'named') continue;
+		if (!allNames.has(ref.node)) problems.push(`${node} › ${param}: no node is called "${ref.node}"`);
+		else if (!g.byName.has(ref.node)) problems.push(`${node} › ${param}: "${ref.node}" is disabled`);
+		else if (!/isExecuted/.test(ref.raw) && !upstream(g, node).has(ref.node))
+			problems.push(`${node} › ${param}: "${ref.node}" does not run before this node, so the reference has no data`);
+	}
+	const named = refs.filter((r) => r.ref.kind === 'named').length;
+	return check(
+		'C4',
+		problems.length === 0,
+		problems.length ? `${plural(problems.length, 'node reference')} cannot resolve` : `All ${plural(named, 'node reference')} name a node that runs before them`,
+		src.expressions,
+		[...new Set(problems)],
+	);
+}
+
+const DEFINITIVE = new Set(['executed', 'fixture', 'case', 'output-parser']);
+const INDICATIVE = new Set(['node-schema', 'prompt-example', 'ai-root']);
+// Outputs with the real number of array items. The other provenances are schema samples.
+const REAL_OUTPUT = new Set(['executed', 'fixture', 'case']);
+
+/** The output items of a node across all scenarios, with their provenance. */
+function outputsOf(name, g, exec) {
+	const items = [];
+	for (const scenario of exec?.scenarios ?? []) {
+		const out = scenario.nodes?.[name];
+		if (out?.status === 'success' && Array.isArray(out.items)) items.push(...out.items.filter((i) => i && typeof i === 'object'));
+	}
+	const node = g.byName.get(name);
+	let provenance = exec?.executed?.includes(name) ? 'executed' : exec?.pinned?.[name];
+	if (items.length && provenance) return { items, provenance };
+	if (TRIGGER_FIXTURES[node.type]) return { items: TRIGGER_FIXTURES[node.type], provenance: 'fixture' };
+	const schema = loadSdk()?.sdk.discoverOutputSchemaForNode(node.type, node.typeVersion, node.parameters ?? {});
+	if (schema) return { items: [sample(schema, 'item')], provenance: 'node-schema' };
+	return { items: [], provenance: 'unknown' };
+}
+
+function checkFieldPaths(g, refs, exec) {
+	const missing = [];
+	const unconfirmed = [];
+	let checked = 0;
+	let unknown = 0;
+	const seen = new Set();
+	for (const { node, param, ref, code } of refs) {
+		if (code || !ref.path?.length) continue;
+		if (ref.kind !== 'named' && g.subnodes.has(node)) continue;
+		const sources = ref.kind === 'named' ? [ref.node] : g.mainParents(node);
+		if (!sources.length || sources.some((s) => !g.byName.has(s))) continue;
+		const key = `${node}|${param}|${ref.raw}`;
+		if (seen.has(key)) continue;
+		seen.add(key);
+		let found = false;
+		let anyUnknown = false;
+		const misses = [];
+		for (const source of sources) {
+			const { items, provenance } = outputsOf(source, g, exec);
+			if (!items.length || provenance === 'unknown' || provenance === 'empty' || provenance === 'placeholder') {
+				anyUnknown = true;
+				continue;
+			}
+			const results = items.map((item) => lookup(item, ref.path, { sample: !REAL_OUTPUT.has(provenance) }));
+			if (results.includes('found')) found = true;
+			else if (results.includes('unknown')) anyUnknown = true;
+			else misses.push({ source, provenance });
+		}
+		if (found) checked++;
+		else if (anyUnknown || !misses.length) unknown++;
+		else {
+			checked++;
+			const line = `${node} › ${param}: ${ref.raw.trim()} reads "${showPath(ref.path)}", which is not in the output of ${misses.map((m) => `${m.source} (${m.provenance})`).join(' or ')}`;
+			if (misses.some((m) => DEFINITIVE.has(m.provenance))) missing.push(line);
+			else if (misses.some((m) => INDICATIVE.has(m.provenance))) unconfirmed.push(`Unconfirmed: ${line}`);
+		}
+	}
+	const note = exec?.scenarios?.length ? [] : ['No pinned-data run, so node outputs come from fixtures and node schemas only.'];
+	return check(
+		'C5',
+		missing.length === 0,
+		missing.length
+			? `${plural(missing.length, 'expression')} read a field that the source node does not output`
+			: `${plural(checked, 'expression field')} found in the source node output${unknown ? `, ${unknown} not checkable` : ''}`,
+		src.execution,
+		[...missing, ...unconfirmed, ...note],
+	);
+}
+
+const SECRET_PATTERNS = [
+	/sk-(?:proj-|ant-)?[A-Za-z0-9_-]{20,}/,
+	/xox[abpr]-[A-Za-z0-9-]{10,}/,
+	/AIza[0-9A-Za-z_-]{35}/,
+	/gh[pousr]_[A-Za-z0-9]{36}/,
+	/Bearer\s+(?!\{\{)[A-Za-z0-9._~+/-]{24,}/,
+	/-----BEGIN [A-Z ]*PRIVATE KEY-----/,
+];
+
+function checkSecrets(g) {
+	const hits = [];
+	for (const node of g.nodes) {
+		for (const { path, value } of strings(node.parameters ?? {})) {
+			if (SECRET_PATTERNS.some((re) => re.test(value))) hits.push(`${node.name} › ${path}: a value looks like a key or token`);
+		}
+	}
+	return check('C6', hits.length === 0, hits.length ? 'Secrets are written into node parameters' : 'No secrets in node parameters', src.expressions, hits);
+}
+
+// --- Execution -----------------------------------------------------------------
+
+function executionChecks(exec) {
+	if (!exec) return [check('E0', null, 'The workflow runs with pinned data', src.execution, ['Not run. Run run_scenarios.py first.'])];
+	if (exec.skipped) return [check('E0', null, 'The workflow runs with pinned data', src.execution, [exec.skipped])];
+	const pinned = Object.entries(exec.pinned ?? {}).map(([n, p]) => `${n} (${p})`);
+	const intro = [`Pinned: ${pinned.join(', ') || 'nothing'}.`, `Ran for real: ${(exec.executed ?? []).join(', ') || 'nothing'}.`];
+	const out = [];
+	exec.scenarios.forEach((s, i) => {
+		const errors = Object.entries(s.nodes ?? {}).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
+		const ran = Object.keys(s.nodes ?? {});
+		const ok = s.status === 'success' && errors.length === 0;
+		out.push(check(`E${i + 1}`, ok, `"${s.name}" runs without an error`, src.execution, [
+			...(i === 0 ? intro : []),
+			`Status: ${s.status}${s.error ? ` (${s.error})` : ''}.`,
+			...errors,
+			`Nodes that ran: ${ran.join(', ')}.`,
+		], { executionId: s.executionId }));
+		(s.checks ?? []).forEach((c, j) => {
+			const verb = c.kind === 'reach' ? 'reaches' : 'does not reach';
+			out.push(check(`E${i + 1}.${j + 1}`, c.ok, `"${s.name}" ${verb} a ${c.type} node`, src.execution, []));
+		});
+	});
+	return out;
+}
+
+// --- In-process runs, eval checks and claims -------------------------------------
+
+const WIRE_KINDS = {
+	contract: { id: 'W', title: 'Service calls', gating: true, note: 'The workflow runs in process. Service nodes run for real against recorded API fixtures, so the check sees each HTTP request that a node sends. The AI node is pinned. A failure here changes the verdict.' },
+	edge: { id: 'X', title: 'Edge cases', gating: false, note: 'Inputs that the prompt does not describe but the trigger can send: several messages in one call, status updates, empty data. This score does not change the verdict.' },
+	ai: { id: 'A', title: 'Real AI runs', gating: false, note: 'The AI node calls the real model on labeled questions. Service nodes use the recorded API fixtures. Model output can change between runs, so this score does not change the verdict.' },
+};
+
+function wireCheckText(name, c) {
+	if (c.kind === 'reach') return `"${name}" reaches a ${c.type} node`;
+	if (c.kind === 'skip') return `"${name}" does not reach a ${c.type} node`;
+	if (c.kind === 'noWrites') return `"${name}" sends no write request`;
+	if (c.kind === 'eachItem') return `"${name}" sends a request for each of the ${c.ids.length} items`;
+	const what = c.contains?.length ? ` that carries ${c.contains.map((m) => m.split(' or ').map((x) => (x.startsWith('@') ? `\`${x}\`` : x)).join(' or ')).join(', ')}` : '';
+	const count = c.count === undefined ? 'a request' : plural(c.count, 'request');
+	return `"${name}" sends ${count} from a ${c.type} node${what}`;
+}
+
+function requestLines(s) {
+	return (s.requests ?? []).map((r) => `Request: ${r.node} › ${r.method} ${r.url}${r.answeredBy && r.answeredBy !== 'fixture' ? ` (${r.answeredBy})` : ''}`);
+}
+
+function wireGroups(wire) {
+	if (!wire) return [];
+	const groups = [];
+	for (const [kind, meta] of Object.entries(WIRE_KINDS)) {
+		const scenarios = (wire.scenarios ?? []).filter((s) => s.kind === kind);
+		if (!scenarios.length && !(kind === 'contract' && wire.skipped)) continue;
+		const checks = [];
+		if (wire.skipped) checks.push(check(`${meta.id}0`, null, 'The workflow runs in process', src.wire, [wire.skipped]));
+		scenarios.forEach((s, i) => {
+			const id = `${meta.id}${i + 1}`;
+			if (s.status === 'skipped') {
+				checks.push(check(id, null, `"${s.name}" runs`, src.wire, [s.error]));
+				return;
+			}
+			const errors = Object.entries(s.nodes ?? {}).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
+			const ai = Object.entries(s.aiOutputs ?? {}).map(([node, items]) => `AI output of ${node}: ${JSON.stringify(items?.[0]?.output ?? items?.[0] ?? null)}`);
+			const ran = Object.entries(s.nodes ?? {}).map(([name, n]) => `${name} (${plural((n.outputCounts ?? []).reduce((a, b) => a + b, 0), 'item')})`);
+			checks.push(check(id, s.status === 'success' && !errors.length, `"${s.name}" runs without an error`, src.wire, [
+				...(s.why ? [s.why[0].toUpperCase() + s.why.slice(1)] : []),
+				`Status: ${s.status}${s.error ? ` (${s.error})` : ''}.`,
+				...errors,
+				`Nodes that ran: ${ran.join(', ') || 'none'}.`,
+				...ai,
+				...requestLines(s),
+				...(s.blocked?.length ? [`Blocked network calls: ${s.blocked.join(', ')}`] : []),
+			]));
+			(s.checks ?? []).forEach((c, j) => checks.push(check(`${id}.${j + 1}`, c.ok, wireCheckText(s.name, c), src.wire, c.details ?? [])));
+		});
+		const scored = checks.filter((c) => c.ok !== null);
+		const filled = kind === 'contract' && wire.filled?.length ? [`Setup values filled for the run: ${wire.filled.join(', ')}.`] : [];
+		groups.push({
+			key: kind === 'contract' ? 'contract' : kind,
+			title: meta.title,
+			gating: meta.gating,
+			note: [meta.note, ...filled].join(' '),
+			...(meta.gating ? {} : { score: { passed: scored.filter((c) => c.ok).length, total: scored.length } }),
+			checks,
+		});
+	}
+	return groups;
+}
+
+function binaryGroup(binary) {
+	if (!binary?.outcomes) return null;
+	const measured = binary.outcomes.filter((o) => o.status !== 'n_a');
+	const checks = measured.map((o, i) =>
+		check(`B${i + 1}`, o.status === 'error' ? null : o.status === 'pass', `${o.name}: ${o.description}`, { label: `Instance AI eval binary checks (${o.dimension})`, path: binary.source }, o.comment ? [o.comment] : []),
+	);
+	const scored = checks.filter((c) => c.ok !== null);
+	const na = binary.outcomes.length - measured.length;
+	return {
+		key: 'binary',
+		title: 'Eval binary checks',
+		gating: false,
+		note: `The deterministic binary checks of the Instance AI eval suite.${na ? ` ${na} do not apply to this workflow.` : ''} This score does not change the verdict.`,
+		score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
+		checks,
+	};
+}
+
+function claimsGroup(claims) {
+	if (!claims) return null;
+	const checks = claims.checks.map((c) => check(c.id, c.ok, c.text, src.claims, c.details));
+	if (claims.error) checks.unshift(check('M0', null, 'The final message and trace are readable', src.claims, [claims.error]));
+	const scored = checks.filter((c) => c.ok !== null);
+	return {
+		key: 'claims',
+		title: 'Final message claims',
+		gating: false,
+		note: 'What the final message says about the build, compared with the trace tool calls, the workflow and the service-call runs. This score does not change the verdict.',
+		score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
+		checks,
+	};
+}
+
+// --- Standards -----------------------------------------------------------------
+
+function unattended(g) {
+	return g.triggers.some((t) => !isManualTrigger(t.type) && !typeMatches(t.type, 'chatTrigger') && !typeMatches(t.type, 'formTrigger'));
+}
+
+function errorWorkflowRule(workflow, g) {
+	if (!unattended(g)) return null;
+	const set = Boolean(workflow.settings?.errorWorkflow);
+	return check('S1', set, 'An unattended workflow has an error workflow', src.errorHandling, set ? [] : ['settings.errorWorkflow is not set, so a failure goes nowhere.']);
+}
+
+function errorOutputRule(g) {
+	const problems = [];
+	for (const node of g.nodes) {
+		if (node.onError === 'continueRegularOutput') problems.push(`${node.name}: onError continueRegularOutput passes the error on as normal data`);
+		// The error output comes after the normal outputs, so it is never output 0.
+		if (node.onError === 'continueErrorOutput' && !g.mainChildren(node.name).some((e) => e.output > 0))
+			problems.push(`${node.name}: onError continueErrorOutput has no node on its error output`);
+	}
+	if (!g.nodes.some((n) => n.onError)) return null;
+	return check('S12', problems.length === 0, 'Failed nodes go to a wired error output, not on as data', src.errorHandling, problems);
+}
+
+function retryRule(g) {
+	const nodes = g.nodes.filter((n) => isNetwork(g, n));
+	if (!nodes.length) return null;
+	const missing = nodes.filter((n) => !n.retryOnFail).map((n) => `${n.name}: no retryOnFail`);
+	return check('S2', missing.length === 0, `Nodes that call a service retry on failure (${nodes.length - missing.length} of ${nodes.length})`, src.errorHandling, missing);
+}
+
+function namedRefRule(refs) {
+	const uses = refs.filter((r) => !r.code && (r.ref.kind === 'json' || r.ref.kind === 'input'));
+	if (!refs.some((r) => !r.code)) return null;
+	const where = [...new Set(uses.map((r) => `${r.node} › ${r.param}`))];
+	return check('S3', uses.length === 0, uses.length ? `${plural(where.length, 'parameter')} read $json instead of a node name` : 'Expressions read data by node name', src.expressionsSkill, where);
+}
+
+function parserParams(g, root) {
+	const parser = g.aiInputs(root.name, 'ai_outputParser')[0];
+	return parser ? { parser, p: parser.parameters ?? {} } : null;
+}
+
+function structuredRule(g) {
+	const roots = g.nodes.filter((n) => isAiRoot(g, n) && g.mainChildren(n.name).length);
+	if (!roots.length) return null;
+	const problems = [];
+	for (const root of roots) {
+		const found = parserParams(g, root);
+		if (!found) {
+			const declared = strings(root.parameters ?? {}).some(({ value }) => /json/i.test(value));
+			if (declared) problems.push(`${root.name}: the JSON format is only in the prompt, with no structured output parser`);
+			continue;
+		}
+		if ((found.p.schemaType ?? 'fromJson') !== 'manual') problems.push(`${found.parser.name}: schemaType is not 'manual'`);
+		if (!found.p.autoFix) problems.push(`${found.parser.name}: autoFix is off`);
+	}
+	return check('S4', problems.length === 0, 'Structured AI output uses a manual schema with autoFix', src.structured, problems);
+}
+
+const ATTRIBUTION_FIELDS = ['appendAttribution', 'includeLinkToWorkflow'];
+const definitionCache = new Map();
+
+/** The text of the node definition file for a node, or ''. */
+function definitionText(node) {
+	const [pkg, short] = node.type.startsWith(LC) ? ['@n8n/nodes-langchain', node.type.slice(LC.length)] : ['nodes-base', shortType(node.type)];
+	const pkgDir = node.type.startsWith(LC) ? '@n8n/n8n-nodes-langchain' : 'n8n-nodes-base';
+	const dir = join(repo, 'packages', pkg, 'dist/node-definitions/nodes', pkgDir, short, `v${String(node.typeVersion ?? 1).replace('.', '')}`);
+	const snake = (s) => String(s ?? '').replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase();
+	const p = node.parameters ?? {};
+	const files = [join(dir, `resource_${snake(p.resource)}`, `operation_${snake(p.operation)}.ts`), `${dir}.ts`];
+	if (!p.resource && p.operation && existsSync(dir)) {
+		for (const entry of readdirSync(dir)) if (entry.startsWith('resource_')) files.push(join(dir, entry, `operation_${snake(p.operation)}.ts`));
+	}
+	const key = files.join('|');
+	if (!definitionCache.has(key)) definitionCache.set(key, files.filter(existsSync).map((f) => readFileSync(f, 'utf8')).join('\n'));
+	return definitionCache.get(key);
+}
+
+function findKey(value, key) {
+	if (!value || typeof value !== 'object') return undefined;
+	if (key in value) return value[key];
+	for (const v of Object.values(value)) {
+		const hit = findKey(v, key);
+		if (hit !== undefined) return hit;
+	}
+	return undefined;
+}
+
+function attributionRule(g) {
+	const nodes = [];
+	for (const node of g.nodes) {
+		const text = definitionText(node);
+		const field = ATTRIBUTION_FIELDS.find((f) => text.includes(`${f}?:`));
+		if (field) nodes.push({ node, field });
+	}
+	if (!nodes.length) return null;
+	const on = nodes.filter(({ node, field }) => findKey(node.parameters, field) !== false).map(({ node, field }) => `${node.name}: ${field} is not false (the default adds an n8n attribution)`);
+	return check('S5', on.length === 0, 'Messages do not carry the n8n attribution', src.comms, on);
+}
+
+function multiItemRule(g, refs) {
+	const multi = multiItemNodes(g);
+	if (!multi.size) return null;
+	const hits = refs.filter((r) => r.ref.kind === 'named' && ['first', 'last', 'all-index'].includes(r.ref.accessor) && multi.has(r.ref.node));
+	const where = [...new Set(hits.map((r) => `${r.node} › ${r.param}: ${r.ref.raw.trim()}`))];
+	const triggers = g.triggers.filter((t) => multi.has(t.name));
+	const source = triggers.some((t) => typeMatches(t.type, 'whatsAppTrigger')) ? { ...src.multiItem, also: src.expressionsSkill } : src.expressionsSkill;
+	return check(
+		'S6',
+		where.length === 0,
+		where.length ? `${plural(where.length, 'reference')} read only the first item of a node that can output several` : 'References keep each item paired with its own data',
+		source,
+		where.length ? [...where, `Nodes that can output several items: ${[...multi].join(', ')}.`] : [],
+	);
+}
+
+/** AI root nodes whose output decides a branch, with the IF or Switch nodes that they feed. */
+function decidingRoots(g) {
+	return g.nodes
+		.filter((n) => isAiRoot(g, n))
+		.map((root) => ({ root, branches: [...downstream(g, root.name)].map((n) => g.byName.get(n)).filter((n) => n && isBranch(n)) }))
+		.filter((r) => r.branches.length);
+}
+
+function temperatureRule(g) {
+	const roots = decidingRoots(g);
+	if (!roots.length) return null;
+	const problems = [];
+	for (const { root } of roots) {
+		for (const model of g.aiInputs(root.name, 'ai_languageModel')) {
+			const t = findKey(model.parameters?.options, 'temperature') ?? findKey(model.parameters, 'temperature');
+			if (t === undefined) problems.push(`${model.name}: temperature is not set, so the model default is used`);
+			else if (typeof t === 'number' && t > 0.2) problems.push(`${model.name}: temperature is ${t}`);
+		}
+	}
+	return check('S7', problems.length === 0, 'A model that decides a branch has a temperature of 0.2 or lower', src.triage, problems);
+}
+
+/** Field names of a JSON Schema, or of a JSON example when the value is not a schema. */
+function schemaKeys(value, out = new Set()) {
+	if (!value || typeof value !== 'object') return out;
+	if (Array.isArray(value)) {
+		value.forEach((v) => schemaKeys(v, out));
+		return out;
+	}
+	const isSchema = value.properties || value.items || typeof value.type === 'string';
+	if (isSchema) {
+		for (const [k, v] of Object.entries(value.properties ?? {})) {
+			out.add(k);
+			schemaKeys(v, out);
+		}
+		if (value.items) schemaKeys(value.items, out);
+		return out;
+	}
+	for (const [k, v] of Object.entries(value)) {
+		out.add(k);
+		schemaKeys(v, out);
+	}
+	return out;
+}
+
+function reasoningRule(g) {
+	const roots = decidingRoots(g);
+	if (!roots.length) return null;
+	const problems = [];
+	let judged = 0;
+	for (const { root } of roots) {
+		const found = parserParams(g, root);
+		let text = '';
+		if (found) text = String(found.p.inputSchema ?? found.p.jsonSchemaExample ?? found.p.jsonSchema ?? '');
+		else text = strings(root.parameters ?? {}).map((s) => s.value).join('\n');
+		if (!text.includes('{')) continue;
+		judged++;
+		let keys;
+		try {
+			keys = [...schemaKeys(JSON.parse(text))];
+		} catch {
+			keys = [...text.matchAll(/"([A-Za-z_]\w*)"\s*:/g)].map((m) => m[1]);
+		}
+		if (!keys.some((k) => /reason|rationale|explanation|why/i.test(k))) problems.push(`${root.name}: the output has no reasoning field (fields: ${keys.slice(0, 8).join(', ')})`);
+	}
+	if (!judged) return null;
+	return check('S8', problems.length === 0, 'AI output that decides a branch includes a reasoning field', src.triage, problems);
+}
+
+function fallbackRule(g) {
+	const roots = decidingRoots(g);
+	if (!roots.length) return null;
+	const problems = [];
+	const branches = new Map(roots.flatMap((r) => r.branches.map((b) => [b.name, b])));
+	for (const node of branches.values()) {
+		const used = new Set(g.mainChildren(node.name).map((e) => e.output));
+		if (typeMatches(node.type, 'if')) {
+			for (const [i, label] of [[0, 'true'], [1, 'false']]) if (!used.has(i)) problems.push(`${node.name}: the ${label} output goes nowhere`);
+		} else {
+			const rules = node.parameters?.rules?.values ?? node.parameters?.rules?.rules ?? [];
+			const fallback = node.parameters?.options?.fallbackOutput;
+			if (fallback !== 'extra') problems.push(`${node.name}: options.fallbackOutput is not 'extra', so unmatched items are dropped`);
+			else if (!used.has(rules.length)) problems.push(`${node.name}: the fallback output goes nowhere`);
+			rules.forEach((_, i) => !used.has(i) && problems.push(`${node.name}: output ${i} goes nowhere`));
+		}
+	}
+	return check('S9', problems.length === 0, 'Every branch after an AI decision, including the fallback, goes somewhere', src.triage, problems);
+}
+
+function memoryRule(g) {
+	const chat = g.triggers.some((t) => MESSAGING_TRIGGERS.some((m) => typeMatches(t.type, m)));
+	const agents = g.nodes.filter((n) => typeMatches(n.type, `${LC}agent`) && !g.subnodes.has(n.name));
+	if (!chat || !agents.length) return null;
+	const missing = agents.filter((a) => !g.aiInputs(a.name, 'ai_memory').length).map((a) => `${a.name}: no memory sub-node`);
+	return check('S10', missing.length === 0, 'A chatbot agent has memory', src.chatbot, missing);
+}
+
+function sessionKeyRule(g) {
+	const memories = g.nodes.filter((n) => g.subnodes.has(n.name) && /memory/i.test(shortType(n.type)) && n.parameters?.sessionIdType === 'customKey');
+	if (!memories.length) return null;
+	const problems = [];
+	for (const m of memories) {
+		const key = String(m.parameters?.sessionKey ?? '');
+		if (!key.startsWith('=')) problems.push(`${m.name}: the session key is a fixed value, so every conversation shares one memory`);
+		else if (/\$json\b/.test(key)) problems.push(`${m.name}: the session key reads $json, which a memory sub-node does not have`);
+	}
+	return check('S11', problems.length === 0, 'The memory session key names a node and identifies the conversation', { ...src.chatbot, also: src.memory }, problems);
+}
+
+// --- Main ----------------------------------------------------------------------
+
+export function deriveChecks(workflow, canvas, exec, { wire, binary, claims } = {}) {
+	const g = graph(workflow);
+	const refs = workflowReferences(g);
+	const correctness = [
+		checkDefinitions(workflow),
+		...checkCanvas(workflow, canvas),
+		checkReachable(g),
+		checkNamedRefs(workflow, g, refs),
+		checkFieldPaths(g, refs, exec),
+		checkSecrets(g),
+	];
+	const standards = [
+		errorWorkflowRule(workflow, g),
+		errorOutputRule(g),
+		retryRule(g),
+		namedRefRule(refs),
+		structuredRule(g),
+		attributionRule(g),
+		multiItemRule(g, refs),
+		temperatureRule(g),
+		reasoningRule(g),
+		fallbackRule(g),
+		memoryRule(g),
+		sessionKeyRule(g),
+	].filter(Boolean);
+	const scored = standards.filter((c) => c.ok !== null);
+	return {
+		version: VERSION,
+		groups: [
+			{ key: 'correctness', title: 'Correctness', gating: true, note: 'From the node definitions, the n8n editor and the workflow graph. A failure here changes the verdict.', checks: correctness },
+			{ key: 'execution', title: 'Execution with pinned data', gating: true, note: 'Triggers, service nodes and AI nodes get pinned data. Logic nodes run for real. A failure here changes the verdict.', checks: executionChecks(exec) },
+			...wireGroups(wire),
+			claimsGroup(claims),
+			{
+				key: 'standards',
+				title: 'n8n standards',
+				gating: false,
+				note: 'Rules from n8n-io/skills and the workflow-sdk best practices. The builders did not get these rules. This score does not change the verdict.',
+				score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
+				checks: standards,
+			},
+			binaryGroup(binary),
+		].filter(Boolean),
+	};
+}
+
+if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+	const runDir = resolve(process.argv[2] ?? '.');
+	for (const entry of readdirSync(runDir, { withFileTypes: true })) {
+		const dir = join(runDir, entry.name);
+		if (!entry.isDirectory() || !existsSync(join(dir, 'workflow.json'))) continue;
+		const result = deriveChecks(readJson(join(dir, 'workflow.json')), readJson(join(dir, 'canvas-issues.json')), readJson(join(dir, 'exec.json')), {
+			wire: readJson(join(dir, 'wire.json')),
+			binary: readJson(join(dir, 'binary-checks.json')),
+			claims: readJson(join(dir, 'claims.json')),
+		});
+		writeFileSync(join(dir, 'auto-checks.json'), JSON.stringify(result, null, 1));
+		const line = result.groups.map((group) => {
+			const judged = group.checks.filter((c) => c.ok !== null && c.level !== 'info');
+			return `${group.key} ${judged.filter((c) => c.ok).length}/${judged.length}`;
+		});
+		console.log(`${entry.name}: auto-checks.json written (${line.join(', ')})`);
+	}
+}

.agents/skills/testmax-n8n-ai/scripts/langsmith_usage.py

@@ -0,0 +1,160 @@
+#!/usr/bin/env python3
+"""Fetch per-call token usage and cost of an Instance AI run from LangSmith.
+
+n8n does not store token usage. LangSmith has it when the instance runs with
+LANGSMITH_TRACING=true. The script reads LANGSMITH_API_KEY, LANGSMITH_ENDPOINT and
+LANGSMITH_PROJECT from the config (see n8n_api.config).
+
+  langsmith_usage.py --run-dir <run-dir>/iai
+
+It takes all LLM runs in the project between startedAt and finishedAt from run.json.
+Do not start other Instance AI work on the same instance during the run, or use --by-trace.
+
+  langsmith_usage.py --run-dir <dir> --by-trace
+
+--by-trace keeps only the calls of the traces in run.json "langsmithRunIds" (the root run id of
+each run-start event is also its LangSmith trace id). Use it when other Instance AI work runs
+at the same time.
+
+The script also writes context.json: for each agent, the system prompt, the messages before
+the first model response, and the tools, from the first LLM call of that agent. Tools that the
+agent loads later with load_tool come from its last call.
+"""
+
+import argparse
+import json
+import urllib.parse
+import urllib.request
+from datetime import datetime, timedelta
+from pathlib import Path
+
+from n8n_api import setting
+
+
+def langsmith_client():
+	key = setting('LANGSMITH_API_KEY')
+	if not key:
+		raise SystemExit('LANGSMITH_API_KEY is not set. Instance AI cost and tokens are not available.')
+	base = setting('LANGSMITH_ENDPOINT', default='https://api.smith.langchain.com').rstrip('/')
+
+	def call(method, path, body=None):
+		req = urllib.request.Request(
+			base + path, method=method,
+			data=json.dumps(body).encode() if body is not None else None,
+			headers={'x-api-key': key, 'content-type': 'application/json'},
+		)
+		with urllib.request.urlopen(req, timeout=60) as res:
+			return json.load(res)
+
+	return call
+
+
+def fetch_calls(started_at, finished_at, traces=None):
+	"""LLM calls in the LangSmith project between two ISO times. traces keeps only the calls of these traces."""
+	call = langsmith_client()
+	project = setting('LANGSMITH_PROJECT', 'LANGCHAIN_PROJECT', default='instance-ai')
+
+	sessions = call('GET', '/api/v1/sessions?' + urllib.parse.urlencode({'name': project}))
+	if not sessions:
+		raise SystemExit(f'LangSmith project "{project}" not found.')
+	session_id = sessions[0]['id']
+
+	start = datetime.fromisoformat(started_at) - timedelta(seconds=5)
+	end = datetime.fromisoformat(finished_at) + timedelta(seconds=5)
+	calls, cursor = [], None
+	while True:
+		body = {
+			'session': [session_id], 'run_type': 'llm', 'limit': 100,
+			'start_time': start.isoformat(),
+			'select': ['id', 'name', 'start_time', 'end_time', 'prompt_tokens', 'completion_tokens',
+			           'total_cost', 'prompt_token_details', 'extra', 'trace_id'],
+		}
+		if cursor:
+			body['cursor'] = cursor
+		res = call('POST', '/api/v1/runs/query', body)
+		for run in res.get('runs', []):
+			started = datetime.fromisoformat(run['start_time'].replace('Z', '+00:00'))
+			if started.tzinfo is None:
+				started = started.replace(tzinfo=start.tzinfo)
+			if started > end or (traces is not None and run.get('trace_id') not in traces):
+				continue
+			details = run.get('prompt_token_details') or {}
+			metadata = (run.get('extra') or {}).get('metadata') or {}
+			calls.append({
+				'id': run.get('id'), 'trace': run.get('trace_id'), 'name': run.get('name'),
+				'start': started.isoformat(), 'end': run.get('end_time'),
+				'in': run.get('prompt_tokens') or 0,
+				'cache_read': details.get('cache_read') or 0,
+				'cache_write': details.get('cache_creation') or 0,
+				'out': run.get('completion_tokens') or 0,
+				'cost': float(run.get('total_cost') or 0),
+				'model': metadata.get('ls_model_name'),
+			})
+		cursor = (res.get('cursors') or {}).get('next')
+		if not cursor:
+			break
+	return sorted(calls, key=lambda c: c['start'])
+
+
+def message_text(content):
+	if isinstance(content, str):
+		return content
+	return '\n'.join(p.get('text', '') for p in content or [] if isinstance(p, dict) and p.get('type') == 'text')
+
+
+def tool_spec(t):
+	return {'name': t.get('name') or t.get('id'), 'description': t.get('description') or '',
+	        'schema': t.get('inputSchema') or t.get('parameters'), 'type': t.get('type')}
+
+
+def fetch_contexts(calls):
+	"""What each agent got before its first model response, in the order that the agents started."""
+	call = langsmith_client()
+	by_agent = {}
+	for c in calls:
+		if c.get('id'):
+			by_agent.setdefault(c['name'], []).append(c)
+	contexts = []
+	for name, agent_calls in by_agent.items():
+		first = call('GET', f'/api/v1/runs/{agent_calls[0]["id"]}').get('inputs') or {}
+		messages = []
+		for m in first.get('input') or []:
+			if m.get('role') not in ('system', 'user'):
+				break
+			messages.append({'role': m['role'], 'text': message_text(m.get('content'))})
+		tools = [tool_spec(t) for t in first.get('tools') or []]
+		loaded = []
+		if len(agent_calls) > 1:
+			last = call('GET', f'/api/v1/runs/{agent_calls[-1]["id"]}').get('inputs') or {}
+			known = {t['name'] for t in tools}
+			loaded = [tool_spec(t) for t in last.get('tools') or [] if (t.get('name') or t.get('id')) not in known]
+		contexts.append({'agent': name, 'messages': messages, 'tools': tools, 'loadedTools': loaded})
+	return contexts
+
+
+def totals(calls):
+	return {
+		'calls': len(calls), 'models': sorted({c['model'] for c in calls if c['model']}),
+		'input': sum(c['in'] for c in calls), 'output': sum(c['out'] for c in calls),
+		'cost': round(sum(c['cost'] for c in calls), 4),
+	}
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--run-dir', required=True)
+	ap.add_argument('--by-trace', action='store_true')
+	args = ap.parse_args()
+	run_dir = Path(args.run_dir)
+	meta = json.loads((run_dir / 'run.json').read_text())
+	traces = set(meta.get('langsmithRunIds') or [])
+	if args.by_trace and not traces:
+		raise SystemExit('run.json has no langsmithRunIds. Is LANGSMITH_TRACING=true on the instance?')
+	calls = fetch_calls(meta['startedAt'], meta['finishedAt'], traces if args.by_trace else None)
+	(run_dir / 'usage.json').write_text(json.dumps(calls, indent=1))
+	(run_dir / 'context.json').write_text(json.dumps(fetch_contexts(calls), indent=1))
+	print(json.dumps(totals(calls), indent=2))
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/live.py

@@ -0,0 +1,84 @@
+"""Live views of a run.
+
+COMPARE_LIVE=1 (default): compare.py serves a live page (live_server.py). COMPARE_LIVE=0 turns it off.
+COMPARE_LIVE_WINDOWS=1 also opens the Instance AI thread in the browser and a terminal window for
+each CLI builder. If a view cannot open, the script prints how to open it.
+COMPARE_RECORD=0 skips the screen recordings that the report replays.
+"""
+
+import shlex
+import shutil
+import subprocess
+import sys
+import webbrowser
+
+from n8n_api import setting
+
+
+def is_on(name, default='1'):
+	return setting(name, default=default).strip().lower() not in ('0', 'false', 'no', 'off')
+
+
+def enabled():
+	return is_on('COMPARE_LIVE')
+
+
+def windows():
+	return enabled() and is_on('COMPARE_LIVE_WINDOWS', default='0')
+
+
+def recording():
+	return is_on('COMPARE_RECORD')
+
+
+def open_url(url):
+	if not windows():
+		return
+	try:
+		opened = webbrowser.open(url)
+	except webbrowser.Error:
+		opened = False
+	if not opened:
+		print(f'open {url} to watch the run', flush=True)
+
+
+def log_window_launchers(command):
+	if sys.platform == 'darwin' and shutil.which('osascript'):
+		# The command goes in as an argument, so it needs no AppleScript quoting.
+		yield ['osascript', '-e', 'on run argv', '-e',
+		       'tell application "Terminal" to do script (item 1 of argv)', '-e', 'end run', command]
+	for terminal, flag in (('x-terminal-emulator', '-e'), ('gnome-terminal', '--'), ('konsole', '-e'), ('xterm', '-e')):
+		if shutil.which(terminal):
+			yield [terminal, flag, 'sh', '-c', command]
+			return
+
+
+def open_terminal(title, command, cols=None, rows=None):
+	"""Open a terminal window with a title that runs a shell command. Returns False if none opened."""
+	if not windows():
+		return True
+	prefix = f"printf '\\033]0;%s\\007' {shlex.quote(title)}; "
+	if cols and rows:
+		# Resize the window so that the full screen of the CLI fits.
+		prefix += f"printf '\\033[8;{rows};{cols}t'; "
+	for launcher in log_window_launchers(prefix + command):
+		# osascript returns at once, but xterm and other emulators run as long as the window is open.
+		# A launcher that still runs after a short wait has opened its window.
+		try:
+			proc = subprocess.Popen(launcher, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
+			                        stderr=subprocess.DEVNULL, start_new_session=True)
+		except OSError:
+			continue
+		try:
+			if proc.wait(timeout=3) == 0:
+				return True
+		except subprocess.TimeoutExpired:
+			return True
+	return False
+
+
+def open_log_window(title, path):
+	"""Open a terminal window that follows the log file at path."""
+	quoted = shlex.quote(str(path))
+	if not open_terminal(title, f'tail -n +1 -f {quoted}'):
+		print(f'watch {title}: tail -f {quoted}', flush=True)

.agents/skills/testmax-n8n-ai/scripts/live_server.py

@@ -0,0 +1,635 @@
+#!/usr/bin/env python3
+"""Serve a page that shows the builder runs while they run.
+
+  live_server.py --run-dir <run-dir> [--parent-pid <pid>]
+
+compare.py starts this server in the background. The server listens on 127.0.0.1 only and
+writes its address to <run-dir>/live-url.txt. Routes:
+
+  /report/live       the live page
+  /report/<file>     the final report and its files, when they exist
+  /api/state         the status, table row and LLM steps of each builder, as JSON
+  /api/cast/<name>   the terminal screen of a CLI builder, as Server-Sent Events in the
+                     live format of the asciinema player
+  /api/shot/<name>   the last screenshot of the Instance AI thread
+
+compare.py writes the status of each builder to <run-dir>/status.json. The server stops
+30 minutes after the last request, when the run is done or compare.py has stopped.
+
+Other skills reuse the server: they call serve() with their own app (see BuilderApp).
+"""
+
+import argparse
+import json
+import mimetypes
+import os
+import re
+import threading
+import time
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from urllib.parse import unquote, urlparse
+
+import build_report as br
+from run_cli_builder import find_session_file
+
+IDLE_STOP_S = 1800
+KINDS = {'iai': 'instance-ai', 'claude': 'claude', 'codex': 'codex'}
+STATUS_LABELS = {'waiting': 'Waiting', 'running': 'Running', 'done': 'Done', 'failed': 'Failed', 'skipped': 'Skipped'}
+mimetypes.add_type('font/woff2', '.woff2')
+mimetypes.add_type('text/javascript', '.js')
+mimetypes.add_type('video/webm', '.webm')
+
+
+def read_status(run_dir):
+	try:
+		return json.loads((run_dir / 'status.json').read_text())
+	except (OSError, ValueError):
+		return {'phase': 'running', 'builders': {}}
+
+
+def empty_data(model=None):
+	return {'events': [], 'calls': [], 'model': model, 'duration': '-', 'cost': '-', 'costSource': None}
+
+
+def load_instance_ai_live(sub):
+	"""Trace events from the event stream that run_instance_ai.py writes while the thread runs."""
+	path = sub / 'events.jsonl'
+	rows = br.read_jsonl(path) if path.exists() else []
+	if not rows:
+		return empty_data()
+	first = next((r for r in rows if r.get('type') == 'run-start'), rows[0])
+	start, root = first['ts'] / 1000, first.get('agentId')
+	events, texts, tools = [], {}, {}
+	for r in rows:
+		kind, p, agent = r.get('type'), r.get('payload') or {}, r.get('agentId')
+		t = (r.get('ts') or first['ts']) / 1000 - start
+		label = '' if agent in (root, None) else f'{agent.split("-run_")[0]} › '
+		if kind in ('text-delta', 'reasoning-delta'):
+			event_kind = 'text' if kind == 'text-delta' else 'reasoning'
+			key = (agent, r.get('responseId'), event_kind)
+			if key not in texts:
+				texts[key] = {'kind': event_kind, 't': t, 'text': ''}
+				events.append(texts[key])
+			texts[key]['text'] += p.get('text') or ''
+		elif kind == 'tool-call':
+			tools[p.get('toolCallId')] = {'kind': 'tool', 't': t, 'name': label + (p.get('toolName') or ''),
+			                              'args': p.get('args'), 'result': 'Waiting for the result…',
+			                              'error': False, 'duration': None}
+			events.append(tools[p.get('toolCallId')])
+		elif kind in ('tool-result', 'tool-error') and p.get('toolCallId') in tools:
+			result = p.get('result') if kind == 'tool-result' else p.get('error')
+			tools[p['toolCallId']].update(
+				result=result, duration=t - tools[p['toolCallId']]['t'],
+				error=kind == 'tool-error' or (isinstance(result, dict) and (result.get('success') is False or 'error' in result)))
+		elif kind == 'confirmation-request' and p.get('toolCallId') in tools:
+			tools[p['toolCallId']]['note'] = 'Confirmation card answered by the auto-approve policy.'
+	data = empty_data()
+	data['events'] = [e for e in events if e['kind'] == 'tool' or e['text'].strip()]
+	return data
+
+
+def live_run(run_dir, name, case):
+	"""Meta and trace data of one builder. A finished builder has run.json. A running one does not."""
+	sub = run_dir / name
+	meta_file = sub / 'run.json'
+	if meta_file.exists():
+		meta = json.loads(meta_file.read_text())
+		if meta.get('kind') in br.LOADERS:
+			return meta, br.load_run(sub, meta, case.get('prices'))
+	meta = {'kind': KINDS.get(name, name)}
+	if meta['kind'] == 'instance-ai':
+		return meta, load_instance_ai_live(sub)
+	session = find_session_file(meta['kind'], sub / 'home') if (sub / 'home').exists() else None
+	if not session:
+		return meta, empty_data()
+	meta['sessionFile'] = str(session.relative_to(sub))
+	return meta, br.load_run(sub, meta, case.get('prices'))
+
+
+def seconds(value):
+	return f'{value:.0f}s'
+
+
+def gather(run_dir):
+	case, status, verdicts = br.load_case(run_dir), read_status(run_dir), br.load_verdicts(run_dir)
+	builders = status.get('builders', {})
+	names = sorted((n for n, b in builders.items() if b.get('status') != 'skipped'),
+	               key=lambda n: (br.RUN_ORDER.get(KINDS.get(n, n), 9), n))
+	now, runs = time.time(), []
+	for name in names:
+		entry = builders[name]
+		state = entry.get('status', 'waiting')
+		meta, data = live_run(run_dir, name, case)
+		running = state in ('waiting', 'running')
+		if running and data.get('loadError'):
+			# A running builder can leave a file in a state that the loader cannot read yet.
+			data = empty_data(data.get('model'))
+		started, finished = entry.get('startedAt'), entry.get('finishedAt')
+		if started and (running or data['duration'] == '-'):
+			data['duration'] = seconds((finished or now) - started)
+		if running:
+			data['cost'] = '-'
+		runs.append((name, meta, data, entry))
+	return case, status, verdicts, runs
+
+
+def status_badge(state):
+	return f'<span class="badge st-{br.esc(state)}">{STATUS_LABELS.get(state, br.esc(state))}</span>'
+
+
+def clock(status):
+	started = status.get('startedAt')
+	elapsed = seconds((status.get('finishedAt') or time.time()) - started) if started else ''
+	mode = 'Parallel' if status.get('mode') == 'parallel' else 'One at a time'
+	return ' · '.join(x for x in (mode, elapsed, status.get('step')) if x)
+
+
+def render_run(run_dir, case, name, meta, data, entry, verdict, scale):
+	row = br.render_row_cells(name, meta, data, status_badge(entry.get('status', 'waiting')))
+	canvas = br.render_canvas_box(name, meta, run_dir / 'report', case.get('baseUrl', 'http://localhost:5678'),
+	                              empty='The canvas appears when all builders finish.')
+	view, auto = br.verdict_view(verdict, run_dir / name, case.get('expected') or '')
+	return row, br.render_panel_body(data, view, canvas, *scale, case['prompt'], case.get('expected') or '', auto=auto,
+	                                 pending=True)
+
+
+def render_live_box(status, runs):
+	cards = []
+	for name, meta, _, entry in runs:
+		title = br.esc(br.run_title(name, meta))
+		screen = (status.get('screens') or {}).get(name)
+		if screen == 'cast':
+			inner = (f'<div data-cast="{name}"></div><button type="button" class="live-zoom overlay" '
+			         f'data-zoom="{name}" aria-label="Show {title} full screen"></button>')
+		elif screen == 'shot':
+			inner = (f'<button type="button" class="live-zoom" data-zoom="{name}" aria-label="Show {title} full screen">'
+			         f'<img data-shot="{name}" alt="{title} in the n8n UI" hidden></button>'
+			         f'<div class="live-empty" data-shot-empty="{name}">Waiting for the first screenshot</div>')
+		else:
+			inner = '<div class="live-empty">No live screen for this builder. The LLM steps below update live.</div>'
+		cards.append(
+			f'<div class="live-card"><div class="live-screen">{inner}</div>'
+			f'<div class="reel-meta"><span class="run-name">{title}</span>'
+			f'<span data-live-badge="{name}">{status_badge(entry.get("status", "waiting"))}</span></div></div>'
+		)
+	return (f'<div class="box"><div class="head"><h2>Live</h2><span class="mono-label" id="live-clock">'
+	        f'{br.esc(clock(status))}</span></div><div class="live-grid">{"".join(cards)}</div></div>')
+
+
+def render_page(run_dir):
+	case, status, verdicts, runs = gather(run_dir)
+	scale = br.token_scale([r[:3] for r in runs])
+	rows, panels = [], []
+	for i, (name, meta, data, entry) in enumerate(runs):
+		active = ' active' if i == 0 else ''
+		row, panel = render_run(run_dir, case, name, meta, data, entry, verdicts.get(name), scale)
+		rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">{row}</tr>')
+		panels.append(f'<section class="run{active}" id="{name}">{panel}</section>')
+	try:
+		br.ensure_player_cache()
+		player_js, player_css = (f'vendor/{n}' for n in br.PLAYER_FILES)
+	except OSError:
+		player_js, player_css = (br.PLAYER_CDN + n for n in br.PLAYER_FILES)
+	notes = ('This page updates while the builders run, and changes to the full report when the run is done. '
+	         'Costs, canvases, checks and verdicts come after the run. Times are seconds from the start of each builder.')
+	return br.fill_template(
+		br.esc(case.get('title') or 'Builder comparison'), ''.join(rows), br.esc(notes), ''.join(panels),
+		replay=render_live_box(status, runs), head=f'<link rel="stylesheet" href="{player_css}">',
+		scripts=f'<script src="{player_js}"></script><script>const SCREENS = [];</script><script>{LIVE_JS}</script>',
+		prompt=br.esc(case['prompt']), expected=br.esc(case.get('expected') or '-'))
+
+
+def render_state(run_dir):
+	case, status, verdicts, runs = gather(run_dir)
+	scale = br.token_scale([r[:3] for r in runs])
+	state = {'phase': status.get('phase', 'running'), 'clock': clock(status), 'runs': {}}
+	if state['phase'] == 'done' and (run_dir / 'report' / 'index.html').exists():
+		state['report'] = '/report/index.html'
+	for name, meta, data, entry in runs:
+		row, panel = render_run(run_dir, case, name, meta, data, entry, verdicts.get(name), scale)
+		shot = run_dir / name / 'live.png'
+		state['runs'][name] = {'row': row, 'panel': panel, 'badge': status_badge(entry.get('status', 'waiting')),
+		                       'shot': shot.stat().st_mtime if shot.exists() else None}
+	return state
+
+
+LIVE_JS = """
+window.addEventListener('DOMContentLoaded', ()=>{
+  const seen = {}, shots = {}, userFolds = {};
+  // A box that the user opened or closed stays that way when its panel updates. Other boxes take the new default.
+  document.addEventListener('click', e=>{
+    const summary = e.target.closest && e.target.closest('details.fold > summary');
+    if (!summary || e.target.closest('.run-picker')) return;
+    const section = summary.closest('section.run');
+    if (section) (userFolds[section.id] ??= {})[summary.querySelector('h2').textContent] = !summary.parentElement.open;
+  });
+  document.addEventListener('change', e=>{
+    if (!e.target.matches || !e.target.matches('.run-picker')) return;
+    const steps = document.getElementById(e.target.value)?.querySelector('.steps');
+    if (steps) (userFolds[e.target.value] ??= {})['LLM steps'] = steps.open;
+  });
+  document.querySelectorAll('[data-cast]').forEach(el=>{
+    AsciinemaPlayer.create({driver: 'eventsource', url: `/api/cast/${el.dataset.cast}`, bufferTime: 0}, el,
+      {fit: 'both', autoPlay: true, controls: false});
+  });
+  // A live screen opens larger in a full-screen viewer. The viewer follows the newest screenshot, and a
+  // terminal gets its own player, because the stream starts with the whole recording so far.
+  const zoomNames = [...document.querySelectorAll('[data-zoom]')].map(b=>b.dataset.zoom);
+  const viewer = document.createElement('div');
+  let zoomed = null, opener = null, castPlayer = null, castName = null;
+  viewer.className = 'theater live-theater';
+  viewer.hidden = true;
+  viewer.setAttribute('role', 'dialog');
+  viewer.setAttribute('aria-modal', 'true');
+  viewer.setAttribute('aria-label', 'Live screen');
+  viewer.innerHTML = '<div class="theater-bar"><h2>Live</h2><span data-zoom-badge></span>'
+    + '<button type="button" class="btn" data-zoom-close>Close</button></div><div class="tabs"></div>'
+    + '<div class="live-zoomed"><img alt=""><div class="live-empty">Waiting for the first screenshot</div>'
+    + '<div class="live-zoomed-cast" hidden></div></div>'
+    + '<p class="fineprint">The screen updates every few seconds. Press the left or right arrow key to change the '
+    + 'scenario. Press Esc to close.</p>';
+  for (const name of zoomNames) {
+    const tab = Object.assign(document.createElement('button'), {type: 'button', className: 'tab'});
+    tab.dataset.zoomTab = name;
+    tab.textContent = document.querySelector(`[data-zoom="${name}"]`).closest('.live-card').querySelector('.run-name').textContent;
+    tab.addEventListener('click', ()=>showZoom(name));
+    viewer.querySelector('.tabs').append(tab);
+  }
+  document.body.append(viewer);
+  function stopCast(){
+    castPlayer?.dispose();
+    castPlayer = castName = null;
+    viewer.querySelector('.live-zoomed-cast').replaceChildren();
+  }
+  function showZoom(name){
+    zoomed = name;
+    viewer.querySelectorAll('[data-zoom-tab]').forEach(t=>t.classList.toggle('active', t.dataset.zoomTab === name));
+    const shot = document.querySelector(`img[data-shot="${name}"]`);
+    const isCast = !!document.querySelector(`[data-cast="${name}"]`);
+    const img = viewer.querySelector('.live-zoomed img'), castBox = viewer.querySelector('.live-zoomed-cast');
+    if (castName !== name) stopCast();
+    castBox.hidden = !isCast;
+    if (isCast && !castPlayer) {
+      castName = name;
+      castPlayer = AsciinemaPlayer.create({driver: 'eventsource', url: `/api/cast/${name}`, bufferTime: 0}, castBox,
+        {fit: 'both', autoPlay: true, controls: false});
+    }
+    const shotHidden = !shot || shot.hidden;
+    img.hidden = isCast || shotHidden;
+    if (shot && !shot.hidden && img.getAttribute('src') !== shot.getAttribute('src')) { img.src = shot.src; img.alt = shot.alt; }
+    viewer.querySelector('.live-zoomed .live-empty').hidden = isCast || !shotHidden;
+    viewer.querySelector('[data-zoom-badge]').innerHTML = document.querySelector(`[data-live-badge="${name}"]`)?.innerHTML || '';
+  }
+  function closeZoom(){
+    stopCast();
+    viewer.hidden = true;
+    zoomed = null;
+    document.body.classList.remove('theater-open');
+    opener?.focus();
+  }
+  document.querySelectorAll('[data-zoom]').forEach(b=>b.addEventListener('click', ()=>{
+    opener = b;
+    viewer.hidden = false;
+    document.body.classList.add('theater-open');
+    showZoom(b.dataset.zoom);
+    viewer.querySelector('[data-zoom-close]').focus();
+  }));
+  viewer.querySelector('[data-zoom-close]').addEventListener('click', closeZoom);
+  document.addEventListener('keydown', e=>{
+    if (zoomed === null) return;
+    if (e.key === 'Escape') closeZoom();
+    else if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') {
+      const step = e.key === 'ArrowRight' ? 1 : -1;
+      showZoom(zoomNames[(zoomNames.indexOf(zoomed) + step + zoomNames.length) % zoomNames.length]);
+    }
+  });
+  function selectQuietly(root, step){
+    root.querySelectorAll('.step').forEach(b=>b.classList.toggle('active', b.dataset.step === step));
+    root.querySelectorAll('.step-detail').forEach(d=>{ d.hidden = d.dataset.step !== step; });
+  }
+  // Keeps the selected step, the scroll positions and the open sections. When the last step is
+  // selected, the newest step is selected after the update.
+  function swapPanel(section, html, first){
+    const old = section.querySelector('.steps');
+    const items = old ? [...old.querySelectorAll('.step')] : [];
+    const active = items.find(b=>b.classList.contains('active'));
+    const follow = first || !active || active === items[items.length - 1];
+    // A run with no trace yet has a .steps section without the list and the pane.
+    const listTop = old?.querySelector('.step-list')?.scrollTop ?? 0;
+    const paneTop = old?.querySelector('.step-pane')?.scrollTop ?? 0;
+    const openState = {};
+    section.querySelectorAll('.step-detail').forEach(d=>d.querySelectorAll('details').forEach((x, i)=>{ openState[`${d.dataset.step}:${i}`] = x.open; }));
+    section.innerHTML = html;
+    section.querySelectorAll('details.fold').forEach(d=>{
+      const open = (userFolds[section.id] || {})[d.querySelector('summary h2')?.textContent];
+      if (open !== undefined) d.open = open;
+    });
+    addRunPickers();
+    section.querySelectorAll('.step-detail').forEach(d=>d.querySelectorAll('details').forEach((x, i)=>{
+      const key = `${d.dataset.step}:${i}`;
+      if (key in openState) x.open = openState[key];
+    }));
+    const root = section.querySelector('.steps');
+    const next = root ? [...root.querySelectorAll('.step')] : [];
+    if (!next.length) return;
+    const list = root.querySelector('.step-list'), pane = root.querySelector('.step-pane');
+    if (!list || !pane) return;
+    if (follow) {
+      selectQuietly(root, next[next.length - 1].dataset.step);
+      list.scrollTop = list.scrollHeight;
+    } else {
+      selectQuietly(root, active.dataset.step);
+      list.scrollTop = listTop;
+      pane.scrollTop = paneTop;
+    }
+  }
+  async function poll(){
+    let state;
+    try {
+      state = await (await fetch('/api/state', {cache: 'no-store'})).json();
+    } catch {
+      $('live-clock').textContent = 'Lost the connection to the live server. Trying again…';
+      setTimeout(poll, 3000);
+      return;
+    }
+    if (state.report) { location.replace(state.report); return; }
+    $('live-clock').textContent = state.clock;
+    for (const [name, run] of Object.entries(state.runs)) {
+      const row = document.querySelector(`.row[data-run="${name}"]`);
+      if (row && seen[`row:${name}`] !== run.row) { row.innerHTML = run.row; seen[`row:${name}`] = run.row; }
+      const section = document.getElementById(name);
+      if (section && seen[`panel:${name}`] !== run.panel) {
+        swapPanel(section, run.panel, seen[`panel:${name}`] === undefined);
+        seen[`panel:${name}`] = run.panel;
+      }
+      const badge = document.querySelector(`[data-live-badge="${name}"]`);
+      if (badge) badge.innerHTML = run.badge;
+      const img = document.querySelector(`img[data-shot="${name}"]`);
+      if (img && run.shot && shots[name] !== run.shot) {
+        shots[name] = run.shot;
+        img.src = `/api/shot/${name}?v=${run.shot}`;
+        img.hidden = false;
+        document.querySelector(`[data-shot-empty="${name}"]`)?.remove();
+      }
+    }
+    if (zoomed !== null) showZoom(zoomed);
+    setTimeout(poll, 2000);
+  }
+  poll();
+});
+"""
+
+
+class BuilderApp:
+	"""What the server shows for compare.py: one live card, table row and panel for each builder.
+
+	Another skill can serve its own page with the same server: give LiveServer an object with
+	these methods."""
+
+	def __init__(self, run_dir):
+		self.run_dir = run_dir
+
+	def entry(self, name):
+		"""The status.json entry of a live card, or None for an unknown name."""
+		return read_status(self.run_dir).get('builders', {}).get(name)
+
+	def finished(self, name):
+		state = (self.entry(name) or {}).get('status')
+		return state in ('done', 'failed', 'skipped') or (self.run_dir / name / 'run.json').exists()
+
+	def done(self):
+		return read_status(self.run_dir).get('phase') == 'done' and (self.run_dir / 'report' / 'index.html').exists()
+
+	def page(self):
+		return render_page(self.run_dir)
+
+	def state(self):
+		return render_state(self.run_dir)
+
+	def shot(self, name):
+		return self.run_dir / name / 'live.png'
+
+	def casts(self, name):
+		"""The live recording, then the place where the runner moves it when the builder stops."""
+		sub = self.run_dir / name
+		return sub / '.screen' / 'screen.cast', sub / 'screen.cast'
+
+	def file(self, path):
+		"""A file for a route that the server does not know, or None."""
+		return None
+
+
+class LiveServer(ThreadingHTTPServer):
+	daemon_threads = True
+
+	def __init__(self, address, app):
+		super().__init__(address, Handler)
+		port = self.server_address[1]
+		self.app, self.run_dir = app, app.run_dir
+		self.allowed_hosts = {f'127.0.0.1:{port}', f'localhost:{port}'}
+		self.last_request = time.time()
+
+
+class Handler(BaseHTTPRequestHandler):
+	server_version = 'cwb-live'
+
+	def log_message(self, *args):
+		pass
+
+	def do_GET(self):
+		srv = self.server
+		srv.last_request = time.time()
+		# Only same-origin requests. A page that resolves its own host name to 127.0.0.1 gets a 403.
+		if self.headers.get('Host', '') not in srv.allowed_hosts:
+			return self.send_error(403)
+		path = unquote(urlparse(self.path).path)
+		app = srv.app
+		try:
+			if path in ('/', '/report', '/report/'):
+				return self.redirect('/report/index.html' if app.done() else '/report/live')
+			if path == '/report/live':
+				return self.send_body(app.page().encode(), 'text/html; charset=utf-8')
+			if path == '/api/state':
+				return self.send_body(json.dumps(app.state()).encode(), 'application/json')
+			name = path.rsplit('/', 1)[-1]
+			if path.startswith(('/api/cast/', '/api/shot/')) and not app.entry(name):
+				return self.send_error(404)
+			if path.startswith('/api/cast/'):
+				return self.stream_cast(name)
+			if path.startswith('/api/shot/'):
+				return self.send_file(app.shot(name))
+			if path.startswith('/report/'):
+				return self.send_file(self.static_file(path.removeprefix('/report/')))
+			self.send_file(app.file(path))
+		except (BrokenPipeError, ConnectionResetError):
+			pass
+
+	def redirect(self, location):
+		self.send_response(302)
+		self.send_header('Location', location)
+		self.send_header('Content-Length', '0')
+		self.end_headers()
+
+	def send_body(self, body, content_type):
+		self.send_response(200)
+		self.send_header('Content-Type', content_type)
+		self.send_header('Content-Length', str(len(body)))
+		self.send_header('Cache-Control', 'no-store')
+		self.end_headers()
+		self.wfile.write(body)
+
+	def static_file(self, rel):
+		"""A file of the report. Fonts, logos and the player come from the asset cache until the report exists."""
+		base = (self.server.run_dir / 'report').resolve()
+		candidate = (base / rel).resolve()
+		if candidate.is_relative_to(base) and candidate.is_file():
+			return candidate
+		parts = Path(rel).parts
+		if parts[:1] in (('fonts',), ('brand',)):
+			cache = br.ASSET_CACHE.resolve()
+			candidate = (cache / rel).resolve()
+			return candidate if candidate.is_relative_to(cache) and candidate.is_file() else None
+		if len(parts) == 2 and parts[0] == 'vendor' and parts[1] in br.PLAYER_FILES:
+			return br.PLAYER_CACHE / parts[1]
+		return None
+
+	def send_file(self, file):
+		if not file or not file.is_file():
+			return self.send_error(404)
+		size = file.stat().st_size
+		start, end = 0, size - 1
+		match = re.fullmatch(r'bytes=(\d*)-(\d*)', self.headers.get('Range') or '')
+		if match and size and (match[1] or match[2]):
+			# Videos need byte ranges to seek.
+			if match[1]:
+				start, end = int(match[1]), int(match[2]) if match[2] else size - 1
+			else:
+				start = max(size - int(match[2]), 0)
+			end = min(end, size - 1)
+			if start > end:
+				self.send_response(416)
+				self.send_header('Content-Range', f'bytes */{size}')
+				self.end_headers()
+				return
+			self.send_response(206)
+			self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
+		else:
+			self.send_response(200)
+		self.send_header('Content-Type', mimetypes.guess_type(file.name)[0] or 'application/octet-stream')
+		self.send_header('Content-Length', str(end - start + 1))
+		self.send_header('Accept-Ranges', 'bytes')
+		self.send_header('Cache-Control', 'no-cache')
+		self.end_headers()
+		with file.open('rb') as f:
+			f.seek(start)
+			left = end - start + 1
+			while left > 0:
+				chunk = f.read(min(left, 1 << 16))
+				if not chunk:
+					break
+				self.wfile.write(chunk)
+				left -= len(chunk)
+
+	def sse(self, data, event=None):
+		self.wfile.write(((f'event: {event}\n' if event else '') + f'data: {data}\n\n').encode())
+		self.wfile.flush()
+
+	def stream_cast(self, name):
+		"""Send the screen recording so far as one reset frame, then each new frame as it arrives."""
+		srv = self.server
+		self.send_response(200)
+		self.send_header('Content-Type', 'text/event-stream')
+		self.send_header('Cache-Control', 'no-store')
+		self.end_headers()
+		path = None
+		while path is None:
+			path = next((p for p in srv.app.casts(name) if p.exists()), None)
+			if path is None:
+				if srv.app.finished(name):
+					return self.sse('{}', 'done')
+				self.wfile.write(b': waiting\n\n')
+				self.wfile.flush()
+				srv.last_request = time.time()
+				time.sleep(1)
+		# The runner moves the recording when the builder stops. The open file stays readable.
+		with path.open('rb') as f:
+			buffer = b''
+
+			def complete_lines():
+				nonlocal buffer
+				buffer += f.read()
+				*lines, buffer = buffer.split(b'\n')
+				return [line for line in lines if line.strip()]
+
+			lines = complete_lines()
+			while not lines:
+				if srv.app.finished(name):
+					return self.sse('{}', 'done')
+				time.sleep(0.2)
+				lines = complete_lines()
+			header, frames = json.loads(lines[0]), [json.loads(line) for line in lines[1:]]
+			output = [f for f in frames if f[1] == 'o']
+			self.sse(json.dumps({'cols': header.get('width'), 'rows': header.get('height'),
+			                     'time': output[-1][0] if output else 0, 'init': ''.join(f[2] for f in output)}))
+			last_data = last_ping = time.time()
+			while True:
+				lines = complete_lines()
+				for line in lines:
+					frame = json.loads(line)
+					if frame[1] == 'o':
+						self.sse(json.dumps([frame[0], 'o', frame[2]]))
+				now = time.time()
+				srv.last_request = now
+				if lines:
+					last_data = now
+					continue
+				if srv.app.finished(name) and now - last_data > 3:
+					return self.sse('{}', 'done')
+				if now - last_ping > 15:
+					self.wfile.write(b': ping\n\n')
+					self.wfile.flush()
+					last_ping = now
+				time.sleep(0.2)
+
+
+def pid_alive(pid):
+	try:
+		os.kill(pid, 0)
+	except ProcessLookupError:
+		return False
+	except PermissionError:
+		return True
+	return True
+
+
+def stop_when_idle(server, parent_pid):
+	while True:
+		time.sleep(30)
+		if time.time() - server.last_request < IDLE_STOP_S:
+			continue
+		done = read_status(server.run_dir).get('phase') == 'done'
+		if done or (parent_pid and not pid_alive(parent_pid)):
+			server.shutdown()
+			return
+
+
+def serve(app, parent_pid=None, port=0):
+	"""Serve app until it is idle. Writes the address to <run-dir>/live-url.txt."""
+	server = LiveServer(('127.0.0.1', port), app)
+	(app.run_dir / 'live-url.txt').write_text(f'http://127.0.0.1:{server.server_address[1]}\n')
+	threading.Thread(target=stop_when_idle, args=(server, parent_pid), daemon=True).start()
+	server.serve_forever()
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--run-dir', required=True)
+	ap.add_argument('--parent-pid', type=int)
+	ap.add_argument('--port', type=int, default=0)
+	args = ap.parse_args()
+	serve(BuilderApp(Path(args.run_dir).resolve()), args.parent_pid, args.port)
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/merge_verdicts.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""Merge the blind judge panel into verdicts.json.
+
+  merge_verdicts.py <run-dir>
+
+Reads judge/blind-map.json and judge/verdicts-*.json. Each judge file has one entry for each
+builder letter:
+
+  {"A": {"summary": "...", "goalMet": true, "goalReason": "...", "verdict": "pass",
+         "checks": [{"id": "4", "ok": true, "text": "...", "evidence": [{"node": "...", "path": "...", "value": "..."}]}]}}
+
+For each criterion id, the merged check takes the value that all judges or a majority of the judges
+gave. A split is listed under disagreements. Without a majority, the check is not confirmed (ok null).
+The summary and the findings outside the criteria come from the first judge.
+"""
+
+import json
+import sys
+from collections import Counter
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from build_report import load_case, parse_criteria  # noqa: E402
+
+VERSION = 2
+VOTE_TEXT = {True: 'met', False: 'not met', None: 'not confirmed'}
+
+
+def majority(values):
+	"""The value that more than half of the votes have, or None when no value has a majority."""
+	value, count = Counter(values).most_common(1)[0]
+	return value if count * 2 > len(values) else None
+
+
+def first_check(judge, cid):
+	return next((c for c in judge.get('checks') or [] if str(c.get('id')) == cid), None)
+
+
+def merge_quality(judges):
+	"""The quality checks by majority, in the order of the first judge that answered each check."""
+	ids = list(dict.fromkeys(str(q.get('id')) for j in judges for q in j.get('quality') or [] if q.get('id')))
+	merged = []
+	for qid in ids:
+		found = [next((q for q in j.get('quality') or [] if str(q.get('id')) == qid), None) for j in judges]
+		votes = [q.get('ok') if q else None for q in found]
+		ok = majority(votes)
+		pick = next((q for q, v in zip(found, votes) if q and v == ok), None) or next(q for q in found if q)
+		item = {'id': qid, 'ok': ok, 'text': pick.get('text', ''), 'votes': votes}
+		if len(set(votes)) > 1:
+			item['note'] = f'{Counter(votes)[ok]} of {len(judges)} judges.' if ok is not None else 'Judges disagree.'
+		merged.append(item)
+	return merged
+
+
+def merge_builder(judges, criteria_ids):
+	n = len(judges)
+	lead = judges[0]
+	checks, disagreements = [], []
+	for cid in criteria_ids:
+		found = [first_check(j, cid) for j in judges]
+		votes = [c.get('ok') if c else None for c in found]
+		if all(c is None for c in found):
+			continue
+		ok = majority(votes)
+		pick = next((c for c, v in zip(found, votes) if c and v == ok), None) or next(c for c in found if c)
+		check = {k: v for k, v in pick.items() if k != 'ok'}
+		check.update(id=cid, ok=ok, votes=votes)
+		if len(set(votes)) > 1:
+			counts = ', '.join(f'{count} {VOTE_TEXT[v]}' for v, count in Counter(votes).items())
+			disagreements.append({'id': cid, 'votes': votes, 'texts': [c.get('text') if c else None for c in found]})
+			note = f'Judges disagree ({counts}).' if ok is None else f'{Counter(votes)[ok]} of {n} judges.'
+			check['note'] = note
+		checks.append(check)
+	checks += [dict(c) for c in lead.get('checks') or [] if str(c.get('id')) not in criteria_ids]
+
+	goal_votes = [j.get('goalMet') for j in judges]
+	goal = majority(goal_votes)
+	if goal is None:
+		goal = False if goal_votes.count(False) * 2 >= n else None
+	if len(set(goal_votes)) > 1:
+		reasons = [j.get('goalReason') for j in judges if j.get('goalMet') is False and j.get('goalReason')]
+		disagreements.append({'id': 'goal', 'votes': goal_votes, 'texts': reasons})
+	verdicts = [j.get('verdict') for j in judges if j.get('verdict')]
+	return {
+		'quality': merge_quality(judges),
+		'version': VERSION,
+		'judges': n,
+		'verdict': majority(verdicts) if verdicts else None,
+		'summary': lead.get('summary', ''),
+		'goalMet': goal,
+		'goalReason': next((j.get('goalReason') for j in judges if j.get('goalMet') is goal and j.get('goalReason')), ''),
+		'checks': checks,
+		'disagreements': disagreements,
+	}
+
+
+def main():
+	run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
+	judge_dir = run_dir / 'judge'
+	blind_map = json.loads((judge_dir / 'blind-map.json').read_text())
+	files = sorted(judge_dir.glob('verdicts-*.json'))
+	if not files:
+		sys.exit(f'No judge/verdicts-*.json in {run_dir}.')
+	panel = [json.loads(f.read_text()) for f in files]
+	groups = parse_criteria(load_case(run_dir).get('expected') or '') or []
+	criteria_ids = [num for _, items in groups for num, _ in items]
+
+	merged = {}
+	for letter, builder in blind_map.items():
+		judges = [p[letter] for p in panel if letter in p]
+		if not judges:
+			print(f'{builder}: no judge gave a verdict for builder {letter}')
+			continue
+		merged[builder] = merge_builder(judges, criteria_ids)
+		split = len(merged[builder]['disagreements'])
+		print(f'{builder} (builder {letter}): {len(judges)} judges, {split} disagreement{"" if split == 1 else "s"}')
+	(run_dir / 'verdicts.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False))
+	print(run_dir / 'verdicts.json')
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/n8n_api.py

@@ -0,0 +1,123 @@
+"""Config loader and small REST client for a local n8n instance. Standard library only."""
+
+import http.cookiejar
+import json
+import os
+import stat
+import urllib.error
+import urllib.request
+import uuid
+from functools import cache
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[4]
+DEFAULT_BASE_URL = 'http://localhost:5678'
+TOKEN_FILE = Path.home() / '.config' / 'n8n-builder-compare' / 'mcp-token'
+
+
+def read_env_file(path):
+	"""Return KEY=VALUE pairs from a dotenv file. Values are never printed."""
+	values = {}
+	p = Path(path)
+	if not p.exists():
+		return values
+	for line in p.read_text().splitlines():
+		line = line.strip()
+		if line.startswith('export '):
+			line = line[7:]
+		if not line or line.startswith('#') or '=' not in line:
+			continue
+		key, value = line.split('=', 1)
+		value = value.strip()
+		if value[:1] in ('"', "'"):
+			value = value[1:].split(value[0], 1)[0]
+		else:
+			value = value.split(' #', 1)[0].strip()
+		values[key.strip()] = value
+	return values
+
+
+def env_files():
+	"""Config files from lowest to highest priority."""
+	custom = os.environ.get('AI_SURFACES_ENV')
+	return [REPO_ROOT / '.env.local', Path(custom) if custom else REPO_ROOT / '.env.ai-surfaces']
+
+
+@cache
+def config():
+	"""Merged settings: .env.local, then .env.ai-surfaces, then the process environment."""
+	merged = {}
+	for path in env_files():
+		merged.update({k: v for k, v in read_env_file(path).items() if v != ''})
+	merged.update({k: v for k, v in os.environ.items() if v != ''})
+	return merged
+
+
+def setting(*names, default=None):
+	"""Return the first name that has a value."""
+	values = config()
+	for name in names:
+		if values.get(name):
+			return values[name]
+	return default
+
+
+class N8nClient:
+	def __init__(self):
+		self.base_url = setting('N8N_BASE_URL', default=DEFAULT_BASE_URL).rstrip('/')
+		self.email = setting('N8N_EVAL_EMAIL', default='nathan@n8n.io')
+		self.password = setting('N8N_EVAL_PASSWORD', default='PlaywrightTest123')
+		# The auth cookie is bound to the browser-id header that was sent at login.
+		self.browser_id = str(uuid.uuid4())
+		self.cookies = http.cookiejar.CookieJar()
+		self.opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(self.cookies))
+
+	def headers(self, extra=None):
+		h = {'content-type': 'application/json', 'browser-id': self.browser_id}
+		h.update(extra or {})
+		return h
+
+	def request(self, method, path, body=None, timeout=60):
+		data = json.dumps(body).encode() if body is not None else None
+		req = urllib.request.Request(self.base_url + path, method=method, data=data, headers=self.headers())
+		try:
+			with self.opener.open(req, timeout=timeout) as res:
+				raw = res.read()
+		except urllib.error.HTTPError as e:
+			detail = e.read()[:500].decode(errors='replace')
+			raise RuntimeError(f'{method} {path} -> {e.code}: {detail}') from None
+		parsed = json.loads(raw) if raw else None
+		if isinstance(parsed, dict) and set(parsed) == {'data'}:
+			return parsed['data']
+		return parsed
+
+	def open_stream(self, path):
+		req = urllib.request.Request(
+			self.base_url + path, headers=self.headers({'accept': 'text/event-stream'})
+		)
+		return self.opener.open(req, timeout=600)
+
+	def login(self):
+		self.request('POST', '/rest/login', {'emailOrLdapLoginId': self.email, 'password': self.password})
+		return self
+
+	def workflow(self, workflow_id):
+		return self.request('GET', f'/rest/workflows/{workflow_id}')
+
+
+def save_token(token):
+	TOKEN_FILE.parent.mkdir(parents=True, exist_ok=True)
+	TOKEN_FILE.write_text(token)
+	os.chmod(TOKEN_FILE, stat.S_IRUSR | stat.S_IWUSR)
+
+
+def load_token():
+	"""N8N_MCP_TOKEN from the config wins over the cached token."""
+	token = setting('N8N_MCP_TOKEN')
+	if token:
+		return token
+	return TOKEN_FILE.read_text().strip() if TOKEN_FILE.exists() else None
+
+
+def is_full_token(value):
+	return isinstance(value, str) and value.count('.') == 2 and '*' not in value

.agents/skills/testmax-n8n-ai/scripts/n8n_browser.mjs

@@ -0,0 +1,50 @@
+// Shared browser helpers: settings, Playwright launch, and n8n sign-in.
+// Uses the Playwright copy in the repo test package. Falls back to the local Chrome install.
+import { createRequire } from 'node:module';
+import { existsSync, readFileSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const here = dirname(fileURLToPath(import.meta.url));
+const repo = resolve(here, '../../../..');
+const require = createRequire(join(repo, 'packages/quality/testing/playwright/package.json'));
+const { chromium, expect, selectors } = require('@playwright/test');
+export { expect, selectors };
+// pngjs, as Playwright bundles it. Image diffs use it, so the skills need no extra package.
+export const loadPng = () =>
+	createRequire(require.resolve('@playwright/test'))('playwright-core/lib/utilsBundle').PNG;
+
+// Same order as n8n_api.config(): .env.local, then .env.ai-surfaces, then the environment.
+const settings = {};
+for (const file of [
+	join(repo, '.env.local'),
+	process.env.AI_SURFACES_ENV ?? join(repo, '.env.ai-surfaces'),
+]) {
+	if (!existsSync(file)) continue;
+	for (const line of readFileSync(file, 'utf8').split('\n')) {
+		const match = line.match(/^\s*(?:export\s+)?([A-Z0-9_]+)\s*=\s*(.*)$/);
+		if (!match) continue;
+		const value = match[2].trim().replace(/^(['"])(.*)\1$/, '$2');
+		if (value) settings[match[1]] = value;
+	}
+}
+export const setting = (name, fallback) => process.env[name] || settings[name] || fallback;
+
+export const baseUrl = setting('N8N_BASE_URL', 'http://localhost:5678').replace(/\/$/, '');
+
+export async function launchBrowser() {
+	const launch = existsSync(chromium.executablePath()) ? {} : { channel: 'chrome' };
+	return await chromium.launch({ headless: true, ...launch });
+}
+
+export async function signIn(page) {
+	await page.goto(`${baseUrl}/signin`);
+	await page
+		.getByRole('textbox', { name: 'Email' })
+		.fill(setting('N8N_EVAL_EMAIL', 'nathan@n8n.io'));
+	await page
+		.getByRole('textbox', { name: 'Password' })
+		.fill(setting('N8N_EVAL_PASSWORD', 'PlaywrightTest123'));
+	await page.getByRole('button', { name: 'Sign in', exact: true }).click();
+	await page.waitForURL((url) => !url.pathname.startsWith('/signin'), { timeout: 30_000 });
+}

.agents/skills/testmax-n8n-ai/scripts/prepare.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+"""Check the local setup and make the n8n MCP token available.
+
+Prints a JSON status object. It never prints a token or a key.
+
+  prepare.py            check everything, reuse the cached MCP token
+  prepare.py --rotate   create a new MCP key (this breaks other clients that use the old key)
+"""
+
+import argparse
+import json
+import shutil
+import subprocess
+import sys
+import urllib.request
+
+from n8n_api import TOKEN_FILE, N8nClient, env_files, is_full_token, load_token, save_token, setting
+
+
+def mcp_handshake(base_url, token):
+	body = json.dumps({
+		'jsonrpc': '2.0', 'id': 1, 'method': 'initialize',
+		'params': {'protocolVersion': '2025-03-26', 'capabilities': {},
+		           'clientInfo': {'name': 'testmax-n8n-ai', 'version': '1'}},
+	}).encode()
+	req = urllib.request.Request(base_url + '/mcp-server/http', data=body, method='POST', headers={
+		'content-type': 'application/json',
+		'accept': 'application/json, text/event-stream',
+		'authorization': f'Bearer {token}',
+	})
+	try:
+		with urllib.request.urlopen(req, timeout=20) as res:
+			return res.status == 200
+	except Exception:
+		return False
+
+
+def docker_ready():
+	if not shutil.which('docker'):
+		return False
+	return subprocess.run(['docker', 'info'], capture_output=True).returncode == 0
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--rotate', action='store_true')
+	args = ap.parse_args()
+
+	status = {
+		'configFiles': [str(p) for p in env_files() if p.exists()],
+		'tokenFile': str(TOKEN_FILE),
+		'builders': setting('COMPARE_BUILDERS', default='iai,claude,codex'),
+		'keys': {
+			'claude': bool(setting('CLAUDE_API_KEY', 'ANTHROPIC_API_KEY')),
+			'codex': bool(setting('CODEX_API_KEY', 'OPENAI_API_KEY')),
+			'langsmith': bool(setting('LANGSMITH_API_KEY')),
+		},
+	}
+	client = N8nClient()
+	status['baseUrl'] = client.base_url
+	try:
+		client.login()
+		status['login'] = True
+	except Exception as e:
+		status['login'] = False
+		status['error'] = str(e)
+		print(json.dumps(status, indent=2))
+		sys.exit(1)
+
+	try:
+		settings = client.request('GET', '/rest/instance-ai/settings')
+		status['instanceAiEnabled'] = bool(settings.get('enabled'))
+	except Exception:
+		status['instanceAiEnabled'] = False
+
+	client.request('PATCH', '/rest/mcp/settings', {'mcpAccessEnabled': True})
+	status['mcpAccessEnabled'] = True
+
+	token = None
+	if args.rotate:
+		if setting('N8N_MCP_TOKEN'):
+			# The clients would keep the configured token, which the rotation makes invalid.
+			sys.exit('Remove N8N_MCP_TOKEN from the config before you rotate the MCP key, then run prepare.py --rotate again.')
+		token = client.request('POST', '/rest/mcp/api-key/rotate')['apiKey']
+		save_token(token)
+	else:
+		current = client.request('GET', '/rest/mcp/api-key')['apiKey']
+		if is_full_token(current):
+			# A new key was created for this user, so the full value is visible once.
+			save_token(current)
+		token = load_token()
+	status['mcpTokenSource'] = 'N8N_MCP_TOKEN' if setting('N8N_MCP_TOKEN') else 'cache'
+
+	status['mcpTokenAvailable'] = bool(token)
+	status['mcpTokenWorks'] = bool(token) and mcp_handshake(client.base_url, token)
+	if not status['mcpTokenWorks']:
+		status['mcpTokenHelp'] = (
+			'The MCP token is missing or invalid. Set N8N_MCP_TOKEN in .env.ai-surfaces, or run '
+			'prepare.py --rotate after the user agrees. Rotation invalidates the old key in every other MCP client.'
+		)
+
+	status['docker'] = docker_ready()
+	status['claudeOnHost'] = bool(shutil.which('claude'))
+	status['codexOnHost'] = bool(shutil.which('codex'))
+	print(json.dumps(status, indent=2))
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/record_iai.mjs

@@ -0,0 +1,82 @@
+#!/usr/bin/env node
+// Record a video of an Instance AI thread in the n8n UI while the run continues.
+//
+//   node record_iai.mjs <out-dir> [--live]
+//
+// --live also saves the screen to <out-dir>/live.png every few seconds, for the live page.
+//
+// Protocol on stdin and stdout, one line each:
+//   prints "ready" after sign-in
+//   reads  "open <thread-url>" and opens the thread
+//   reads  "stop", or stdin closes: keeps the last screen for a moment, then saves <out-dir>/screen.webm
+//                                   and the last screen as <out-dir>/screen-poster.png
+// The sign-in happens in a separate context, so the video starts at the thread.
+import { copyFileSync, mkdtempSync, renameSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join, resolve } from 'node:path';
+import { createInterface } from 'node:readline';
+
+import { launchBrowser, signIn } from './n8n_browser.mjs';
+
+const outDir = resolve(process.argv[2] ?? '.');
+const live = process.argv.includes('--live');
+const SIZE = { width: 1600, height: 1000 };
+const LIVE_SHOT_MS = 2500;
+
+const browser = await launchBrowser();
+const login = await browser.newContext();
+const loginPage = await login.newPage();
+await signIn(loginPage);
+const storageState = await login.storageState();
+await login.close();
+
+const videoDir = mkdtempSync(join(tmpdir(), 'cwb-video-'));
+const context = await browser.newContext({
+	viewport: SIZE,
+	storageState,
+	recordVideo: { dir: videoDir, size: SIZE },
+});
+const page = await context.newPage();
+console.log('ready');
+
+let liveTimer;
+let shooting = false;
+async function saveLiveShot() {
+	if (shooting) return;
+	shooting = true;
+	// The live page can read the file at any time, so it gets only complete images.
+	const partial = join(outDir, 'live.tmp.png');
+	try {
+		await page.screenshot({ path: partial });
+		renameSync(partial, join(outDir, 'live.png'));
+	} catch {
+		// The page can be in the middle of a navigation. The next interval tries again.
+	} finally {
+		shooting = false;
+	}
+}
+
+const lines = createInterface({ input: process.stdin });
+for await (const line of lines) {
+	if (line.startsWith('open ')) {
+		await page.goto(line.slice(5).trim(), { waitUntil: 'domcontentloaded' });
+		console.log('opened');
+		if (live && !liveTimer) liveTimer = setInterval(saveLiveShot, LIVE_SHOT_MS);
+	} else if (line.trim() === 'stop') {
+		break;
+	}
+}
+
+clearInterval(liveTimer);
+await page.waitForTimeout(3000);
+if (live) await saveLiveShot();
+// The video starts on a blank page, so the report shows the last screen until playback starts.
+await page.screenshot({ path: join(outDir, 'screen-poster.png') }).catch(() => {});
+const video = page.video();
+await context.close();
+if (video) {
+	copyFileSync(await video.path(), join(outDir, 'screen.webm'));
+	console.log('saved screen.webm');
+}
+await browser.close();
+rmSync(videoDir, { recursive: true, force: true });

.agents/skills/testmax-n8n-ai/scripts/request_recorder.py

@@ -0,0 +1,192 @@
+"""Record the request bodies that a CLI sends to its model API. Standard library only.
+
+The recorder is a local HTTP proxy. The CLI sends its model requests to the recorder, and the
+recorder sends them to the real API without changes. It writes the request bodies to a JSONL file.
+It never writes headers, so the API key stays out of the file.
+
+Each row has the request settings without the conversation input. The tools and the instructions
+are in a row only when they differ from the previous request of the same thread, because the CLI
+sends them every turn.
+The tools field also holds the tools of additional_tools input items.
+"""
+
+import gzip
+import http.client
+import json
+import threading
+import time
+import zlib
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from urllib.parse import urlparse
+
+# Headers that apply to one connection only. The recorder sets its own values.
+HOP_HEADERS = {'connection', 'keep-alive', 'proxy-connection', 'proxy-authorization', 'te', 'trailer',
+               'transfer-encoding', 'upgrade', 'host', 'content-length', 'accept-encoding'}
+CHANGING_KEYS = ('tools', 'instructions')
+
+
+def decode_body(data, encoding):
+	encoding = (encoding or '').lower()
+	if encoding == 'gzip':
+		return gzip.decompress(data)
+	if encoding == 'deflate':
+		return zlib.decompress(data)
+	if encoding == 'zstd':
+		from compression import zstd
+		return zstd.decompress(data)
+	return data
+
+
+class RequestRecorder:
+	"""Start with start(). Point the CLI at base_url(host). Call stop() when the CLI is done."""
+
+	def __init__(self, upstream, out_file, bind='127.0.0.1'):
+		self.upstream = urlparse(upstream)
+		self.out_file = out_file
+		self.lock = threading.Lock()
+		self.idle = threading.Condition(self.lock)
+		self.active = 0
+		self.last = {}
+		self.count = 0
+		self.started = time.time()
+		self.server = ThreadingHTTPServer((bind, 0), self.handler())
+		self.server.daemon_threads = True
+		self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
+
+	@property
+	def port(self):
+		return self.server.server_address[1]
+
+	def base_url(self, host):
+		return f'http://{host}:{self.port}{self.upstream.path.rstrip("/")}'
+
+	def start(self):
+		self.out_file.parent.mkdir(parents=True, exist_ok=True)
+		self.out_file.write_text('')
+		self.thread.start()
+		return self
+
+	def stop(self, timeout=30):
+		"""Stop the server. Wait until the open requests are written, so count is final."""
+		self.server.shutdown()
+		with self.idle:
+			self.idle.wait_for(lambda: self.active == 0, timeout=timeout)
+		self.server.server_close()
+
+	def begin(self):
+		with self.lock:
+			self.active += 1
+
+	def end(self):
+		with self.idle:
+			self.active -= 1
+			self.idle.notify_all()
+
+	def record(self, method, path, body, encoding, status, error_text):
+		row = {'t': round(time.time() - self.started, 3), 'method': method, 'path': path, 'status': status}
+		try:
+			payload = json.loads(decode_body(body, encoding)) if body else None
+		except (ValueError, OSError, ImportError):
+			payload, row['bodyBytes'] = None, len(body)
+		with self.lock:
+			if isinstance(payload, dict):
+				items = payload.get('input') if isinstance(payload.get('input'), list) else []
+				if items:
+					row['inputItems'] = len(items)
+				# Codex sends its tools as an additional_tools input item, not in the top-level tools field.
+				extra = [t for i in items if isinstance(i, dict) and i.get('type') == 'additional_tools'
+				         for t in i.get('tools') or []]
+				if extra:
+					payload = {**payload, 'tools': (payload.get('tools') or []) + extra}
+				# Side tasks, such as a title request, run on their own thread with other tools.
+				metadata = payload.get('client_metadata') if isinstance(payload.get('client_metadata'), dict) else {}
+				thread = metadata.get('thread_id') or payload.get('prompt_cache_key') or ''
+				for key, value in payload.items():
+					if key == 'input':
+						continue
+					if key in CHANGING_KEYS:
+						if self.last.get((thread, key)) == value:
+							continue
+						self.last[(thread, key)] = value
+					row[key] = value
+			if error_text:
+				row['error'] = error_text
+			self.count += 1
+			with self.out_file.open('a') as f:
+				f.write(json.dumps(row, ensure_ascii=False) + '\n')
+
+	def handler(self):
+		recorder = self
+
+		class Handler(BaseHTTPRequestHandler):
+			protocol_version = 'HTTP/1.1'
+
+			def log_message(self, *args):
+				pass
+
+			def read_body(self):
+				if self.headers.get('Content-Length'):
+					return self.rfile.read(int(self.headers['Content-Length']))
+				if 'chunked' not in (self.headers.get('Transfer-Encoding') or '').lower():
+					return b''
+				parts = []
+				while True:
+					size = int(self.rfile.readline().split(b';')[0].strip() or b'0', 16)
+					if not size:
+						self.rfile.readline()
+						return b''.join(parts)
+					parts.append(self.rfile.read(size))
+					self.rfile.readline()
+
+			def forward(self):
+				recorder.begin()
+				try:
+					self.forward_and_record()
+				finally:
+					recorder.end()
+
+			def forward_and_record(self):
+				body = self.read_body()
+				up = recorder.upstream
+				conn_class = http.client.HTTPSConnection if up.scheme == 'https' else http.client.HTTPConnection
+				conn = conn_class(up.hostname, up.port, timeout=900)
+				headers = {k: v for k, v in self.headers.items() if k.lower() not in HOP_HEADERS}
+				headers['Host'] = up.netloc
+				headers['Content-Length'] = str(len(body))
+				status, error_text, replied = 502, None, False
+				try:
+					conn.request(self.command, self.path, body=body, headers=headers)
+					resp = conn.getresponse()
+					status, replied = resp.status, True
+					self.send_response(resp.status, resp.reason)
+					for key, value in resp.getheaders():
+						if key.lower() not in HOP_HEADERS:
+							self.send_header(key, value)
+					self.send_header('Transfer-Encoding', 'chunked')
+					self.end_headers()
+					kept = b''
+					while chunk := resp.read1(65536):
+						if status >= 400 and len(kept) < 4000:
+							kept += chunk
+						self.wfile.write(b'%x\r\n%s\r\n' % (len(chunk), chunk))
+						self.wfile.flush()
+					self.wfile.write(b'0\r\n\r\n')
+					self.wfile.flush()
+					if kept:
+						error_text = kept[:4000].decode('utf-8', 'replace')
+				except (OSError, http.client.HTTPException) as e:
+					error_text = f'recorder: {type(e).__name__}'
+					if not replied:
+						try:
+							self.send_error(502, 'Recorder could not reach the API')
+						except OSError:
+							pass
+					self.close_connection = True
+				finally:
+					conn.close()
+					recorder.record(self.command, self.path, body, self.headers.get('Content-Encoding'),
+					                status, error_text)
+
+			do_GET = do_POST = do_PUT = do_DELETE = do_PATCH = forward
+
+		return Handler

.agents/skills/testmax-n8n-ai/scripts/run_cli_builder.py

@@ -0,0 +1,613 @@
+#!/usr/bin/env python3
+"""Run one prompt through Claude Code or Codex with only the n8n MCP server.
+
+  run_cli_builder.py --builder claude --out <run-dir>/claude --prompt-file <run-dir>/prompt.txt
+  run_cli_builder.py --builder codex  --out <run-dir>/codex  --prompt-file <run-dir>/prompt.txt
+
+Isolation modes:
+  container  (default) Docker container. The only mount is the output directory.
+             The builder cannot see host files, skills, memory or settings.
+  clean      Claude only. A clean process with an empty config directory, no built-in
+             tools and only the n8n MCP server. It needs an API key, not a subscription login.
+
+Screen modes:
+  interactive  (default in a container) The normal terminal screen of the CLI runs in tmux.
+               screen.cast records the exact screen, and a terminal window can watch it live.
+               The trace comes from the session file of the CLI.
+  headless     `claude -p` or `codex exec` with a JSON event stream. Clean isolation uses this mode.
+
+Secrets: the model key and the MCP token go through environment variables only.
+They are never written to a file or put on a command line.
+
+Codex sends its model requests through a local recorder (request_recorder.py), which writes the
+request bodies, but not the headers, to requests.jsonl. Set COMPARE_RECORD_REQUESTS=0 to turn it off.
+"""
+
+import argparse
+import hashlib
+import json
+import os
+import re
+import shutil
+import subprocess
+import sys
+import threading
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from urllib.parse import urlparse
+
+from live import enabled as live_enabled
+from live import open_log_window, open_terminal
+from n8n_api import DEFAULT_BASE_URL, load_token, setting
+from request_recorder import RequestRecorder
+
+SKILL_DIR = Path(__file__).resolve().parents[1]
+IMAGE = 'n8n-builder-sandbox'
+# The builder-specific name comes first, so the CLI can use a different key than
+# the ANTHROPIC_API_KEY or OPENAI_API_KEY that the instance reads from .env.local.
+KEY_SETTINGS = {'claude': ['CLAUDE_API_KEY', 'ANTHROPIC_API_KEY'], 'codex': ['CODEX_API_KEY', 'OPENAI_API_KEY']}
+# Codex features that add tools or downloads beyond the n8n MCP server. shell_snapshot is off
+# because it writes the environment, with the API key, to a file.
+CODEX_FEATURES_OFF = ['apps', 'browser_use', 'browser_use_external', 'computer_use', 'image_generation',
+                      'in_app_browser', 'plugins', 'remote_plugin', 'shell_snapshot']
+SCREEN = {'cols': 120, 'rows': 36}
+OPENAI_BASE_URL = 'https://api.openai.com/v1'
+# Interactive mode: the variable that the CLI reads the model key from.
+SANDBOX_KEY = {'claude': 'CLAUDE_SANDBOX_KEY', 'codex': 'CODEX_SANDBOX_KEY'}
+IDLE_LIMIT_S = 600
+
+
+def now_iso():
+	return datetime.now(timezone.utc).isoformat()
+
+
+def ensure_image(rebuild):
+	# The label holds a hash of the Dockerfile, so an edit to the Dockerfile rebuilds the image.
+	digest = hashlib.sha256((SKILL_DIR / 'sandbox' / 'Dockerfile').read_bytes()).hexdigest()[:16]
+	found = subprocess.run(['docker', 'image', 'inspect', '--format',
+	                        '{{index .Config.Labels "dockerfile-sha"}}', IMAGE], capture_output=True, text=True)
+	if found.returncode == 0 and found.stdout.strip() == digest and not rebuild:
+		return
+	print(f'building docker image {IMAGE}', flush=True)
+	subprocess.run(['docker', 'build', '--label', f'dockerfile-sha={digest}', '-t', IMAGE,
+	                str(SKILL_DIR / 'sandbox')], check=True)
+
+
+def mcp_url(base_url, isolation):
+	if isolation != 'container':
+		return f'{base_url}/mcp-server/http'
+	parsed = urlparse(base_url)
+	port = parsed.port or (443 if parsed.scheme == 'https' else 80)
+	host = 'host.docker.internal' if parsed.hostname in ('localhost', '127.0.0.1') else parsed.hostname
+	return f'{parsed.scheme}://{host}:{port}/mcp-server/http'
+
+
+def claude_tool_args(tools):
+	# 'default' gives the normal Claude Code tools, all allowed without a question, like Codex in the
+	# container. Only a container may use it, because Bash and Write can change the files they see.
+	if tools == 'default':
+		return ['--permission-mode', 'bypassPermissions']
+	# dontAsk allows the pre-approved n8n tools and denies the rest without a prompt.
+	return ['--tools', '', '--allowedTools', 'mcp__n8n', '--permission-mode', 'dontAsk']
+
+
+def claude_args(prompt, model, config_path, bare, tools='none'):
+	# --bare also replaces the system prompt with a minimal one. A container already isolates the CLI,
+	# so only the clean mode, which runs on the host, uses it.
+	args = [
+		'claude', '-p', prompt, *(['--bare'] if bare else []),
+		'--strict-mcp-config', '--mcp-config', config_path, *claude_tool_args(tools),
+		'--disable-slash-commands', '--output-format', 'stream-json', '--verbose',
+	]
+	if model:
+		args += ['--model', model]
+	return args
+
+
+def claude_interactive_args(model, tools='none'):
+	args = [
+		'claude', '--strict-mcp-config', '--mcp-config', '/out/mcp.json', *claude_tool_args(tools),
+		'--disable-slash-commands', '--settings', '/out/.screen/claude-settings.json',
+	]
+	if model:
+		args += ['--model', model]
+	return args
+
+
+def codex_args(prompt, model):
+	args = ['codex', 'exec', '--json', '--skip-git-repo-check', '--sandbox', 'read-only']
+	if model:
+		args += ['--model', model]
+	return args + [prompt]
+
+
+def codex_interactive_args(model):
+	# The background app server needs `ps`, which the slim image does not have.
+	args = ['codex', '--no-daemon']
+	if model:
+		args += ['--model', model]
+	return args
+
+
+def write_codex_config(home, url, key_env=None, api_base=OPENAI_BASE_URL):
+	"""Write config.toml. With key_env, a provider reads the key from that variable, so the
+	interactive screen does not ask for a sign-in and no key is stored in auth.json."""
+	codex_home = home / '.codex'
+	codex_home.mkdir(parents=True, exist_ok=True)
+	lines = ['approval_policy = "never"', 'sandbox_mode = "read-only"']
+	if key_env:
+		lines.append('model_provider = "sandbox-openai"')
+	lines += ['', '[features]'] + [f'{name} = false' for name in CODEX_FEATURES_OFF]
+	if key_env:
+		lines += ['', '[model_providers.sandbox-openai]', 'name = "OpenAI"',
+		          f'base_url = "{api_base}"', f'env_key = "{key_env}"', 'wire_api = "responses"']
+	lines += ['', '[projects."/out/work"]', 'trust_level = "trusted"',
+	          '', '[mcp_servers.n8n]', f'url = "{url}"', 'bearer_token_env_var = "N8N_MCP_TOKEN"', '']
+	(codex_home / 'config.toml').write_text('\n'.join(lines))
+
+
+def write_claude_interactive_config(out, home, tools):
+	# Skip the first-run screens: theme, onboarding and the folder trust question.
+	(home / '.claude' / '.claude.json').write_text(json.dumps({
+		'hasCompletedOnboarding': True, 'theme': 'dark',
+		'projects': {'/out/work': {'hasTrustDialogAccepted': True, 'hasCompletedProjectOnboarding': True}},
+	}))
+	# With ANTHROPIC_API_KEY the screen asks to confirm the key and shows part of it.
+	# apiKeyHelper reads the key from the environment without that question.
+	settings = {'apiKeyHelper': f'printenv {SANDBOX_KEY["claude"]}'}
+	if tools == 'default':
+		# The bypass-mode warning screen also has a ❯ marker, so the prompt would answer "No, exit".
+		settings['skipDangerousModePermissionPrompt'] = True
+	(out / '.screen' / 'claude-settings.json').write_text(json.dumps(settings))
+
+
+def live_line(builder, event):
+	"""One short terminal line for an interesting stream event, or None."""
+	if builder == 'claude':
+		if event.get('subtype') == 'permission_denied':
+			return f'denied {event.get("tool_name")}'
+		if event.get('type') == 'assistant':
+			for block in (event.get('message') or {}).get('content') or []:
+				if block.get('type') == 'tool_use':
+					return f'tool {block.get("name")}'
+				if block.get('type') == 'text' and block.get('text', '').strip():
+					return 'text ' + block['text'].strip().splitlines()[0][:120]
+		if event.get('type') == 'result':
+			return f'result {event.get("subtype")} turns={event.get("num_turns")}'
+	else:
+		item = event.get('item') or {}
+		if event.get('type') == 'item.started' and item.get('type') == 'mcp_tool_call':
+			return f'tool {item.get("server")}.{item.get("tool")}'
+		if event.get('type') == 'item.completed' and item.get('type') == 'agent_message':
+			return 'text ' + (item.get('text') or '').strip()[:120]
+		if event.get('type') in ('turn.completed', 'turn.failed', 'error'):
+			return event['type']
+	return None
+
+
+def session_lines(builder, row):
+	"""Short terminal lines for one session-file row, and the end state: None, 'done' or 'error'."""
+	lines = []
+	if builder == 'claude':
+		if row.get('type') != 'assistant':
+			return lines, None
+		msg = row.get('message') or {}
+		for block in msg.get('content') or []:
+			if block.get('type') == 'tool_use':
+				lines.append(f'tool {block.get("name")}')
+			elif block.get('type') == 'text' and block.get('text', '').strip():
+				lines.append('text ' + block['text'].strip().splitlines()[0][:120])
+		if row.get('isApiErrorMessage'):
+			return lines, 'error'
+		return lines, 'done' if msg.get('stop_reason') == 'end_turn' else None
+	payload = row.get('payload') or {}
+	if row.get('type') != 'event_msg':
+		return lines, None
+	kind, item = payload.get('type'), payload.get('item') or {}
+	if kind == 'item_completed' and item.get('type') == 'McpToolCall':
+		lines.append(f'tool {item.get("server")}.{item.get("tool")}')
+	elif kind == 'item_completed' and item.get('type') == 'AgentMessage':
+		text = ' '.join(c.get('text', '') for c in item.get('content') or [] if isinstance(c, dict))
+		lines.append('text ' + text.strip()[:120])
+	elif kind == 'error':
+		lines.append(f'error {str(payload.get("message"))[:200]}')
+	return lines, {'task_complete': 'done', 'turn_aborted': 'error'}.get(kind)
+
+
+BOLD, DIM, CYAN, GREEN, RED, RESET = '\033[1m', '\033[2m', '\033[36m', '\033[32m', '\033[31m', '\033[0m'
+LABELS = {'claude': 'Claude Code + n8n MCP', 'codex': 'Codex + n8n MCP'}
+
+
+def short(value, limit=400):
+	"""Readable one-line text for a tool input or result."""
+	if isinstance(value, dict) and isinstance(value.get('content'), list):
+		value = value['content']
+	if isinstance(value, list) and all(isinstance(v, dict) and 'text' in v for v in value):
+		value = ' '.join(v['text'] for v in value)
+	text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
+	text = ' '.join(text.split())
+	return text if len(text) <= limit else text[:limit] + ' …'
+
+
+def log_entries(builder, event):
+	"""Detailed, colored lines for live.log, the file that the headless live window follows."""
+	lines = []
+	if builder == 'claude':
+		# A permission_denied event has a string message, not an API message.
+		if event.get('subtype') == 'permission_denied':
+			lines.append(f'{RED}denied {event.get("tool_name")}{RESET}')
+		message = event.get('message')
+		content = message.get('content') if isinstance(message, dict) else None
+		for block in content if isinstance(content, list) else []:
+			kind = block.get('type')
+			if kind == 'thinking' and block.get('thinking', '').strip():
+				lines.append(f'{DIM}thinking: {short(block["thinking"], 600)}{RESET}')
+			elif kind == 'text' and block.get('text', '').strip():
+				lines.append(block['text'].strip())
+			elif kind == 'tool_use':
+				name = block.get('name', '').removeprefix('mcp__n8n__')
+				lines.append(f'{BOLD}{CYAN}→ {name}{RESET} {short(block.get("input"))}')
+			elif kind == 'tool_result':
+				color = RED if block.get('is_error') else DIM
+				lines.append(f'{color}← {short(block.get("content"))}{RESET}')
+		if event.get('type') == 'result':
+			cost = event.get('total_cost_usd')
+			lines.append(f'{GREEN}done: {event.get("subtype")}, {event.get("num_turns")} turns'
+			             + (f', ${cost:.2f} (Claude Code estimate)' if cost is not None else '') + RESET)
+	else:
+		item, etype = event.get('item') or {}, event.get('type')
+		kind = item.get('type')
+		if etype == 'item.started' and kind == 'mcp_tool_call':
+			lines.append(f'{BOLD}{CYAN}→ {item.get("tool")}{RESET} {short(item.get("arguments"))}')
+		elif etype == 'item.completed' and kind == 'mcp_tool_call':
+			if item.get('error'):
+				lines.append(f'{RED}← {short(item["error"])}{RESET}')
+			else:
+				lines.append(f'{DIM}← {short(item.get("result"))}{RESET}')
+		elif etype == 'item.completed' and kind == 'reasoning' and item.get('text'):
+			lines.append(f'{DIM}thinking: {short(item["text"], 600)}{RESET}')
+		elif etype == 'item.completed' and kind == 'agent_message' and item.get('text'):
+			lines.append(item['text'].strip())
+		elif etype == 'turn.completed':
+			lines.append(f'{GREEN}done: turn completed{RESET}')
+		elif etype in ('turn.failed', 'error'):
+			lines.append(f'{RED}{etype}: {short(event.get("error") or event.get("message"))}{RESET}')
+	return lines
+
+
+WORKFLOW_ID = re.compile(r'(?:workflowId|WORKFLOW_ID)\\*"?\s*[:=]\s*\\*"?([A-Za-z0-9]{10,})')
+
+
+def session_candidates(builder, home):
+	if builder == 'claude':
+		# Subagent transcripts are in deeper folders. The main session is at projects/<cwd>/<id>.jsonl.
+		root = home / '.claude' / 'projects'
+		return list(root.glob('*/*.jsonl')) if root.exists() else []
+	root = home / '.codex' / 'sessions'
+	return list(root.rglob('rollout-*.jsonl')) if root.exists() else []
+
+
+def find_session_file(builder, home, session_id=None):
+	matches = session_candidates(builder, home)
+	if session_id:
+		matches = [p for p in matches if session_id in p.name] or matches
+	matches.sort(key=lambda p: p.stat().st_mtime)
+	return matches[-1] if matches else None
+
+
+class SessionReader:
+	"""Reads the rows that the CLI appends to its session file."""
+
+	def __init__(self, builder, home):
+		self.builder, self.home, self.path, self.offset = builder, home, None, 0
+
+	def rows(self):
+		self.path = self.path or find_session_file(self.builder, self.home)
+		if not self.path:
+			return []
+		with self.path.open('rb') as f:
+			f.seek(self.offset)
+			data = f.read()
+		end = data.rfind(b'\n') + 1
+		self.offset += end
+		rows = []
+		for line in data[:end].splitlines():
+			try:
+				rows.append(json.loads(line))
+			except json.JSONDecodeError:
+				continue
+		return rows
+
+
+def docker_command(container, out, env_names):
+	cmd = [
+		'docker', 'run', '--rm', '-i', '--name', container,
+		'--user', f'{os.getuid()}:{os.getgid()}',
+		'--cap-drop', 'ALL', '--security-opt', 'no-new-privileges',
+		'--read-only', '--tmpfs', '/tmp',
+		'--add-host', 'host.docker.internal:host-gateway',
+		'-v', f'{out}:/out', '-w', '/out/work',
+		'-e', 'HOME=/out/home',
+		'-e', 'CLAUDE_CONFIG_DIR=/out/home/.claude',
+		'-e', 'CODEX_HOME=/out/home/.codex',
+		'-e', 'CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1',
+		'-e', 'COLORTERM=truecolor',
+	]
+	# `-e NAME` without a value copies the value from this process, so secrets stay off argv.
+	for name in env_names:
+		cmd += ['-e', name]
+	return cmd + [IMAGE]
+
+
+def check_no_secrets(out, secrets):
+	"""Warn if a secret value ended up in a file of the run directory. Never prints the value."""
+	needles = [s.encode() for s in secrets if s and len(s) >= 12]
+	for path in out.rglob('*'):
+		if not path.is_file() or path.stat().st_size > 50_000_000:
+			continue
+		data = path.read_bytes()
+		if any(n in data for n in needles):
+			print(f'WARNING: a secret value is in {path}. Delete the file before you share the run.', flush=True)
+
+
+def run_headless(args, out, prompt, cmd, run_env, cwd, container):
+	stream_path = out / 'stream.jsonl'
+	session_id, workflow_ids, result, errors_in_row = None, [], None, 0
+	live_log = (out / 'live.log').open('w', buffering=1)
+	live_log.write(f'{BOLD}{LABELS[args.builder]}{RESET}  model: {args.model or "CLI default"}, '
+	               f'isolation: {args.isolation}\n{DIM}prompt: {short(prompt, 600)}{RESET}\n\n')
+	open_log_window(LABELS[args.builder], out / 'live.log')
+	# The Codex stream has no timestamps, so stream.times keeps the arrival time of each line.
+	with stream_path.open('w') as stream, (out / 'stream.times').open('w') as times, \
+			(out / 'stderr.log').open('w') as err, live_log:
+		proc = subprocess.Popen(cmd, env=run_env, cwd=cwd, stdin=subprocess.DEVNULL,
+		                        stdout=subprocess.PIPE, stderr=err, text=True)
+		watchdog = threading.Timer(args.timeout_min * 60, proc.kill)
+		watchdog.start()
+		for line in proc.stdout:
+			try:
+				event = json.loads(line)
+			except json.JSONDecodeError:
+				continue
+			stream.write(line)
+			stream.flush()
+			times.write(f'{time.time():.3f}\n')
+			if event.get('type') == 'system' and event.get('subtype') == 'init':
+				session_id = event.get('session_id')
+			if event.get('type') == 'thread.started':
+				session_id = event.get('thread_id')
+			if event.get('type') == 'result':
+				result = event
+			for match in WORKFLOW_ID.findall(line):
+				if match not in workflow_ids:
+					workflow_ids.append(match)
+			text = live_line(args.builder, event)
+			if text:
+				print(f'[{args.builder}] {text}', flush=True)
+			for entry in log_entries(args.builder, event):
+				live_log.write(entry + '\n\n')
+			errors_in_row = errors_in_row + 1 if event.get('type') == 'error' else 0
+			if errors_in_row >= 10:
+				print(f'[{args.builder}] stopped after 10 errors in a row: {event.get("message", "")[:200]}', flush=True)
+				proc.kill()
+				break
+		proc.wait()
+		if not watchdog.is_alive():
+			print('timeout: killed the builder', flush=True)
+		watchdog.cancel()
+		if args.isolation == 'container':
+			subprocess.run(['docker', 'kill', container], capture_output=True)
+		live_log.write(f'{BOLD}=== finished, exit code {proc.returncode}. You can close this window. ==={RESET}\n')
+	return {'exitCode': proc.returncode, 'sessionId': session_id, 'workflowIds': workflow_ids,
+	        'reportedCostUsd': (result or {}).get('total_cost_usd')}
+
+
+def run_interactive(args, out, home, prompt, run_env, container):
+	screen_dir = out / '.screen'
+	argv = (claude_interactive_args(args.model, args.claude_tools) if args.builder == 'claude'
+	        else codex_interactive_args(args.model))
+	# Claude Code shows its input box before the MCP servers connect, so it waits a few seconds more.
+	ready = {'claude': ('❯', 5000), 'codex': ('›', 2000)}[args.builder]
+	(screen_dir / 'command.json').write_text(json.dumps(
+		{'argv': argv, **SCREEN, 'input': prompt, 'readyText': ready[0], 'settleMs': ready[1]}))
+	cmd = docker_command(container, out, [n for n in run_env if n in (SANDBOX_KEY[args.builder], 'N8N_MCP_TOKEN')])
+	cmd += ['node', '/out/.screen/screen.mjs', 'run', '/out/.screen']
+
+	reader, workflow_ids, state, timed_out = SessionReader(args.builder, home), [], None, False
+	with (out / 'stderr.log').open('w') as err:
+		proc = subprocess.Popen(cmd, env=run_env, stdin=subprocess.DEVNULL, stdout=err, stderr=err)
+		for _ in range(120):
+			if (screen_dir / 'screen.cast').exists() or proc.poll() is not None:
+				break
+			time.sleep(0.25)
+		watch = f'docker exec -it {container} tmux attach -r -t run'
+		if live_enabled() and not open_terminal(LABELS[args.builder], watch, SCREEN['cols'], SCREEN['rows']):
+			print(f'watch {LABELS[args.builder]}: {watch}', flush=True)
+		deadline, last_growth = time.time() + args.timeout_min * 60, time.time()
+		while proc.poll() is None and not state:
+			rows = reader.rows()
+			if rows:
+				last_growth = time.time()
+			for row in rows:
+				lines, row_state = session_lines(args.builder, row)
+				for line in lines:
+					print(f'[{args.builder}] {line}', flush=True)
+				for match in WORKFLOW_ID.findall(json.dumps(row)):
+					if match not in workflow_ids:
+						workflow_ids.append(match)
+				state = state or row_state
+			if time.time() > deadline:
+				print('timeout: stopped the builder', flush=True)
+				timed_out = True
+				break
+			if reader.path and time.time() - last_growth > IDLE_LIMIT_S:
+				print(f'no progress for {IDLE_LIMIT_S // 60} minutes: stopped the builder', flush=True)
+				timed_out = True
+				break
+			time.sleep(1)
+		(screen_dir / 'stop').write_text('')
+		try:
+			proc.wait(timeout=30)
+		except subprocess.TimeoutExpired:
+			subprocess.run(['docker', 'kill', container], capture_output=True)
+			proc.wait()
+	for row in reader.rows():
+		for match in WORKFLOW_ID.findall(json.dumps(row)):
+			if match not in workflow_ids:
+				workflow_ids.append(match)
+	if (screen_dir / 'screen.cast').exists():
+		shutil.move(screen_dir / 'screen.cast', out / 'screen.cast')
+	# After a finished turn the driver stops tmux, so the exit code of the CLI means nothing.
+	exited = screen_dir / 'exited'
+	if state == 'done' and not timed_out:
+		code = 0
+	else:
+		code = int(exited.read_text() or 1) if exited.exists() else 1
+		code = code or 1
+	session = reader.path
+	return {'exitCode': code, 'sessionId': session.stem.split('rollout-')[-1] if session else None,
+	        'workflowIds': workflow_ids, 'reportedCostUsd': None}
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--builder', choices=['claude', 'codex'], required=True)
+	ap.add_argument('--out', required=True)
+	ap.add_argument('--prompt-file', required=True)
+	ap.add_argument('--isolation', choices=['container', 'clean'],
+	                default=setting('COMPARE_ISOLATION', default='container'))
+	ap.add_argument('--mode', choices=['interactive', 'headless'], default=setting('COMPARE_CLI_MODE'),
+	                help='Default: interactive in a container, headless in clean isolation.')
+	ap.add_argument('--model', help='Model id. Default: CLAUDE_MODEL or CODEX_MODEL, else the CLI default.')
+	ap.add_argument('--key-env', help='Name of the setting with the model API key.')
+	ap.add_argument('--claude-tools', choices=['none', 'default'],
+	                default=setting('COMPARE_CLAUDE_TOOLS', default='none'),
+	                help='none: only the n8n tools. default: also the built-in Claude Code tools, all allowed.')
+	ap.add_argument('--rebuild-image', action='store_true')
+	ap.add_argument('--timeout-min', type=float, default=30)
+	args = ap.parse_args()
+
+	if args.claude_tools not in ('none', 'default'):
+		sys.exit(f'COMPARE_CLAUDE_TOOLS must be none or default, not {args.claude_tools!r}.')
+	if args.builder == 'codex' and args.isolation != 'container':
+		sys.exit('Codex runs only in a container. Its shell tool can read the host disk.')
+	if args.isolation != 'container':
+		args.mode = 'headless'
+		if args.builder == 'claude' and args.claude_tools != 'none':
+			print('[claude] clean isolation runs on the host, so COMPARE_CLAUDE_TOOLS=default is ignored.', flush=True)
+			args.claude_tools = 'none'
+	args.mode = args.mode or 'interactive'
+
+	prefix = args.builder.upper()
+	args.model = args.model or setting(f'{prefix}_MODEL')
+	# Gateway-style ids such as anthropic/claude-opus-5-5 are not valid for the CLIs.
+	provider = {'claude': 'anthropic/', 'codex': 'openai/'}[args.builder]
+	if args.model and args.model.startswith(provider):
+		args.model = args.model.removeprefix(provider)
+	key_names = [args.key_env] if args.key_env else KEY_SETTINGS[args.builder]
+	api_key = setting(*key_names)
+	if not api_key:
+		sys.exit(f'Set {" or ".join(key_names)} in .env.ai-surfaces, .env.local or the environment.')
+	token = load_token()
+	if not token:
+		sys.exit('No MCP token. Run prepare.py first.')
+
+	out = Path(args.out).resolve()
+	if out.exists():
+		shutil.rmtree(out)
+	work, home = out / 'work', out / 'home'
+	work.mkdir(parents=True)
+	(home / '.claude').mkdir(parents=True)
+	prompt = Path(args.prompt_file).read_text().strip()
+	base_url = setting('N8N_BASE_URL', default=DEFAULT_BASE_URL).rstrip('/')
+	url = mcp_url(base_url, args.isolation)
+	container = f'cwb-{args.builder}-{int(time.time())}'
+	interactive = args.mode == 'interactive'
+
+	# Codex does not save the tools it sends to the model, so a local proxy records the request bodies.
+	recorder = None
+	if args.builder == 'codex' and setting('COMPARE_RECORD_REQUESTS', default='1') != '0':
+		recorder = RequestRecorder(OPENAI_BASE_URL, out / 'requests.jsonl',
+		                           setting('COMPARE_RECORDER_BIND', default='127.0.0.1')).start()
+	api_base = recorder.base_url('host.docker.internal') if recorder else OPENAI_BASE_URL
+
+	child_env = {'N8N_MCP_TOKEN': token}
+	if args.builder == 'claude':
+		(out / 'mcp.json').write_text(json.dumps({'mcpServers': {'n8n': {
+			'type': 'http', 'url': url, 'headers': {'Authorization': 'Bearer ${N8N_MCP_TOKEN}'},
+		}}}))
+	if interactive:
+		(out / '.screen').mkdir()
+		shutil.copy(SKILL_DIR / 'sandbox' / 'screen.mjs', out / '.screen' / 'screen.mjs')
+		child_env[SANDBOX_KEY[args.builder]] = api_key
+		if args.builder == 'claude':
+			write_claude_interactive_config(out, home, args.claude_tools)
+		else:
+			write_codex_config(home, url, SANDBOX_KEY['codex'], api_base)
+	elif args.builder == 'claude':
+		child_env['ANTHROPIC_API_KEY'] = api_key
+	else:
+		child_env['OPENAI_API_KEY'] = api_key
+		child_env['CODEX_API_KEY'] = api_key
+		write_codex_config(home, url, 'CODEX_API_KEY' if recorder else None, api_base)
+
+	started_at = now_iso()
+	if args.isolation == 'container':
+		ensure_image(args.rebuild_image)
+		run_env = {**os.environ, **child_env}
+		try:
+			if interactive:
+				result = run_interactive(args, out, home, prompt, run_env, container)
+			else:
+				cmd = docker_command(container, out, child_env)
+				cmd += (claude_args(prompt, args.model, '/out/mcp.json', False, args.claude_tools)
+				        if args.builder == 'claude' else codex_args(prompt, args.model))
+				result = run_headless(args, out, prompt, cmd, run_env, None, container)
+		finally:
+			if recorder:
+				recorder.stop()
+		if recorder and not recorder.count:
+			print(f'[{args.builder}] the request recorder saw no requests, so the report has no Codex tools.', flush=True)
+	else:
+		if not shutil.which('claude'):
+			sys.exit('claude is not on PATH.')
+		run_env = {
+			'PATH': os.environ.get('PATH', '/usr/bin:/bin'),
+			'HOME': str(home),
+			'CLAUDE_CONFIG_DIR': str(home / '.claude'),
+			'DISABLE_AUTOUPDATER': '1',
+			'DISABLE_TELEMETRY': '1',
+			**child_env,
+		}
+		cmd = claude_args(prompt, args.model, str(out / 'mcp.json'), True)
+		result = run_headless(args, out, prompt, cmd, run_env, str(work), container)
+
+	session_file = find_session_file(args.builder, home, result['sessionId'])
+	meta = {
+		'kind': args.builder,
+		'isolation': args.isolation,
+		'mode': args.mode,
+		'model': args.model,
+		**({'claudeTools': args.claude_tools} if args.builder == 'claude' else {}),
+		'startedAt': started_at,
+		'finishedAt': now_iso(),
+		'exitCode': result['exitCode'],
+		'sessionId': result['sessionId'],
+		'sessionFile': str(session_file.relative_to(out)) if session_file else None,
+		'workflowIds': result['workflowIds'],
+		'workflowId': result['workflowIds'][-1] if result['workflowIds'] else None,
+		'reportedCostUsd': result['reportedCostUsd'],
+		'screen': 'screen.cast' if (out / 'screen.cast').exists() else None,
+		'requests': 'requests.jsonl' if recorder and recorder.count else None,
+	}
+	(out / 'run.json').write_text(json.dumps(meta, indent=2))
+	shutil.rmtree(home / '.codex' / 'shell_snapshots', ignore_errors=True)
+	check_no_secrets(out, [api_key, token])
+	print(json.dumps({k: meta[k] for k in ('exitCode', 'sessionId', 'workflowId', 'sessionFile', 'screen')}, indent=2))
+	if meta['exitCode']:
+		print(f'see {out / "stderr.log"}', flush=True)
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/run_instance_ai.py

@@ -0,0 +1,294 @@
+#!/usr/bin/env python3
+"""Run one prompt through Instance AI on the local n8n instance.
+
+The script sends the prompt to a new thread, answers every confirmation card with
+a fixed policy, waits for the thread to go idle, and exports the trace.
+
+  run_instance_ai.py --out <run-dir>/iai --prompt-file <run-dir>/prompt.txt
+
+Approval policy (the same choices for every run, so runs are comparable):
+  - approval, plan review, web search, continue: approve
+  - text input: approve with "Use sensible defaults and continue."
+  - questions: pick the first option, or skip a free-text question
+  - domain access: allow once
+  - workflow or credential setup cards: defer (the builder must not wait for real credentials)
+  - local gateway resources and MCP connect cards: deny
+"""
+
+import argparse
+import json
+import os
+import queue
+import re
+import shutil
+import subprocess
+import sys
+import threading
+import time
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+from live import enabled as live_enabled
+from live import open_url, recording
+from n8n_api import N8nClient
+
+DEFAULT_TEXT_ANSWER = 'Use sensible defaults and continue.'
+HERE = Path(__file__).resolve().parent
+
+
+class Recorder:
+	"""Video of the thread in the n8n UI, made by record_iai.mjs."""
+
+	def __init__(self, out):
+		self.proc, self.lines = None, queue.Queue()
+		node = shutil.which('node')
+		if not recording() or not node:
+			return
+		live_flag = ['--live'] if live_enabled() else []
+		self.proc = subprocess.Popen([node, str(HERE / 'record_iai.mjs'), str(out.resolve()), *live_flag], cwd=HERE,
+		                             stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+		                             stderr=(out / 'record.log').open('w'), text=True)
+		threading.Thread(target=self._read, daemon=True).start()
+
+	def _read(self):
+		for line in self.proc.stdout:
+			self.lines.put(line.strip())
+
+	def wait_for(self, word, timeout):
+		deadline = time.time() + timeout
+		while self.proc and time.time() < deadline:
+			try:
+				if self.lines.get(timeout=min(max(deadline - time.time(), 0.1), 1)) == word:
+					return True
+			except queue.Empty:
+				if self.proc.poll() is not None:
+					return False
+		return False
+
+	def send(self, line):
+		if self.proc and self.proc.poll() is None:
+			self.proc.stdin.write(line + '\n')
+			self.proc.stdin.flush()
+
+	def stop(self):
+		if not self.proc:
+			return
+		self.send('stop')
+		self.proc.stdin.close()
+		try:
+			self.proc.wait(timeout=120)
+		except subprocess.TimeoutExpired:
+			self.proc.kill()
+
+
+def now_iso():
+	return datetime.now(timezone.utc).isoformat()
+
+
+def local_time_zone():
+	zone = os.environ.get('TZ')
+	if zone:
+		return zone
+	link = os.path.realpath('/etc/localtime')
+	return link.split('zoneinfo/', 1)[1] if 'zoneinfo/' in link else 'UTC'
+
+
+def answer_questions(questions):
+	answers = []
+	for q in questions or []:
+		options = q.get('options') or []
+		if q.get('type') in ('single', 'multi') and options:
+			answers.append({'questionId': q['id'], 'selectedOptions': [options[0]]})
+		elif q.get('required'):
+			answers.append({'questionId': q['id'], 'selectedOptions': [], 'customText': DEFAULT_TEXT_ANSWER})
+		else:
+			answers.append({'questionId': q['id'], 'selectedOptions': [], 'skipped': True})
+	return {'kind': 'questions', 'answers': answers}
+
+
+def decide(payload):
+	"""Return (confirm body, short label) for one confirmation-request payload."""
+	input_type = payload.get('inputType') or 'approval'
+	if payload.get('credentialDestination'):
+		origin = payload['credentialDestination'].get('origin')
+		return {'kind': 'credentialDestination', 'origin': origin, 'approved': False}, 'defer credential destination'
+	if payload.get('setupRequests') or payload.get('credentialRequests') or payload.get('credentialFlow'):
+		return {'kind': 'approval', 'approved': False}, 'defer setup'
+	if payload.get('domainAccess'):
+		return {'kind': 'domainAccessApprove', 'domainAccessAction': 'allow_once'}, 'allow domain once'
+	if payload.get('mcpConnectRequest'):
+		return {'kind': 'mcpConnect', 'approved': False}, 'deny MCP connect'
+	if input_type == 'resource-decision':
+		return {'kind': 'resourceDecision', 'resourceDecision': 'denyOnce'}, 'deny local resource'
+	if input_type == 'questions':
+		return answer_questions(payload.get('questions')), 'answer questions'
+	if input_type == 'text':
+		return {'kind': 'approval', 'approved': True, 'userInput': DEFAULT_TEXT_ANSWER}, 'answer text'
+	return {'kind': 'approval', 'approved': True}, f'approve {input_type}'
+
+
+def read_stream(client, thread_id, out_queue, stop):
+	try:
+		res = client.open_stream(f'/rest/instance-ai/events/{thread_id}')
+	except Exception as e:
+		out_queue.put(('error', str(e)))
+		return
+	out_queue.put(('open', None))
+	data_lines, event_name = [], None
+	while not stop.is_set():
+		raw = res.readline()
+		if not raw:
+			out_queue.put(('closed', None))
+			return
+		line = raw.decode(errors='replace').rstrip('\r\n')
+		if line == '':
+			if data_lines:
+				try:
+					out_queue.put(('event', (event_name, json.loads('\n'.join(data_lines)))))
+				except json.JSONDecodeError:
+					pass
+			data_lines, event_name = [], None
+		elif line.startswith('data:'):
+			data_lines.append(line[5:].lstrip())
+		elif line.startswith('event:'):
+			event_name = line[6:].strip()
+
+
+def collect_workflow_ids(value, found):
+	text = json.dumps(value)
+	for match in re.finditer(r'"workflowId"\s*:\s*"([A-Za-z0-9]{10,})"', text):
+		if match.group(1) not in found:
+			found.append(match.group(1))
+
+
+def main():
+	ap = argparse.ArgumentParser()
+	ap.add_argument('--out', required=True)
+	ap.add_argument('--prompt-file', required=True)
+	ap.add_argument('--timeout-min', type=float, default=30)
+	args = ap.parse_args()
+
+	out = Path(args.out)
+	out.mkdir(parents=True, exist_ok=True)
+	prompt = Path(args.prompt_file).read_text().strip()
+
+	# The recorder signs in while the event stream opens, so the video starts soon after the prompt.
+	recorder = Recorder(out)
+	client = N8nClient().login()
+	thread_id = str(uuid.uuid4())
+	events_file = (out / 'events.jsonl').open('w')
+	q, stop = queue.Queue(), threading.Event()
+	threading.Thread(target=read_stream, args=(client, thread_id, q, stop), daemon=True).start()
+
+	kind, detail = q.get(timeout=30)
+	if kind != 'open':
+		sys.exit(f'Event stream did not open: {detail}')
+
+	project = client.request('GET', '/rest/projects/personal')
+	client.request('POST', '/rest/instance-ai/threads',
+	               {'threadId': thread_id, 'projectId': project['id'], 'source': 'assistant_page'})
+
+	if recorder.proc and not recorder.wait_for('ready', 60):
+		print(f'recording failed to start, see {out / "record.log"}', flush=True)
+		recorder.proc.kill()
+		recorder.proc = None
+
+	started_at = now_iso()
+	chat = client.request('POST', f'/rest/instance-ai/chat/{thread_id}',
+	                      {'message': prompt, 'timeZone': local_time_zone()})
+	print(f'thread {thread_id} run {chat.get("runId") if isinstance(chat, dict) else chat}', flush=True)
+	thread_url = f'{client.base_url}/instance-ai/{thread_id}'
+	# The UI shows the user message only after the chat request, so the recorder opens the thread now.
+	recorder.send(f'open {thread_url}')
+	print(f'watch: {thread_url}', flush=True)
+	open_url(thread_url)
+
+	workflow_ids, decisions, langsmith_run_ids = [], [], []
+	saw_finish, idle_since = False, None
+	deadline = time.time() + args.timeout_min * 60
+	while time.time() < deadline:
+		try:
+			kind, detail = q.get(timeout=5)
+		except queue.Empty:
+			kind, detail = 'tick', None
+
+		if kind == 'event':
+			_, event = detail
+			events_file.write(json.dumps(event) + '\n')
+			events_file.flush()
+			etype, payload = event.get('type'), event.get('payload') or {}
+			if etype == 'run-start':
+				if payload.get('langsmithRunId'):
+					langsmith_run_ids.append(payload['langsmithRunId'])
+				idle_since = None
+			elif etype == 'run-finish':
+				saw_finish = True
+				print(f'run-finish {payload.get("status", "")}', flush=True)
+			elif etype == 'tool-call':
+				print(f'tool {payload.get("toolName")}', flush=True)
+			elif etype in ('tool-result', 'confirmation-request'):
+				collect_workflow_ids(payload, workflow_ids)
+			if etype == 'confirmation-request':
+				body, label = decide(payload)
+				try:
+					client.request('POST', f'/rest/instance-ai/confirm/{payload["requestId"]}', body)
+					ok = True
+				except RuntimeError as e:
+					ok = False
+					label += f' (failed: {e})'
+				decisions.append({'at': now_iso(), 'toolName': payload.get('toolName'),
+				                  'message': payload.get('message'), 'decision': label, 'ok': ok})
+				print(f'confirm: {label} [{payload.get("toolName")}]', flush=True)
+		elif kind in ('closed', 'error'):
+			print(f'event stream {kind}: {detail or ""}', flush=True)
+
+		if saw_finish:
+			status = client.request('GET', f'/rest/instance-ai/threads/{thread_id}/status')
+			running = [t for t in status.get('backgroundTasks', []) if t.get('status') == 'running']
+			if status.get('hasActiveRun') or status.get('isSuspended') or running:
+				idle_since = None
+			elif idle_since is None:
+				idle_since = time.time()
+			elif time.time() - idle_since > 10:
+				break
+	else:
+		print('timeout: the thread did not go idle', flush=True)
+
+	finished_at = now_iso()
+	stop.set()
+	events_file.close()
+	recorder.stop()
+
+	trace = {}
+	for key, path in (
+		('rich', f'/threads/{thread_id}/messages?limit=200'),
+		('raw', f'/threads/{thread_id}/messages?limit=200&raw=true'),
+		('thread', f'/threads/{thread_id}'),
+		('debug', f'/debug/threads/{thread_id}/runs'),
+	):
+		try:
+			trace[key] = client.request('GET', '/rest/instance-ai' + path)
+		except RuntimeError as e:
+			trace[key] = None
+			print(f'export {key} failed: {str(e)[:200]}', flush=True)
+	(out / 'trace.json').write_text(json.dumps(trace, indent=1))
+
+	meta = {
+		'kind': 'instance-ai',
+		'threadId': thread_id,
+		'startedAt': started_at,
+		'finishedAt': finished_at,
+		'workflowIds': workflow_ids,
+		'workflowId': workflow_ids[-1] if workflow_ids else None,
+		'langsmithRunIds': langsmith_run_ids,
+		'decisions': decisions,
+		'threadUrl': thread_url,
+		'screen': 'screen.webm' if (out / 'screen.webm').exists() else None,
+	}
+	(out / 'run.json').write_text(json.dumps(meta, indent=2))
+	print(json.dumps({k: meta[k] for k in ('threadId', 'workflowId', 'startedAt', 'finishedAt')}, indent=2))
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/run_scenarios.py

@@ -0,0 +1,172 @@
+#!/usr/bin/env python3
+"""Run each workflow of a run directory once for each scenario, with pinned data.
+
+  run_scenarios.py <run-dir>
+
+Reads <run-dir>/scenarios.json (optional, see SKILL.md) and <builder>/workflow.json.
+sdk.mjs makes the pin data: triggers, service nodes and AI root nodes are pinned, and logic
+nodes (Set, IF, Code, Aggregate, ...) run for real. The n8n MCP tool test_workflow runs the
+workflow, and get_workflow_execution reads the result of each node.
+Writes <builder>/exec.json. MCP runs a workflow only when its MCP access is on, so the script
+turns it on through the REST API when necessary, records that in exec.json, and turns it off
+again after the scenarios. With several triggers, each scenario starts from the trigger that
+sdk.mjs picks (scenarios.json "triggerNode", or the trigger that reaches the "reach" types).
+"""
+
+import json
+import shutil
+import subprocess
+import sys
+import urllib.request
+from pathlib import Path
+
+from n8n_api import N8nClient, load_token
+
+HERE = Path(__file__).resolve().parent
+TIMEOUT_SECONDS = 90
+ITEMS_PER_NODE = 3
+
+
+class Mcp:
+	def __init__(self, base_url, token):
+		self.url, self.token = base_url + '/mcp-server/http', token
+
+	def call(self, tool, arguments):
+		body = json.dumps({'jsonrpc': '2.0', 'id': 1, 'method': 'tools/call',
+		                   'params': {'name': tool, 'arguments': arguments}}).encode()
+		req = urllib.request.Request(self.url, data=body, method='POST', headers={
+			'content-type': 'application/json',
+			'accept': 'application/json, text/event-stream',
+			'authorization': f'Bearer {self.token}',
+		})
+		with urllib.request.urlopen(req, timeout=TIMEOUT_SECONDS + 60) as res:
+			raw = res.read().decode()
+		lines = [line[5:].strip() for line in raw.splitlines() if line.startswith('data:')]
+		reply = json.loads(lines[-1] if lines else raw)
+		if 'error' in reply:
+			raise RuntimeError(f'{tool}: {reply["error"].get("message")}')
+		result = reply['result']
+		return result.get('structuredContent') or json.loads(result['content'][0]['text'])
+
+
+def matches(node_type, pattern):
+	short = node_type.rsplit('.', 1)[-1]
+	return node_type == pattern or short.lower() == pattern.lower()
+
+
+def read_run(run_data, types):
+	"""Per node: did it run, its error, and its first output items (all outputs)."""
+	nodes = {}
+	for name, runs in (run_data or {}).items():
+		run = runs[-1] if runs else {}
+		items = []
+		for output in ((run.get('data') or {}).get('main') or []):
+			items.extend(item.get('json') for item in (output or [])[:ITEMS_PER_NODE])
+		error = run.get('error')
+		nodes[name] = {
+			'type': types.get(name),
+			'status': run.get('executionStatus'),
+			'error': (error.get('message') or error.get('description')) if isinstance(error, dict) else error,
+			'items': items[:ITEMS_PER_NODE * 2],
+		}
+	return nodes
+
+
+def expectations(expect, nodes):
+	"""Check reach and skip: node types that must run, and node types that must not run."""
+	results = []
+	ran = [n for n in nodes.values() if n['status'] == 'success' and not n['error']]
+	for pattern in expect.get('reach') or []:
+		results.append({'kind': 'reach', 'type': pattern, 'ok': any(matches(n['type'] or '', pattern) for n in ran)})
+	for pattern in expect.get('skip') or []:
+		results.append({'kind': 'skip', 'type': pattern,
+		                'ok': not any(matches(n['type'] or '', pattern) for n in nodes.values())})
+	return results
+
+
+def set_mcp_access(client, workflow_id, on):
+	"""Set MCP access for the workflow. Returns True when the script changed the setting."""
+	wf = client.workflow(workflow_id)
+	settings = wf.get('settings') or {}
+	if bool(settings.get('availableInMCP')) == on:
+		return False
+	client.request('PATCH', f'/rest/workflows/{workflow_id}',
+	               {'versionId': wf['versionId'], 'settings': {**settings, 'availableInMCP': on}})
+	return True
+
+
+def run_builder(sub, scenarios_file, client, mcp, node):
+	meta = json.loads((sub / 'run.json').read_text())
+	workflow_id = meta.get('workflowId')
+	wf_file = sub / 'workflow.json'
+	out = {'workflowId': workflow_id, 'scenarios': []}
+	if not workflow_id or not wf_file.exists():
+		out['skipped'] = 'No workflow.'
+		return out
+	workflow = json.loads(wf_file.read_text())
+	types = {n['name']: n['type'] for n in workflow.get('nodes', [])}
+	cmd = [node, str(HERE / 'sdk.mjs'), 'pin-plan', str(wf_file)] + ([str(scenarios_file)] if scenarios_file else [])
+	plan = json.loads(subprocess.run(cmd, cwd=HERE, capture_output=True, text=True, check=True).stdout)
+	out.update(pinned=plan['pinned'], executed=plan['executed'])
+	try:
+		out['mcpAccessTurnedOn'] = set_mcp_access(client, workflow_id, True)
+	except Exception as e:
+		out['skipped'] = f'Could not turn on MCP access: {e}'
+		return out
+	try:
+		run_plan(plan, workflow_id, types, mcp, sub, out)
+	finally:
+		if out['mcpAccessTurnedOn']:
+			try:
+				set_mcp_access(client, workflow_id, False)
+			except Exception as e:
+				out['mcpAccessRestoreError'] = str(e)
+				print(f'[scenarios] {sub.name}: could not turn MCP access off again: {e}', flush=True)
+	return out
+
+
+def run_plan(plan, workflow_id, types, mcp, sub, out):
+	for scenario in plan['scenarios']:
+		entry = {'name': scenario['name'], 'expect': scenario['expect'], 'provenance': scenario['provenance']}
+		args = {'workflowId': workflow_id, 'pinData': scenario['pinData'], 'timeout': TIMEOUT_SECONDS}
+		if scenario.get('triggerNode'):
+			args['triggerNodeName'] = entry['triggerNode'] = scenario['triggerNode']
+		try:
+			result = mcp.call('test_workflow', args)
+			entry.update(status=result.get('status'), error=result.get('error'), executionId=result.get('executionId'))
+			if result.get('executionId'):
+				execution = mcp.call('get_workflow_execution', {'workflowId': workflow_id,
+				                                                'executionId': result['executionId'], 'includeData': True})
+				run_data = ((execution.get('data') or {}).get('resultData') or {}).get('runData')
+				entry['nodes'] = read_run(run_data, types)
+				entry['checks'] = expectations(scenario['expect'], entry['nodes'])
+		except Exception as e:
+			entry.update(status='error', error=f'{type(e).__name__}: {e}')
+		print(f'[scenarios] {sub.name}: {scenario["name"]} -> {entry.get("status")}'
+		      + (f' ({entry["error"]})' if entry.get('error') else ''), flush=True)
+		out['scenarios'].append(entry)
+
+
+def main():
+	run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.').resolve()
+	node = shutil.which('node')
+	token = load_token()
+	if not node or not token:
+		sys.exit('Scenarios need node on PATH and a working MCP token.')
+	client = N8nClient().login()
+	mcp = Mcp(client.base_url, token)
+	scenarios_file = run_dir / 'scenarios.json'
+	scenarios_file = scenarios_file if scenarios_file.exists() else None
+	for meta_file in sorted(run_dir.glob('*/run.json')):
+		sub = meta_file.parent
+		try:
+			out = run_builder(sub, scenarios_file, client, mcp, node)
+		except Exception as e:
+			out = {'skipped': f'{type(e).__name__}: {e}', 'scenarios': []}
+		(sub / 'exec.json').write_text(json.dumps(out, indent=1))
+		if out.get('skipped'):
+			print(f'[scenarios] {sub.name}: skipped, {out["skipped"]}', flush=True)
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/run_wire.mjs

@@ -0,0 +1,313 @@
+#!/usr/bin/env node
+// Run each workflow of a run directory with the real node code, and record its service requests.
+//
+//   node run_wire.mjs <run-dir>
+//
+// wire.mjs runs the workflow in this process. Only the trigger is pinned, and the AI root
+// nodes too, except in the real-AI scenarios. Every other node runs, and the HTTP requests of
+// the service nodes get answers in the documented shape of each API. A node that runs must be on
+// the allow-list (runsInProcess), else the scenarios that run it are skipped. Three kinds of scenario:
+//   contract  the scenarios of scenarios.json, with the AI output pinned as in run_scenarios.py
+//   edge      the edge cases of the trigger (TRIGGER_KITS in sdk.mjs) and scenarios.json "edge"
+//   ai        scenarios.json "questions": the AI nodes call their real model with each question
+// Writes <builder>/wire.json.
+import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { join, resolve } from 'node:path';
+
+import { LOGIC_TYPES, TRIGGER_KITS, graph, isAiNode, isTrigger, pinPlan, typeMatches } from './sdk.mjs';
+import { MODEL_CREDENTIALS, fillSetup, isWrite, loadEngine, modelKey, nodeCredentialTypes, runWorkflow, startNode } from './wire.mjs';
+
+const ITEMS_PER_NODE = 3;
+const BODY_CHARS = 1500;
+// This process holds the settings and has no sandbox. The eval hook answers only the requests that
+// go through the n8n request helpers. Database, SSH, FTP and SMTP clients, files and a shell bypass
+// it, so only the node types below run here.
+// These nodes run code from the builder. The pinned-data runs run them in the n8n task runner.
+const CODE_NODE_TYPES = new Set([
+	'n8n-nodes-base.code',
+	'n8n-nodes-base.function',
+	'n8n-nodes-base.functionItem',
+	'n8n-nodes-base.aiTransform',
+	'@n8n/n8n-nodes-langchain.code',
+	'@n8n/n8n-nodes-langchain.toolCode',
+]);
+// Telegram is here because its credential has no authenticate block.
+const IN_PROCESS_TYPES = new Set([
+	...[...LOGIC_TYPES].filter((type) => !CODE_NODE_TYPES.has(type)),
+	'n8n-nodes-base.httpRequest',
+	'n8n-nodes-base.wait',
+	'n8n-nodes-base.telegram',
+]);
+// Each model node has a key in MODEL_CREDENTIALS. The other nodes call only the model or the helpers.
+const IN_PROCESS_AI_TYPES = new Set(
+	[
+		'agent', 'chainLlm', 'chainSummarization', 'informationExtractor', 'textClassifier', 'sentimentAnalysis',
+		'openAi', 'anthropic', 'googleGemini',
+		'lmChatOpenAi', 'lmChatAnthropic', 'lmChatGoogleGemini', 'lmChatMistralCloud', 'lmChatGroq',
+		'outputParserStructured', 'outputParserAutofixing', 'outputParserItemList',
+		'memoryBufferWindow', 'toolCalculator', 'toolThink', 'toolHttpRequest',
+	].map((type) => `@n8n/n8n-nodes-langchain.${type}`),
+);
+const readJson = (file) => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
+
+/** Read a path such as `messages[0].from` from a value. */
+export function readPath(value, path) {
+	const keys = Array.isArray(path) ? path : String(path).match(/[^.[\]]+/g) ?? [];
+	return keys.reduce((v, k) => (v == null ? undefined : v[/^\d+$/.test(k) ? Number(k) : k]), value);
+}
+
+const requestText = (r) => JSON.stringify({ url: r.url, qs: r.qs ?? undefined, body: r.body ?? undefined });
+
+function nodeSummary(runData, types) {
+	const nodes = {};
+	for (const [name, runs] of Object.entries(runData ?? {})) {
+		const run = runs.at(-1) ?? {};
+		const outputs = (run.data?.main ?? []).map((o) => o ?? []);
+		const error = run.error;
+		nodes[name] = {
+			type: types[name],
+			status: run.executionStatus,
+			error: error ? error.message ?? error.description ?? String(error) : null,
+			runs: runs.length,
+			outputCounts: outputs.map((o) => o.length),
+			items: outputs.flat().slice(0, ITEMS_PER_NODE).map((i) => i.json),
+		};
+	}
+	return nodes;
+}
+
+const trim = (value) => {
+	const text = value === undefined ? undefined : JSON.stringify(value);
+	return text && text.length > BODY_CHARS ? `${text.slice(0, BODY_CHARS)}…` : value;
+};
+const trimRequest = (r) => ({ ...r, body: trim(r.body), response: trim(r.response) });
+
+/**
+ * Resolve a marker of an expectation:
+ *   @trigger.<path>          a value of the first trigger item, such as @trigger.messages[0].from
+ *   @response.<type>.<path>  a value of the response to the last write request of a node type,
+ *                            such as @response.notion.id (the id of the page that Notion created)
+ * Other strings stay as they are.
+ */
+function resolveMarker(value, triggerItem, requests) {
+	if (typeof value !== 'string') return value;
+	if (value.startsWith('@trigger.')) return readPath(triggerItem, value.slice('@trigger.'.length));
+	const response = value.match(/^@response\.([^.]+)\.(.+)$/);
+	if (response) {
+		const last = requests.filter((r) => isWrite(r) && typeMatches(r.nodeType, response[1])).at(-1);
+		return last ? readPath(last.response, response[2]) : undefined;
+	}
+	return value;
+}
+
+/** Check the expectations of a scenario against the nodes that ran and the requests that they sent. */
+export function evaluate(scenario, nodes, requests, trigger, kit) {
+	const checks = [];
+	const expect = scenario.expect ?? {};
+	const ran = Object.values(nodes).filter((n) => n.status === 'success' && !n.error);
+	for (const pattern of expect.reach ?? []) {
+		checks.push({ kind: 'reach', type: pattern, ok: ran.some((n) => typeMatches(n.type ?? '', pattern)) });
+	}
+	for (const pattern of expect.skip ?? []) {
+		checks.push({ kind: 'skip', type: pattern, ok: !Object.values(nodes).some((n) => typeMatches(n.type ?? '', pattern)) });
+	}
+	for (const want of expect.requests ?? []) {
+		// An array in contains means any one of its markers.
+		const contains = (want.contains ?? []).map((c) => (Array.isArray(c) ? c : [c]).map((m) => ({ marker: m, value: resolveMarker(m, trigger[0], requests) })));
+		const sent = requests.filter((r) => typeMatches(r.nodeType, want.type) && (!want.method || r.method === want.method.toUpperCase()));
+		const has = (r, any) => any.some((c) => c.value !== undefined && c.value !== null && requestText(r).includes(String(c.value)));
+		const matching = sent.filter((r) => contains.every((any) => has(r, any)));
+		const ok = want.count === undefined ? matching.length > 0 : matching.length === want.count;
+		const show = (c) => (c.marker === c.value ? `"${c.value}"` : `${c.marker} ("${c.value ?? 'not found'}")`);
+		const details = sent.length
+			? sent.map((r) => {
+					const missing = contains.filter((any) => !has(r, any)).map((any) => any.map(show).join(' or '));
+					if (missing.length) return `${r.node}: ${r.method} ${r.url} does not carry ${missing.join(', ')}`;
+					const body = r.body === undefined ? '' : JSON.stringify(r.body);
+					return `${r.node}: ${r.method} ${r.url}${body ? ` with \`${body.length > 300 ? `${body.slice(0, 300)}…` : body}\`` : ''}`;
+				})
+			: [`No ${want.method ?? ''} request from a ${want.type} node.`.replace('  ', ' ')];
+		checks.push({ kind: 'request', type: want.type, contains: contains.map((any) => any.map((c) => c.marker).join(' or ')), count: want.count, ok, details });
+	}
+	if (expect.noWrites) {
+		const writes = requests.filter(isWrite);
+		checks.push({ kind: 'noWrites', ok: writes.length === 0, details: writes.map((r) => `${r.node}: ${r.method} ${r.url}`) });
+	}
+	if (scenario.derive === 'each-item' && kit) {
+		const ids = trigger.map((item) => String(readPath(item, kit.identity)));
+		const carriers = requests.filter((r) => isWrite(r) && ids.some((id) => requestText(r).includes(id)));
+		const byNode = new Map();
+		for (const r of carriers) byNode.set(r.node, [...(byNode.get(r.node) ?? []), r]);
+		const details = [];
+		let ok = byNode.size > 0;
+		for (const [node, list] of byNode) {
+			const reached = ids.filter((id) => list.some((r) => requestText(r).includes(id)));
+			const missed = ids.filter((id) => !reached.includes(id));
+			if (missed.length) ok = false;
+			details.push(`${node}: ${list.length} request${list.length === 1 ? '' : 's'}, for ${reached.join(', ') || 'nobody'}${missed.length ? `, nothing for ${missed.join(', ')}` : ''}`);
+		}
+		if (!byNode.size) details.push(`No outgoing request names a sender (${ids.join(', ')}).`);
+		checks.push({ kind: 'eachItem', ids, ok, details });
+	}
+	return checks;
+}
+
+/** The scenarios of one workflow: contract, edge and AI, with their pin data and case rows. */
+function buildScenarios(workflow, cfg) {
+	const g = graph(workflow);
+	const triggerNode = g.triggers.find((t) => isTrigger(t.type));
+	const kit = triggerNode ? TRIGGER_KITS[triggerNode.type] : undefined;
+	const contract = cfg.scenarios?.length ? cfg.scenarios : null;
+	const edges = [...(kit?.edges ?? []), ...(cfg.edge ?? [])];
+	// The triggerNode of pinPlan stays: a name from scenarios.json can be missing in this workflow.
+	const merge = (s, entry, kind) => ({ ...s, ...entry, triggerNode: s.triggerNode, kind });
+	const list = [
+		...(contract ? pinPlan(workflow, cfg, contract).scenarios.map((s, i) => merge(s, contract[i], 'contract')) : pinPlan(workflow, cfg).scenarios.map((s) => merge(s, {}, 'contract'))),
+		...pinPlan(workflow, cfg, edges).scenarios.map((s, i) => merge(s, edges[i], 'edge')),
+		...pinPlan(workflow, cfg, cfg.questions ?? []).scenarios.map((s, i) => merge(s, cfg.questions[i], 'ai')),
+	];
+	const keep = (name) => {
+		const node = g.byName.get(name);
+		return node && (isTrigger(node.type) || isAiNode(g, node));
+	};
+	return {
+		g,
+		scenarios: list.map((s) => {
+			const pinData = Object.fromEntries(
+				Object.entries(s.pinData).filter(([name]) => keep(name) && !(s.kind === 'ai' && isAiNode(g, g.byName.get(name)))),
+			);
+			return { ...s, pinData };
+		}),
+	};
+}
+
+/** Model sub-nodes and vendor model nodes, and whether a key exists for each, for the real-AI scenarios. */
+function modelReadiness(g) {
+	const subnodes = g.edges.filter((e) => e.type === 'ai_languageModel').map((e) => g.byName.get(e.from));
+	const vendors = g.nodes.filter((n) => isAiNode(g, n) && !g.aiInputs(n.name, 'ai_languageModel').length);
+	return [...new Set([...subnodes, ...vendors])].map((node) => {
+		const types = nodeCredentialTypes(node).filter((t) => MODEL_CREDENTIALS[t]);
+		const type = types.find((t) => modelKey(t)) ?? types[0];
+		const p = node.parameters ?? {};
+		return { node: node.name, credentialType: type ?? null, model: p.modelName ?? p.model?.value ?? p.model ?? p.modelId?.value ?? null, ready: Boolean(type && modelKey(type)) };
+	});
+}
+
+const isHttpCredential = (name, { credentialType, parentTypes }) =>
+	credentialType(name).authenticate !== undefined || [name, ...parentTypes(name)].some((p) => p === 'oAuth1Api' || p === 'oAuth2Api');
+
+/**
+ * True for a node on the allow-list: the types above, declarative nodes, and nodes whose credentials
+ * all authenticate HTTP requests. A tool variant (slackTool) counts as its base node.
+ */
+export function runsInProcess(node) {
+	const base = node.type.startsWith('n8n-nodes-base.') && node.type.endsWith('Tool') ? node.type.slice(0, -'Tool'.length) : node.type;
+	if (IN_PROCESS_TYPES.has(base) || IN_PROCESS_AI_TYPES.has(base)) return true;
+	if (!base.startsWith('n8n-nodes-base.')) return false;
+	const engine = loadEngine();
+	try {
+		const { description } = engine.nodeTypes.getByNameAndVersion(base, node.typeVersion);
+		if (description.requestDefaults !== undefined) return true;
+		const credentials = (description.credentials ?? []).map((c) => c.name);
+		return credentials.length > 0 && credentials.every((name) => isHttpCredential(name, engine));
+	} catch {
+		return false;
+	}
+}
+
+/**
+ * The names and types of the nodes that run from the start trigger and are not on the allow-list.
+ * Pinned nodes do not run. A disabled node passes its input on, so the walk goes through it.
+ */
+function outsideAllowList(workflow, g, start, pinData) {
+	const seen = new Set([start]);
+	const running = new Set();
+	// A Set visits the entries that the loop adds.
+	for (const name of seen) {
+		for (const targets of workflow.connections?.[name]?.main ?? []) {
+			for (const { node } of targets ?? []) {
+				if (seen.has(node)) continue;
+				seen.add(node);
+				if (g.byName.has(node) && !Object.hasOwn(pinData, node)) running.add(node);
+			}
+		}
+	}
+	for (const name of running) {
+		for (const e of g.edges) if (e.to === name && e.type.startsWith('ai_')) running.add(e.from);
+	}
+	return [...running].map((name) => g.byName.get(name)).filter((n) => !runsInProcess(n)).map((n) => `${n.name} (${n.type})`);
+}
+
+async function runBuilder(dir, cfg) {
+	const workflow = readJson(join(dir, 'workflow.json'));
+	const { g, scenarios } = buildScenarios(workflow, cfg);
+	const canvas = readJson(join(dir, 'canvas-issues.json'));
+	const { workflow: filledWorkflow, filled } = fillSetup(workflow, canvas);
+	const types = Object.fromEntries((workflow.nodes ?? []).map((n) => [n.name, n.type]));
+	const models = modelReadiness(g);
+	const out = { filled, models, scenarios: [] };
+	for (const scenario of scenarios) {
+		const entry = { name: scenario.name, kind: scenario.kind, why: scenario.why, expect: scenario.expect ?? {}, derive: scenario.derive, pinned: Object.keys(scenario.pinData) };
+		if (scenario.kind === 'ai' && !models.length) {
+			out.scenarios.push({ ...entry, status: 'skipped', error: 'The workflow has no model sub-node.' });
+			continue;
+		}
+		const start = startNode(workflow, scenario.pinData, scenario.triggerNode);
+		const outside = start ? outsideAllowList(workflow, g, start.name, scenario.pinData) : [];
+		if (outside.length) {
+			out.scenarios.push({ ...entry, status: 'skipped', error: `Nodes outside the in-process allow-list: ${outside.join(', ')}.` });
+			continue;
+		}
+		if (scenario.kind === 'ai' && models.some((m) => !m.ready)) {
+			const missing = models.filter((m) => !m.ready).map((m) => `${m.node} (${m.credentialType ?? 'unknown credential'}: set ${MODEL_CREDENTIALS[m.credentialType]?.keys[0] ?? 'a supported model'})`);
+			out.scenarios.push({ ...entry, status: 'skipped', error: `No model key for ${missing.join(', ')}.` });
+			continue;
+		}
+		const rows = (node) => Object.entries(scenario.byType ?? {}).find(([pattern]) => typeMatches(node.type, pattern))?.[1];
+		const trigger = start ? scenario.pinData[start.name].map((i) => i.json) : [];
+		const kit = start ? TRIGGER_KITS[start.type] : undefined;
+		const result = await runWorkflow(filledWorkflow, { pinData: scenario.pinData, triggerNode: scenario.triggerNode, rows, realModels: scenario.kind === 'ai' });
+		const nodes = nodeSummary(result.runData, types);
+		const errors = Object.entries(nodes).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
+		Object.assign(entry, {
+			status: result.status,
+			error: result.error,
+			ms: result.ms,
+			nodes,
+			requests: result.requests.map(trimRequest),
+			blocked: result.blocked,
+			checks: evaluate(scenario, nodes, result.requests, trigger, kit),
+		});
+		if (scenario.kind === 'ai') {
+			entry.aiOutputs = Object.fromEntries(Object.entries(nodes).filter(([name]) => isAiNode(g, g.byName.get(name))).map(([name, n]) => [name, n.items]));
+		}
+		console.log(`[wire] ${dir.split('/').pop()}: ${scenario.kind} "${scenario.name}" -> ${result.status}${errors.length ? ` (${errors[0]})` : result.error ? ` (${result.error})` : ''}`);
+		out.scenarios.push(entry);
+	}
+	return out;
+}
+
+if (process.argv[1] && resolve(process.argv[1]) === new URL(import.meta.url).pathname) {
+	const runDir = resolve(process.argv[2] ?? '.');
+	const loaded = loadEngine();
+	const cfg = readJson(join(runDir, 'scenarios.json')) ?? {};
+	for (const entry of readdirSync(runDir, { withFileTypes: true })) {
+		const dir = join(runDir, entry.name);
+		if (!entry.isDirectory() || !existsSync(join(dir, 'workflow.json'))) continue;
+		let out;
+		if (loaded.error) out = { skipped: loaded.error, scenarios: [] };
+		else {
+			try {
+				out = await runBuilder(dir, cfg);
+			} catch (error) {
+				out = { skipped: `${error?.name ?? 'Error'}: ${error?.message ?? error}`, scenarios: [] };
+			}
+		}
+		// A failed real model request can echo its URL, with the key, in the error message.
+		const keys = Object.keys(MODEL_CREDENTIALS).map(modelKey).filter(Boolean);
+		writeFileSync(join(dir, 'wire.json'), keys.reduce((text, key) => text.replaceAll(key, '[redacted]'), JSON.stringify(out, null, 1)));
+		if (out.skipped) console.log(`[wire] ${entry.name}: skipped, ${out.skipped}`);
+	}
+	// Model SDKs can keep sockets open.
+	process.exit(0);
+}

.agents/skills/testmax-n8n-ai/scripts/sdk.mjs

@@ -0,0 +1,600 @@
+#!/usr/bin/env node
+// Shared workflow knowledge for derive_checks.mjs and run_scenarios.py: the workflow SDK of
+// the repository, graph helpers, expression references, JSON Schema samples and pin data plans.
+//
+//   node sdk.mjs pin-plan <workflow.json> [scenarios.json]    print the pin data of each scenario
+import { createRequire } from 'node:module';
+import { existsSync, readFileSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const here = dirname(fileURLToPath(import.meta.url));
+export const repo = resolve(here, '../../../..');
+const require = createRequire(import.meta.url);
+const SDK_DIST = join(repo, 'packages/@n8n/workflow-sdk/dist/index.js');
+// The same directories that the n8n node catalog gives to setSchemaBaseDirs().
+const NODE_DEFINITION_DIRS = [
+	join(repo, 'packages/nodes-base/dist/node-definitions/'),
+	join(repo, 'packages/@n8n/nodes-langchain/dist/node-definitions/'),
+];
+
+let loaded;
+/** The workflow SDK with the node parameter schemas set, or null when the package is not built. */
+export function loadSdk() {
+	if (loaded !== undefined) return loaded;
+	if (!existsSync(SDK_DIST)) return (loaded = null);
+	const sdk = require(SDK_DIST);
+	const schemaDirs = NODE_DEFINITION_DIRS.filter((dir) => existsSync(dir));
+	// Without schema directories validateNodeConfig reports every node as valid.
+	sdk.setSchemaBaseDirs(schemaDirs);
+	return (loaded = { sdk, schemaDirs, missingDirs: NODE_DEFINITION_DIRS.filter((d) => !existsSync(d)) });
+}
+
+// --- Node kinds ----------------------------------------------------------------
+
+const BASE = 'n8n-nodes-base.';
+const LC = '@n8n/n8n-nodes-langchain.';
+
+/** Nodes that run for real in a scenario. They need no credentials and call no service. */
+export const LOGIC_TYPES = new Set(
+	[
+		'set', 'if', 'switch', 'filter', 'merge', 'aggregate', 'splitOut', 'code', 'function', 'functionItem',
+		'noOp', 'splitInBatches', 'itemLists', 'sort', 'limit', 'removeDuplicates', 'summarize', 'dateTime',
+		'renameKeys', 'compareDatasets', 'markdown', 'html', 'xml', 'crypto', 'executionData', 'stopAndError',
+		'respondToWebhook', 'convertToFile', 'extractFromFile', 'debugHelper',
+	].map((t) => BASE + t),
+);
+
+/** Nodes that end up with one item, so `.first()` on them reads all of their output. */
+export const SINGLE_ITEM_TYPES = new Set(['aggregate', 'summarize'].map((t) => BASE + t));
+
+const TRIGGER_TYPES = new Set(['webhook', 'cron', 'interval', 'emailReadImap', 'start'].map((t) => BASE + t));
+
+export const isSticky = (type) => type === `${BASE}stickyNote`;
+export const isTrigger = (type) => TRIGGER_TYPES.has(type) || type.toLowerCase().includes('trigger');
+export const isManualTrigger = (type) => type === `${BASE}manualTrigger`;
+export const shortType = (type) => type.slice(type.lastIndexOf('.') + 1);
+
+/** True when type matches a full type name, or its part after the last dot (case-insensitive). */
+export function typeMatches(type, pattern) {
+	return type === pattern || shortType(type).toLowerCase() === pattern.toLowerCase();
+}
+
+const AI_ROOT_PLACEHOLDER = {
+	[`${LC}agent`]: (v) => ({ output: v }),
+	[`${LC}openAiAssistant`]: (v) => ({ output: v }),
+	[`${LC}informationExtractor`]: (v) => ({ output: v }),
+	[`${LC}chainLlm`]: (v) => ({ text: v }),
+	[`${LC}chainRetrievalQa`]: (v) => ({ response: v }),
+	[`${LC}chainSummarization`]: (v) => ({ output: { output_text: v } }),
+	[`${LC}sentimentAnalysis`]: (v) => ({ sentimentAnalysis: { category: v } }),
+	[`${LC}googleGemini`]: (v) => ({ content: { parts: [{ text: v }], role: 'model' }, finishReason: 'STOP', index: 0 }),
+};
+
+/** Vendor nodes that call a model themselves, without a model sub-node. */
+const AI_VENDOR_TYPES = new Set(['googleGemini', 'openAi', 'anthropic'].map((t) => LC + t));
+
+// --- Graph ---------------------------------------------------------------------
+
+/** Nodes, edges and lookups of a workflow. Sticky notes and disabled nodes are left out. */
+export function graph(workflow) {
+	const nodes = (workflow.nodes ?? []).filter((n) => !isSticky(n.type) && !n.disabled);
+	const byName = new Map(nodes.map((n) => [n.name, n]));
+	const edges = [];
+	for (const [from, outputs] of Object.entries(workflow.connections ?? {})) {
+		for (const [type, branches] of Object.entries(outputs ?? {})) {
+			(branches ?? []).forEach((targets, output) =>
+				(targets ?? []).forEach((t) => {
+					if (byName.has(from) && byName.has(t.node))
+						edges.push({ from, to: t.node, type, output, input: t.index ?? 0 });
+				}),
+			);
+		}
+	}
+	const subnodes = new Set(edges.filter((e) => e.type.startsWith('ai_')).map((e) => e.from));
+	const mainParents = (name) => [...new Set(edges.filter((e) => e.type === 'main' && e.to === name).map((e) => e.from))];
+	const mainChildren = (name) => edges.filter((e) => e.type === 'main' && e.from === name);
+	const aiInputs = (name, type) => edges.filter((e) => e.to === name && e.type === type).map((e) => byName.get(e.from));
+	const triggers = nodes.filter((n) => isTrigger(n.type) && !subnodes.has(n.name));
+	return { nodes, byName, edges, subnodes, mainParents, mainChildren, aiInputs, triggers };
+}
+
+/** Names of the nodes that a trigger reaches. AI sub-nodes count when they serve a reached node. */
+export function reachable(g) {
+	const seen = new Set(g.triggers.map((t) => t.name));
+	const queue = [...seen];
+	while (queue.length) {
+		const name = queue.shift();
+		for (const e of g.edges) {
+			if (e.type === 'main' && e.from === name && !seen.has(e.to)) {
+				seen.add(e.to);
+				queue.push(e.to);
+			}
+		}
+	}
+	// A sub-node can serve another sub-node, for example a model inside an auto-fixing parser.
+	let grew = true;
+	while (grew) {
+		grew = false;
+		for (const e of g.edges) {
+			if (e.type.startsWith('ai_') && seen.has(e.to) && !seen.has(e.from)) {
+				seen.add(e.from);
+				grew = true;
+			}
+		}
+	}
+	return seen;
+}
+
+/** Names of the nodes downstream of name, through main connections. */
+export function downstream(g, name) {
+	const seen = new Set();
+	const queue = [name];
+	while (queue.length) {
+		for (const e of g.mainChildren(queue.shift())) {
+			if (!seen.has(e.to)) {
+				seen.add(e.to);
+				queue.push(e.to);
+			}
+		}
+	}
+	return seen;
+}
+
+// --- Parameters and expressions --------------------------------------------------
+
+/** Every string in a parameter tree, with its dotted path. */
+export function strings(value, path = '') {
+	if (typeof value === 'string') return [{ path, value }];
+	if (Array.isArray(value)) return value.flatMap((v, i) => strings(v, `${path}[${i}]`));
+	if (value && typeof value === 'object')
+		return Object.entries(value).flatMap(([k, v]) => strings(v, path ? `${path}.${k}` : k));
+	return [];
+}
+
+const CODE_PARAMS = new Set(['jsCode', 'pythonCode', 'functionCode', 'code']);
+
+/** The expression text of a parameter value, or null. Code fields count, because they use the same references. */
+export function expressionText(path, value) {
+	if (value.startsWith('=')) return value.slice(1);
+	const key = path.split('.').pop().replace(/\[\d+\]$/, '');
+	return CODE_PARAMS.has(key) ? value : null;
+}
+
+const IDENT = /^[A-Za-z_$][\w$]*/;
+
+/** Read a property path such as `.a?.b['c'][0]` from text at index i. Returns { path, end, call }. */
+function readPath(text, i) {
+	const path = [];
+	let dynamic = false;
+	for (;;) {
+		if (text.startsWith('?.', i)) i += 2;
+		else if (text[i] === '.') i += 1;
+		else if (text[i] !== '[') break;
+		if (text[i] === '[') {
+			const close = text.indexOf(']', i);
+			if (close < 0) break;
+			const inner = text.slice(i + 1, close).trim();
+			if (/^\d+$/.test(inner)) path.push(Number(inner));
+			else if (/^(['"`])(.*)\1$/.test(inner)) path.push(inner.slice(1, -1));
+			else {
+				dynamic = true;
+				i = close + 1;
+				break;
+			}
+			i = close + 1;
+			continue;
+		}
+		const match = text.slice(i).match(IDENT);
+		if (!match) break;
+		path.push(match[0]);
+		i += match[0].length;
+	}
+	const call = /^\s*\(/.test(text.slice(i));
+	// A trailing method call, such as `.toUpperCase()`, is not a field.
+	if (call && path.length) path.pop();
+	return { path, end: i, call, dynamic };
+}
+
+const ACCESSORS = [
+	{ re: /^\.item\b/, kind: 'item' },
+	{ re: /^\.first\(\s*\)/, kind: 'first' },
+	{ re: /^\.last\(\s*\)/, kind: 'last' },
+	{ re: /^\.all\(\s*\)\[\s*\d+\s*\]/, kind: 'all-index' },
+	{ re: /^\.all\(\s*\)/, kind: 'all' },
+	{ re: /^\.itemMatching\([^)]*\)/, kind: 'item' },
+	{ re: /^\.pairedItem\([^)]*\)/, kind: 'item' },
+];
+
+/**
+ * References in expression text: { node, kind, accessor, path, raw }.
+ * node is null for `$json` and `$input`, which read the main input of the node.
+ * path is null when the reference does not read item JSON, for example `$('A').params`.
+ */
+export function references(text) {
+	const refs = [];
+	const named = /\$\(\s*(['"`])((?:\\.|(?!\1).)+)\1\s*\)|\$node\[\s*(['"`])((?:\\.|(?!\3).)+)\3\s*\]|\$items\(\s*(['"`])((?:\\.|(?!\5).)+)\5/g;
+	for (const m of text.matchAll(named)) {
+		const node = (m[2] ?? m[4] ?? m[6]).replace(/\\(.)/g, '$1');
+		let i = m.index + m[0].length;
+		let accessor = m[4] !== undefined ? 'node' : m[6] !== undefined ? 'items' : null;
+		if (accessor === null) {
+			const rest = text.slice(i);
+			const hit = ACCESSORS.find((a) => a.re.test(rest));
+			if (hit) {
+				accessor = hit.kind;
+				i += rest.match(hit.re)[0].length;
+			}
+		}
+		let path = null;
+		if (text.startsWith('.json', i) && accessor !== 'all' && accessor !== 'items') {
+			const read = readPath(text, i + '.json'.length);
+			path = read.dynamic ? null : read.path;
+		}
+		refs.push({ node, kind: 'named', accessor, path, raw: text.slice(m.index, i + 40).split(/[}\n]/)[0] });
+	}
+	for (const m of text.matchAll(/\$json\b/g)) {
+		const read = readPath(text, m.index + '$json'.length);
+		refs.push({ node: null, kind: 'json', accessor: 'item', path: read.dynamic ? null : read.path,
+			raw: text.slice(m.index, read.end) });
+	}
+	for (const m of text.matchAll(/\$input\.(item|first\(\s*\)|last\(\s*\)|all\(\s*\)\[\s*\d+\s*\])\.json\b/g)) {
+		const read = readPath(text, m.index + m[0].length);
+		const accessor = m[1].startsWith('first') ? 'first' : m[1].startsWith('last') ? 'last' : m[1] === 'item' ? 'item' : 'all-index';
+		refs.push({ node: null, kind: 'input', accessor, path: read.dynamic ? null : read.path,
+			raw: text.slice(m.index, read.end) });
+	}
+	return refs;
+}
+
+/** Every reference in a workflow: { node (the node that holds it), param, ref, code }. */
+export function workflowReferences(g) {
+	const out = [];
+	for (const node of g.nodes) {
+		for (const { path, value } of strings(node.parameters ?? {})) {
+			const text = expressionText(path, value);
+			if (text === null) continue;
+			const code = !value.startsWith('=');
+			for (const ref of references(text)) out.push({ node: node.name, param: path, ref, code });
+		}
+	}
+	return out;
+}
+
+/**
+ * Look up a path in a value. Returns 'found', 'missing' or 'unknown'.
+ * 'unknown' means that a null or an opaque value hides the rest of the path. With sample, the
+ * value comes from a schema, which shows the shape of one array item and not the count, so an
+ * index past the end of an array is 'unknown'. In real output, that index is 'missing'.
+ */
+export function lookup(value, path, { sample = false } = {}) {
+	let current = value;
+	for (const key of path) {
+		if (current === null || current === undefined) return current === null ? 'unknown' : 'missing';
+		if (Array.isArray(current)) {
+			if (typeof key === 'number') {
+				if (key >= current.length) return sample ? 'unknown' : 'missing';
+				current = current[key];
+			} else if (key === 'length') return 'found';
+			else return 'missing';
+			continue;
+		}
+		if (typeof current === 'string') return key === 'length' ? 'found' : 'missing';
+		if (typeof current !== 'object') return 'missing';
+		if (!(key in current)) return 'missing';
+		current = current[key];
+	}
+	return current === undefined ? 'missing' : 'found';
+}
+
+export const showPath = (path) =>
+	path.map((k, i) => (typeof k === 'number' ? `[${k}]` : /^[A-Za-z_$][\w$]*$/.test(k) ? (i ? `.${k}` : k) : `['${k}']`)).join('');
+
+// --- JSON Schema samples ---------------------------------------------------------
+
+/** A deterministic sample value for a JSON Schema. key names the field, for readable strings. */
+export function sample(schema, key = 'value', depth = 0) {
+	if (!schema || typeof schema !== 'object') return null;
+	if (schema.example !== undefined) return schema.example;
+	if (schema.const !== undefined) return schema.const;
+	if (Array.isArray(schema.enum) && schema.enum.length) return schema.enum[0];
+	if (schema.default !== undefined) return schema.default;
+	const variant = schema.anyOf ?? schema.oneOf;
+	if (Array.isArray(variant) && variant.length) return sample(variant.find((v) => v.type !== 'null') ?? variant[0], key, depth);
+	let type = Array.isArray(schema.type) ? schema.type.find((t) => t !== 'null') : schema.type;
+	if (!type) type = schema.properties ? 'object' : schema.items ? 'array' : 'string';
+	switch (type) {
+		case 'object': {
+			if (depth > 6) return {};
+			return Object.fromEntries(
+				Object.entries(schema.properties ?? {}).map(([k, v]) => [k, sample(v, k, depth + 1)]),
+			);
+		}
+		case 'array':
+			return schema.items && depth <= 6 ? [sample(schema.items, key, depth + 1)] : [];
+		case 'integer':
+		case 'number':
+			return 1;
+		case 'boolean':
+			return true;
+		case 'null':
+			return null;
+		default: {
+			const k = String(key);
+			if (schema.format === 'date-time' || /(^|_)(date|time|at)$|At$/.test(k)) return '2026-01-01T09:00:00.000Z';
+			if (schema.format === 'email' || /email/i.test(k)) return 'ada@example.com';
+			if (schema.format === 'uri' || /url|link|href/i.test(k)) return `https://example.com/${k}`;
+			if (/(^|_)id$|Id$/.test(k)) return `${k}-1`;
+			return `sample ${k}`;
+		}
+	}
+}
+
+/** Set every boolean in a value to flag. */
+function setBooleans(value, flag) {
+	if (Array.isArray(value)) return value.map((v) => setBooleans(v, flag));
+	if (value && typeof value === 'object')
+		return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, setBooleans(v, flag)]));
+	return typeof value === 'boolean' ? flag : value;
+}
+
+function deepMerge(base, extra) {
+	if (!base || typeof base !== 'object' || Array.isArray(base) || !extra || typeof extra !== 'object') return extra;
+	const out = { ...base };
+	for (const [k, v] of Object.entries(extra)) out[k] = deepMerge(base[k], v);
+	return out;
+}
+
+// --- Pin data ------------------------------------------------------------------
+
+/**
+ * Trigger output for a scenario, in the shape that the trigger code emits.
+ * WhatsApp: WhatsAppTrigger.node.ts emits `{ ...change.value, field }` for each change,
+ * and change.value is the Cloud API webhook payload of a text message.
+ */
+export const TRIGGER_FIXTURES = {
+	[`${BASE}whatsAppTrigger`]: [{
+		messaging_product: 'whatsapp',
+		metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
+		contacts: [{ profile: { name: 'Ada Lovelace' }, wa_id: '4915112345678' }],
+		messages: [{ from: '4915112345678', id: 'wamid.HBgLNDkxNTExMjM0NTY3OBUCABIYFjNFQjA', timestamp: '1767258000',
+			text: { body: 'What are your opening hours?' }, type: 'text' }],
+		field: 'messages',
+	}],
+	[`${BASE}webhook`]: [{ headers: { 'content-type': 'application/json' }, params: {}, query: {},
+		body: { message: 'Hello' }, webhookUrl: 'http://localhost:5678/webhook/test', executionMode: 'test' }],
+	[`${LC}chatTrigger`]: [{ sessionId: 'session-1', action: 'sendMessage', chatInput: 'Hello' }],
+	[`${BASE}manualTrigger`]: [{}],
+	[`${BASE}scheduleTrigger`]: [{ timestamp: '2026-01-01T09:00:00.000+00:00', 'Readable date': 'January 1st 2026, 9:00:00 am',
+		'Day of week': 'Thursday', Year: '2026', Month: 'January', 'Day of month': '01', Hour: '09', Minute: '00',
+		Second: '00', Timezone: 'UTC' }],
+	[`${BASE}telegramTrigger`]: [{ update_id: 1, message: { message_id: 1, date: 1767258000, text: 'Hello',
+		chat: { id: 1001, type: 'private', first_name: 'Ada' }, from: { id: 1001, is_bot: false, first_name: 'Ada' } } }],
+	[`${BASE}formTrigger`]: [{ submittedAt: '2026-01-01T09:00:00.000Z', formMode: 'test' }],
+};
+
+const whatsAppMessage = (from, name, body, id) => ({
+	messaging_product: 'whatsapp',
+	metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
+	contacts: [{ profile: { name }, wa_id: from }],
+	messages: [{ from, id, timestamp: '1767258000', text: { body }, type: 'text' }],
+	field: 'messages',
+});
+
+/**
+ * Per trigger type: identity is the path of the value that names the sender in one item,
+ * withMessage puts a scenario text into the fixture, and edges are the edge cases that every
+ * workflow with this trigger must survive. derive 'each-item' means that each trigger item
+ * must get its own outgoing request that names its sender.
+ */
+export const TRIGGER_KITS = {
+	[`${BASE}whatsAppTrigger`]: {
+		identity: ['messages', 0, 'from'],
+		withMessage: (items, text) => items.map((i) => ({ ...i, messages: (i.messages ?? []).map((m) => ({ ...m, text: { body: text } })) })),
+		edges: [
+			{
+				name: 'Two messages arrive in one webhook call',
+				why: 'WhatsAppTrigger.node.ts emits one item for each change, and Meta can batch several changes in one call',
+				derive: 'each-item',
+				ai: { booleans: true },
+				trigger: [
+					whatsAppMessage('4915112345678', 'Ada Lovelace', 'What are your opening hours?', 'wamid.A1'),
+					whatsAppMessage('4917699887766', 'Alan Turing', 'Do you ship abroad?', 'wamid.B2'),
+				],
+			},
+			{
+				name: 'A delivery status update arrives, not a customer message',
+				why: 'the messages update also delivers sent, delivered and read statuses, with no messages array',
+				expect: { noWrites: true },
+				trigger: [{
+					messaging_product: 'whatsapp',
+					metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
+					statuses: [{ id: 'wamid.HBgLNDkxNTExMjM0NTY3OBUCABEYEjQ', status: 'delivered', timestamp: '1767258060', recipient_id: '4915112345678' }],
+					field: 'messages',
+				}],
+			},
+		],
+	},
+};
+
+/** The trigger items of a scenario: its own trigger, the case trigger, or the fixture with the scenario message. */
+export function triggerItems(node, scenario = {}, scenarios = {}) {
+	const explicit = scenario.trigger ?? scenarios.trigger;
+	if (explicit) return { items: Array.isArray(explicit) ? explicit : [explicit], source: 'case' };
+	const fixture = TRIGGER_FIXTURES[node.type];
+	if (!fixture) return null;
+	const kit = TRIGGER_KITS[node.type];
+	if (scenario.message !== undefined && kit) return { items: kit.withMessage(fixture, scenario.message), source: 'case' };
+	return { items: fixture, source: 'fixture' };
+}
+
+export const isAiRootNode = (g, node) => node.type.startsWith(LC) && !g.subnodes.has(node.name) && g.aiInputs(node.name, 'ai_languageModel').length > 0;
+
+/** A node whose output comes from a model: an AI root node or a vendor node such as Google Gemini. */
+export const isAiNode = (g, node) => Boolean(node) && (isAiRootNode(g, node) || (AI_VENDOR_TYPES.has(node.type) && !g.subnodes.has(node.name)));
+
+function parserSchema(parser) {
+	const p = parser?.parameters ?? {};
+	const parse = (text) => {
+		try {
+			return typeof text === 'string' ? JSON.parse(text) : text && typeof text === 'object' ? text : null;
+		} catch {
+			return null;
+		}
+	};
+	const schemaType = p.schemaType ?? 'fromJson';
+	if (schemaType === 'manual' && parse(p.inputSchema)) return { kind: 'schema', value: parse(p.inputSchema) };
+	if (schemaType !== 'manual' && parse(p.jsonSchemaExample)) return { kind: 'example', value: parse(p.jsonSchemaExample) };
+	if (parse(p.jsonSchema)) return { kind: 'schema', value: parse(p.jsonSchema) };
+	return null;
+}
+
+/** JSON objects that a prompt shows as examples, in order. Objects with a boolean come first. */
+function promptExamples(node) {
+	const found = [];
+	for (const { value } of strings(node.parameters ?? {})) {
+		for (let start = value.indexOf('{'); start >= 0; start = value.indexOf('{', start + 1)) {
+			let depth = 0;
+			for (let i = start; i < value.length; i++) {
+				if (value[i] === '{') depth++;
+				else if (value[i] === '}' && --depth === 0) {
+					try {
+						const parsed = JSON.parse(value.slice(start, i + 1));
+						if (parsed && typeof parsed === 'object' && !Array.isArray(parsed) && Object.keys(parsed).length)
+							found.push(parsed);
+					} catch {}
+					break;
+				}
+			}
+		}
+	}
+	return found.sort((a, b) => Number(hasBoolean(b)) - Number(hasBoolean(a)));
+}
+
+const hasBoolean = (value) => Object.values(value).some((v) => typeof v === 'boolean');
+
+/** Apply the ai settings of a scenario to a model result: every boolean, then explicit fields. */
+function applyAi(value, ai) {
+	let out = typeof ai.booleans === 'boolean' ? setBooleans(value, ai.booleans) : value;
+	if (ai.set && out && typeof out === 'object') out = deepMerge(out, ai.set);
+	return out;
+}
+
+/**
+ * The output of an AI root node for a scenario. Returns { json, provenance }.
+ * With a structured output parser, the result follows the parser schema. Without one, the
+ * model returns text: ai.text when the scenario sets it, else the first JSON example in the
+ * prompt of the node (the contract that the builder declared), else a plain sentence.
+ */
+function aiRootOutput(g, node, ai = {}) {
+	const parserNode = g.aiInputs(node.name, 'ai_outputParser')[0];
+	const parser = parserNode ? parserSchema(parserNode) : null;
+	if (parser) {
+		const parsed = parser.kind === 'schema' ? sample(parser.value, 'output') : parser.value;
+		return { json: { output: applyAi(parsed, ai) }, provenance: 'output-parser' };
+	}
+	const wrap = AI_ROOT_PLACEHOLDER[node.type] ?? ((v) => ({ output: v }));
+	if (ai.text !== undefined) return { json: wrap(ai.text), provenance: 'case' };
+	const example = promptExamples(node)[0];
+	if (example) return { json: wrap(JSON.stringify(applyAi(example, ai))), provenance: 'prompt-example' };
+	return { json: wrap('Sample answer from the model.'), provenance: AI_ROOT_PLACEHOLDER[node.type] ? 'ai-root' : 'placeholder' };
+}
+
+/**
+ * The pin data of each scenario. Logic nodes run for real. Triggers, service nodes and AI root
+ * nodes get pin data, with the provenance of each node, so a check knows how far to trust it:
+ *   fixture, case       the shape of real data (trigger code, or the case file)
+ *   node-schema         the __schema__ file of the node
+ *   output-parser       the schema of the attached structured output parser
+ *   prompt-example      a JSON example in the prompt of an AI root node without a parser
+ *   ai-root             the known output envelope of the AI root node
+ *   placeholder, empty  nothing describes the output, so the shape is a guess or one empty item
+ */
+export function pinPlan(workflow, scenarios = {}, only = null) {
+	const g = graph(workflow);
+	const sdk = loadSdk()?.sdk;
+	const list = only ?? (scenarios.scenarios?.length ? scenarios.scenarios : defaultScenarios(g));
+	const plan = { pinned: {}, executed: [], subnodes: [...g.subnodes], scenarios: [] };
+	for (const node of g.nodes) {
+		if (!g.subnodes.has(node.name) && LOGIC_TYPES.has(node.type)) plan.executed.push(node.name);
+	}
+	for (const scenario of list) {
+		const pinData = {};
+		const provenance = {};
+		const byType = mergeByType(scenarios.pinByType, scenario.pinByType);
+		for (const node of g.nodes) {
+			if (g.subnodes.has(node.name) || LOGIC_TYPES.has(node.type)) continue;
+			let items;
+			let source;
+			const caseItems = Object.entries(byType).find(([pattern]) => typeMatches(node.type, pattern))?.[1];
+			const explicit = scenario.trigger ?? scenarios.trigger;
+			const trigger = isTrigger(node.type) && (explicit || !caseItems) ? triggerItems(node, scenario, scenarios) : null;
+			if (trigger) {
+				items = trigger.items;
+				source = trigger.source;
+			} else if (caseItems) {
+				items = caseItems;
+				source = 'case';
+			} else if (isAiRootNode(g, node) || (isAiNode(g, node) && AI_ROOT_PLACEHOLDER[node.type])) {
+				const out = aiRootOutput(g, node, scenario.ai);
+				items = [out.json];
+				source = out.provenance;
+			} else {
+				const schema = sdk?.discoverOutputSchemaForNode(node.type, node.typeVersion, node.parameters ?? {});
+				items = [schema ? sample(schema, 'item') : {}];
+				source = schema ? 'node-schema' : 'empty';
+			}
+			pinData[node.name] = (Array.isArray(items) ? items : [items]).map((json) => ({ json }));
+			provenance[node.name] = source;
+		}
+		plan.scenarios.push({ name: scenario.name, expect: scenario.expect ?? {}, pinData, provenance, byType, triggerNode: scenarioTrigger(g, scenario) });
+		Object.assign(plan.pinned, provenance);
+	}
+	return plan;
+}
+
+/**
+ * The trigger that a scenario starts from, or null when the workflow has one trigger. Without
+ * scenario.triggerNode, it is the trigger whose downstream nodes match the most expect.reach types.
+ */
+function scenarioTrigger(g, scenario) {
+	if (g.triggers.length < 2) return null;
+	if (scenario.triggerNode && g.triggers.some((t) => t.name === scenario.triggerNode)) return scenario.triggerNode;
+	const reach = scenario.expect?.reach ?? [];
+	const score = (trigger) => {
+		const types = [...downstream(g, trigger.name)].map((name) => g.byName.get(name)?.type ?? '');
+		return reach.filter((pattern) => types.some((type) => typeMatches(type, pattern))).length;
+	};
+	return g.triggers.reduce((best, t) => (score(t) > score(best) ? t : best)).name;
+}
+
+/** Case pin data by node type. A scenario entry replaces a case entry for the same short type. */
+function mergeByType(base = {}, extra = {}) {
+	const out = { ...base };
+	for (const [pattern, items] of Object.entries(extra ?? {})) {
+		for (const key of Object.keys(out)) if (shortType(key).toLowerCase() === shortType(pattern).toLowerCase()) delete out[key];
+		out[pattern] = items;
+	}
+	return out;
+}
+
+/** Without a case file: one scenario for each value of the booleans that an AI node returns. */
+function defaultScenarios(g) {
+	const hasAi = g.nodes.some((n) => g.aiInputs(n.name, 'ai_languageModel').length && !g.subnodes.has(n.name));
+	return hasAi
+		? [{ name: 'AI flags true', ai: { booleans: true } }, { name: 'AI flags false', ai: { booleans: false } }]
+		: [{ name: 'Default run' }];
+}
+
+// --- CLI -----------------------------------------------------------------------
+
+if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+	const [command, workflowFile, scenariosFile] = process.argv.slice(2);
+	if (command !== 'pin-plan' || !workflowFile) {
+		console.error('usage: node sdk.mjs pin-plan <workflow.json> [scenarios.json]');
+		process.exit(2);
+	}
+	const workflow = JSON.parse(readFileSync(workflowFile, 'utf8'));
+	const scenarios = scenariosFile && existsSync(scenariosFile) ? JSON.parse(readFileSync(scenariosFile, 'utf8')) : {};
+	process.stdout.write(JSON.stringify(pinPlan(workflow, scenarios)));
+}

.agents/skills/testmax-n8n-ai/scripts/summarize_runs.py

@@ -0,0 +1,215 @@
+#!/usr/bin/env python3
+"""Print the evidence of each run for the judge step.
+
+  summarize_runs.py <run-dir>            print the summary with the builder names
+  summarize_runs.py <run-dir> --blind    write judge/summary-blind.txt and judge/blind-map.json
+
+The summary has the expected outcome criteria with their ids, and for each builder: the final
+agent message, every node with all its parameters and settings, the connections, and the facts
+from auto-checks.json. Nothing is truncated, so the judge can cite exact values.
+
+In blind mode the builders are called A, B and C in a random order, and builder and model names
+are removed from the final messages. blind-map.json maps each letter back to its builder.
+"""
+
+import argparse
+import json
+import random
+import re
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from build_report import load_case, parse_criteria  # noqa: E402
+
+NODE_SETTINGS = ('disabled', 'onError', 'retryOnFail', 'maxTries', 'waitBetweenTries', 'executeOnce',
+                 'alwaysOutputData', 'notesInFlow', 'notes')
+BUILDER_NAMES = re.compile(
+	r'\b(Instance AI|n8n Assistant|Claude Code|Claude|Anthropic|Codex|OpenAI|ChatGPT|GPT-[\w.-]+|Sonnet|Opus|Haiku)\b',
+	re.IGNORECASE)
+MARK = {True: '✓', False: '✗', None: '?'}
+REPO = Path(__file__).resolve().parents[4]
+LLM_CHECKS_DIR = REPO / 'packages/@n8n/instance-ai/evaluations/binaryChecks/checks'
+# The LLM binary checks of the eval suite that the judges answer as quality checks.
+QUALITY_CHECKS = ('valid-data-flow', 'correct-node-operations', 'handles-multiple-items', 'descriptive-node-names',
+                  'response-describes-changes-accurately')
+
+
+def quality_checks():
+	"""[(name, description, rules)] read from the check sources, so the rules stay those of the eval suite."""
+	out = []
+	for file in QUALITY_CHECKS:
+		path = LLM_CHECKS_DIR / f'{file}.ts'
+		if not path.exists():
+			continue
+		text = path.read_text()
+		name = re.search(r"name:\s*'([^']+)'", text)
+		desc = re.search(r"description:\s*(['\"])(.*?)\1,", text, re.S)
+		rules = re.search(r'systemPrompt:\s*`((?:\\`|[^`])*)`', text)
+		if name and rules:
+			out.append((name.group(1), desc.group(2) if desc else '', rules.group(1).replace('\\`', '`').strip()))
+	return out
+
+
+def quality_text():
+	checks = quality_checks()
+	if not checks:
+		return ''
+	lines = ['QUALITY CHECKS (from the Instance AI eval suite). For each builder, answer each check once in "quality",',
+	         'with the check name as the id. The final message is the narration of the agent. Each build starts',
+	         'from an empty workflow, so the START workflow is empty.']
+	for name, desc, rules in checks:
+		lines += [f'--- [{name}] {desc}', rules]
+	return '\n'.join(lines) + '\n'
+
+
+def session_final_message(sub, meta):
+	"""Final agent message from the session file of an interactive CLI run."""
+	if not meta.get('sessionFile'):
+		return None
+	text = None
+	for line in (sub / meta['sessionFile']).read_text().splitlines():
+		row = json.loads(line)
+		payload = row.get('payload') or {}
+		if row.get('type') == 'event_msg' and payload.get('type') == 'task_complete':
+			text = payload.get('last_agent_message') or text
+		if row.get('type') == 'assistant':
+			blocks = (row.get('message') or {}).get('content') or []
+			texts = [b['text'] for b in blocks if isinstance(b, dict) and b.get('type') == 'text' and b.get('text', '').strip()]
+			text = '\n'.join(texts) or text
+	return text
+
+
+def final_message(sub, meta):
+	kind = meta.get('kind')
+	if kind == 'instance-ai':
+		trace = json.loads((sub / 'trace.json').read_text())
+		rich = trace['rich'].get('data', trace['rich'])
+		assistant = [m for m in rich['messages'] if m.get('role') == 'assistant']
+		return (assistant[-1].get('agentTree') or {}).get('textContent') or assistant[-1].get('content') if assistant else None
+	if not (sub / 'stream.jsonl').exists():
+		return session_final_message(sub, meta)
+	text = None
+	for line in (sub / 'stream.jsonl').read_text().splitlines():
+		event = json.loads(line)
+		if event.get('type') == 'result':
+			text = event.get('result')
+		item = event.get('item') or {}
+		if event.get('type') == 'item.completed' and item.get('type') == 'agent_message':
+			text = item.get('text')
+	return text
+
+
+def connections(workflow):
+	lines = []
+	for source, outputs in (workflow.get('connections') or {}).items():
+		for kind, branches in outputs.items():
+			for index, targets in enumerate(branches or []):
+				for target in targets or []:
+					branch = f'[{index}]' if len(branches) > 1 else ''
+					# Merge and other multi-input nodes need the input index to show which side an edge feeds.
+					target_index = target.get('index') or 0
+					inlet = f'[in {target_index}]' if target_index else ''
+					via = '' if kind == 'main' else f' ({kind})'
+					lines.append(f'  {source}{branch} -> {target["node"]}{inlet}{via}')
+	return lines
+
+
+def criteria_text(expected):
+	"""The expected outcome with the id of each criterion, as the judge must cite it."""
+	groups = parse_criteria(expected or '')
+	if not groups:
+		return (expected or '(none)').strip()
+	lines = []
+	for heading, items in groups:
+		if heading:
+			lines.append(f'{heading}:')
+		lines.extend(f'  [{num}] {text}' for num, text in items)
+	return '\n'.join(lines)
+
+
+def auto_check_lines(sub):
+	path = sub / 'auto-checks.json'
+	if not path.exists():
+		return ['  (none: derive_checks.mjs did not run)']
+	lines = []
+	for group in json.loads(path.read_text()).get('groups', []):
+		score = group.get('score')
+		suffix = f' · score {score["passed"]}/{score["total"]}' if score else ''
+		lines.append(f'  {group["title"]}{suffix} — {group.get("note", "")}')
+		for c in group.get('checks', []):
+			mark = 'i' if c.get('level') == 'info' else MARK.get(c.get('ok'), '?')
+			lines.append(f'    [{mark}] {c["id"]} {c["text"]}')
+			lines.extend(f'        · {d}' for d in c.get('details') or [])
+	return lines
+
+
+def run_section(sub, meta, label, blind=False):
+	out = [f'===== {label} workflow={meta.get("workflowId")} =====']
+	try:
+		message = (final_message(sub, meta) or '(none)').strip()
+		out.append('FINAL MESSAGE:\n' + (scrub(message, meta) if blind else message))
+	except Exception as e:
+		out.append(f'FINAL MESSAGE: unreadable ({e})')
+	wf_file = sub / 'workflow.json'
+	if not wf_file.exists():
+		out.append('WORKFLOW: none\n')
+		return '\n'.join(out)
+	workflow = json.loads(wf_file.read_text())
+	out.append(f'WORKFLOW: {workflow.get("name")}  nodes={len(workflow.get("nodes", []))}')
+	out.append(f'SETTINGS: {json.dumps(workflow.get("settings") or {}, ensure_ascii=False)}')
+	for node in workflow.get('nodes', []):
+		creds = ', '.join((node.get('credentials') or {}).keys())
+		out.append(f'- {node["name"]} | {node["type"]} v{node.get("typeVersion")}' + (f' | creds: {creds}' if creds else ''))
+		settings = {k: node[k] for k in NODE_SETTINGS if k in node}
+		if settings:
+			out.append(f'    (settings) {json.dumps(settings, ensure_ascii=False)}')
+		for key, value in (node.get('parameters') or {}).items():
+			text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
+			out.append(f'    {key}: {text}')
+	out.append('CONNECTIONS:')
+	out.append('\n'.join(connections(workflow)) or '  (none)')
+	out.append('AUTO-CHECK FACTS (deterministic, from the node definitions, the n8n editor, pinned-data runs, '
+	           'in-process runs with recorded API requests, the eval binary checks and the final-message claims):')
+	out.extend(auto_check_lines(sub))
+	return '\n'.join(out) + '\n'
+
+
+def scrub(text, meta):
+	"""Remove builder and model names from a final message. The workflow itself stays as built."""
+	for name in filter(None, [meta.get('model'), meta.get('kind')]):
+		text = re.sub(re.escape(name), 'the builder', text, flags=re.IGNORECASE)
+	return BUILDER_NAMES.sub('the builder', text)
+
+
+def main():
+	ap = argparse.ArgumentParser(description='Print the evidence of each run for the judge step.')
+	ap.add_argument('run_dir', nargs='?', default='.')
+	ap.add_argument('--blind', action='store_true', help='Write a summary with the builders called A, B and C.')
+	ap.add_argument('--seed', type=int, help='Seed of the blind order, for a reproducible test.')
+	args = ap.parse_args()
+	run_dir = Path(args.run_dir)
+	case = load_case(run_dir)
+	runs = [(f.parent, json.loads(f.read_text())) for f in sorted(run_dir.glob('*/run.json')) if f.parent.name != 'report']
+	header = ('EXPECTED OUTCOME CRITERIA (cite each criterion by its id in brackets):\n'
+	          f'{criteria_text(case.get("expected"))}\n\n{quality_text()}')
+
+	if not args.blind:
+		print(header)
+		for sub, meta in runs:
+			print(run_section(sub, meta, f'{sub.name} ({meta.get("kind")})'))
+		return
+
+	order = runs[:]
+	random.Random(args.seed).shuffle(order)
+	letters = [chr(ord('A') + i) for i in range(len(order))]
+	sections = [run_section(sub, meta, f'Builder {letter}', blind=True) for letter, (sub, meta) in zip(letters, order)]
+	judge_dir = run_dir / 'judge'
+	judge_dir.mkdir(exist_ok=True)
+	(judge_dir / 'summary-blind.txt').write_text(header + '\n' + '\n'.join(sections))
+	(judge_dir / 'blind-map.json').write_text(json.dumps({letter: sub.name for letter, (sub, _) in zip(letters, order)}, indent=2))
+	print(judge_dir / 'summary-blind.txt')
+
+
+if __name__ == '__main__':
+	main()

.agents/skills/testmax-n8n-ai/scripts/wire.mjs

@@ -0,0 +1,459 @@
+// Run a workflow in this process with the real node code, and answer its HTTP requests here.
+//
+// The n8n execution engine and the nodes load from the compiled dist folders of the repository,
+// as in the execution evaluator of ai-workflow-builder.ee. n8n-core calls additionalData
+// .evalLlmMockHandler before it authenticates or sends a request (the Instance AI eval hook),
+// so every service request reaches respond() below and no request leaves the machine.
+// Credentials come from buildEvalMockCredentials, the same placeholders that the eval uses.
+// AI root nodes either get pinned data, or call their real model with a key from the settings.
+import { createRequire } from 'node:module';
+import { existsSync, readFileSync } from 'node:fs';
+import { join } from 'node:path';
+
+import { repo } from './sdk.mjs';
+
+const coreRequire = createRequire(join(repo, 'packages/core/package.json'));
+const PACKAGES = [
+	['n8n-nodes-base', join(repo, 'packages/nodes-base')],
+	['@n8n/n8n-nodes-langchain', join(repo, 'packages/@n8n/nodes-langchain')],
+];
+const REQUIRED_DIST = [
+	'packages/core/dist/execution-engine/index.js',
+	'packages/nodes-base/dist/known/nodes.json',
+	'packages/@n8n/nodes-langchain/dist/known/nodes.json',
+];
+
+// --- Settings ------------------------------------------------------------------
+
+// Same order as n8n_api.config(): .env.local, then .env.ai-surfaces, then the environment.
+const fileSettings = {};
+for (const file of [join(repo, '.env.local'), process.env.AI_SURFACES_ENV ?? join(repo, '.env.ai-surfaces')]) {
+	if (!existsSync(file)) continue;
+	for (const line of readFileSync(file, 'utf8').split('\n')) {
+		const match = line.match(/^\s*(?:export\s+)?([A-Z0-9_]+)\s*=\s*(.*)$/);
+		const value = match?.[2].trim().replace(/^(['"])(.*)\1$/, '$2');
+		if (value) fileSettings[match[1]] = value;
+	}
+}
+export const setting = (name) => process.env[name] || fileSettings[name] || '';
+
+/** Model credential types, the settings that can hold their key, and the hosts of their API. */
+export const MODEL_CREDENTIALS = {
+	googlePalmApi: { keys: ['GOOGLE_GENERATIVE_AI_API_KEY', 'GEMINI_API_KEY', 'GOOGLE_API_KEY'], field: 'apiKey', hosts: ['generativelanguage.googleapis.com'] },
+	openAiApi: { keys: ['OPENAI_API_KEY'], field: 'apiKey', hosts: ['api.openai.com'] },
+	anthropicApi: { keys: ['ANTHROPIC_API_KEY'], field: 'apiKey', hosts: ['api.anthropic.com'] },
+	mistralCloudApi: { keys: ['MISTRAL_API_KEY'], field: 'apiKey', hosts: ['api.mistral.ai'] },
+	groqApi: { keys: ['GROQ_API_KEY'], field: 'apiKey', hosts: ['api.groq.com'] },
+};
+export const modelKey = (type) => MODEL_CREDENTIALS[type]?.keys.map(setting).find(Boolean) ?? '';
+
+// --- Engine --------------------------------------------------------------------
+
+let engine;
+
+/** The execution engine, node types and credential types. Returns { error } when a dist folder is missing. */
+export function loadEngine() {
+	if (engine) return engine;
+	const missing = REQUIRED_DIST.filter((p) => !existsSync(join(repo, p)));
+	if (missing.length) return { error: `Build these first: ${missing.join(', ')}.` };
+	coreRequire('reflect-metadata');
+	const workflowLib = coreRequire('n8n-workflow');
+	const core = coreRequire(join(repo, 'packages/core/dist/execution-engine/index.js'));
+	const nodes = new Map();
+	const credentials = new Map();
+	for (const [prefix, dir] of PACKAGES) {
+		const read = (file) => JSON.parse(readFileSync(join(dir, 'dist/known', file), 'utf8'));
+		for (const [name, info] of Object.entries(read('nodes.json'))) nodes.set(`${prefix}.${name}`, { ...info, dir });
+		for (const [name, info] of Object.entries(read('credentials.json'))) credentials.set(name, { ...info, dir });
+	}
+	const instances = new Map();
+	const instance = (index, name) => {
+		const key = `${index === nodes ? 'n' : 'c'}:${name}`;
+		if (!instances.has(key)) {
+			const info = index.get(name);
+			if (!info) throw new Error(`Unknown type ${name}`);
+			const mod = createRequire(join(info.dir, 'package.json'))(join(info.dir, info.sourcePath));
+			instances.set(key, new mod[info.className]());
+		}
+		return instances.get(key);
+	};
+	const nodeTypes = {
+		getByName: (type) => instance(nodes, type),
+		getByNameAndVersion: (type, version) => {
+			const loaded = instance(nodes, type);
+			const versioned = 'nodeVersions' in loaded ? workflowLib.NodeHelpers.getVersionedNodeType(loaded, version) : loaded;
+			// The same rule as shouldAssignExecuteMethod in packages/cli/src/utils.ts. Without it a
+			// declarative node with a webhook (WhatsApp send and wait) gets no input data for $json.
+			const declarative = versioned.description?.requestDefaults !== undefined;
+			if (!versioned.execute && !versioned.supplyData && !versioned.poll && !versioned.trigger && (!versioned.webhook || declarative) && (!versioned.methods || declarative)) {
+				versioned.execute = async function () {
+					return (await new core.RoutingNode(this, versioned).runNode()) ?? [];
+				};
+			}
+			return versioned;
+		},
+		getKnownTypes: () => ({}),
+	};
+	const credentialType = (name) => instance(credentials, name);
+	const parentTypes = (name) => {
+		try {
+			return (credentialType(name).extends ?? []).flatMap((p) => [p, ...parentTypes(p)]);
+		} catch {
+			return [];
+		}
+	};
+	const credentialProperties = (name) => [...(credentialType(name).extends ?? []).flatMap(credentialProperties), ...(credentialType(name).properties ?? [])];
+	engine = { workflowLib, core, nodeTypes, nodes, credentialType, parentTypes, credentialProperties };
+	return engine;
+}
+
+/** Credential types that a node can use, from its node type description. */
+export function nodeCredentialTypes(node) {
+	try {
+		return (loadEngine().nodeTypes.getByNameAndVersion(node.type, node.typeVersion).description.credentials ?? []).map((c) => c.name);
+	} catch {
+		return [];
+	}
+}
+
+/** Apply the generic authenticate rule of a credential type, such as `qs.key` of Google Gemini. */
+function applyGenericAuth(credentialType, credentials, request) {
+	const auth = credentialType.authenticate;
+	if (!auth || typeof auth !== 'object' || auth.type !== 'generic') return request;
+	const fill = (v) => (typeof v === 'string' ? v.replace(/^=/, '').replace(/\{\{\s*\$credentials\.(\w+)\s*\}\}/g, (_m, k) => credentials[k] ?? '') : v);
+	for (const part of ['headers', 'qs', 'body']) {
+		const props = auth.properties?.[part];
+		if (props) request[part] = { ...(request[part] ?? {}), ...Object.fromEntries(Object.entries(props).map(([k, v]) => [k, fill(v)])) };
+	}
+	return request;
+}
+
+function credentialsHelper(realModels) {
+	const { core, parentTypes, credentialProperties, credentialType } = loadEngine();
+	return {
+		getParentTypes: parentTypes,
+		getDecrypted: async (_data, _details, type) => {
+			const mocked = core.buildEvalMockCredentials(credentialProperties(type));
+			const model = MODEL_CREDENTIALS[type];
+			if (realModels && model) mocked[model.field] = modelKey(type);
+			return mocked;
+		},
+		// Vendor model nodes send their request through the n8n helpers, so the real key must be applied here.
+		authenticate: async (credentials, type, request) => (realModels && MODEL_CREDENTIALS[type] ? applyGenericAuth(credentialType(type), credentials, request) : request),
+		preAuthentication: async () => undefined,
+		runPreAuthentication: async () => undefined,
+		getCredentials: async () => ({}),
+		getCredentialsProperties: (type) => credentialProperties(type),
+		updateCredentials: async () => {},
+		updateCredentialsOauthTokenData: async () => {},
+		isCredentialUsableByNode: () => true,
+	};
+}
+
+// --- Setup values ----------------------------------------------------------------
+
+/** Values that a node accepts for a parameter that the builder left for setup. */
+const SETUP_VALUES = {
+	'googleSheets.sheetName': 'gid=0',
+};
+
+const isEmptyValue = (v) => v === undefined || v === null || v === '';
+
+/**
+ * A copy of the workflow with placeholder values in the parameters that the builder left for
+ * the user: empty resource locators, and empty parameters that the editor reports as required.
+ * Returns { workflow, filled }.
+ */
+export function fillSetup(workflow, canvas) {
+	const copy = structuredClone(workflow);
+	const filled = [];
+	const required = new Map();
+	for (const node of canvas?.nodes ?? []) {
+		for (const [param, messages] of Object.entries(node.issues?.parameters ?? {})) {
+			if (messages.some((m) => /is required/i.test(m))) required.set(`${node.name}|${param}`, true);
+		}
+	}
+	for (const node of copy.nodes ?? []) {
+		const short = node.type.slice(node.type.lastIndexOf('.') + 1);
+		for (const [param, value] of Object.entries(node.parameters ?? {})) {
+			const placeholder = SETUP_VALUES[`${short}.${param}`] ?? `mock-${param}`;
+			if (value && typeof value === 'object' && value.__rl && isEmptyValue(value.value)) {
+				node.parameters[param] = { ...value, value: placeholder };
+				filled.push(`${node.name} › ${param}`);
+			} else if (isEmptyValue(value) && required.has(`${node.name}|${param}`)) {
+				node.parameters[param] = placeholder;
+				filled.push(`${node.name} › ${param}`);
+			}
+		}
+	}
+	return { workflow: copy, filled };
+}
+
+// --- Wire responses --------------------------------------------------------------
+
+const WRITE_METHODS = new Set(['POST', 'PUT', 'PATCH', 'DELETE']);
+export const isWrite = (request) => WRITE_METHODS.has(request.method);
+
+/** Rows for a read node: [{ Question, Answer }, ...] become the header row and the value rows. */
+function sheetValues(rows) {
+	const headers = [...new Set(rows.flatMap((r) => Object.keys(r ?? {})))];
+	if (!headers.length) return [['Question', 'Answer']];
+	return [headers, ...rows.map((r) => headers.map((h) => (r?.[h] === undefined ? '' : String(r[h]))))];
+}
+
+/** Page properties as Notion returns them: each with an id, a type, and plain_text in rich text. */
+function notionProperties(sent = {}) {
+	const out = {};
+	for (const [name, value] of Object.entries(sent ?? {})) {
+		const type = value?.type ?? Object.keys(value ?? {}).find((k) => k !== 'id' && k !== 'type') ?? 'rich_text';
+		let content = value?.[type];
+		if (Array.isArray(content)) content = content.map((t) => ({ type: 'text', ...t, plain_text: t?.plain_text ?? t?.text?.content ?? '' }));
+		out[name] = { id: `prop-${Object.keys(out).length}`, type, [type]: content ?? null };
+	}
+	if (!Object.values(out).some((p) => p.type === 'title')) out.Name = { id: 'title', type: 'title', title: [] };
+	return out;
+}
+
+const bodyOf = (request) => {
+	if (typeof request.body !== 'string') return request.body ?? {};
+	try {
+		return JSON.parse(request.body);
+	} catch {
+		return {};
+	}
+};
+
+/**
+ * Responses in the shape that each API documents. A fixture matches on host, method and path.
+ * rows(node) gives the case rows for a read node (scenarios.json pinByType).
+ */
+const FIXTURES = [
+	{
+		service: 'Google Sheets',
+		host: /(^|\.)sheets\.googleapis\.com$/,
+		method: 'GET',
+		path: /^\/v4\/spreadsheets\/[^/]+$/,
+		respond: () => ({ spreadsheetId: 'mock-sheet', properties: { title: 'FAQ' }, sheets: [{ properties: { sheetId: 0, title: 'Sheet1', index: 0, gridProperties: { rowCount: 100, columnCount: 26 } } }] }),
+	},
+	{
+		service: 'Google Sheets',
+		host: /(^|\.)sheets\.googleapis\.com$/,
+		method: 'GET',
+		path: /^\/v4\/spreadsheets\/[^/]+\/values\//,
+		respond: (_r, ctx, node) => ({ range: 'Sheet1!A1:Z100', majorDimension: 'ROWS', values: sheetValues(ctx.rows(node) ?? []) }),
+	},
+	{
+		service: 'Google Sheets',
+		host: /(^|\.)sheets\.googleapis\.com$/,
+		method: 'POST',
+		path: /:append$|values:batchUpdate$|:batchUpdate$/,
+		respond: () => ({ spreadsheetId: 'mock-sheet', updates: { updatedRange: 'Sheet1!A2:B2', updatedRows: 1 }, replies: [] }),
+	},
+	{
+		service: 'Notion',
+		host: /(^|\.)api\.notion\.com$/,
+		method: 'GET',
+		path: /^\/v1\/(databases|data_sources)\/[^/]+$/,
+		respond: (r) => ({
+			object: r.url.includes('data_sources') ? 'data_source' : 'database',
+			id: 'mock-database',
+			title: [{ type: 'text', plain_text: 'Support tickets', text: { content: 'Support tickets' } }],
+			properties: { Name: { id: 'title', name: 'Name', type: 'title', title: {} } },
+			data_sources: [{ id: 'mock-data-source', name: 'Support tickets' }],
+		}),
+	},
+	{
+		service: 'Notion',
+		host: /(^|\.)api\.notion\.com$/,
+		method: 'POST',
+		path: /^\/v1\/pages$/,
+		respond: (r) => {
+			const body = bodyOf(r);
+			return { object: 'page', id: '59833787-2cf9-4fdf-8782-e53db20768a5', created_time: '2026-01-01T09:00:00.000Z', last_edited_time: '2026-01-01T09:00:00.000Z', archived: false, in_trash: false, parent: body.parent ?? {}, properties: notionProperties(body.properties), url: 'https://www.notion.so/Ticket-598337872cf94fdf8782e53db20768a5', public_url: null };
+		},
+	},
+	{
+		service: 'Notion',
+		host: /(^|\.)api\.notion\.com$/,
+		method: 'PATCH',
+		path: /^\/v1\/blocks\/[^/]+\/children$/,
+		respond: (r) => ({ object: 'list', results: (bodyOf(r).children ?? []).map((c, i) => ({ object: 'block', id: `mock-block-${i}`, ...c })), next_cursor: null, has_more: false }),
+	},
+	{
+		service: 'Slack',
+		host: /(^|\.)slack\.com$/,
+		method: 'POST',
+		path: /^\/api\/chat\.postMessage$/,
+		respond: (r) => {
+			const body = bodyOf(r);
+			return { ok: true, channel: body.channel ?? 'C0000000000', ts: '1767258000.000100', message: { type: 'message', text: body.text ?? '', ts: '1767258000.000100', bot_id: 'B0000000000' } };
+		},
+	},
+	{
+		service: 'WhatsApp Cloud API',
+		host: /(^|\.)graph\.facebook\.com$/,
+		method: 'POST',
+		path: /^\/v\d+(\.\d+)?\/[^/]+\/messages$/,
+		respond: (r) => {
+			const to = bodyOf(r).to ?? '';
+			return { messaging_product: 'whatsapp', contacts: [{ input: to, wa_id: to }], messages: [{ id: 'wamid.HBgLMTU1NTAwMDExMTEVAgARGBI5QTNDQTVCM0Q0Q0Q2RTY3RTcA' }] };
+		},
+	},
+	{
+		service: 'Telegram Bot API',
+		host: /(^|\.)api\.telegram\.org$/,
+		method: 'POST',
+		path: /\/sendMessage$/,
+		respond: (r) => ({ ok: true, result: { message_id: 2, date: 1767258000, chat: { id: bodyOf(r).chat_id ?? 1001, type: 'private' }, text: bodyOf(r).text ?? '' } }),
+	},
+	{
+		service: 'Gmail',
+		host: /(^|\.)gmail\.googleapis\.com$|(^|\.)www\.googleapis\.com$/,
+		method: 'POST',
+		path: /\/messages\/send$/,
+		respond: () => ({ id: 'mock-message', threadId: 'mock-thread', labelIds: ['SENT'] }),
+	},
+];
+
+/**
+ * A request handler for evalLlmMockHandler. Every request is recorded in ctx.requests with the
+ * fixture that answered it. A request with no fixture gets a generic answer: a read gets an
+ * empty object, a write gets its own body back with an id. A request to a host in passHosts
+ * goes to the network: these are the model APIs of the real-AI scenarios.
+ */
+export function wireHandler(ctx, passHosts = new Set()) {
+	return async (request, node) => {
+		const url = new URL(request.url ?? '', 'https://unknown.invalid');
+		if (request.baseURL && !/^https?:/.test(request.url ?? '')) {
+			const base = new URL(request.baseURL);
+			url.host = base.host;
+			url.pathname = `${base.pathname.replace(/\/$/, '')}/${(request.url ?? '').replace(/^\//, '')}`;
+		}
+		const method = String(request.method ?? 'GET').toUpperCase();
+		if (passHosts.has(url.hostname)) {
+			ctx.requests.push({ node: node.name, nodeType: node.type, method, url: url.toString(), body: request.body, answeredBy: 'real model' });
+			return undefined;
+		}
+		const fixture = FIXTURES.find((f) => f.host.test(url.hostname) && f.method === method && f.path.test(url.pathname));
+		const record = { node: node.name, nodeType: node.type, method, url: url.toString(), qs: request.qs, body: request.body, answeredBy: fixture?.service ?? 'generic' };
+		let body;
+		try {
+			body = fixture ? fixture.respond({ ...request, url: url.toString() }, ctx, node) : method === 'GET' ? {} : { id: 'mock-1', ...(bodyOf(request) ?? {}) };
+		} catch (error) {
+			body = { error: String(error) };
+		}
+		ctx.requests.push({ ...record, response: body });
+		return { statusCode: 200, headers: { 'content-type': 'application/json' }, body };
+	};
+}
+
+// --- Network guard ---------------------------------------------------------------
+
+/**
+ * Replace global fetch while a run is active. Model SDKs use fetch directly, so this is the
+ * only path that could reach the network. Only the hosts in allow pass, the rest is blocked.
+ */
+function guardFetch(allow, blocked) {
+	const original = globalThis.fetch;
+	globalThis.fetch = async (input, init) => {
+		const url = new URL(typeof input === 'string' ? input : input.url ?? String(input));
+		if (allow.has(url.hostname)) return await original(input, init);
+		blocked.push(`${init?.method ?? 'GET'} ${url.origin}${url.pathname}`);
+		throw new Error(`Blocked network request to ${url.hostname} during a test run.`);
+	};
+	return () => {
+		globalThis.fetch = original;
+	};
+}
+
+// --- Code nodes ------------------------------------------------------------------
+
+/**
+ * additionalData.startRunnerTask. The n8n server sends Code node jobs to a sandboxed task runner.
+ * This process has no sandbox, so it never runs builder code. run_wire.mjs skips workflows with a
+ * Code node. This refusal catches a node type that it does not know.
+ */
+async function startRunnerTask(_data, jobType) {
+	return { ok: false, error: { message: `${jobType} code does not run in the in-process run.` } };
+}
+
+// --- Run -------------------------------------------------------------------------
+
+const CANCEL_GRACE_MS = 15_000;
+
+/** The value of promise, or null after ms. */
+async function within(promise, ms) {
+	let timer;
+	try {
+		return await Promise.race([promise, new Promise((resolve) => (timer = setTimeout(() => resolve(null), ms)))]);
+	} finally {
+		clearTimeout(timer);
+	}
+}
+
+/** The pinned trigger that a run starts from: triggerNode, or else the first pinned trigger. */
+export const startNode = (workflow, pinData, triggerNode) =>
+	workflow.nodes.find((n) => (!triggerNode || n.name === triggerNode) && Object.hasOwn(pinData, n.name) && !n.disabled && Object.keys(workflow.connections ?? {}).includes(n.name) && /trigger|webhook/i.test(n.type));
+
+/**
+ * Run a workflow once. pinData holds the trigger and any pinned nodes. triggerNode names the
+ * trigger to start from. With realModels, model sub-nodes get the real key and fetch can reach
+ * their API hosts. Returns { status, error, runData, requests, blocked, ms }.
+ */
+export async function runWorkflow(workflow, { pinData, triggerNode, rows = () => undefined, realModels = false, timeoutMs = 90_000 }) {
+	const { workflowLib, core, nodeTypes } = loadEngine();
+	const started = Date.now();
+	const ctx = { requests: [], rows };
+	const blocked = [];
+	const allow = new Set(realModels ? Object.values(MODEL_CREDENTIALS).flatMap((m) => m.hosts) : []);
+	const restore = guardFetch(allow, blocked);
+	let stillRunning = false;
+	try {
+		const instance = new workflowLib.Workflow({ id: 'wire-run', name: workflow.name, nodes: workflow.nodes, connections: workflow.connections, nodeTypes, active: false, settings: workflow.settings ?? {} });
+		const start = startNode(workflow, pinData, triggerNode);
+		if (!start) return { status: 'error', error: 'No pinned trigger to start from.', runData: {}, requests: [], blocked, ms: 0 };
+		const hooks = new core.ExecutionLifecycleHooks('manual', 'wire-run', workflow);
+		const finished = new Promise((resolve) => hooks.addHandler('workflowExecuteAfter', (run) => resolve(run)));
+		const values = {
+			executionId: 'wire-run',
+			hooks,
+			credentialsHelper: credentialsHelper(realModels),
+			evalLlmMockHandler: wireHandler(ctx, allow),
+			startRunnerTask,
+			getRunnerStatus: () => ({ available: false }),
+			variables: {},
+			restApiUrl: 'http://localhost:5678/rest',
+			instanceBaseUrl: 'http://localhost:5678',
+			webhookBaseUrl: 'http://localhost:5678/webhook',
+			webhookTestBaseUrl: 'http://localhost:5678/webhook-test',
+			webhookWaitingBaseUrl: 'http://localhost:5678/webhook-waiting',
+			formWaitingBaseUrl: 'http://localhost:5678/form-waiting',
+			userId: 'wire-run-user',
+		};
+		// Unset fields stay undefined. Methods that n8n calls for logs and UI updates do nothing.
+		const additionalData = new Proxy({}, {
+			get: (_t, prop) => (prop in values ? values[prop] : typeof prop === 'string' && /^(log|send|set|on|emit|track)/.test(prop) ? () => undefined : undefined),
+		});
+		const runExecutionData = workflowLib.createRunExecutionData({
+			executionData: { nodeExecutionStack: [{ node: start, data: { main: [pinData[start.name]] }, source: null }] },
+			resultData: { pinData },
+		});
+		const execute = new core.WorkflowExecute(additionalData, 'manual', runExecutionData);
+		const execution = execute.processRunExecutionData(instance);
+		const run = await within(execution.then(() => finished), timeoutMs);
+		if (!run) {
+			// Cancel aborts the running node. Wait for the engine to stop, so no request runs after the guard is gone.
+			execution.cancel();
+			stillRunning = !(await within(execution.then(() => true, () => true), CANCEL_GRACE_MS));
+			const error = `No result after ${timeoutMs / 1000} s.${stillRunning ? ' The run did not stop after the cancel.' : ''}`;
+			return { status: 'timeout', error, runData: {}, requests: ctx.requests, blocked, ms: Date.now() - started };
+		}
+		const error = run.data?.resultData?.error;
+		return { status: run.status, error: error ? error.message ?? String(error) : null, runData: run.data?.resultData?.runData ?? {}, requests: ctx.requests, blocked, ms: Date.now() - started };
+	} catch (error) {
+		return { status: 'error', error: `${error?.name ?? 'Error'}: ${error?.message ?? error}`, runData: {}, requests: ctx.requests, blocked, ms: Date.now() - started };
+	} finally {
+		// A run that did not stop keeps the guard. The process exits when all runs are done.
+		if (!stillRunning) restore();
+	}
+}

.aikido

@@ -8,7 +8,7 @@ exclude:
     # Machine-generated instance-AI expectation traces. Contain ephemeral
     # credential reference IDs (nanoids) that re-trip the secrets scanner
     # on every regeneration. No secret material.
-    - packages/testing/playwright/expectations
+    - packages/quality/testing/playwright/expectations
 
     # Test code repo-wide. Fixtures routinely contain synthetic PEM blocks,
     # basic-auth strings, and deliberately fake keys (including the secret

.boundaries-baseline.json

@@ -1,3 +1,3 @@
 {
-  "issues": 141
+  "issues": 110
 }

.claude/plugins/n8n/skills/env-var-docs

@@ -0,0 +1 @@
+../../../../.agents/skills/env-var-docs

.claude/plugins/n8n/skills/testmax-assistant-ui

@@ -0,0 +1 @@
+../../../../.agents/skills/testmax-assistant-ui

.claude/plugins/n8n/skills/testmax-n8n-ai

@@ -0,0 +1 @@
+../../../../.agents/skills/testmax-n8n-ai

.devcontainer/codespaces/README.md

@@ -2,7 +2,7 @@
 
 Run long-lived, human-steered agent sessions (Claude Code / OpenCode) on a
 GitHub Codespace instead of your laptop: start a task, close the lid, steer it
-from anywhere with a terminal, resume tomorrow.
+from anywhere, resume tomorrow.
 
 This is a separate devcontainer config from the laptop one in
 `.devcontainer/` — it ships both agent CLIs, `tmux` for session persistence,
@@ -11,17 +11,61 @@ Playwright system deps, and Docker-in-Docker (for testcontainers and
 
 ## One-time setup (~5 min)
 
-1. Add provider keys at [github.com/settings/codespaces](https://github.com/settings/codespaces).
+1. Give the GitHub CLI the codespace scope:
-   Add `ANTHROPIC_API_KEY` for Claude Code. Add `OPENROUTER_API_KEY` for OpenCode.
-   Give both secrets access to `n8n-io/n8n`.
-   (Alternative for Max subscriptions: `CLAUDE_CODE_OAUTH_TOKEN` from `claude setup-token`.)
-2. Give the GitHub CLI the codespace scope:
 
    ```bash
    gh auth refresh -h github.com -s codespace
    ```
 
-## Daily flow
+2. Add provider keys at [github.com/settings/codespaces](https://github.com/settings/codespaces)
+   if your client needs them. Give each secret access to `n8n-io/n8n`.
+   - `ANTHROPIC_API_KEY` for Claude Code in the terminal (`pnpm session`).
+     Alternative for Max subscriptions: `CLAUDE_CODE_OAUTH_TOKEN` from
+     `claude setup-token`. The desktop app does not need either secret.
+   - `OPENROUTER_API_KEY` for OpenCode.
+
+## Choose a client
+
+All clients run the agent, its tools, and builds in the Codespace. Your laptop
+shows the conversation only.
+
+| Client | Command | Use it when |
+|---|---|---|
+| Claude Code desktop app | `pnpm session ssh-config` once | You want the full desktop experience. |
+| Claude Code in VS Code | Open the codespace in VS Code or the browser | You already work in VS Code. |
+| Claude Code in a terminal | `pnpm session` | You want a plain terminal, or you are on a remote machine. |
+| OpenCode | `pnpm session:opencode` | You use OpenCode. See [Local OpenCode clients](#local-opencode-clients). |
+
+### Claude Code desktop app
+
+1. From a local checkout, add an SSH host for your codespace:
+
+   ```bash
+   pnpm session ssh-config
+   ```
+
+   The command creates or starts the codespace. It writes the host
+   `n8n-codespace` to `~/.ssh/n8n-codespace.conf` and includes that file from
+   `~/.ssh/config`.
+2. In the desktop app, add an SSH connection to the host `n8n-codespace`.
+3. Open the folder `/workspaces/n8n`, or a worktree under `/workspaces`.
+
+The desktop app installs its own Claude Code on the codespace and signs in
+with your desktop account. Sessions continue when you close the app. They stop
+when the codespace stops.
+
+- **Run `pnpm session ssh-config` again after you recreate the codespace.**
+- **If the connection times out**, the codespace is probably starting. Run
+  `pnpm session ssh-config`, then connect again.
+- **Test the connection** with `ssh n8n-codespace`.
+
+### Claude Code in VS Code
+
+The dev container installs the Claude Code extension. Open the codespace in
+VS Code or the browser, then open the Claude Code panel. Sign in from the
+extension if you did not add `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN`.
+
+### Claude Code in a terminal
 
 ```bash
 pnpm session                       # attach Claude Code (creates everything on first run)
@@ -33,6 +77,7 @@ pnpm session:opencode --legacy     # use the remote TUI in tmux
 pnpm session fix-flaky             # Claude Code in a separate worktree
 pnpm session ls                    # what's running
 pnpm session tunnel                # forward n8n ports (default 5678, 8080); Ctrl-C to stop
+pnpm session ssh-config            # add the n8n-codespace SSH host for the desktop app
 pnpm session stop                  # end of day: billing stops, disk survives
 pnpm session rm                    # delete the codespace
 ```
@@ -332,11 +377,15 @@ session rarely needs a cold `pnpm install` or a full `pnpm build`. Both are slow
 Claude sessions and the headless OpenCode worker get the `flaky` MCP server automatically: Currents
 flaky/quarantine data, the `qa_*` BigQuery dataset, Sentry RCA, live Linear,
 and repo investigation. The worker keeps the token in its environment and puts
-only an environment reference in the OpenCode config. Claude login registers
+only an environment reference in the OpenCode config. Forks have no secrets and
-the same server without writing the token to disk. Forks have no secrets and
 skip it. Tell the agent to call `get_flaky_context` first — it returns the rules
 the tools assume.
 
+`post-start.mjs` registers the server for Claude Code on each container start.
+Its `headersHelper` reads the token from the secrets file on each connect, so
+the server works in every client. The token is not copied into the Claude Code
+MCP configuration.
+
 ## Quality and security skills (Claude plugins)
 
 Claude sessions can also load the private skills from the
@@ -398,7 +447,8 @@ to pull the skills into context.
 
 ## Viewing the dev UI locally
 
-Two terminal windows:
+`pnpm session tunnel` works with every client. With the terminal client, use
+two terminal windows:
 
 ```bash
 pnpm session          # window 1: attach the agent session
@@ -448,10 +498,10 @@ After a stop, `pnpm session <name>` restarts the codespace (~30–60 s); run
   agent injects them into VS Code sessions only; they're delivered
   base64-encoded to `/workspaces/.codespaces/shared/.env-secrets`. The image
   sources `/usr/local/lib/codespaces-env.sh` in login shells (profile.d), in
-  interactive shells (bashrc), and in the `pnpm session` prelude. If Claude
+  interactive shells (bashrc), and in the `pnpm session` prelude. Desktop app
-  Code shows `Missing environment variables: FLAKY_MCP_TOKEN`, the shell that
+  sessions do not source it. `git`, `gh`, `scripts/codespace-env.mjs`, and the
-  started Claude did not source the file. Run
+  `flaky` MCP entry read the file when they run, so they work everywhere. New
-  `. /usr/local/lib/codespaces-env.sh` and start Claude again.
+  code that needs a secret must do the same.
 - **Do not read `CODESPACE_NAME` or `GITHUB_USER` from the process env** — use
   `scripts/codespace-env.mjs`. Codespaces gives these variables to VS Code
   sessions only. Other processes read them from `codespaces-env.sh`, and a
@@ -460,7 +510,7 @@ After a stop, `pnpm session <name>` restarts the codespace (~30–60 s); run
   polled correctly as its owner while `dev:up` in the same session saw an empty
   box name, printed the localhost URL, and did not share the port. The helper
   reads `/workspaces/.codespaces/shared`, which is always correct.
-- **You cannot paste images into a remote Claude session.** Image paste reads
+- **You cannot paste images into a terminal Claude session.** Image paste reads
   the clipboard of the machine where `claude` runs — the codespace, not your
   laptop. Drag the file into the VS Code explorer (or
   `gh codespace cp shot.png remote:/workspaces/n8n/`) and give Claude the

.devcontainer/codespaces/codespaces-secrets.sh

@@ -12,14 +12,6 @@
 	'{"hasCompletedOnboarding":true,"theme":"dark","projects":{"/workspaces/n8n":{"hasTrustDialogAccepted":true}}}' \
 	>"$HOME/.claude.json"
 
-# Register the Flaky MCP server for Claude Code. The config keeps a literal
-# ${FLAKY_MCP_TOKEN}; Claude Code expands it at connect time, so the token is
-# not written to disk. Forks have no repo secrets and skip this.
-if [ -n "$FLAKY_MCP_URL" ] && ! grep -qs '"flaky"' "$HOME/.claude.json" && command -v claude >/dev/null 2>&1; then
-	claude mcp add --scope user --transport http flaky "$FLAKY_MCP_URL" \
-		--header 'Authorization: Bearer ${FLAKY_MCP_TOKEN}' >/dev/null 2>&1 || true
-fi
-
 # Register the credential helper in the user config, because Codespaces
 # regenerates the managed /etc/gitconfig. When the env token is missing, the
 # system helper exits 0 without output, and git falls through to ours.

.devcontainer/codespaces/devcontainer.json

@@ -18,9 +18,13 @@
 		"ghcr.io/devcontainers/features/docker-in-docker:2": {}
 	},
 	"forwardPorts": [8080, 5678],
-	// Grant the Codespace token read access to the private skills and harness repositories.
-	// Each user authorizes this access once when they create the Codespace.
 	"customizations": {
+		// Claude Code in VS Code or the browser editor. It runs in the codespace.
+		"vscode": {
+			"extensions": ["anthropic.claude-code"]
+		},
+		// Grant the Codespace token read access to the private skills and harness repositories.
+		// Each user authorizes this access once when they create the Codespace.
 		"codespaces": {
 			"repositories": {
 				"n8n-io/n8n-agent-skills": { "permissions": { "contents": "read" } },

.devcontainer/codespaces/post-start.mjs

@@ -1,10 +1,11 @@
 #!/usr/bin/env node
 // Runs on each Codespace start. Installs the skills and harness, then starts the worker.
 import { execFileSync } from 'node:child_process';
-import { rmSync, writeFileSync } from 'node:fs';
+import { readFileSync, rmSync, writeFileSync } from 'node:fs';
 import { homedir } from 'node:os';
 import { join } from 'node:path';
 import { installAgentHarness } from '../../scripts/agent-harness.mjs';
+import { codespaceSecret } from '../../scripts/codespace-env.mjs';
 import { MARKETPLACE, PLUGINS } from './plugins.mjs';
 
 const STATUS_FILE = '/tmp/post-start-status.json';
@@ -49,6 +50,42 @@ function addMarketplace() {
 	return add('marketplace add (retry)');
 }
 
+// Claude Code reads the Flaky token when it connects. An env reference such as
+// ${FLAKY_MCP_TOKEN} fails in processes that did not load the secrets file, for
+// example the desktop app's SSH server. The helper reads the file on each connect,
+// so it also gets a rotated token. The token is not copied into ~/.claude.json.
+const FLAKY_HEADERS_HELPER =
+	'. /usr/local/lib/codespaces-env.sh; printf %s "{\\"Authorization\\":\\"Bearer $FLAKY_MCP_TOKEN\\"}"';
+
+function registerFlakyMcp() {
+	// Forks have no repository secrets.
+	const url = codespaceSecret('FLAKY_MCP_URL');
+	if (!url || !codespaceSecret('FLAKY_MCP_TOKEN')) return;
+
+	let current;
+	try {
+		current = JSON.parse(readFileSync(join(homedir(), '.claude.json'), 'utf8')).mcpServers?.flaky;
+	} catch {
+		// No readable config yet: register the server below.
+	}
+	if (current?.url === url && current.headersHelper === FLAKY_HEADERS_HELPER && !current.headers) {
+		return;
+	}
+
+	if (current) tryRun('flaky mcp remove', 'claude', ['mcp', 'remove', '--scope', 'user', 'flaky']);
+	const config = { type: 'http', url, headersHelper: FLAKY_HEADERS_HELPER };
+	tryRun('flaky mcp add', 'claude', [
+		'mcp',
+		'add-json',
+		'--scope',
+		'user',
+		'flaky',
+		JSON.stringify(config),
+	]);
+}
+
+registerFlakyMcp();
+
 tryRun('skills repo reachable', 'git', ['ls-remote', `https://github.com/${MARKETPLACE}`, 'HEAD']);
 
 const installed = [];

.devcontainer/codespaces/ssh-config.test.mjs

@@ -0,0 +1,32 @@
+import assert from 'node:assert/strict';
+import { test } from 'node:test';
+
+import { aliasHostConfig, withInclude } from '../../scripts/cloud-session-ssh-config.mjs';
+
+const GH_CONFIG = `Host cs.ominous-doodle.master
+	User node
+	ProxyCommand gh cs ssh -c ominous-doodle --stdio
+	UserKnownHostsFile=/dev/null
+`;
+
+test('replaces the gh host name with the fixed alias', () => {
+	assert.equal(
+		aliasHostConfig(GH_CONFIG),
+		GH_CONFIG.replace('cs.ominous-doodle.master', 'n8n-codespace'),
+	);
+});
+
+test('rejects gh output without exactly one proxied host', () => {
+	assert.throws(() => aliasHostConfig(''));
+	assert.throws(() => aliasHostConfig(`${GH_CONFIG}Host other\n`));
+	assert.throws(() => aliasHostConfig('Host cs.x\n\tUser node\n'));
+});
+
+test('adds the Include line at the top once', () => {
+	const include = 'Include ~/.ssh/n8n-codespace.conf';
+	assert.equal(withInclude(''), `${include}\n`);
+
+	const updated = withInclude('Host github.com\n\tUser git\n');
+	assert.equal(updated, `${include}\n\nHost github.com\n\tUser git\n`);
+	assert.equal(withInclude(updated), undefined);
+});

.dockerignore

@@ -26,7 +26,7 @@
 !docker/images/runners
 !docker/images/runners/n8n-task-runners.json
 
-# === benchmark image (packages/@n8n/benchmark/Dockerfile) ===
+# === benchmark image (packages/quality/efficiency/scale/benchmark/Dockerfile) ===
 !package.json
 !pnpm-lock.yaml
 !pnpm-workspace.yaml
@@ -34,8 +34,10 @@
 !patches/**
 !scripts
 !scripts/**
-!packages/@n8n/benchmark
+!packages/quality
-!packages/@n8n/benchmark/**
+!packages/quality/efficiency
+!packages/quality/efficiency/scale
+!packages/quality/efficiency/scale/benchmark
+!packages/quality/efficiency/scale/benchmark/**
 !packages/@n8n/typescript-config
 !packages/@n8n/typescript-config/**
-

.env.ai-surfaces.example

@@ -0,0 +1,75 @@
+# Example env for comparing AI workflow builders (Instance AI, Claude Code + n8n MCP,
+# Codex + n8n MCP) on a local instance.
+#
+# Copy to `.env.ai-surfaces` (gitignored) and fill in the values that you need.
+# Never commit keys. The testmax-n8n-ai skill reads the files in this
+# order, from low to high priority: .env.local, .env.ai-surfaces, the shell
+# environment. Empty values are ignored, so keys that are already in .env.local
+# work without a change. Set AI_SURFACES_ENV to use a file in a different place.
+#
+# See .agents/skills/testmax-n8n-ai/SKILL.md for how the values are used.
+
+# --- n8n ---------------------------------------------------------------------
+# Empty values use .env.local, then the defaults: http://localhost:5678,
+# nathan@n8n.io and PlaywrightTest123.
+N8N_BASE_URL=
+N8N_EVAL_EMAIL=
+N8N_EVAL_PASSWORD=
+# Optional. The MCP key of the user above (Settings > MCP). When set, the skill
+# does not need the cached token and never asks to rotate the key.
+N8N_MCP_TOKEN=
+
+# --- Run defaults --------------------------------------------------------------
+COMPARE_BUILDERS=iai,claude,codex
+# container (Docker) or clean (Claude only, no Docker)
+COMPARE_ISOLATION=container
+COMPARE_RUNS_DIR=/tmp/builder-compare
+# 1 runs the builders at the same time. 0 runs them one at a time.
+COMPARE_PARALLEL=1
+# 1 opens a local page that shows the runs live and then the report. 0 turns it off.
+COMPARE_LIVE=1
+# 1 also opens the Instance AI thread in the browser and a terminal window for
+# each CLI builder.
+COMPARE_LIVE_WINDOWS=0
+# 1 records a video of the Instance AI thread for the replay on the page.
+COMPARE_RECORD=1
+# interactive records the real terminal screen of Claude Code and Codex.
+# headless uses `claude -p` and `codex exec` and has no screen recording.
+COMPARE_CLI_MODE=interactive
+
+# --- Claude Code + n8n MCP -----------------------------------------------------
+# Empty model uses the Claude Code default.
+CLAUDE_MODEL=
+# none: Claude Code has only the n8n tools. default: it also has its built-in tools,
+# such as Bash, Read, Write and WebFetch, and uses all of them without a question,
+# like Codex in the same container. Clean isolation always uses none.
+COMPARE_CLAUDE_TOOLS=none
+# 1 runs each created workflow with pinned data through the n8n MCP server, once
+# for each scenario in scenarios.json. 0 turns it off.
+COMPARE_SCENARIOS=1
+# 1 runs each created workflow in process with recorded API fixtures, for the
+# scenarios, edge cases and questions in scenarios.json. 0 turns it off.
+COMPARE_WIRE=1
+# 1 runs the deterministic binary checks of the Instance AI eval suite.
+COMPARE_BINARY_CHECKS=1
+# Model keys for the real AI runs of the questions in scenarios.json. A workflow
+# model without a key skips those runs. Set the key of each model provider that
+# the builders can pick: GOOGLE_GENERATIVE_AI_API_KEY, OPENAI_API_KEY,
+# ANTHROPIC_API_KEY, MISTRAL_API_KEY or GROQ_API_KEY.
+GOOGLE_GENERATIVE_AI_API_KEY=
+# Falls back to ANTHROPIC_API_KEY.
+CLAUDE_API_KEY=
+
+# --- Codex + n8n MCP -----------------------------------------------------------
+# Empty model uses the Codex default.
+CODEX_MODEL=
+# Falls back to OPENAI_API_KEY.
+CODEX_API_KEY=
+
+# --- Instance AI cost (LangSmith) ----------------------------------------------
+# The n8n instance must trace to the same project (LANGSMITH_TRACING=true).
+LANGSMITH_API_KEY=
+# Use the same region as the instance. Empty uses https://api.smith.langchain.com.
+# For the EU region, set https://eu.api.smith.langchain.com.
+LANGSMITH_ENDPOINT=
+LANGSMITH_PROJECT=instance-ai

.githooks/check-pnpm.mjs

@@ -0,0 +1,36 @@
+#!/usr/bin/env node
+// Git hook helper (post-merge, post-rewrite, post-checkout): warn when the
+// installed pnpm major differs from the one pinned in package.json.
+// Usage: node .githooks/check-pnpm.mjs [checkout-type]
+// For post-checkout, pass lefthook's {3}: "0" is a file checkout, which we skip.
+import { spawnSync } from 'node:child_process';
+import { readFileSync } from 'node:fs';
+import { homedir } from 'node:os';
+import { fileURLToPath } from 'node:url';
+
+if (process.argv[2] === '0') process.exit(0);
+
+const packageJsonPath = fileURLToPath(new URL('../package.json', import.meta.url));
+const { packageManager = '' } = JSON.parse(readFileSync(packageJsonPath, 'utf8'));
+const wanted = /^pnpm@(\d+)/.exec(packageManager)?.[1];
+
+if (!wanted) process.exit(0);
+
+// Run from the home dir so pnpm reports the installed version,
+// not the version pinned by this repo.
+// `shell` lets Windows resolve pnpm.cmd. The command has no user input.
+const result = spawnSync('pnpm --version', { cwd: homedir(), shell: true, encoding: 'utf8' });
+const current = result.status === 0 ? result.stdout.trim().split('.')[0] : '';
+if (wanted === current) process.exit(0);
+
+const installCommand =
+	process.platform === 'win32'
+		? 'powershell -c "irm https://get.pnpm.io/install.ps1 | iex"'
+		: 'curl -fsSL https://get.pnpm.io/install.sh | sh -';
+
+console.log(`
+⚠ This repo now pins pnpm ${wanted} (you have ${current || 'none'}).
+  pnpm should auto-download it on the next install command.
+  If that fails (some pnpm 10 versions can't install v12), reinstall:
+    ${installCommand}
+`);

.githooks/check-pnpm.sh

@@ -1,13 +0,0 @@
-#!/bin/sh
-# .githooks/check-pnpm — symlink or call from post-merge, post-rewrite, post-checkout
-
-WANTED=$(sed -n 's/.*"packageManager": *"pnpm@\([0-9]*\).*/\1/p' package.json)
-CURRENT=$(cd ~ && pnpm --version 2>/dev/null | cut -d. -f1)
-[ -z "$WANTED" ] || [ "$WANTED" = "$CURRENT" ] && exit 0
-echo ""
-echo "⚠ This repo now pins pnpm $WANTED (you have ${CURRENT:-none})."
-echo "  pnpm should auto-download it on the next install command."
-echo "  If that fails (some pnpm 10 versions can't install v12), reinstall:"
-echo "    curl -fsSL https://get.pnpm.io/install.sh | sh -"
-echo ""
-exit 0

.github/CI-TELEMETRY.md

@@ -81,11 +81,12 @@ return 'blacksmith';
 
 | Telemetry | Source | Metrics |
 |-----------|--------|---------|
-| Playwright perf/benchmark | `packages/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
+| Playwright perf/benchmark | `packages/quality/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
-| Accessibility buckets | `packages/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
+| Accessibility buckets | `packages/quality/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
 | Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
 | Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
-| Container stack | `packages/testing/containers/telemetry.ts` | E2E stack startup times per service |
+| Dive image efficiency | `packages/quality/efficiency/images/image-efficiency.spec.ts` | Layer content size, estimated waste, efficiency score, and layer count through the shared Playwright reporter |
+| Container stack | `packages/quality/environments/containers/telemetry.ts` | E2E stack startup times per service |
 
 ## Secrets
 

.github/CODEOWNERS

@@ -1,11 +0,0 @@
-# Temporary: kept side by side with the top-level OWNERS file while the
-# "Required Reviews" enforcement is on trial. Do not add entries here;
-# add them to OWNERS. See "Transition from CODEOWNERS" in WORKFLOWS.md.
-packages/@n8n/db/src/migrations/ @n8n-io/migrations-review
-/OWNERS @n8n-io/qa-dx
-.github/workflows @n8n-io/qa-dx
-.github/scripts @n8n-io/qa-dx
-.github/actions @n8n-io/qa-dx
-.github/poutine-rules @n8n-io/qa-dx
-docker/get-n8n.sh @n8n-io/qa-dx
-.code-health-baseline.json @n8n-io/qa-dx

.github/WORKFLOWS.md

@@ -10,7 +10,6 @@ Complete reference for n8n's `.github/` folder.
 .github/
 ├── WORKFLOWS.md                          # This document
 ├── CI-TELEMETRY.md                       # Telemetry & metrics guide
-├── CODEOWNERS                            # Temporary, side by side with OWNERS during the trial
 ├── pull_request_template.md              # PR description template
 ├── pull_request_title_conventions.md     # Title format rules (Angular)
 ├── actionlint.yml                        # Workflow linter config
@@ -178,7 +177,7 @@ These only run if specific files changed:
 | `docker/images/n8n-base/Dockerfile`                                    | `build-base-image.yml`      | any        |
 | `**/package.json`, `**/turbo.json`                                     | `build-windows.yml`         | master     |
 | `packages/@n8n/ai-workflow-builder.ee/evaluations/programmatic/python/**` | `test-evals-python.yml`  | any        |
-| `packages/@n8n/benchmark/**`                                           | `build-benchmark-image.yml` | master     |
+| `packages/quality/efficiency/scale/benchmark/**`                       | `build-benchmark-image.yml` | master     |
 | `packages/cli/src/public-api/**/*.yml`, `packages/cli/src/public-api/**/*.yaml`, `packages/cli/src/public-api/**/*.css`, `packages/cli/src/public-api/v1/openapi-gen/**/*.ts`, `packages/cli/scripts/build.mjs`, `packages/cli/package.json` | `util-publish-api-schema.yml` | master   |
 | `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts`, `packages/@n8n/agents/src/**` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
 | `docker/get-n8n.sh`, `docker/get-n8n-compose.yml`, `docker/test-get-n8n.sh` | `test-get-n8n.yml`          | any        |
@@ -237,14 +236,6 @@ parallelism). See the `--build-via-mcp` section in
 | PR merged to `release/*`           | `release-publish.yml`          |
 | PR closed with `codespace-preview` | `util-codespace-preview.yml`   |
 
-### Manual Triggers (PR Comments)
-
-| Command            | Workflow                     | Permissions         |
-|--------------------|------------------------------|---------------------|
-| `/test-workflows`  | `test-workflows-callable.yml`| admin/write/maintain|
-
-**Why:** Re-run tests without pushing commits. Useful for flaky test investigation.
-
 ### Label Triggers
 
 | Label                 | Workflow                     | Effect                                             |
@@ -420,7 +411,9 @@ Run-workflow dropdown, which only a user with write access can pick.
 |-----------------------------|---------------------------------------------------------|
 | `util-codespace-preview.yml`| Wake, re-serve or delete a PR preview instance by hand   |
 | `util-data-tooling.yml`     | SQLite/PostgreSQL export/import validation (manual)     |
+| `test-image-efficiency.yml` | Measure n8n and runners image size and layer waste with Dive |
 | `util-probe-registry.yml`   | Diagnose slow npm metadata fetches (temporary)          |
+| `sec-sync-bundle-branches.yml` | Sync both bundle branches on demand, including before a bundle cut |
 
 ---
 
@@ -454,10 +447,6 @@ test-sbom-nightly.yml
 
 test-workflows-nightly.yml  (manual dispatch only — nightly schedule disabled, DEVP-544)
     └──────────────────────────▶  test-workflows-callable.yml
-
-PR Comment Dispatchers (triggered by /command in PR comments):
-test-workflows-pr-comment.yml
-    └──────────────────────────▶  test-workflows-callable.yml
 ```
 
 ---
@@ -598,7 +587,7 @@ Push to master
 | Daily 03:00               | `sec-sync-bundle-branches.yml`    | Merge the base into `bundle/*` |
 | Daily 00:00               | `docker-build-push.yml`           | Nightly Docker images    |
 | Daily 00:00               | `test-db.yml`                     | Database compatibility   |
-| Daily 00:00               | `test-e2e-performance-reusable.yml`| Performance E2E         |
+| Daily 00:00               | `test-e2e-performance-reusable.yml`| Browser performance and idle memory baselines |
 | Daily 00:00               | `release-storybook.yml`       | Storybook deploy         |
 | Daily 00:00               | `release-chromatic.yml`       | Visual regression        |
 | Daily 00:00               | `util-check-docs-urls.yml`        | Doc link validation      |
@@ -611,6 +600,7 @@ Push to master
 | Daily 08:00               | `build-v3-nightly.yml`            | Nightly v3 Docker images |
 | Monday 00:00              | `util-update-node-popularity.yml` | Node usage stats         |
 | Monday 02:00              | `test-e2e-coverage-weekly.yml`    | Weekly E2E coverage      |
+| Wednesday 04:00           | `util-update-get-n8n-versions.yml` | get-n8n stable versions |
 | Saturday 22:00            | `test-evals-ai.yml`               | AI workflow evals        |
 | 1st of month 04:00        | `util-refresh-cubic-schema.yml`   | Refresh vendored cubic schema |
 
@@ -850,9 +840,8 @@ See **[CI-TELEMETRY.md](CI-TELEMETRY.md)** for:
 
 ## OWNERS
 
-Team ownership lives in the top-level `OWNERS` file (this replaces the
+Team ownership lives in the top-level `OWNERS` file. The scripts that consume
-GitHub-native `CODEOWNERS` file; see the transition note below). The scripts
+it live in `.github/scripts/owners/`. Line format:
-that consume it live in `.github/scripts/owners/`. Line format:
 
 ```
 <pattern> <@org/team> [required]
@@ -884,8 +873,10 @@ The file drives four workflows:
 ### Required reviews
 
 An entry with the `required` option makes team approval mandatory: when a PR
-changes a file whose winning entry carries `required`, a member of each listed
+changes a non-test file whose winning entry carries `required`, a member of each
-team must approve the PR. `ci-owners-required-reviews.yml` evaluates this on
+listed team must approve the PR. Test files match the shared patterns in
+`test-files.mjs` and do not trigger required reviews.
+`ci-owners-required-reviews.yml` evaluates this on
 PR changes and review events, and reports a commit status
 named **Required Reviews** on the head SHA. A missing approval reports
 `pending` ("Waiting for approval from: …"), not `failure`, so an unreviewed PR
@@ -920,8 +911,7 @@ Reviews** status succeed without required OWNERS team approvals. A team member
 must apply the `large-scale-change` label. The gate checks the actor from the
 label event and verifies their current team membership. The label alone is not
 sufficient. Label changes re-evaluate the status. Removing the label removes
-the exemption. During the CODEOWNERS trial, GitHub still enforces its separate
+the exemption.
-code-owner review requirement.
 
 Every path that writes the status runs in the base repository context, because
 a fork-context run has no secrets and a read-only token. PR changes arrive
@@ -935,16 +925,6 @@ contribution whose runs still wait for approval gets no review-event
 re-evaluation until a maintainer approves the runs; `workflow_dispatch` with
 the PR number is the manual fallback.
 
-### Transition from CODEOWNERS
-
-During a trial period, `.github/CODEOWNERS` stays in place next to OWNERS:
-GitHub's native code-owner enforcement keeps gating merges while the
-"Required Reviews" status runs side by side. The two must agree — CODEOWNERS
-holds exactly the `required` entries of OWNERS (plus the OWNERS file itself)
-and must not gain new entries; new ownership goes into OWNERS. After the
-trial, delete `.github/CODEOWNERS`, remove "Require review from Code Owners"
-from the master ruleset, and delete this section (tracked in DEVP-887).
-
 ---
 
 ## Runner Selection
@@ -1161,40 +1141,28 @@ open the PR there. That PR **must stay a single-parent squash** — the publish
 posts to `#alerts-build` when that gate fails on a PR opened *from* `bundle/2.x` or
 `bundle/1.x` (link only, no PR title, since the branch is embargoed).
 
-`sec-sync-bundle-branches.yml` keeps those branches current, daily plus whenever a PR is
+`sec-sync-bundle-branches.yml` keeps those branches current nightly at 03:00 UTC and on
-merged into one (and on `workflow_dispatch`). It **merges the base into** the bundle branch
+`workflow_dispatch`, including before a bundle cut. It **merges the public base into** the
-via [`scripts/sync-bundle-branch.mjs`](scripts/sync-bundle-branch.mjs) and pushes without
+bundle branch via
-forcing. Every push is verified to carry exactly the tree a merge of the two sides would
+[`scripts/sync-bundle-branch.mjs`](scripts/sync-bundle-branch.mjs) and pushes without forcing.
-produce (`git merge-tree`); a mismatch, or a conflict marker, fails the run instead of pushing.
+Every push is verified to carry exactly the tree a merge of the two sides would produce (`git
+merge-tree`); a mismatch, or a conflict marker, fails the run instead of pushing.
 
-**A bundle branch is only ever built on published history.** Before it creates or merges
+**A bundle branch is only ever built on published history.** The sync fetches `master` or `1.x`
-anything, the sync fetches the same-named branch from `https://github.com/n8n-io/n8n.git`
+directly from `https://github.com/n8n-io/n8n.git`. It creates a missing bundle branch at that
-(anonymously — its token is scoped to the private repo) and checks that the private base tip is
+exact public SHA and merges that public SHA into an existing bundle branch. It never reads the
-contained in it. This matters because the `chore: Bundle/*` squash on private `master` is
+private base branch. This matters because the `chore: Bundle/*` squash on private `master` is
-*private-only*: the mirror above discards it in favour of the public cherry-pick of the same
+private-only until publication. The mirror can replace that commit with the public copy after
-changes. A sync in that window would root the branch on a commit that is about to disappear,
+the bundle branch is deleted. The next bundle branch must not inherit the replaced commit.
-leaving it carrying two commits for one set of fixes — and every fix PR cut from it inherits the
-dead one. The window is real: this workflow's daily cron and the mirror's hourly cron both fire
-at `:00` with nothing ordering them, which is why the check lives in the script rather than in
-step ordering.
-
-An unpublished cut is a **skip**, not a failure: the mirror discards that commit every hour
-regardless, so the run exits green with the reason in the log and the next one proceeds. A
-missing branch also self-heals, because the mirror re-dispatches this workflow while one is
-absent. A base ahead of public for **any other reason** fails the run — the mirror is stuck and
-will not clear it on its own, so fix that first (remove the commit, or dispatch **Security: Sync
-from Public** with `force`) and re-run. To sync a bundle branch immediately, dispatch the mirror
-and then this workflow. The blocking commits are listed in the run log; the failure annotation
-carries only a count, since a subject hints at the fix.
 
 Deleting a bundle branch takes its open PRs with it: GitHub moves each one onto the deleted
-branch's own base, and re-creating the branch does not move them back. So the sync then calls
+branch's own base, and re-creating the branch does not move them back. The next nightly or manual
+sync re-creates the branch and then calls
 [`sec-sync-retarget-prs.yml`](workflows/sec-sync-retarget-prs.yml), which moves every open PR
 on `master` onto `bundle/2.x` and every one on `1.x` onto `bundle/1.x`. It skips a bundle
 branch that does not exist, and skips PRs whose *head* is `bundle/*` — those are the
 `chore: Bundle/*` cut PRs, which target the base on purpose. It is also dispatchable on its
-own. `sec-sync-public-to-private.yml` only *dispatches* the bundle sync when it finds a branch
+own. The retarget runs inside the bundle sync after a missing branch is recreated.
-missing; the retarget runs inside that dispatched run, once the branch exists.
 
 **`bundle/*` is append-only — never rebase it, never force-push it.** These branches receive
 PRs, and rewriting a branch that receives PRs orphans the copies of its commits that the open
@@ -1207,21 +1175,19 @@ itself. To refresh a fix branch, use GitHub's **Update branch** button or
 sibling PR's merge base untouched, which is why only a rewrite breaks this.
 
 The costs of merging are deliberate and paid for: a merge commit per run, and fixes that have
-already been published staying in the branch's log (the old rebase dropped them as empty
+already been published staying in the branch's log. Neither reaches anything downstream,
-commits). Neither reaches anything downstream, because a bundle publishes as one squashed
+because a bundle publishes as one squashed commit taken from the tree rather than the history —
-commit taken from the tree rather than the history — the `chore: Bundle/*` PR's **diff** stays
+the `chore: Bundle/*` PR's **diff** stays
 exactly the pending fixes even when its commit list does not. For a list of what a bundle
-actually carries, read the fix PRs merged into the branch since the last cut, not
+actually carries, read the fix PRs merged into the branch since the last cut, not `base..bundle`.
-`base..bundle`. A lower cadence than the base's is fine too: a base push never re-triggered
-CI on the fix PRs, so syncing more often bought them nothing.
 
 There is **one job per bundle branch**. A conflict is detected from the merge tree before the
 working tree is touched, so the branch is left exactly as it was, that job **fails** (no
 green runs hiding a stalled branch) and `#alerts-security` gets a run link — while the other
-branch still syncs. Recovery is deliberate: merge the base into the branch locally, resolve,
+branch still syncs. Recovery is deliberate: merge the public base into the branch locally,
-push, then re-run the workflow — and that resolution then lives in the merge commit instead of
+resolve, push, then re-run the workflow — and that resolution then lives in the merge commit
-being re-litigated on every later run. The sync never resolves a conflict itself, unlike
+instead of being re-litigated on every later run. The sync never resolves a conflict itself,
-`util-sync-master-to-3x.yml`.
+unlike `util-sync-master-to-3x.yml`.
 
 See **[`../AGENTS.md`](../AGENTS.md)** ("Security Fix Hygiene") for the naming rules that
 keep the vulnerability out of public branch names, commits, and test descriptions.
@@ -1291,12 +1257,6 @@ Adding a new channel requires inviting the bot first; the first run otherwise fa
 
 ## Future Vision
 
-### Redundancy Review
-
-Comment trigger (`/test-workflows`) is a workaround.
-
-Long-term: Main CI should be reliable enough to not need these.
-
 ### Workflow Testability
 
 - Tools like `act` for local testing

.github/actions/ci-filter/__tests__/ci-filter.test.ts

@@ -58,6 +58,15 @@ describe('matchGlob', () => {
 		);
 	});
 
+	it('backend module pattern matches packaged module files', () => {
+		assert.ok(
+			matchGlob(
+				'packages/modules/insights/backend/src/insights.service.ts',
+				'packages/modules/*/backend/**',
+			),
+		);
+	});
+
 	it('* matches single-level only', () => {
 		assert.ok(matchGlob('README.md', '*.md'));
 		assert.ok(!matchGlob('docs/README.md', '*.md'));
@@ -242,7 +251,7 @@ describe('runtime filter', () => {
 		'!**/CHANGELOG.md',
 		'!**/*.test.ts',
 		'!**/*.spec.ts',
-		'!packages/testing/playwright/**',
+		'!packages/quality/testing/playwright/**',
 		'!packages/frontend/@n8n/storybook/**',
 		'!scripts/agent-setup.mjs',
 		'!scripts/backend-module/**',
@@ -265,7 +274,7 @@ describe('runtime filter', () => {
 
 	it('does not trigger on playwright tests', () => {
 		assert.equal(
-			evaluateFilter(['packages/testing/playwright/tests/x.spec.ts'], runtimePatterns),
+			evaluateFilter(['packages/quality/testing/playwright/tests/x.spec.ts'], runtimePatterns),
 			false,
 		);
 	});

.github/docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   postgres:
-    image: ${TEST_IMAGE_POSTGRES:?set it from packages/testing/containers/postgres-versions.json}
+    image: ${TEST_IMAGE_POSTGRES:?set it from packages/quality/environments/containers/postgres-versions.json}
     restart: always
     environment:
       - POSTGRES_DB=n8n

.github/pull_request_template.md

@@ -1,8 +1,15 @@
 ## Summary
 
 <!--
-Describe what the PR does.
+Start with 1 to 3 sentences of background for a new reader. Then describe what the PR does, in prose, not as a list of edits.
-Photos and videos are recommended.
+Add photos or videos for UI changes.
+-->
+
+## Why
+
+<!--
+State the problem or the constraint that makes the current code unfit. Do not repeat the Summary.
+Add a rejected alternative only when a reviewer would ask about it, in one or two sentences.
 -->
 
 ## How to test

.github/scripts/db-test-matrix.mjs

@@ -7,7 +7,7 @@ import { fileURLToPath } from 'node:url';
 
 const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
 
-export const POSTGRES_VERSIONS_PATH = 'packages/testing/containers/postgres-versions.json';
+export const POSTGRES_VERSIONS_PATH = 'packages/quality/environments/containers/postgres-versions.json';
 
 /** The matrix wall is bounded by its slowest leg, so a larger runner buys nothing. */
 const RUNNER = 'blacksmith-4vcpu-ubuntu-2204';

.github/scripts/glob.test.mjs

@@ -30,9 +30,9 @@ describe('matchesGlob', () => {
 	});
 
 	it('matches a prefix pattern', () => {
-		assert.equal(matchesGlob('packages/testing/playwright/a.ts', 'packages/testing/**'), true);
+		assert.equal(matchesGlob('packages/quality/testing/playwright/a.ts', 'packages/quality/**'), true);
-		assert.equal(matchesGlob('packages/testing', 'packages/testing/**'), false);
+		assert.equal(matchesGlob('packages/quality', 'packages/quality/**'), false);
-		assert.equal(matchesGlob('packages/cli/a.ts', 'packages/testing/**'), false);
+		assert.equal(matchesGlob('packages/cli/a.ts', 'packages/quality/**'), false);
 	});
 
 	it('matches a literal path only exactly', () => {
@@ -55,8 +55,8 @@ describe('matchesGlob', () => {
 			'packages/cli/__snapshots__/a.ts.snap',
 			'packages/nodes-base/nodes/Foo/test/fixtures/a.json',
 			'packages/frontend/editor-ui/src/__mocks__/a.ts',
-			'packages/testing/playwright/a.ts',
+			'packages/quality/testing/playwright/a.ts',
-			'packages/testing',
+			'packages/quality',
 			'.github/scripts/glob.test.mjs',
 			'.github/workflows/ci.yml',
 			'a/.hidden/b.spec.js',

.github/scripts/owners/owners.mjs

@@ -1,6 +1,8 @@
 import { readFileSync, statSync } from "node:fs";
 import { join } from "node:path";
 
+import { isTestFile } from "../test-files.mjs";
+
 /**
  * @typedef OwnersEntry
  * @property { string } pattern
@@ -236,7 +238,7 @@ export function assignOwnership(files, entries) {
 
 /**
  * Determine which teams must approve the changeset: a team is required when
- * its `required` entry wins (last-match) for a changed file.
+ * its `required` entry wins (last-match) for a changed non-test file.
  *
  * @param { Set<string> } files
  * @param { OwnersEntry[] } entries
@@ -247,6 +249,8 @@ export function resolveRequiredTeams(files, entries) {
 	const teamToFiles = new Map();
 
 	for (const file of [...files].sort()) {
+		if (isTestFile(file)) continue;
+
 		const entry = findOwningEntry(file, entries);
 		if (!entry?.required) continue;
 

.github/scripts/owners/owners.test.mjs

@@ -293,6 +293,22 @@ describe('resolveRequiredTeams', () => {
 		assert.equal(result.size, 0);
 	});
 
+	it('ignores test files whose winning entry is required', () => {
+		const result = resolveRequiredTeams(
+			new Set([
+				'.github/scripts/check.test.mjs',
+				'.github/workflows/__tests__/check.ts',
+				'.github/workflows/ci.yml',
+			]),
+			[
+				entry({ pattern: '.github/scripts/', team: '@n8n-io/qa-dx', required: true }),
+				entry({ pattern: '.github/workflows/', team: '@n8n-io/qa-dx', required: true }),
+			],
+		);
+
+		assert.deepEqual(result.get('@n8n-io/qa-dx'), ['.github/workflows/ci.yml']);
+	});
+
 	it('a later non-required entry overrides an earlier required one', () => {
 		const result = resolveRequiredTeams(new Set(['db/migrations/1-init.ts']), [
 			entry({ pattern: 'db/', team: '@n8n-io/migrations-review', required: true }),

.github/scripts/owners/required-reviews.mjs

@@ -1,7 +1,7 @@
 /**
  * Required-review enforcement for OWNERS entries marked `required`.
  *
- * Computes the teams whose approval the changeset needs (per OWNERS)
+ * Computes the teams whose approval the non-test changes need (per OWNERS)
  * and reports the verdict as a "Required Reviews" commit status on the PR
  * head SHA. A ruleset that lists this status as a required check blocks the
  * merge until a member of each required team has approved the PR. Merge-queue

.github/scripts/owners/review-recommendations.mjs

@@ -31,6 +31,38 @@ import {
  */
 
 const BOT_MARKER = '<!-- pr-recommendations -->';
+const REQUIRED_FILES_DISPLAY_LIMIT = 5;
+
+/**
+ * @param { string } value
+ * @returns { string }
+ */
+function escapeHtml(value) {
+	return value
+		.replaceAll('&', '&amp;')
+		.replaceAll('<', '&lt;')
+		.replaceAll('>', '&gt;')
+		.replaceAll('|', '&#124;');
+}
+
+/**
+ * @param { string[] } files
+ * @returns { string }
+ */
+function formatRequiredFiles(files) {
+	const displayedFiles = files
+		.slice(0, REQUIRED_FILES_DISPLAY_LIMIT)
+		.map((file) => `<code>${escapeHtml(file)}</code>`);
+	const omittedFileCount = files.length - displayedFiles.length;
+
+	if (omittedFileCount > 0) {
+		displayedFiles.push(
+			`<em>${omittedFileCount} more file${omittedFileCount === 1 ? '' : 's'}</em>`,
+		);
+	}
+
+	return displayedFiles.join('<br>');
+}
 
 function createEmptyLineStats() {
 	return {
@@ -210,9 +242,11 @@ export function buildRequiredReviewsSection(requiredTeamFiles) {
 		'',
 		'Some changed files have a `required` owner in `OWNERS`. A member of each of these teams must approve this PR before it can merge:',
 		'',
-		'| Team | Files |',
+		'| Team | Files owned | Owned files |',
-		'| --- | ---: |',
+		'| --- | ---: | --- |',
-		...[...requiredTeamFiles].map(([team, files]) => `| ${team} | ${files.length} |`),
+		...[...requiredTeamFiles].map(
+			([team, files]) => `| ${team} | ${files.length} | ${formatRequiredFiles(files)} |`,
+		),
 		'',
 		`Request a review from the team${plural ? 's' : ''} — GitHub assigns reviewers according to the team's review settings. The \`Auto-assign reviewers\` label does this for all owning teams.`,
 	].join('\n');

.github/scripts/owners/review-recommendations.test.mjs

@@ -217,7 +217,7 @@ describe('computeLineStats', () => {
 			['*.snap extension', 'src/foo.snap'],
 			['fixtures/ directory', 'src/fixtures/data.json'],
 			['__mocks__/ directory', 'src/__mocks__/axios.ts'],
-			['packages/testing/**', 'packages/testing/playwright/spec.ts'],
+			['packages/quality/**', 'packages/quality/testing/playwright/spec.ts'],
 			['dot-directory *.test.mjs', '.github/scripts/owners/owners.test.mjs'],
 		]) {
 			it(`classifies ${label} as testFiles`, () => {
@@ -322,7 +322,7 @@ describe('buildRequiredReviewsSection', () => {
 		assert.equal(buildRequiredReviewsSection(new Map()), null);
 	});
 
-	it('lists each required team with its file count', () => {
+	it('lists each required team with its file count and owned files', () => {
 		const section = buildRequiredReviewsSection(
 			new Map([
 				['@n8n-io/qa-dx', ['a.yml', 'b.yml']],
@@ -331,8 +331,30 @@ describe('buildRequiredReviewsSection', () => {
 		);
 
 		assert.match(section, /### Required reviews/);
-		assert.match(section, /\| @n8n-io\/qa-dx \| 2 \|/);
+		assert.match(
-		assert.match(section, /\| @n8n-io\/migrations-review \| 1 \|/);
+			section,
+			/\| @n8n-io\/qa-dx \| 2 \| <code>a\.yml<\/code><br><code>b\.yml<\/code> \|/,
+		);
+		assert.match(section, /\| @n8n-io\/migrations-review \| 1 \| <code>m\.ts<\/code> \|/);
+	});
+
+	it('shows no more than five files for each team', () => {
+		const section = buildRequiredReviewsSection(
+			new Map([['@n8n-io/qa-dx', ['a.yml', 'b.yml', 'c.yml', 'd.yml', 'e.yml', 'f.yml', 'g.yml']]]),
+		);
+
+		assert.match(section, /<code>a\.yml<\/code>/);
+		assert.match(section, /<code>e\.yml<\/code>/);
+		assert.doesNotMatch(section, /<code>f\.yml<\/code>/);
+		assert.match(section, /<em>2 more files<\/em>/);
+	});
+
+	it('escapes file paths before adding them to the table', () => {
+		const section = buildRequiredReviewsSection(
+			new Map([['@n8n-io/qa-dx', ['packages/a|b/<script>.ts']]]),
+		);
+
+		assert.match(section, /<code>packages\/a&#124;b\/&lt;script&gt;\.ts<\/code>/);
 	});
 
 	it('prompts to request review from the team, plural when several teams are required', () => {
@@ -447,7 +469,7 @@ describe('run', () => {
 
 		const body = postOrUpdateComment.mock.calls[0].arguments[1];
 		assert.match(body, /### Required reviews/);
-		assert.match(body, /\| @n8n-io\/qa-dx \| 1 \|/);
+		assert.match(body, /\| @n8n-io\/qa-dx \| 1 \| <code>\.github\/workflows\/ci\.yml<\/code> \|/);
 		assert.match(body, /Request a review from the team —/);
 	});
 });

.github/scripts/pnpm-lock.yaml

@@ -747,8 +747,8 @@ packages:
   fast-deep-equal@3.1.3:
     resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==}
 
-  fast-uri@3.1.5:
+  fast-uri@3.1.8:
-    resolution: {integrity: sha512-gHwA1O9LDIcKunMKhObS/HimwtehO1nPUECKAu5TpKgaO19fcWEl4bliWe1jWxVFvIXztJjjQ4L8XQ1EU9f7Jw==}
+    resolution: {integrity: sha512-GZMtZUTNRpOVIECoXwLNZS5xUGE+mVNbTB8h/7Rwh2TFWcBQiPzTgyZi05BF9UMZKkLJv8XBRJTlU7zg8+ZfMg==}
 
   fd-package-json@2.0.0:
     resolution: {integrity: sha512-jKmm9YtsNXN789RS/0mSzOC1NUq9mkVd65vbSSVsKdjGvYXBuE4oWe2QOEoFeRmJg+lPuZxpmrfFclNhoRMneQ==}
@@ -1694,7 +1694,7 @@ snapshots:
   ajv@8.20.0:
     dependencies:
       fast-deep-equal: 3.1.3
-      fast-uri: 3.1.5
+      fast-uri: 3.1.8
       json-schema-traverse: 1.0.0
       require-from-string: 2.0.2
 
@@ -1994,7 +1994,7 @@ snapshots:
 
   fast-deep-equal@3.1.3: {}
 
-  fast-uri@3.1.5: {}
+  fast-uri@3.1.8: {}
 
   fd-package-json@2.0.0:
     dependencies:

.github/scripts/quality/check-pr-size.mjs

@@ -14,32 +14,13 @@
 
 import { initGithub, getEventFromGithubEventPath } from '../github-helpers.mjs';
 import { matchesGlob } from '../glob.mjs';
+import { isTestFile, TEST_PATTERNS } from '../test-files.mjs';
+
+export { isTestFile, TEST_PATTERNS };
 
 export const SIZE_LIMIT = 1000;
 export const OVERRIDE_COMMAND = '/size-limit-override';
 
-export const TEST_PATTERNS = [
-	// Test files (by extension)
-	'**/*.test.ts',
-	'**/*.test.js',
-	'**/*.test.mjs',
-	'**/*.spec.ts',
-	'**/*.spec.js',
-	'**/*.spec.mjs',
-	// Test directories
-	'**/test/**',
-	'**/tests/**',
-	'**/__tests__/**',
-	// Snapshots
-	'**/__snapshots__/**',
-	'**/*.snap',
-	// Fixtures and mocks
-	'**/fixtures/**',
-	'**/__mocks__/**',
-	// Dedicated testing package
-	'packages/testing/**',
-];
-
 export const MISC_PATTERNS = [
 	// Lock file (can produce massive diffs on dependency changes)
 	'pnpm-lock.yaml',
@@ -56,7 +37,7 @@ export const EXCLUDE_PATTERNS = [...TEST_PATTERNS, ...MISC_PATTERNS];
  * @returns { 'testFiles' | 'misc' | 'sourceCode' }
  */
 export function categorizeFile(filename) {
-	if (TEST_PATTERNS.some((pattern) => matchesGlob(filename, pattern))) return 'testFiles';
+	if (isTestFile(filename)) return 'testFiles';
 	if (MISC_PATTERNS.some((pattern) => matchesGlob(filename, pattern))) return 'misc';
 	return 'sourceCode';
 }

.github/scripts/quality/check-pr-size.test.mjs

@@ -203,14 +203,21 @@ describe('countFilteredAdditions', () => {
 		assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
 	});
 
-	it('applies EXCLUDE_PATTERNS to packages/testing and pnpm-lock.yaml', () => {
+	it('excludes former testing workspaces but counts benchmark CLI source', () => {
 		const files = [
 			{ filename: 'packages/cli/src/service.ts', additions: 50 },
-			{ filename: 'packages/testing/playwright/tests/workflow.spec.ts', additions: 100 },
+			{ filename: 'packages/quality/testing/playwright/tests/workflow.spec.ts', additions: 100 },
-			{ filename: 'packages/testing/playwright/pages/CanvasPage.ts', additions: 100 },
+			{ filename: 'packages/quality/testing/playwright/pages/CanvasPage.ts', additions: 100 },
+			{ filename: 'packages/quality/policy/code-health/src/cli.ts', additions: 100 },
+			{ filename: 'packages/quality/environments/containers/service-stack.ts', additions: 100 },
+			{
+				filename: 'packages/quality/efficiency/microbenchmarks/benchmarks/bench-options.ts',
+				additions: 100,
+			},
+			{ filename: 'packages/quality/efficiency/scale/benchmark/src/commands/list.ts', additions: 100 },
 			{ filename: 'pnpm-lock.yaml', additions: 500 },
 		];
-		assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
+		assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 150);
 	});
 
 	it('applies EXCLUDE_PATTERNS to markdown files', () => {

.github/scripts/quality/cubic-config.schema.json

@@ -5,7 +5,9 @@
 	"description": "JSON schema for cubic.yaml (and the legacy .cubic.yaml) so editors can validate and autocomplete repository settings.",
 	"type": "object",
 	"additionalProperties": true,
-	"required": ["version"],
+	"required": [
+		"version"
+	],
 	"properties": {
 		"version": {
 			"const": 1,
@@ -16,13 +18,26 @@
 			"description": "AI review behavior and optional YAML-defined custom agents.",
 			"additionalProperties": false,
 			"properties": {
+				"senior_reviewers": {
+					"type": "array",
+					"maxItems": 5,
+					"items": {
+						"type": "string",
+						"pattern": "^[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,37}[a-zA-Z0-9])?$"
+					},
+					"description": "GitHub usernames to learn review patterns from. Overrides dashboard selections; an empty list disables senior reviewer learning."
+				},
 				"enabled": {
 					"type": "boolean",
 					"description": "Master toggle for AI reviews on this repository."
 				},
 				"sensitivity": {
 					"type": "string",
-					"enum": ["low", "medium", "high"],
+					"enum": [
+						"low",
+						"medium",
+						"high"
+					],
 					"description": "Controls how strictly Cubic flags issues."
 				},
 				"incremental_commits": {
@@ -63,12 +78,21 @@
 				},
 				"auto_approve_behavior": {
 					"type": "string",
-					"enum": ["disabled", "shadow", "live"],
+					"enum": [
+						"disabled",
+						"shadow",
+						"live"
+					],
 					"description": "Controls whether auto-approval is disabled, simulated in shadow mode, or submitted live to GitHub."
 				},
 				"auto_approve": {
 					"type": "string",
-					"enum": ["disabled", "always", "low_risk_only", "custom"],
+					"enum": [
+						"disabled",
+						"always",
+						"low_risk_only",
+						"custom"
+					],
 					"description": "Auto-approve mode: disabled, always approve clean runs, use low-risk heuristics, or use a custom prompt."
 				},
 				"auto_approve_custom_prompt": {
@@ -80,12 +104,20 @@
 				},
 				"ultrareview": {
 					"type": "string",
-					"enum": ["disabled", "manual", "automatic"],
+					"enum": [
+						"disabled",
+						"manual",
+						"automatic"
+					],
 					"description": "Master switch for ultrareviews. disabled blocks all ultrareviews (manual and automatic), manual allows only manual triggers (the default), automatic also enables the automatic triggers."
 				},
 				"auto_ultrareview": {
 					"type": "string",
-					"enum": ["disabled", "high_risk_only", "custom"],
+					"enum": [
+						"disabled",
+						"high_risk_only",
+						"custom"
+					],
 					"description": "Auto-trigger ultrareview mode: disabled, let cubic decide per PR using built-in high-risk criteria, or use a custom prompt."
 				},
 				"auto_ultrareview_custom_prompt": {
@@ -159,7 +191,10 @@
 				},
 				"coding_agent_provider": {
 					"type": "string",
-					"enum": ["cubic", "cursor_cloud_agent"],
+					"enum": [
+						"cubic",
+						"cursor_cloud_agent"
+					],
 					"description": "Which coding agent applies automatic fixes. 'cubic' uses cubic's built-in agent; 'cursor_cloud_agent' dispatches a Cursor cloud agent (requires Cursor configuration)."
 				}
 			}
@@ -222,8 +257,20 @@
 				},
 				"exclude_external_contributors": {
 					"type": "boolean",
-					"default": false,
+					"default": true,
-					"description": "If true, public-repository PRs from external contributors are reviewed but never auto-approved."
+					"description": "If true, public-repository PRs from external contributors, or from authors cubic cannot verify, are reviewed but never auto-approved."
+				},
+				"allow_open_issues": {
+					"type": "string",
+					"enum": [
+						"none",
+						"p3",
+						"p2_and_p3",
+						"p1_to_p3",
+						"all"
+					],
+					"default": "none",
+					"description": "Open cubic findings that don't block auto-approval, by priority. none blocks on any open finding; p3, p2_and_p3, and p1_to_p3 allow that range; all allows every open finding. A finding without a recorded priority blocks unless this is all. Every other auto-approval rule still applies."
 				},
 				"exclude_authors": {
 					"$ref": "#/$defs/globList",
@@ -271,13 +318,19 @@
 			"type": "object",
 			"description": "Definition of a YAML-managed custom agent.",
 			"additionalProperties": false,
-			"required": ["name"],
+			"required": [
+				"name"
+			],
 			"anyOf": [
 				{
-					"required": ["description"]
+					"required": [
+						"description"
+					]
 				},
 				{
-					"required": ["file_paths"]
+					"required": [
+						"file_paths"
+					]
 				}
 			],
 			"properties": {

.github/scripts/split-jscpd-report.mjs

@@ -0,0 +1,68 @@
+#!/usr/bin/env node
+// Splits a jscpd JSON report's `duplicates` array into N bucket files so a
+// caller can download and sample one small file instead of the whole report
+// (which can run into tens of MB on a large monorepo). Round-robin, not
+// contiguous chunks, so a run of similarly-sized clusters in scan order
+// doesn't pile up in one bucket.
+
+import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
+import { join } from 'node:path';
+
+export function splitDuplicates(duplicates, bucketCount) {
+	const buckets = Array.from({ length: bucketCount }, () => []);
+	duplicates.forEach((dup, i) => buckets[i % bucketCount].push(dup));
+	return buckets;
+}
+
+export function writeBuckets(buckets, outputDir, deps = {}) {
+	const mkdirImpl = deps.mkdir ?? mkdirSync;
+	const writeImpl = deps.write ?? writeFileSync;
+	mkdirImpl(outputDir, { recursive: true });
+	return buckets.map((duplicates, i) => {
+		const path = join(outputDir, `bucket-${i}.json`);
+		writeImpl(path, JSON.stringify({ duplicates }));
+		return path;
+	});
+}
+
+export function parseBucketCount(raw) {
+	const count = Number(raw ?? 20);
+	if (!Number.isInteger(count) || count <= 0) {
+		throw new Error(`BUCKET_COUNT must be a positive integer, got '${raw}'`);
+	}
+	return count;
+}
+
+export function loadDuplicates(reportPath, deps = {}) {
+	const readImpl = deps.read ?? readFileSync;
+	const report = JSON.parse(readImpl(reportPath, 'utf8'));
+	if (!Array.isArray(report.duplicates)) {
+		throw new Error('report.duplicates must be an array');
+	}
+	return report.duplicates;
+}
+
+// Exits non-zero on any failure. The workflow step already runs with
+// continue-on-error, so this can't fail the job — it only stops a broken
+// split (zero buckets written) from reporting success.
+function main() {
+	const reportPath = process.env.REPORT_PATH;
+	const outputDir = process.env.OUTPUT_DIR;
+	if (!reportPath || !outputDir) {
+		process.stderr.write('split-jscpd-report: REPORT_PATH and OUTPUT_DIR are required\n');
+		process.exit(1);
+	}
+	try {
+		const bucketCount = parseBucketCount(process.env.BUCKET_COUNT);
+		const duplicates = loadDuplicates(reportPath);
+		const buckets = splitDuplicates(duplicates, bucketCount);
+		writeBuckets(buckets, outputDir);
+	} catch (err) {
+		process.stderr.write(`split-jscpd-report: ${err.message}\n`);
+		process.exit(1);
+	}
+}
+
+if (import.meta.url === `file://${process.argv[1]}`) {
+	main();
+}

.github/scripts/split-jscpd-report.test.mjs

@@ -0,0 +1,111 @@
+/**
+ * Run these tests by running
+ *
+ * node --test .github/scripts/split-jscpd-report.test.mjs
+ * */
+
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { splitDuplicates, writeBuckets, parseBucketCount, loadDuplicates } from './split-jscpd-report.mjs';
+
+describe('splitDuplicates', () => {
+	it('distributes entries round-robin across buckets', () => {
+		const duplicates = [1, 2, 3, 4, 5, 6];
+		const buckets = splitDuplicates(duplicates, 3);
+		assert.deepEqual(buckets, [
+			[1, 4],
+			[2, 5],
+			[3, 6],
+		]);
+	});
+
+	it('leaves later buckets empty when there are fewer entries than buckets', () => {
+		const buckets = splitDuplicates([1, 2], 5);
+		assert.equal(buckets.length, 5);
+		assert.deepEqual(buckets[0], [1]);
+		assert.deepEqual(buckets[1], [2]);
+		assert.deepEqual(buckets[2], []);
+	});
+
+	it('returns bucketCount empty arrays for an empty report', () => {
+		const buckets = splitDuplicates([], 4);
+		assert.deepEqual(buckets, [[], [], [], []]);
+	});
+});
+
+describe('writeBuckets', () => {
+	it('writes one JSON file per bucket, named by index', () => {
+		const written = [];
+		const paths = writeBuckets([[{ a: 1 }], [{ b: 2 }]], '/tmp/buckets', {
+			mkdir: () => {},
+			write: (path, contents) => written.push({ path, contents }),
+		});
+		assert.deepEqual(paths, ['/tmp/buckets/bucket-0.json', '/tmp/buckets/bucket-1.json']);
+		assert.deepEqual(JSON.parse(written[0].contents), { duplicates: [{ a: 1 }] });
+		assert.deepEqual(JSON.parse(written[1].contents), { duplicates: [{ b: 2 }] });
+	});
+
+	it('creates the output directory before writing', () => {
+		let mkdirCalledWith = null;
+		writeBuckets([[]], '/tmp/buckets', {
+			mkdir: (dir, opts) => {
+				mkdirCalledWith = { dir, opts };
+			},
+			write: () => {},
+		});
+		assert.deepEqual(mkdirCalledWith, { dir: '/tmp/buckets', opts: { recursive: true } });
+	});
+});
+
+describe('parseBucketCount', () => {
+	it('parses a numeric string', () => {
+		assert.equal(parseBucketCount('20'), 20);
+	});
+
+	it('defaults to 20 when unset', () => {
+		assert.equal(parseBucketCount(undefined), 20);
+	});
+
+	it('rejects zero', () => {
+		assert.throws(() => parseBucketCount('0'), /positive integer/);
+	});
+
+	it('rejects a negative count', () => {
+		assert.throws(() => parseBucketCount('-1'), /positive integer/);
+	});
+
+	it('rejects a non-integer', () => {
+		assert.throws(() => parseBucketCount('1.5'), /positive integer/);
+	});
+
+	it('rejects a non-numeric value', () => {
+		assert.throws(() => parseBucketCount('abc'), /positive integer/);
+	});
+});
+
+describe('loadDuplicates', () => {
+	it('returns the duplicates array from a valid report', () => {
+		const duplicates = loadDuplicates('/tmp/report.json', {
+			read: () => JSON.stringify({ duplicates: [{ a: 1 }] }),
+		});
+		assert.deepEqual(duplicates, [{ a: 1 }]);
+	});
+
+	it('rejects a report missing the duplicates array', () => {
+		assert.throws(
+			() => loadDuplicates('/tmp/report.json', { read: () => JSON.stringify({}) }),
+			/duplicates must be an array/,
+		);
+	});
+
+	it('rejects a report whose duplicates field is not an array', () => {
+		assert.throws(
+			() => loadDuplicates('/tmp/report.json', { read: () => JSON.stringify({ duplicates: 'x' }) }),
+			/duplicates must be an array/,
+		);
+	});
+
+	it('propagates a JSON parse failure', () => {
+		assert.throws(() => loadDuplicates('/tmp/report.json', { read: () => 'not json' }), SyntaxError);
+	});
+});

.github/scripts/sync-bundle-branch.mjs

@@ -12,7 +12,7 @@
  * why only a rewrite breaks this.
  *
  * The cost of merging is a merge commit per run, plus already-published fixes staying in the
- * branch's log (the rebase used to drop them as empty). Neither reaches anything downstream:
+ * branch's log. Neither reaches anything downstream:
  * a bundle is published as one squashed, deliberately obfuscated commit, and the squash is
  * taken from the tree, not the history. For a list of what a bundle actually carries, read
  * the fix PRs merged into the branch since the last cut, not `base..bundle`.
@@ -26,17 +26,13 @@
  * of being re-litigated on every later run. Contrast `sync-master-to-3x.mjs`, which
  * auto-resolves mechanical files and opens a conflict PR.
  *
- * PUBLISHED HISTORY ONLY: the base is synced in only once the public repo already has it. A cut
+ * PUBLIC HISTORY ONLY: the base is fetched directly from the public repo. Private master can
- * squash-merges the bundle branch into private master as `chore: Bundle …`, and the hourly
+ * temporarily contain a bundle cut that public master does not have yet. Using the private base
- * public→private mirror then DISCARDS that commit in favour of the public cherry-pick of the same
+ * would strand that commit on a recreated bundle branch after the private mirror replaces it.
- * changes. Building on it would strand the discarded commit on a branch that fix PRs are cut
- * from, and the replacement gets merged in on top of it later. An unpublished cut is a SKIP (the
- * mirror clears it within the hour, whatever the cut is, so waiting cannot deadlock); a base
- * ahead of public for any other reason is a FAILURE, because the mirror will not clear it.
  *
  * Runs from a full checkout (fetch-depth 0) of any branch. Assumes credentials are NOT
- * persisted by checkout — every private fetch and the push go through an explicit token URL,
+ * persisted by checkout. The bundle fetch and push use an explicit token URL. The public base
- * which matters because the repository is private. The public fetch is anonymous.
+ * fetch is anonymous.
  *
  * Env: BUNDLE_BRANCH (e.g. bundle/2.x), BASE_BRANCH (e.g. master),
  *      GH_TOKEN (installation token with contents:write),
@@ -60,10 +56,6 @@ const BOT_EMAIL = 'n8n-assistant[bot]@users.noreply.github.com';
 // installation token this script holds is scoped to the private repo.
 const PUBLIC_REMOTE = 'https://github.com/n8n-io/n8n.git';
 
-// What a cut lands on private base before it is published. Anything else ahead of public means
-// the mirror is stuck, not that a cut is in flight.
-const BUNDLE_SUBJECT = /^chore: Bundle/;
-
 function required(env, name) {
 	const value = env[name];
 	if (!value) throw new Error(`${name} env var is required`);
@@ -76,49 +68,8 @@ export function annotation(title, message) {
 	return `::error title=${title}::${message.replace(/\n/g, '%0A')}`;
 }
 
-/**
- * Whether the base tip is still private-only, and what to do about it.
- *
- * `null` means the public repo already has it, so the sync may proceed. The predicate is
- * containment in public rather than "is not a bundle commit", so a direct push that bypassed
- * ci-restrict-private-merges.yml is caught too; the subjects only decide how to react.
- */
-function unconvergedBase({ git, log, bundle, base, baseSha, publicSha }) {
-	if (isAncestor(git, baseSha, publicSha)) return null;
-
-	const ahead = git(['log', '--format=%h %s', `${publicSha}..${baseSha}`])
-		.split('\n')
-		.filter(Boolean);
-	const subjects = ahead.map((line) => line.slice(line.indexOf(' ') + 1));
-
-	// All-or-nothing: one unexplained commit means the mirror will not clear the base on its own.
-	// An empty range here means unrelated histories, which `every` treats as a skip — the safe
-	// direction, since forcing past an ancestry check we could not evaluate is never right.
-	if (subjects.every((subject) => BUNDLE_SUBJECT.test(subject))) {
-		log(
-			`${base} ${baseSha} has not reached public ${base} ${publicSha} yet; leaving ${bundle} ` +
-				'untouched. The hourly public → private sync clears this.',
-		);
-		return { status: 'unconverged' };
-	}
-
-	// Subjects go to the run log, which is private. The annotation stays a count: it is the
-	// headline of a failed run, and a subject hints at the fix.
-	log(ahead.join('\n'));
-	log(
-		annotation(
-			`${base} is ahead of public`,
-			`${base} carries ${ahead.length} commit(s) that public does not have and that are not ` +
-				`bundle cuts, so the mirror cannot clear them. Fix the mirror, then re-run this workflow.`,
-		),
-	);
-	throw new Error(`${base} is ahead of public by commits that are not bundle cuts.`);
-}
-
-// Takes `baseSha` explicitly. This path never fetches the bundle branch, so FETCH_HEAD still
-// holds the PUBLIC tip here, not the base — never swap the argument for FETCH_HEAD.
 function createBundleBranch({ git, log, bundle, base, baseSha, remote }) {
-	log(`${bundle} does not exist yet; creating it at ${base} ${baseSha}.`);
+	log(`${bundle} does not exist yet; creating it at public ${base} ${baseSha}.`);
 	git(['checkout', '--force', '-B', bundle, baseSha]);
 
 	const push = attempt(git, ['push', remote, `HEAD:refs/heads/${bundle}`]);
@@ -132,28 +83,15 @@ function createBundleBranch({ git, log, bundle, base, baseSha, remote }) {
 
 /**
  * One fetch-merge-push cycle. `status: 'rejected'` means the branch moved under us — the
- * caller re-runs from a fresh fetch rather than forcing anything. `status: 'unconverged'` means
+ * caller re-runs from a fresh fetch rather than forcing anything.
- * the base is not published yet and nothing was touched.
  */
 function mergeBaseIntoBundle({ git, log, bundle, base, remote }) {
-	// Pin every side to the SHA fetched for it — fetching by URL never updates the origin/*
+	// Pin the base to the public SHA. Private master can temporarily contain an unpublished cut.
-	// tracking refs, so FETCH_HEAD is the only handle, and each of the three fetches below
-	// overwrites the last. Read FETCH_HEAD on the line after its own fetch; never separate a
-	// fetch from the read that consumes it.
-	git(['fetch', remote, base]);
-	const baseSha = git(['rev-parse', 'FETCH_HEAD']);
-
-	// The base branch is named the same on both sides (master, 1.x). If that ever stops being
-	// true, map it here rather than at the call site.
 	const publicFetch = attempt(git, ['fetch', PUBLIC_REMOTE, base]);
 	if (!publicFetch.ok) {
-		// A guard that cannot be evaluated must stop the run, not be assumed satisfied.
 		throw new Error(`Could not fetch ${base} from the public repo:\n${publicFetch.out}`);
 	}
-	const publicSha = git(['rev-parse', 'FETCH_HEAD']);
+	const baseSha = git(['rev-parse', 'FETCH_HEAD']);
-
-	const unconverged = unconvergedBase({ git, log, bundle, base, baseSha, publicSha });
-	if (unconverged) return unconverged;
 
 	const listed = attempt(git, ['ls-remote', '--heads', remote, `refs/heads/${bundle}`]);
 	if (!listed.ok) {
@@ -179,7 +117,9 @@ function mergeBaseIntoBundle({ git, log, bundle, base, remote }) {
 		'--format=%h %s',
 		`${baseSha}..${preHead}`,
 	]);
-	log(`Merging ${base} ${baseSha} into ${bundle}. Commits on ${bundle}:\n${pending || '(none)'}`);
+	log(
+		`Merging public ${base} ${baseSha} into ${bundle}. Commits on ${bundle}:\n${pending || '(none)'}`,
+	);
 
 	// The content the merge must produce — and whether the two sides conflict at all. Computed
 	// without touching the working tree, so a conflict costs the branch nothing.
@@ -211,7 +151,7 @@ function mergeBaseIntoBundle({ git, log, bundle, base, remote }) {
 		if (push.out) log(push.out);
 		return { status: 'rejected' };
 	}
-	log(`Merged ${base} into ${bundle} — nothing rewritten.`);
+	log(`Merged public ${base} into ${bundle}; nothing was rewritten.`);
 	return { status: 'merged' };
 }
 

.github/scripts/sync-bundle-branch.test.mjs

@@ -28,7 +28,6 @@ const fail =
 	};
 
 const PRE_HEAD = 'PREHEAD';
-const BASE = 'BASESHA';
 const PUBLIC = 'PUBLICSHA';
 const MERGE_TREE = 'MERGETREEOID';
 const REMOTE = 'https://x-access-token:tok@github.com/n8n-io/n8n-private.git';
@@ -52,37 +51,27 @@ const conflictedMergeTree = (...paths) =>
 	fail(`${MERGE_TREE}\n${paths.join('\n')}\n\nCONFLICT (content): Merge conflict in ${paths[0]}`);
 
 // FETCH_HEAD is one slot holding whatever the last fetch brought down, so the stub has to answer
-// by looking back at that fetch. Resolving both fetches to the same SHA would let the
+// by looking back at that fetch.
-// convergence guard pass by accident and assert nothing.
 const fetchHead = (_args, calls) => {
 	const [, url, ref] = [...calls].reverse().find((a) => a[0] === 'fetch') ?? [];
 	if (url === PUBLIC_REMOTE) return PUBLIC;
 	if (ref?.startsWith('bundle/')) return PRE_HEAD;
-	return BASE;
+	return '';
 };
 
-// Routes shared by every path: base fetched and already published, the bundle branch hasn't
+// Routes shared by every path: the bundle branch has not absorbed the public base yet, the
-// absorbed it yet, the merge tree is computable, and the merge lands on exactly that tree. git
+// merge tree is computable, and the merge lands on exactly that tree. git grep exits non-zero
-// grep exits non-zero => no markers.
+// => no markers.
 const baseGitRoutes = [
 	[(a) => a[0] === 'ls-remote', `${PRE_HEAD}\trefs/heads/bundle/2.x`],
 	[(a) => a[0] === 'rev-parse' && a[1] === 'FETCH_HEAD', fetchHead],
 	[(a) => a[0] === 'rev-parse' && a[1] === 'HEAD', PRE_HEAD],
-	// Split by descendant: the base has reached public, but the bundle branch has not absorbed
-	// it. One `merge-base` matcher would answer both and decide convergence by accident.
-	[(a) => a[0] === 'merge-base' && a[3] === PUBLIC, ''],
 	[(a) => a[0] === 'merge-base' && a[3] === PRE_HEAD, fail()],
 	[(a) => a[0] === 'merge-tree', MERGE_TREE],
 	[(a) => a[0] === 'rev-parse' && a[1] === 'HEAD^{tree}', MERGE_TREE],
 	[(a) => a[0] === 'grep', fail()],
 ];
 
-// A base that is ahead of public by the commits given, so the guard trips.
-const unpublishedRoutes = (...lines) => [
-	[(a) => a[0] === 'merge-base' && a[3] === PUBLIC, fail()],
-	[(a) => a[0] === 'log' && a[1] === '--format=%h %s', lines.join('\n')],
-];
-
 const silent = () => {};
 
 test('a missing branch env var fails before any git command runs', () => {
@@ -107,14 +96,14 @@ test('a clean sync merges the base in and pushes without forcing', () => {
 		'--no-edit',
 		'-m',
 		'Merge master into bundle/2.x',
-		BASE,
+		PUBLIC,
 	]);
 	// No lease and no --force: the branch is append-only, so a non-fast-forward must be
 	// refused by git rather than resolved by overwriting whatever landed.
 	assert.deepEqual(git.calls.find(isPush), ['push', REMOTE, 'HEAD:refs/heads/bundle/2.x']);
 });
 
-test('the private fetches are authenticated and the public one is not', () => {
+test('the bundle fetch is authenticated and the public base fetch is not', () => {
 	const git = makeStub(baseGitRoutes);
 	syncBundleBranch({ git, env, log: silent });
 
@@ -122,7 +111,6 @@ test('the private fetches are authenticated and the public one is not', () => {
 	assert.deepEqual(
 		git.calls.filter((a) => a[0] === 'fetch'),
 		[
-			['fetch', REMOTE, 'master'],
 			['fetch', PUBLIC_REMOTE, 'master'],
 			['fetch', REMOTE, 'bundle/2.x'],
 		],
@@ -145,8 +133,6 @@ test('the public tip is read straight after its own fetch', () => {
 	assert.deepEqual(
 		git.calls.filter((a) => a[0] === 'fetch' || (a[0] === 'rev-parse' && a[1] === 'FETCH_HEAD')),
 		[
-			['fetch', REMOTE, 'master'],
-			['rev-parse', 'FETCH_HEAD'],
 			['fetch', PUBLIC_REMOTE, 'master'],
 			['rev-parse', 'FETCH_HEAD'],
 			['fetch', REMOTE, 'bundle/2.x'],
@@ -156,8 +142,6 @@ test('the public tip is read straight after its own fetch', () => {
 
 test('a bundle branch that already contains its base is left alone', () => {
 	const git = makeStub([
-		// Only the bundle-branch ancestry is overridden; the convergence probe keeps its own
-		// answer from baseGitRoutes, or this would pass without the guard being exercised.
 		[(a) => a[0] === 'merge-base' && a[3] === PRE_HEAD, ''], // --is-ancestor succeeds
 		...baseGitRoutes,
 	]);
@@ -226,9 +210,8 @@ test('a fix landing mid-run is retried from a fresh fetch, never forced', () =>
 
 	assert.deepEqual(result, { status: 'merged' });
 	assert.equal(pushes, 2);
-	// The retry re-fetches rather than reusing the tip it already merged onto — including the
+	// The retry re-fetches both tips instead of reusing the merge that lost the push race.
-	// public tip, so the convergence guard is re-evaluated instead of carried over.
+	assert.equal(git.calls.filter((a) => a[0] === 'fetch').length, 4);
-	assert.equal(git.calls.filter((a) => a[0] === 'fetch').length, 6);
 	assert.equal(git.calls.filter((a) => a[1] === PUBLIC_REMOTE).length, 2);
 	assert.equal(
 		git.calls.every((a) => !a.some((arg) => String(arg).includes('force-with-lease'))),
@@ -244,22 +227,19 @@ test('a branch that keeps moving fails instead of forcing', () => {
 	});
 });
 
-test('a bundle branch that does not exist yet is created at its base', () => {
+test('a bundle branch that does not exist yet is created at the public base', () => {
 	const git = makeStub([[(a) => a[0] === 'ls-remote', ''], ...baseGitRoutes]);
 	const result = syncBundleBranch({ git, env, log: silent });
 
 	assert.deepEqual(result, { status: 'created' });
 	assert.deepEqual(
 		git.calls.find((a) => a[0] === 'checkout'),
-		['checkout', '--force', '-B', 'bundle/2.x', BASE],
+		['checkout', '--force', '-B', 'bundle/2.x', PUBLIC],
 	);
 	assert.equal(git.calls.some(isMerge), false);
 	assert.deepEqual(
 		git.calls.filter((a) => a[0] === 'fetch'),
-		[
+		[['fetch', PUBLIC_REMOTE, 'master']],
-			['fetch', REMOTE, 'master'],
-			['fetch', PUBLIC_REMOTE, 'master'],
-		],
 	);
 	assert.deepEqual(git.calls.find(isPush), ['push', REMOTE, 'HEAD:refs/heads/bundle/2.x']);
 });
@@ -301,97 +281,6 @@ test('annotation keeps a multi-line message on one line', () => {
 	assert.equal(annotation('t', 'a\nb'), '::error title=t::a%0Ab');
 });
 
-test('a base that has not reached the public repo is left alone', () => {
-	const git = makeStub([
-		...unpublishedRoutes('abc1234 chore: Bundle 2.x (#123)'),
-		...baseGitRoutes,
-	]);
-	const logs = [];
-
-	const result = syncBundleBranch({ git, env, log: (m) => logs.push(m) });
-
-	assert.deepEqual(result, { status: 'unconverged' });
-	// Nothing about the bundle branch is even asked: no existence probe, no fetch, no checkout.
-	assert.equal(
-		git.calls.some((a) => a[0] === 'ls-remote'),
-		false,
-	);
-	assert.equal(
-		git.calls.some((a) => a[0] === 'checkout'),
-		false,
-	);
-	assert.equal(git.calls.some(isMerge), false);
-	assert.equal(git.calls.some(isPush), false);
-	assert.equal(git.calls.filter((a) => a[0] === 'fetch').length, 2);
-	assert.match(logs.join('\n'), /has not reached public master/);
-});
-
-test('a missing bundle branch is not created from an unpublished base', () => {
-	const git = makeStub([
-		[(a) => a[0] === 'ls-remote', ''],
-		...unpublishedRoutes('abc1234 chore: Bundle/1.x'),
-		...baseGitRoutes,
-	]);
-
-	// One guard covers both paths: the branch must not be conjured onto a doomed commit either.
-	assert.deepEqual(syncBundleBranch({ git, env, log: silent }), { status: 'unconverged' });
-	assert.equal(
-		git.calls.some((a) => a[0] === 'checkout'),
-		false,
-	);
-	assert.equal(git.calls.some(isPush), false);
-});
-
-test('a base ahead of public by a non-bundle commit fails loudly', () => {
-	const git = makeStub([
-		...unpublishedRoutes('abc1234 fix(core): Something a human pushed directly'),
-		...baseGitRoutes,
-	]);
-	const logs = [];
-
-	assert.throws(() => syncBundleBranch({ git, env, log: (m) => logs.push(m) }), {
-		message: /master is ahead of public by commits that are not bundle cuts/,
-	});
-	assert.equal(git.calls.some(isPush), false);
-
-	// The annotation is the headline of a failed run: a count, never a subject.
-	const annotations = logs.filter((m) => m.startsWith('::error'));
-	assert.match(annotations.join('\n'), /carries 1 commit\(s\)/);
-	assert.doesNotMatch(annotations.join('\n'), /Something a human pushed directly/);
-});
-
-test('one stray commit among bundle cuts still fails', () => {
-	const git = makeStub([
-		...unpublishedRoutes('abc1234 chore: Bundle 2.x (#123)', 'def5678 fix(core): Stray'),
-		...baseGitRoutes,
-	]);
-
-	// All-or-nothing: a single unexplained commit means the mirror cannot clear the base.
-	assert.throws(() => syncBundleBranch({ git, env, log: silent }), {
-		message: /not bundle cuts/,
-	});
-});
-
-test('an unrelated history skips rather than forcing anything', () => {
-	const git = makeStub([...unpublishedRoutes(), ...baseGitRoutes]);
-
-	// Ancestry fails but the range is empty, so there is nothing to classify. Skipping is the
-	// only safe default here.
-	assert.deepEqual(syncBundleBranch({ git, env, log: silent }), { status: 'unconverged' });
-	assert.equal(git.calls.some(isPush), false);
-});
-
-test('an unconverged base is not retried as a push race', () => {
-	const git = makeStub([...unpublishedRoutes('abc1234 chore: Bundle 2.x'), ...baseGitRoutes]);
-	const logs = [];
-
-	syncBundleBranch({ git, env, log: (m) => logs.push(m) });
-
-	// Only 'rejected' earns a second attempt; the mirror runs hourly, not within this run.
-	assert.equal(git.calls.filter((a) => a[0] === 'fetch').length, 2);
-	assert.doesNotMatch(logs.join('\n'), /retrying/);
-});
-
 test('a public fetch that fails stops the run instead of guessing', () => {
 	const git = makeStub([
 		[(a) => a[0] === 'fetch' && a[1] === PUBLIC_REMOTE, fail('could not read from remote')],
@@ -423,7 +312,6 @@ test('the 1.x bundle branch is checked against public 1.x', () => {
 	assert.deepEqual(
 		git.calls.filter((a) => a[0] === 'fetch'),
 		[
-			['fetch', REMOTE, '1.x'],
 			['fetch', PUBLIC_REMOTE, '1.x'],
 			['fetch', REMOTE, 'bundle/1.x'],
 		],

.github/scripts/test-files.mjs

@@ -0,0 +1,34 @@
+import { matchesGlob } from './glob.mjs';
+
+export const TEST_PATTERNS = [
+	// Test files (by extension)
+	'**/*.test.ts',
+	'**/*.test.js',
+	'**/*.test.mjs',
+	'**/*.spec.ts',
+	'**/*.spec.js',
+	'**/*.spec.mjs',
+	// Test directories
+	'**/test/**',
+	'**/tests/**',
+	'**/__tests__/**',
+	// Snapshots
+	'**/__snapshots__/**',
+	'**/*.snap',
+	// Fixtures and mocks
+	'**/fixtures/**',
+	'**/__mocks__/**',
+	// Former packages/testing workspaces
+	'packages/quality/policy/**',
+	'packages/quality/testing/**',
+	'packages/quality/environments/**',
+	'packages/quality/efficiency/microbenchmarks/**',
+];
+
+/**
+ * @param { string } filename
+ * @returns { boolean }
+ */
+export function isTestFile(filename) {
+	return TEST_PATTERNS.some((pattern) => matchesGlob(filename, pattern));
+}

.github/workflows/build-base-image.yml

@@ -32,11 +32,11 @@ jobs:
         # when updating.
         include:
           - node_version: '22'
-            dhi_ref: dhi.io/node:22.23.2-alpine3.24-dev@sha256:155a95d59244a71e9479e191ff718f03a7197d6829612e0707608be92988cb87
+            dhi_ref: dhi.io/node:22.23.3-alpine3.24-dev@sha256:23ee07ace431ea7d14e22c0e52fb52b0bd1853dabdacd89ba75f937f6dff8a8d
-          - node_version: '24.19.0'
+          - node_version: '24.21.0'
-            dhi_ref: dhi.io/node:24.19.0-alpine3.24-dev@sha256:dc2989ad23938772abaf549ed7bde5d61d70377005229bb55bef94bff40f8bbd
+            dhi_ref: dhi.io/node:24.21.0-alpine3.24-dev@sha256:d22ba0e9ab2a54609ae533df4ebfd6f36c1f233b662a4c547f5a3195e8c21937
-          - node_version: '26.7.0'
+          - node_version: '26.10.0'
-            dhi_ref: dhi.io/node:26.7.0-alpine3.24-dev@sha256:4b494d89fb26c950ce97865acf45b480dc7a6868fdc2b81c2d66599702eeac3f
+            dhi_ref: dhi.io/node:26.10.0-alpine3.24-dev@sha256:8da859df5dc853c923ace53c4cfe994a909a28917f7fcfc8508869beab8cf132
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 

.github/workflows/build-benchmark-image.yml

@@ -6,7 +6,7 @@ on:
     branches:
       - master
     paths:
-      - 'packages/@n8n/benchmark/**'
+      - 'packages/quality/efficiency/scale/benchmark/**'
       - 'pnpm-lock.yaml'
       - 'pnpm-workspace.yaml'
       - '.github/workflows/build-benchmark-image.yml'
@@ -33,7 +33,7 @@ jobs:
           DOCKER_BUILD_SUMMARY: false
         with:
           context: .
-          file: ./packages/@n8n/benchmark/Dockerfile
+          file: ./packages/quality/efficiency/scale/benchmark/Dockerfile
           platforms: linux/amd64
           provenance: false
           push: true

.github/workflows/build-node-pc-image.yml

@@ -25,7 +25,7 @@ on:
         type: boolean
 
 env:
-  NODE_VERSION: '26.7.0'
+  NODE_VERSION: '26.10.0'
 
 jobs:
   build:

.github/workflows/build-v3-nightly.yml

@@ -147,6 +147,8 @@ jobs:
           DOCKER_BASE: ${{ secrets.DOCKER_USERNAME }}
           N8N_IMAGE: ${{ needs.build.outputs.n8n_image }}
           N8N_DIGEST: ${{ needs.build.outputs.n8n_digest }}
+          N8N_PC_IMAGE: ${{ needs.build.outputs.n8n_pc_image }}
+          N8N_PC_DIGEST: ${{ needs.build.outputs.n8n_pc_digest }}
           RUNNERS_IMAGE: ${{ needs.build.outputs.runners_image }}
           RUNNERS_DIGEST: ${{ needs.build.outputs.runners_digest }}
           DISTROLESS_IMAGE: ${{ needs.build.outputs.runners_distroless_image }}
@@ -195,5 +197,6 @@ jobs:
           } >> "$GITHUB_STEP_SUMMARY"
 
           promote "$N8N_IMAGE" "$N8N_DIGEST" "${DOCKER_BASE}/n8n" ""
+          promote "$N8N_PC_IMAGE" "$N8N_PC_DIGEST" "${DOCKER_BASE}/n8n" "-pc"
           promote "$RUNNERS_IMAGE" "$RUNNERS_DIGEST" "${DOCKER_BASE}/runners" ""
           promote "$DISTROLESS_IMAGE" "$DISTROLESS_DIGEST" "${DOCKER_BASE}/runners" "-distroless"

.github/workflows/build-windows.yml

@@ -23,6 +23,7 @@ on:
     paths:
       - '**/package.json'
       - '**/turbo.json'
+      - 'packages/cli/scripts/build.mjs'
       - '.github/workflows/build-windows.yml'
       - '.github/actions/setup-nodejs/**'
 

.github/workflows/ci-cla-check.yml

@@ -69,8 +69,8 @@ jobs:
     steps:
       # Defaults for the CLA endpoints, status context, comment marker and label
       # are already n8n's production values.
-    - uses: n8n-io/github-actions/cla-check@ea082925743265a61d9fd0a69a32b211551f88f0 # v1.0.2
+      - uses: n8n-io/github-actions/cla-check@ea082925743265a61d9fd0a69a32b211551f88f0 # v1.0.2
-      with:
+        with:
           app-id: ${{ secrets.N8N_ASSISTANT_APP_ID }}
           private-key: ${{ secrets.N8N_ASSISTANT_PRIVATE_KEY }}
           pr-number: ${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.pull_request.number || github.event.issue.number }}

.github/workflows/ci-pr-quality.yml

@@ -99,8 +99,8 @@ jobs:
           mode: filter
           filters: |
             janitor:
-              packages/testing/playwright/**
+              packages/quality/testing/playwright/**
-              packages/testing/janitor/**
+              packages/quality/testing/janitor/**
 
   check-static-analysis:
     name: Static Analysis