🚀 Release 2.41.4 (#39940)

Co-authored-by: Matsuuu <16068444+Matsuuu@users.noreply.github.com>

.agents/review-rules/README.md

@@ -17,7 +17,7 @@ clear, when to stay silent, what never to say — lives once in
 | `backend/`  | Backend  | `cli`, `@n8n/db`, `core`, `workflow`, node packages |
 | `db-migrations/` | DB migrations | `@n8n/db` migrations + their tests in `cli` |
 | `frontend/` | Frontend | `packages/frontend`                                 |
-| `qa-dx/`    | QA & DX  | `.github`, `docker`, `scripts`, `patches`, `packages/quality`, the lint/test/TS config packages, baselines |
+| `qa-dx/`    | QA & DX  | `.github`, `docker`, `scripts`, `patches`, `packages/testing`, the lint/test/TS config packages, baselines |
 | `testing/`  | Backend + Frontend | any package with a test suite            |
 
 A directory maps to one agent unless, like `testing/`, the policy is identical
@@ -27,7 +27,7 @@ never a copy per directory. Security and QA & DX deliberately don't link
 agents shouldn't be able to produce.
 
 All five slots are used, so a new domain now merges into an existing agent. QA &
-DX covers the build, test, and CI surface — the same paths `OWNERS`
+DX covers the build, test, and CI surface — the same paths `.github/OWNERS`
 assigns to `@n8n-io/qa-dx`. Code-quality rules that happen to apply broadly
 (error classes, `any`, lazy imports) are backend rules, not QA & DX ones. DB
 migrations is split out of Backend because a migration is permanent and runs

.agents/review-rules/qa-dx/ratchets-and-allowlists.md

@@ -13,9 +13,8 @@ Flag a diff that **adds** entries to any of these, and ask for the fix instead:
 |------|---------|
 | `.code-health-baseline.json` | `@n8n/code-health` violations |
 | `.boundaries-baseline.json` | `turbo boundaries` issue count |
-| `packages/quality/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
-| `.code-health-baseline.json` | the `typeorm-persistence-boundary` baseline, which only shrinks as business logic moves queries into repositories |
-| `packages/cli/eslint.config.mjs` | the public-API allowlists captioned "NEVER add to this list" |
+| `packages/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
+| `packages/cli/eslint.config.mjs` | the `misplaced-n8n-typeorm-import` and public-API allowlists, each captioned "NEVER add to this list" |
 | `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
 
 Removals are the healthy direction and need no comment.

.agents/review-rules/qa-dx/workflow-safety.md

@@ -35,7 +35,7 @@ fed from a `needs:` list. Flag:
 Jobs gated on a `ci-filter` filter only run when a matching file changed, so a
 filter that is too narrow means the test quietly stops covering new code. When
 a PR adds a directory that an existing filter was meant to cover — a new package
-under `packages/quality/`, a new script under `.github/scripts/` — check the
+under `packages/testing/`, a new script under `.github/scripts/` — check the
 filter still matches it.
 
 ## Least privilege

.agents/skills/agent-design-language/SKILL.md

@@ -1,24 +0,0 @@
----
-name: n8n:agent-design-language
-description: Design or review Agent Builder interfaces. Use for Agent configuration patterns, modal flows, responsive behavior, status treatments, and other Agent-only UI work.
----
-
-# Agent design language
-
-Use this skill only for interface work in
-`packages/frontend/editor-ui/src/features/agents`.
-
-Before you change code:
-
-1. Read `packages/frontend/editor-ui/src/features/agents/AGENTS.md`.
-2. Read `packages/frontend/editor-ui/src/features/agents/agents-design-language.md`
-   completely.
-3. Use [ui-design](../ui-design/SKILL.md) for visual implementation rules.
-4. Use [content-design](../content-design/SKILL.md) when you add or change UI
-   text.
-
-Keep Agent-specific components in the Agent feature. Do not change the Design
-System unless the user explicitly asks for this change.
-
-Update the design language document when you establish a reusable Agent
-interface pattern. Add or update tests for the behavior that you change.

.agents/skills/community-pr-readiness-check/SKILL.md

@@ -113,7 +113,7 @@ Run when `AutoReject` is `null`. Full rules for each in `reference/checks.md`:
 
 ## Step 4 — Identify the responsible team
 
-Run `node .github/scripts/owners/owners.mjs` against the changed file list and map the winning GitHub team to a Linear team. Full mapping table, sub-agent fallback procedure, and label rules: see `reference/teams.md`.
+Run `node .github/scripts/owners.mjs` against the changed file list and map the winning GitHub team to a Linear team. Full mapping table, sub-agent fallback procedure, and label rules: see `reference/teams.md`.
 
 ## Step 5 — Extract the Linear ticket
 

.agents/skills/community-pr-readiness-check/reference/teams.md

@@ -16,7 +16,7 @@ Owner resolution, the GitHub-team → Linear-team → label mapping, and the Lin
 
 ## Identifying the owning team
 
-Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parses `OWNERS` with last-match-wins semantics and returns allocations sorted by file count with a `share` percentage. Using the script keeps this skill consistent with whatever CI uses.
+Use the canonical owners script at `.github/scripts/owners.mjs`. It parses `.github/OWNERS` with last-match-wins semantics and returns allocations sorted by file count with a `share` percentage. Using the script keeps this skill consistent with whatever CI uses.
 
 1. Write the PR's changed file paths (from the `files` list) to a temp file, one per line:
    ```bash
@@ -24,7 +24,7 @@ Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parse
    ```
 2. Run the script:
    ```bash
-   node .github/scripts/owners/owners.mjs /tmp/pr-<number>-files.txt
+   node .github/scripts/owners.mjs /tmp/pr-<number>-files.txt
    ```
 3. The script prints JSON of the form:
    ```json
@@ -38,10 +38,10 @@ Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parse
    ```
    Allocations are already sorted by `fileCount` descending — take the first entry as the winning team.
 4. Clean up: `rm /tmp/pr-<number>-files.txt`.
-5. Strip the `@n8n-io/` prefix from `allocations[0].team` — the GitHub team slug is `nodes`, `iam`, `ai`, etc. If `allocations` is empty (no file matched any rule, which is possible only if `OWNERS` lost its catch-all), fall back to `catalysts`.
+5. Strip the `@n8n-io/` prefix from `allocations[0].team` — the GitHub team slug is `nodes`, `iam`, `ai`, etc. If `allocations` is empty (no file matched any rule, which is possible only if `.github/OWNERS` lost its catch-all), fall back to `catalysts`.
 6. Map the GitHub team slug to its Linear team name and PR label using the table below. The `team` field in the JSON output is the **Linear team name**. If the resolved GitHub team slug has no entry in the table, fall back to `Engineering`.
 
-**Sub-agent fallback**: if `node` execution is denied by the sandbox, read `OWNERS` directly and apply last-match-wins by hand. All the active rules fit on one screen.
+**Sub-agent fallback**: if `node` execution is denied by the sandbox, read `.github/OWNERS` directly and apply last-match-wins by hand. All the active rules fit on one screen.
 
 ## GitHub team → Linear team → GitHub label
 

.agents/skills/create-agent-builder-eval/SKILL.md

@@ -20,7 +20,7 @@ Run this check before sourcing, drafting, or writing an eval. Run it from
 `packages/@n8n/instance-ai`:
 
 ```bash
-pnpm exec dotenvx run -f ../../../.env.eval -- \
+pnpm exec dotenvx run -f ../../../.env.local -- \
   sh -c 'test -n "${LANGTRACER_URL:-}" && test -n "${LANGTRACER_API_KEY:-}"'
 ```
 
@@ -28,7 +28,7 @@ If the check fails, stop before creating an eval file. Ask the user to:
 
 1. Generate a key on the
    [LangTracer API page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
-2. Add these variables to the repository root `.env.eval` file:
+2. Add these variables to the repository root `.env.local` file:
 
    ```env
    LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
@@ -45,8 +45,9 @@ value. Rerun the check after the user confirms. Continue only when it passes.
 - Author the case at
   `packages/@n8n/instance-ai/evaluations/data/agents/<slug>.json`.
 - Set `"datasets": ["agents"]`.
-- Ask the driver for the target suite before you do anything else. See
-  "Ask for the target suite first" below. Do not assume `agents`.
+- Push general Agent Builder cases to
+  [Instance AI capabilities — agents](https://lang-tracer.n8n-maintenance.workers.dev/suites/10).
+  Its suite slug is `agents`.
 - Do not commit the case JSON. LangTracer is the durable source of truth.
 - Commit changes to this skill, the harness, and CI when applicable.
 
@@ -69,189 +70,9 @@ or outcome expectations can prove the behavior.
 Use the substitution test for every expectation. A correct alternative build
 must pass. A build that misses the requested behavior must fail.
 
-## Write expectations about the Agent, not the builder's internals
-
-Agent Builder runs as a delegated sub-agent architecture today. That design
-can change. Do not write an expectation that depends on it. Follow
-"Keep expectations free of internal mechanics" in
-[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md).
-
-Do not write:
-
-- "The sub-agent runs/tests the Agent before it responds."
-- "Instance AI delegates to `build-agent`" or "calls `build-agent` once."
-- "The builder calls `call_agent` to verify the Agent."
-- "The sub-agent reads the skill, then sets the model."
-
-Write what a user or a reviewer can observe:
-
-| ❌ mechanical | ✅ intent |
-|---|---|
-| "The sub-agent tests the Agent before it reports" | "The final response says the Agent was tested only if a test ran, and it reports the result of that test honestly" |
-| "`build-agent` is called with the Slack credential" | "The Agent uses the Slack credential the user has" |
-| "The sub-agent adds a tool for the lookup" | "The Agent can look up the order status when a user asks for it" |
-| "The builder sets `model` after the catalog lookup" | "The Agent uses a model that the connected credential supports" |
-
-Put each one where it belongs:
-
-- The Agent's configuration, tools, skills, and behavior go in
-  `outcomeExpectations`.
-- What the user is told, asked, or shown goes in `processExpectations`.
-- The final response is the place to check claims ("says it is ready",
-  "lists the tools it added"). Do not check which component made the claim.
-
-The calibration steps below ask you to confirm that `build-agent` ran. That is
-a check for you, to be sure the case reached Agent Builder. Do not add it to the
-case.
-
 For multi-turn, seeded, or capability-gap cases, follow the case-shape and
 calibration rules in [create-instance-ai-eval](../create-instance-ai-eval/SKILL.md).
 This skill overrides its workflow directory and suite guidance for Agent cases.
-The sections below add the Agent-specific parts. Do not skip the general rules
-they point to.
-
-## Ask for the target suite first
-
-Follow "Ask for the target suite first" in
-[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md). Do it after the
-LangTracer preflight and before you source or draft anything. If the request
-already names a suite, state it and do not ask. Otherwise ask, in autonomous
-mode too, in the same message as the autonomy question.
-
-For Agent cases, recommend
-[Instance AI capabilities — agents](https://lang-tracer.n8n-maintenance.workers.dev/suites/10)
-(slug `agents`). Offer `baseline` as well: the Instance AI (INS) team monitors
-it and it runs nightly. Get the full list from `list_suites`. Use the suite the
-driver picks in every push command, in place of `agents` in the examples below.
-Keep the `agents` dataset and the `data/agents/` directory. They do not depend
-on the suite.
-
-## Set the autonomy level first
-
-Follow "Set the autonomy level first" in
-[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md). Ask the driver
-for *autonomous* or *checkpoint* mode before you source or draft anything. The
-four gates are the same: selection, shape and expectations, calibration, push.
-
-## Source the case from a real thread
-
-The best Agent cases come from a real conversation. Discover the conversation
-with LangTracer. Author a synthetic case from what you learn.
-
-1. List candidates: `list_conversations` with `usedAgentBuilder: true`. Do not
-   use `source: "agent-builder"`. That pool is a different product, has
-   unscrubbed payloads, and has no Agent snapshots.
-2. List what the thread did to the Agent: `list_conversation_agent_snapshots`.
-   `target-resolved` is the state that a turn opened on. `config-updated` is the
-   state after the builder changed it.
-3. Pick the shape from the snapshot sequence (next section).
-4. Fetch the state to seed with `get_conversation_agent_snapshot`. Seed from the
-   `target-resolved` row of the turn under test.
-5. Read the raw turn before you write any expectation. Follow
-   "First reproduce, then reclassify" in the general skill.
-
-Two snapshot patterns to look for:
-
-- A `target-resolved` hash that differs from the previous `config-updated` hash.
-  The user edited the Agent in the UI between turns. This is the best seed for a
-  repair or update case.
-- A run of turns that each end in `config-updated`. The user added one rule per
-  turn. Use it for a "new rule does not erase old rules" case.
-
-Credentials arrive as `[redacted]`. Instructions and skill bodies do not. Scrub
-the prose before it enters a case. See the `agents` section in
-[case-shapes.md](../create-instance-ai-eval/case-shapes.md) (the `agents` seed section).
-
-## Pick the shape: create or update
-
-| Shape | Question it answers | How to write it |
-|---|---|---|
-| **Create** | Does the builder make the right Agent from a fresh request? | One user turn. No seed. Assert on the Agent artifact. |
-| **Update** | Does the builder change only what the user asked for? | `seed.agents` with the Agent before the change. One live turn with the change. Assert on the new behavior and that the untouched parts survive. |
-| **Repair** | Does the builder find the cause of a failure and fix it? | Seed the broken Agent. The live turn reports the failure. Assert on the cause found and the fix applied. |
-| **Incremental rules** | Do earlier rules survive when a new rule lands? | Seed the Agent after several rules. Add one conflicting or overlapping rule. Assert that the old rules are still in force. |
-
-Rules that apply to all four:
-
-- An update that rebuilds the Agent from scratch is a failure. Assert on a
-  specific untouched instruction, tool, or skill that must still be there.
-- Write the conversation in English, even when the source thread is not.
-- Trim to the fewest turns that reproduce the behavior. Do not copy a long
-  thread.
-- Put only the live turn in `conversation`. Put earlier turns in `seed.messages`.
-
-## Confirm the precondition fired
-
-A pass can mean the builder did the right thing. It can also mean the situation
-never happened. Check before you trust a green.
-
-- For "the builder did not do X", confirm that it reached the turn where X was
-  possible. A builder that never edited the Agent passes "did not delete the old
-  rule".
-- For a repair case, confirm that the seeded Agent really has the defect. Read
-  the seeded config in the rendered artifact.
-- For an update case, compare the seeded config with the final config. Confirm
-  that the requested change is the only difference you expected.
-- For a "say you cannot" case, confirm that the tool or data was really missing
-  in the run.
-
-## A red is a finding
-
-Follow "A red is signal" in the general skill. For Agent cases, add this:
-
-1. Classify the red as a product gap, a harness limitation, or non-determinism.
-2. Keep a product-gap red. Start the case `description` with
-   `Capability-gap finding:`.
-3. **Propose a Linear ticket for the Agent team.** The Linear team is named
-   `Agent`. Propose the ticket. Do not create it unless the driver says go.
-   Follow "Capability gap → propose a Linear ticket" in the general skill. Also
-   check whether the gap already has a ticket.
-   - Write the ticket in English. Use Simplified Technical English, as
-     [AGENTS.md](../../../AGENTS.md) requires. Translate the user's words. Do
-     not paste them.
-   - This repository is public. Keep customer names, company names, and real
-     URLs out of the ticket. Describe the use case in neutral words.
-   - Link the LangTracer case and the source thread. Do not copy raw user
-     messages or agent instructions from the thread.
-   - Reproduce the gap first. This is a default, not a hard rule. Propose a
-     ticket only after a case is red on a real build. Quote the failing
-     expectation and the judge reason. Follow "Reproduce first" in the general
-     skill.
-   - The driver may approve an exception, for example when the harness cannot
-     reach the mechanism after about three attempts. Then the ticket must say
-     "Not reproduced in the eval. Based on trace analysis."
-4. **Check the builder's own guidance for the gap.** Read the files that steer
-   Agent Builder:
-   `packages/cli/src/modules/agents/builder/agents-builder-prompts.ts`,
-   `packages/cli/src/modules/agents/builder/prompts/`, and
-   `packages/cli/src/modules/agents/builder/skills/`. Also read the Instance AI
-   orchestrator prompt in `packages/@n8n/instance-ai/src/agent/system-prompt.ts`.
-   If none of them covers the failing situation, add that to the ticket as a
-   proposed prompt or skill change.
-5. Propose the smallest added line first. Do not delete existing guidance to fix a
-   gap. A measured case: adding one line fixed a routing gap, and deleting two
-   lines did not.
-6. Every prompt or skill change needs a before and after measurement. The new
-   case is the "before" run. Run it again after the change. Use `--iterations 5`.
-
-When the failure is in the Agent's runtime (for example a channel that cannot
-report an error), the builder cannot see it. The right fix is often a builder
-instruction to say so and stop guessing. Write that as a `processExpectations`
-item on the final response.
-
-## Publish and link
-
-Follow "Share links, never bare ids" and "Link the pushed case to its source" in
-the general skill. For a sourced Agent case:
-
-- Report the case, the suite, and the source thread as links. Use the
-  `https://lang-tracer.n8n-maintenance.workers.dev/test-cases/<id>` form.
-- Call `update_test_case` with `sourceThreadId`, `expectedBehavior`, and
-  `failurePattern`.
-- Add a capability tag with `add_case_tags`.
-- Push a case whose *build* is wrong with `--set-kind capability_gap` into a
-  suite of that kind. If no such Agent suite exists, tell the driver. Do not put
-  it in a `regression` suite.
 
 ## Draft the case
 
@@ -270,7 +91,7 @@ Start with this shape:
     "The final response ..."
   ],
   "outcomeExpectations": [
-    "A standalone Agent is the deliverable; any workflow created exists only as a tool the Agent calls.",
+    "A standalone Agent was created and no workflow was created.",
     "The Agent ..."
   ],
   "datasets": ["agents"]
@@ -340,14 +161,14 @@ credential such as `EVAL_OPENAI_API_KEY`. A build-only case does not need one.
 4. Classify each red as a product gap, harness limitation, or non-determinism.
 5. Use `--iterations 5` before adding a case to a gating tier.
 6. Preview the LangTracer change.
-7. Push it to the suite the driver chose (`<suite>` below).
+7. Push it to `agents`.
 
 ```bash
-pnpm exec dotenvx run -f ../../../.env.eval -- \
-  pnpm eval:langtracer-push --suite <suite> --dry-run --changed
+pnpm exec dotenvx run -f ../../../.env.local -- \
+  pnpm eval:langtracer-push --suite agents --dry-run --changed
 
-pnpm exec dotenvx run -f ../../../.env.eval -- \
-  pnpm eval:langtracer-push --suite <suite> --changed
+pnpm exec dotenvx run -f ../../../.env.local -- \
+  pnpm eval:langtracer-push --suite agents --changed
 ```
 
 The push needs `LANGTRACER_URL` and `LANGTRACER_API_KEY`. Generate a key on the
@@ -358,20 +179,12 @@ after a successful push.
 ## Completion checklist
 
 - The LangTracer environment preflight passed before authoring.
-- The driver chose the target suite before any work, including in autonomous mode.
-- The autonomy level was set and stated before any work.
-- A sourced case links to its source thread, and the source turn was read raw.
-- Each conditional expectation has proof that its precondition fired.
-- A kept product-gap red has a proposed Linear ticket and a check of the builder guidance.
 - The case is in `data/agents/` and uses the `agents` dataset.
 - The strict loader accepts it.
 - A local run captures a standalone Agent.
 - The transcript proves each process expectation was exercised.
 - The rendered Agent artifact proves each outcome expectation.
-- No expectation names a sub-agent, an internal tool, or a delegation step.
-  Each one passes for a correct build from a different architecture.
 - Repeated runs are stable enough for the selected tier.
-- The case is pushed to the suite the driver chose, and the report links to it.
-- Agent Builder PR changes select the `agents` suite in CI. A case pushed to
-  another suite does not run in that lane.
+- The case is pushed to `Instance AI capabilities — agents` with suite slug `agents`.
+- Agent Builder PR changes select the `agents` suite in CI.
 - The local case JSON is not committed.

.agents/skills/create-agent-builder-eval/local-setup.md

@@ -4,7 +4,7 @@
 
 Generate a key on the
 [LangTracer API keys page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
-Add it to `.env.eval` at the repository root:
+Add it to `.env.local` at the repository root:
 
 ```env
 LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
@@ -16,7 +16,7 @@ that both variables exist before it creates an eval.
 
 ## Remaining environment
 
-Create `.env.eval` at the repository root. Use one Instance AI model provider
+Create `.env.local` at the repository root. Use one Instance AI model provider
 and one sandbox provider.
 
 ```env
@@ -76,7 +76,7 @@ N8N_USER_FOLDER=<printed-temp-directory> \
 E2E_TESTS=true \
 N8N_ENABLED_MODULES=instance-ai,agents \
 N8N_AI_ENABLED=true \
-pnpm exec dotenvx run -f ../../.env.eval -- pnpm start
+pnpm exec dotenvx run -f ../../.env.local -- pnpm start
 ```
 
 Use different ports when 5680 or 5681 is busy.
@@ -96,7 +96,7 @@ curl -sf -X POST http://localhost:5680/rest/e2e/reset \
 From `packages/@n8n/instance-ai`:
 
 ```bash
-pnpm exec dotenvx run -f ../../../.env.eval -- \
+pnpm exec dotenvx run -f ../../../.env.local -- \
   env -u LANGSMITH_API_KEY -u LANGSMITH_TRACING \
   pnpm eval:instance-ai \
   --base-url http://localhost:5680 \

.agents/skills/create-instance-ai-eval/SKILL.md

@@ -25,9 +25,7 @@ exhaustive field reference; this skill is the opinionated *how*.
 > approach.** Author the file locally (uncommitted), calibrate it against a real
 > build, then **push it to a lang-tracer suite** with `eval:langtracer-push`
 > (see [Push to a lang-tracer suite](#push-to-a-lang-tracer-suite)) —
-> the suite the driver chose (see
-> [Ask for the target suite first](#ask-for-the-target-suite-first)) —
-> `baseline` for the consolidated corpus n8n CI runs, or a dedicated
+> `--suite baseline` for the consolidated corpus n8n CI runs, or a dedicated
 > capability suite like `agents`.
 > The suite is the home for the case; the eval CLI reads it back via
 > `--source langtracer`. You still write the JSON file — it's just the input to
@@ -40,34 +38,6 @@ exhaustive field reference; this skill is the opinionated *how*.
 > pruned and has no durable home. Don't commit a replay case either — derive a
 > synthetic case from it. See [`case-shapes.md`](case-shapes.md).
 
-## Ask for the target suite first
-
-**Before you source, draft, or run anything, ask the driver which LangTracer
-suite receives every eval you create in this session.** Ask this in both
-autonomy levels, including autonomous mode. Never pick the suite yourself and
-never default silently. A case pushed to the wrong suite runs in the wrong CI
-lane and alerts the wrong team.
-
-Ask the suite question and the autonomy question together, in one message.
-Recommend `baseline`:
-
-- `baseline` (recommended): the general corpus. The Instance AI (INS) team
-  monitors it, and it runs nightly.
-- Any other suite, such as `agents` or `node-gauntlet`: call `list_suites` and
-  offer the live list. Do not use a hardcoded list, because suites are added
-  and retired.
-
-Rules:
-
-- The driver's answer applies to all cases in the session. Do not ask again per
-  case.
-- If the driver already named a suite in the request, state it back in one line
-  and continue. Do not ask again.
-- Check that the suite kind matches the case. Push a case with a wrong build
-  (`capability_gap`) only into a suite that allows that kind. If the suite does
-  not allow it, tell the driver and ask again.
-- Put the chosen suite slug in the final decision log, as a link.
-
 ## Set the autonomy level first
 
 **Before you source, draft, or run anything, decide how hands-on the driver
@@ -170,7 +140,7 @@ discover → verify → encode workflow.
 
 ## Pick the case shape first
 
-The corpus is five archetypes. Decide which you're writing before you draft — it
+The corpus is four archetypes. Decide which you're writing before you draft — it
 determines the fields, the grading, and how you validate. They compose (a seeded
 case can still assert outcome), but the primary shape drives the work.
 
@@ -180,25 +150,10 @@ case can still assert outcome), but the primary shape drives the work.
 | **Behaviour / process** | Does the agent *converse* correctly (ask the right clarifying question, not re-ask, honour a correction, respect plan approval)? | `processExpectations` + multi-turn director script; often **build-only** |
 | **Credential** | Does the build behave correctly given a specific credential view? | `credentials[]` |
 | **Seeded** | Start mid-thread, with prior work already in place, and drive the turn under test | `seed` (authored `mode: "inline"`; `"replay"` for a local check) |
-| **Context** | Does the agent still work once the conversation is long — does it reuse what it already read, and keep what matters after observational memory compacts the history? | `processExpectations` + a long `seed`; `requiresMemoryCompaction` for the compaction ones |
 
-**Build** is documented in full below. The other four, the director-script
+**Build** is documented in full below. The other three, the director-script
 vocabulary, and the seeding modes are in [`case-shapes.md`](case-shapes.md).
 
-The judge also sees **token ground truth** for workflow-build cases: per-turn
-input and output tokens inline in each transcript turn header, a build-wide
-total, a cache read/write split, and the opening step's cost. So an expectation
-may reference cost or consumption directly ("does not re-read the same node's
-schema in turn 2"). The numbers cover the orchestrator's own LLM steps only. A
-delegated Agent build runs in a sub-agent whose steps are not in the snapshot,
-so do not write cost expectations on an Agent case. Those numbers come from run-debug snapshots, so they need
-`N8N_INSTANCE_AI_RUN_DEBUG_ENABLED=true` on the instance under test; without it
-the judge reads `(no run debug captured)` and cost expectations are ungradeable.
-
-The judge also sees the thread's **observation rows** — what observational
-memory kept after it compacted — so a memory case can grade the summary itself.
-That block is a separate REST read and needs no flag.
-
 ## Core principle (all shapes)
 
 **Write expectations from intent, then calibrate against a real build.** Decide
@@ -498,16 +453,6 @@ read the run assumes someone already owns it. So once a red is classified as a
 real gap (and the driver has confirmed it, per the autonomy level), **propose a
 Linear ticket for it.**
 
-**Reproduce first. This is a default, not a hard rule.** Propose a ticket only
-after a case has reproduced the gap. That means the case is red on a real build,
-the precondition was confirmed to fire, and you re-read the raw thread (see
-[First reproduce, then reclassify](#first-reproduce-then-reclassify)). A gap that
-comes from trace analysis alone is a hypothesis. A ticket for it costs the owning
-team time. The driver may approve an exception, for example when the harness
-cannot reach the mechanism after about three attempts. In that case the ticket
-must say that the eval did not reproduce the gap. Never file an unreproduced gap
-without the driver's approval.
-
 **Propose, don't create.** Per [AGENTS.md](../../../AGENTS.md), never open a
 Linear ticket unasked. Put the draft in front of the driver — interactively in
 checkpoint mode, in the decision log in autonomous mode — with a title, a team,
@@ -655,16 +600,9 @@ nodes get LLM-generated pin data). So:
   static data** (`removeItemsSeenInPreviousExecutions`, `$getWorkflowStaticData`)
   is **not** seedable, because static data starts empty every run, so such a
   scenario reds vacuously (it sees everything as "new"). To get a seedable
-  change-detection scenario, say "keep the state in a data table" in the prompt;
-  otherwise
+  change-detection scenario, steer the build toward a Data Table; otherwise
   accept the static-data red as a harness limit and carry the logic in
   `outcomeExpectations`. Note the agent may *choose* static-data dedup on its own.
-- **Message an Agent runs the called Agent with its real model.** The harness
-  mocks the Agent's tools, MCP servers and web search, and records those calls
-  under the calling node. It turns off the Agent's memory, vector stores and chat
-  integrations. Only OpenAI models are supported today, and only when
-  `EVAL_OPENAI_API_KEY` is set (CI sets it): put the Agent on an `openai/*` model
-  and declare an `openAiApi` credential.
 - Don't assert exact counts that depend on mock generation ("exactly 7 posts").
   Say "fewer than the original 10".
 
@@ -758,9 +696,8 @@ Both are natural-language assertions graded by the same Sonnet judge, and each
   They run everywhere, including prebuilt/MCP runs (no transcript needed).
 - **`processExpectations`** — **how the agent behaved during the build**, judged
   from the transcript. Assert clarifying questions asked (or not re-asked),
-  user-visible behaviour, plan/approval handling, batching, honouring a
-  correction, ordering. Do not assert internal mechanics such as sub-agents or
-  tool names (see "Keep expectations free of internal mechanics"). They need a transcript, so they're **skipped in prebuilt/MCP runs**.
+  tool-call behaviour, plan/approval handling, batching, honouring a correction,
+  ordering. They need a transcript, so they're **skipped in prebuilt/MCP runs**.
 
 Rule of thumb: an assertion about *the artifact* is an outcome expectation; an
 assertion about *the conversation or the agent's choices along the way* is a
@@ -798,54 +735,6 @@ case, where the source and channel were **left unspecified**):
 Put intent the conversation only *implied* (a preferred but unstated channel) in
 `processExpectations`, not `outcomeExpectations`.
 
-## Keep expectations free of internal mechanics
-
-An expectation describes what the user or the artifact sees. It does not
-describe how Instance AI is built inside. The internal architecture changes:
-sub-agents, tool names, delegation, planning steps, and run ordering all move.
-An expectation that names them goes red when the design changes, although the
-behavior is still correct. It also stops proving the behavior: a build that
-solves the problem by another route fails it.
-
-**Do not name these in an expectation:**
-
-- Sub-agents, delegation, or hand-off ("the sub-agent runs ...", "the
-  orchestrator delegates ...").
-- Internal tool names or call counts ("calls `build-agent`", "calls
-  `search-nodes` twice").
-- Internal stages, task lists, or step order that the user cannot see.
-- Which component produced a result.
-
-**Do name these:**
-
-- What the user sees in the conversation: the question asked, the plan shown
-  for approval, the claim in the final response.
-- What the artifact contains: the workflow or Agent, its configuration, and
-  how it behaves.
-- A user-visible sequence, only when the user asked for it ("asks before it
-  builds").
-
-Rewrite test. Delete the internal term. State the result the term produced.
-
-| ❌ mechanical | ✅ intent |
-|---|---|
-| "The sub-agent runs the workflow before it reports success" | "The final response reports only results that the build verified; it does not claim an untested step works" |
-| "The orchestrator calls `ask-user` before delegating" | "The agent asks for the missing Slack channel before it builds, and does not guess one" |
-| "The builder calls `search-nodes` for the trigger" | "The workflow starts from a trigger that matches the requested schedule" |
-
-Second test. Imagine the internals are replaced by one agent with different
-tools. A correct build must still pass. If the expectation would fail or become
-meaningless, rewrite it.
-
-Some mechanics are a legitimate subject: a case whose purpose is to guard a
-specific tool contract (for example, a regression in how a tool reports an
-error). Keep these rare. State the reason in `description`. Do not mix them with
-intent expectations in the same case.
-
-**Calibration steps are not expectations.** Reading the transcript to confirm
-that a build ran, or that a sub-agent was reached, is a good calibration check.
-Keep it in your own run notes. Do not copy it into the case JSON.
-
 ## Robust design vs harness flakiness
 
 Two different things — keep them apart:
@@ -932,24 +821,16 @@ drifted, leaves the rest unchanged, and never prunes. It's the inverse of
 
 ```bash
 cd packages/@n8n/instance-ai
-# <suite> is the slug the driver chose (e.g. baseline)
 # preview first — no writes:
-pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite> --dry-run --changed
+pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite baseline --dry-run --changed
 # then push (drop --dry-run):
-pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite> --changed
+pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite baseline --changed
 ```
 
 - **Selectors** (at least one required — no accidental push-all): positional
   `<slugs...>` (exact file slugs), `--changed` (new/untracked + staged + modified
   `data/{workflows,agents}/*.json`, ideal right after authoring an uncommitted case),
   `--filter`/`--tier` (with `--exclude` as a modifier).
-- **Validation is selective.** Exact slugs and `--changed` read only the named
-  files, so an unrelated invalid file in `data/workflows` (a case authored on a
-  newer branch, a half-written draft) never blocks your push. `--filter` parses
-  only the files whose slug matches; `--tier` reads the tier from inside each
-  file, so it parses them all. An invalid file either one parses prints
-  `⚠ skipped invalid case file …` and the push continues without it. A file you
-  named still fails the push when it is invalid.
 - **Multiple positional slugs? Skip pnpm — call the script directly.** `pnpm
   eval:langtracer-push … slugA slugB` forwards the slugs as one joined argument
   (`"slugA slugB"`), so no case file matches and nothing is pushed. Either use a

.agents/skills/create-instance-ai-eval/case-shapes.md

@@ -1,7 +1,7 @@
 # Case shapes beyond the plain build
 
 The [SKILL](SKILL.md) covers the **build** archetype. This file covers the other
-four — **behaviour/process**, **credential**, **seeded**, **context** — plus the
+three — **behaviour/process**, **credential**, **seeded** — plus the
 director-note vocabulary multi-turn cases rely on. Field-level docs live in the
 eval [README](../../../packages/@n8n/instance-ai/evaluations/README.md); this is
 the opinionated *how* and the traps.
@@ -662,24 +662,6 @@ really had one — and the agent has to guess from prose that deliberately names
 nothing. It will list workflows and pick, or ask which one, and you will score a
 clarification failure the real user never hit.
 
-#### Seeded faults must really fail
-
-`seed.priorRuns[].hints` steer the mock for the staged run's outbound calls and
-nothing else. The builder's own runs are never mocked, and Code, Set, IF, Filter
-and Merge run for real. So:
-
-- **The fault must fail in n8n.** A Code node that returns a plain object does not
-  fail; n8n wraps it as one item. Return an array of plain values, reference a
-  node that does not exist (`$('Missing')`), or throw.
-- **The fault fires before any external call.** The builder's rerun reaches a
-  fixture host and dies there (DNS, 401), so a fault behind an HTTP node is never
-  reached. Never grade the rerun itself; grade the fix with `outcomeExpectations`
-  and the harness's `executionScenarios`.
-- **An empty branch needs a Filter that keeps nothing.** Split Out over an empty
-  array still emits a placeholder item.
-- **Walk the item flow before asserting a count.** A Slack node fed three items
-  posts three times.
-
 #### Before you ship a seeded case — three checks
 
 1. **The defect still bites.** A seed whose workflow isn't broken any more makes the
@@ -696,139 +678,3 @@ build". A seeded thread starts with an empty sandbox, so the agent re-reads the
 workflow from the database and re-derives SDK source a real resumed session would
 still have on disk. The bias is *harder* than reality, so those numbers read worse
 for a reason that has nothing to do with the builder.
-
----
-
-## Context cases (long conversations, token cost, compaction)
-
-A context case asks whether the agent still works once the conversation is
-long. Two questions, two mechanics.
-
-### The judge already has the token numbers
-
-Every case's judge context carries token ground truth, so no new field is
-needed to grade cost:
-
-- Each transcript turn header reads
-  `Turn 2 (3 steps, 47446 tokens in [41220 cached], 812 tokens out)`.
-- A **Total** block sums every LLM step across the whole build, main run and
-  resumes, plus a cache read/write split and a **Fixed overhead** line — the
-  opening step's input, which is instructions and tool schemas plus one user
-  message, so it reads as the floor cost of any turn on that instance. Use it
-  to separate "the prompt got bigger" from "this conversation got expensive".
-
-So `processExpectations` may reference consumption directly. Write them as
-statements about *behaviour the numbers reveal*, not as thresholds:
-
-```json
-"processExpectations": [
-  "The agent answers the second question without calling get_node_details on the Notion node again — it read that schema in turn 1 and the answer needs nothing new.",
-  "Turn 2's input tokens are mostly cache reads, which is what an unchanged prefix looks like. A turn that re-reads the history from scratch shows a cache-read share near zero."
-]
-```
-
-Absolute token budgets rot: a prompt edit moves every number and the case goes
-red for a reason that has nothing to do with the agent. Grade the *shape* —
-what got re-read, what came from cache, which turn is the expensive one.
-
-Two limits to know before you write a cost expectation:
-
-- It needs `N8N_INSTANCE_AI_RUN_DEBUG_ENABLED=true` on the instance under test.
-  Without it the blocks render `(no run debug captured)` and the expectation is
-  unjudgeable — it fails or passes at random.
-- The capture hooks **only the orchestrator's own stream**. A workflow build
-  runs inside that loop, so a workflow case's numbers are its real cost. A
-  delegated **Agent** build (the `build-agent` tool) runs on a separate stream
-  with no hooks, so its tokens are missing from every block. Don't write a cost
-  expectation on an `data/agents/` case.
-
-### `requiresMemoryCompaction` — grade the post-compaction window
-
-Observational memory compacts a long thread: an Observer summarises it into
-observation rows, the summary is rendered into the system prompt, and the early
-turns are masked out of the agent's window. A case that tests whether a decision
-*survives* that has a setup problem — production compacts at 30k tokens of
-visible message content, which is a conversation too long to hand-author.
-
-Set the flag and the harness handles it:
-
-```json
-"requiresMemoryCompaction": true
-```
-
-Two things happen, both per-thread, so no other case in the run is affected:
-
-1. The harness sends a low observer threshold with every turn of that thread
-   only: **1,000 tokens** of visible message content, the request field's floor.
-   The seed plus the live turns must cross it, or the observer never runs — a
-   short seed does not test memory, it reports **not judged** (below). Size the
-   seed generously; a few real turns of tool output cross 1,000 tokens easily.
-   The Observer's prompt, masking and cursor logic are the production ones at
-   any threshold.
-2. After the build, the harness checks the premise actually held. If it did
-   not, every expectation on the case is reported **not judged** — an
-   `incomplete` verdict, excluded from scoring, not a red.
-
-The premise check is the point of the flag. Uncompacted, the raw early turns
-are still in the window, so the agent answers off them and every expectation
-passes for free — a green that tests nothing.
-
-The evidence is structural. `GET /rest/instance-ai/eval/threads/:threadId/memory`
-returns the observation rows and the compaction cursor from their own tables. A
-cursor means the observer ran and everything up to `lastObservedMessageId` is
-masked out; the rows are what replaced it. Missing either, the case had nothing
-to test and is reported not judged. Nothing reads the system prompt, so this
-needs **no debug flag** and a prompt or SDK rename cannot quietly turn "never
-compacted" into the answer.
-
-The judge never sees the premise check — it grades the conversation, not whether
-the harness configured the scenario. A misconfigured lane must not read as a
-quality regression.
-
-### Grade the summary, not just the reply
-
-The judge *does* see the observation rows, as ground truth:
-
-```
-## Observational memory after compaction (ground truth — do not recount)
-
-- [CRITICAL] User ruled OUT the native Slack node; will post through the HTTP
-  Request node instead, because the workspace is pinned to a legacy Slack API
-  version. Do not revisit.
-- [CRITICAL] Alert fires when queue depth goes above 4700 items.
-```
-
-So write expectations at **both** layers:
-
-```json
-"processExpectations": [
-  "The observational-memory block retains the alert threshold as 4700 queued items. A summary that keeps the topic but loses the number is the specific failure this looks for.",
-  "In its reply to the final turn, the agent gives the alert threshold as 4700 queued items."
-]
-```
-
-The first grades the memory system, the second grades the agent. Grading only
-the reply is weaker than it looks: the agent can answer correctly from the
-unmasked tail, or guess well, and the case goes green while memory dropped the
-detail. The `[MARKER]` is the Observer's own priority label, so "retained as
-CRITICAL" is checkable too.
-
-### Writing the conversation
-
-The shape that makes a compaction case real:
-
-1. **State the anchors once, in a `user` turn near the start of the seed**, and
-   never restate them. Anything the agent repeats later lands *after* the
-   cursor and survives for the wrong reason.
-2. **Make live turn 1 off-topic.** Its answer must not re-surface the anchors
-   into the post-cursor window. Compaction runs at the end of it.
-3. **Ask for the anchors back in live turn 2.** That turn runs against the
-   compacted context.
-4. **Force the antecedent** — a lost anchor must produce a *confident wrong*
-   answer, not a vague one. "Posting goes through HTTP Request, not the native
-   Slack node" works because the Slack node is the obvious guess. "The
-   threshold is 4700" works because a guess reaches for 5000. See
-   [negative expectations need a forced antecedent](SKILL.md#core-principle-all-shapes).
-
-Tag these `context-consumption` and put them in a `context` dataset, so a
-tier that needs run debug enabled can select them.

.agents/skills/create-instance-ai-eval/running-evals.md

@@ -218,44 +218,6 @@ deliberately does not re-sync tier-only edits to an existing case. **Only
 promote to `pr` after `--iterations 5+` shows it's reliably green** — a flaky
 case in the gate poisons it.
 
-## Calibrating a batch of new cases on the dispatchers
-
-One case costs 10–20 minutes per iteration on a laptop, and docker lanes have
-crashed it. A batch calibrates on the LangTracer dispatchers:
-
-1. Author the JSON locally. Static checks only: a `--dry-run` push (schema) and
-   the similarity check against the verification suites.
-2. Push the batch to its suite with the tag `calibration-pending`.
-3. Dispatch one manual sweep for those case ids:
-   `gh workflow run eval-run.yml --repo n8n-io/lang-tracer -f image=n8nio/n8n:nightly -f case_ids=<ids> -f iterations=3 -f trigger=manual`.
-   `trigger=manual` keeps it out of the nightly baseline.
-4. Read `list_eval_runs` → `get_eval_run`. Classify each red: **builder** (keep
-   it red, tag `capability-gap-finding`, propose a ticket), **harness** (fix it or
-   design around it; never leave it to charge the model), **authoring** (fix the
-   case).
-5. Re-push the fixed cases and re-dispatch only those ids.
-6. Swap `calibration-pending` for `calibrated` and stamp the evidence into the
-   description, e.g. "Calibration (Opus 4.8, N=3): sweep 174 3/4, sweep 176 4/4".
-
-## Validating a harness change before merge
-
-```bash
-gh workflow run test-evals-instance-ai.yml --repo n8n-io/n8n \
-  -f branch=<your-branch> -f cache-sha=$(git rev-parse origin/<your-branch>) \
-  -f suite=<suite> -f filter=<slug-substrings> \
-  -f iterations=3 -f experiment-name=<change-name>
-```
-
-`cache-sha` is what puts a backend change under test: without it the workflow
-restores the n8n image cached for master's head, and only the eval CLI runs from
-your branch. A harness-only change works either way.
-
-CI reads cases from LangTracer only. A case that exists only on disk goes into a
-throwaway suite first (`eval:langtracer-push --suite <scratch-suite> <slug>`).
-Never pass `experiment-name=instance-ai-baseline`; that name refreshes the shared
-baseline. Results: the run log's summary table and the
-`instance-ai-workflow-eval-results` artifact (`eval-results.json` + HTML report).
-
 ## Baselines & regression
 
 When `LANGSMITH_API_KEY` is set, every run auto-compares against the most recent

.agents/skills/create-instance-ai-eval/sourcing-cases.md

@@ -47,20 +47,9 @@ ids. Full table in
 1. **Scan cluster themes** — `list_cluster_runs` / `get_latest_cluster_run`
    return capability-gap themes with a `label`, `summary`, and `mechanism`: the
    real, recurring failure modes.
-2. **Pull the conversations.** Start from what LangTracer already detected.
-   `list_signal_rates` lists every code check and label rule with its rate over a
-   window: what fails most right now. Then `list_conversations` with `signals`,
-   one token per signal, ANDed — `check:has_nodes`,
-   `label:task_outcome=agent-gave-up`, `analysis:bad`, `keeper:*`, `review:*` — puts
-   the matching entries on each row, each with its evidence (the verdict comment,
-   the judge's words, Keeper's reason) and its version. On a candidate,
-   `get_conversation_signals` shows what every source said and which ones agreed:
-   a thread flagged by a code check, the judge and Keeper is a stronger source
-   than one flagged by a single model pass. A source marked `not_run` never looked
-   at the thread, so read its silence as unknown, not as clean. The older path
-   still works: `list_conversations` (e.g. `verdict:"bad"`, `analyzed:"yes"`) →
-   `get_conversation` (raw trace) + `get_conversation_analysis` (findings).
-   `get_linear_ticket_context` links a thread to its ticket. To target
+2. **Pull the conversations** — `list_conversations` (e.g. `verdict:"bad"`,
+   `analyzed:"yes"`) → `get_conversation` (raw trace) + `get_conversation_analysis`
+   (findings). `get_linear_ticket_context` links a thread to its ticket. To target
    **execution failures**, add `funnelDrop:"05"` (built + launched an execution
    that never *succeeded*). **Caveat: the funnel drop is a *conversion* signal, not
    a build-quality one** — most `funnelDrop:"05"` threads are healthy builds that
@@ -72,8 +61,7 @@ ids. Full table in
    (was `build-workflow` called on this turn?), which it reads reliably — but a
    content-dependent claim ("invented ID", "missing node") can be wrong when it
    couldn't see the built workflow. Confirm against the raw trace before building
-   a case around it. The same goes for a signal's `evidence`: it is what that
-   source saw, not the trace.
+   a case around it.
 4. **Optionally record the selection as an observation.** Offer to save why this
    thread was selected and what the developer observed. This is a best-effort
    LangTracer note, not a gate: skip it when the developer declines or has not

.agents/skills/create-pr/SKILL.md

@@ -34,7 +34,7 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
 - `benchmark` - Benchmark CLI changes
 - `core` - Core/backend/private API
 - `editor` - Editor UI changes
-- `engine` - New workflow execution engine v2 (@n8n/engine package & engine-v2 module in cli)
+- `engine` - New workflow execution engine v2 (@n8n/engine package)
 - `* Node` - Specific node (e.g., `Slack Node`, `GitHub Node`)
 
 ### Summary Rules
@@ -43,7 +43,7 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
 - Capitalize first letter
 - No period at the end
 - No ticket IDs (e.g., N8N-1234)
-- Add `(no-changelog)` suffix to exclude from changelog, unless the type already excludes it
+- Add `(no-changelog)` suffix to exclude from changelog
 
 ## Steps
 
@@ -78,24 +78,11 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
    body structure, then populate each section with actual content before
    creating the PR:
    - **Summary**: describe what the PR does
-   - **Why**: explain why the change is needed (see Why below)
    - **How to test**: describe how to test the changes. Include an example workflow if appropriate
    - **Related tickets**: add the Linear ticket URL (`https://linear.app/n8n/issue/[TICKET-ID]`) and any GitHub issue links
    - **Checklist**: keep as-is from the template
 	 - Add a "🤖 PR Summary generated by AI" at the end of the body
 
-   Before you create the PR, proofread the body with these checks. Rewrite
-   each sentence that fails a check:
-   1. The Summary and the Why together are under the word budget. If not,
-      first remove the details that the reviewer reads in the diff, then
-      shorten the Why.
-   2. Each sentence names who does what: a component, a process, or a person
-      as the subject, with a plain technical verb.
-   3. Each domain term has a plain definition where it first appears.
-   4. Each sentence is true for the diff.
-
-   When a later commit changes a fact in the body, update the body.
-
    ```bash
    gh pr create --draft --title "<type>(<scope>): <summary>" --body "$(cat <<'EOF'
    <populated body based on pull_request_template.md>
@@ -107,43 +94,12 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
 
 Based on `.github/pull_request_template.md`:
 
-Write for a reviewer who does not know this area of the code. The reviewer
-must understand the change before they open the diff. Use ASD-STE100
-Simplified Technical English: short sentences, active voice.
-1. Keep the Summary and the Why together under about 200 words. A small PR
-   needs much less.
-2. Define each domain term in plain words where it first appears. Do not use
-   a ticket ID or an internal name as an explanation.
-3. Leave out details that the reviewer reads in the diff: error cases, status
-   codes, renamed fields, the order of calls, file lists, test names. Mention
-   one only when it changes behaviour that a reviewer could miss. Also leave
-   out what reviewers already know (framework behaviour).
-4. Use plain technical verbs, not metaphors ("passes the value", not
-   "hands the value" or "lets the value through").
+Write the PR body in ASD-STE100 Simplified Technical English: use short
+sentences, the active voice, and one instruction for each sentence.
 
-### Summary: what the PR does
-1. Start with one to three sentences of background for a reader who is new to
-   the area. Then state what the PR does.
-2. Write **prose** about behaviour, not a list of code edits. Name a class or
-   a function when it represents an essential/core concept and it helps
-   the reader to navigate the change.
-3. If the PR changes no product code, say so in the first sentence
-   (for example: "This PR only adds tests.").
-4. Do not justify the change here. The justification goes in the Why.
-5. Include screenshots or videos for UI changes.
-
-### Why: the reasoning
-Always fill the `## Why` section. The Summary and the core of the
-Why are what the reviewer needs most. Keep the core short:
-1. Do not repeat the Summary.
-2. State the problem or the constraint that makes the pre-existing code unfit.
-3. If there are accepted gaps, or changes for existing code paths, state them
-   in a separate short paragraph.
-
-Add rejected alternatives only when a reviewer would otherwise ask about them,
-if any. Use one or two sentences for each alternative: state the concrete
-reason it fails. Write an alternative that was never built in the conditional.
-Together, the alternatives must be shorter than the core Why section.
+### Summary Section
+- Describe what the PR does
+- Include screenshots/videos for UI changes
 
 ### How to test Section
 - Explain how to test the changes
@@ -191,7 +147,7 @@ feat(API)!: Remove deprecated v1 endpoints
 
 ### No changelog entry
 ```
-feat(core): Experimental feature not enabled yet (no-changelog)
+refactor(core): Simplify error handling (no-changelog)
 ```
 
 ### No scope (affects multiple areas)

.agents/skills/db-migrations/SKILL.md

@@ -110,7 +110,6 @@ Run through this before requesting review. Each item is a real, recurring review
 - [ ] Identifiers go through **`escape.tableName(...)` / `escape.columnName(...)`**. Never hand-write `n8n_table` prefixes. — [Always escape identifiers](#always-escape-identifiers)
 - [ ] **Match column type to value semantics.** Native `uuid` for UUIDs, `timestampTimezone()` for timestamps, a numeric type for numbers, `bool` for booleans, `json` for structured data. Never `varchar` as a catch-all. — [Column types](#column-types)
 - [ ] **Pick the narrowest sane type within that category:** `int`/`smallint` not `bigint` when range allows; `text` not `varchar(255)` for unbounded strings; never `double` for version numbers. — [Column types](#column-types)
-- [ ] **A generated `id` key is an integer** (`int.primary.autoGenerate2`). A UUID key is a UUIDv7. For a UUID or string key, write the reason in the PR description. A join table uses its foreign-key columns as the key, not an `id` column. — [Primary Keys](#primary-keys)
 - [ ] **Default `notNull`**, relax only when justified. PK is implicitly NOT NULL. Migration's `notNull` matches the entity's nullability. — [NOT NULL and entity parity](#not-null-and-entity-parity)
 - [ ] **Enum-like columns** carry `.withEnumCheck([...])` AND `.comment('explains values')`. Opaque IDs / unix timestamps / JSON shapes also get `.comment()`. — [Constrain enum-like strings](#constrain-enum-like-strings), [Add comments on columns](#add-comments-on-columns)
 - [ ] **Every reference column has an explicit FK** with deliberate `onDelete`. Name FKs explicitly when SQLite recreate cycles risk duplicating them. Avoid polymorphic `(typeCol, idCol)` patterns. — [Foreign Key Constraints](#foreign-key-constraints), [General Design Guidance](#general-design-guidance)
@@ -440,32 +439,11 @@ Don't set the default of a `status` column to a terminal value — `"running"` m
 
 ### Primary Keys
 
-Every table needs a primary key. **Use an integer key by default.** Read the options below in order. Use the first option that fits. Do not skip an option only because you like a later one better.
+Every table needs a primary key. Choose the type in this order:
 
-1. **Integer (default)** — `column('id').int.primary.autoGenerate2`. Use this for every new table. Use a different option only when one of its conditions is true. Integer keys are small and make joins fast. `autoGenerate2` uses Postgres `IDENTITY`. Do not use `autoGenerate`. It uses `serial`, which is deprecated.
-2. **UUID** — `column('id').uuid.primary`. Use this only when one of these conditions is true:
-   - The application must know the ID before the insert. For example, the client makes the ID.
-   - The ID is in a URL or a public API. Users must not see how many rows exist or guess other IDs.
-
-   **You must use UUIDv7.** Make the ID in application code with the `uuid` package: `import { v7 as uuidv7 } from 'uuid'`.
-
-   Why: a UUIDv7 starts with a timestamp. Thus, new rows go near the end of the index. A UUIDv4 is random, for example `randomUUID()` from `node:crypto`. New rows go to random positions in the index. This splits index pages and makes the index larger.
-
-   A UUIDv7 does not give an exact creation order. Two IDs made in the same millisecond can sort in the wrong order. Clock differences between hosts can also cause this. If you need the exact order, sort by a timestamp column.
-
-   Do **not** chain `.autoGenerate2` on `.uuid`. The DSL throws an error. The database default makes a UUIDv4. Also, `uuid_generate_v4()` needs the `uuid-ossp` extension. Some managed Postgres services, such as Supabase, do not have it. Use `.uuid`, not `.varchar(36)`.
-3. **String** — `column('id').varchar(36).primary`. Use this only when one of these conditions is true:
-   - The entity already uses IDs in a format that is not a UUID. For example, the public API or the UI already shows nanoid IDs for this entity.
-   - The UI shows the ID to users. A UUID is too long to read, copy, or type.
-
-   Entity IDs are usually nanoids with 16 characters.
-
-**Give a reason for every key that is not an integer.** Write the reason in the PR description. Reviewers will ask for it. These reasons are not sufficient:
-- "Other tables use string IDs."
-- "This module already does it this way."
-- "We might need it later."
-
-**These options apply to a generated `id` column only.** A join table that only links other tables does not need an `id` column. Use its foreign-key columns as a composite key (see below). A foreign-key column always has the same type as the key that it refers to.
+1. **Integer** — `column('id').int.primary.autoGenerate2`. Preferred for new tables: compact, fast joins, no ordering surprises. `autoGenerate2` uses Postgres `IDENTITY` (preferred over the deprecated `serial`-based `autoGenerate`).
+2. **UUID** — `column('id').uuid.primary`. Use when IDs are generated client-side, exposed in URLs, or need to be unguessable. Generate UUIDs in application code via `randomUUID()` from `node:crypto`; do **not** chain `.autoGenerate2` on `.uuid` (the DSL throws — `DEFAULT uuid_generate_v4()` fails on managed Postgres like Supabase because it needs the `uuid-ossp` extension in `public`). Use `.uuid` instead of `.varchar(36)`.
+3. **String** — `column('id').varchar(36).primary` for IDs whose format isn't a UUID (e.g. nanoid-style IDs). Convention: nanoid length 16 for entity IDs.
 
 **Keep ID-column types consistent across related tables.** Mixing `uuid` and `varchar(36)` for what is "the same kind of ID" creates JOIN footguns.
 
@@ -718,7 +696,7 @@ If only Postgres needs the change, just put the file in `postgresdb/`; don't wri
 
 ### Postgres-version-aware UUID generation
 
-`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, make a UUIDv7 in application code — see [Primary Keys](#primary-keys).
+`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, generate at the application level with `randomUUID()` — see [Primary Keys](#primary-keys).
 
 ### SQLite doesn't enforce `varchar(N)` length
 

.agents/skills/env-var-docs/SKILL.md

@@ -1,158 +0,0 @@
----
-name: n8n:env-var-docs
-description: >-
-  Writes and reviews the JSDoc on `@Env(...)` config properties. The docs team
-  extracts these comments from source into the public environment variable
-  reference, so they are user documentation, not code comments. Use when adding
-  or changing an `@Env` property, a config class, or reviewing a PR that touches
-  one.
----
-
-# Environment variable docs
-
-The JSDoc above every `@Env('...')` property is exported almost verbatim into
-the [environment variables reference](https://docs.n8n.io/deploy/host-n8n/configure-n8n/basic-configuration/use-environment-variables)
-on docs.n8n.io. The generated table has four columns: variable name,
-description, data type, default. Your JSDoc is the description column. The
-reader is a self-hosting operator, not an n8n developer.
-
-Follow the [n8n docs style guide](https://docs.n8n.io/contribute/contribution-guide-for-n8n-docs/style-guide-for-n8n-docs)
-for these comments. It overrides the ASD-STE100 rule in `AGENTS.md` for this
-one case. The rules below are the parts of that guide that matter for a
-one-paragraph description.
-
-## Rules
-
-- **Every `@Env` property has a JSDoc.** No exceptions. A missing comment
-  ships as an empty description cell.
-- **Write for the operator.** Say what the setting does, when to change it,
-  and what happens when it's unset or wrong. Don't explain why the code is
-  written the way it is.
-- **No code references.** No file names, class names, function names,
-  constants, license keys, or other internals the reader can't open. Values
-  the operator types into the variable are fine and often needed, for example
-  a credential type ID like `googleOAuth2Api` or a mode like `postgresdb`.
-  The test: would the operator paste this into a config file, or would they
-  need the source to know what it means? Details for developers go in a
-  regular `//` comment on the implementation, not in the JSDoc.
-- **No attack scenarios.** Say what the setting protects, not how an
-  attacker would exploit it. This is a public page.
-- **Don't repeat the default.** The default renders in its own column. Don't
-  write "Defaults to true" or "Default: 1 hour". Do say what the unset state
-  means when it's not obvious, for example "When unset, n8n uses the Redis
-  host".
-- **State units and formats.** Say "in milliseconds", "in bytes", "a
-  comma-separated list", "a JSON object". The data type column only says
-  `Number` or `String`.
-- **Use allowed values when the set is small.** "One of `sqlite`,
-  `postgresdb`." For a list, show one inline example in the sentence.
-- **No JSDoc tags.** No `@example`, `@default`, `@see`, `@deprecated`. The
-  exporter copies the tag name into the docs. Put the example or the
-  deprecation notice in a sentence instead.
-- **Product names:** "Gateway credits", not "n8n Connect" or "AI Gateway".
-  "Publish a workflow", not "activate". `n8n` is always lowercase.
-- **Style:** present tense, active voice, second person. Use contractions.
-  No em dashes, use a comma or a new sentence. No "e.g." or "i.e.", write
-  "for example" and "that is". Spell out zero to nine, numerals from 10.
-  Values the operator types stay numerals, for example "Set to 0".
-  Sentence case. One idea per sentence. End every sentence with a period.
-- **Plain words:** "use" not "utilize", "lets you" not "allows you to", cut
-  "powerful", "seamless", "simply", "it's important to note".
-- **Length:** one to three sentences. If the setting needs more, the extra
-  belongs in a docs page the comment can't link to yet; keep the comment to
-  what fits a table cell.
-
-## Examples
-
-Bad, as found in the generated docs:
-
-```ts
-/**
- * Instance-level enablement for n8n Connect (AI Gateway). Licensed instances
- * are enabled by default; false opts out. Budget / free allowance still comes
- * from the license (`quota:aiGatewayBudget`).
- */
-```
-
-Wrong product name, a restated default, and an internal license key. Good:
-
-```ts
-/**
- * Whether Gateway credits are available on this instance. Set to false to
- * turn them off on a licensed instance. The credit allowance comes from your
- * license.
- */
-```
-
-Bad:
-
-```ts
-/**
- * Time to live in milliseconds for Redis cache entries. Set to 0 to disable
- * expiry. Default: 1 hour.
- */
-```
-
-The default is already in the table. Good:
-
-```ts
-/**
- * Time to live in milliseconds for Redis cache entries. Set to 0 to keep
- * entries until Redis evicts them.
- */
-```
-
-Bad:
-
-```ts
-/**
- * Comma-separated list of credential types for which overwrites are skipped
- * when the credential has been customized.
- * @example `N8N_SKIP_CREDENTIAL_OVERWRITE=zohoOAuth2Api,salesforceOAuth2Api`
- */
-```
-
-The `@example` tag lands in the description as literal text. Good:
-
-```ts
-/**
- * Comma-separated list of credential types to skip when applying credential
- * overwrites, for example `zohoOAuth2Api,salesforceOAuth2Api`. n8n skips an
- * overwrite only when a user has already changed one of its fields.
- */
-```
-
-Bad:
-
-```ts
-/**
- * Bind OAuth credential callbacks to the browser that initiated the flow.
- * When enabled, a session-scoped `n8n-oauth-binding` cookie is set on flow
- * initiation and verified on callback. Defeats phishing attacks where an
- * attacker injects an OAuth authorize URL into a victim's browser to land
- * tokens on the attacker's credential.
- */
-```
-
-Attack narrative and an internal cookie name on a public page. Good:
-
-```ts
-/**
- * Whether an OAuth credential connection must finish in the same browser
- * that started it. When enabled, n8n rejects the callback if it arrives
- * from a different browser.
- */
-```
-
-## Review checklist
-
-When you review a PR that adds or changes an `@Env` property, check each
-comment against this list and quote the fix in the review:
-
-1. JSDoc present.
-2. No code identifiers, file names, or internal keys.
-3. No default restated.
-4. Units, format, or allowed values stated.
-5. No JSDoc tags.
-6. No em dashes, Latin abbreviations, or filler words.
-7. Reads correctly as a sentence in a table cell with no surrounding context.

.agents/skills/nathan/SKILL.md

@@ -13,11 +13,11 @@ the same commands from the repo via `pnpm nathan`.
 ## Prerequisites
 
 - **No tunnel setup needed.** Nathan replies asynchronously through a short-lived
-  Cloudflare Quick Tunnel that the script opens for you (`npx cloudflared`). A
-  `deploy` to a **new** `test-<name>` also polls that instance URL directly, so it
-  still reports success even if the tunnel drops. (Redeploying a name that's
-  *already up* skips the poll — it can't tell the old instance from the new — so
-  it falls back to the tunnel; prefer a fresh name when you need the reliable report.)
+  public tunnel that the script opens for you (`npx localtunnel`). A `deploy` to a
+  **new** `test-<name>` also polls that instance URL directly, so it still reports
+  success even if the tunnel drops. (Redeploying a name that's *already up* skips
+  the poll — it can't tell the old instance from the new — so it falls back to the
+  tunnel; prefer a fresh name when you need the reliable report.)
 - **A token in `~/.n8n/dev/nathan-token`.** If a command reports no token, **ask the
   user for one** — point them at the form
   (`https://internal.users.n8n.cloud/form/d6d34a2f-4899-4ee8-afc8-f8c41a8a243d`),

.agents/skills/public-api/SKILL.md

@@ -21,7 +21,8 @@ check the code, not license to drop a team default.
 ## Non-negotiable rules
 
 - New endpoints are `@PublicApiController` classes under `v1/controllers/`, one
-  `*.public.controller.ts` per feature.
+  `*.public.controller.ts` per feature. A controller is a class — never
+  `export =` (the legacy tuple style; `require-public-api-controller` flags it).
 - Public API and internal REST are separate HTTP surfaces. A public controller
   never calls an internal controller/endpoint; both reuse the same service.
 - Controllers and handlers delegate to a service — never import a repository or
@@ -30,6 +31,7 @@ check the code, not license to drop a team default.
   `@ApiResponse(Dto)`.
 - Register each controller via a side-effect import in `v1/controllers/index.ts`
   (`public-api-controllers.test.ts` fails otherwise).
+- Don't add business logic to legacy `express-openapi-validator` (EOV) handlers.
 - Migrating a legacy endpoint must not change its public contract.
 
 These are `n8n-local-rules` ESLint rules (see `packages/cli/eslint.config.mjs`)
@@ -39,7 +41,6 @@ add to it.
 
 ## Team defaults
 
-- Write code that acts as its own documentation. The schema, the decorator, and the test should make the rule clear on their own without a comment.
 - List endpoints: cursor-based pagination (internal API uses both cursor- and
   page-based — don't copy an internal endpoint's model).
 - Pagination args are always `offset` and `limit` — on service methods, handler
@@ -107,10 +108,10 @@ model; reuse only what applies. Decorators, all from `@n8n/decorators`:
 | `@Get/@Post/@Put/@Patch/@Delete('/path')` | Route method. |
 | `@ApiKeyScope('res:action')` | API-key grant check. |
 | `@ProjectScope/@GlobalScope('res:action')` | User RBAC check. |
-| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO or a binary body — throws. For a binary response body (returned as `{ body, headers? }`), use `@ApiResponse(status, { mediaType, description?, headers? })` — see [Binary response bodies](reference.md#binary-response-bodies). |
+| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO — throws. |
 | `@ApiErrorResponse(status)` | Declares an additional documented non-2xx status (e.g. `404`, `409`). Stack multiple for more than one. `400`/`401`/`403` are added automatically (body/query present, always, and `@ApiKeyScope` present, respectively) — don't declare those yourself. |
 | `@ApiSummary(text)` / `@ApiDescription(text)` / `@ApiTags([...])` | OpenAPI summary/description/tags. `@ApiTags` sorts alphabetically regardless of the order you pass. All optional but expected on every real route. |
-| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. `@Body` is JSON by default; `@Body({ mediaType: 'multipart/form-data', uploadLimits })` takes a `multipart/form-data` body instead — see [Request body media types](reference.md#request-body-media-types). |
+| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. |
 | `@Licensed('feat')` | Gates the route on a single `BooleanLicenseFeature`; `PublicApiControllerRegistry` runs its own license middleware (after auth/`@ApiKeyScope`/`@ProjectScope`|`@GlobalScope`, before the handler) and 403s unlicensed requests. Only takes one feature — if the gate is an any-of/all-of combination (e.g. `LicenseState.isProvisioningLicensed()`, which is `feat:saml` OR `feat:oidc`), `@Licensed` can't express that; check manually in the handler instead, same as the internal `provisioning.controller.ee.ts`/`role-mapping-rule.controller.ee.ts` do today (throwing `ForbiddenError` on failure). |
 
 ## Authorization (easy to get wrong)
@@ -198,5 +199,6 @@ existing tests.
 - [Test-before-save endpoints](reference.md#test-before-save-endpoints)
 - [Errors](reference.md#errors)
 - [Testing matrix](reference.md#testing-matrix)
+- [Migrating legacy EOV endpoints](reference.md#migrating-legacy-eov-endpoints)
 - [Verifying a migration](reference.md#verifying-a-migration)
 - [CI and merging](reference.md#ci-and-merging)

.agents/skills/public-api/reference.md

@@ -103,76 +103,55 @@ scope, RBAC denial. Add whichever apply, matching the nearest existing tests:
   unchanged. Tests alone cannot show this — see
   [Verifying a migration](#verifying-a-migration).
 
-## Request body media types
+## Migrating legacy EOV endpoints
 
-`@Body` defaults to `application/json`; the body is already parsed by the
-app-wide `bodyParser` before the registry sees it. Declaring
-`@Body({ mediaType: 'multipart/form-data', uploadLimits })` instead takes a
-`multipart/form-data` body:
+Legacy `express-openapi-validator` endpoints live under
+`v1/handlers/`, wired through `openapi.yml` with `x-eov-operation-*` and request
+types in `packages/cli/src/public-api/types.ts`. Treat these as migration targets,
+not templates.
 
-- `uploadLimits: () => MultipartUploadLimits` is a thunk, read once the route
-  handles its first request (not at startup) — read live config inside it
-  (e.g. `Container.get(GlobalConfig)`), don't inline a literal.
-- The route's `@Body` DTO validates text fields merged with uploaded files,
-  not `req.body` alone. A text field and a file share the same flat object;
-  a file field uses `publicApiUploadedFileSchema` (`@n8n/api-types`) — a
-  multer-file-shaped schema that documents itself as `{ type: 'string',
-  format: 'binary' }`. Several files under the same field name become an
-  array.
-- An unknown field (one the DTO doesn't declare, on a `{ strict: true }` DTO)
-  fails with `Unexpected form field "<name>"` — not the default "unrecognized
-  keys" wording.
-- Multer parsing errors map to these statuses: `413` for a size or count limit,
-  `500` for `LIMIT_UNEXPECTED_FILE`, `400` for any other multer error, and `500`
-  for anything else (a malformed body). The `500` is not masked, because it is a
-  `ResponseError`. A missing boundary gives `400` with `multipart file(s) required`.
-- The body is parsed **after** every auth/scope/license/quota gate and
-  **before** controller/route middlewares — a caller those gates would reject
-  never has their (possibly huge) body read off the socket, and a middleware
-  that reads `req.body` sees it already parsed.
-- `/discover` shows no request schema for a multipart route (same as a legacy
-  multipart route today) — a client can't assume a JSON schema it never gets.
-- The generator documents the body under the `multipart/form-data` content
-  key (not `application/json`) and adds `413` alongside `415` to the route's
-  documented responses automatically.
-
-Everything above lives in `packages/cli/src/public-api/media-types/`, one
-handler per media type (`REQUEST_BODY_HANDLERS` in
-`media-types/request-body/index.ts`). Adding a further media type (e.g.
-`application/octet-stream`) means: add it to `RequestBodyMediaOptions` in
-`@n8n/decorators`'s `controller/types.ts`, write a handler implementing
-`RequestBodyHandler` in `media-types/request-body/`, and register it in
-`REQUEST_BODY_HANDLERS` — the registry, resolver, generator and `/discover`
-need no change, since they all read the handler, not the media type.
-
-## Binary response bodies
-
-`@ApiResponse(status, { mediaType, description?, headers? })` declares a
-success body that the controller method returns as `{ body, headers? }`, for
-example an `application/gzip` stream. The framework treats options whose
-`mediaType` is in `BINARY_RESPONSE_MEDIA_TYPES` as a binary response.
-`mediaType` accepts only those types.
-
-- `body` is a `Buffer` or a `Readable`. The method returns the result and never
-  writes to `res`.
-- `headers` holds the response headers. Every header declared in `headers` must
-  be in the result. A missing header fails the request with a `500`. Matching is
-  case-insensitive.
-- The registry checks the headers, then sets the declared status, the headers,
-  and `Content-Type: <mediaType>`. A header in the result cannot replace the
-  `Content-Type`.
-- A stream's first chunk is read before any header is set. So a stream that
-  fails before its first chunk gives the normal JSON error response, with no
-  binary `Content-Type`. A method that throws before it returns does the same.
-- After the first chunk is sent, an error ends the response. A client that
-  disconnects is not an error, and the framework destroys the stream.
-- A method that returns no binary body fails with a `500`.
-- The generator documents the body as `{ type: string, format: binary }`
-  under the `mediaType` content key, with the description and headers.
-  The description defaults to `Operation successful.`.
-
-The runtime part is `sendBinaryResponse` in
-`packages/cli/src/public-api/media-types/binary-response.ts`.
+- Prefer migrating to `@PublicApiController` over extending the handler.
+- Preserve the public contract: path, method, scopes, status codes, response
+  shape, and pagination.
+- Move HTTP concerns into the controller and business orchestration into the
+  shared service; keep the controller a thin HTTP boundary. If the legacy
+  handler was itself already just a thin wrapper around an internal
+  `@RestController` (calling its methods directly, e.g.
+  `Container.get(SomeController).createThing(req, res, payload)`), the new
+  public controller can call that same internal controller directly — no need
+  to duplicate its validation/business logic.
+- A route must be served by either the EOV handler or a controller, not both —
+  the build's `mergeDecoratorDocument` (`v1/openapi-gen/generate.ts`) throws on
+  a path+method declared by both sides. Remove the legacy wiring (its path's
+  `$ref` entry in `openapi.yml`, the `x-eov-operation-*` handler, and its
+  `handler.ts`) only after the new controller is registered, `pnpm build`
+  regenerates the spec cleanly, and tests are updated.
+- Fully delete the migrated legacy files: the handler's `.ts`, its
+	`spec/paths/*.yml` and `spec/schemas/*.yml`, and any now-dead request type
+	in `packages/cli/src/public-api/types.ts`. Then check
+  `v1/shared/spec/schemas/_index.yml` and `v1/shared/spec/parameters/_index.yml`
+  for entries that `$ref` one of the deleted schema/parameter files — those are
+  separate from the path's own `$ref` in `openapi.yml` and are easy to miss;
+  left dangling, the next bundle fails on a broken `$ref`.
+- If the legacy handler gated on a license (`isLicensed('feat:x')` middleware),
+  `@Licensed('feat:x')` now replicates that for a controller route (see the
+  decorator table in [SKILL.md](SKILL.md#declaring-a-controller)) — but only for
+  a single feature. If the legacy check was an any-of/all-of over several flags
+  (e.g. `LicenseState.isProvisioningLicensed()`), `@Licensed` can't express
+  that; replicate it manually in the controller instead, don't drop it - this
+  is exactly what the internal `provisioning.controller.ee.ts` and
+  `role-mapping-rule.controller.ee.ts` already do, since neither uses
+  `@Licensed` for that reason.
+- As a legacy file drops repository access / the `export =` tuple, remove its
+  entry from the `off` allowlists for `no-repository-in-public-api-handler` and
+  `require-public-api-controller` in `packages/cli/eslint.config.mjs` (shrink-only
+  — never extend them).
+- For complex legacy-only, multipart, or non-standard endpoints, study the
+  nearest existing handler first.
+- Keep each field in its original position when you extract a request shape shared
+  by two routes, and destructure out the ones a route doesn't take. The generator
+  emits properties in shape order, so a moved field rewrites the `*.generated.yml`
+  of a route the PR wasn't changing.
 
 ## Verifying a migration
 

.agents/skills/reproduce-bug/SKILL.md

@@ -32,7 +32,7 @@ Based on the affected area, pick the test layer and pattern:
 | CLI / API | Vitest integration | setupTestServer + supertest | `packages/cli/test/integration/` |
 | Config | Vitest unit | GlobalConfig + Container | `packages/@n8n/config/src/configs/__tests__/` |
 | Editor UI | Vitest | Vue Test Utils + Pinia | `packages/frontend/editor-ui/src/**/__tests__/` |
-| E2E / Canvas | Playwright | Test containers + composables | `packages/quality/testing/playwright/` |
+| E2E / Canvas | Playwright | Test containers + composables | `packages/testing/playwright/` |
 
 ## Step 3: Locate Source Files
 

.agents/skills/testmax-assistant-ui/SKILL.md

@@ -1,464 +0,0 @@
----
-name: n8n:testmax-assistant-ui
-description: >-
-  Tests an n8n Assistant (Instance AI) UI feature end to end. Drafts the test
-  paths from the feature and the diff, writes one Playwright script for each
-  path, runs the paths in parallel on fresh n8n instances with a real LLM, and
-  compares the main Assistant screens with master to find visual
-  regressions. Shows live screenshots during the run, and builds one HTML
-  report with video replays, step screenshots, LLM steps, tokens, cost and
-  visual diffs. Use when the user wants to test an Instance AI or Assistant
-  UI feature end to end, check that a UI change does not break existing
-  screens, or says /testmax-assistant-ui.
----
-
-# Test an Instance AI feature
-
-Use this skill to test a UI feature of the n8n Assistant end to end. In code,
-the Assistant is "Instance AI". You draft the test paths, the user approves
-them, and the skill runs them all at the same time. A local page shows each run
-live. At the end, you judge each run and build one report.
-
-The skill uses the scripts of `testmax-n8n-ai` for the n8n API, the
-LangSmith usage, the live page server, and the report template. Do not copy
-those scripts.
-
-## What the skill does
-
-```mermaid
-flowchart LR
-    A[Feature + diff] --> B[Draft scenarios.json<br/>and scripts]
-    B --> T[--try and --estimate]
-    T --> C{User approves<br/>plan + cost}
-    C --> D[run.py]
-    D --> E[One fresh branch instance<br/>for each feature + regression]
-    D --> F[Fresh branch instance<br/>visual]
-    D --> G[Fresh master instance<br/>visual, or a kept run]
-    E & F & G --> H[Live page]
-    F & G --> I[Pixel + accessibility tree diff]
-    E & F & G --> J[summary.txt with triage]
-    J --> K[You judge: verdicts.json]
-    K --> L[Report rebuilds itself]
-```
-
-- Each run is one scenario: a Playwright script that drives the real
-  Assistant UI in Chrome, with a real LLM.
-- There are three kinds of scenario:
-  - `feature`: tests a path of the new feature.
-  - `regression`: tests an existing flow that the feature touches.
-  - `visual`: captures screens. The skill runs it on the branch and on master,
-    then compares the captures.
-- All instances are fresh: a new data folder, a new owner, and no workflows or
-  threads. Thus the screens of the branch and master are the same when the
-  code is the same. A fresh instance starts in about 5 seconds, because it
-  copies a data folder that is already migrated and seeded.
-- Each feature and regression scenario gets its own fresh instance, so the
-  scenarios do not see each other's data. The visual scenarios share one
-  branch instance and one master instance.
-- The feature flags are the same on all instances. The instances do not ask
-  PostHog for flags, so no instance gets a random experiment variant. Set the
-  flags that the feature needs in the plan, with `featureFlags`.
-- `run.py` builds this checkout first (with turbo, so a build that is up to
-  date takes about 1 second). The master instance starts from a git worktree
-  at `~/.cache/n8n-feature-test/master`, at the merge base of `HEAD` and
-  `origin/master`. Thus the visual check shows only the changes of this
-  branch, not the changes that master got after the branch started.
-- A master visual run that passed is kept. The next run with the same merge
-  base, the same script and the same flags uses it again, and does not start
-  master.
-
-## Requirements
-
-Run `python3 .agents/skills/testmax-assistant-ui/scripts/run.py --check`
-before the first run. The commands below use `$S` for this scripts folder.
-`--check` checks the tools,
-Chrome, the settings, the disk space and the master build, and prints the
-plan folder for this branch.
-
-- Google Chrome. The scripts use the Playwright package of the repository.
-- The Instance AI model settings in `.env.local` or `.env.ai-surfaces`, as for
-  your own instance. Fresh instances get the same settings, without the
-  license, the database, the port and the encryption key.
-- `LANGSMITH_API_KEY` and LangSmith tracing, for the tokens and cost. Without
-  them, the report shows no tokens or cost. Do not set
-  `N8N_DIAGNOSTICS_ENABLED=false`: then Instance AI sends no traces.
-- About 3 GB of disk space for the master worktree and its build.
-
-Never put a key on the command line. Read settings from the env files and the
-environment only.
-
-## 1. Understand the feature
-
-Find what changed and what the user wants to test:
-
-```bash
-git diff origin/master...HEAD --stat -- packages/frontend packages/@n8n/instance-ai packages/cli/src/modules/instance-ai
-git diff origin/master...HEAD -- <the files of the feature>
-```
-
-Read the changed Vue components, the stores, and the backend events that they
-use. Find the `data-test-id` values of the new elements. If the user did not
-describe the feature, ask for one or two sentences about it.
-
-If the plan has visual scenarios, start the master baseline now, in the
-background. The first build takes 15 to 20 minutes. Later builds take less
-than a minute when the turbo cache has the files. You can draft the plan while
-it builds:
-
-```bash
-S=.agents/skills/testmax-assistant-ui/scripts
-python3 $S/run.py --check         # the setup, and the plan folder
-python3 $S/baseline.py prepare    # in the background
-python3 $S/baseline.py status     # later: is it ready?
-```
-
-## 2. Draft the test paths
-
-Write the plan folder that `--check` prints:
-`~/.cache/n8n-feature-test/plans/<branch>/<feature>/`. The folder stays after
-a reboot. It holds `scenarios.json` and one script for each scenario. The
-folder name is the name of the run folders.
-
-Draft these paths:
-
-- **Feature paths.** The main path, then each branch of it: other user
-  choices, empty and long inputs, cancel and stop, a reload in the middle, a
-  second thread, errors that the user can see. One path for each scenario.
-- **Regression paths.** The existing flows near the change: a plain chat
-  reply, approvals, the plan review, the preview panel, the thread list. The
-  library has `smoke-chat` for the plain chat reply.
-- **Visual screens.** The screens that the change can affect, on the branch
-  and on master. Capture each screen in a stable state. Mask all text that
-  comes from the LLM, and all data that changes between runs. The library has
-  `visual-assistant-home` and `visual-thread-reply`.
-
-Keep each scenario short and focused. A scenario that fails must show where it
-failed. Use more scenarios, not longer ones. A visual scenario must not need
-the new feature, because it also runs on master.
-
-Some UI states show only for a moment, for example a status line that folds
-away when the run ends. Check them while the run is active, not after it.
-
-If the feature is behind a feature flag or an experiment, set the flag in
-`featureFlags`. Without it, the instances use the default value of each flag.
-
-Before you show the plan to the user, check the new scripts and get the
-estimate (see [Check the scripts](#check-the-scripts)). Then show the user the
-list of paths: the id, the kind, the title and the expected result of each.
-Also show the estimate: the number of LLM runs, the cost and the time. Change
-the plan until the user approves it.
-
-### scenarios.json
-
-```json
-{
-  "title": "Stop button in the Assistant",
-  "feature": "The user can stop a run while the Assistant works.",
-  "scenarios": [
-    {
-      "id": "stop-mid-run",
-      "title": "Stop a run in the middle",
-      "kind": "feature",
-      "script": "stop-mid-run.mjs",
-      "expected": "The stop button shows during the run. A click ends the run without an error message.",
-      "timeoutMin": 6
-    },
-    {
-      "id": "smoke-chat",
-      "title": "A prompt gets a reply",
-      "kind": "regression",
-      "script": "lib:smoke-chat.mjs",
-      "expected": "A short prompt gets the reply OK."
-    }
-  ]
-}
-```
-
-The plan has these fields: `title` (required), `feature` (the feature, for the
-judge), and `featureFlags` (optional). `featureFlags` maps a flag name to
-`true`, `false` or a variant name, for example
-`{ "<flag name>": "variant" }`. Each instance gets these flags, on the
-branch and on master. They go into `N8N_FEATURE_FLAG_OVERRIDES` and replace
-the values in your env files.
-
-Each scenario has these fields:
-
-| Field | Required | Description |
-|---|---|---|
-| `id` | yes | Lowercase letters, digits and `-`. No `--`. Unique. |
-| `title` | yes | A short name for the report. |
-| `kind` | yes | `feature`, `regression` or `visual`. |
-| `script` | yes | A path relative to `scenarios.json`, or `lib:<file>` for a library script. |
-| `expected` | yes | The expected result, for the judge. |
-| `autoConfirm` | no | Answer the Assistant's confirmations automatically. Default `true`. Credential and setup cards get "Later". Set `false` when the script answers a confirmation or a card. |
-| `isolated` | no | Run the scenario on its own fresh instance. Default `true`. Instances start ahead of their scenario, so this costs almost no time. Set `false` to run the scenario on one branch instance that it shares with the other `false` scenarios. Visual scenarios ignore this field. |
-| `timeoutMin` | no | The time limit of the scenario. Default 10. |
-| `viewport` | no | The window size, for example `1600x1000` (the default). |
-
-`examples/stop-button/` is a full example plan.
-
-### Scenario scripts
-
-A script exports one async function. It gets the kit `t`:
-
-```js
-export default async function (t) {
-	await t.step('Open the Assistant', () => t.goto());
-	await t.step('Send a prompt', () => t.send('Reply with exactly the word OK and nothing else.'));
-	await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
-	await t.check('The reply says OK', () => t.expect(t.ai.assistantMessages().last()).toContainText('OK'));
-	await t.capture('thread', { mask: [t.ai.assistantMessages()] });
-}
-```
-
-| Kit member | What it does |
-|---|---|
-| `t.step(title, fn)` | Runs an action. Records a screenshot and the time. A failure stops the scenario. |
-| `t.check(title, fn)` | Runs an assertion, with a screenshot. A failure is recorded, and the scenario continues. |
-| `t.note(title)` | Adds a step with a screenshot and no action. |
-| `t.goto(path = '/assistant')` | Opens a page and waits for the chat input. Closes the "Try new n8n Assistant" dialog. |
-| `t.send(text)` | Types into the chat input, sends, and waits for the thread URL. Records the thread for the trace. |
-| `t.waitForIdle({timeout, settle})` | Waits until the thread has no active run. Returns `'idle'`, or `'suspended'` when the run waits for the user. |
-| `t.capture(name, {mask, maskText, root, fullPage})` | Saves a screen and its accessibility tree for the visual check. `mask` takes locators, and `maskText` takes texts: all elements with that text. Masked areas show as one color. In the tree, the subtree of a `mask` element and each masked text show as `[masked]`. |
-| `t.api(method, path, body)` | Calls the n8n REST API as the signed-in user. Returns `data`. |
-| `t.threadStatus(threadId)` | Returns `{active, suspended, background}`: the thread has a run, the run waits for the user, and the number of running background tasks. |
-| `t.ai.*` | Locators for the Assistant: `chatInput`, `sendButton`, `stopButton`, `userMessages`, `assistantMessages`, `assistantText(text)`, `panelText(text)`, `sidebarToggle`, `threadList`, `newThreadButton`, `approveButton`, `denyButton`, `planApproveButton`, `previewPanel`, `previewCanvasNodes`, and more in `scripts/kit.mjs`. |
-| `t.expect`, `t.page`, `t.baseUrl`, `t.sleep(ms)` | Playwright `expect`, the page, the instance URL, and a pause. |
-
-Rules for scripts:
-
-- Use `data-test-id` locators (`page.getByTestId`). Use text only for text that
-  the test controls.
-- Use `t.expect(...).toBeVisible()` and other assertions that wait. Do not use
-  `t.sleep` to wait for the UI.
-- LLM answers change from run to run. Check the UI state, not the exact words.
-  Ask for a fixed reply, such as "Reply with exactly the word OK", when you
-  need a known answer.
-- To test an approval, set `"autoConfirm": false`, wait with
-  `t.waitForIdle()` until it returns `'suspended'`, then click the button.
-
-### Check the scripts
-
-Check the new scripts on fresh instances before you show the plan:
-
-```bash
-python3 $S/run.py --plan <plan>/scenarios.json --try --only id1,id2
-```
-
-`--try` does not run master, does not start the live page, and builds no
-report. It prints a table with the status, the steps and the first error of
-each run. The runs are in `/tmp/feature-test/try/`. Fix the scripts that fail
-because of the script, then try them again. Each LLM scenario in a try costs
-money too, so try only the new scripts.
-
-Then get the estimate for the full run:
-
-```bash
-python3 $S/run.py --plan <plan>/scenarios.json --estimate
-```
-
-The estimate uses the cost and time of earlier runs of the same scenario ids.
-A scenario without an earlier run shows `*` and the median of all earlier
-runs. A master run that is kept shows `(kept)` and costs nothing.
-
-## 3. Run
-
-Open the live page for the user on each run. Do not skip this step. Do this:
-
-1. Find out if your harness has a browser tool that opens a tab in the
-   editor. In Cursor, this is `browser_navigate` of the `cursor-ide-browser`
-   MCP server.
-2. If it has one, start the run in the background with `--no-open`:
-
-   ```bash
-   python3 $S/run.py --plan <plan>/scenarios.json --no-open
-   ```
-
-   Wait for the `live page: <url>` line in the output, which comes in a few
-   seconds. Then open that URL in a new editor tab that the user can see (in
-   Cursor: `newTab: true` and `position: "active"`). If the tab does not open,
-   run `open <url>` (macOS) or `xdg-open <url>` (Linux).
-3. If it has none, start the run in the background without `--no-open`.
-   `run.py` opens the live page in the default browser:
-
-   ```bash
-   python3 $S/run.py --plan <plan>/scenarios.json
-   ```
-
-4. Give the user the URL in both cases.
-
-`run.py` does not read `$BROWSER`, so do not set it.
-
-`run.py` does these steps:
-
-1. Prints the estimate and checks the setup. It stops when the setup has an
-   error, for example a missing Chrome.
-2. Copies the plan into a new run folder,
-   `/tmp/feature-test/<time>-<plan folder name>/`.
-3. Starts the live page and prints its URL. Without `--no-open`, it opens the
-   page in the default browser.
-4. Builds this checkout with turbo. A build error stops the run, and the
-   output shows the end of `build.log`.
-5. Starts the instances: one for each feature and regression scenario, one
-   branch instance for the visual scenarios, and one master instance. It
-   starts the instances ahead of their scenario. Master starts only for the
-   master runs that it does not keep from an earlier run. It prepares the
-   master build first, if it is not ready.
-6. Runs the scenarios, 4 at a time. Each run records a video, the steps with
-   screenshots, the browser errors, the thread events, and the LangSmith ids.
-7. Stops the instances, compares the visual captures, writes `summary.txt`,
-   and builds the report without verdicts.
-8. Prints a result table, the command to run the plan again, and one JSON
-   object on several lines: `runDir`, `report`, `live`, `summary`, `results`
-   (the status of each run), `visual` (the number of captures, the changed
-   ones, and the error when the visual comparison failed), `rerun` and
-   `next`.
-9. Gets the tokens and cost from LangSmith, then builds the report again.
-   LangSmith receives traces with a delay, so this can take up to 90 seconds.
-   You can start to judge while it runs.
-
-| Option | Use |
-|---|---|
-| `--check` | Check the setup and print the plan folder, then stop. |
-| `--estimate` | Print the expected cost and time, then stop. |
-| `--try` | Check new scripts: no master, no live page, no report. See [Check the scripts](#check-the-scripts). |
-| `--only id1,id2` | Run only these scenarios. |
-| `--rerun last` | Run the plan of the last run again, in a new folder. |
-| `--rerun <run-dir>` | Run the plan of this earlier run again, in a new folder. If the plan folder of that run still exists, it uses the current plan and scripts from there. Thus a fixed script runs in its fixed version. |
-| `--concurrency N` | Runs at the same time. Default 4, or `FEATURE_TEST_CONCURRENCY`. |
-| `--branch-url URL` | Use a running instance for all branch runs, not fresh ones. The threads stay on that instance. The scenarios share its data, and the visual check is not reliable. |
-| `--no-build` | Do not build this checkout. The output warns when source files are newer than their build. |
-| `--no-baseline` | Do not run master. There is no visual check. |
-| `--no-cache` | Run master again, also when a kept master run exists. |
-| `--no-fetch` | Do not fetch `origin/master`. Use the merge base with the `origin/master` that git has. |
-| `--no-open` | Do not open the live page in the default browser. Use this when you open it in the editor tab. |
-| `--no-live` | Do not start the live page. Use this only when the user asks for it. |
-| `--keep-instances` | Keep the fresh instances running after the run, for a manual check. The output gives `instancePorts`. Each instance is at `http://localhost:<port>`. Stop them yourself. |
-| `--run-dir DIR` | Use this run folder. |
-
-| Setting | Default |
-|---|---|
-| `FEATURE_TEST_RUNS_DIR` | `/tmp/feature-test` |
-| `FEATURE_TEST_CACHE` | `~/.cache/n8n-feature-test`: the plans, the master worktree, the kept master runs, and the instance data templates |
-| `FEATURE_TEST_CONCURRENCY` | `4` |
-
-### Watch the runs
-
-The live page shows each run as a row, with its status and its last
-screenshot. The screenshot changes every few seconds. When the run is done,
-the same address shows the report. The page stops 30 minutes after its last
-request.
-
-## 4. Judge
-
-Read `<runDir>/summary.txt`. It starts with a triage: each run is `clean`, or
-`look closer` with the reasons (a failed step or check, a browser error, a
-failed tool call, or a visual change). Then it has, for each run: the expected
-result, the steps and their status with the path of each screenshot, the
-prompts, the last reply, the browser errors, and the visual results.
-
-For a clean run, look at the last screenshot and check it against the expected
-result. For a run to look at closer, look at the screenshots of the steps in
-the reasons, and the diff images in `visual/<id>/`.
-
-A script that passed is not always a pass. Check that the run shows the
-expected result, not only that the script ended. Check for these problems:
-
-- a browser error or a failed request that comes from the feature
-- a step that passed, but the screenshot shows the wrong state
-- a visual change that the feature did not intend
-- a script that failed because of the script, not the feature. Say so, and
-  fix the script
-
-Write `<runDir>/verdicts.json`. Use the run names as keys. A master run has
-the name `<id>--master`, and does not need a verdict:
-
-```json
-{
-  "stop-mid-run": {
-    "verdict": "pass",
-    "summary": "One sentence about the result.",
-    "checks": [
-      { "ok": true, "text": "The stop button ends the run" },
-      { "ok": false, "text": "The send button comes back after 9 seconds" },
-      { "ok": null, "text": "Could not confirm the stopped label" }
-    ]
-  }
-}
-```
-
-Use `pass` when all checks pass, `fail` when the main goal is not met, and
-`partial` for all other results. For a visual scenario, judge each change:
-intended by the feature, a regression, or noise (for example, a timestamp
-that is not masked). Be strict and factual.
-
-Write each check as one short sentence with one fact. Put selectors, test
-IDs and UI text in backticks. The page shows them as code. The page shows
-the failed checks first.
-
-## 5. The report
-
-While the live page runs, it builds the report again each time
-`verdicts.json` changes. The open tab loads the new report by itself, and
-keeps the selected run and the scroll position. Thus you do not need to load
-the page again. When the live page does not run, build the report yourself:
-
-```bash
-python3 $S/report.py <runDir>
-```
-
-The report is `<runDir>/report/index.html`. It is one folder with all its
-files, so you can copy or share it. It has, from top to bottom:
-
-- the video replays of all runs
-- a table of all runs: kind, result, steps passed, time, LLM calls, tool calls,
-  tokens, cost and browser errors. The result is your verdict. A master run
-  shows `Baseline`, and a run without a verdict shows `Not judged`, or
-  `Failed` when its script failed
-- the visual check: the branch screen, the master screen, and the diff image
-  of each capture, with the accessibility tree diff
-- for each run: the video of the run, the expected result and the verdict,
-  the steps with their screenshots, the browser errors, and the LLM steps
-  from the trace. A click on a row of the table shows the panel and the
-  video of that run, and pauses the video of the other runs
-
-Each box shows closed. It opens and closes when you click its title. On the
-live page, a box that you open or close stays that way when the page
-updates.
-
-A master run that is kept from an earlier run shows "Reused" in its run box.
-
-Give the user the path and the `live` URL. Tell the user the result of each
-scenario, the visual changes, and the problems that you found.
-
-## Single steps
-
-| Step | Command |
-|---|---|
-| Prepare the master baseline | `python3 $S/baseline.py prepare` |
-| Show the baseline state | `python3 $S/baseline.py status` |
-| Run one script on any instance | `N8N_BASE_URL=<url> node $S/run_scenario.mjs --out <dir> --script <file>` |
-| Compare the visual captures again | `node $S/visual_diff.mjs <runDir>` |
-| Print the judge summary | `python3 $S/report.py <runDir> --summary` |
-| Serve the live page of a run | `python3 $S/live_page.py --run-dir <runDir>` |
-
-## Limits
-
-- LLM answers change from run to run, and a run can fail because of the LLM.
-  Run a failed scenario again with `--rerun last --only <id>` before you call
-  it a bug.
-- A stopped LLM call does not show in LangSmith. The cost of a run with a stop
-  includes only the calls that ended.
-- Each LLM scenario costs money. The tokens and cost show in the report.
-- Parallel runs share the CPU and the LLM rate limits. The times include the
-  shared load.
-- The visual check compares pixels with a small tolerance. Fonts and
-  animations can cause small changes. Masks hide the parts that change between
-  runs. The accessibility tree diff shows changes in structure and text.
-- The worktree shows in `git worktree list`. To remove it, run
-  `git worktree remove ~/.cache/n8n-feature-test/master`.
-- Kept master runs older than 14 days are deleted. The instance data
-  templates in `~/.cache/n8n-feature-test/templates` change when the
-  migrations change. You can delete old templates at any time.
-- A run folder is about 35 MB. The run folders are in `/tmp`, so the system
-  deletes them on reboot.

.agents/skills/testmax-assistant-ui/examples/stop-button/scenarios.json

@@ -1,38 +0,0 @@
-{
-	"title": "Stop button in the Assistant",
-	"feature": "The user can stop a run while the Assistant works. After the stop, the composer is ready and a new message gets a reply. The Assistant screens look the same as on master.",
-	"scenarios": [
-		{
-			"id": "stop-mid-run",
-			"title": "Stop a run in the middle",
-			"kind": "feature",
-			"script": "stop-mid-run.mjs",
-			"expected": "The stop button shows during the run. A click ends the run without an error message. The send button comes back, and a follow-up message gets a reply.",
-			"timeoutMin": 6
-		},
-		{
-			"id": "smoke-chat",
-			"title": "A prompt gets a reply",
-			"kind": "regression",
-			"script": "lib:smoke-chat.mjs",
-			"expected": "A short prompt opens a new thread and gets the reply OK. The composer is ready after the reply.",
-			"timeoutMin": 5
-		},
-		{
-			"id": "assistant-home",
-			"title": "Assistant home and chat history",
-			"kind": "visual",
-			"script": "lib:visual-assistant-home.mjs",
-			"expected": "The home screen and the chat history popover look the same as on master.",
-			"timeoutMin": 3
-		},
-		{
-			"id": "thread-reply",
-			"title": "Thread with one reply",
-			"kind": "visual",
-			"script": "lib:visual-thread-reply.mjs",
-			"expected": "A thread with one short exchange looks the same as on master.",
-			"timeoutMin": 5
-		}
-	]
-}

.agents/skills/testmax-assistant-ui/examples/stop-button/stop-mid-run.mjs

@@ -1,31 +0,0 @@
-// Feature: the user stops a run while the Assistant works, and can send a new message after it.
-export default async function (t) {
-	await t.step('Open the Assistant', () => t.goto());
-	await t.step('Send a prompt that takes a while', () =>
-		t.send(
-			'Explain step by step, in at least 30 numbered points, how to design a workflow that syncs ' +
-				'contacts between two CRMs. Do not build anything.',
-		),
-	);
-	await t.step('The stop button shows while the run is active', () =>
-		t.expect(t.ai.stopButton()).toBeVisible({ timeout: 30_000 }),
-	);
-	await t.step('Click stop', () => t.ai.stopButton().click());
-	await t.check('The stop button goes away', () =>
-		t.expect(t.ai.stopButton()).toBeHidden({ timeout: 15_000 }),
-	);
-	await t.check('The send button comes back', () =>
-		t.expect(t.ai.sendButton()).toBeVisible({ timeout: 15_000 }),
-	);
-	await t.step('Wait until the thread is idle', () => t.waitForIdle({ timeout: 60_000 }));
-	await t.check('The thread shows no error message', async () => {
-		await t.expect(t.ai.panelText(/something went wrong/i)).toHaveCount(0);
-	});
-	await t.step('Send a follow-up message', () =>
-		t.send('Reply with exactly the word OK and nothing else.'),
-	);
-	await t.step('Wait for the follow-up reply', () => t.waitForIdle({ timeout: 120_000 }));
-	await t.check('The follow-up gets a reply', () =>
-		t.expect(t.ai.assistantMessages().last()).toContainText('OK'),
-	);
-}

.agents/skills/testmax-assistant-ui/scenarios/smoke-chat.mjs

@@ -1,17 +0,0 @@
-// Regression: a prompt gets a reply in a new thread.
-export default async function (t) {
-	await t.step('Open the Assistant', () => t.goto());
-	await t.step('Send a short prompt', () =>
-		t.send('Reply with exactly the word OK and nothing else.'),
-	);
-	await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
-	await t.check('The user message shows', () =>
-		t.expect(t.ai.userMessages().first()).toBeVisible(),
-	);
-	await t.check('The reply says OK', () =>
-		t.expect(t.ai.assistantMessages().last()).toContainText('OK'),
-	);
-	await t.check('The composer is ready for the next message', () =>
-		t.expect(t.ai.sendButton()).toBeVisible(),
-	);
-}

.agents/skills/testmax-assistant-ui/scenarios/visual-assistant-home.mjs

@@ -1,11 +0,0 @@
-// Visual: the Assistant home screen, and the chat history popover.
-export default async function (t) {
-	await t.step('Open the Assistant', () => t.goto());
-	await t.capture('assistant-home');
-	await t.step('Open the chat history', async () => {
-		await t.ai.sidebarToggle().click();
-		await t.expect(t.ai.threadList()).toBeVisible();
-	});
-	// Other visual scenarios create threads at the same time, so the list content changes between runs.
-	await t.capture('chat-history', { mask: [t.ai.threadList()] });
-}

.agents/skills/testmax-assistant-ui/scenarios/visual-thread-reply.mjs

@@ -1,11 +0,0 @@
-// Visual: a thread with one short exchange, and the composer after the reply.
-export default async function (t) {
-	await t.step('Open the Assistant', () => t.goto());
-	await t.step('Send a short prompt', () =>
-		t.send('Reply with exactly the word OK and nothing else.'),
-	);
-	await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
-	// The reply text and the thread title come from the LLM. The title also shows in the main sidebar.
-	const title = (await t.ai.sidebarToggle().innerText()).trim();
-	await t.capture('thread-reply', { mask: [t.ai.assistantMessages()], maskText: [title] });
-}

.agents/skills/testmax-assistant-ui/scripts/baseline.py

@@ -1,117 +0,0 @@
-#!/usr/bin/env python3
-"""Keep a built copy of master for the visual check.
-
-  baseline.py status              JSON: the worktree, the built commit, and the commit to build
-  baseline.py prepare [--no-fetch]  fetch origin/master, update the worktree, install, and build
-
-The commit to build is the merge base of HEAD and origin/master: the master commit that this branch
-starts from. Then a visual change comes from the branch, not from newer commits on master. The
-build runs only when the merge base moves, for example after a rebase.
-
-The worktree is in <FEATURE_TEST_CACHE>/master (default ~/.cache/n8n-feature-test/master). It is a
-detached git worktree of this repository, so `git worktree list` shows it. The first build takes 15
-to 20 minutes. The output goes to <FEATURE_TEST_CACHE>/build.log.
-"""
-
-import argparse
-import fcntl
-import json
-import os
-import subprocess
-import sys
-import time
-from datetime import datetime, timezone
-
-from common import CACHE, git
-
-WORKTREE = CACHE / 'master'
-STATE = CACHE / 'built.json'
-BUILD_LOG = CACHE / 'build.log'
-
-
-def built():
-	try:
-		state = json.loads(STATE.read_text())
-	except (OSError, ValueError):
-		return None
-	cli = WORKTREE / 'packages' / 'cli' / 'dist'
-	editor = WORKTREE / 'packages' / 'frontend' / 'editor-ui' / 'dist' / 'index.html'
-	return state if cli.exists() and editor.exists() else None
-
-
-def target(fetch=False):
-	"""The master commit to build: the merge base of HEAD and origin/master."""
-	if fetch:
-		git('fetch', '--quiet', 'origin', 'master')
-	try:
-		return git('merge-base', 'HEAD', 'origin/master')
-	except subprocess.CalledProcessError:
-		return git('rev-parse', 'origin/master')
-
-
-def status():
-	state = built() or {}
-	try:
-		commit = target()
-	except subprocess.CalledProcessError:
-		commit = None
-	return {'worktree': str(WORKTREE), 'exists': WORKTREE.exists(), 'builtCommit': state.get('commit'),
-	        'builtAt': state.get('builtAt'), 'baseCommit': commit,
-	        'upToDate': bool(commit) and state.get('commit') == commit, 'log': str(BUILD_LOG)}
-
-
-def run_logged(cmd, log, env=None):
-	log.write(f'\n$ {" ".join(cmd)}\n')
-	log.flush()
-	code = subprocess.run(cmd, cwd=WORKTREE, stdout=log, stderr=subprocess.STDOUT, env=env).returncode
-	if code != 0:
-		raise RuntimeError(f'{" ".join(cmd)} failed with exit code {code}. See {BUILD_LOG}')
-
-
-def prepare(fetch=True, say=print):
-	"""Return {"root", "commit"} of a built merge base. Builds only when the merge base moved."""
-	CACHE.mkdir(parents=True, exist_ok=True)
-	with (CACHE / 'build.lock').open('w') as lock:
-		# Two runs at the same time must not build into the same worktree.
-		fcntl.flock(lock, fcntl.LOCK_EX)
-		if fetch:
-			say('fetch origin/master')
-		commit = target(fetch)
-		if not (WORKTREE / '.git').exists():
-			say(f'create the master worktree in {WORKTREE}')
-			git('worktree', 'add', '--detach', str(WORKTREE), commit)
-		elif git('rev-parse', 'HEAD', cwd=WORKTREE) != commit:
-			git('checkout', '--detach', '--force', commit, cwd=WORKTREE)
-		state = built()
-		if state and state.get('commit') == commit:
-			say(f'master {commit[:11]} is built')
-			return {'root': WORKTREE, 'commit': commit}
-		started = time.time()
-		say(f'build master {commit[:11]} (the merge base). The first build takes 15 to 20 minutes. Log: {BUILD_LOG}')
-		with BUILD_LOG.open('w') as log:
-			# DOCKER_BUILD skips the git hook install of the prepare script, so the hooks of the
-			# main checkout stay as they are.
-			run_logged(['pnpm', 'install', '--frozen-lockfile'], log, env={**os.environ, 'DOCKER_BUILD': 'true'})
-			run_logged(['pnpm', 'turbo', 'run', 'build', '--filter=n8n...'], log)
-		STATE.write_text(json.dumps({'commit': commit, 'builtAt': datetime.now(timezone.utc).isoformat()}, indent=2))
-		say(f'master {commit[:11]} built in {time.time() - started:.0f}s')
-		return {'root': WORKTREE, 'commit': commit}
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('action', choices=['status', 'prepare'])
-	ap.add_argument('--no-fetch', action='store_true')
-	args = ap.parse_args()
-	if args.action == 'status':
-		print(json.dumps(status(), indent=2))
-		return
-	try:
-		result = prepare(fetch=not args.no_fetch, say=lambda m: print(m, flush=True))
-	except (RuntimeError, subprocess.CalledProcessError) as e:
-		sys.exit(str(e))
-	print(json.dumps({'root': str(result['root']), 'commit': result['commit']}, indent=2))
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-assistant-ui/scripts/common.py

@@ -1,175 +0,0 @@
-"""Shared helpers: paths, settings, and fresh n8n instances for the scenario runs. Standard library only.
-
-The skill reuses the scripts of testmax-n8n-ai (settings, REST client, report parts,
-live server). This module puts them on the import path.
-"""
-
-import fcntl
-import hashlib
-import json
-import os
-import shutil
-import signal
-import socket
-import subprocess
-import sys
-import time
-import urllib.request
-from pathlib import Path
-
-HERE = Path(__file__).resolve().parent
-SHARED = HERE.parents[1] / 'testmax-n8n-ai' / 'scripts'
-sys.path.insert(0, str(SHARED))
-
-from n8n_api import REPO_ROOT, N8nClient, config, setting  # noqa: E402
-
-CACHE = Path(setting('FEATURE_TEST_CACHE', default=str(Path.home() / '.cache' / 'n8n-feature-test'))).expanduser().resolve()
-RUNS_ROOT = Path(setting('FEATURE_TEST_RUNS_DIR', default='/tmp/feature-test')).expanduser().resolve()
-PLANS_ROOT = CACHE / 'plans'
-TEMPLATE_VERSION = '1'
-# These settings belong to the instance that the user runs. A fresh instance must not reuse them.
-INSTANCE_ENV_SKIP = ('N8N_LICENSE_', 'DB_', 'N8N_PORT', 'N8N_USER_FOLDER', 'N8N_EDITOR_BASE_URL', 'WEBHOOK_URL',
-                     'N8N_RUNNERS_BROKER_PORT', 'N8N_HOST', 'N8N_PROTOCOL', 'N8N_ENCRYPTION_KEY', 'N8N_BASE_URL')
-
-
-def free_port():
-	with socket.socket() as s:
-		s.bind(('127.0.0.1', 0))
-		return s.getsockname()[1]
-
-
-def instance_env(port, home, feature_flags=None):
-	"""The settings of the user's instance (.env.local and .env.ai-surfaces), with a new port and data folder.
-
-	Feature flags do not come from PostHog: PostHog gives each new instance its own experiment variants,
-	so the screens of two instances with the same code can differ. The flags are their defaults, plus
-	N8N_FEATURE_FLAG_OVERRIDES, plus feature_flags.
-	"""
-	env = {k: v for k, v in os.environ.items() if not k.startswith(INSTANCE_ENV_SKIP)}
-	env.update({k: v for k, v in config().items() if not k.startswith(INSTANCE_ENV_SKIP)})
-	# Keep N8N_DIAGNOSTICS_ENABLED as the user set it: when it is false, Instance AI sends no LangSmith traces.
-	env.update(
-		N8N_PORT=str(port), N8N_RUNNERS_BROKER_PORT=str(free_port()), N8N_USER_FOLDER=str(home),
-		N8N_EDITOR_BASE_URL=f'http://localhost:{port}', WEBHOOK_URL=f'http://localhost:{port}/',
-		N8N_PERSONALIZATION_ENABLED='false', N8N_INSTANCE_AI_EVAL_INSTANCE='true',
-	)
-	if feature_flags:
-		try:
-			overrides = json.loads(env.get('N8N_FEATURE_FLAG_OVERRIDES') or '{}')
-		except ValueError:
-			overrides = {}
-		env['N8N_FEATURE_FLAG_OVERRIDES'] = json.dumps({**overrides, **feature_flags})
-	return env
-
-
-def template_dir(root):
-	"""The data folder template of a checkout. The key has the database migrations and the owner, so a build
-	with other migrations or another N8N_EVAL_EMAIL gets a new template. Change TEMPLATE_VERSION when seed()
-	changes."""
-	migrations = Path(root) / 'packages' / '@n8n' / 'db' / 'dist' / 'migrations'
-	names = sorted(str(p.relative_to(migrations)) for p in migrations.rglob('*.js')) if migrations.exists() else []
-	owner = client('http://localhost')
-	parts = [TEMPLATE_VERSION, str(Path(root).resolve()), owner.email, owner.password, *names]
-	return CACHE / 'templates' / hashlib.sha256('\n'.join(parts).encode()).hexdigest()[:16]
-
-
-class Instance:
-	"""A fresh n8n instance with its own port and data folder, started from a checkout that is built.
-
-	The first instance of a checkout runs the database migrations and the owner setup, then saves its
-	data folder as a template. Later instances start from a copy of the template, which is faster.
-	"""
-
-	def __init__(self, name, root, home, log, feature_flags=None):
-		self.name, self.root, self.home, self.log_path = name, Path(root), Path(home), Path(log)
-		self.feature_flags = feature_flags
-		self.port = free_port()
-		self.url = f'http://localhost:{self.port}'
-		self.proc = None
-
-	def start(self, timeout=180):
-		template = template_dir(self.root)
-		if (template / '.ready').exists():
-			shutil.copytree(template / '.n8n', self.home / '.n8n', dirs_exist_ok=True)
-			self._boot(timeout)
-			return self
-		template.parent.mkdir(parents=True, exist_ok=True)
-		with (template.parent / f'{template.name}.lock').open('w') as lock:
-			# Only one instance makes the template. The others start without it and do not wait.
-			try:
-				fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
-				owner = True
-			except BlockingIOError:
-				owner = False
-			self._boot(timeout)
-			seed(self.url)
-			if owner and not (template / '.ready').exists():
-				self._save_template(template)
-		return self
-
-	def _save_template(self, template):
-		"""Stop, copy the data folder, and start again on the same port."""
-		self.stop()
-		shutil.rmtree(template, ignore_errors=True)
-		shutil.copytree(self.home / '.n8n', template / '.n8n', ignore=shutil.ignore_patterns('n8nEventLog*', '*.log'))
-		(template / '.ready').touch()
-		self._boot(60)
-
-	def _boot(self, timeout):
-		self.home.mkdir(parents=True, exist_ok=True)
-		log = self.log_path.open('a')
-		# A new session, so stop() can end n8n and its task runners together.
-		self.proc = subprocess.Popen(['node', str(self.root / 'packages' / 'cli' / 'bin' / 'n8n'), 'start'],
-		                             cwd=self.home, env=instance_env(self.port, self.home, self.feature_flags),
-		                             stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
-		deadline = time.time() + timeout
-		while time.time() < deadline:
-			if self.proc.poll() is not None:
-				raise RuntimeError(f'{self.name} instance stopped at start (exit {self.proc.returncode}), see {self.log_path}')
-			try:
-				with urllib.request.urlopen(f'{self.url}/healthz/readiness', timeout=2) as res:
-					if res.status == 200:
-						return
-			except OSError:
-				pass
-			time.sleep(0.25)
-		raise RuntimeError(f'{self.name} instance did not become ready in {timeout}s, see {self.log_path}')
-
-	def stop(self, clean=False):
-		"""Stop n8n. clean also deletes the static file cache (about 80 MB), which n8n writes again at each start."""
-		if self.proc and self.proc.poll() is None:
-			try:
-				os.killpg(self.proc.pid, signal.SIGTERM)
-				self.proc.wait(timeout=30)
-			except (ProcessLookupError, subprocess.TimeoutExpired):
-				try:
-					os.killpg(self.proc.pid, signal.SIGKILL)
-				except ProcessLookupError:
-					pass
-		if clean:
-			shutil.rmtree(self.home / '.cache', ignore_errors=True)
-
-
-def client(base_url):
-	c = N8nClient()
-	c.base_url = base_url.rstrip('/')
-	return c
-
-
-def seed(base_url):
-	"""Create the owner (N8N_EVAL_EMAIL and N8N_EVAL_PASSWORD) and finish the Assistant setup."""
-	c = client(base_url)
-	c.request('POST', '/rest/owner/setup',
-	          {'email': c.email, 'firstName': 'Feature', 'lastName': 'Test', 'password': c.password})
-	c.login()
-	# Web search is optional. Without a choice, the Assistant shows its setup screen instead of the chat.
-	# A search provider from the environment makes the choice, and the API rejects searchDisabled then.
-	current = c.request('GET', '/rest/instance-ai/settings') or {}
-	update = {'enabled': True}
-	if not current.get('searchEnvConfigured'):
-		update['searchDisabled'] = True
-	c.request('PUT', '/rest/instance-ai/settings', update)
-
-
-def git(*args, cwd=REPO_ROOT):
-	return subprocess.run(['git', *args], cwd=cwd, capture_output=True, text=True, check=True).stdout.strip()

.agents/skills/testmax-assistant-ui/scripts/kit.mjs

@@ -1,351 +0,0 @@
-// The helper object `t` that a scenario script receives. See SKILL.md for the full API.
-//
-//   export default async function (t) {
-//     await t.step('Open the Assistant', () => t.goto());
-//     await t.step('Send a prompt', () => t.send('Reply with exactly: OK'));
-//     await t.step('Wait for the reply', () => t.waitForIdle());
-//     await t.check('The reply shows', () => t.expect(t.ai.assistantMessages().first()).toBeVisible());
-//     await t.capture('thread', { mask: [t.ai.assistantMessages()] });
-//   }
-//
-// A step that throws ends the scenario. A check that fails is recorded and the scenario goes on.
-// Each step and check writes a screenshot. The runner shows them in the report.
-import { mkdirSync, renameSync, writeFileSync } from 'node:fs';
-import { join } from 'node:path';
-
-import { expect } from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
-
-const THREAD_URL = /\/assistant\/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/;
-const RUN_GRACE_MS = 10_000;
-
-/**
- * Replace the accessibility subtree of each masked element with one line. Masked texts alone keep
- * the structure, so a list with a different number of rows still shows as a change. Each subtree
- * replaces only its first unmasked match, so an unmasked element with the same content stays.
- */
-function maskSubtrees(tree, subtrees) {
-	const lines = tree.split('\n');
-	for (const subtree of subtrees) {
-		const sub = subtree.split('\n').filter((l) => l.trim());
-		if (!sub.length || sub[0].startsWith('#')) continue;
-		for (let i = 0; i + sub.length <= lines.length; i++) {
-			if (!lines[i].endsWith(sub[0])) continue;
-			const indent = lines[i].slice(0, lines[i].length - sub[0].length);
-			if (indent.trim() || !sub.every((l, j) => lines[i + j] === indent + l)) continue;
-			lines.splice(i, sub.length, `${indent}- text: "[masked]"`);
-			break;
-		}
-	}
-	return lines.join('\n');
-}
-
-/** The accessibility tree without the masked texts and the instance port, so branch and master compare. */
-function maskTree(tree, texts) {
-	let out = tree.replace(/localhost(:|%3A)\d+/g, 'localhost$1PORT');
-	for (const text of [...texts].sort((a, b) => b.length - a.length))
-		out = out.split(text).join('[masked]');
-	return out;
-}
-
-export class StepFailed extends Error {}
-
-/** Locators of the Instance AI page. They mirror pages/InstanceAiPage.ts in the Playwright tests. */
-function locators(page) {
-	const container = () => page.getByTestId('instance-ai-container');
-	return {
-		container,
-		chatInput: () => container().getByRole('textbox').or(container().getByRole('combobox')),
-		composer: () => container().getByTestId('instance-ai-composer'),
-		sendButton: () => container().getByTestId('instance-ai-send-button'),
-		stopButton: () => container().getByTestId('instance-ai-stop-button'),
-		userMessages: () => container().getByTestId('instance-ai-user-message'),
-		assistantMessages: () => container().getByTestId('instance-ai-assistant-message'),
-		assistantText: (text) =>
-			container().getByTestId('instance-ai-assistant-message').getByText(text),
-		panelText: (text) => container().getByText(text),
-		emptyState: () => container().getByTestId('instance-ai-empty-state'),
-		statusBar: () => container().getByTestId('instance-ai-status-bar'),
-		sidebarToggle: () => container().getByTestId('instance-ai-sidebar-toggle'),
-		threadList: () => page.getByTestId('instance-ai-thread-list'),
-		newThreadButton: () =>
-			page
-				.getByTestId('project-instance-ai-menu-item')
-				.getByRole('menuitem', { name: 'Assistant', exact: true }),
-		mentionButton: () => container().getByTestId('instance-ai-mention-button'),
-		mentionMenu: () => page.getByTestId('instance-ai-mention-menu-content'),
-		fileInput: () => container().locator('input[type="file"]'),
-		approveButton: () => container().getByTestId('approval-card-allow-once'),
-		alwaysAllowButton: () => container().getByTestId('approval-card-always-allow'),
-		denyButton: () => container().getByTestId('approval-card-deny'),
-		domainAllowOnce: () => container().getByTestId('domain-access-allow-once'),
-		planApproveButton: () => container().getByTestId('instance-ai-plan-approve'),
-		credentialContinue: () => container().getByTestId('instance-ai-credential-continue-button'),
-		previewPanel: () => container().getByTestId('instance-ai-preview-panel'),
-		previewCanvasNodes: () =>
-			container().getByTestId('workflow-canvas-host').locator('[data-test-id="canvas-node"]'),
-		previewToggle: () => container().getByTestId('instance-ai-artifacts-preview-toggle'),
-		artifactsSidebar: () => container().getByTestId('instance-ai-artifacts-sidebar'),
-		outOfCredits: () => container().getByTestId('instance-ai-out-of-credits'),
-	};
-}
-
-export function createKit({ page, out, baseUrl, autoConfirm, emit }) {
-	const started = Date.now();
-	const steps = [];
-	const threads = [];
-	const prompts = [];
-	const ai = locators(page);
-	let stopped = false;
-	mkdirSync(join(out, 'captures'), { recursive: true });
-
-	const seconds = () => (Date.now() - started) / 1000;
-	function save() {
-		const partial = join(out, 'steps.tmp.json');
-		writeFileSync(partial, JSON.stringify(steps, null, 1));
-		renameSync(partial, join(out, 'steps.json'));
-	}
-	async function shot(file) {
-		try {
-			await page.screenshot({ path: join(out, file) });
-			return file;
-		} catch {
-			return null;
-		}
-	}
-	function message(error) {
-		return String(error?.message ?? error)
-			.replace(/\u001b\[[0-9;]*m/g, '')
-			.slice(0, 2000);
-	}
-
-	async function record(kind, title, fn) {
-		if (stopped) throw new StepFailed('The scenario has stopped.');
-		const entry = { i: steps.length + 1, kind, title, status: 'running', t: seconds() };
-		steps.push(entry);
-		save();
-		emit(`${kind} ${title}`);
-		let value;
-		let failed = false;
-		try {
-			value = typeof fn === 'function' ? await fn() : fn;
-			if (value === false) throw new Error('The check returned false.');
-		} catch (error) {
-			failed = true;
-			entry.error = message(error);
-		}
-		// After a stop, run_scenario.mjs owns the steps. A late result must not change them.
-		if (stopped) throw new StepFailed('The scenario has stopped.');
-		entry.status = failed ? 'failed' : 'passed';
-		if (failed) emit(`${kind} failed: ${entry.error.split('\n')[0]}`);
-		entry.duration = seconds() - entry.t;
-		entry.shot = await shot(`${kind}-${String(entry.i).padStart(2, '0')}.png`);
-		if (!stopped) save();
-		if (failed && kind === 'step') throw new StepFailed(entry.error);
-		return failed ? undefined : value;
-	}
-
-	async function api(method, path, body) {
-		return await page.evaluate(
-			async ({ method, path, body }) => {
-				const res = await fetch(path, {
-					method,
-					headers: {
-						'content-type': 'application/json',
-						'browser-id': localStorage.getItem('n8n-browserId') ?? '',
-					},
-					body: body === undefined ? undefined : JSON.stringify(body),
-				});
-				const text = await res.text();
-				if (!res.ok) throw new Error(`${method} ${path} -> ${res.status}: ${text.slice(0, 300)}`);
-				const json = text ? JSON.parse(text) : null;
-				return json && typeof json === 'object' && Object.keys(json).length === 1 && 'data' in json
-					? json.data
-					: json;
-			},
-			{ method, path, body },
-		);
-	}
-
-	function currentThread() {
-		return page.url().match(THREAD_URL)?.[1] ?? null;
-	}
-
-	function noteThread(id) {
-		if (id && !threads.includes(id)) {
-			threads.push(id);
-			emit(`thread ${id}`);
-		}
-	}
-
-	async function threadStatus(id) {
-		const s = await api('GET', `/rest/instance-ai/threads/${id}/status`);
-		const background = (s?.backgroundTasks ?? []).filter((x) => x.status === 'running').length;
-		return { active: Boolean(s?.hasActiveRun), suspended: Boolean(s?.isSuspended), background };
-	}
-
-	const t = {
-		page,
-		ai,
-		expect,
-		baseUrl,
-		threads,
-		prompts,
-
-		/** A step. When it throws, the scenario ends as failed. */
-		step: (title, fn) => record('step', title, fn),
-		/** A soft check. A failure is recorded and the scenario goes on. fn can return false to fail. */
-		check: (title, fn) => record('check', title, fn),
-		/** A line in the step list, with a screenshot. */
-		note: (title) => record('note', title, () => true),
-
-		/** Open a page of the instance. Default: the Assistant, with the composer ready. */
-		async goto(path = '/assistant') {
-			await page.goto(baseUrl + path, { waitUntil: 'domcontentloaded' });
-			if (path.startsWith('/assistant')) {
-				const enable = page.getByRole('dialog').filter({ hasText: 'Try new n8n Assistant' });
-				if (await enable.isVisible({ timeout: 1500 }).catch(() => false)) {
-					await enable
-						.getByRole('button', { name: /Enable n8n Assistant on this instance/ })
-						.click();
-					await enable.getByRole('button', { name: /^(Continue|Enable)$/ }).click();
-				}
-				await expect(ai.chatInput()).toBeVisible({ timeout: 30_000 });
-			}
-			noteThread(currentThread());
-		},
-
-		/** Type a message in the composer and send it. Returns the thread id. */
-		async send(text) {
-			const before = await ai.assistantMessages().count();
-			await expect(ai.chatInput()).toBeVisible({ timeout: 30_000 });
-			await ai.chatInput().fill(text);
-			await expect(ai.sendButton()).toBeEnabled({ timeout: 30_000 });
-			await ai.sendButton().click();
-			await page.waitForURL(THREAD_URL, { timeout: 30_000 });
-			const id = currentThread();
-			noteThread(id);
-			prompts.push({
-				t: seconds(),
-				text,
-				thread: id,
-				assistantMessagesBefore: before,
-				sentAt: Date.now(),
-			});
-			return id;
-		},
-
-		/**
-		 * Wait until the thread has no active run, no open confirmation, and no running background task.
-		 * Returns 'idle', or 'suspended' when a confirmation card waits for the scenario
-		 * (autoConfirm is off, or the card stays open for 20 seconds).
-		 */
-		async waitForIdle({ timeout = 600_000, settle = 3000 } = {}) {
-			const id = currentThread() ?? threads.at(-1);
-			if (!id) throw new Error('No thread is open. Call t.send() first.');
-			const last = prompts.filter((p) => p.thread === id).at(-1);
-			const deadline = Date.now() + timeout;
-			let seenRun = false;
-			let idleSince = null;
-			let suspendedSince = null;
-			while (!stopped && Date.now() < deadline) {
-				let s;
-				try {
-					s = await threadStatus(id);
-				} catch {
-					await page.waitForTimeout(1000);
-					continue;
-				}
-				// The status can show no run for a moment after the message is sent. A run that the user
-				// stops at once can end before it shows, so after a grace period "no run" means "done".
-				if (!seenRun && Date.now() - (last?.sentAt ?? 0) > RUN_GRACE_MS) {
-					seenRun = true;
-				}
-				if (!seenRun) {
-					const replies = await ai
-						.assistantMessages()
-						.count()
-						.catch(() => 0);
-					seenRun =
-						s.active ||
-						s.suspended ||
-						s.background > 0 ||
-						replies > (last?.assistantMessagesBefore ?? 0);
-				}
-				if (s.suspended) {
-					suspendedSince ??= Date.now();
-					if (!autoConfirm || Date.now() - suspendedSince > 20_000) return 'suspended';
-				} else {
-					suspendedSince = null;
-				}
-				const idle = seenRun && !s.active && !s.suspended && s.background === 0;
-				if (!idle) {
-					idleSince = null;
-				} else {
-					idleSince ??= Date.now();
-					if (Date.now() - idleSince >= settle) {
-						await ai
-							.sendButton()
-							.waitFor({ state: 'visible', timeout: 10_000 })
-							.catch(() => {});
-						return 'idle';
-					}
-				}
-				await page.waitForTimeout(1000);
-			}
-			throw new Error(`The thread did not go idle in ${Math.round(timeout / 1000)}s.`);
-		},
-
-		/**
-		 * Save a screen for the visual check. Visual scenarios run on this branch and on master, and the
-		 * report compares the screens with the same name. mask hides changing parts, such as LLM text.
-		 * root is the part of the page for the accessibility tree diff (default: the whole page).
-		 */
-		async capture(name, { mask = [], maskText = [], root, fullPage = false } = {}) {
-			if (!/^[a-z0-9][a-z0-9-]*$/.test(name))
-				throw new Error(`Capture name "${name}": use a-z, 0-9 and "-".`);
-			return await record('capture', `Capture ${name}`, async () => {
-				await page.waitForLoadState('networkidle', { timeout: 5000 }).catch(() => {});
-				await page.waitForTimeout(500);
-				const texts = new Set(maskText.filter(Boolean));
-				for (const locator of mask) {
-					for (const inner of await locator.allInnerTexts().catch(() => [])) {
-						for (const line of inner.split('\n'))
-							if (line.trim().length > 1) texts.add(line.trim());
-					}
-				}
-				const textMasks = maskText.filter(Boolean).map((s) => page.getByText(s, { exact: true }));
-				await page.screenshot({
-					path: join(out, 'captures', `${name}.png`),
-					mask: [...mask, ...textMasks],
-					maskColor: '#FF00FF',
-					animations: 'disabled',
-					caret: 'hide',
-					fullPage,
-				});
-				const tree = await (root ?? page.locator('body'))
-					.ariaSnapshot({ timeout: 10_000 })
-					.catch((e) => `# ${message(e)}`);
-				const subtrees = [];
-				for (const locator of mask) {
-					for (const element of await locator.all().catch(() => [])) {
-						subtrees.push(await element.ariaSnapshot({ timeout: 5_000 }).catch(() => ''));
-					}
-				}
-				writeFileSync(
-					join(out, 'captures', `${name}.aria.yml`),
-					maskTree(maskSubtrees(tree, subtrees), texts) + '\n',
-				);
-			});
-		},
-
-		/** A REST call as the signed-in user. Returns the `data` field of the answer when it has one. */
-		api,
-		threadStatus,
-		sleep: (ms) => page.waitForTimeout(ms),
-	};
-	/** End the scenario: later steps throw, and a running step does not change the steps. */
-	const stop = () => {
-		stopped = true;
-	};
-	return { t, steps, threads, prompts, stop };
-}

.agents/skills/testmax-assistant-ui/scripts/live_page.py

@@ -1,55 +0,0 @@
-#!/usr/bin/env python3
-"""Serve the live page of a feature test run, then its report.
-
-  live_page.py --run-dir <run-dir> [--parent-pid <pid>] [--port <port>]
-
-run.py starts it in the background. It uses the server of testmax-n8n-ai
-(live_server.py): 127.0.0.1 only, the address in <run-dir>/live-url.txt, and it stops
-30 minutes after the last request when the run is done. When verdicts.json changes after the
-run, the page builds the report again, and the open report reloads.
-"""
-
-import argparse
-import threading
-import time
-from pathlib import Path
-
-import common  # noqa: F401  (puts the shared scripts on the import path)
-import report
-from live_server import read_status, serve
-from report import ScenarioApp
-
-
-def rebuild_on_verdicts(run_dir):
-	verdicts, built = run_dir / 'verdicts.json', None
-	while True:
-		time.sleep(2)
-		if read_status(run_dir).get('phase') != 'done' or not verdicts.exists():
-			continue
-		mtime = verdicts.stat().st_mtime
-		if mtime == built:
-			continue
-		try:
-			report.build(run_dir)
-			built = mtime
-		except ValueError:
-			# The judge is still writing the file. The next check reads it again.
-			pass
-		except Exception as e:
-			print(f'cannot build the report: {e}', flush=True)
-			built = mtime
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--run-dir', required=True)
-	ap.add_argument('--parent-pid', type=int)
-	ap.add_argument('--port', type=int, default=0)
-	args = ap.parse_args()
-	run_dir = Path(args.run_dir).resolve()
-	threading.Thread(target=rebuild_on_verdicts, args=(run_dir,), daemon=True).start()
-	serve(ScenarioApp(run_dir), args.parent_pid, args.port)
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-assistant-ui/scripts/report.py

@@ -1,690 +0,0 @@
-#!/usr/bin/env python3
-"""Build the report of a feature test run, and the live page while it runs.
-
-  report.py <run-dir>          build <run-dir>/report/index.html (run it again after verdicts.json)
-  report.py <run-dir> --summary  print the summary for the judge
-
-Inputs in <run-dir>:
-  plan/scenarios.json    the plan (run.py copies it)
-  status.json            the status of each run (run.py writes it)
-  runs/<name>/           the files of one run (run_scenario.mjs and run.py write them)
-  visual.json            the visual check (visual_diff.mjs and run.py write it)
-  verdicts.json          optional: {"<name>": {"verdict": "pass|partial|fail", "summary": "...",
-                         "checks": [{"ok": true|false|null, "text": "..."}]}}
-
-The page uses the template and the LLM steps view of testmax-n8n-ai.
-"""
-
-import fcntl
-import json
-import shutil
-import sys
-import time
-from datetime import datetime
-from pathlib import Path
-
-import common  # noqa: F401  (puts the shared scripts on the import path)
-import build_report as br
-import live_server as ls
-
-STATUS = {'waiting': ('st-waiting', 'Waiting'), 'starting': ('st-waiting', 'Starting'), 'running': ('st-running', 'Running'),
-          'passed': ('st-done', 'Passed'), 'failed': ('st-failed', 'Failed'), 'error': ('st-failed', 'Error'),
-          'skipped': ('st-skipped', 'Skipped')}
-FINISHED = ('passed', 'failed', 'error', 'skipped')
-KIND_LABELS = {'feature': 'Feature', 'regression': 'Regression', 'visual': 'Visual'}
-VISUAL = {'changed': ('vis-changed', 'Changed'), 'unchanged': ('vis-same', 'Unchanged'),
-          'branch-only': ('vis-only', 'Only on branch'), 'master-only': ('vis-only', 'Only on master')}
-THEAD = ('<tr><th>Scenario</th><th>Kind</th><th>Result</th><th>Steps</th><th>Time</th><th>LLM calls</th>'
-         '<th>Tool calls</th><th>Input tokens</th><th>Output tokens</th><th>Cost</th><th>Browser errors</th></tr>')
-
-CSS = """<style>
-.run-video video{display:block;width:100%;max-height:70vh;aspect-ratio:16/10;object-fit:contain;background:var(--black);border-radius:var(--r8)}
-.badge.vis-changed{background:var(--pink-70);color:var(--bg)}
-.badge.vis-same{background:var(--bg-3);color:var(--black)}
-.badge.vis-only{box-shadow:inset 0 0 0 1px var(--line-strong);color:var(--dim)}
-.facts{display:grid;grid-template-columns:max-content minmax(0,1fr);gap:var(--s4) var(--s16);margin:0}
-.facts dt{font:10px/18px var(--mono);text-transform:uppercase;letter-spacing:.02em;color:var(--dim)}
-.facts dd{margin:0;word-break:break-word}
-.run-error{margin:var(--s12) 0 0;background:var(--pink-10);color:var(--pink-70);border-radius:var(--r8);padding:var(--s12);white-space:pre-wrap;word-break:break-word;font:12px/16px var(--mono);max-height:240px;overflow:auto}
-.tsteps{list-style:none;margin:0;padding:0;display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--s16)}
-@media (max-width:1000px){.tsteps{grid-template-columns:repeat(2,minmax(0,1fr))}}
-@media (max-width:640px){.tsteps{grid-template-columns:minmax(0,1fr)}}
-.tstep{display:flex;flex-direction:column;min-width:0;background:var(--white);border:1px solid var(--line);border-radius:var(--r8);overflow:hidden}
-.tstep.failed{background:var(--pink-10);border-color:var(--pink-70);box-shadow:0 0 0 1px var(--pink-70)}
-.tstep-shot{display:block;width:100%;aspect-ratio:16/10;padding:0;border:0;border-bottom:1px solid var(--line);background:var(--bg-3);cursor:zoom-in;overflow:hidden}
-.tstep-shot img{display:block;width:100%;height:100%;object-fit:cover;object-position:top center;transition:transform .15s}
-.tstep-shot:hover img{transform:scale(1.03)}
-.tstep-shot:focus-visible{outline:2px solid var(--pink);outline-offset:-2px}
-@media (prefers-reduced-motion:reduce){.tstep-shot img{transition:none}}
-.tstep-noshot{display:grid;place-items:center;aspect-ratio:16/10;background:var(--bg-2);border-bottom:1px solid var(--line);color:var(--gray-50);font:10px/14px var(--mono);text-transform:uppercase;letter-spacing:.02em}
-.tstep-info{display:flex;gap:var(--s12);align-items:flex-start;padding:var(--s12);min-width:0}
-.tstep-info > div{flex:1;min-width:0;display:flex;flex-direction:column;gap:var(--s4)}
-.tstep-info.running .num{background:var(--pink-10);color:var(--pink-70)}
-.tstep-info.passed .num{background:var(--pink)}
-.tstep-info.failed .num{background:var(--pink-70);color:var(--bg)}
-.tstep-title{font-weight:500;padding-top:3px;word-break:break-word}
-.tstep-title .mono-label{font-weight:400;margin-right:var(--s8)}
-.tstep-info pre{margin:var(--s4) 0 0;white-space:pre-wrap;word-break:break-word;font:12px/16px var(--mono);color:var(--pink-70);max-height:200px;overflow:auto}
-.shot-viewer .theater-bar{flex-wrap:nowrap;align-items:flex-start}
-.shot-viewer-info{flex:1;min-width:0;padding-top:3px}
-.shot-viewer-actions{flex:none;display:flex;align-items:center;gap:var(--s16)}
-.shot-viewer-info .tstep-info{padding:0}
-.shot-viewer-info .tstep-title{font-size:18px;line-height:22px}
-.shot-viewer-img{flex:1;min-height:0;display:flex;align-items:center;justify-content:center;cursor:zoom-out}
-.shot-viewer-img img{display:block;max-width:100%;max-height:100%;border-radius:var(--r8);box-shadow:0 0 0 1px var(--line);cursor:default}
-.err-list{margin:0;padding-left:var(--s16);font:12px/16px var(--mono)}
-.err-list li{margin-bottom:var(--s4);word-break:break-word}
-.vis{background:var(--white);border-radius:var(--r8);padding:var(--s12);box-shadow:inset 0 0 0 1px var(--line);margin-top:var(--s12)}
-.vis.changed{box-shadow:inset 4px 0 0 var(--pink-70),inset 0 0 0 1px var(--line)}
-.vis-grid{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--s12)}
-.vis-grid figure{margin:0}
-.vis-grid figcaption{margin-bottom:var(--s4)}
-.vis-grid img{display:block;width:100%;height:auto;border-radius:var(--r4);box-shadow:0 0 0 1px var(--line)}
-.vis details summary{cursor:pointer;color:var(--dim);margin-top:var(--s8);font:10px/14px var(--mono);text-transform:uppercase;letter-spacing:.02em}
-.vis pre{background:var(--bg);border-radius:var(--r4);padding:var(--s12);margin:var(--s8) 0 0;overflow:auto;max-height:320px;font-size:12px;line-height:16px;white-space:pre}
-</style>"""
-
-
-def read_json(path, default):
-	try:
-		return json.loads(path.read_text())
-	except (OSError, ValueError):
-		return default
-
-
-def seconds(value):
-	return f'{value:.0f}s'
-
-
-def load_trace(sub, result):
-	"""LLM steps from the thread events, and per-call tokens from usage.json when LangSmith had them."""
-	data = ls.load_instance_ai_live(sub)
-	events_file = sub / 'events.jsonl'
-	rows = br.read_jsonl(events_file) if events_file.exists() else []
-	starts = [r['ts'] / 1000 for r in rows if r.get('type') == 'run-start' and r.get('ts')]
-	usage = read_json(sub / 'usage.json', None)
-	if usage and starts:
-		start = starts[0]
-		data['calls'] = sorted((
-			{'t': br.ts(u['start']) - start, 'start': br.ts(u['start']) - start,
-			 'end': br.ts(u['end']) - start if u.get('end') else None,
-			 'name': (u.get('name') or '').removeprefix('llm: ') or None,
-			 'prompt': u['in'], 'cache_read': u['cache_read'], 'cache_write': u['cache_write'],
-			 'out': u['out'], 'cost': u['cost']} for u in usage), key=lambda c: c['t'])
-		data['model'] = ', '.join(sorted({u['model'] for u in usage if u.get('model')})) or None
-		data['cost'] = f'${sum(c["cost"] for c in data["calls"]):.2f}'
-		data['costSource'] = 'langsmith'
-	if result.get('startedAt') and result.get('finishedAt'):
-		data['duration'] = seconds(br.ts(result['finishedAt']) - br.ts(result['startedAt']))
-	return data
-
-
-def load(run_dir):
-	"""The plan, the status, and one dict for each run in plan order."""
-	plan = read_json(run_dir / 'plan' / 'scenarios.json', {'scenarios': []})
-	status = ls.read_status(run_dir)
-	entries = status.get('runs') or {}
-	runs = []
-	for s in plan['scenarios']:
-		for name in (s['id'], f'{s["id"]}--master'):
-			if name not in entries:
-				continue
-			entry = entries[name]
-			sub = run_dir / 'runs' / name
-			result = read_json(sub / 'result.json', {})
-			data = load_trace(sub, result)
-			if entry.get('status') in ('waiting', 'running'):
-				data['cost'] = '-'
-				if entry.get('startedAt'):
-					data['duration'] = seconds((entry.get('finishedAt') or time.time()) - entry['startedAt'])
-			runs.append({
-				'name': name, 'scenario': s, 'entry': entry, 'master': name.endswith('--master'),
-				'steps': read_json(sub / 'steps.json', []), 'result': result,
-				'console': read_json(sub / 'console.json', {}), 'meta': read_json(sub / 'run.json', {}),
-				'data': data,
-			})
-	return plan, status, runs
-
-
-def status_badge(state):
-	cls, label = STATUS.get(state, ('st-waiting', state))
-	return f'<span class="badge {cls}">{br.esc(label)}</span>'
-
-
-def run_title(run):
-	return run['scenario']['title'] + (' (master)' if run['master'] else '')
-
-
-def check_count(steps):
-	checked = [s for s in steps if s['kind'] in ('step', 'check', 'capture')]
-	return f'{sum(s["status"] == "passed" for s in checked)}/{len(checked)}' if checked else '-'
-
-
-def row_cells(run, result_html):
-	data, result = run['data'], run['result']
-	calls, tools = data['calls'], [e for e in data['events'] if e['kind'] == 'tool']
-	errors = sum(e['error'] for e in tools)
-	browser = (result.get('consoleErrors') or 0) + (result.get('requestErrors') or 0)
-	cells = [
-		f'<span class="run-name">{br.esc(run_title(run))}</span>',
-		KIND_LABELS.get(run['scenario']['kind'], run['scenario']['kind']),
-		result_html, check_count(run['steps']), data['duration'], str(len(calls)) if calls else '-',
-		f'{len(tools)}' + (f' <span class="err">({errors} err)</span>' if errors else ''),
-		br.kilo(sum(c['prompt'] for c in calls)) if calls else '-',
-		br.kilo(sum(c['out'] for c in calls)) if calls else '-',
-		f'<strong>{br.esc(data["cost"])}</strong>',
-		f'<span class="err">{browser}</span>' if browser else '0',
-	]
-	return ''.join(f'<td>{c}</td>' for c in cells)
-
-
-def result_html(run, verdict):
-	"""The judge's verdict. The script status shows only while it runs or when it fails, because the Steps
-	column already counts the steps that passed."""
-	if verdict and verdict.get('verdict'):
-		return br.badge(verdict['verdict'])
-	state = run['entry'].get('status', 'waiting')
-	if state != 'passed':
-		return status_badge(state)
-	if run['master']:
-		return '<span class="badge st-done">Baseline</span>'
-	return br.badge(None)
-
-
-def render_video(screen):
-	"""The video of the run, above the expected outcome. Only the built report has it."""
-	if not screen:
-		return ''
-	poster = f' poster="{br.esc(screen["poster"])}"' if screen.get('poster') else ''
-	return br.fold('Video', f'<span class="mono-label">n8n UI · {br.esc(screen["duration"])}</span>',
-	               f'<div class="run-video"><video controls muted playsinline preload="metadata" '
-	               f'src="{br.esc(screen["src"])}"{poster}></video></div>', cls='brief-card prompt')
-
-
-def render_expected(run, verdict):
-	head = br.verdict_head(verdict) if verdict else ''
-	body = f'<div class="msg">{br.esc(run["scenario"]["expected"])}</div>'
-	if verdict:
-		body += f'<div class="verdict-body">{br.render_checks(verdict)}</div>'
-	elif run['entry'].get('status') in FINISHED:
-		body += '<p class="dim">Not judged yet.</p>'
-	return br.fold('Expected outcome', head, body)
-
-
-def render_run_box(run):
-	entry, result, meta = run['entry'], run['result'], run['meta']
-	facts = [('Instance', f'{"master" if run["master"] else "branch"} · {entry.get("baseUrl") or "-"}')]
-	if entry.get('startedAt'):
-		facts.append(('Started', datetime.fromtimestamp(entry['startedAt']).strftime('%H:%M:%S')))
-	if result.get('threads'):
-		facts.append(('Threads', ', '.join(result['threads'])))
-	if meta.get('cached'):
-		facts.append(('Reused', 'This master run is from an earlier run of the same master commit and script.'))
-	if meta.get('decisions'):
-		facts.append(('Confirmations', f'{len(meta["decisions"])} answered by the auto-approve policy'))
-	if result:
-		facts.append(('Browser errors', f'{result.get("consoleErrors", 0)} console, {result.get("requestErrors", 0)} requests'))
-	dl = ''.join(f'<dt>{br.esc(k)}</dt><dd>{br.esc(v)}</dd>' for k, v in facts)
-	error = result.get('error') or (entry.get('reason') if entry.get('status') in ('error', 'skipped') else None)
-	error_html = f'<pre class="run-error">{br.esc(error)}</pre>' if error else ''
-	return br.fold('Run', status_badge(entry.get('status', 'waiting')), f'<dl class="facts">{dl}</dl>{error_html}')
-
-
-def render_scenario_steps(steps, files):
-	if not steps:
-		return br.fold('Scenario steps', '', '<p class="dim">No steps yet.</p>')
-	items = []
-	for s in steps:
-		status, title = br.esc(s['status']), br.esc(s['title'])
-		duration = f' · took {s["duration"]:.1f}s' if s.get('duration') is not None else ''
-		if s.get('shot'):
-			src = f'{files}{br.esc(s["shot"])}'
-			thumb = (f'<button type="button" class="tstep-shot" data-src="{src}" aria-label="Show step {s["i"]} '
-			         f'full screen"><img src="{src}" loading="lazy" alt="Screen after: {title}"></button>')
-		else:
-			thumb = '<div class="tstep-noshot">No screenshot</div>'
-		error = f'<pre>{br.esc(s["error"])}</pre>' if s.get('error') else ''
-		items.append(
-			f'<li class="tstep {status}">{thumb}<div class="tstep-info {status}"><span class="num">{s["i"]}</span><div>'
-			f'<div class="tstep-title"><span class="mono-label">{br.esc(s["kind"])}</span><span>{title}</span></div>'
-			f'<div class="mono-label">at {s["t"]:.1f}s{duration}</div>{error}</div></div></li>')
-	passed = sum(s['status'] == 'passed' for s in steps)
-	return br.fold('Scenario steps', f'<span class="mono-label">{passed} of {len(steps)} passed</span>',
-	               f'<ol class="tsteps">{"".join(items)}</ol>')
-
-
-def render_errors(console):
-	console_errors, request_errors = console.get('consoleErrors') or [], console.get('requestErrors') or []
-	if not console_errors and not request_errors:
-		return ''
-	items = [f'<li>{br.esc(e.get("text", ""))}</li>' for e in console_errors[:50]]
-	items += [f'<li>{e.get("status") or "failed"} {br.esc(e.get("method", ""))} {br.esc(e.get("url", ""))}'
-	          f'{" · " + br.esc(e["error"]) if e.get("error") else ""}</li>' for e in request_errors[:50]]
-	return br.fold('Browser errors', f'<span class="mono-label">{len(console_errors)} console · '
-	               f'{len(request_errors)} requests</span>', f'<ul class="err-list">{"".join(items)}</ul>')
-
-
-def render_panel(run, verdict, files, scale, screen=None):
-	data = run['data']
-	prompt = (run['result'].get('prompts') or [{}])[0].get('text', '')
-	load_error = f'<div class="box err">Could not read the trace: {br.esc(data["loadError"])}</div>' if data.get('loadError') else ''
-	return (f'{load_error}{render_video(screen)}'
-	        f'{render_expected(run, verdict)}{render_run_box(run)}'
-	        f'{render_scenario_steps(run["steps"], files)}'
-	        f'{br.render_steps(data["calls"], data["events"], *scale, prompt)}'
-	        f'{render_errors(run["console"])}')
-
-
-def render_case(plan, status):
-	branch, master = status.get('branch') or {}, status.get('master')
-	kinds = [s['kind'] for s in plan['scenarios']]
-	lines = [f'Branch {branch.get("commit") or "?"}' + (' with local changes' if branch.get('dirty') else '')
-	         + (f' on {branch["url"]}' if branch.get('url') else ' on a fresh instance')]
-	if master is not None:
-		lines.append(f'Master {(master.get("commit") or "(building)")[:11]} (the merge base) on a fresh instance')
-	lines.append(f'{len(kinds)} scenarios: ' + ', '.join(f'{kinds.count(k)} {k}' for k in KIND_LABELS if kinds.count(k)))
-	lines.append(f'{status.get("concurrency", 1)} at a time')
-	return (f'<div class="case"><div class="box"><div class="mono-label">Feature</div><div class="msg">'
-	        f'{br.esc(plan.get("feature") or "-")}</div></div><div class="box"><div class="mono-label">Run</div>'
-	        f'<div class="msg">{br.esc(chr(10).join(lines))}</div></div></div>')
-
-
-def notes(status, runs, verdicts):
-	parts = []
-	if not verdicts and status.get('phase') == 'done':
-		parts.append('The runs are not judged yet. This page reloads when the verdicts and the token usage are ready.')
-	parts.append('Times are seconds from the start of each scenario. Several scenarios ran at the same time on one '
-	             'machine, so their times include the shared load. Tokens and cost come from LangSmith, matched by the '
-	             'traces of the threads of each scenario. An asterisk on a time marks text placed at the previous tool call.')
-	if any(r['master'] for r in runs):
-		parts.append('Visual scenarios ran on a fresh branch instance and on a fresh master instance, so both start '
-		             'from the same empty state. Master is the merge base of the branch.')
-	if status.get('stale'):
-		parts.append('Warning: these packages had source changes newer than their build, so the branch instance ran '
-		             f'an old build: {", ".join(status["stale"])}.')
-	return br.esc(' '.join(parts))
-
-
-def render_visual(results, runs, files):
-	if not results:
-		return ''
-	titles = {r['scenario']['id']: r['scenario']['title'] for r in runs}
-	cards = []
-	for r in results:
-		cls, label = VISUAL.get(r['status'], ('vis-only', r['status']))
-		ratio = f'{100 * r["ratio"]:.2f}% of pixels' if r.get('ratio') is not None else ''
-		sizes = (f' · size {r["masterSize"][0]}×{r["masterSize"][1]} → {r["branchSize"][0]}×{r["branchSize"][1]}'
-		         if r.get('branchSize') and r['branchSize'] != r.get('masterSize') else '')
-		figures = ''
-		for caption, key in (('Master', 'master'), ('Branch', 'branch'), ('Difference', 'diff')):
-			src = files(r, key)
-			figures += (f'<figure><figcaption class="mono-label">{caption}</figcaption>'
-			            + (f'<a href="{src}" target="_blank"><img src="{src}" loading="lazy" alt="{caption}: {br.esc(r["name"])}"></a>'
-			               if src else '<p class="dim">No screen.</p>') + '</figure>')
-		tree = (f'<details><summary>Accessibility tree diff</summary><pre>{br.esc(r["ariaDiff"])}</pre></details>'
-		        if r.get('ariaDiff') else '')
-		cards.append(
-			f'<div class="vis {"changed" if r["status"] != "unchanged" else ""}"><div class="head">'
-			f'<h3>{br.esc(titles.get(r["scenario"], r["scenario"]))} › {br.esc(r["name"])}</h3>'
-			f'<span class="mono-label">{ratio}{sizes}</span><span class="badge {cls}">{label}</span></div>'
-			f'<div class="vis-grid">{figures}</div>{tree}</div>')
-	changed = sum(r['status'] != 'unchanged' for r in results)
-	return br.fold('Visual check against master',
-	               f'<span class="mono-label">{len(results)} captures · {changed} changed</span>',
-	               '<p class="fineprint">Pink pixels in the difference changed by more than a small color threshold. '
-	               'Magenta boxes are masked parts, such as LLM text. The accessibility tree diff shows changes in '
-	               'structure, roles and labels.</p>' + ''.join(cards))
-
-
-STEP_VIEWER_JS = """
-// A step screenshot opens full screen. The listeners are delegated, because the live page replaces the panels.
-(()=>{
-  const viewer = document.createElement('div');
-  viewer.className = 'theater shot-viewer';
-  viewer.hidden = true;
-  viewer.setAttribute('role', 'dialog');
-  viewer.setAttribute('aria-modal', 'true');
-  viewer.setAttribute('aria-label', 'Step screenshot');
-  viewer.innerHTML = '<div class="theater-bar"><div class="shot-viewer-info"></div><div class="shot-viewer-actions">'
-    + '<span class="mono-label" data-shot-count></span>'
-    + '<button type="button" class="btn" data-shot-move="-1">Previous</button>'
-    + '<button type="button" class="btn" data-shot-move="1">Next</button>'
-    + '<button type="button" class="btn accent" data-shot-close>Close</button></div></div>'
-    + '<div class="shot-viewer-img"><img alt=""></div>'
-    + '<p class="fineprint">Press the left or right arrow key to change the step. Press Esc to close.</p>';
-  document.body.append(viewer);
-  let run = null, index = 0, opener = null;
-  const shots = () => [...(document.getElementById(run)?.querySelectorAll('.tstep-shot') || [])];
-  function render(){
-    const items = shots();
-    if (!items.length) return close();
-    index = Math.min(index, items.length - 1);
-    const button = items[index], img = viewer.querySelector('.shot-viewer-img img');
-    img.src = button.dataset.src;
-    img.alt = button.querySelector('img').alt;
-    viewer.querySelector('.shot-viewer-info').replaceChildren(button.closest('.tstep').querySelector('.tstep-info').cloneNode(true));
-    viewer.querySelector('[data-shot-count]').textContent = `${index + 1} of ${items.length}`;
-  }
-  function open(button){
-    run = button.closest('.run').id;
-    index = Math.max(0, shots().indexOf(button));
-    opener = button;
-    viewer.hidden = false;
-    document.body.classList.add('theater-open');
-    render();
-    viewer.querySelector('[data-shot-close]').focus();
-  }
-  function close(){
-    viewer.hidden = true;
-    run = null;
-    document.body.classList.remove('theater-open');
-    if (opener?.isConnected) opener.focus();
-  }
-  function move(step){
-    const count = shots().length;
-    if (!count) return close();
-    index = (index + step + count) % count;
-    render();
-  }
-  document.addEventListener('click', e=>{
-    const button = e.target.closest('.tstep-shot');
-    if (button) open(button);
-  });
-  viewer.addEventListener('click', e=>{
-    const step = e.target.closest('[data-shot-move]');
-    if (step) move(Number(step.dataset.shotMove));
-    else if (e.target.closest('[data-shot-close]') || e.target.classList.contains('shot-viewer-img')) close();
-  });
-  document.addEventListener('keydown', e=>{
-    if (viewer.hidden) return;
-    if (e.key === 'Escape') close();
-    else if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') { e.preventDefault(); move(e.key === 'ArrowRight' ? 1 : -1); }
-  });
-})();
-"""
-
-VIDEO_JS = """
-// The video of a run plays only while its panel shows.
-const showPanel = show;
-show = (id)=>{
-  showPanel(id);
-  document.querySelectorAll('.run:not(.active) .run-video video').forEach(v=>v.pause());
-};
-"""
-
-RELOAD_JS = """
-// Served by the live page: reload when the report is built again, for example after the verdicts.
-(()=>{
-  if (!location.protocol.startsWith('http')) return;
-  const saved = JSON.parse(sessionStorage.getItem('report-view') || 'null');
-  if (saved) {
-    sessionStorage.removeItem('report-view');
-    if (document.getElementById(saved.run)) show(saved.run);
-    window.scrollTo(0, saved.y);
-  }
-  setInterval(async ()=>{
-    try {
-      const version = (await (await fetch('version.txt', {cache: 'no-store'})).text()).trim();
-      if (version && version !== REPORT_VERSION) {
-        const run = document.querySelector('.run.active')?.id;
-        sessionStorage.setItem('report-view', JSON.stringify({run, y: window.scrollY}));
-        location.reload();
-      }
-    } catch {}
-  }, 3000);
-})();
-"""
-
-
-def build(run_dir):
-	"""Write report/index.html with its images and videos. Returns its path. Two builds of one run can
-	happen at the same time (run.py, report.py, the live page), so they take turns."""
-	run_dir.mkdir(parents=True, exist_ok=True)
-	with (run_dir / 'report.lock').open('w') as lock:
-		fcntl.flock(lock, fcntl.LOCK_EX)
-		return _build(run_dir)
-
-
-def _build(run_dir):
-	plan, status, runs = load(run_dir)
-	verdicts = br.load_verdicts(run_dir)
-	report = run_dir / 'report'
-	for sub in ('runs', 'screens', 'visual'):
-		shutil.rmtree(report / sub, ignore_errors=True)
-	report.mkdir(parents=True, exist_ok=True)
-	for sub in ('fonts', 'brand'):
-		if (br.ASSET_CACHE / sub).exists():
-			shutil.copytree(br.ASSET_CACHE / sub, report / sub, dirs_exist_ok=True)
-
-	scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
-	rows, panels, screens = [], [], []
-	for i, run in enumerate(runs):
-		name, sub = run['name'], run_dir / 'runs' / run['name']
-		shots = report / 'runs' / name
-		for png in sub.glob('*.png'):
-			if png.name not in ('live.png', 'screen-poster.png'):
-				shots.mkdir(parents=True, exist_ok=True)
-				shutil.copy(png, shots / png.name)
-		verdict = verdicts.get(name)
-		screen = None
-		if (sub / 'screen.webm').exists():
-			(report / 'screens').mkdir(exist_ok=True)
-			shutil.copy(sub / 'screen.webm', report / 'screens' / f'{name}.webm')
-			screen = {'name': name, 'title': run_title(run), 'duration': run['data']['duration'], 'type': 'video',
-			          'src': f'screens/{name}.webm', 'verdict': (verdict or {}).get('verdict'),
-			          'label': KIND_LABELS.get(run['scenario']['kind'], '')}
-			if (sub / 'screen-poster.png').exists():
-				shutil.copy(sub / 'screen-poster.png', report / 'screens' / f'{name}-poster.png')
-				screen['poster'] = f'screens/{name}-poster.png'
-			screens.append(screen)
-		active = ' active' if i == 0 else ''
-		rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">'
-		            + row_cells(run, result_html(run, verdict)) + '</tr>')
-		panels.append(f'<section class="run{active}" id="{name}">'
-		              + render_panel(run, verdict, f'runs/{name}/', scale, screen) + '</section>')
-
-	results = read_json(run_dir / 'visual.json', [])
-	for r in results:
-		target = report / 'visual' / r['scenario']
-		target.mkdir(parents=True, exist_ok=True)
-		for key, source in (('branch', run_dir / 'runs' / r['scenario'] / 'captures' / f'{r["name"]}.png'),
-		                    ('master', run_dir / 'runs' / f'{r["scenario"]}--master' / 'captures' / f'{r["name"]}.png'),
-		                    ('diff', run_dir / (r.get('diff') or 'missing'))):
-			if source.is_file():
-				shutil.copy(source, target / f'{r["name"]}.{key}.png')
-
-	def visual_file(r, key):
-		return (f'visual/{r["scenario"]}/{r["name"]}.{key}.png'
-		        if (report / 'visual' / r['scenario'] / f'{r["name"]}.{key}.png').exists() else None)
-
-	replay = br.render_replay(screens, 'Each video shows one scenario in the n8n UI. Press Esc to close.')
-	version = str(time.time_ns())
-	page = br.fill_template(
-		br.esc(plan.get('title') or 'Instance AI feature test'), ''.join(rows), notes(status, runs, verdicts),
-		''.join(panels), replay=replay, after_table=render_visual(results, runs, visual_file), head=CSS,
-		scripts=(f'<script>const SCREENS = {json.dumps(screens)}; const REPORT_VERSION = "{version}";</script>'
-		         f'<script>window.addEventListener("DOMContentLoaded", ()=>{{{VIDEO_JS}{RELOAD_JS}}});</script>'
-		         f'<script>{STEP_VIEWER_JS}</script>'),
-		case=render_case(plan, status), thead=THEAD, kicker='Assistant UI test')
-	(report / 'index.html').write_text(page)
-	(report / 'version.txt').write_text(version)
-	return report / 'index.html'
-
-
-def clock(status):
-	started = status.get('startedAt')
-	elapsed = seconds((status.get('finishedAt') or time.time()) - started) if started else ''
-	return ' · '.join(x for x in (f'{status.get("concurrency", 1)} at a time', elapsed, status.get('step')) if x)
-
-
-def live_box(status, runs):
-	cards = []
-	for run in runs:
-		name, title = run['name'], br.esc(run_title(run))
-		cards.append(
-			f'<div class="live-card"><div class="live-screen"><button type="button" class="live-zoom" '
-			f'data-zoom="{name}" aria-label="Show {title} full screen"><img data-shot="{name}" '
-			f'alt="{title} in the n8n UI" hidden></button>'
-			f'<div class="live-empty" data-shot-empty="{name}">Waiting for the first screenshot</div></div>'
-			f'<div class="reel-meta"><span class="run-name">{title}</span>'
-			f'<span data-live-badge="{name}">{status_badge(run["entry"].get("status", "waiting"))}</span></div></div>')
-	return (f'<div class="box"><div class="head"><h2>Live</h2><span class="mono-label" id="live-clock">'
-	        f'{br.esc(clock(status))}</span></div><div class="live-grid">{"".join(cards)}</div></div>')
-
-
-def render_live(run_dir):
-	plan, status, runs = load(run_dir)
-	verdicts = br.load_verdicts(run_dir)
-	scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
-	rows, panels = [], []
-	for i, run in enumerate(runs):
-		name, active = run['name'], ' active' if i == 0 else ''
-		rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">'
-		            + row_cells(run, result_html(run, verdicts.get(name))) + '</tr>')
-		panels.append(f'<section class="run{active}" id="{name}">'
-		              + render_panel(run, verdicts.get(name), f'/files/runs/{name}/', scale) + '</section>')
-	text = ('This page updates while the scenarios run, and changes to the full report when the run is done. '
-	        'Tokens, cost, the visual check and verdicts come after the run.')
-	return br.fill_template(
-		br.esc(plan.get('title') or 'Instance AI feature test'), ''.join(rows), br.esc(text), ''.join(panels),
-		replay=live_box(status, runs), head=CSS,
-		scripts=f'<script>const SCREENS = [];</script><script>{ls.LIVE_JS}</script><script>{STEP_VIEWER_JS}</script>',
-		case=render_case(plan, status), thead=THEAD, kicker='Assistant UI test')
-
-
-class ScenarioApp:
-	"""The live page for live_server.serve(): one card, table row and panel for each scenario run."""
-
-	def __init__(self, run_dir):
-		self.run_dir = run_dir
-
-	def entry(self, name):
-		return ls.read_status(self.run_dir).get('runs', {}).get(name)
-
-	def finished(self, name):
-		return (self.entry(name) or {}).get('status') in FINISHED
-
-	def done(self):
-		return ls.read_status(self.run_dir).get('phase') == 'done' and (self.run_dir / 'report' / 'index.html').exists()
-
-	def page(self):
-		return render_live(self.run_dir)
-
-	def state(self):
-		plan, status, runs = load(self.run_dir)
-		verdicts = br.load_verdicts(self.run_dir)
-		scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
-		state = {'phase': status.get('phase', 'running'), 'clock': clock(status), 'runs': {}}
-		if self.done():
-			state['report'] = '/report/index.html'
-		for run in runs:
-			name = run['name']
-			shot = self.shot(name)
-			state['runs'][name] = {
-				'row': row_cells(run, result_html(run, verdicts.get(name))),
-				'panel': render_panel(run, verdicts.get(name), f'/files/runs/{name}/', scale),
-				'badge': status_badge(run['entry'].get('status', 'waiting')),
-				'shot': shot.stat().st_mtime if shot.exists() else None,
-			}
-		return state
-
-	def shot(self, name):
-		return self.run_dir / 'runs' / name / 'live.png'
-
-	def casts(self, name):
-		return ()
-
-	def file(self, path):
-		"""The step screenshots of a run while it runs: /files/runs/<name>/<file>.png."""
-		if not path.startswith('/files/runs/') or not path.endswith('.png'):
-			return None
-		base = (self.run_dir / 'runs').resolve()
-		candidate = (self.run_dir / path.removeprefix('/files/')).resolve()
-		return candidate if candidate.is_relative_to(base) and candidate.is_file() else None
-
-
-def triage(run, visual):
-	"""The reasons to look closer at a run. An empty list means that the run looks clean: every step and check
-	passed, with no browser error, no failed tool call, and no visual change."""
-	reasons = []
-	status = run['entry'].get('status')
-	if status != 'passed':
-		reasons.append(f'the run {status}')
-	failed = [s for s in run['steps'] if s['status'] == 'failed']
-	if failed:
-		reasons.append(f'{len(failed)} failed step(s) or check(s)')
-	console = run['console']
-	errors = len(console.get('consoleErrors') or []) + len(console.get('requestErrors') or [])
-	if errors:
-		reasons.append(f'{errors} browser error(s)')
-	tool_errors = sum(e['error'] for e in run['data']['events'] if e['kind'] == 'tool')
-	if tool_errors:
-		reasons.append(f'{tool_errors} failed tool call(s)')
-	changed = [r['name'] for r in visual if r['scenario'] == run['scenario']['id'] and r['status'] != 'unchanged']
-	if changed and not run['master']:
-		reasons.append(f'visual change in {", ".join(changed)}')
-	return reasons
-
-
-def summarize(run_dir):
-	"""Plain text for the judge: a triage list, then each run with its steps, final reply, errors and visual
-	results."""
-	plan, status, runs = load(run_dir)
-	visual = read_json(run_dir / 'visual.json', [])
-	out = [f'# {plan.get("title", "")}', '', 'Feature:', plan.get('feature') or '-', '', '## Triage', '',
-	       'Clean runs passed every step and check, with no browser error, failed tool call or visual change.',
-	       'For these, check that the last reply and the final screenshot match the expected result.',
-	       'Look closely at the other runs: their screenshots, errors and diffs.', '']
-	judged = [r for r in runs if not r['master']]
-	for run in judged:
-		reasons = triage(run, visual)
-		out.append(f'- {run["name"]}: ' + ('clean' if not reasons else 'look closer: ' + '; '.join(reasons)))
-	out.append('')
-	for run in runs:
-		s, result, data = run['scenario'], run['result'], run['data']
-		out += [f'## {run["name"]}: {run_title(run)} ({s["kind"]}, {"master" if run["master"] else "branch"})',
-		        f'Expected: {s["expected"]}',
-		        f'Status: {run["entry"].get("status")}' + (f' ({result["error"]})' if result.get('error') else ''),
-		        'Steps:']
-		for st in run['steps']:
-			shot = f'  (runs/{run["name"]}/{st["shot"]})' if st.get('shot') else ''
-			out.append(f'  {st["i"]}. [{st["status"]}] {st["kind"]}: {st["title"]}{shot}')
-			if st.get('error'):
-				out.append('     ' + st['error'].replace('\n', '\n     ')[:1500])
-		for p in result.get('prompts') or []:
-			out.append(f'Prompt (thread {p.get("thread")}): {p["text"]}')
-		tools = [e for e in data['events'] if e['kind'] == 'tool']
-		if tools:
-			out.append('Tool calls: ' + ', '.join(e['name'] + (' (error)' if e['error'] else '') for e in tools))
-		texts = [e['text'] for e in data['events'] if e['kind'] == 'text']
-		if texts:
-			out.append('Last reply: ' + texts[-1].strip()[:2000])
-		for d in run['meta'].get('decisions') or []:
-			out.append(f'Confirmation: {d["decision"]} [{d.get("toolName")}]')
-		console = run['console']
-		for e in (console.get('consoleErrors') or [])[:8]:
-			out.append(f'Console error: {e.get("text", "")[:300]}')
-		for e in (console.get('requestErrors') or [])[:8]:
-			out.append(f'Request error: {e.get("status")} {e.get("method")} {e.get("url")}')
-		if s['kind'] == 'visual' and not run['master']:
-			for r in (v for v in visual if v['scenario'] == s['id']):
-				ratio = f', {100 * r["ratio"]:.2f}% of pixels' if r.get('ratio') is not None else ''
-				out.append(f'Visual {r["name"]}: {r["status"]}{ratio}')
-				if r.get('ariaDiff'):
-					out.append('  Accessibility tree diff:\n    ' + '\n    '.join(r['ariaDiff'].splitlines()[:40]))
-		out.append('')
-	return '\n'.join(out)
-
-
-def main():
-	if len(sys.argv) < 2:
-		sys.exit(__doc__)
-	run_dir = Path(sys.argv[1]).resolve()
-	if '--summary' in sys.argv:
-		print(summarize(run_dir))
-	else:
-		print(build(run_dir))
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-assistant-ui/scripts/run.py

@@ -1,931 +0,0 @@
-#!/usr/bin/env python3
-"""Run a test plan of Instance AI scenarios and build one report. The judge step is separate.
-
-  run.py --plan <dir>/scenarios.json [--only id1,id2] [--concurrency 4]
-  run.py --rerun last|<run-dir>             the plan of an earlier run, again in a new folder
-  run.py --plan ... --try                   a quick check of new scripts: no master, no live page, no report
-  run.py --plan ... --estimate              print the expected cost and time, then stop
-  run.py --check                            check the setup, then stop
-
-Steps:
-  1. Check the setup, and build this checkout with turbo (a few seconds when nothing changed).
-  2. Start fresh n8n instances from the build. Each feature and regression scenario gets its own
-     instance, unless it sets "isolated": false. Instances start ahead of their scenario, so a
-     scenario does not wait for one.
-  3. Run the scenarios, several at the same time.
-  4. At the same time: build the master merge base (baseline.py), start a fresh master instance and
-     a fresh branch instance, and run each visual scenario on both. A master run that passed is kept
-     and used again while the merge base and the script stay the same.
-  5. Compare the visual captures, write summary.txt for the judge, and build the report.
-  6. Fetch the token usage from LangSmith and build the report again.
-A local page shows the runs live (live_page.py). It opens in the default browser, unless --no-open.
-Settings come from .env.local and .env.ai-surfaces.
-"""
-
-import argparse
-import difflib
-import hashlib
-import json
-import os
-import re
-import shutil
-import signal
-import socket
-import statistics
-import subprocess
-import sys
-import threading
-import time
-import webbrowser
-from concurrent.futures import ThreadPoolExecutor
-from datetime import datetime, timezone
-from pathlib import Path
-
-import baseline
-import report
-from common import CACHE, HERE, PLANS_ROOT, REPO_ROOT, RUNS_ROOT, SHARED, Instance, client, git, setting
-from langsmith_usage import fetch_calls, totals
-from live import enabled as live_enabled
-from run_instance_ai import decide
-
-KINDS = ('feature', 'regression', 'visual')
-LIB = HERE.parent / 'scenarios'
-ID = re.compile(r'^[a-z0-9][a-z0-9-]{0,60}$')
-MASTER_CACHE = CACHE / 'master-runs'
-MASTER_CACHE_DAYS = 14
-PRINT_LOCK = threading.Lock()
-RUNNERS = set()
-S = '.agents/skills/testmax-assistant-ui/scripts'
-
-
-def say(line, prefix=None):
-	with PRINT_LOCK:
-		print(f'[{prefix}] {line}' if prefix else line, flush=True)
-
-
-def step(title):
-	say(f'\n=== {title} ===')
-
-
-def now_iso():
-	return datetime.now(timezone.utc).isoformat()
-
-
-def slug(text, words=8):
-	return '-'.join(re.findall(r'[a-z0-9]+', text.lower())[:words]) or 'run'
-
-
-def script_path(spec, base):
-	return LIB / spec[4:] if spec.startswith('lib:') else (base / spec).resolve()
-
-
-def load_plan(path):
-	"""Read and check scenarios.json. Returns the plan with each script path resolved."""
-	path = Path(path).resolve()
-	plan = json.loads(path.read_text())
-	errors, seen = [], set()
-	if not plan.get('scenarios'):
-		errors.append('The plan has no scenarios.')
-	if not isinstance(plan.get('featureFlags', {}), dict):
-		errors.append('featureFlags must be an object of flag names and values.')
-	for s in plan.get('scenarios', []):
-		sid = s.get('id', '')
-		if not ID.match(sid) or '--' in sid:
-			errors.append(f'Scenario id "{sid}": use a-z, 0-9 and single "-" characters.')
-		if sid in seen:
-			errors.append(f'Scenario id "{sid}" is used two times.')
-		seen.add(sid)
-		if s.get('kind') not in KINDS:
-			errors.append(f'Scenario "{sid}": kind must be one of {", ".join(KINDS)}.')
-		if not s.get('script') or not script_path(s['script'], path.parent).is_file():
-			errors.append(f'Scenario "{sid}": script {s.get("script")} not found.')
-		if not s.get('title') or not s.get('expected'):
-			errors.append(f'Scenario "{sid}": give a title and an expected outcome.')
-	if errors:
-		sys.exit('The plan is not valid:\n  ' + '\n  '.join(errors))
-	for s in plan['scenarios']:
-		s['scriptPath'] = str(script_path(s['script'], path.parent))
-	plan.setdefault('source', str(path))
-	plan['name'] = path.parent.name if path.parent.name != 'plan' else slug(plan.get('title') or 'run')
-	return plan
-
-
-def run_dirs():
-	"""Earlier run folders, newest first."""
-	if not RUNS_ROOT.exists():
-		return []
-	found = [d for d in RUNS_ROOT.iterdir() if (d / 'plan' / 'scenarios.json').is_file()]
-	return sorted(found, key=lambda d: (d / 'plan' / 'scenarios.json').stat().st_mtime, reverse=True)
-
-
-def rerun_plan(arg):
-	"""The plan to run again. The source plan wins over the copy, so a fixed script is used."""
-	if arg == 'last':
-		found = run_dirs()
-		if not found:
-			sys.exit(f'No earlier run in {RUNS_ROOT}.')
-		run_dir = found[0]
-	else:
-		run_dir = Path(arg).expanduser().resolve()
-	copy = run_dir / 'plan' / 'scenarios.json'
-	if not copy.is_file():
-		sys.exit(f'{run_dir} has no plan/scenarios.json.')
-	source = json.loads(copy.read_text()).get('source')
-	if source and Path(source).is_file():
-		say(f'rerun of {run_dir.name}, with the current plan and scripts in {Path(source).parent}')
-		return source
-	say(f'rerun of {run_dir.name}, with the scripts copied into that run')
-	return copy
-
-
-def new_run_dir(root, name):
-	base = root / f'{datetime.now():%Y%m%d-%H%M%S}-{name}'
-	run_dir, n = base, 2
-	while run_dir.exists():
-		run_dir, n = base.with_name(f'{base.name}-{n}'), n + 1
-	run_dir.mkdir(parents=True)
-	return run_dir
-
-
-def copy_plan(plan, run_dir):
-	"""Keep the plan and its scripts in the run folder, so --rerun can repeat the run."""
-	plan_dir = run_dir / 'plan'
-	(plan_dir / 'scripts').mkdir(parents=True, exist_ok=True)
-	saved = json.loads(json.dumps(plan))
-	saved.pop('name', None)
-	for s in saved['scenarios']:
-		target = plan_dir / 'scripts' / f'{s["id"]}.mjs'
-		shutil.copy(s.pop('scriptPath'), target)
-		s['script'] = f'scripts/{target.name}'
-	(plan_dir / 'scenarios.json').write_text(json.dumps(saved, indent=2))
-
-
-class Progress:
-	"""Writes status.json, which the live page reads."""
-
-	def __init__(self, run_dir, data):
-		self.path, self.lock, self.data = run_dir / 'status.json', threading.Lock(), data
-		self.save()
-
-	def save(self):
-		tmp = self.path.with_suffix('.tmp')
-		tmp.write_text(json.dumps(self.data, indent=2))
-		os.replace(tmp, self.path)
-
-	def update(self, **fields):
-		with self.lock:
-			self.data.update(fields)
-			self.save()
-
-	def run(self, name, **fields):
-		with self.lock:
-			self.data['runs'][name].update(fields)
-			self.save()
-
-	def finish(self):
-		with self.lock:
-			for entry in self.data['runs'].values():
-				if entry['status'] in ('waiting', 'starting'):
-					entry.update(status='skipped', reason=entry.get('reason') or 'The run did not start.')
-				elif entry['status'] == 'running':
-					entry['status'] = 'error'
-			if self.data.get('phase') != 'done':
-				self.data.update(phase='done', step='Done', finishedAt=time.time())
-			self.save()
-
-
-def close_stream(res):
-	"""Close an event stream that another thread reads. close() alone waits for the next server
-	keep-alive (about 13 seconds), because the reader holds the buffer lock. A socket shutdown ends the
-	read at once."""
-	try:
-		with socket.socket(fileno=os.dup(res.fileno())) as sock:
-			sock.shutdown(socket.SHUT_RDWR)
-	except (OSError, ValueError):
-		pass
-	try:
-		res.close()
-	except Exception:
-		pass
-
-
-class ThreadEvents:
-	"""Reads the event stream of each thread that a scenario opens. The stream starts at the first event."""
-
-	def __init__(self, base_url, out, auto_confirm, prefix):
-		self.client = client(base_url).login()
-		self.out, self.auto_confirm, self.prefix = out, auto_confirm, prefix
-		self.file, self.lock = (out / 'events.jsonl').open('a'), threading.Lock()
-		self.run_ids, self.decisions, self.streams, self.closed = [], [], [], False
-
-	def follow(self, thread_id):
-		threading.Thread(target=self._read, args=(thread_id,), daemon=True).start()
-
-	def _read(self, thread_id):
-		try:
-			res = self.client.open_stream(f'/rest/instance-ai/events/{thread_id}')
-		except Exception as e:
-			say(f'event stream of {thread_id} did not open: {e}', self.prefix)
-			return
-		self.streams.append(res)
-		data, name = [], None
-		try:
-			for raw in res:
-				line = raw.decode(errors='replace').rstrip('\r\n')
-				if line.startswith('data:'):
-					data.append(line[5:].lstrip())
-				elif line.startswith('event:'):
-					name = line[6:].strip()
-				elif line == '' and data:
-					try:
-						self._event(json.loads('\n'.join(data)))
-					except json.JSONDecodeError:
-						pass
-					data, name = [], None
-		except Exception:
-			# close() ends the stream when the scenario stops.
-			pass
-
-	def _event(self, event):
-		with self.lock:
-			if self.closed:
-				return
-			self.file.write(json.dumps(event) + '\n')
-			self.file.flush()
-		payload = event.get('payload') or {}
-		if event.get('type') == 'run-start' and payload.get('langsmithRunId'):
-			self.run_ids.append(payload['langsmithRunId'])
-		if event.get('type') == 'confirmation-request' and self.auto_confirm:
-			body, label = decide(payload)
-			try:
-				self.client.request('POST', f'/rest/instance-ai/confirm/{payload["requestId"]}', body)
-				ok = True
-			except RuntimeError as e:
-				ok, label = False, f'{label} (failed: {str(e)[:200]})'
-			self.decisions.append({'at': now_iso(), 'toolName': payload.get('toolName'), 'decision': label, 'ok': ok})
-			say(f'confirm: {label} [{payload.get("toolName")}]', self.prefix)
-
-	def close(self):
-		with self.lock:
-			self.closed = True
-			self.file.close()
-		for res in self.streams:
-			close_stream(res)
-
-
-def run_job(name, scenario, url, instance, run_dir, progress, live):
-	sub = run_dir / 'runs' / name
-	sub.mkdir(parents=True, exist_ok=True)
-	progress.run(name, status='running', startedAt=time.time(), baseUrl=url)
-	auto_confirm = scenario.get('autoConfirm', True)
-	timeout_min = float(scenario.get('timeoutMin', 10))
-	started = now_iso()
-	try:
-		events = ThreadEvents(url, sub, auto_confirm, name)
-	except Exception as e:
-		progress.run(name, status='error', finishedAt=time.time(), reason=f'Cannot sign in to {url}: {e}')
-		return
-	cmd = ['node', str(HERE / 'run_scenario.mjs'), '--out', str(sub), '--script', scenario['scriptPath'],
-	       '--timeout-min', str(timeout_min), '--viewport', scenario.get('viewport', '1600x1000')]
-	cmd += ['--live'] if live else []
-	cmd += ['--auto-confirm'] if auto_confirm else []
-	proc = subprocess.Popen(cmd, cwd=HERE, env={**os.environ, 'N8N_BASE_URL': url},
-	                        stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
-	RUNNERS.add(proc)
-	# The scenario has its own time limit. This one covers a runner that hangs outside the scenario.
-	watchdog = threading.Timer(timeout_min * 60 + 180, proc.kill)
-	watchdog.start()
-	with (sub / 'runner.log').open('w') as log:
-		for line in proc.stdout:
-			log.write(line)
-			say(line.rstrip(), name)
-			match = re.match(r'thread (\S+)', line)
-			if match:
-				events.follow(match[1])
-	proc.wait()
-	RUNNERS.discard(proc)
-	watchdog.cancel()
-	# The last events of a run can arrive just after the scenario ends.
-	time.sleep(1)
-	events.close()
-	try:
-		result = json.loads((sub / 'result.json').read_text())
-	except (OSError, ValueError):
-		result = {'status': 'error', 'error': f'The runner stopped without a result (exit {proc.returncode}). See runner.log.'}
-		(sub / 'result.json').write_text(json.dumps(result, indent=2))
-	meta = {
-		'kind': 'scenario', 'scenario': scenario['id'], 'instance': instance, 'baseUrl': url,
-		'threadIds': result.get('threads', []), 'langsmithRunIds': events.run_ids, 'decisions': events.decisions,
-		'startedAt': started, 'finishedAt': now_iso(),
-		'screen': 'screen.webm' if (sub / 'screen.webm').exists() else None,
-	}
-	(sub / 'run.json').write_text(json.dumps(meta, indent=2))
-	progress.run(name, status=result['status'], finishedAt=time.time(), reason=result.get('error'))
-
-
-def stale_packages():
-	"""Packages with a source change in this branch that is newer than their build output."""
-	try:
-		base = subprocess.run(['git', 'merge-base', 'HEAD', 'origin/master'], cwd=REPO_ROOT, capture_output=True,
-		                      text=True, check=True).stdout.strip()
-		changed = subprocess.run(['git', 'diff', '--name-only', base], cwd=REPO_ROOT, capture_output=True,
-		                         text=True, check=True).stdout.split()
-		changed += [line[3:] for line in subprocess.run(['git', 'status', '--porcelain'], cwd=REPO_ROOT,
-		            capture_output=True, text=True, check=True).stdout.splitlines()]
-	except subprocess.CalledProcessError:
-		return []
-	newest_dist, stale = {}, set()
-	for rel in changed:
-		path = REPO_ROOT / rel
-		if not rel.startswith('packages/') or '/src/' not in rel or re.search(r'(\.test\.|__tests__|\.md$)', rel):
-			continue
-		if not path.is_file():
-			continue
-		pkg = REPO_ROOT / rel.split('/src/', 1)[0]
-		dist = pkg / 'dist'
-		if not dist.exists():
-			continue
-		if pkg not in newest_dist:
-			newest_dist[pkg] = max((f.stat().st_mtime for f in dist.rglob('*') if f.is_file()), default=0)
-		if path.stat().st_mtime > newest_dist[pkg]:
-			stale.add(str(pkg.relative_to(REPO_ROOT)))
-	return sorted(stale)
-
-
-def build_branch(log_path):
-	"""Build n8n and its packages with turbo. Unchanged packages come from the turbo cache in seconds."""
-	started = time.time()
-	with log_path.open('w') as log:
-		code = subprocess.run(['pnpm', 'turbo', 'run', 'build', '--filter=n8n...', '--output-logs=errors-only'],
-		                      cwd=REPO_ROOT, stdout=log, stderr=subprocess.STDOUT).returncode
-	text = log_path.read_text()
-	if code != 0:
-		sys.exit(f'The build failed. The end of {log_path}:\n' + '\n'.join(text.splitlines()[-30:]))
-	cached = re.search(r'Cached:\s+(\d+) cached, (\d+) total', text)
-	if cached and cached[1] == cached[2]:
-		say(f'build: up to date ({time.time() - started:.0f}s)')
-	else:
-		say(f'build: done in {time.time() - started:.0f}s')
-
-
-def preflight(build, need_master, local_instances=True):
-	"""Problems that stop a run (errors) or make the report less useful (warnings).
-	local_instances is False with --branch-url: the branch runs then need no local build."""
-	errors, warnings = [], []
-	for tool in ('node', 'git') + (('pnpm',) if build or need_master else ()):
-		if not shutil.which(tool):
-			errors.append(f'{tool} is not on the PATH.')
-	if shutil.which('node'):
-		probe = ("import(process.argv[1]).then(async (m) => { const b = await m.launchBrowser(); await b.close(); })"
-		         ".catch((e) => { console.error(e.message.split('\\n')[0]); process.exit(1); })")
-		res = subprocess.run(['node', '-e', probe, str(SHARED / 'n8n_browser.mjs')], capture_output=True, text=True,
-		                     timeout=60)
-		if res.returncode != 0:
-			errors.append('Playwright cannot start a browser: '
-			              f'{(res.stderr or res.stdout).strip()[:300]}. Run pnpm install, and install Google Chrome.')
-	if not build and local_instances:
-		for path in ('packages/cli/dist', 'packages/frontend/editor-ui/dist/index.html'):
-			if not (REPO_ROOT / path).exists():
-				errors.append(f'{path} does not exist. Build first, or run without --no-build.')
-	if not setting('N8N_INSTANCE_AI_MODEL'):
-		warnings.append('N8N_INSTANCE_AI_MODEL is not set in .env.local, .env.ai-surfaces or the environment. '
-		                'Scenarios that send a prompt can fail.')
-	if not setting('LANGSMITH_API_KEY'):
-		warnings.append('LANGSMITH_API_KEY is not set. The report has no tokens or cost.')
-	elif (setting('LANGSMITH_TRACING') or '').lower() != 'true':
-		warnings.append('LANGSMITH_TRACING is not true. The instances send no traces, so the report has no tokens or cost.')
-	if (setting('N8N_DIAGNOSTICS_ENABLED') or '').lower() == 'false':
-		warnings.append('N8N_DIAGNOSTICS_ENABLED is false. Instance AI sends no traces, so the report has no tokens or cost.')
-	RUNS_ROOT.mkdir(parents=True, exist_ok=True)
-	free = shutil.disk_usage(RUNS_ROOT).free / 1e9
-	if free < 2:
-		warnings.append(f'Only {free:.1f} GB free in {RUNS_ROOT}. A run needs about 0.5 GB.')
-	if need_master and not baseline.built():
-		CACHE.mkdir(parents=True, exist_ok=True)
-		free = shutil.disk_usage(CACHE).free / 1e9
-		warnings.append('The master build does not exist yet. The first build takes 15 to 20 minutes'
-		                + (f', and needs about 3 GB (free: {free:.1f} GB).' if free < 5 else '.'))
-	return errors, warnings
-
-
-def history():
-	"""Cost and time of earlier runs, by scenario id. Master runs and runs without a result are left out."""
-	by_id, every = {}, []
-	for run_dir in run_dirs():
-		try:
-			runs = json.loads((run_dir / 'status.json').read_text()).get('runs', {})
-		except (OSError, ValueError):
-			continue
-		for name, entry in runs.items():
-			if name.endswith('--master') or entry.get('status') not in ('passed', 'failed'):
-				continue
-			if not (entry.get('startedAt') and entry.get('finishedAt')):
-				continue
-			usage = run_dir / 'runs' / name / 'usage.json'
-			try:
-				if usage.exists():
-					cost = sum(c['cost'] for c in json.loads(usage.read_text()))
-				else:
-					meta = json.loads((run_dir / 'runs' / name / 'run.json').read_text())
-					cost = None if meta.get('langsmithRunIds') else 0.0
-			except (OSError, ValueError):
-				cost = None
-			sample = (cost, entry['finishedAt'] - entry['startedAt'])
-			by_id.setdefault(entry.get('scenario', name), []).append(sample)
-			every.append(sample)
-	return by_id, every
-
-
-def estimate(jobs, concurrency, compare_master, flags):
-	"""Expected cost and time of the jobs, from the earlier runs of the same scenario ids. A master run that
-	can be reused costs nothing."""
-	by_id, every = history()
-	try:
-		base = baseline.target() if compare_master else None
-	except subprocess.CalledProcessError:
-		base = None
-
-	def median(samples, index, default):
-		values = [x[index] for x in samples if x[index] is not None]
-		return statistics.median(values) if values else default
-
-	any_cost, any_time = median(every, 0, None), median(every, 1, 60)
-	rows, cost_total, known = [], 0.0, True
-	for name, s, role in jobs:
-		if role == 'master' and base and (MASTER_CACHE / master_key(s, base, flags) / 'result.json').is_file():
-			rows.append((name + ' (kept)', 0.0, 0, True))
-			continue
-		samples = by_id.get(s['id'], [])
-		cost = median(samples, 0, any_cost)
-		seconds = median(samples, 1, any_time)
-		if cost is None:
-			known = False
-		else:
-			cost_total += cost
-		rows.append((name, cost, seconds, bool(samples)))
-	times = sorted((r[2] for r in rows), reverse=True)
-	wall = max(times[0] if times else 0, sum(times) / max(1, concurrency)) + 15
-	return {'rows': rows, 'cost': cost_total if known else None, 'seconds': wall,
-	        'masterBuild': compare_master and not baseline.built()}
-
-
-def print_estimate(est):
-	say('Estimate (median of earlier runs; * means no earlier run of this scenario):')
-	for name, cost, seconds, seen in est['rows']:
-		price = f'${cost:.2f}' if cost is not None else '  ?  '
-		say(f'  {name:44} {price:>7} {seconds:5.0f}s{"" if seen else " *"}')
-	total = f'about ${est["cost"]:.2f}' if est['cost'] is not None else 'unknown cost'
-	extra = ', plus 15 to 20 minutes for the first master build' if est['masterBuild'] else ''
-	say(f'  {len(est["rows"])} runs, {total}, about {est["seconds"] / 60:.1f} minutes{extra}')
-
-
-def fetch_usage(run_dir, names, timeout=90):
-	"""Token usage and cost of each run from LangSmith, in one query for all runs. LangSmith receives traces
-	with a delay, so the query repeats until each run has calls and the count stops changing."""
-	if not setting('LANGSMITH_API_KEY'):
-		return
-	metas = {}
-	for name in names:
-		file = run_dir / 'runs' / name / 'run.json'
-		meta = json.loads(file.read_text()) if file.exists() else {}
-		if meta.get('langsmithRunIds') and not meta.get('cached'):
-			metas[name] = meta
-	if not metas:
-		return
-	start = min(m['startedAt'] for m in metas.values())
-	end = max(m['finishedAt'] for m in metas.values())
-	traces = {t for m in metas.values() for t in m['langsmithRunIds']}
-	deadline, previous, calls = time.time() + timeout, None, []
-	while True:
-		try:
-			calls = fetch_calls(start, end, traces)
-		except (SystemExit, OSError, ValueError) as e:
-			say(f'usage failed: {str(e)[:200]}')
-			return
-		seen = {c['trace'] for c in calls}
-		complete = all(set(m['langsmithRunIds']) & seen for m in metas.values())
-		if (complete and len(calls) == previous) or time.time() > deadline:
-			break
-		previous = len(calls)
-		time.sleep(10)
-	for name, meta in metas.items():
-		own = [c for c in calls if c['trace'] in set(meta['langsmithRunIds'])]
-		(run_dir / 'runs' / name / 'usage.json').write_text(json.dumps(own, indent=1))
-		usage = totals(own)
-		say(f'{usage["calls"]} LLM calls, ${usage["cost"]:.2f}', name)
-
-
-def compare_trees(run_dir):
-	"""Add the accessibility tree diff of each capture to visual.json."""
-	path = run_dir / 'visual.json'
-	if not path.exists():
-		return []
-	results = json.loads(path.read_text())
-	for r in results:
-		trees = []
-		for name in (f'{r["scenario"]}--master', r['scenario']):
-			file = run_dir / 'runs' / name / 'captures' / f'{r["name"]}.aria.yml'
-			trees.append(file.read_text().splitlines() if file.exists() else [])
-		diff = '\n'.join(difflib.unified_diff(*trees, 'master', 'branch', lineterm='', n=2))
-		r['ariaDiff'] = diff
-		r['pixelStatus'] = r['status']
-		if diff and r['status'] == 'unchanged':
-			r['status'] = 'changed'
-	path.write_text(json.dumps(results, indent=2))
-	return results
-
-
-def master_key(scenario, commit, flags):
-	"""A master run gives the same screens while the commit, the flags, the script and the runner stay the same."""
-	h = hashlib.sha256()
-	overrides = setting('N8N_FEATURE_FLAG_OVERRIDES') or ''
-	for part in (commit, json.dumps(flags or {}, sort_keys=True), overrides, scenario.get('viewport', '1600x1000'),
-	             str(scenario.get('autoConfirm', True))):
-		h.update(part.encode() + b'\0')
-	for file in (Path(scenario['scriptPath']), HERE / 'kit.mjs', HERE / 'run_scenario.mjs'):
-		h.update(file.read_bytes())
-	return h.hexdigest()[:20]
-
-
-def save_master_run(sub, key):
-	if json.loads((sub / 'result.json').read_text()).get('status') != 'passed':
-		return
-	target = MASTER_CACHE / key
-	shutil.rmtree(target, ignore_errors=True)
-	shutil.copytree(sub, target, ignore=shutil.ignore_patterns('live.png', 'usage.json'))
-	cutoff = time.time() - MASTER_CACHE_DAYS * 86400
-	for old in MASTER_CACHE.iterdir():
-		if old.is_dir() and old.stat().st_mtime < cutoff:
-			shutil.rmtree(old, ignore_errors=True)
-
-
-def reuse_master_run(sub, key):
-	"""Copy a kept master run into this run. Returns False when there is none."""
-	source = MASTER_CACHE / key
-	if not (source / 'result.json').is_file():
-		return False
-	shutil.copytree(source, sub, dirs_exist_ok=True)
-	meta = json.loads((sub / 'run.json').read_text())
-	meta['cached'] = True
-	(sub / 'run.json').write_text(json.dumps(meta, indent=2))
-	return True
-
-
-def start_live_server(run_dir):
-	url_file = run_dir / 'live-url.txt'
-	url_file.unlink(missing_ok=True)
-	log = (run_dir / 'live-server.log').open('w')
-	# A new session, so the page still serves the report after this script ends.
-	subprocess.Popen([sys.executable, str(HERE / 'live_page.py'), '--run-dir', str(run_dir),
-	                  '--parent-pid', str(os.getpid())],
-	                 cwd=HERE, stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
-	for _ in range(50):
-		if url_file.exists() and url_file.read_text().strip():
-			return url_file.read_text().strip()
-		time.sleep(0.1)
-	return None
-
-
-def open_in_default_browser(url):
-	"""Opens url in the default browser. Returns False if no browser opened.
-
-	$BROWSER is not used: agents often set it to a command that does nothing.
-	"""
-	command = {'darwin': ['open'], 'linux': ['xdg-open']}.get(sys.platform)
-	if command and shutil.which(command[0]):
-		try:
-			return subprocess.run(command + [url], capture_output=True, timeout=15).returncode == 0
-		except (OSError, subprocess.TimeoutExpired):
-			return False
-	try:
-		return webbrowser.open(url)
-	except webbrowser.Error:
-		return False
-
-
-def result_table(run_dir, runs):
-	"""A plain text table of the results, with the first problem of each run that did not pass."""
-	lines = [f'  {"run":44} {"status":15} {"steps":>7} {"time":>6} {"errors":>6}']
-	for name, entry in runs.items():
-		sub = run_dir / 'runs' / name
-		steps = report.read_json(sub / 'steps.json', [])
-		result = report.read_json(sub / 'result.json', {})
-		checked = [s for s in steps if s['kind'] in ('step', 'check', 'capture')]
-		passed = f'{sum(s["status"] == "passed" for s in checked)}/{len(checked)}' if checked else '-'
-		took = (entry.get('finishedAt') or 0) - (entry.get('startedAt') or 0)
-		errors = (result.get('consoleErrors') or 0) + (result.get('requestErrors') or 0)
-		status = entry['status'] + (' (kept)' if report.read_json(sub / 'run.json', {}).get('cached') else '')
-		lines.append(f'  {name:44} {status:15} {passed:>7} {took:5.0f}s {errors:>6}')
-		failed = next((s for s in steps if s['status'] == 'failed'), None)
-		if failed:
-			problem = f'{failed["kind"]} "{failed["title"]}": {failed.get("error") or ""}'
-		else:
-			problem = entry.get('reason') if entry['status'] != 'passed' else ''
-		if (problem or '').strip():
-			lines.append(f'      {problem.strip().splitlines()[0][:160]}')
-	return '\n'.join(lines)
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--plan', help='scenarios.json')
-	ap.add_argument('--rerun', help='"last", or an earlier run folder. Uses the current plan and scripts when they exist.')
-	ap.add_argument('--only', help='Comma-separated scenario ids.')
-	ap.add_argument('--run-dir')
-	ap.add_argument('--concurrency', type=int, default=int(setting('FEATURE_TEST_CONCURRENCY', default='4')))
-	ap.add_argument('--branch-url', help='Use this running instance for the branch runs instead of fresh ones.')
-	ap.add_argument('--try', dest='try_mode', action='store_true',
-	                help='A quick check of new scripts: no master, no live page, no usage, no report.')
-	ap.add_argument('--estimate', action='store_true', help='Print the expected cost and time, then stop.')
-	ap.add_argument('--check', action='store_true', help='Check the setup, then stop.')
-	ap.add_argument('--no-build', action='store_true', help='Do not build this checkout first.')
-	ap.add_argument('--no-baseline', action='store_true', help='Skip master. Visual scenarios run on the branch only.')
-	ap.add_argument('--no-cache', action='store_true', help='Run master again, also when a kept master run exists.')
-	ap.add_argument('--no-fetch', action='store_true', help='Use origin/master as it is, without git fetch.')
-	ap.add_argument('--no-live', action='store_true', help='Do not start the live page.')
-	ap.add_argument('--no-open', action='store_true',
-	                help='Do not open the live page in the default browser. Use this when you open it in the editor browser.')
-	ap.add_argument('--keep-instances', action='store_true', help='Keep the fresh instances running after the run.')
-	args = ap.parse_args()
-	if args.try_mode:
-		args.no_baseline = args.no_live = True
-	build = not args.no_build and not args.branch_url
-
-	if args.check and not (args.plan or args.rerun):
-		errors, warnings = preflight(build, need_master=True)
-		for w in warnings:
-			say(f'warning: {w}')
-		for e in errors:
-			say(f'error: {e}')
-		branch = git('rev-parse', '--abbrev-ref', 'HEAD').replace('/', '-')
-		say(f'plan folder for this branch: {PLANS_ROOT / branch}/<feature>/')
-		say('The setup is ready.' if not errors else 'Fix the errors, then run again.')
-		sys.exit(1 if errors else 0)
-	if not (args.plan or args.rerun):
-		sys.exit('Give --plan <scenarios.json>, --rerun last|<run-dir>, or --check.')
-	# The fresh instances run in their own sessions. Exit through the cleanup, so they stop too.
-	for sig in (signal.SIGTERM, signal.SIGHUP):
-		signal.signal(sig, lambda signum, _frame: sys.exit(128 + signum))
-
-	plan = load_plan(rerun_plan(args.rerun) if args.rerun else args.plan)
-	if args.only:
-		wanted = {x.strip() for x in args.only.split(',')}
-		plan['scenarios'] = [s for s in plan['scenarios'] if s['id'] in wanted]
-		if not plan['scenarios']:
-			sys.exit(f'No scenario matches --only {args.only}.')
-
-	visual = [s for s in plan['scenarios'] if s['kind'] == 'visual']
-	compare_master = bool(visual) and not args.no_baseline
-	jobs = []  # (name, scenario, instance role)
-	for s in plan['scenarios']:
-		if args.branch_url:
-			role = 'branch'
-		elif s['kind'] == 'visual':
-			role = 'branch-visual'
-		else:
-			role = 'isolated' if s.get('isolated', True) else 'branch'
-		jobs.append((s['id'], s, role))
-		if s['kind'] == 'visual' and compare_master:
-			jobs.append((f'{s["id"]}--master', s, 'master'))
-
-	est = estimate(jobs, args.concurrency, compare_master, plan.get('featureFlags'))
-	print_estimate(est)
-	if args.estimate:
-		return
-	errors, warnings = preflight(build, compare_master, local_instances=not args.branch_url)
-	for w in warnings:
-		say(f'warning: {w}')
-	if errors:
-		sys.exit('The setup is not ready:\n  ' + '\n  '.join(errors))
-
-	if args.run_dir:
-		run_dir = Path(args.run_dir).expanduser().resolve()
-		run_dir.mkdir(parents=True, exist_ok=True)
-	else:
-		run_dir = new_run_dir(RUNS_ROOT / 'try' if args.try_mode else RUNS_ROOT, plan['name'])
-	copy_plan(plan, run_dir)
-	say(f'run directory: {run_dir}')
-
-	branch_commit = subprocess.run(['git', 'rev-parse', '--short', 'HEAD'], cwd=REPO_ROOT, capture_output=True,
-	                               text=True).stdout.strip()
-	dirty = bool(subprocess.run(['git', 'status', '--porcelain', '--', 'packages'], cwd=REPO_ROOT,
-	                            capture_output=True, text=True).stdout.strip())
-	live = live_enabled() and not args.no_live
-	progress = Progress(run_dir, {
-		'phase': 'running', 'step': 'Building the branch' if build else 'Starting the instances',
-		'startedAt': time.time(), 'concurrency': args.concurrency,
-		'branch': {'commit': branch_commit, 'dirty': dirty, 'url': args.branch_url},
-		'master': {'commit': None} if compare_master else None,
-		'runs': {name: {'status': 'waiting', 'scenario': s['id'], 'instance': role.split('-')[0]}
-		         for name, s, role in jobs},
-	})
-
-	live_url = None
-	if live:
-		live_url = start_live_server(run_dir)
-		if live_url:
-			say(f'live page: {live_url}/report/live')
-			if not args.no_open and not open_in_default_browser(f'{live_url}/report/live'):
-				say(f'no browser opened. Open {live_url}/report/live to watch the runs.')
-		else:
-			say(f'the live page did not start, see {run_dir / "live-server.log"}')
-
-	instances, lock = [], threading.Lock()
-	keep = args.keep_instances
-	pool = ThreadPoolExecutor(max_workers=max(1, args.concurrency))
-	# Shared instances start at once. Isolated instances start ahead of their scenario, with at most one
-	# more alive than the number of scenarios that run at the same time.
-	shared_pool = ThreadPoolExecutor(max_workers=3)
-	boot_pool = ThreadPoolExecutor(max_workers=args.concurrency + 1)
-	alive = threading.BoundedSemaphore(args.concurrency + 1)
-	futures, shared = [], {}
-
-	def start_instance(name, root):
-		inst = Instance(name, root, run_dir / 'instances' / name, run_dir / 'instances' / f'{name}.log',
-		                plan.get('featureFlags'))
-		with lock:
-			instances.append(inst)
-		started = time.time()
-		inst.start()
-		# The port only: Cursor opens an editor tab for each new localhost URL that a command prints.
-		say(f'{name} instance ready in {time.time() - started:.0f}s on port {inst.port}')
-		return inst
-
-	def boot_isolated(name):
-		alive.acquire()
-		try:
-			return start_instance(name, REPO_ROOT)
-		except Exception:
-			alive.release()
-			raise
-
-	def run_on(name, scenario, role, instance):
-		"""Run one job when its instance is ready. instance is a future of an Instance, or a URL."""
-		progress.run(name, status='starting')
-		try:
-			inst = instance.result() if hasattr(instance, 'result') else None
-		except Exception as e:
-			progress.run(name, status='error', finishedAt=time.time(), reason=f'The instance did not start: {e}')
-			return
-		url = inst.url if inst else instance
-		try:
-			run_job(name, scenario, url, role.split('-')[0], run_dir, progress, live)
-			if role == 'master' and not args.no_cache:
-				save_master_run(run_dir / 'runs' / name,
-				                master_key(scenario, progress.data['master']['commit'], plan.get('featureFlags')))
-		finally:
-			if role == 'isolated':
-				if not keep:
-					inst.stop(clean=True)
-				alive.release()
-
-	def submit(name, scenario, role, instance):
-		futures.append(pool.submit(run_on, name, scenario, role, instance))
-
-	def skip(role, reason):
-		for name, _, r in jobs:
-			if r == role and progress.data['runs'][name]['status'] == 'waiting':
-				progress.run(name, status='skipped', reason=reason)
-
-	master_jobs = [(name, s) for name, s, r in jobs if r == 'master']
-	base_commit = None
-	if compare_master:
-		try:
-			base_commit = baseline.target(fetch=not args.no_fetch)
-		except subprocess.CalledProcessError as e:
-			say(f'cannot find the master merge base: {e}', 'master')
-		if base_commit:
-			progress.update(master={'commit': base_commit})
-			if not args.no_cache:
-				left = []
-				for name, s in master_jobs:
-					if reuse_master_run(run_dir / 'runs' / name, master_key(s, base_commit, plan.get('featureFlags'))):
-						now = time.time()
-						progress.run(name, status='passed', startedAt=now, finishedAt=now)
-						say(f'master run kept from an earlier run of {base_commit[:11]}', name)
-					else:
-						left.append((name, s))
-				master_jobs = left
-
-	def master_side():
-		try:
-			built = baseline.prepare(fetch=False, say=lambda m: say(m, 'master'))
-			progress.update(master={'commit': built['commit']})
-			inst = start_instance('master', built['root'])
-			for name, s in master_jobs:
-				submit(name, s, 'master', inst.url)
-		except Exception as e:
-			say(f'master is not available: {e}', 'master')
-			skip('master', f'Master is not available: {e}')
-
-	visual_error = None
-	try:
-		if build:
-			step('Build this checkout')
-			build_branch(run_dir / 'build.log')
-		elif not args.branch_url:
-			stale = stale_packages()
-			if stale:
-				say('warning: these packages have source changes that are newer than their build: '
-				    f'{", ".join(stale)}. The fresh instances run the old build.')
-				progress.update(stale=stale)
-
-		step(f'Run {len(jobs)} scenario runs, {args.concurrency} at a time')
-		progress.update(step='Running the scenarios')
-		for role in ('branch', 'branch-visual'):
-			if any(r == role for _, _, r in jobs):
-				shared[role] = args.branch_url or shared_pool.submit(start_instance, role, REPO_ROOT)
-		boots = {name: boot_pool.submit(boot_isolated, name) for name, _, r in jobs if r == 'isolated'}
-		master_thread = None
-		if compare_master and master_jobs and base_commit:
-			master_thread = threading.Thread(target=master_side, daemon=True)
-			master_thread.start()
-		elif compare_master and not base_commit:
-			skip('master', 'The master merge base is not known.')
-		for name, s, role in jobs:
-			if role == 'isolated':
-				submit(name, s, role, boots[name])
-			elif role in shared:
-				submit(name, s, role, shared[role])
-		if master_thread:
-			master_thread.join()
-		# Jobs can be added while the pool runs, so wait until the list stops growing.
-		while any(not f.done() for f in list(futures)):
-			time.sleep(1)
-		for f in futures:
-			if f.exception():
-				say(f'a run failed: {f.exception()}')
-		pool.shutdown()
-		if not keep:
-			for inst in instances:
-				inst.stop(clean=True)
-
-		names = [name for name, _, _ in jobs]
-		if compare_master:
-			step('Compare the visual captures with master')
-			progress.update(step='Comparing the captures with master')
-			diff = subprocess.run(['node', 'visual_diff.mjs', str(run_dir)], cwd=HERE)
-			if diff.returncode != 0:
-				visual_error = f'visual_diff.mjs failed with exit code {diff.returncode}. The visual check is not complete.'
-				say(f'error: {visual_error}')
-				progress.update(visualError=visual_error)
-			trees = [r for r in compare_trees(run_dir) if r.get('ariaDiff')]
-			if trees:
-				say(f'{len(trees)} captures with a difference in the accessibility tree: '
-				    + ', '.join(f'{r["scenario"]}/{r["name"]}' for r in trees))
-
-		step('Summarize the runs for the judge')
-		summary = report.summarize(run_dir)
-		if visual_error:
-			summary = f'VISUAL CHECK FAILED: {visual_error}\n\n{summary}'
-		(run_dir / 'summary.txt').write_text(summary)
-		if not args.try_mode:
-			step('Build the report')
-			progress.update(step='Building the report')
-			report.build(run_dir)
-		progress.finish()
-	finally:
-		pool.shutdown(wait=False, cancel_futures=True)
-		boot_pool.shutdown(wait=False, cancel_futures=True)
-		shared_pool.shutdown(wait=False, cancel_futures=True)
-		for proc in list(RUNNERS):
-			proc.kill()
-		if not keep:
-			for inst in instances:
-				inst.stop(clean=True)
-		progress.finish()
-
-	runs = progress.data['runs']
-	visual_results = json.loads((run_dir / 'visual.json').read_text()) if (run_dir / 'visual.json').exists() else []
-	not_passed = [n for n, e in runs.items() if e['status'] != 'passed' and not n.endswith('--master')]
-	rerun = f'python3 {S}/run.py --rerun {run_dir}' + (f' --only {",".join(not_passed)}' if not_passed else '')
-	usage_later = not args.try_mode and bool(setting('LANGSMITH_API_KEY'))
-	say('\n=== Result ===')
-	say(result_table(run_dir, runs))
-	if not_passed:
-		say(f'Run the ones that did not pass again: {rerun}')
-	print(json.dumps({
-		'runDir': str(run_dir),
-		'report': None if args.try_mode else str(run_dir / 'report' / 'index.html'),
-		'live': f'{live_url}/report/index.html' if live_url else None,
-		'summary': str(run_dir / 'summary.txt'),
-		'results': {name: runs[name]['status'] for name in runs},
-		'visual': {'captures': len(visual_results),
-		           'changed': [f'{r["scenario"]}/{r["name"]}' for r in visual_results if r['status'] != 'unchanged'],
-		           'error': visual_error},
-		'instancePorts': {i.name: i.port for i in instances} if keep else None,
-		'rerun': rerun,
-		'usage': 'The token usage comes in the next lines. The report updates when it is done.' if usage_later else None,
-		'next': ('Fix the scripts that failed, then run --try again, or run the full plan.' if args.try_mode else
-		         'Judge each run from summary.txt and write verdicts.json. The live page rebuilds the report.'),
-	}, indent=2), flush=True)
-
-	if usage_later:
-		step('Fetch the token usage from LangSmith')
-		fetch_usage(run_dir, [n for n in runs])
-		report.build(run_dir)
-		say('usage: done, the report is updated')
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-assistant-ui/scripts/run_scenario.mjs

@@ -1,203 +0,0 @@
-#!/usr/bin/env node
-// Run one scenario script in a browser that is signed in to an n8n instance.
-//
-//   node run_scenario.mjs --out <dir> --script <file.mjs> [--live] [--auto-confirm]
-//                         [--timeout-min 10] [--viewport 1600x1000]
-//
-// N8N_BASE_URL selects the instance. Writes to <dir>:
-//   steps.json            the steps, checks and captures with their screenshots (updated while it runs)
-//   step-NN.png ...       one screenshot for each step, check and capture
-//   captures/<name>.png   screens for the visual check, with <name>.aria.yml
-//   live.png              the screen every 2.5 seconds, for the live page (with --live)
-//   screen.webm           the video, with screen-poster.png
-//   console.json          browser errors and failed requests to the instance
-//   result.json           {status: passed|failed|error, error, threads, prompts, startedAt, finishedAt, ...}
-// Prints "thread <id>" for each thread that the scenario opens, so run.py can read its events.
-import { randomUUID } from 'node:crypto';
-import { copyFileSync, mkdirSync, mkdtempSync, renameSync, rmSync, writeFileSync } from 'node:fs';
-import { tmpdir } from 'node:os';
-import { join, resolve } from 'node:path';
-import { pathToFileURL } from 'node:url';
-import { parseArgs } from 'node:util';
-
-import {
-	baseUrl,
-	launchBrowser,
-	selectors,
-	setting,
-	signIn,
-} from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
-import { StepFailed, createKit } from './kit.mjs';
-
-const { values: args } = parseArgs({
-	options: {
-		out: { type: 'string' },
-		script: { type: 'string' },
-		live: { type: 'boolean', default: false },
-		'auto-confirm': { type: 'boolean', default: false },
-		'timeout-min': { type: 'string', default: '10' },
-		viewport: { type: 'string', default: '1600x1000' },
-	},
-});
-const out = resolve(args.out);
-mkdirSync(out, { recursive: true });
-const [width, height] = args.viewport.split('x').map(Number);
-const size = { width, height };
-const LIVE_SHOT_MS = 2500;
-const emit = (line) => console.log(line.replace(/\s+/g, ' ').slice(0, 300));
-
-/**
- * Sign in through the REST API, which is about 1.3 seconds faster than the sign-in page. The auth cookie
- * is bound to the browser-id header of the login, so the page gets the same id in localStorage.
- * Falls back to the sign-in page.
- */
-async function signedInState(browser) {
-	const login = await browser.newContext();
-	try {
-		const browserId = randomUUID();
-		const res = await login.request.post(`${baseUrl}/rest/login`, {
-			headers: { 'browser-id': browserId },
-			data: {
-				emailOrLdapLoginId: setting('N8N_EVAL_EMAIL', 'nathan@n8n.io'),
-				password: setting('N8N_EVAL_PASSWORD', 'PlaywrightTest123'),
-			},
-		});
-		if (res.ok()) {
-			const state = await login.storageState();
-			state.origins = [
-				{
-					origin: new URL(baseUrl).origin,
-					localStorage: [{ name: 'n8n-browserId', value: browserId }],
-				},
-			];
-			return state;
-		}
-		await signIn(await login.newPage());
-		return await login.storageState();
-	} finally {
-		await login.close();
-	}
-}
-
-selectors.setTestIdAttribute('data-test-id');
-const browser = await launchBrowser();
-const storageState = await signedInState(browser);
-
-// The video starts after the sign-in, at the first page of the scenario.
-const videoDir = mkdtempSync(join(tmpdir(), 'feature-test-video-'));
-const context = await browser.newContext({
-	viewport: size,
-	storageState,
-	recordVideo: { dir: videoDir, size },
-});
-const page = await context.newPage();
-
-const origin = new URL(baseUrl).origin;
-const consoleErrors = [];
-const requestErrors = [];
-page.on('console', (m) => {
-	if (m.type() === 'error')
-		consoleErrors.push({ text: m.text().slice(0, 1000), url: m.location()?.url });
-});
-page.on('pageerror', (e) => consoleErrors.push({ text: `Uncaught: ${e.message}`.slice(0, 1000) }));
-page.on('response', (r) => {
-	if (r.status() >= 400 && r.url().startsWith(origin)) {
-		requestErrors.push({
-			status: r.status(),
-			method: r.request().method(),
-			url: r.url().slice(origin.length),
-		});
-	}
-});
-page.on('requestfailed', (r) => {
-	const failure = r.failure()?.errorText ?? '';
-	// Closed event streams and aborted navigations are normal.
-	if (r.url().startsWith(origin) && !/ERR_ABORTED|NS_BINDING_ABORTED/.test(failure)) {
-		requestErrors.push({
-			status: 0,
-			method: r.method(),
-			url: r.url().slice(origin.length),
-			error: failure,
-		});
-	}
-});
-
-let shooting = false;
-async function liveShot() {
-	if (shooting) return;
-	shooting = true;
-	const partial = join(out, 'live.tmp.png');
-	try {
-		await page.screenshot({ path: partial });
-		renameSync(partial, join(out, 'live.png'));
-	} catch {
-		// The page can be in the middle of a navigation. The next interval tries again.
-	} finally {
-		shooting = false;
-	}
-}
-const liveTimer = args.live ? setInterval(liveShot, LIVE_SHOT_MS) : null;
-
-const { t, threads, prompts, steps, stop } = createKit({
-	page,
-	out,
-	baseUrl,
-	autoConfirm: args['auto-confirm'],
-	emit,
-});
-const result = { status: 'passed', startedAt: new Date().toISOString(), baseUrl };
-const timeoutMs = Number(args['timeout-min']) * 60_000;
-let timer;
-try {
-	const scenario = (await import(pathToFileURL(resolve(args.script)).href)).default;
-	if (typeof scenario !== 'function')
-		throw new Error('The script must export a default async function (t).');
-	await Promise.race([
-		scenario(t),
-		new Promise((_, reject) => {
-			timer = setTimeout(() => {
-				// The race does not end the scenario function. stop() makes its next step throw.
-				stop();
-				reject(new Error(`The scenario did not finish in ${args['timeout-min']} minutes.`));
-			}, timeoutMs);
-		}),
-	]);
-	if (steps.some((s) => s.status === 'failed')) result.status = 'failed';
-} catch (error) {
-	result.status = error instanceof StepFailed || steps.length ? 'failed' : 'error';
-	result.error = String(error?.message ?? error)
-		.replace(/\u001b\[[0-9;]*m/g, '')
-		.slice(0, 2000);
-	if (!(error instanceof StepFailed)) emit(`error: ${result.error.split('\n')[0]}`);
-} finally {
-	clearTimeout(timer);
-}
-
-stop();
-clearInterval(liveTimer);
-for (const s of steps.filter((x) => x.status === 'running')) {
-	s.status = 'failed';
-	s.error = 'The scenario stopped during this step.';
-}
-writeFileSync(join(out, 'steps.json'), JSON.stringify(steps, null, 1));
-await page.waitForTimeout(1500);
-if (args.live) await liveShot();
-await page.screenshot({ path: join(out, 'screen-poster.png') }).catch(() => {});
-const video = page.video();
-await context.close();
-if (video) copyFileSync(await video.path(), join(out, 'screen.webm'));
-await browser.close();
-rmSync(videoDir, { recursive: true, force: true });
-
-Object.assign(result, {
-	finishedAt: new Date().toISOString(),
-	threads,
-	prompts,
-	consoleErrors: consoleErrors.length,
-	requestErrors: requestErrors.length,
-});
-writeFileSync(join(out, 'console.json'), JSON.stringify({ consoleErrors, requestErrors }, null, 1));
-writeFileSync(join(out, 'result.json'), JSON.stringify(result, null, 2));
-emit(`result ${result.status}`);
-// A scenario that timed out can still wait on its own timers. The results are written, so end here.
-process.exit(0);

.agents/skills/testmax-assistant-ui/scripts/visual_diff.mjs

@@ -1,96 +0,0 @@
-#!/usr/bin/env node
-// Compare the captures of each visual scenario: this branch against master.
-//
-//   node visual_diff.mjs <run-dir>
-//
-// Reads runs/<id>/captures/*.png (branch) and runs/<id>--master/captures/*.png (master) for each
-// visual scenario in plan/scenarios.json. Writes visual/<id>/<name>.diff.png and visual.json:
-//   [{scenario, name, status: unchanged|changed|branch-only|master-only, changedPixels, ratio,
-//     branchSize, masterSize, diff}]
-// A pixel counts as changed when one color channel differs by more than PIXEL_THRESHOLD. A capture
-// counts as changed when more than RATIO_THRESHOLD of its pixels changed, or its size changed.
-import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
-import { join, resolve } from 'node:path';
-
-import { loadPng } from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
-
-const PIXEL_THRESHOLD = 32;
-const RATIO_THRESHOLD = 0.0005;
-const PNG = loadPng();
-const runDir = resolve(process.argv[2] ?? '.');
-const plan = JSON.parse(readFileSync(join(runDir, 'plan', 'scenarios.json'), 'utf8'));
-
-const read = (file) => PNG.sync.read(readFileSync(file));
-const names = (dir) =>
-	existsSync(dir)
-		? readdirSync(dir)
-				.filter((f) => f.endsWith('.png'))
-				.map((f) => f.slice(0, -4))
-		: [];
-
-function compare(branch, master) {
-	const width = Math.max(branch.width, master.width);
-	const height = Math.max(branch.height, master.height);
-	const diff = new PNG({ width, height });
-	let changed = 0;
-	for (let y = 0; y < height; y++) {
-		for (let x = 0; x < width; x++) {
-			const o = (y * width + x) * 4;
-			const inB = x < branch.width && y < branch.height;
-			const inM = x < master.width && y < master.height;
-			const b = inB ? (y * branch.width + x) * 4 : -1;
-			const m = inM ? (y * master.width + x) * 4 : -1;
-			let differs = !inB || !inM;
-			if (!differs) {
-				for (let c = 0; c < 4 && !differs; c++)
-					differs = Math.abs(branch.data[b + c] - master.data[m + c]) > PIXEL_THRESHOLD;
-			}
-			if (differs) {
-				changed++;
-				diff.data.set([224, 17, 95, 255], o);
-			} else {
-				// The unchanged page shows as a faint gray copy of the branch screen.
-				const gray =
-					0.299 * branch.data[b] + 0.587 * branch.data[b + 1] + 0.114 * branch.data[b + 2];
-				const v = Math.round(255 - (255 - gray) * 0.25);
-				diff.data.set([v, v, v, 255], o);
-			}
-		}
-	}
-	return { diff, changed, ratio: changed / (width * height) };
-}
-
-const results = [];
-for (const s of plan.scenarios.filter((x) => x.kind === 'visual')) {
-	const branchDir = join(runDir, 'runs', s.id, 'captures');
-	const masterDir = join(runDir, 'runs', `${s.id}--master`, 'captures');
-	const all = [...new Set([...names(branchDir), ...names(masterDir)])].sort();
-	for (const name of all) {
-		const entry = { scenario: s.id, name };
-		const hasB = existsSync(join(branchDir, `${name}.png`));
-		const hasM = existsSync(join(masterDir, `${name}.png`));
-		if (!hasB || !hasM) {
-			results.push({ ...entry, status: hasB ? 'branch-only' : 'master-only' });
-			continue;
-		}
-		const branch = read(join(branchDir, `${name}.png`));
-		const master = read(join(masterDir, `${name}.png`));
-		const { diff, changed, ratio } = compare(branch, master);
-		const out = join(runDir, 'visual', s.id);
-		mkdirSync(out, { recursive: true });
-		writeFileSync(join(out, `${name}.diff.png`), PNG.sync.write(diff));
-		const resized = branch.width !== master.width || branch.height !== master.height;
-		results.push({
-			...entry,
-			status: resized || ratio > RATIO_THRESHOLD ? 'changed' : 'unchanged',
-			changedPixels: changed,
-			ratio,
-			branchSize: [branch.width, branch.height],
-			masterSize: [master.width, master.height],
-			diff: `visual/${s.id}/${name}.diff.png`,
-		});
-	}
-}
-writeFileSync(join(runDir, 'visual.json'), JSON.stringify(results, null, 2));
-const changed = results.filter((r) => r.status !== 'unchanged').length;
-console.log(`${results.length} captures compared, ${changed} with a difference`);

.agents/skills/testmax-n8n-ai/SKILL.md

@@ -1,745 +0,0 @@
----
-name: n8n:testmax-n8n-ai
-description: >-
-  Runs one workflow prompt through Instance AI, Claude Code with the n8n MCP
-  server, and Codex with the n8n MCP server on a local n8n instance. Collects
-  the traces, per-call tokens, cost, and canvas screenshots, judges each run
-  against the expected outcome, and builds one HTML comparison page. Claude
-  Code and Codex run in a sandbox that sees only the n8n MCP server. Use when
-  the user wants to compare builders, test a prompt across Instance AI and
-  MCP, or says /testmax-n8n-ai.
----
-
-# Compare workflow builders
-
-This skill sends the same prompt to up to three builders on the local n8n
-instance. The builders run at the same time, and a local page shows them
-live. It then builds one page that shows, for each run:
-
-- the result against the expected outcome (pass, partial or fail, with reasons
-  and the evidence that a panel of blind judges cited)
-- deterministic checks of each workflow: the node definitions, the node
-  issues in the n8n editor, the graph, the expression references, and a run
-  of each branch with pinned data
-- a separate n8n standards score, from the rules in
-  [n8n-io/skills](https://github.com/n8n-io/skills) and the workflow-sdk best
-  practices. This score does not change the verdict
-- the canvas of the workflow that the builder made
-- a replay of the run: the exact terminal screen, or a video of the n8n UI
-- the cost
-- the LLM steps: one entry for each LLM call, with its tokens (cache read,
-  cache write, uncached input, output) and the agent text, reasoning and tool
-  calls (with input and output) that the call made
-
-The page uses the light mode of the
-[n8n brand guidelines](https://guidelines.n8n.design/).
-
-![An example comparison page](assets/comparison-page.png)
-
-All scripts are in `scripts/` next to this file. They use only the Python
-standard library and the Playwright copy in the repository.
-
-```bash
-S="$(git rev-parse --show-toplevel)/.agents/skills/testmax-n8n-ai/scripts"
-```
-
-## Settings
-
-All settings are in `.env.ai-surfaces` in the repository root. Git ignores
-this file. The template is `.env.ai-surfaces.example`, also in the
-repository root. The scripts read,
-from low to high priority: `.env.local`, `.env.ai-surfaces`, the shell
-environment. Empty values are ignored. Thus keys that are already in
-`.env.local` work without a change.
-
-If `.env.ai-surfaces` does not exist, copy `.env.ai-surfaces.example` to it. Tell the user
-which values to fill in. Never read the values back to the user, never print
-them, and never put a secret on a command line.
-
-The most important settings:
-
-| Setting | Use |
-|---|---|
-| `COMPARE_BUILDERS` | Default builders, for example `iai,claude,codex` |
-| `COMPARE_PARALLEL` | `1` (default) runs the builders at the same time. `0` runs them one at a time |
-| `COMPARE_LIVE` | `1` (default) opens a live page of the runs. `0` turns it off |
-| `COMPARE_LIVE_WINDOWS` | `1` also opens the Instance AI thread and a terminal window for each CLI. Default `0` |
-| `COMPARE_RECORD` | `1` (default) records the Instance AI video for the replay. `0` turns it off |
-| `COMPARE_CLI_MODE` | `interactive` (default) records the real CLI screen. `headless` uses the JSON event stream |
-| `COMPARE_RECORD_REQUESTS` | `1` (default) records the Codex request bodies, without headers, for the tool list. `0` turns it off |
-| `COMPARE_SCENARIOS` | `1` (default) runs each workflow with pinned data through the n8n MCP server. `0` turns it off |
-| `COMPARE_WIRE` | `1` (default) runs each workflow in process with recorded API fixtures (`run_wire.mjs`). `0` turns it off |
-| `COMPARE_BINARY_CHECKS` | `1` (default) runs the deterministic binary checks of the Instance AI eval suite. `0` turns it off |
-| `GOOGLE_GENERATIVE_AI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `MISTRAL_API_KEY`, `GROQ_API_KEY` | Model keys for the real AI runs. A workflow model without a key skips those runs |
-| `CLAUDE_MODEL`, `CODEX_MODEL` | Model ids. Empty uses the CLI default. A prefix such as `anthropic/` is removed |
-| `COMPARE_CLAUDE_TOOLS` | `none` (default) gives Claude Code only the n8n tools. `default` also gives it its built-in tools, all allowed (see "How the builders run") |
-| `CLAUDE_API_KEY` | Anthropic key for Claude Code. Falls back to `ANTHROPIC_API_KEY` |
-| `CODEX_API_KEY` | OpenAI key for Codex. Falls back to `OPENAI_API_KEY` |
-| `N8N_MCP_TOKEN` | MCP key of the n8n user. Optional |
-| `LANGSMITH_*` | Instance AI cost and tokens |
-
-## 1. Ask the user
-
-Ask for these two items. If the harness has a structured question tool, use
-it. If not, ask in chat. Do not guess them.
-
-1. **Prompt.** The exact text to send to each builder.
-2. **Expected outcome.** What a correct result does. Ask for concrete checks,
-   for example "WhatsApp trigger", "Gemini compares the question to the FAQ
-   sheet", "the false branch makes a Notion ticket and a Slack message".
-
-Use the builders and models from the settings. Ask about them only if the
-user wants a change for this run.
-
-Write one expected outcome item on each line, numbered or bulleted. The
-number of an item is its id: the judges cite it, and the page groups the
-checks by it.
-
-Then write the scenarios for the pinned-data runs, from the expected
-outcome. You do not need to ask the user for them. A scenario sets the data
-of the trigger, the service nodes and the AI nodes, and says which nodes
-must or must not run. Logic nodes (IF, Switch, Set, Code, Aggregate and
-others) run for real. Make one scenario for each branch of the expected
-outcome:
-
-```json
-{
-  "pinByType": {
-    "googleSheets": [{ "Question": "What are your opening hours?", "Answer": "9:00 to 17:00" }]
-  },
-  "scenarios": [
-    { "name": "The FAQ answers the question", "ai": { "booleans": true },
-      "expect": { "reach": ["whatsApp"], "skip": ["notion", "slack"] } },
-    { "name": "The FAQ does not answer the question", "ai": { "booleans": false },
-      "expect": { "reach": ["notion", "slack"] } }
-  ]
-}
-```
-
-- `pinByType`: output items for every node of a type, in all scenarios. A
-  type is the full name or the part after the last dot. A scenario can have
-  its own `pinByType`.
-- `trigger`: output items of the trigger. Without it, the trigger gets a
-  fixture in the shape that the trigger code emits (WhatsApp, Webhook, Chat,
-  Schedule, Telegram, Form and Manual triggers have one).
-- `ai`: the output of each AI node. `booleans` sets every boolean in the
-  output, `set` sets fields, and `text` sets the whole text of a node
-  without a structured output parser. The output follows the schema of the
-  structured output parser, or the JSON example in the prompt of the node.
-- `expect`: `reach` lists node types that must run without an error,
-  `skip` lists node types that must not run.
-- `triggerNode`: the name of the trigger that the scenario starts from, for a
-  workflow with more than one trigger. Without it, the scenario starts from
-  the trigger that reaches the most `reach` types.
-
-Without a scenarios file, `run_scenarios.py` makes one scenario for each
-value of the AI booleans, with no expectations.
-
-`run_wire.mjs` runs the same scenarios in process. There, service nodes
-send real requests that recorded fixtures answer, so a scenario can also
-check the API contract. Add these keys to the scenarios file:
-
-```json
-{
-  "scenarios": [
-    { "name": "The FAQ does not answer the question", "ai": { "booleans": false },
-      "expect": { "reach": ["notion", "slack"], "requests": [
-        { "type": "notion", "method": "POST", "contains": ["\"title\"", "@trigger.messages[0].from"] },
-        { "type": "slack", "method": "POST",
-          "contains": ["@trigger.messages[0].text.body", ["@response.notion.id", "@response.notion.url"]] } ] } }
-  ],
-  "edge": [
-    { "name": "The FAQ sheet is empty", "pinByType": { "googleSheets": [] }, "ai": { "booleans": false },
-      "expect": { "reach": ["notion", "slack"] } }
-  ],
-  "questions": [
-    { "name": "Opening hours, which the FAQ answers", "message": "Hi! What time do you open?",
-      "expect": { "skip": ["notion"], "requests": [
-        { "type": "whatsApp", "method": "POST", "contains": ["@trigger.messages[0].from", "9:00"] } ] } }
-  ]
-}
-```
-
-- `expect.requests`: each entry needs a request from a node of `type`,
-  optionally with `method` and an exact `count`, whose URL, query or body
-  contains every value in `contains`. An array in `contains` means any one
-  of its values. `@trigger.<path>` is a value of the trigger item.
-  `@response.<type>.<path>` is a value of the response to the last write
-  request of that node type, for example the id of the Notion page.
-- `expect.noWrites`: the scenario must send no write request.
-- `edge`: inputs that the prompt does not describe. The trigger kits in
-  `sdk.mjs` add standard edge cases, for example two WhatsApp messages in one
-  webhook call (each sender must get a request) and a status update (no
-  writes). Edge cases are a score, not part of the verdict.
-- `questions`: the AI nodes call the real model with this customer
-  `message`. The trigger fixture carries the message. A real-AI run needs a
-  model key, and is a score, not part of the verdict.
-
-## 2. Run the builders
-
-```bash
-python3 $S/compare.py --title "<short title>" \
-  --prompt-file <file with the prompt> --expected-file <file with the outcome> \
-  --scenarios-file <file with the scenarios>
-```
-
-Write the prompt, the expected outcome and the scenarios to files with the
-file tool first. Do not put them on the command line, because they can
-contain quotes. Use `--builders iai,claude` to change the builders for one
-run. Use `--rerun <old run dir>` to run the same case again. It also copies
-`scenarios.json`.
-
-The command takes 5 to 30 minutes. Start it in the background if the harness
-supports it, and read its output until it ends. It prints one line for each
-tool call, with the builder name in front, so the user can follow the run.
-It does these steps:
-
-1. Checks the setup and skips a builder that cannot run. It prints the
-   reason.
-2. Starts the live page (see below).
-3. Runs the builders at the same time. `--sequential` or
-   `COMPARE_PARALLEL=0` runs them one at a time.
-4. Fetches the Instance AI cost from LangSmith.
-5. Takes a screenshot of each canvas, and saves the node issues that the
-   n8n editor shows (`canvas-issues.json`).
-6. Runs each workflow once for each scenario, with pinned data, through the
-   `test_workflow` tool of the n8n MCP server (`exec.json`). It turns on
-   "Available in MCP" for a workflow that does not have it.
-7. Runs each workflow in process with recorded API fixtures, for the
-   scenarios, the edge cases and the real AI questions (`wire.json`).
-8. Runs the deterministic eval binary checks (`binary-checks.json`) and
-   checks the final-message claims (`claims.json`).
-9. Derives the deterministic checks (`auto-checks.json`). See "Checks on
-   the page".
-10. Writes `summary.txt`, and `judge/summary-blind.txt` for the judges.
-11. Builds the page without verdicts.
-
-The last output is JSON with `runDir`, `report`, `live`, `parallel`,
-`summary`, `blindSummary`, `models` and `skipped`.
-
-Parallel runs share one n8n instance and one machine. Thus the times of the
-builders include the shared load. The page says "run in parallel" under
-the title. Use
-`--sequential` when the user wants to compare times exactly.
-
-### Watch the runs
-
-`compare.py` starts `live_server.py` and opens its page in the default
-browser. It prints `live page: <url>`. Give the user this URL. The page
-shows each builder while it runs, and updates every 2 seconds without a
-reload:
-
-- the table: status, time, LLM calls, tool calls and tokens of each builder
-- **Live:** the real terminal screen of Claude Code and Codex, streamed from
-  the tmux recording, and a screenshot of the Instance AI thread every
-  2.5 seconds
-- the LLM steps of each builder. Claude Code and Codex steps come from the
-  session files of the CLIs, with tokens. Instance AI steps come from its
-  event stream, without tokens until the run ends
-
-The cost, the canvases and the Instance AI tokens come after the run. Until
-then, the Expected outcome box shows "Not verified yet", and the
-Correctness, Execution and n8n standards boxes show "Not run yet". When
-the report is ready, the page changes to it. The server listens on
-`127.0.0.1` only. It stops 30 minutes after the last request, when the run
-is done.
-
-The report shows the price and judge steps (steps 3 and 4) as queued until
-you start them. The Result column shows "Judge queued" or "Judging", and
-the Cost column of a CLI builder without a price shows "Price queued" or
-"Finding price". Each `build_report.py --step` call builds the page again.
-While a step is queued or running, the page on the live server reloads
-itself when the page changes. A page that you open from the file needs a
-manual reload.
-
-Headless mode and clean isolation have no terminal screen. For them, the
-page shows only the table and the LLM steps. `COMPARE_RECORD=0` removes the
-Instance AI screenshot.
-
-`COMPARE_LIVE_WINDOWS=1` also opens the old live views:
-
-- **Instance AI:** the thread opens in the default browser. The browser must
-  be logged in as `N8N_EVAL_EMAIL`, because a thread is visible only to its
-  owner. The script answers the confirmation cards, so the user must not
-  click them.
-- **Claude Code and Codex:** a terminal window shows the real screen of the
-  CLI, read-only (`docker exec … tmux attach -r`). In headless mode, the
-  window follows `<builder>/live.log` instead. On macOS the window opens in
-  Terminal. The first time, macOS asks if the agent's app can control
-  Terminal.
-
-Use `--no-live` or `COMPARE_LIVE=0` for runs without a display, for example
-in CI.
-
-### Replay on the page
-
-The Replay section shows the last screen of each builder. The play button
-on a screen plays that run in place on the page. **Expand** opens a
-full-screen player. In that player, the user sees all builders side by
-side, or picks one builder, and plays the runs at 1× to 8×:
-
-- **Claude Code and Codex:** `<builder>/screen.cast` is an asciinema
-  recording of the exact terminal screen. The page plays it with the
-  asciinema player.
-- **Instance AI:** `iai/screen.webm` is a video of the thread in the n8n UI.
-  A headless browser records it while the run continues.
-
-`COMPARE_RECORD=0` skips the Instance AI video. The terminal recordings come
-with interactive mode. `COMPARE_CLI_MODE=headless` has no terminal
-recording.
-
-### Canvas on the page
-
-`capture_canvas.mjs` expands the node groups, zooms to fit, and crops the
-screenshot to the nodes, their labels and their subtitles. Empty click areas
-and hidden hover toolbars do not change the crop. The page shows the canvas
-next to the expected outcome, at its real size and at most 360 px high.
-
-A click on the canvas opens a full-screen viewer. It has a tab for each run
-that has a canvas, the verdict and the n8n link of the run, and a zoom
-button. The left and right arrow keys switch runs. A click on the image
-zooms in to that point. Esc closes the viewer, and the page then shows the
-last run that the user looked at. Cmd-click or Ctrl-click still opens the
-image in a new tab.
-
-### Checks on the page
-
-Under the expected outcome and the canvas, each run has sections from
-`derive_checks.mjs`. No LLM makes these checks, except the model calls of
-the real AI runs. Correctness, Execution with pinned data and Service calls
-change the verdict. The other sections show a score only.
-
-**Correctness.** A failed check changes the verdict.
-
-| Id | Check | Source |
-|---|---|---|
-| C1 | Node parameters match the node definitions | `validateWorkflow` of `@n8n/workflow-sdk`, with the built node definitions |
-| C2 | The n8n editor shows no node errors | The issues of each node in the editor. Missing credentials and empty required pickers are setup for the user (C2s, information only) |
-| C3 | Every node can be reached from a trigger | The connections |
-| C4 | Each `$('Node')` names a node that exists and runs before the node that reads it | The expressions |
-| C5 | Each expression field is in the output of the node that it reads | The real output of the pinned-data runs |
-| C6 | No keys or tokens in node parameters | The parameters |
-
-C5 fails only when the output is real: a node that ran, the trigger
-fixture, the case data, or the schema of the structured output parser. A
-field that is missing only from a node schema sample or a prompt example
-shows as "Unconfirmed".
-
-**Execution with pinned data.** One check for each scenario that ran without
-an error, and one for each `reach` or `skip` expectation. A failed check
-changes the verdict.
-
-**Service calls.** `run_wire.mjs` loads the n8n execution engine from the
-built `packages/core` and runs each workflow in process. Triggers and AI
-nodes get pinned data. Service nodes run their real code, and the core
-eval hook answers each HTTP request with a recorded fixture before any
-credential is used. The process has no sandbox, and the hook sees only the
-requests that go through the n8n request helpers. Thus each node that runs
-must be on an allow-list (`runsInProcess` in `run_wire.mjs`):
-
-- logic nodes from `LOGIC_TYPES`, HTTP Request, Wait and Telegram
-- declarative nodes
-- nodes whose credentials all authenticate HTTP requests (an `authenticate`
-  block or OAuth)
-- a fixed set of AI nodes, with models that have a key in `MODEL_CREDENTIALS`
-
-Code, Function and AI Transform nodes, and nodes with database, SSH, FTP,
-SMTP, file or shell access, are not on the list. Each scenario checks the
-nodes that run from its trigger. Pinned nodes do not run, so a node under
-a pinned AI node counts only in the real AI runs. If a node outside the list
-would run, the scenario is skipped. The report shows it as skipped, not
-failed, with the names of the nodes.
-The pinned-data runs still run Code nodes in the n8n task runner. No request
-leaves the machine, except the model calls of the real AI runs. Empty pickers get
-`mock-<parameter>` values for the run. Each scenario row lists the nodes
-that ran with their item counts, and each request. A failed check changes
-the verdict.
-
-**Edge cases** and **Real AI runs.** The same in-process runs for the
-`edge` and `questions` entries of the scenarios file. A score.
-
-**Final message claims.** `claims.py` compares the final message with the
-trace and the workflow. A score.
-
-| Id | Claim | Source |
-|---|---|---|
-| M1 | The workflow id or URL is a workflow of this run | `run.json` |
-| M2 | The publish state (draft or active) | `workflow.json` `active` |
-| M3 | The workflow passed validation | The last validation tool call |
-| M4 | The workflow was tested, or was not run | Test and execution tool calls. Scripts that report passed tests count, with a note that they ran outside n8n |
-| M5 | Values in `code spans` | The workflow, the prompt and the tool calls |
-| M6 | A claim that the workflow was checked agrees with the service-call runs | `wire.json` |
-
-**Eval binary checks.** `binary_checks.mts` runs the deterministic binary
-checks of the Instance AI eval suite
-(`packages/@n8n/instance-ai/evaluations/binaryChecks`), for example
-`no_empty_set_nodes` and `no_unnecessary_code_nodes`. Checks that do not
-apply are left out. A score.
-
-**Eval quality checks.** The judges answer the LLM binary checks of the
-same suite: `valid_data_flow`, `correct_node_operations`,
-`handles_multiple_items`, `descriptive_node_names` and
-`response_describes_changes_accurately`. `summarize_runs.py` reads their
-rules from the check sources. A score.
-
-**n8n standards.** A score, not part of the verdict. The builders do not get
-these rules: Claude Code and Codex run without the n8n skills, and Instance
-AI uses its own prompts. Only the rules that apply to the workflow count.
-
-| Id | Rule | Source |
-|---|---|---|
-| S1 | An unattended workflow has an error workflow | `n8n-error-handling-official` |
-| S12 | A failed node goes to a wired error output, not on as data | `n8n-error-handling-official` |
-| S2 | Nodes that call a service have `retryOnFail` | `n8n-error-handling-official` |
-| S3 | Expressions read data by node name, not `$json` | `n8n-expressions-official` |
-| S4 | Structured AI output uses a manual schema with `autoFix` | `n8n-ai-nodes-official` STRUCTURED_OUTPUT.md |
-| S5 | Messages do not carry the n8n attribution | `n8n-node-configuration-official` COMMS_NODES.md, and the node definitions |
-| S6 | References do not read only the first item of a node that can output several | `n8n-expressions-official`, and the trigger code |
-| S7 | A model that decides a branch has a temperature of 0.2 or lower | workflow-sdk best practices: triage |
-| S8 | AI output that decides a branch has a reasoning field | workflow-sdk best practices: triage |
-| S9 | Every branch after an AI decision, including the fallback, goes somewhere | workflow-sdk best practices: triage |
-| S10 | A chatbot agent has memory | workflow-sdk best practices: chatbot |
-| S11 | The memory session key names a node and identifies the conversation | workflow-sdk best practices: chatbot, and `n8n-ai-nodes-official` MEMORY.md |
-
-The skill rules come from commit `27b6b9f` of n8n-io/skills. Each check
-links to its source.
-
-**The verdict** comes from the checks, not from the judges alone:
-
-- `fail` when there is no workflow, or the judges say that the workflow does
-  not do what the prompt asks (`goalMet: false`)
-- `pass` when every expected outcome item is met, and no correctness,
-  execution or service-call check failed
-- `partial` for all other results
-
-A met item must cite evidence that the workflow contains. Without evidence,
-or with evidence that is not in the workflow, the item shows as not
-confirmed. The Result column shows the verdict and the standards score
-(`Std 5/10`).
-
-An old `verdicts.json` without ids or evidence keeps the verdict of the
-judge. A failed correctness or execution check changes its `pass` to
-`partial`.
-
-### LLM steps on the page
-
-Each run has a list of LLM steps, like the LLM steps view in the Instance AI
-debug panel. The box is collapsed by default. Click its title to open it. On
-the live page, a box that you open stays open when the page updates. The list
-shows the tools or text of each call, its time and a
-token bar. Select a step, or use the arrow keys, to see the token counts and
-what the call did. Step 1 starts with what the model gets before the first
-call, then the user message. Tool output is collapsed, except when the tool
-failed.
-
-- **Claude Code:** the system prompt and the tool definitions, from the
-  request shape in the session file. The tools show as chips in groups:
-  System (the built-in tools of the CLI), one group for each MCP server, and
-  Other (for example tools that the model provider runs). System and MCP
-  show "None" when the model had no such tools. The tools that the run
-  called are highlighted, with a call count and a dot for failures. Select a chip to see the description and a table of
-  the parameters. A filter box searches names and descriptions.
-- **Codex:** the system prompt (base instructions), the developer messages
-  and the environment context, from the session file. Codex does not save
-  its tool definitions, so the runner records them (see "Isolation"). The
-  tools use the same chips. Codex declares most system tools inside the
-  description of its `exec` tool, so they show as "System · in exec" with
-  their TypeScript declaration. Codex does not put MCP tools in the prompt:
-  its scripts look them up in `ALL_TOOLS` at run time. The MCP group thus
-  shows only the MCP tools that the run called, with a dashed border and no
-  schema. Runs from before the recorder show no Codex tools.
-- **Instance AI:** the system prompt, the context that n8n puts before the
-  prompt (for example `<thread-context>`), and the tools, from the first
-  LangSmith LLM call of each agent. `langsmith_usage.py` saves them in
-  `context.json`. Tools that the agent loads later with `load_tool` show as
-  "System · loaded with load_tool". A sub-agent shows its own system prompt,
-  briefing and tools at its first step. Runs from before `context.json` show
-  no Instance AI context: run `langsmith_usage.py` again for them.
-
-- **Claude Code:** the session file links each event to its LLM call.
-- **Codex:** text goes to the response that wrote it. A tool call goes to the
-  response that asked for it. Codex calls the n8n tools from scripts that it
-  runs with its `exec` tool. The page shows these scripts too, but does not
-  count them as tool calls.
-- **Instance AI:** the LangSmith call times place each event. Text times are
-  approximate. An asterisk on a time marks text that the page puts at the
-  previous tool call.
-
-### When a check fails
-
-- Cannot log in: n8n is not running, or `N8N_BASE_URL`, `N8N_EVAL_EMAIL` or
-  `N8N_EVAL_PASSWORD` is wrong.
-- The MCP token does not work: n8n shows the full MCP key only once. Ask the
-  user to put the key in `N8N_MCP_TOKEN`, or ask before you run
-  `python3 $S/prepare.py --rotate`. Rotation makes the old key stop working
-  in every other MCP client of this user.
-- Docker is not running: Codex is skipped, and Claude runs as a clean
-  process.
-
-## 3. Find the prices
-
-Codex reports no cost, and the estimate of Claude Code can be out of date
-for new models. Thus, find the published price of each model that a CLI
-builder used. The models are in `models` in the output of `compare.py`, and
-in `<runDir>/models.json`. Instance AI cost comes from LangSmith, so do not
-look up its model.
-
-Do not skip this step. In interactive mode neither CLI reports a cost, so
-without prices the page shows "not reported" for both. `build_report.py`
-prints `models without a published price in case.json: …` while a model has
-no price.
-
-First, tell the page that you look for the prices:
-
-```bash
-python3 $S/build_report.py <runDir> --step prices=running
-```
-
-For each model:
-
-1. Search the web for the official price page of the provider, for example
-   the Anthropic pricing page or the OpenAI pricing page. Do not use
-   third-party price lists if the provider page has the model.
-2. Read four prices in USD per million tokens: input, cache read, cache
-   write, output. If the provider has no cache-write price, use the input
-   price. Anthropic has two cache-write prices. Use the 5-minute price, and
-   tell the user that the 1-hour price is higher.
-3. If you cannot find the model, write no price for it. The page then shows
-   the estimate of Claude Code, or "not reported" for Codex. Tell the user.
-
-Add the prices to `<runDir>/case.json`, with the URL that you used:
-
-```json
-"prices": {
-  "<model id as in models.json>": {
-    "input": 5, "cached": 0.5, "cacheWrite": 6.25, "output": 25,
-    "source": "https://..."
-  }
-}
-```
-
-The page calculates the cost of each call from its tokens. The tooltip of a
-Cost cell shows the source of the cost, the prices and the URL.
-
-## 4. Judge each run
-
-Show the costs and tell the page that you judge the runs:
-
-```bash
-python3 $S/build_report.py <runDir> --step prices=done --step judge=running
-```
-
-A panel of three judges judges the runs blind. Each judge reads only
-`<runDir>/judge/summary-blind.txt`. In that file the builders are called A,
-B and C in a random order, and the final messages have no builder or model
-names. `judge/blind-map.json` maps the letters back. Do not give a judge
-`summary.txt`, the map, the trace files or this conversation.
-
-If the harness can start sub-agents, start three sub-agents with a fresh
-context, one for each judge. Use a model that is not one of the builder
-models when the harness lets you choose. If the harness cannot start
-sub-agents, judge once yourself from the blind file only, and write
-`verdicts-1.json`.
-
-Give each judge these instructions:
-
-> Read `<runDir>/judge/summary-blind.txt`. It lists the expected outcome
-> criteria with their ids, and for each builder the final message, every
-> node with all its parameters, the connections, and deterministic facts
-> (AUTO-CHECK FACTS). Trust the facts: they come from the node definitions,
-> the n8n editor and real pinned-data runs. For each builder, make one check
-> for each criterion id. A met check must cite evidence: the node name, the
-> parameter path, and the exact value in the workflow. Without evidence,
-> set `ok` to `null`. Set `goalMet` to false when the workflow does not do
-> what the prompt asks, and give the reason in `goalReason`. Then answer
-> each QUALITY CHECK in `quality`, with the check name as the `id`. Write
-> the result to `<runDir>/judge/verdicts-<n>.json`.
-
-The format of each judge file, with the builder letters as keys:
-
-```json
-{
-  "A": {
-    "summary": "One or two sentences about the result.",
-    "goalMet": true,
-    "goalReason": "",
-    "checks": [
-      { "id": "1", "ok": true, "text": "A WhatsApp trigger starts the workflow on `messages`",
-        "evidence": [{ "node": "WhatsApp Message Received", "path": "parameters.updates", "value": "messages" }] },
-      { "id": "9", "ok": false, "text": "Send Reply reads `$json.from`, which does not exist" },
-      { "id": "16", "ok": null, "text": "Could not confirm the sheet columns" },
-      { "ok": false, "text": "The Slack node is not connected" }
-    ],
-    "quality": [
-      { "id": "valid_data_flow", "ok": true, "text": "Every expression reads a field that its source outputs" },
-      { "id": "descriptive_node_names", "ok": false, "text": "Two nodes keep the default names `Code` and `IF`" }
-    ]
-  }
-}
-```
-
-- `id` is the id of the criterion in brackets, as a string. Leave out `id`
-  for a problem that is not a criterion. The page shows it under "Other
-  findings".
-- `evidence` lists `{node, path, value}` entries. `path` is a dotted path in
-  the node, for example `parameters.textBody` or
-  `parameters.conditions.conditions[0].leftValue`. The page checks that the
-  node has the path and that the path contains the value. An entry with
-  `"source": "message"` and a `value` quotes the final message and is not
-  checked.
-- Write each check as one short sentence with one fact. Put expressions,
-  field paths and parameter names in backticks. The page shows them as
-  code. Do not repeat the failed checks in the summary.
-
-Then merge the panel into `verdicts.json`:
-
-```bash
-python3 $S/merge_verdicts.py <runDir>
-```
-
-For each criterion, the merged check takes the value that all judges or a
-majority gave. A split shows on the page ("3 judges · 1 split", and "2 of 3
-judges" on the check). Without a majority, the check is not confirmed. The
-summary and the other findings come from judge 1. The page computes the
-verdict (see "Checks on the page").
-
-## 5. Build the page again
-
-```bash
-python3 $S/build_report.py <runDir> --step judge=done
-```
-
-If the command prints `models without a published price`, do step 3 for
-those models and build the page again.
-
-The page is `<runDir>/report/index.html`. It works offline. Give the user the
-path, and the `live` URL from the output of `compare.py` while the server
-runs. Then, in a few sentences, report:
-
-- the verdict of each builder, with the most important reason
-- failed correctness, execution or service-call checks, and judge splits
-- the standards, edge-case, real-AI, claims and eval scores of each
-  builder, as separate results
-- the cost, time and number of LLM calls of each builder
-- builders that were skipped, and data that is missing
-
-## How the builders run
-
-**Instance AI** runs through the REST API of the instance, with its model.
-The runner answers every confirmation card with one fixed policy, so runs
-can be compared. It approves actions and plans, picks the first option of a
-question, allows a domain once, defers workflow and credential setup cards,
-and denies access to local files and MCP connections. `iai/run.json` lists
-each decision.
-
-**Claude Code and Codex** run in a Docker container (`COMPARE_ISOLATION=container`):
-
-- The container mounts only the run folder of the builder. The builder
-  cannot see host files, skills, memory, `CLAUDE.md`, `AGENTS.md` or user
-  settings.
-- The container has no Linux capabilities and a read-only root file system.
-- Claude starts with a new config folder, no built-in tools, and only the
-  `n8n` MCP server. It does not use `--bare`, because `--bare` replaces the
-  normal system prompt with a 150-byte one. The container already keeps out
-  host settings, so Claude gets its normal system prompt, like Codex does.
-  `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` stops update checks and
-  telemetry.
-- `COMPARE_CLAUDE_TOOLS=default` gives Claude the built-in tools of a normal
-  Claude Code session, such as Bash, Read, Write, Edit and WebFetch, with
-  `--permission-mode bypassPermissions`. Claude then uses every tool without
-  a question, as Codex does. The tools can change files in the run folder
-  and read the environment, which holds the model key and the MCP token.
-  The secret check after the run finds a key that a tool wrote to a file.
-  Claude Code also turns on tool search: the model gets only the names of
-  the n8n tools and of some built-in tools, and loads a schema with
-  `ToolSearch`. The page shows these deferred tools with a dashed border.
-  The default `none` keeps runs comparable with older runs, because the
-  extra tools change the tokens and the cost.
-- Codex gets a new `CODEX_HOME` that holds only the n8n MCP server.
-  Its plugins, apps, browser use, computer use and image generation are off.
-- The container can still reach the internet and the host network. It needs
-  them for the model API and for n8n.
-- Codex sends its model requests through a local recorder
-  (`request_recorder.py`). The recorder sends each request to OpenAI without
-  changes and writes the request body, but not the headers, to
-  `codex/requests.jsonl`. The key is in a header, so it stays out of the file.
-  The file does not hold the conversation input. It holds the tools and the
-  instructions only when they change. The recorder listens on `127.0.0.1`,
-  which Docker Desktop forwards to `host.docker.internal`. On Linux, set
-  `COMPARE_RECORDER_BIND` to the address of the Docker bridge. Set
-  `COMPARE_RECORD_REQUESTS=0` to send Codex straight to OpenAI.
-
-By default the CLIs run in interactive mode (`COMPARE_CLI_MODE=interactive`):
-
-- The normal terminal screen of the CLI runs in tmux, at 120 × 36
-  characters. tmux answers the terminal queries of the CLI, and its output
-  goes to `screen.cast`.
-- The CLIs connect MCP servers in the background. Thus the script pastes the
-  prompt when the input box shows, plus a short wait, not as an argument.
-- Claude uses `--permission-mode dontAsk`: the n8n tools are pre-approved,
-  and all other tools are denied without a question.
-- The model key reaches the CLI through a variable that only the CLI reads
-  (`apiKeyHelper` for Claude, a provider `env_key` for Codex). Thus no screen
-  asks to confirm the key or shows part of it, and no key is in a file.
-- The script reads the session file of the CLI for the progress lines, the
-  end of the turn, the trace and the tokens. Claude Code then shows no cost
-  estimate, so the cost needs the published prices.
-- After each run, the script searches the run folder for the model key and
-  the MCP token. It prints a warning with the file name if it finds one.
-
-`COMPARE_CLI_MODE=headless` uses `claude -p` and `codex exec` with a JSON
-event stream instead.
-
-The first run builds the image `n8n-builder-sandbox` from
-`sandbox/Dockerfile`. A change to the Dockerfile rebuilds the image on the
-next run. The image has `python3`, `jq` and `ripgrep`, because Codex and
-Claude Code with `COMPARE_CLAUDE_TOOLS=default` use them in their shell. To get new CLI
-versions, run `docker rmi n8n-builder-sandbox`.
-
-`COMPARE_ISOLATION=clean` runs Claude as a clean process on the host: an
-empty environment, a new config folder, no built-in tools, and only the n8n
-MCP server. It ignores `COMPARE_CLAUDE_TOOLS`. It uses `--bare` to keep out host settings, so it gets the
-minimal system prompt. It needs an API key, because `--bare` does not read
-the subscription login. Codex has no clean mode, because its shell tool can
-read the host disk.
-
-## Single steps
-
-`compare.py` calls these scripts. Use them to repeat one step.
-
-| Script | Step |
-|---|---|
-| `prepare.py` | Check the setup. `--rotate` makes a new MCP key |
-| `run_instance_ai.py --out <dir>/iai --prompt-file <file>` | Run Instance AI |
-| `run_cli_builder.py --builder claude\|codex --out <dir>/<name> --prompt-file <file>` | Run a CLI builder. `--mode headless` skips the screen |
-| `node record_iai.mjs <dir>/iai` | Record the Instance AI video. `run_instance_ai.py` starts it |
-| `live_server.py --run-dir <dir>` | Serve the live page and the report of a run folder. Prints nothing. The URL is in `<dir>/live-url.txt` |
-| `langsmith_usage.py --run-dir <dir>/iai` | Fetch Instance AI usage and model context |
-| `node capture_canvas.mjs <dir>` | Save `workflow.json`, `canvas.png` and `canvas-issues.json` for each run |
-| `run_scenarios.py <dir>` | Run each workflow with pinned data. Writes `<name>/exec.json` |
-| `node sdk.mjs pin-plan <workflow.json> [scenarios.json]` | Print the pin data of each scenario, without a run |
-| `node run_wire.mjs <dir>` | Run each workflow in process with recorded API fixtures. Writes `<name>/wire.json` |
-| `pnpm --filter @n8n/instance-ai exec tsx $S/binary_checks.mts <dir>` | Run the deterministic eval binary checks. Writes `<name>/binary-checks.json` |
-| `claims.py <dir>` | Check the final-message claims. Writes `<name>/claims.json` |
-| `node derive_checks.mjs <dir>` | Write `<name>/auto-checks.json` from all the files above |
-| `summarize_runs.py <dir> [--blind]` | Print the judge summary. `--blind` writes `judge/summary-blind.txt` and `judge/blind-map.json` |
-| `merge_verdicts.py <dir>` | Merge `judge/verdicts-*.json` into `verdicts.json` |
-| `build_report.py <dir> [--step judge\|prices=queued\|running\|done]` | Build the page. `--step` first sets the state of a step after the run |
-
-If a runner did not find a workflow id, read its trace for the id. Set
-`workflowId` in that `run.json`, then run `capture_canvas.mjs`,
-`run_scenarios.py`, `run_wire.mjs`, `binary_checks.mts`, `claims.py`,
-`derive_checks.mjs` and `summarize_runs.py` again.
-
-## Notes
-
-- Cost sources: Instance AI cost comes from LangSmith. For Claude Code and
-  Codex, the cost is tokens × the published prices in `case.json`. Without a
-  price, Claude Code in headless mode shows its own estimate. All other runs
-  show "not reported".
-  The tooltip of each Cost cell shows the source of the cost.
-- The workflows stay in the n8n instance. Archive them in n8n when they are
-  no longer necessary. The pinned-data runs add test executions to them.
-- The n8n skills stay out of the sandboxes, so the standards score shows
-  what each builder does without them. Do not add the skills to the
-  sandboxes to raise the score.

.agents/skills/testmax-n8n-ai/sandbox/Dockerfile

@@ -1,21 +0,0 @@
-# Sandbox for CLI workflow builders. The container sees only the run directory
-# that is mounted at /out. It reaches n8n through host.docker.internal.
-FROM node:26-slim
-
-ARG CLAUDE_CODE_VERSION=latest
-ARG CODEX_VERSION=latest
-
-# Codex uses the system trust store. Claude Code uses the CA list in Node.
-# tmux runs the interactive CLI screens so that they can be recorded and watched.
-# Claude Code (with built-in tools) and Codex use python3, jq and ripgrep in their shell to read large tool results.
-RUN apt-get update \
-	&& apt-get install -y --no-install-recommends ca-certificates tmux python3 jq ripgrep \
-	&& rm -rf /var/lib/apt/lists/*
-
-RUN npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" "@openai/codex@${CODEX_VERSION}" \
-	&& npm cache clean --force
-
-ENV DISABLE_AUTOUPDATER=1 \
-	DISABLE_TELEMETRY=1 \
-	DISABLE_ERROR_REPORTING=1 \
-	CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

.agents/skills/testmax-n8n-ai/sandbox/screen.mjs

@@ -1,100 +0,0 @@
-// Run an interactive CLI inside tmux and record its screen as an asciinema v2 file.
-//
-//   node screen.mjs run <dir>      start tmux, record, and wait for <dir>/stop or the CLI exit
-//   node screen.mjs launch <dir>   (inside tmux) start the command in <dir>/command.json
-//   node screen.mjs record <dir>   (tmux pipe-pane) write the pane output to <dir>/screen.cast
-//
-// <dir>/command.json is {"argv": [...], "cols": 120, "rows": 36, "input": "prompt",
-// "readyText": "text on the screen when the CLI accepts input", "settleMs": 0}. tmux answers the terminal
-// queries of the CLI, so the CLI behaves as in a real terminal. The recording has the raw
-// output of the CLI, so a player shows the exact screen.
-import { spawn, spawnSync } from 'node:child_process';
-import { createWriteStream, existsSync, readFileSync, writeFileSync } from 'node:fs';
-import { join } from 'node:path';
-
-const [mode, dir] = process.argv.slice(2);
-const command = JSON.parse(readFileSync(join(dir, 'command.json'), 'utf8'));
-const READY = '/tmp/screen-ready';
-const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
-const tmux = (...args) => spawnSync('tmux', args, { stdio: 'inherit' }).status;
-const screenText = () =>
-	spawnSync('tmux', ['capture-pane', '-p', '-t', 'run'], { encoding: 'utf8' }).stdout ?? '';
-
-// The CLIs connect MCP servers in the background. A prompt on the command line can start
-// before the n8n tools exist, so the prompt is pasted when the input box is ready.
-async function typePrompt({ input, readyText, settleMs = 0 }) {
-	const deadline = Date.now() + 60_000;
-	while (readyText && !screenText().includes(readyText) && Date.now() < deadline) await sleep(250);
-	await sleep(settleMs);
-	tmux('set-buffer', '-b', 'prompt', '--', input);
-	tmux('paste-buffer', '-p', '-d', '-b', 'prompt', '-t', 'run');
-	await sleep(500);
-	tmux('send-keys', '-t', 'run', 'Enter');
-}
-
-if (mode === 'record') {
-	const out = createWriteStream(join(dir, 'screen.cast'));
-	out.write(
-		JSON.stringify({
-			version: 2,
-			width: command.cols,
-			height: command.rows,
-			timestamp: Math.floor(Date.now() / 1000),
-			env: { TERM: 'xterm-256color' },
-		}) + '\n',
-	);
-	const start = process.hrtime.bigint();
-	const decoder = new TextDecoder('utf-8');
-	process.stdin.on('data', (chunk) => {
-		const text = decoder.decode(chunk, { stream: true });
-		if (!text) return;
-		const t = Number(process.hrtime.bigint() - start) / 1e9;
-		out.write(JSON.stringify([Number(t.toFixed(6)), 'o', text]) + '\n');
-	});
-	process.stdin.on('end', () => out.end());
-} else if (mode === 'launch') {
-	while (!existsSync(READY)) await sleep(50);
-	const [bin, ...args] = command.argv;
-	const child = spawn(bin, args, { stdio: 'inherit' });
-	child.on('exit', (code) => {
-		writeFileSync(join(dir, 'exited'), String(code ?? 1));
-		process.exit(code ?? 1);
-	});
-	// A CLI that cannot start emits error, not exit. Without the marker, the run waits for ever.
-	child.on('error', (error) => {
-		console.error(`Cannot start ${bin}: ${error.message}`);
-		writeFileSync(join(dir, 'exited'), '127');
-		process.exit(127);
-	});
-} else if (mode === 'run') {
-	const self = new URL(import.meta.url).pathname;
-	// mouse and focus-events stop the tmux tips that Claude Code shows in the recorded screen.
-	writeFileSync(
-		'/tmp/tmux.conf',
-		'set -g status off\nset -g default-terminal xterm-256color\nset -g mouse on\nset -g focus-events on\n',
-	);
-	tmux(
-		'-f',
-		'/tmp/tmux.conf',
-		'new-session',
-		'-d',
-		'-s',
-		'run',
-		'-x',
-		String(command.cols),
-		'-y',
-		String(command.rows),
-		`node ${self} launch ${dir}`,
-	);
-	// A watcher in another terminal size must not resize the recorded screen. tmux 3.5 stops
-	// when this option is in the startup config, so it is set after the session starts.
-	tmux('set-option', '-g', 'window-size', 'manual');
-	tmux('pipe-pane', '-O', '-t', 'run', `node ${self} record ${dir}`);
-	writeFileSync(READY, '');
-	if (command.input) await typePrompt(command);
-	while (!existsSync(join(dir, 'stop')) && !existsSync(join(dir, 'exited'))) await sleep(500);
-	// Keep the final screen in the recording for a moment.
-	await sleep(2500);
-	tmux('kill-server');
-	await sleep(500);
-}

.agents/skills/testmax-n8n-ai/scripts/binary_checks.mts

@@ -1,50 +0,0 @@
-// Runs the deterministic binary checks of the Instance AI eval suite on each workflow of a run.
-// Run it with tsx from the instance-ai package, because the checks are TypeScript sources:
-//
-//   pnpm --filter @n8n/instance-ai exec tsx <skill>/scripts/binary_checks.mts <run-dir>
-//
-// Writes <builder>/binary-checks.json with {source, outcomes: [{name, description, dimension, status, comment}]}.
-import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
-import { dirname, join, resolve } from 'node:path';
-import { fileURLToPath, pathToFileURL } from 'node:url';
-
-const repo = resolve(dirname(fileURLToPath(import.meta.url)), '../../../..');
-const SOURCE = 'packages/@n8n/instance-ai/evaluations/binaryChecks/checks';
-
-type Check = {
-	name: string;
-	description: string;
-	kind: 'deterministic' | 'llm';
-	dimension: string;
-	run(workflow: unknown, ctx: Record<string, unknown>): unknown;
-};
-type Result = { pass: boolean; comment?: string; applicable?: boolean; errored?: boolean };
-
-const readJson = (file: string): unknown => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
-
-const runDir = resolve(process.argv[2] ?? '.');
-const { DETERMINISTIC_CHECKS } = (await import(pathToFileURL(join(repo, SOURCE, 'index.ts')).href)) as { DETERMINISTIC_CHECKS: Check[] };
-const prompt = existsSync(join(runDir, 'prompt.txt')) ? readFileSync(join(runDir, 'prompt.txt'), 'utf8') : '';
-
-for (const entry of readdirSync(runDir, { withFileTypes: true })) {
-	const dir = join(runDir, entry.name);
-	const workflow = entry.isDirectory() ? readJson(join(dir, 'workflow.json')) : null;
-	if (!workflow) continue;
-	const outcomes = [];
-	for (const check of DETERMINISTIC_CHECKS) {
-		let status: string;
-		let comment: string | undefined;
-		try {
-			const r = (await check.run(workflow, { prompt })) as Result;
-			status = r.errored ? 'error' : r.applicable === false ? 'n_a' : r.pass ? 'pass' : 'fail';
-			comment = r.comment;
-		} catch (error) {
-			status = 'error';
-			comment = error instanceof Error ? error.message : String(error);
-		}
-		outcomes.push({ name: check.name, description: check.description, dimension: check.dimension, status, ...(comment ? { comment } : {}) });
-	}
-	writeFileSync(join(dir, 'binary-checks.json'), JSON.stringify({ source: SOURCE, outcomes }, null, 1));
-	const scored = outcomes.filter((o) => o.status === 'pass' || o.status === 'fail');
-	console.log(`${entry.name}: binary-checks.json written (${scored.filter((o) => o.status === 'pass').length}/${scored.length})`);
-}

.agents/skills/testmax-n8n-ai/scripts/capture_canvas.mjs

@@ -1,162 +0,0 @@
-#!/usr/bin/env node
-// Take a cropped canvas screenshot of each workflow that a run created.
-//
-//   node capture_canvas.mjs <run-dir>
-//
-// Reads <run-dir>/<builder>/run.json and writes <run-dir>/<builder>/canvas.png, workflow.json and
-// canvas-issues.json.
-import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
-import { join, resolve } from 'node:path';
-
-import { baseUrl, launchBrowser, signIn } from './n8n_browser.mjs';
-
-const runDir = resolve(process.argv[2] ?? '.');
-
-// The status icons that CanvasNodeStatusIcons.vue draws on a node. n8n computes them from the node
-// type description, so they are the verdict of n8n itself on each node.
-const ISSUE_MARKERS = ['node-issues', 'node-setup-required', 'node-not-installed', 'node-restricted'];
-
-/**
- * The status icon of each node, with the issues that the editor computed for it. The tooltips
- * are teleported and show after a delay, so the issues come from the workflow document store.
- */
-async function readIssues(page, workflowId) {
-	return await page.evaluate(
-		({ markers, workflowId }) => {
-			const icons = {};
-			for (const el of document.querySelectorAll(
-				markers.map((m) => `[data-node-name] [data-test-id="${m}"]`).join(','),
-			)) {
-				icons[el.closest('[data-node-name]').dataset.nodeName] = el.dataset.testId;
-			}
-			const pinia = document.querySelector('#app')?.__vue_app__?.config.globalProperties.$pinia;
-			const key = [...(pinia?._s.keys() ?? [])].find((k) =>
-				k.startsWith(`workflowDocuments/${workflowId}@`),
-			);
-			const storeNodes = key ? (pinia._s.get(key).allNodes ?? []) : [];
-			const nodes = storeNodes
-				.filter((n) => icons[n.name] || (n.issues && Object.keys(n.issues).length))
-				.map((n) => ({
-					name: n.name,
-					type: n.type,
-					marker: icons[n.name] ?? null,
-					issues: n.issues ?? {},
-				}));
-			for (const [name, marker] of Object.entries(icons)) {
-				if (!nodes.some((n) => n.name === name)) nodes.push({ name, marker, issues: {} });
-			}
-			return { markers, store: Boolean(key), nodes };
-		},
-		{ markers: ISSUE_MARKERS, workflowId },
-	);
-}
-
-const runs = readdirSync(runDir, { withFileTypes: true })
-	.filter((d) => d.isDirectory() && existsSync(join(runDir, d.name, 'run.json')))
-	.map((d) => ({
-		dir: join(runDir, d.name),
-		meta: JSON.parse(readFileSync(join(runDir, d.name, 'run.json'), 'utf8')),
-	}))
-	.filter((r) => r.meta.workflowId);
-
-if (runs.length === 0) {
-	console.log('No run has a workflowId. Nothing to capture.');
-	process.exit(0);
-}
-
-const browser = await launchBrowser();
-const VIEWPORT = { width: 1800, height: 1000 };
-const context = await browser.newContext({ viewport: VIEWPORT, deviceScaleFactor: 2 });
-const page = await context.newPage();
-await signIn(page);
-
-for (const { dir, meta } of runs) {
-	const id = meta.workflowId;
-	const res = await page.request.get(`${baseUrl}/rest/workflows/${id}`, {
-		headers: {
-			'browser-id': await page.evaluate(() => localStorage.getItem('n8n-browserId') ?? ''),
-		},
-	});
-	if (res.ok())
-		writeFileSync(join(dir, 'workflow.json'), JSON.stringify((await res.json()).data, null, 1));
-	else console.log(`${id}: workflow fetch failed with ${res.status()}`);
-
-	await page.goto(`${baseUrl}/workflow/${id}`);
-	try {
-		await page.locator('[data-test-id="canvas-node"]').first().waitFor({ timeout: 30_000 });
-	} catch {
-		console.log(`${id}: no canvas nodes, skipped`);
-		continue;
-	}
-	// Expanding one group re-renders the others, so repeat until no collapsed group is left.
-	for (let round = 0; round < 5; round++) {
-		const clicked = await page.evaluate(() => {
-			const buttons = [...document.querySelectorAll('button[aria-label="Expand"]')];
-			buttons.forEach((b) => b.click());
-			return buttons.length;
-		});
-		if (clicked === 0) break;
-		await page.waitForTimeout(400);
-	}
-	// The layout moves after groups expand. Fit twice so the second fit sees the final size.
-	for (let i = 0; i < 2; i++) {
-		await page.waitForTimeout(1000);
-		await page
-			.locator('[data-test-id="zoom-to-fit"]')
-			.click({ timeout: 5_000 })
-			.catch(() => {});
-	}
-	await page.waitForTimeout(800);
-
-	const box = await page.evaluate(() => {
-		const rects = [...document.querySelectorAll('.vue-flow__node')].map((n) =>
-			n.getBoundingClientRect(),
-		);
-		if (rects.length === 0) return null;
-		// Labels, subtitles and expanded groups overflow their node element, so use the descendants too.
-		// Only count what draws: hit areas and hover toolbars are empty boxes around the node.
-		const draws = (el) =>
-			el instanceof SVGSVGElement ||
-			el instanceof HTMLImageElement ||
-			[...el.childNodes].some((c) => c.nodeType === Node.TEXT_NODE && c.textContent.trim());
-		const visible = (el) => {
-			for (let e = el; e && !e.classList.contains('vue-flow__node'); e = e.parentElement) {
-				const style = getComputedStyle(e);
-				if (
-					style.visibility === 'hidden' ||
-					style.display === 'none' ||
-					Number(style.opacity) === 0
-				)
-					return false;
-			}
-			return true;
-		};
-		const all = rects.concat(
-			[...document.querySelectorAll('.vue-flow__node *')]
-				.filter((el) => draws(el) && visible(el))
-				.map((n) => n.getBoundingClientRect())
-				.filter((r) => r.width > 0 && r.height > 0),
-		);
-		const left = Math.min(...all.map((r) => r.left));
-		const top = Math.min(...all.map((r) => r.top));
-		const right = Math.max(...all.map((r) => r.right));
-		const bottom = Math.max(...all.map((r) => r.bottom));
-		return { left, top, right, bottom };
-	});
-	const pad = 40;
-	const clip = box
-		? {
-				x: Math.max(0, box.left - pad),
-				y: Math.max(0, box.top - pad),
-				width: Math.min(VIEWPORT.width, box.right + pad) - Math.max(0, box.left - pad),
-				height: Math.min(VIEWPORT.height, box.bottom + pad) - Math.max(0, box.top - pad),
-			}
-		: undefined;
-	await page.screenshot({ path: join(dir, 'canvas.png'), clip });
-	console.log(`${id}: canvas.png written`);
-	const issues = await readIssues(page, id);
-	writeFileSync(join(dir, 'canvas-issues.json'), JSON.stringify(issues, null, 1));
-	console.log(`${id}: canvas-issues.json written (${issues.nodes.length} nodes with a status icon)`);
-}
-
-await browser.close();

.agents/skills/testmax-n8n-ai/scripts/claims.py

@@ -1,180 +0,0 @@
-#!/usr/bin/env python3
-"""Check what the final message of each builder says against the trace, the workflow and the runs.
-
-  claims.py <run-dir>
-
-Writes <builder>/claims.json with {message, checks: [{id, ok, text, details}]}. No LLM is involved.
-A check is about one kind of claim: the workflow id, the publish state, validation, testing, the
-values in `code spans`, and if a claim that the workflow works agrees with the service-call runs.
-A message that does not make a kind of claim gets no check for it.
-"""
-
-import json
-import re
-import sys
-from pathlib import Path
-
-sys.path.insert(0, str(Path(__file__).resolve().parent))
-from build_report import load_run  # noqa: E402
-from summarize_runs import final_message  # noqa: E402
-
-TEST_TOOL = re.compile(r'test|execut|verify|run[-_]?workflow|simulat', re.I)
-SHELL_TOOL = re.compile(r'command|shell|bash', re.I)
-SCRIPT_PASSED = re.compile(r'"passed"\s*:\s*true|\b\d+ (?:tests? )?passed\b|\ball tests passed\b', re.I)
-VALIDATE_TOOL = re.compile(r'validat', re.I)
-NEGATION = re.compile(r"\b(not|never|no|nothing|without)\b|n't\b", re.I)
-# A clause about what can or will happen later is not a claim about the run.
-MODAL = re.compile(r"\b(can|can't|cannot|will|won't|could|would|should|once|until|after that|let me know|need)\b", re.I)
-TESTED = re.compile(r'\b(tested|test run|ran|run|runs|executed|simulat\w*|routing checks?|checks? passed)\b', re.I)
-VALIDATED = re.compile(r'\b(validat\w*|valid)\b', re.I)
-PASSED = re.compile(r'\b(pass\w*|succeed\w*|valid|ok|clean)\b', re.I)
-DRAFT = re.compile(r"\b(draft|inactive|unpublished|not (?:yet )?(?:live|active|published)|isn't (?:yet )?(?:live|active)|not turned on)\b", re.I)
-LIVE = re.compile(r'\b(?:is|now) (?:live|active|published)\b', re.I)
-WORKFLOW_ID = re.compile(r'/workflow/([A-Za-z0-9]{8,})|\bID `?([A-Za-z0-9]{16})`?')
-CODE_SPAN = re.compile(r'`([^`\n]{2,80})`')
-INVALID = re.compile(r'"valid"\s*:\s*false|"success"\s*:\s*false', re.I)
-# A test tool can answer without a transport error and still report a failed execution.
-RUN_FAILED = re.compile(r'"status"\s*:\s*"(?:error|failed|crashed|canceled|cancelled)"|"success"\s*:\s*false', re.I)
-
-
-def clauses(message):
-	"""The message split into clauses, with Markdown emphasis removed."""
-	text = re.sub(r'[*_]{1,2}([^*_]+)[*_]{1,2}', r'\1', message or '')
-	# Streamed messages can join sentences with no space, as in "done.Next".
-	return [c.strip() for c in re.split(r'(?<=[.!?:])\s+|(?<=[a-z][.!?])(?=[A-Z])|\n+|;\s+|,\s+but\s+|\s+but\s+', text) if c.strip()]
-
-
-def claim(message, pattern, extra=None):
-	"""(positive, negative) clauses that match pattern. extra must also match a positive clause."""
-	pos, neg = [], []
-	for c in clauses(message):
-		if not pattern.search(c) or MODAL.search(c):
-			continue
-		if NEGATION.search(c):
-			neg.append(c)
-		elif extra is None or extra.search(c):
-			pos.append(c)
-	return pos, neg
-
-
-def tool_text(e):
-	return ' '.join(x if isinstance(x, str) else json.dumps(x, ensure_ascii=False) for x in (e.get('args'), e.get('code'), e.get('result')) if x)
-
-
-def check(id_, ok, text, details):
-	return {'id': id_, 'ok': ok, 'text': text, 'details': details}
-
-
-def quote(clauses_):
-	return [f'Message: "{c}"' for c in clauses_[:3]]
-
-
-def id_check(message, meta, workflow):
-	ids = {a or b for a, b in WORKFLOW_ID.findall(message or '')}
-	if not ids:
-		return None
-	known = set(meta.get('workflowIds') or []) | {meta.get('workflowId'), (workflow or {}).get('id')}
-	wrong = sorted(ids - known)
-	return check('M1', not wrong, 'The workflow id in the message is the workflow that the run created',
-	             [f'{i} is not a workflow of this run' for i in wrong] or [f'{", ".join(sorted(ids))} matches.'])
-
-
-def state_check(message, workflow):
-	draft, live = DRAFT.findall(message or ''), LIVE.findall(message or '')
-	if workflow is None or not (draft or live):
-		return None
-	active = bool(workflow.get('active'))
-	says = 'active' if live and not draft else 'a draft'
-	ok = active == (says == 'active')
-	return check('M2', ok, f'The message says the workflow is {says}', [f'workflow.json active is {str(active).lower()}.'])
-
-
-def validation_check(message, events):
-	pos, _ = claim(message, VALIDATED, PASSED)
-	if not pos:
-		return None
-	calls = [e for e in events if (e['kind'] == 'tool' and VALIDATE_TOOL.search(e.get('name') or ''))
-	         or (e['kind'] == 'code' and VALIDATE_TOOL.search(e.get('code') or ''))]
-	if not calls:
-		return check('M3', False, 'The message says the workflow passed validation', quote(pos) + ['No validation tool call in the trace.'])
-	last = calls[-1]
-	failed = last.get('error') or INVALID.search(str(last.get('result') or ''))
-	return check('M3', not failed, 'The message says the workflow passed validation', quote(pos) + [
-		f'{len(calls)} validation call(s). The last one {"failed" if failed else "passed"}: {last.get("name")}.'])
-
-
-def test_check(message, events):
-	pos, neg = claim(message, TESTED)
-	runs = [e for e in events if e['kind'] == 'tool' and TEST_TOOL.search(e.get('name') or '')
-	        and not VALIDATE_TOOL.search(e.get('name') or '') and not SHELL_TOOL.search(e.get('name') or '')]
-	scripts = [e for e in events if e['kind'] == 'code' and SCRIPT_PASSED.search(str(e.get('result') or ''))]
-	if pos:
-		ok_runs = [e for e in runs if not e.get('error') and not RUN_FAILED.search(tool_text({'result': e.get('result')}))]
-		if ok_runs:
-			names = sorted({e['name'] for e in ok_runs})
-			return check('M4', True, 'The message says the workflow was tested', quote(pos) + [f'{len(ok_runs)} successful test call(s): {", ".join(names)}.'])
-		if scripts:
-			return check('M4', True, 'The message says the workflow was tested', quote(pos) + [
-				f'{len(scripts)} script(s) in the trace report passed tests. These ran outside n8n, not as a workflow execution.'])
-		return check('M4', False, 'The message says the workflow was tested', quote(pos) + ['No test or execution call in the trace.'])
-	if neg and runs:
-		return check('M4', False, 'The message says the workflow was not run', quote(neg) + [f'The trace has {len(runs)} test call(s).'])
-	if neg:
-		return check('M4', True, 'The message says the workflow was not run', quote(neg) + ['No test or execution call in the trace.'])
-	return None
-
-
-def span_check(message, events, workflow, prompt):
-	spans = sorted({s.strip() for s in CODE_SPAN.findall(message or '')})
-	if not spans:
-		return None
-	haystack = '\n'.join([json.dumps(workflow or {}, ensure_ascii=False), prompt] + [tool_text(e) for e in events])
-	loose = lambda s: re.sub(r'\s+', ' ', s).lower()
-	hay = loose(haystack.replace('\\"', '"'))
-	missing = [s for s in spans if loose(s) not in hay]
-	return check('M5', not missing, f'Values in code spans are in the workflow, the prompt or a tool result ({len(spans) - len(missing)} of {len(spans)})',
-	             [f'`{s}` is not in the workflow, the prompt or any tool call' for s in missing])
-
-
-def works_check(message, wire):
-	pos, _ = claim(message, re.compile(TESTED.pattern + '|' + VALIDATED.pattern, re.I), None)
-	if not pos or not wire or wire.get('skipped'):
-		return None
-	failed = [s for s in wire.get('scenarios', []) if s.get('kind') == 'contract'
-	          and (s.get('status') != 'success' or any(c.get('ok') is False for c in s.get('checks') or []))]
-	details = quote(pos) + [f'"{s["name"]}" fails in the service-call run: {s.get("error") or "a check fails"}.' for s in failed]
-	return check('M6', not failed, 'A claim that the workflow was checked agrees with the service-call runs', details)
-
-
-def claims_for(sub, meta, prompt):
-	workflow = json.loads((sub / 'workflow.json').read_text()) if (sub / 'workflow.json').exists() else None
-	wire = json.loads((sub / 'wire.json').read_text()) if (sub / 'wire.json').exists() else None
-	try:
-		message = final_message(sub, meta)
-	except Exception as e:
-		return {'message': None, 'error': f'The final message is not readable: {e}', 'checks': []}
-	data = load_run(sub, meta, {})
-	events = data.get('events') or []
-	checks = [id_check(message, meta, workflow), state_check(message, workflow), validation_check(message, events),
-	          test_check(message, events), span_check(message, events, workflow, prompt), works_check(message, wire)]
-	out = {'message': message, 'checks': [c for c in checks if c]}
-	if data.get('loadError'):
-		out['error'] = f'The trace is not readable: {data["loadError"]}'
-	return out
-
-
-def main():
-	run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
-	prompt = (run_dir / 'prompt.txt').read_text() if (run_dir / 'prompt.txt').exists() else ''
-	for f in sorted(run_dir.glob('*/run.json')):
-		sub = f.parent
-		if sub.name == 'report':
-			continue
-		result = claims_for(sub, json.loads(f.read_text()), prompt)
-		(sub / 'claims.json').write_text(json.dumps(result, indent=1, ensure_ascii=False))
-		ok = sum(1 for c in result['checks'] if c['ok'])
-		print(f'{sub.name}: claims.json written ({ok}/{len(result["checks"])})')
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/compare.py

@@ -1,328 +0,0 @@
-#!/usr/bin/env python3
-"""Run the full comparison in one command, except the judge step.
-
-  compare.py --prompt "..." --expected "..." [--title "..."] [--builders iai,claude,codex]
-  compare.py --prompt-file p.txt --expected-file e.txt [--scenarios-file s.json]
-  compare.py --rerun /tmp/builder-compare/<old-run>     same prompt, expected outcome and scenarios, new run
-
-Steps: check the setup, run the builders at the same time (--sequential: one at a time),
-fetch Instance AI usage, capture the canvases and their node issues, run each workflow with
-pinned data (run_scenarios.py), run each workflow in process with recorded API requests
-(run_wire.mjs), run the eval binary checks (binary_checks.mts), check the final-message claims
-(claims.py), derive the deterministic checks (derive_checks.mjs), write
-summary.txt and judge/summary-blind.txt for the judges, and build the report. A local page
-shows the runs live (see live_server.py).
-Settings come from .env.ai-surfaces (template: .env.ai-surfaces.example in the repository root).
-"""
-
-import argparse
-import json
-import os
-import re
-import shutil
-import subprocess
-import sys
-import threading
-import time
-import webbrowser
-from datetime import datetime
-from pathlib import Path
-
-from live import enabled as live_enabled
-from live import is_on, recording
-from n8n_api import setting
-
-HERE = Path(__file__).resolve().parent
-RUNNERS = {'iai', 'claude', 'codex'}
-PRINT_LOCK = threading.Lock()
-
-
-def slug(text):
-	words = re.findall(r'[a-z0-9]+', text.lower())[:5]
-	return '-'.join(words) or 'run'
-
-
-def step(title):
-	with PRINT_LOCK:
-		print(f'\n=== {title} ===', flush=True)
-
-
-def run(cmd, log=None, prefix=None):
-	"""Run a script, show its output, and keep a copy in log. Returns the exit code."""
-	with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, cwd=HERE) as proc:
-		lines = []
-		for line in proc.stdout:
-			shown = line if not prefix or line.startswith(f'[{prefix}]') else f'[{prefix}] {line}'
-			with PRINT_LOCK:
-				print(shown, end='', flush=True)
-			lines.append(line)
-	if log:
-		Path(log).write_text(''.join(lines))
-	return proc.returncode
-
-
-class Progress:
-	"""Writes status.json, which the live page reads."""
-
-	def __init__(self, run_dir, mode, builders, skipped, screens):
-		self.path, self.lock = run_dir / 'status.json', threading.Lock()
-		self.data = {
-			'mode': mode, 'phase': 'running', 'step': 'Running the builders', 'startedAt': time.time(),
-			'screens': screens,
-			'builders': {**{b: {'status': 'waiting'} for b in builders},
-			             **{b: {'status': 'skipped', 'reason': r} for b, r in skipped.items()}},
-		}
-		self.save()
-
-	def save(self):
-		tmp = self.path.with_suffix('.tmp')
-		tmp.write_text(json.dumps(self.data, indent=2))
-		os.replace(tmp, self.path)
-
-	def update(self, **fields):
-		with self.lock:
-			self.data.update(fields)
-			self.save()
-
-	def builder(self, name, **fields):
-		with self.lock:
-			self.data['builders'][name].update(fields)
-			self.save()
-
-	def finish(self):
-		with self.lock:
-			for entry in self.data['builders'].values():
-				if entry['status'] in ('waiting', 'running'):
-					entry['status'] = 'failed'
-			self.data.update(phase='done', step='Done', finishedAt=time.time())
-			self.save()
-
-
-def start_live_server(run_dir):
-	"""Start live_server.py in the background. Returns its URL, or None."""
-	url_file = run_dir / 'live-url.txt'
-	url_file.unlink(missing_ok=True)
-	log = (run_dir / 'live-server.log').open('w')
-	# A new session, so the page still serves the final report after this script ends.
-	subprocess.Popen([sys.executable, str(HERE / 'live_server.py'), '--run-dir', str(run_dir),
-	                  '--parent-pid', str(os.getpid())],
-	                 cwd=HERE, stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
-	for _ in range(50):
-		if url_file.exists() and url_file.read_text().strip():
-			return url_file.read_text().strip()
-		time.sleep(0.1)
-	return None
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--prompt')
-	ap.add_argument('--prompt-file')
-	ap.add_argument('--expected')
-	ap.add_argument('--expected-file')
-	ap.add_argument('--title')
-	ap.add_argument('--builders', default=setting('COMPARE_BUILDERS', default='iai,claude,codex'))
-	ap.add_argument('--run-dir')
-	ap.add_argument('--scenarios-file', help='Pinned-data scenarios for run_scenarios.py (see SKILL.md).')
-	ap.add_argument('--rerun', help='Copy prompt.txt, case.json and scenarios.json from an earlier run directory.')
-	ap.add_argument('--no-live', action='store_true', help='Do not start the live page.')
-	ap.add_argument('--sequential', action='store_true', default=not is_on('COMPARE_PARALLEL'),
-	                help='Run the builders one at a time. Default: COMPARE_PARALLEL=1 runs them at the same time.')
-	args = ap.parse_args()
-	if args.no_live:
-		os.environ['COMPARE_LIVE'] = '0'
-
-	if args.rerun:
-		old = Path(args.rerun)
-		prompt = (old / 'prompt.txt').read_text().strip()
-		case = json.loads((old / 'case.json').read_text())
-	else:
-		prompt = args.prompt or (Path(args.prompt_file).read_text().strip() if args.prompt_file else None)
-		expected = args.expected or (Path(args.expected_file).read_text().strip() if args.expected_file else None)
-		if not prompt or not expected:
-			sys.exit('Give a prompt and an expected outcome (--prompt/--prompt-file and --expected/--expected-file).')
-		case = {'title': args.title or prompt[:60], 'expected': expected}
-	builders = [b.strip() for b in args.builders.split(',') if b.strip()]
-	unknown = set(builders) - RUNNERS
-	if unknown:
-		sys.exit(f'Unknown builders: {", ".join(sorted(unknown))}. Use iai, claude, codex.')
-
-	root = Path(setting('COMPARE_RUNS_DIR', default='/tmp/builder-compare'))
-	# The child scripts run from HERE, so a relative path must not reach them.
-	run_dir = (Path(args.run_dir) if args.run_dir else root / f'{datetime.now():%Y%m%d-%H%M}-{slug(case["title"])}').resolve()
-	run_dir.mkdir(parents=True, exist_ok=True)
-	(run_dir / 'prompt.txt').write_text(prompt + '\n')
-	(run_dir / 'case.json').write_text(json.dumps(case, indent=2))
-	scenarios = Path(args.scenarios_file) if args.scenarios_file else (Path(args.rerun) / 'scenarios.json' if args.rerun else None)
-	if scenarios and scenarios.exists():
-		json.loads(scenarios.read_text())
-		shutil.copy(scenarios, run_dir / 'scenarios.json')
-	elif args.scenarios_file:
-		sys.exit(f'No scenarios file at {scenarios}.')
-	print(f'run directory: {run_dir}', flush=True)
-
-	step('Check the setup')
-	check = subprocess.run([sys.executable, 'prepare.py'], cwd=HERE, capture_output=True, text=True)
-	print(check.stdout, end='')
-	try:
-		status = json.loads(check.stdout)
-	except json.JSONDecodeError:
-		sys.exit(check.stderr or 'prepare.py failed')
-	if not status.get('login'):
-		sys.exit('Cannot log in to n8n. Is it running? Check N8N_BASE_URL, N8N_EVAL_EMAIL and N8N_EVAL_PASSWORD.')
-	isolation = setting('COMPARE_ISOLATION', default='container')
-	skipped = {}
-	if 'iai' in builders and not status.get('instanceAiEnabled'):
-		skipped['iai'] = 'Instance AI is not enabled on this instance.'
-	for cli in ('claude', 'codex'):
-		if cli in builders and not status.get('mcpTokenWorks'):
-			skipped[cli] = status.get('mcpTokenHelp', 'The MCP token does not work.')
-		elif cli in builders and not status['keys'].get(cli):
-			skipped[cli] = f'No API key. Set {cli.upper()}_API_KEY in .env.ai-surfaces.'
-	if isolation == 'container' and not status.get('docker'):
-		skipped.setdefault('codex', 'Docker is not running. Codex runs only in a container.')
-		if 'claude' in builders and 'claude' not in skipped:
-			print('Docker is not running. Claude falls back to clean-process isolation.', flush=True)
-			isolation = 'clean'
-	if isolation == 'clean':
-		skipped.setdefault('codex', 'Codex runs only in a container. Clean isolation is for Claude only.')
-		if not status.get('claudeOnHost'):
-			skipped.setdefault('claude', 'Clean isolation runs claude on the host, and claude is not on PATH.')
-	skipped = {name: reason for name, reason in skipped.items() if name in builders}
-	for name, reason in skipped.items():
-		print(f'skip {name}: {reason}', flush=True)
-	builders = [b for b in builders if b not in skipped]
-	parallel = not args.sequential and len(builders) > 1
-
-	cli_mode = setting('COMPARE_CLI_MODE', default='interactive')
-	screens = {b: ('shot' if recording() else None) if b == 'iai'
-	           else ('cast' if isolation == 'container' and cli_mode == 'interactive' else None) for b in builders}
-	progress = Progress(run_dir, 'parallel' if parallel else 'sequential', builders, skipped, screens)
-	live_url = None
-	if live_enabled() and not args.no_live:
-		live_url = start_live_server(run_dir)
-		if live_url:
-			print(f'live page: {live_url}/report/live', flush=True)
-			webbrowser.open(f'{live_url}/report/live')
-		else:
-			print(f'the live page did not start, see {run_dir / "live-server.log"}', flush=True)
-
-	results = {}
-	try:
-		if parallel and isolation == 'container' and {'claude', 'codex'} & set(builders):
-			# Both CLI runners need the image. Build it once before they start.
-			from run_cli_builder import ensure_image
-			ensure_image(False)
-
-		def run_builder(builder):
-			progress.builder(builder, status='running', startedAt=time.time())
-			started = time.time()
-			if builder == 'iai':
-				cmd = [sys.executable, 'run_instance_ai.py', '--out', str(run_dir / 'iai'),
-				       '--prompt-file', str(run_dir / 'prompt.txt')]
-			else:
-				cmd = [sys.executable, 'run_cli_builder.py', '--builder', builder, '--out', str(run_dir / builder),
-				       '--prompt-file', str(run_dir / 'prompt.txt'), '--isolation', isolation]
-			code = run(cmd, run_dir / f'{builder}.log', prefix=builder if parallel else None)
-			# run_cli_builder.py exits 0 when the CLI fails. run.json has the exit code of the CLI.
-			meta_file = run_dir / builder / 'run.json'
-			if code == 0 and meta_file.exists():
-				code = json.loads(meta_file.read_text()).get('exitCode') or 0
-			results[builder] = {'exitCode': code, 'seconds': round(time.time() - started)}
-			progress.builder(builder, status='done' if code == 0 else 'failed', finishedAt=time.time(), exitCode=code)
-
-		if parallel:
-			step(f'Run {", ".join(builders)} at the same time')
-			threads = [threading.Thread(target=run_builder, args=(b,)) for b in builders]
-			for t in threads:
-				t.start()
-			for t in threads:
-				t.join()
-		else:
-			for builder in builders:
-				step(f'Run {builder}')
-				run_builder(builder)
-
-		if 'iai' in builders and (run_dir / 'iai' / 'run.json').exists():
-			step('Fetch Instance AI usage from LangSmith')
-			progress.update(step='Fetching Instance AI usage')
-			if status['keys'].get('langsmith'):
-				# LangSmith receives traces with a delay.
-				finished = progress.data['builders']['iai'].get('finishedAt') or time.time()
-				time.sleep(max(0.0, 30 - (time.time() - finished)))
-				run([sys.executable, 'langsmith_usage.py', '--run-dir', str(run_dir / 'iai')])
-			else:
-				print('LANGSMITH_API_KEY is not set. The report has no Instance AI cost or tokens.', flush=True)
-
-		step('Capture the canvases')
-		progress.update(step='Capturing the canvases')
-		node = shutil.which('node')
-		if node:
-			run([node, 'capture_canvas.mjs', str(run_dir)])
-		else:
-			print('node is not on PATH. No canvases.', flush=True)
-
-		if not is_on('COMPARE_SCENARIOS'):
-			print('COMPARE_SCENARIOS=0. The workflows do not run with pinned data.', flush=True)
-		elif not node or not status.get('mcpTokenWorks'):
-			print('Pinned-data runs need node and a working MCP token. Skipped.', flush=True)
-		else:
-			step('Run each workflow with pinned data')
-			progress.update(step='Running the workflows with pinned data')
-			run([sys.executable, 'run_scenarios.py', str(run_dir)], run_dir / 'scenarios.log')
-
-		if node and is_on('COMPARE_WIRE'):
-			step('Run each workflow in process with recorded API requests')
-			progress.update(step='Running the workflows with recorded API requests')
-			run([node, 'run_wire.mjs', str(run_dir)], run_dir / 'wire.log')
-		elif node:
-			print('COMPARE_WIRE=0. No in-process runs with recorded API requests.', flush=True)
-
-		pnpm = shutil.which('pnpm')
-		if pnpm and is_on('COMPARE_BINARY_CHECKS'):
-			step('Run the eval binary checks')
-			progress.update(step='Running the eval binary checks')
-			run([pnpm, '--filter', '@n8n/instance-ai', 'exec', 'tsx', str(HERE / 'binary_checks.mts'), str(run_dir)],
-			    run_dir / 'binary-checks.log')
-
-		step('Check the final-message claims')
-		run([sys.executable, 'claims.py', str(run_dir)])
-
-		if node:
-			step('Derive the deterministic checks')
-			progress.update(step='Deriving the deterministic checks')
-			run([node, 'derive_checks.mjs', str(run_dir)])
-
-		step('Summarize the runs for the judges')
-		summary = subprocess.run([sys.executable, 'summarize_runs.py', str(run_dir)], cwd=HERE,
-		                         capture_output=True, text=True)
-		(run_dir / 'summary.txt').write_text(summary.stdout)
-		run([sys.executable, 'summarize_runs.py', str(run_dir), '--blind'])
-
-		step('Build the report')
-		# The agent does the price and judge steps after this script. The page shows them as queued until then.
-		progress.update(step='Building the report', postRun={'prices': 'queued', 'judge': 'queued'})
-		run([sys.executable, 'build_report.py', str(run_dir)])
-	finally:
-		progress.finish()
-
-	models_file = run_dir / 'models.json'
-	print(json.dumps({
-		'runDir': str(run_dir),
-		'report': str(run_dir / 'report' / 'index.html'),
-		'live': f'{live_url}/report/index.html' if live_url else None,
-		'parallel': parallel,
-		'summary': str(run_dir / 'summary.txt'),
-		'blindSummary': str(run_dir / 'judge' / 'summary-blind.txt'),
-		'models': json.loads(models_file.read_text()) if models_file.exists() else [],
-		'results': results,
-		'skipped': skipped,
-		'next': 'Run build_report.py --step prices=running, add published prices for the CLI models to case.json, '
-		        'run build_report.py --step prices=done --step judge=running, give judge/summary-blind.txt to '
-		        '3 judges with a fresh context, write judge/verdicts-1.json to verdicts-3.json, run '
-		        'merge_verdicts.py, then run build_report.py --step judge=done.',
-	}, indent=2))
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/derive_checks.mjs

@@ -1,743 +0,0 @@
-#!/usr/bin/env node
-// Deterministic checks for each workflow that a run created. No LLM is involved.
-//
-//   node derive_checks.mjs <run-dir>
-//
-// Reads <builder>/workflow.json, canvas-issues.json and exec.json, and the optional wire.json,
-// claims.json and binary-checks.json. Writes <builder>/auto-checks.json. A group with gating true
-// changes the verdict. The other groups show a separate score.
-//   correctness  node definitions, canvas issues, graph and expression references (gating)
-//   execution    the pinned-data scenarios of run_scenarios.py (gating)
-//   contract     service-call requests of run_wire.mjs (gating)
-//   edge, ai     edge-case and real-AI scenarios of run_wire.mjs
-//   claims       the final message against the trace, from claims.py
-//   standards    rules from n8n-io/skills and the workflow-sdk best practices
-//   binary       the deterministic eval binary checks, from binary_checks.mts
-import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
-import { join, resolve } from 'node:path';
-import { fileURLToPath } from 'node:url';
-
-import {
-	LOGIC_TYPES,
-	SINGLE_ITEM_TYPES,
-	TRIGGER_FIXTURES,
-	downstream,
-	graph,
-	isManualTrigger,
-	isTrigger,
-	loadSdk,
-	lookup,
-	reachable,
-	repo,
-	sample,
-	shortType,
-	showPath,
-	strings,
-	typeMatches,
-	workflowReferences,
-} from './sdk.mjs';
-
-const VERSION = 1;
-const SKILLS = 'https://github.com/n8n-io/skills/blob/27b6b9f429ac0be90cae764512dbc579d7107e95/';
-const GUIDES = 'packages/@n8n/workflow-sdk/src/prompts/best-practices/guides/';
-const LC = '@n8n/n8n-nodes-langchain.';
-
-const src = {
-	validator: { label: '@n8n/workflow-sdk validateWorkflow with the node definitions' },
-	canvas: { label: 'Node issues in the n8n editor (canvas status icons)' },
-	graph: { label: 'Workflow connections' },
-	expressions: { label: 'Expression references in node parameters' },
-	execution: { label: 'Real output of the pinned-data runs (exec.json)' },
-	wire: { label: 'In-process run with recorded API fixtures (wire.json)' },
-	claims: { label: 'Final message, trace tool calls and workflow (claims.json)' },
-	errorHandling: { label: 'n8n-error-handling-official', url: `${SKILLS}skills/n8n-error-handling-official/SKILL.md` },
-	expressionsSkill: { label: 'n8n-expressions-official', url: `${SKILLS}skills/n8n-expressions-official/SKILL.md` },
-	structured: {
-		label: 'n8n-ai-nodes-official STRUCTURED_OUTPUT.md',
-		url: `${SKILLS}skills/n8n-ai-nodes-official/references/STRUCTURED_OUTPUT.md`,
-	},
-	memory: { label: 'n8n-ai-nodes-official MEMORY.md', url: `${SKILLS}skills/n8n-ai-nodes-official/references/MEMORY.md` },
-	comms: {
-		label: 'n8n-node-configuration-official COMMS_NODES.md',
-		url: `${SKILLS}skills/n8n-node-configuration-official/references/COMMS_NODES.md`,
-	},
-	triage: { label: 'workflow-sdk best practices: triage', path: `${GUIDES}triage.ts` },
-	chatbot: { label: 'workflow-sdk best practices: chatbot', path: `${GUIDES}chatbot.ts` },
-	multiItem: {
-		label: 'WhatsAppTrigger.node.ts emits one item for each webhook change',
-		path: 'packages/nodes-base/nodes/WhatsApp/WhatsAppTrigger.node.ts',
-	},
-};
-
-const check = (id, ok, text, source, details = [], extra = {}) => ({ id, ok, text, source, details, ...extra });
-const plural = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`;
-const readJson = (file) => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
-
-// --- Node kinds ----------------------------------------------------------------
-
-const isAiRoot = (g, node) => node.type.startsWith(LC) && !g.subnodes.has(node.name) && g.aiInputs(node.name, 'ai_languageModel').length > 0;
-const isBranch = (node) => typeMatches(node.type, 'if') || typeMatches(node.type, 'switch');
-/** Nodes that call a service: not logic, not a trigger, not a sub-node. AI root nodes count. */
-const isNetwork = (g, node) =>
-	!g.subnodes.has(node.name) && !isTrigger(node.type) && !LOGIC_TYPES.has(node.type) && !typeMatches(node.type, 'stickyNote');
-
-// Triggers that start one execution with one item. Other triggers can batch several events.
-const SINGLE_ITEM_TRIGGERS = ['manualTrigger', 'scheduleTrigger', 'cron', 'interval', 'webhook', 'chatTrigger', 'formTrigger', 'telegramTrigger', 'slackTrigger'];
-const MESSAGING_TRIGGERS = ['whatsAppTrigger', 'telegramTrigger', 'slackTrigger', 'chatTrigger', 'discordTrigger', 'microsoftTeamsTrigger', 'gmailTrigger'];
-const ITEM_SOURCES = ['splitOut', 'code', 'itemLists'];
-
-/** Names of the nodes that can output more than one item in one execution. */
-function multiItemNodes(g) {
-	const multi = new Set();
-	const order = [...reachable(g)];
-	let grew = true;
-	while (grew) {
-		grew = false;
-		for (const name of order) {
-			if (multi.has(name)) continue;
-			const node = g.byName.get(name);
-			if (!node || g.subnodes.has(name) || SINGLE_ITEM_TYPES.has(node.type) || node.executeOnce) continue;
-			const fromTrigger = isTrigger(node.type) && !SINGLE_ITEM_TRIGGERS.some((t) => typeMatches(node.type, t));
-			const source = ITEM_SOURCES.some((t) => typeMatches(node.type, t)) || (isNetwork(g, node) && !isAiRoot(g, node) && /^(read|getAll|search|list|getMany)$/.test(node.parameters?.operation ?? ''));
-			if (fromTrigger || source || g.mainParents(name).some((p) => multi.has(p))) {
-				multi.add(name);
-				grew = true;
-			}
-		}
-	}
-	return multi;
-}
-
-/** Names of the nodes upstream of name. A sub-node uses the nodes upstream of the root that it serves. */
-function upstream(g, name) {
-	const roots = new Set([name]);
-	for (let grew = true; grew; ) {
-		grew = false;
-		for (const e of g.edges) {
-			if (e.type.startsWith('ai_') && roots.has(e.from) && !roots.has(e.to)) {
-				roots.add(e.to);
-				grew = true;
-			}
-		}
-	}
-	const seen = new Set();
-	const queue = [...roots];
-	while (queue.length) {
-		for (const parent of g.mainParents(queue.shift())) {
-			if (!seen.has(parent)) {
-				seen.add(parent);
-				queue.push(parent);
-			}
-		}
-	}
-	return seen;
-}
-
-// --- Correctness ---------------------------------------------------------------
-
-function checkDefinitions(workflow) {
-	const loaded = loadSdk();
-	if (!loaded || loaded.missingDirs.length) {
-		const missing = loaded ? loaded.missingDirs.map((d) => d.replace(repo + '/', '')) : ['packages/@n8n/workflow-sdk/dist'];
-		return check('C1', null, 'Node parameters match the node definitions', src.validator, [
-			`Not checked. Build these first: ${missing.join(', ')}.`,
-		]);
-	}
-	const result = loaded.sdk.validateWorkflow(workflow, { allowDisconnectedNodes: true, allowNoTrigger: true });
-	const problems = [...result.errors, ...result.warnings.filter((w) => w.severity !== 'informational')];
-	const details = problems.map((p) =>
-		p.nodeName && !p.message.includes(`"${p.nodeName}"`) ? `${p.nodeName}: ${p.message}` : p.message,
-	);
-	const info = result.warnings.filter((w) => w.severity === 'informational').map((w) => `Info: ${w.message}`);
-	return check(
-		'C1',
-		problems.length === 0,
-		problems.length ? `${plural(problems.length, 'node parameter problem')} against the node definitions` : 'Node parameters match the node definitions',
-		src.validator,
-		details.concat(info),
-	);
-}
-
-const isEmpty = (value) =>
-	value === undefined || value === null || value === '' || (typeof value === 'object' && value.__rl && (value.value === '' || value.value == null));
-
-/** Sort the editor issues into setup work for the user and real problems. */
-function classifyCanvas(workflow, canvas) {
-	const byName = new Map((workflow.nodes ?? []).map((n) => [n.name, n]));
-	const setup = [];
-	const problems = [];
-	for (const node of canvas.nodes) {
-		if (node.marker === 'node-not-installed') problems.push(`${node.name}: the node type is not installed`);
-		if (node.marker === 'node-restricted') problems.push(`${node.name}: the node type is restricted`);
-		for (const [kind, issue] of Object.entries(node.issues ?? {})) {
-			if (kind === 'credentials') {
-				for (const type of Object.keys(issue)) setup.push(`${node.name}: connect ${type} credentials`);
-			} else if (kind === 'parameters') {
-				for (const [param, messages] of Object.entries(issue)) {
-					const value = byName.get(node.name)?.parameters?.[param];
-					for (const message of messages) {
-						if (/is required/i.test(message) && isEmpty(value)) setup.push(`${node.name}: ${message.replace(/\.$/, '')} (left for the user)`);
-						else problems.push(`${node.name}: ${message}`);
-					}
-				}
-			} else if (issue === true || (issue && typeof issue === 'object' && Object.keys(issue).length)) {
-				const text = typeof issue === 'object' ? Object.values(issue).flat().join(' ') : '';
-				problems.push(`${node.name}: ${kind === 'typeUnknown' ? 'n8n does not know this node type' : `${kind} issue`}${text ? ` (${text})` : ''}`);
-			}
-		}
-	}
-	return { setup, problems };
-}
-
-function checkCanvas(workflow, canvas) {
-	if (!canvas) return [check('C2', null, 'The n8n editor shows no node errors', src.canvas, ['Not checked. Run capture_canvas.mjs first.'])];
-	if (!canvas.store && canvas.nodes.some((n) => n.marker === 'node-issues')) {
-		return [check('C2', null, 'The n8n editor shows no node errors', src.canvas, [
-			'The editor store was not readable, so the node issues are not known. Icons: ' +
-				canvas.nodes.map((n) => `${n.name} (${n.marker})`).join(', '),
-		])];
-	}
-	const { setup, problems } = classifyCanvas(workflow, canvas);
-	return [
-		check('C2', problems.length === 0, problems.length ? `The n8n editor shows ${plural(problems.length, 'node error')}` : 'The n8n editor shows no node errors', src.canvas, problems),
-		check('C2s', true, setup.length ? `Setup left for the user: ${plural(setup.length, 'item')}` : 'No setup left for the user', src.canvas, setup, { level: 'info' }),
-	];
-}
-
-function checkReachable(g) {
-	if (g.triggers.length === 0) return check('C3', false, 'The workflow has a trigger', src.graph, ['No trigger node.']);
-	const seen = reachable(g);
-	const orphans = g.nodes.filter((n) => !seen.has(n.name)).map((n) => n.name);
-	return check(
-		'C3',
-		orphans.length === 0,
-		orphans.length ? `${plural(orphans.length, 'node')} cannot be reached from a trigger` : 'Every node can be reached from a trigger',
-		src.graph,
-		orphans.map((n) => `${n} is not connected to the flow`),
-	);
-}
-
-function checkNamedRefs(workflow, g, refs) {
-	const allNames = new Set((workflow.nodes ?? []).map((n) => n.name));
-	const problems = [];
-	for (const { node, param, ref } of refs) {
-		if (ref.kind !== 'named') continue;
-		if (!allNames.has(ref.node)) problems.push(`${node} › ${param}: no node is called "${ref.node}"`);
-		else if (!g.byName.has(ref.node)) problems.push(`${node} › ${param}: "${ref.node}" is disabled`);
-		else if (!/isExecuted/.test(ref.raw) && !upstream(g, node).has(ref.node))
-			problems.push(`${node} › ${param}: "${ref.node}" does not run before this node, so the reference has no data`);
-	}
-	const named = refs.filter((r) => r.ref.kind === 'named').length;
-	return check(
-		'C4',
-		problems.length === 0,
-		problems.length ? `${plural(problems.length, 'node reference')} cannot resolve` : `All ${plural(named, 'node reference')} name a node that runs before them`,
-		src.expressions,
-		[...new Set(problems)],
-	);
-}
-
-const DEFINITIVE = new Set(['executed', 'fixture', 'case', 'output-parser']);
-const INDICATIVE = new Set(['node-schema', 'prompt-example', 'ai-root']);
-// Outputs with the real number of array items. The other provenances are schema samples.
-const REAL_OUTPUT = new Set(['executed', 'fixture', 'case']);
-
-/** The output items of a node across all scenarios, with their provenance. */
-function outputsOf(name, g, exec) {
-	const items = [];
-	for (const scenario of exec?.scenarios ?? []) {
-		const out = scenario.nodes?.[name];
-		if (out?.status === 'success' && Array.isArray(out.items)) items.push(...out.items.filter((i) => i && typeof i === 'object'));
-	}
-	const node = g.byName.get(name);
-	let provenance = exec?.executed?.includes(name) ? 'executed' : exec?.pinned?.[name];
-	if (items.length && provenance) return { items, provenance };
-	if (TRIGGER_FIXTURES[node.type]) return { items: TRIGGER_FIXTURES[node.type], provenance: 'fixture' };
-	const schema = loadSdk()?.sdk.discoverOutputSchemaForNode(node.type, node.typeVersion, node.parameters ?? {});
-	if (schema) return { items: [sample(schema, 'item')], provenance: 'node-schema' };
-	return { items: [], provenance: 'unknown' };
-}
-
-function checkFieldPaths(g, refs, exec) {
-	const missing = [];
-	const unconfirmed = [];
-	let checked = 0;
-	let unknown = 0;
-	const seen = new Set();
-	for (const { node, param, ref, code } of refs) {
-		if (code || !ref.path?.length) continue;
-		if (ref.kind !== 'named' && g.subnodes.has(node)) continue;
-		const sources = ref.kind === 'named' ? [ref.node] : g.mainParents(node);
-		if (!sources.length || sources.some((s) => !g.byName.has(s))) continue;
-		const key = `${node}|${param}|${ref.raw}`;
-		if (seen.has(key)) continue;
-		seen.add(key);
-		let found = false;
-		let anyUnknown = false;
-		const misses = [];
-		for (const source of sources) {
-			const { items, provenance } = outputsOf(source, g, exec);
-			if (!items.length || provenance === 'unknown' || provenance === 'empty' || provenance === 'placeholder') {
-				anyUnknown = true;
-				continue;
-			}
-			const results = items.map((item) => lookup(item, ref.path, { sample: !REAL_OUTPUT.has(provenance) }));
-			if (results.includes('found')) found = true;
-			else if (results.includes('unknown')) anyUnknown = true;
-			else misses.push({ source, provenance });
-		}
-		if (found) checked++;
-		else if (anyUnknown || !misses.length) unknown++;
-		else {
-			checked++;
-			const line = `${node} › ${param}: ${ref.raw.trim()} reads "${showPath(ref.path)}", which is not in the output of ${misses.map((m) => `${m.source} (${m.provenance})`).join(' or ')}`;
-			if (misses.some((m) => DEFINITIVE.has(m.provenance))) missing.push(line);
-			else if (misses.some((m) => INDICATIVE.has(m.provenance))) unconfirmed.push(`Unconfirmed: ${line}`);
-		}
-	}
-	const note = exec?.scenarios?.length ? [] : ['No pinned-data run, so node outputs come from fixtures and node schemas only.'];
-	return check(
-		'C5',
-		missing.length === 0,
-		missing.length
-			? `${plural(missing.length, 'expression')} read a field that the source node does not output`
-			: `${plural(checked, 'expression field')} found in the source node output${unknown ? `, ${unknown} not checkable` : ''}`,
-		src.execution,
-		[...missing, ...unconfirmed, ...note],
-	);
-}
-
-const SECRET_PATTERNS = [
-	/sk-(?:proj-|ant-)?[A-Za-z0-9_-]{20,}/,
-	/xox[abpr]-[A-Za-z0-9-]{10,}/,
-	/AIza[0-9A-Za-z_-]{35}/,
-	/gh[pousr]_[A-Za-z0-9]{36}/,
-	/Bearer\s+(?!\{\{)[A-Za-z0-9._~+/-]{24,}/,
-	/-----BEGIN [A-Z ]*PRIVATE KEY-----/,
-];
-
-function checkSecrets(g) {
-	const hits = [];
-	for (const node of g.nodes) {
-		for (const { path, value } of strings(node.parameters ?? {})) {
-			if (SECRET_PATTERNS.some((re) => re.test(value))) hits.push(`${node.name} › ${path}: a value looks like a key or token`);
-		}
-	}
-	return check('C6', hits.length === 0, hits.length ? 'Secrets are written into node parameters' : 'No secrets in node parameters', src.expressions, hits);
-}
-
-// --- Execution -----------------------------------------------------------------
-
-function executionChecks(exec) {
-	if (!exec) return [check('E0', null, 'The workflow runs with pinned data', src.execution, ['Not run. Run run_scenarios.py first.'])];
-	if (exec.skipped) return [check('E0', null, 'The workflow runs with pinned data', src.execution, [exec.skipped])];
-	const pinned = Object.entries(exec.pinned ?? {}).map(([n, p]) => `${n} (${p})`);
-	const intro = [`Pinned: ${pinned.join(', ') || 'nothing'}.`, `Ran for real: ${(exec.executed ?? []).join(', ') || 'nothing'}.`];
-	const out = [];
-	exec.scenarios.forEach((s, i) => {
-		const errors = Object.entries(s.nodes ?? {}).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
-		const ran = Object.keys(s.nodes ?? {});
-		const ok = s.status === 'success' && errors.length === 0;
-		out.push(check(`E${i + 1}`, ok, `"${s.name}" runs without an error`, src.execution, [
-			...(i === 0 ? intro : []),
-			`Status: ${s.status}${s.error ? ` (${s.error})` : ''}.`,
-			...errors,
-			`Nodes that ran: ${ran.join(', ')}.`,
-		], { executionId: s.executionId }));
-		(s.checks ?? []).forEach((c, j) => {
-			const verb = c.kind === 'reach' ? 'reaches' : 'does not reach';
-			out.push(check(`E${i + 1}.${j + 1}`, c.ok, `"${s.name}" ${verb} a ${c.type} node`, src.execution, []));
-		});
-	});
-	return out;
-}
-
-// --- In-process runs, eval checks and claims -------------------------------------
-
-const WIRE_KINDS = {
-	contract: { id: 'W', title: 'Service calls', gating: true, note: 'The workflow runs in process. Service nodes run for real against recorded API fixtures, so the check sees each HTTP request that a node sends. The AI node is pinned. A failure here changes the verdict.' },
-	edge: { id: 'X', title: 'Edge cases', gating: false, note: 'Inputs that the prompt does not describe but the trigger can send: several messages in one call, status updates, empty data. This score does not change the verdict.' },
-	ai: { id: 'A', title: 'Real AI runs', gating: false, note: 'The AI node calls the real model on labeled questions. Service nodes use the recorded API fixtures. Model output can change between runs, so this score does not change the verdict.' },
-};
-
-function wireCheckText(name, c) {
-	if (c.kind === 'reach') return `"${name}" reaches a ${c.type} node`;
-	if (c.kind === 'skip') return `"${name}" does not reach a ${c.type} node`;
-	if (c.kind === 'noWrites') return `"${name}" sends no write request`;
-	if (c.kind === 'eachItem') return `"${name}" sends a request for each of the ${c.ids.length} items`;
-	const what = c.contains?.length ? ` that carries ${c.contains.map((m) => m.split(' or ').map((x) => (x.startsWith('@') ? `\`${x}\`` : x)).join(' or ')).join(', ')}` : '';
-	const count = c.count === undefined ? 'a request' : plural(c.count, 'request');
-	return `"${name}" sends ${count} from a ${c.type} node${what}`;
-}
-
-function requestLines(s) {
-	return (s.requests ?? []).map((r) => `Request: ${r.node} › ${r.method} ${r.url}${r.answeredBy && r.answeredBy !== 'fixture' ? ` (${r.answeredBy})` : ''}`);
-}
-
-function wireGroups(wire) {
-	if (!wire) return [];
-	const groups = [];
-	for (const [kind, meta] of Object.entries(WIRE_KINDS)) {
-		const scenarios = (wire.scenarios ?? []).filter((s) => s.kind === kind);
-		if (!scenarios.length && !(kind === 'contract' && wire.skipped)) continue;
-		const checks = [];
-		if (wire.skipped) checks.push(check(`${meta.id}0`, null, 'The workflow runs in process', src.wire, [wire.skipped]));
-		scenarios.forEach((s, i) => {
-			const id = `${meta.id}${i + 1}`;
-			if (s.status === 'skipped') {
-				checks.push(check(id, null, `"${s.name}" runs`, src.wire, [s.error]));
-				return;
-			}
-			const errors = Object.entries(s.nodes ?? {}).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
-			const ai = Object.entries(s.aiOutputs ?? {}).map(([node, items]) => `AI output of ${node}: ${JSON.stringify(items?.[0]?.output ?? items?.[0] ?? null)}`);
-			const ran = Object.entries(s.nodes ?? {}).map(([name, n]) => `${name} (${plural((n.outputCounts ?? []).reduce((a, b) => a + b, 0), 'item')})`);
-			checks.push(check(id, s.status === 'success' && !errors.length, `"${s.name}" runs without an error`, src.wire, [
-				...(s.why ? [s.why[0].toUpperCase() + s.why.slice(1)] : []),
-				`Status: ${s.status}${s.error ? ` (${s.error})` : ''}.`,
-				...errors,
-				`Nodes that ran: ${ran.join(', ') || 'none'}.`,
-				...ai,
-				...requestLines(s),
-				...(s.blocked?.length ? [`Blocked network calls: ${s.blocked.join(', ')}`] : []),
-			]));
-			(s.checks ?? []).forEach((c, j) => checks.push(check(`${id}.${j + 1}`, c.ok, wireCheckText(s.name, c), src.wire, c.details ?? [])));
-		});
-		const scored = checks.filter((c) => c.ok !== null);
-		const filled = kind === 'contract' && wire.filled?.length ? [`Setup values filled for the run: ${wire.filled.join(', ')}.`] : [];
-		groups.push({
-			key: kind === 'contract' ? 'contract' : kind,
-			title: meta.title,
-			gating: meta.gating,
-			note: [meta.note, ...filled].join(' '),
-			...(meta.gating ? {} : { score: { passed: scored.filter((c) => c.ok).length, total: scored.length } }),
-			checks,
-		});
-	}
-	return groups;
-}
-
-function binaryGroup(binary) {
-	if (!binary?.outcomes) return null;
-	const measured = binary.outcomes.filter((o) => o.status !== 'n_a');
-	const checks = measured.map((o, i) =>
-		check(`B${i + 1}`, o.status === 'error' ? null : o.status === 'pass', `${o.name}: ${o.description}`, { label: `Instance AI eval binary checks (${o.dimension})`, path: binary.source }, o.comment ? [o.comment] : []),
-	);
-	const scored = checks.filter((c) => c.ok !== null);
-	const na = binary.outcomes.length - measured.length;
-	return {
-		key: 'binary',
-		title: 'Eval binary checks',
-		gating: false,
-		note: `The deterministic binary checks of the Instance AI eval suite.${na ? ` ${na} do not apply to this workflow.` : ''} This score does not change the verdict.`,
-		score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
-		checks,
-	};
-}
-
-function claimsGroup(claims) {
-	if (!claims) return null;
-	const checks = claims.checks.map((c) => check(c.id, c.ok, c.text, src.claims, c.details));
-	if (claims.error) checks.unshift(check('M0', null, 'The final message and trace are readable', src.claims, [claims.error]));
-	const scored = checks.filter((c) => c.ok !== null);
-	return {
-		key: 'claims',
-		title: 'Final message claims',
-		gating: false,
-		note: 'What the final message says about the build, compared with the trace tool calls, the workflow and the service-call runs. This score does not change the verdict.',
-		score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
-		checks,
-	};
-}
-
-// --- Standards -----------------------------------------------------------------
-
-function unattended(g) {
-	return g.triggers.some((t) => !isManualTrigger(t.type) && !typeMatches(t.type, 'chatTrigger') && !typeMatches(t.type, 'formTrigger'));
-}
-
-function errorWorkflowRule(workflow, g) {
-	if (!unattended(g)) return null;
-	const set = Boolean(workflow.settings?.errorWorkflow);
-	return check('S1', set, 'An unattended workflow has an error workflow', src.errorHandling, set ? [] : ['settings.errorWorkflow is not set, so a failure goes nowhere.']);
-}
-
-function errorOutputRule(g) {
-	const problems = [];
-	for (const node of g.nodes) {
-		if (node.onError === 'continueRegularOutput') problems.push(`${node.name}: onError continueRegularOutput passes the error on as normal data`);
-		// The error output comes after the normal outputs, so it is never output 0.
-		if (node.onError === 'continueErrorOutput' && !g.mainChildren(node.name).some((e) => e.output > 0))
-			problems.push(`${node.name}: onError continueErrorOutput has no node on its error output`);
-	}
-	if (!g.nodes.some((n) => n.onError)) return null;
-	return check('S12', problems.length === 0, 'Failed nodes go to a wired error output, not on as data', src.errorHandling, problems);
-}
-
-function retryRule(g) {
-	const nodes = g.nodes.filter((n) => isNetwork(g, n));
-	if (!nodes.length) return null;
-	const missing = nodes.filter((n) => !n.retryOnFail).map((n) => `${n.name}: no retryOnFail`);
-	return check('S2', missing.length === 0, `Nodes that call a service retry on failure (${nodes.length - missing.length} of ${nodes.length})`, src.errorHandling, missing);
-}
-
-function namedRefRule(refs) {
-	const uses = refs.filter((r) => !r.code && (r.ref.kind === 'json' || r.ref.kind === 'input'));
-	if (!refs.some((r) => !r.code)) return null;
-	const where = [...new Set(uses.map((r) => `${r.node} › ${r.param}`))];
-	return check('S3', uses.length === 0, uses.length ? `${plural(where.length, 'parameter')} read $json instead of a node name` : 'Expressions read data by node name', src.expressionsSkill, where);
-}
-
-function parserParams(g, root) {
-	const parser = g.aiInputs(root.name, 'ai_outputParser')[0];
-	return parser ? { parser, p: parser.parameters ?? {} } : null;
-}
-
-function structuredRule(g) {
-	const roots = g.nodes.filter((n) => isAiRoot(g, n) && g.mainChildren(n.name).length);
-	if (!roots.length) return null;
-	const problems = [];
-	for (const root of roots) {
-		const found = parserParams(g, root);
-		if (!found) {
-			const declared = strings(root.parameters ?? {}).some(({ value }) => /json/i.test(value));
-			if (declared) problems.push(`${root.name}: the JSON format is only in the prompt, with no structured output parser`);
-			continue;
-		}
-		if ((found.p.schemaType ?? 'fromJson') !== 'manual') problems.push(`${found.parser.name}: schemaType is not 'manual'`);
-		if (!found.p.autoFix) problems.push(`${found.parser.name}: autoFix is off`);
-	}
-	return check('S4', problems.length === 0, 'Structured AI output uses a manual schema with autoFix', src.structured, problems);
-}
-
-const ATTRIBUTION_FIELDS = ['appendAttribution', 'includeLinkToWorkflow'];
-const definitionCache = new Map();
-
-/** The text of the node definition file for a node, or ''. */
-function definitionText(node) {
-	const [pkg, short] = node.type.startsWith(LC) ? ['@n8n/nodes-langchain', node.type.slice(LC.length)] : ['nodes-base', shortType(node.type)];
-	const pkgDir = node.type.startsWith(LC) ? '@n8n/n8n-nodes-langchain' : 'n8n-nodes-base';
-	const dir = join(repo, 'packages', pkg, 'dist/node-definitions/nodes', pkgDir, short, `v${String(node.typeVersion ?? 1).replace('.', '')}`);
-	const snake = (s) => String(s ?? '').replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase();
-	const p = node.parameters ?? {};
-	const files = [join(dir, `resource_${snake(p.resource)}`, `operation_${snake(p.operation)}.ts`), `${dir}.ts`];
-	if (!p.resource && p.operation && existsSync(dir)) {
-		for (const entry of readdirSync(dir)) if (entry.startsWith('resource_')) files.push(join(dir, entry, `operation_${snake(p.operation)}.ts`));
-	}
-	const key = files.join('|');
-	if (!definitionCache.has(key)) definitionCache.set(key, files.filter(existsSync).map((f) => readFileSync(f, 'utf8')).join('\n'));
-	return definitionCache.get(key);
-}
-
-function findKey(value, key) {
-	if (!value || typeof value !== 'object') return undefined;
-	if (key in value) return value[key];
-	for (const v of Object.values(value)) {
-		const hit = findKey(v, key);
-		if (hit !== undefined) return hit;
-	}
-	return undefined;
-}
-
-function attributionRule(g) {
-	const nodes = [];
-	for (const node of g.nodes) {
-		const text = definitionText(node);
-		const field = ATTRIBUTION_FIELDS.find((f) => text.includes(`${f}?:`));
-		if (field) nodes.push({ node, field });
-	}
-	if (!nodes.length) return null;
-	const on = nodes.filter(({ node, field }) => findKey(node.parameters, field) !== false).map(({ node, field }) => `${node.name}: ${field} is not false (the default adds an n8n attribution)`);
-	return check('S5', on.length === 0, 'Messages do not carry the n8n attribution', src.comms, on);
-}
-
-function multiItemRule(g, refs) {
-	const multi = multiItemNodes(g);
-	if (!multi.size) return null;
-	const hits = refs.filter((r) => r.ref.kind === 'named' && ['first', 'last', 'all-index'].includes(r.ref.accessor) && multi.has(r.ref.node));
-	const where = [...new Set(hits.map((r) => `${r.node} › ${r.param}: ${r.ref.raw.trim()}`))];
-	const triggers = g.triggers.filter((t) => multi.has(t.name));
-	const source = triggers.some((t) => typeMatches(t.type, 'whatsAppTrigger')) ? { ...src.multiItem, also: src.expressionsSkill } : src.expressionsSkill;
-	return check(
-		'S6',
-		where.length === 0,
-		where.length ? `${plural(where.length, 'reference')} read only the first item of a node that can output several` : 'References keep each item paired with its own data',
-		source,
-		where.length ? [...where, `Nodes that can output several items: ${[...multi].join(', ')}.`] : [],
-	);
-}
-
-/** AI root nodes whose output decides a branch, with the IF or Switch nodes that they feed. */
-function decidingRoots(g) {
-	return g.nodes
-		.filter((n) => isAiRoot(g, n))
-		.map((root) => ({ root, branches: [...downstream(g, root.name)].map((n) => g.byName.get(n)).filter((n) => n && isBranch(n)) }))
-		.filter((r) => r.branches.length);
-}
-
-function temperatureRule(g) {
-	const roots = decidingRoots(g);
-	if (!roots.length) return null;
-	const problems = [];
-	for (const { root } of roots) {
-		for (const model of g.aiInputs(root.name, 'ai_languageModel')) {
-			const t = findKey(model.parameters?.options, 'temperature') ?? findKey(model.parameters, 'temperature');
-			if (t === undefined) problems.push(`${model.name}: temperature is not set, so the model default is used`);
-			else if (typeof t === 'number' && t > 0.2) problems.push(`${model.name}: temperature is ${t}`);
-		}
-	}
-	return check('S7', problems.length === 0, 'A model that decides a branch has a temperature of 0.2 or lower', src.triage, problems);
-}
-
-/** Field names of a JSON Schema, or of a JSON example when the value is not a schema. */
-function schemaKeys(value, out = new Set()) {
-	if (!value || typeof value !== 'object') return out;
-	if (Array.isArray(value)) {
-		value.forEach((v) => schemaKeys(v, out));
-		return out;
-	}
-	const isSchema = value.properties || value.items || typeof value.type === 'string';
-	if (isSchema) {
-		for (const [k, v] of Object.entries(value.properties ?? {})) {
-			out.add(k);
-			schemaKeys(v, out);
-		}
-		if (value.items) schemaKeys(value.items, out);
-		return out;
-	}
-	for (const [k, v] of Object.entries(value)) {
-		out.add(k);
-		schemaKeys(v, out);
-	}
-	return out;
-}
-
-function reasoningRule(g) {
-	const roots = decidingRoots(g);
-	if (!roots.length) return null;
-	const problems = [];
-	let judged = 0;
-	for (const { root } of roots) {
-		const found = parserParams(g, root);
-		let text = '';
-		if (found) text = String(found.p.inputSchema ?? found.p.jsonSchemaExample ?? found.p.jsonSchema ?? '');
-		else text = strings(root.parameters ?? {}).map((s) => s.value).join('\n');
-		if (!text.includes('{')) continue;
-		judged++;
-		let keys;
-		try {
-			keys = [...schemaKeys(JSON.parse(text))];
-		} catch {
-			keys = [...text.matchAll(/"([A-Za-z_]\w*)"\s*:/g)].map((m) => m[1]);
-		}
-		if (!keys.some((k) => /reason|rationale|explanation|why/i.test(k))) problems.push(`${root.name}: the output has no reasoning field (fields: ${keys.slice(0, 8).join(', ')})`);
-	}
-	if (!judged) return null;
-	return check('S8', problems.length === 0, 'AI output that decides a branch includes a reasoning field', src.triage, problems);
-}
-
-function fallbackRule(g) {
-	const roots = decidingRoots(g);
-	if (!roots.length) return null;
-	const problems = [];
-	const branches = new Map(roots.flatMap((r) => r.branches.map((b) => [b.name, b])));
-	for (const node of branches.values()) {
-		const used = new Set(g.mainChildren(node.name).map((e) => e.output));
-		if (typeMatches(node.type, 'if')) {
-			for (const [i, label] of [[0, 'true'], [1, 'false']]) if (!used.has(i)) problems.push(`${node.name}: the ${label} output goes nowhere`);
-		} else {
-			const rules = node.parameters?.rules?.values ?? node.parameters?.rules?.rules ?? [];
-			const fallback = node.parameters?.options?.fallbackOutput;
-			if (fallback !== 'extra') problems.push(`${node.name}: options.fallbackOutput is not 'extra', so unmatched items are dropped`);
-			else if (!used.has(rules.length)) problems.push(`${node.name}: the fallback output goes nowhere`);
-			rules.forEach((_, i) => !used.has(i) && problems.push(`${node.name}: output ${i} goes nowhere`));
-		}
-	}
-	return check('S9', problems.length === 0, 'Every branch after an AI decision, including the fallback, goes somewhere', src.triage, problems);
-}
-
-function memoryRule(g) {
-	const chat = g.triggers.some((t) => MESSAGING_TRIGGERS.some((m) => typeMatches(t.type, m)));
-	const agents = g.nodes.filter((n) => typeMatches(n.type, `${LC}agent`) && !g.subnodes.has(n.name));
-	if (!chat || !agents.length) return null;
-	const missing = agents.filter((a) => !g.aiInputs(a.name, 'ai_memory').length).map((a) => `${a.name}: no memory sub-node`);
-	return check('S10', missing.length === 0, 'A chatbot agent has memory', src.chatbot, missing);
-}
-
-function sessionKeyRule(g) {
-	const memories = g.nodes.filter((n) => g.subnodes.has(n.name) && /memory/i.test(shortType(n.type)) && n.parameters?.sessionIdType === 'customKey');
-	if (!memories.length) return null;
-	const problems = [];
-	for (const m of memories) {
-		const key = String(m.parameters?.sessionKey ?? '');
-		if (!key.startsWith('=')) problems.push(`${m.name}: the session key is a fixed value, so every conversation shares one memory`);
-		else if (/\$json\b/.test(key)) problems.push(`${m.name}: the session key reads $json, which a memory sub-node does not have`);
-	}
-	return check('S11', problems.length === 0, 'The memory session key names a node and identifies the conversation', { ...src.chatbot, also: src.memory }, problems);
-}
-
-// --- Main ----------------------------------------------------------------------
-
-export function deriveChecks(workflow, canvas, exec, { wire, binary, claims } = {}) {
-	const g = graph(workflow);
-	const refs = workflowReferences(g);
-	const correctness = [
-		checkDefinitions(workflow),
-		...checkCanvas(workflow, canvas),
-		checkReachable(g),
-		checkNamedRefs(workflow, g, refs),
-		checkFieldPaths(g, refs, exec),
-		checkSecrets(g),
-	];
-	const standards = [
-		errorWorkflowRule(workflow, g),
-		errorOutputRule(g),
-		retryRule(g),
-		namedRefRule(refs),
-		structuredRule(g),
-		attributionRule(g),
-		multiItemRule(g, refs),
-		temperatureRule(g),
-		reasoningRule(g),
-		fallbackRule(g),
-		memoryRule(g),
-		sessionKeyRule(g),
-	].filter(Boolean);
-	const scored = standards.filter((c) => c.ok !== null);
-	return {
-		version: VERSION,
-		groups: [
-			{ key: 'correctness', title: 'Correctness', gating: true, note: 'From the node definitions, the n8n editor and the workflow graph. A failure here changes the verdict.', checks: correctness },
-			{ key: 'execution', title: 'Execution with pinned data', gating: true, note: 'Triggers, service nodes and AI nodes get pinned data. Logic nodes run for real. A failure here changes the verdict.', checks: executionChecks(exec) },
-			...wireGroups(wire),
-			claimsGroup(claims),
-			{
-				key: 'standards',
-				title: 'n8n standards',
-				gating: false,
-				note: 'Rules from n8n-io/skills and the workflow-sdk best practices. The builders did not get these rules. This score does not change the verdict.',
-				score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
-				checks: standards,
-			},
-			binaryGroup(binary),
-		].filter(Boolean),
-	};
-}
-
-if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
-	const runDir = resolve(process.argv[2] ?? '.');
-	for (const entry of readdirSync(runDir, { withFileTypes: true })) {
-		const dir = join(runDir, entry.name);
-		if (!entry.isDirectory() || !existsSync(join(dir, 'workflow.json'))) continue;
-		const result = deriveChecks(readJson(join(dir, 'workflow.json')), readJson(join(dir, 'canvas-issues.json')), readJson(join(dir, 'exec.json')), {
-			wire: readJson(join(dir, 'wire.json')),
-			binary: readJson(join(dir, 'binary-checks.json')),
-			claims: readJson(join(dir, 'claims.json')),
-		});
-		writeFileSync(join(dir, 'auto-checks.json'), JSON.stringify(result, null, 1));
-		const line = result.groups.map((group) => {
-			const judged = group.checks.filter((c) => c.ok !== null && c.level !== 'info');
-			return `${group.key} ${judged.filter((c) => c.ok).length}/${judged.length}`;
-		});
-		console.log(`${entry.name}: auto-checks.json written (${line.join(', ')})`);
-	}
-}

.agents/skills/testmax-n8n-ai/scripts/langsmith_usage.py

@@ -1,160 +0,0 @@
-#!/usr/bin/env python3
-"""Fetch per-call token usage and cost of an Instance AI run from LangSmith.
-
-n8n does not store token usage. LangSmith has it when the instance runs with
-LANGSMITH_TRACING=true. The script reads LANGSMITH_API_KEY, LANGSMITH_ENDPOINT and
-LANGSMITH_PROJECT from the config (see n8n_api.config).
-
-  langsmith_usage.py --run-dir <run-dir>/iai
-
-It takes all LLM runs in the project between startedAt and finishedAt from run.json.
-Do not start other Instance AI work on the same instance during the run, or use --by-trace.
-
-  langsmith_usage.py --run-dir <dir> --by-trace
-
---by-trace keeps only the calls of the traces in run.json "langsmithRunIds" (the root run id of
-each run-start event is also its LangSmith trace id). Use it when other Instance AI work runs
-at the same time.
-
-The script also writes context.json: for each agent, the system prompt, the messages before
-the first model response, and the tools, from the first LLM call of that agent. Tools that the
-agent loads later with load_tool come from its last call.
-"""
-
-import argparse
-import json
-import urllib.parse
-import urllib.request
-from datetime import datetime, timedelta
-from pathlib import Path
-
-from n8n_api import setting
-
-
-def langsmith_client():
-	key = setting('LANGSMITH_API_KEY')
-	if not key:
-		raise SystemExit('LANGSMITH_API_KEY is not set. Instance AI cost and tokens are not available.')
-	base = setting('LANGSMITH_ENDPOINT', default='https://api.smith.langchain.com').rstrip('/')
-
-	def call(method, path, body=None):
-		req = urllib.request.Request(
-			base + path, method=method,
-			data=json.dumps(body).encode() if body is not None else None,
-			headers={'x-api-key': key, 'content-type': 'application/json'},
-		)
-		with urllib.request.urlopen(req, timeout=60) as res:
-			return json.load(res)
-
-	return call
-
-
-def fetch_calls(started_at, finished_at, traces=None):
-	"""LLM calls in the LangSmith project between two ISO times. traces keeps only the calls of these traces."""
-	call = langsmith_client()
-	project = setting('LANGSMITH_PROJECT', 'LANGCHAIN_PROJECT', default='instance-ai')
-
-	sessions = call('GET', '/api/v1/sessions?' + urllib.parse.urlencode({'name': project}))
-	if not sessions:
-		raise SystemExit(f'LangSmith project "{project}" not found.')
-	session_id = sessions[0]['id']
-
-	start = datetime.fromisoformat(started_at) - timedelta(seconds=5)
-	end = datetime.fromisoformat(finished_at) + timedelta(seconds=5)
-	calls, cursor = [], None
-	while True:
-		body = {
-			'session': [session_id], 'run_type': 'llm', 'limit': 100,
-			'start_time': start.isoformat(),
-			'select': ['id', 'name', 'start_time', 'end_time', 'prompt_tokens', 'completion_tokens',
-			           'total_cost', 'prompt_token_details', 'extra', 'trace_id'],
-		}
-		if cursor:
-			body['cursor'] = cursor
-		res = call('POST', '/api/v1/runs/query', body)
-		for run in res.get('runs', []):
-			started = datetime.fromisoformat(run['start_time'].replace('Z', '+00:00'))
-			if started.tzinfo is None:
-				started = started.replace(tzinfo=start.tzinfo)
-			if started > end or (traces is not None and run.get('trace_id') not in traces):
-				continue
-			details = run.get('prompt_token_details') or {}
-			metadata = (run.get('extra') or {}).get('metadata') or {}
-			calls.append({
-				'id': run.get('id'), 'trace': run.get('trace_id'), 'name': run.get('name'),
-				'start': started.isoformat(), 'end': run.get('end_time'),
-				'in': run.get('prompt_tokens') or 0,
-				'cache_read': details.get('cache_read') or 0,
-				'cache_write': details.get('cache_creation') or 0,
-				'out': run.get('completion_tokens') or 0,
-				'cost': float(run.get('total_cost') or 0),
-				'model': metadata.get('ls_model_name'),
-			})
-		cursor = (res.get('cursors') or {}).get('next')
-		if not cursor:
-			break
-	return sorted(calls, key=lambda c: c['start'])
-
-
-def message_text(content):
-	if isinstance(content, str):
-		return content
-	return '\n'.join(p.get('text', '') for p in content or [] if isinstance(p, dict) and p.get('type') == 'text')
-
-
-def tool_spec(t):
-	return {'name': t.get('name') or t.get('id'), 'description': t.get('description') or '',
-	        'schema': t.get('inputSchema') or t.get('parameters'), 'type': t.get('type')}
-
-
-def fetch_contexts(calls):
-	"""What each agent got before its first model response, in the order that the agents started."""
-	call = langsmith_client()
-	by_agent = {}
-	for c in calls:
-		if c.get('id'):
-			by_agent.setdefault(c['name'], []).append(c)
-	contexts = []
-	for name, agent_calls in by_agent.items():
-		first = call('GET', f'/api/v1/runs/{agent_calls[0]["id"]}').get('inputs') or {}
-		messages = []
-		for m in first.get('input') or []:
-			if m.get('role') not in ('system', 'user'):
-				break
-			messages.append({'role': m['role'], 'text': message_text(m.get('content'))})
-		tools = [tool_spec(t) for t in first.get('tools') or []]
-		loaded = []
-		if len(agent_calls) > 1:
-			last = call('GET', f'/api/v1/runs/{agent_calls[-1]["id"]}').get('inputs') or {}
-			known = {t['name'] for t in tools}
-			loaded = [tool_spec(t) for t in last.get('tools') or [] if (t.get('name') or t.get('id')) not in known]
-		contexts.append({'agent': name, 'messages': messages, 'tools': tools, 'loadedTools': loaded})
-	return contexts
-
-
-def totals(calls):
-	return {
-		'calls': len(calls), 'models': sorted({c['model'] for c in calls if c['model']}),
-		'input': sum(c['in'] for c in calls), 'output': sum(c['out'] for c in calls),
-		'cost': round(sum(c['cost'] for c in calls), 4),
-	}
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--run-dir', required=True)
-	ap.add_argument('--by-trace', action='store_true')
-	args = ap.parse_args()
-	run_dir = Path(args.run_dir)
-	meta = json.loads((run_dir / 'run.json').read_text())
-	traces = set(meta.get('langsmithRunIds') or [])
-	if args.by_trace and not traces:
-		raise SystemExit('run.json has no langsmithRunIds. Is LANGSMITH_TRACING=true on the instance?')
-	calls = fetch_calls(meta['startedAt'], meta['finishedAt'], traces if args.by_trace else None)
-	(run_dir / 'usage.json').write_text(json.dumps(calls, indent=1))
-	(run_dir / 'context.json').write_text(json.dumps(fetch_contexts(calls), indent=1))
-	print(json.dumps(totals(calls), indent=2))
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/live.py

@@ -1,84 +0,0 @@
-"""Live views of a run.
-
-COMPARE_LIVE=1 (default): compare.py serves a live page (live_server.py). COMPARE_LIVE=0 turns it off.
-COMPARE_LIVE_WINDOWS=1 also opens the Instance AI thread in the browser and a terminal window for
-each CLI builder. If a view cannot open, the script prints how to open it.
-COMPARE_RECORD=0 skips the screen recordings that the report replays.
-"""
-
-import shlex
-import shutil
-import subprocess
-import sys
-import webbrowser
-
-from n8n_api import setting
-
-
-def is_on(name, default='1'):
-	return setting(name, default=default).strip().lower() not in ('0', 'false', 'no', 'off')
-
-
-def enabled():
-	return is_on('COMPARE_LIVE')
-
-
-def windows():
-	return enabled() and is_on('COMPARE_LIVE_WINDOWS', default='0')
-
-
-def recording():
-	return is_on('COMPARE_RECORD')
-
-
-def open_url(url):
-	if not windows():
-		return
-	try:
-		opened = webbrowser.open(url)
-	except webbrowser.Error:
-		opened = False
-	if not opened:
-		print(f'open {url} to watch the run', flush=True)
-
-
-def log_window_launchers(command):
-	if sys.platform == 'darwin' and shutil.which('osascript'):
-		# The command goes in as an argument, so it needs no AppleScript quoting.
-		yield ['osascript', '-e', 'on run argv', '-e',
-		       'tell application "Terminal" to do script (item 1 of argv)', '-e', 'end run', command]
-	for terminal, flag in (('x-terminal-emulator', '-e'), ('gnome-terminal', '--'), ('konsole', '-e'), ('xterm', '-e')):
-		if shutil.which(terminal):
-			yield [terminal, flag, 'sh', '-c', command]
-			return
-
-
-def open_terminal(title, command, cols=None, rows=None):
-	"""Open a terminal window with a title that runs a shell command. Returns False if none opened."""
-	if not windows():
-		return True
-	prefix = f"printf '\\033]0;%s\\007' {shlex.quote(title)}; "
-	if cols and rows:
-		# Resize the window so that the full screen of the CLI fits.
-		prefix += f"printf '\\033[8;{rows};{cols}t'; "
-	for launcher in log_window_launchers(prefix + command):
-		# osascript returns at once, but xterm and other emulators run as long as the window is open.
-		# A launcher that still runs after a short wait has opened its window.
-		try:
-			proc = subprocess.Popen(launcher, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
-			                        stderr=subprocess.DEVNULL, start_new_session=True)
-		except OSError:
-			continue
-		try:
-			if proc.wait(timeout=3) == 0:
-				return True
-		except subprocess.TimeoutExpired:
-			return True
-	return False
-
-
-def open_log_window(title, path):
-	"""Open a terminal window that follows the log file at path."""
-	quoted = shlex.quote(str(path))
-	if not open_terminal(title, f'tail -n +1 -f {quoted}'):
-		print(f'watch {title}: tail -f {quoted}', flush=True)

.agents/skills/testmax-n8n-ai/scripts/live_server.py

@@ -1,635 +0,0 @@
-#!/usr/bin/env python3
-"""Serve a page that shows the builder runs while they run.
-
-  live_server.py --run-dir <run-dir> [--parent-pid <pid>]
-
-compare.py starts this server in the background. The server listens on 127.0.0.1 only and
-writes its address to <run-dir>/live-url.txt. Routes:
-
-  /report/live       the live page
-  /report/<file>     the final report and its files, when they exist
-  /api/state         the status, table row and LLM steps of each builder, as JSON
-  /api/cast/<name>   the terminal screen of a CLI builder, as Server-Sent Events in the
-                     live format of the asciinema player
-  /api/shot/<name>   the last screenshot of the Instance AI thread
-
-compare.py writes the status of each builder to <run-dir>/status.json. The server stops
-30 minutes after the last request, when the run is done or compare.py has stopped.
-
-Other skills reuse the server: they call serve() with their own app (see BuilderApp).
-"""
-
-import argparse
-import json
-import mimetypes
-import os
-import re
-import threading
-import time
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-from urllib.parse import unquote, urlparse
-
-import build_report as br
-from run_cli_builder import find_session_file
-
-IDLE_STOP_S = 1800
-KINDS = {'iai': 'instance-ai', 'claude': 'claude', 'codex': 'codex'}
-STATUS_LABELS = {'waiting': 'Waiting', 'running': 'Running', 'done': 'Done', 'failed': 'Failed', 'skipped': 'Skipped'}
-mimetypes.add_type('font/woff2', '.woff2')
-mimetypes.add_type('text/javascript', '.js')
-mimetypes.add_type('video/webm', '.webm')
-
-
-def read_status(run_dir):
-	try:
-		return json.loads((run_dir / 'status.json').read_text())
-	except (OSError, ValueError):
-		return {'phase': 'running', 'builders': {}}
-
-
-def empty_data(model=None):
-	return {'events': [], 'calls': [], 'model': model, 'duration': '-', 'cost': '-', 'costSource': None}
-
-
-def load_instance_ai_live(sub):
-	"""Trace events from the event stream that run_instance_ai.py writes while the thread runs."""
-	path = sub / 'events.jsonl'
-	rows = br.read_jsonl(path) if path.exists() else []
-	if not rows:
-		return empty_data()
-	first = next((r for r in rows if r.get('type') == 'run-start'), rows[0])
-	start, root = first['ts'] / 1000, first.get('agentId')
-	events, texts, tools = [], {}, {}
-	for r in rows:
-		kind, p, agent = r.get('type'), r.get('payload') or {}, r.get('agentId')
-		t = (r.get('ts') or first['ts']) / 1000 - start
-		label = '' if agent in (root, None) else f'{agent.split("-run_")[0]} › '
-		if kind in ('text-delta', 'reasoning-delta'):
-			event_kind = 'text' if kind == 'text-delta' else 'reasoning'
-			key = (agent, r.get('responseId'), event_kind)
-			if key not in texts:
-				texts[key] = {'kind': event_kind, 't': t, 'text': ''}
-				events.append(texts[key])
-			texts[key]['text'] += p.get('text') or ''
-		elif kind == 'tool-call':
-			tools[p.get('toolCallId')] = {'kind': 'tool', 't': t, 'name': label + (p.get('toolName') or ''),
-			                              'args': p.get('args'), 'result': 'Waiting for the result…',
-			                              'error': False, 'duration': None}
-			events.append(tools[p.get('toolCallId')])
-		elif kind in ('tool-result', 'tool-error') and p.get('toolCallId') in tools:
-			result = p.get('result') if kind == 'tool-result' else p.get('error')
-			tools[p['toolCallId']].update(
-				result=result, duration=t - tools[p['toolCallId']]['t'],
-				error=kind == 'tool-error' or (isinstance(result, dict) and (result.get('success') is False or 'error' in result)))
-		elif kind == 'confirmation-request' and p.get('toolCallId') in tools:
-			tools[p['toolCallId']]['note'] = 'Confirmation card answered by the auto-approve policy.'
-	data = empty_data()
-	data['events'] = [e for e in events if e['kind'] == 'tool' or e['text'].strip()]
-	return data
-
-
-def live_run(run_dir, name, case):
-	"""Meta and trace data of one builder. A finished builder has run.json. A running one does not."""
-	sub = run_dir / name
-	meta_file = sub / 'run.json'
-	if meta_file.exists():
-		meta = json.loads(meta_file.read_text())
-		if meta.get('kind') in br.LOADERS:
-			return meta, br.load_run(sub, meta, case.get('prices'))
-	meta = {'kind': KINDS.get(name, name)}
-	if meta['kind'] == 'instance-ai':
-		return meta, load_instance_ai_live(sub)
-	session = find_session_file(meta['kind'], sub / 'home') if (sub / 'home').exists() else None
-	if not session:
-		return meta, empty_data()
-	meta['sessionFile'] = str(session.relative_to(sub))
-	return meta, br.load_run(sub, meta, case.get('prices'))
-
-
-def seconds(value):
-	return f'{value:.0f}s'
-
-
-def gather(run_dir):
-	case, status, verdicts = br.load_case(run_dir), read_status(run_dir), br.load_verdicts(run_dir)
-	builders = status.get('builders', {})
-	names = sorted((n for n, b in builders.items() if b.get('status') != 'skipped'),
-	               key=lambda n: (br.RUN_ORDER.get(KINDS.get(n, n), 9), n))
-	now, runs = time.time(), []
-	for name in names:
-		entry = builders[name]
-		state = entry.get('status', 'waiting')
-		meta, data = live_run(run_dir, name, case)
-		running = state in ('waiting', 'running')
-		if running and data.get('loadError'):
-			# A running builder can leave a file in a state that the loader cannot read yet.
-			data = empty_data(data.get('model'))
-		started, finished = entry.get('startedAt'), entry.get('finishedAt')
-		if started and (running or data['duration'] == '-'):
-			data['duration'] = seconds((finished or now) - started)
-		if running:
-			data['cost'] = '-'
-		runs.append((name, meta, data, entry))
-	return case, status, verdicts, runs
-
-
-def status_badge(state):
-	return f'<span class="badge st-{br.esc(state)}">{STATUS_LABELS.get(state, br.esc(state))}</span>'
-
-
-def clock(status):
-	started = status.get('startedAt')
-	elapsed = seconds((status.get('finishedAt') or time.time()) - started) if started else ''
-	mode = 'Parallel' if status.get('mode') == 'parallel' else 'One at a time'
-	return ' · '.join(x for x in (mode, elapsed, status.get('step')) if x)
-
-
-def render_run(run_dir, case, name, meta, data, entry, verdict, scale):
-	row = br.render_row_cells(name, meta, data, status_badge(entry.get('status', 'waiting')))
-	canvas = br.render_canvas_box(name, meta, run_dir / 'report', case.get('baseUrl', 'http://localhost:5678'),
-	                              empty='The canvas appears when all builders finish.')
-	view, auto = br.verdict_view(verdict, run_dir / name, case.get('expected') or '')
-	return row, br.render_panel_body(data, view, canvas, *scale, case['prompt'], case.get('expected') or '', auto=auto,
-	                                 pending=True)
-
-
-def render_live_box(status, runs):
-	cards = []
-	for name, meta, _, entry in runs:
-		title = br.esc(br.run_title(name, meta))
-		screen = (status.get('screens') or {}).get(name)
-		if screen == 'cast':
-			inner = (f'<div data-cast="{name}"></div><button type="button" class="live-zoom overlay" '
-			         f'data-zoom="{name}" aria-label="Show {title} full screen"></button>')
-		elif screen == 'shot':
-			inner = (f'<button type="button" class="live-zoom" data-zoom="{name}" aria-label="Show {title} full screen">'
-			         f'<img data-shot="{name}" alt="{title} in the n8n UI" hidden></button>'
-			         f'<div class="live-empty" data-shot-empty="{name}">Waiting for the first screenshot</div>')
-		else:
-			inner = '<div class="live-empty">No live screen for this builder. The LLM steps below update live.</div>'
-		cards.append(
-			f'<div class="live-card"><div class="live-screen">{inner}</div>'
-			f'<div class="reel-meta"><span class="run-name">{title}</span>'
-			f'<span data-live-badge="{name}">{status_badge(entry.get("status", "waiting"))}</span></div></div>'
-		)
-	return (f'<div class="box"><div class="head"><h2>Live</h2><span class="mono-label" id="live-clock">'
-	        f'{br.esc(clock(status))}</span></div><div class="live-grid">{"".join(cards)}</div></div>')
-
-
-def render_page(run_dir):
-	case, status, verdicts, runs = gather(run_dir)
-	scale = br.token_scale([r[:3] for r in runs])
-	rows, panels = [], []
-	for i, (name, meta, data, entry) in enumerate(runs):
-		active = ' active' if i == 0 else ''
-		row, panel = render_run(run_dir, case, name, meta, data, entry, verdicts.get(name), scale)
-		rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">{row}</tr>')
-		panels.append(f'<section class="run{active}" id="{name}">{panel}</section>')
-	try:
-		br.ensure_player_cache()
-		player_js, player_css = (f'vendor/{n}' for n in br.PLAYER_FILES)
-	except OSError:
-		player_js, player_css = (br.PLAYER_CDN + n for n in br.PLAYER_FILES)
-	notes = ('This page updates while the builders run, and changes to the full report when the run is done. '
-	         'Costs, canvases, checks and verdicts come after the run. Times are seconds from the start of each builder.')
-	return br.fill_template(
-		br.esc(case.get('title') or 'Builder comparison'), ''.join(rows), br.esc(notes), ''.join(panels),
-		replay=render_live_box(status, runs), head=f'<link rel="stylesheet" href="{player_css}">',
-		scripts=f'<script src="{player_js}"></script><script>const SCREENS = [];</script><script>{LIVE_JS}</script>',
-		prompt=br.esc(case['prompt']), expected=br.esc(case.get('expected') or '-'))
-
-
-def render_state(run_dir):
-	case, status, verdicts, runs = gather(run_dir)
-	scale = br.token_scale([r[:3] for r in runs])
-	state = {'phase': status.get('phase', 'running'), 'clock': clock(status), 'runs': {}}
-	if state['phase'] == 'done' and (run_dir / 'report' / 'index.html').exists():
-		state['report'] = '/report/index.html'
-	for name, meta, data, entry in runs:
-		row, panel = render_run(run_dir, case, name, meta, data, entry, verdicts.get(name), scale)
-		shot = run_dir / name / 'live.png'
-		state['runs'][name] = {'row': row, 'panel': panel, 'badge': status_badge(entry.get('status', 'waiting')),
-		                       'shot': shot.stat().st_mtime if shot.exists() else None}
-	return state
-
-
-LIVE_JS = """
-window.addEventListener('DOMContentLoaded', ()=>{
-  const seen = {}, shots = {}, userFolds = {};
-  // A box that the user opened or closed stays that way when its panel updates. Other boxes take the new default.
-  document.addEventListener('click', e=>{
-    const summary = e.target.closest && e.target.closest('details.fold > summary');
-    if (!summary || e.target.closest('.run-picker')) return;
-    const section = summary.closest('section.run');
-    if (section) (userFolds[section.id] ??= {})[summary.querySelector('h2').textContent] = !summary.parentElement.open;
-  });
-  document.addEventListener('change', e=>{
-    if (!e.target.matches || !e.target.matches('.run-picker')) return;
-    const steps = document.getElementById(e.target.value)?.querySelector('.steps');
-    if (steps) (userFolds[e.target.value] ??= {})['LLM steps'] = steps.open;
-  });
-  document.querySelectorAll('[data-cast]').forEach(el=>{
-    AsciinemaPlayer.create({driver: 'eventsource', url: `/api/cast/${el.dataset.cast}`, bufferTime: 0}, el,
-      {fit: 'both', autoPlay: true, controls: false});
-  });
-  // A live screen opens larger in a full-screen viewer. The viewer follows the newest screenshot, and a
-  // terminal gets its own player, because the stream starts with the whole recording so far.
-  const zoomNames = [...document.querySelectorAll('[data-zoom]')].map(b=>b.dataset.zoom);
-  const viewer = document.createElement('div');
-  let zoomed = null, opener = null, castPlayer = null, castName = null;
-  viewer.className = 'theater live-theater';
-  viewer.hidden = true;
-  viewer.setAttribute('role', 'dialog');
-  viewer.setAttribute('aria-modal', 'true');
-  viewer.setAttribute('aria-label', 'Live screen');
-  viewer.innerHTML = '<div class="theater-bar"><h2>Live</h2><span data-zoom-badge></span>'
-    + '<button type="button" class="btn" data-zoom-close>Close</button></div><div class="tabs"></div>'
-    + '<div class="live-zoomed"><img alt=""><div class="live-empty">Waiting for the first screenshot</div>'
-    + '<div class="live-zoomed-cast" hidden></div></div>'
-    + '<p class="fineprint">The screen updates every few seconds. Press the left or right arrow key to change the '
-    + 'scenario. Press Esc to close.</p>';
-  for (const name of zoomNames) {
-    const tab = Object.assign(document.createElement('button'), {type: 'button', className: 'tab'});
-    tab.dataset.zoomTab = name;
-    tab.textContent = document.querySelector(`[data-zoom="${name}"]`).closest('.live-card').querySelector('.run-name').textContent;
-    tab.addEventListener('click', ()=>showZoom(name));
-    viewer.querySelector('.tabs').append(tab);
-  }
-  document.body.append(viewer);
-  function stopCast(){
-    castPlayer?.dispose();
-    castPlayer = castName = null;
-    viewer.querySelector('.live-zoomed-cast').replaceChildren();
-  }
-  function showZoom(name){
-    zoomed = name;
-    viewer.querySelectorAll('[data-zoom-tab]').forEach(t=>t.classList.toggle('active', t.dataset.zoomTab === name));
-    const shot = document.querySelector(`img[data-shot="${name}"]`);
-    const isCast = !!document.querySelector(`[data-cast="${name}"]`);
-    const img = viewer.querySelector('.live-zoomed img'), castBox = viewer.querySelector('.live-zoomed-cast');
-    if (castName !== name) stopCast();
-    castBox.hidden = !isCast;
-    if (isCast && !castPlayer) {
-      castName = name;
-      castPlayer = AsciinemaPlayer.create({driver: 'eventsource', url: `/api/cast/${name}`, bufferTime: 0}, castBox,
-        {fit: 'both', autoPlay: true, controls: false});
-    }
-    const shotHidden = !shot || shot.hidden;
-    img.hidden = isCast || shotHidden;
-    if (shot && !shot.hidden && img.getAttribute('src') !== shot.getAttribute('src')) { img.src = shot.src; img.alt = shot.alt; }
-    viewer.querySelector('.live-zoomed .live-empty').hidden = isCast || !shotHidden;
-    viewer.querySelector('[data-zoom-badge]').innerHTML = document.querySelector(`[data-live-badge="${name}"]`)?.innerHTML || '';
-  }
-  function closeZoom(){
-    stopCast();
-    viewer.hidden = true;
-    zoomed = null;
-    document.body.classList.remove('theater-open');
-    opener?.focus();
-  }
-  document.querySelectorAll('[data-zoom]').forEach(b=>b.addEventListener('click', ()=>{
-    opener = b;
-    viewer.hidden = false;
-    document.body.classList.add('theater-open');
-    showZoom(b.dataset.zoom);
-    viewer.querySelector('[data-zoom-close]').focus();
-  }));
-  viewer.querySelector('[data-zoom-close]').addEventListener('click', closeZoom);
-  document.addEventListener('keydown', e=>{
-    if (zoomed === null) return;
-    if (e.key === 'Escape') closeZoom();
-    else if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') {
-      const step = e.key === 'ArrowRight' ? 1 : -1;
-      showZoom(zoomNames[(zoomNames.indexOf(zoomed) + step + zoomNames.length) % zoomNames.length]);
-    }
-  });
-  function selectQuietly(root, step){
-    root.querySelectorAll('.step').forEach(b=>b.classList.toggle('active', b.dataset.step === step));
-    root.querySelectorAll('.step-detail').forEach(d=>{ d.hidden = d.dataset.step !== step; });
-  }
-  // Keeps the selected step, the scroll positions and the open sections. When the last step is
-  // selected, the newest step is selected after the update.
-  function swapPanel(section, html, first){
-    const old = section.querySelector('.steps');
-    const items = old ? [...old.querySelectorAll('.step')] : [];
-    const active = items.find(b=>b.classList.contains('active'));
-    const follow = first || !active || active === items[items.length - 1];
-    // A run with no trace yet has a .steps section without the list and the pane.
-    const listTop = old?.querySelector('.step-list')?.scrollTop ?? 0;
-    const paneTop = old?.querySelector('.step-pane')?.scrollTop ?? 0;
-    const openState = {};
-    section.querySelectorAll('.step-detail').forEach(d=>d.querySelectorAll('details').forEach((x, i)=>{ openState[`${d.dataset.step}:${i}`] = x.open; }));
-    section.innerHTML = html;
-    section.querySelectorAll('details.fold').forEach(d=>{
-      const open = (userFolds[section.id] || {})[d.querySelector('summary h2')?.textContent];
-      if (open !== undefined) d.open = open;
-    });
-    addRunPickers();
-    section.querySelectorAll('.step-detail').forEach(d=>d.querySelectorAll('details').forEach((x, i)=>{
-      const key = `${d.dataset.step}:${i}`;
-      if (key in openState) x.open = openState[key];
-    }));
-    const root = section.querySelector('.steps');
-    const next = root ? [...root.querySelectorAll('.step')] : [];
-    if (!next.length) return;
-    const list = root.querySelector('.step-list'), pane = root.querySelector('.step-pane');
-    if (!list || !pane) return;
-    if (follow) {
-      selectQuietly(root, next[next.length - 1].dataset.step);
-      list.scrollTop = list.scrollHeight;
-    } else {
-      selectQuietly(root, active.dataset.step);
-      list.scrollTop = listTop;
-      pane.scrollTop = paneTop;
-    }
-  }
-  async function poll(){
-    let state;
-    try {
-      state = await (await fetch('/api/state', {cache: 'no-store'})).json();
-    } catch {
-      $('live-clock').textContent = 'Lost the connection to the live server. Trying again…';
-      setTimeout(poll, 3000);
-      return;
-    }
-    if (state.report) { location.replace(state.report); return; }
-    $('live-clock').textContent = state.clock;
-    for (const [name, run] of Object.entries(state.runs)) {
-      const row = document.querySelector(`.row[data-run="${name}"]`);
-      if (row && seen[`row:${name}`] !== run.row) { row.innerHTML = run.row; seen[`row:${name}`] = run.row; }
-      const section = document.getElementById(name);
-      if (section && seen[`panel:${name}`] !== run.panel) {
-        swapPanel(section, run.panel, seen[`panel:${name}`] === undefined);
-        seen[`panel:${name}`] = run.panel;
-      }
-      const badge = document.querySelector(`[data-live-badge="${name}"]`);
-      if (badge) badge.innerHTML = run.badge;
-      const img = document.querySelector(`img[data-shot="${name}"]`);
-      if (img && run.shot && shots[name] !== run.shot) {
-        shots[name] = run.shot;
-        img.src = `/api/shot/${name}?v=${run.shot}`;
-        img.hidden = false;
-        document.querySelector(`[data-shot-empty="${name}"]`)?.remove();
-      }
-    }
-    if (zoomed !== null) showZoom(zoomed);
-    setTimeout(poll, 2000);
-  }
-  poll();
-});
-"""
-
-
-class BuilderApp:
-	"""What the server shows for compare.py: one live card, table row and panel for each builder.
-
-	Another skill can serve its own page with the same server: give LiveServer an object with
-	these methods."""
-
-	def __init__(self, run_dir):
-		self.run_dir = run_dir
-
-	def entry(self, name):
-		"""The status.json entry of a live card, or None for an unknown name."""
-		return read_status(self.run_dir).get('builders', {}).get(name)
-
-	def finished(self, name):
-		state = (self.entry(name) or {}).get('status')
-		return state in ('done', 'failed', 'skipped') or (self.run_dir / name / 'run.json').exists()
-
-	def done(self):
-		return read_status(self.run_dir).get('phase') == 'done' and (self.run_dir / 'report' / 'index.html').exists()
-
-	def page(self):
-		return render_page(self.run_dir)
-
-	def state(self):
-		return render_state(self.run_dir)
-
-	def shot(self, name):
-		return self.run_dir / name / 'live.png'
-
-	def casts(self, name):
-		"""The live recording, then the place where the runner moves it when the builder stops."""
-		sub = self.run_dir / name
-		return sub / '.screen' / 'screen.cast', sub / 'screen.cast'
-
-	def file(self, path):
-		"""A file for a route that the server does not know, or None."""
-		return None
-
-
-class LiveServer(ThreadingHTTPServer):
-	daemon_threads = True
-
-	def __init__(self, address, app):
-		super().__init__(address, Handler)
-		port = self.server_address[1]
-		self.app, self.run_dir = app, app.run_dir
-		self.allowed_hosts = {f'127.0.0.1:{port}', f'localhost:{port}'}
-		self.last_request = time.time()
-
-
-class Handler(BaseHTTPRequestHandler):
-	server_version = 'cwb-live'
-
-	def log_message(self, *args):
-		pass
-
-	def do_GET(self):
-		srv = self.server
-		srv.last_request = time.time()
-		# Only same-origin requests. A page that resolves its own host name to 127.0.0.1 gets a 403.
-		if self.headers.get('Host', '') not in srv.allowed_hosts:
-			return self.send_error(403)
-		path = unquote(urlparse(self.path).path)
-		app = srv.app
-		try:
-			if path in ('/', '/report', '/report/'):
-				return self.redirect('/report/index.html' if app.done() else '/report/live')
-			if path == '/report/live':
-				return self.send_body(app.page().encode(), 'text/html; charset=utf-8')
-			if path == '/api/state':
-				return self.send_body(json.dumps(app.state()).encode(), 'application/json')
-			name = path.rsplit('/', 1)[-1]
-			if path.startswith(('/api/cast/', '/api/shot/')) and not app.entry(name):
-				return self.send_error(404)
-			if path.startswith('/api/cast/'):
-				return self.stream_cast(name)
-			if path.startswith('/api/shot/'):
-				return self.send_file(app.shot(name))
-			if path.startswith('/report/'):
-				return self.send_file(self.static_file(path.removeprefix('/report/')))
-			self.send_file(app.file(path))
-		except (BrokenPipeError, ConnectionResetError):
-			pass
-
-	def redirect(self, location):
-		self.send_response(302)
-		self.send_header('Location', location)
-		self.send_header('Content-Length', '0')
-		self.end_headers()
-
-	def send_body(self, body, content_type):
-		self.send_response(200)
-		self.send_header('Content-Type', content_type)
-		self.send_header('Content-Length', str(len(body)))
-		self.send_header('Cache-Control', 'no-store')
-		self.end_headers()
-		self.wfile.write(body)
-
-	def static_file(self, rel):
-		"""A file of the report. Fonts, logos and the player come from the asset cache until the report exists."""
-		base = (self.server.run_dir / 'report').resolve()
-		candidate = (base / rel).resolve()
-		if candidate.is_relative_to(base) and candidate.is_file():
-			return candidate
-		parts = Path(rel).parts
-		if parts[:1] in (('fonts',), ('brand',)):
-			cache = br.ASSET_CACHE.resolve()
-			candidate = (cache / rel).resolve()
-			return candidate if candidate.is_relative_to(cache) and candidate.is_file() else None
-		if len(parts) == 2 and parts[0] == 'vendor' and parts[1] in br.PLAYER_FILES:
-			return br.PLAYER_CACHE / parts[1]
-		return None
-
-	def send_file(self, file):
-		if not file or not file.is_file():
-			return self.send_error(404)
-		size = file.stat().st_size
-		start, end = 0, size - 1
-		match = re.fullmatch(r'bytes=(\d*)-(\d*)', self.headers.get('Range') or '')
-		if match and size and (match[1] or match[2]):
-			# Videos need byte ranges to seek.
-			if match[1]:
-				start, end = int(match[1]), int(match[2]) if match[2] else size - 1
-			else:
-				start = max(size - int(match[2]), 0)
-			end = min(end, size - 1)
-			if start > end:
-				self.send_response(416)
-				self.send_header('Content-Range', f'bytes */{size}')
-				self.end_headers()
-				return
-			self.send_response(206)
-			self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
-		else:
-			self.send_response(200)
-		self.send_header('Content-Type', mimetypes.guess_type(file.name)[0] or 'application/octet-stream')
-		self.send_header('Content-Length', str(end - start + 1))
-		self.send_header('Accept-Ranges', 'bytes')
-		self.send_header('Cache-Control', 'no-cache')
-		self.end_headers()
-		with file.open('rb') as f:
-			f.seek(start)
-			left = end - start + 1
-			while left > 0:
-				chunk = f.read(min(left, 1 << 16))
-				if not chunk:
-					break
-				self.wfile.write(chunk)
-				left -= len(chunk)
-
-	def sse(self, data, event=None):
-		self.wfile.write(((f'event: {event}\n' if event else '') + f'data: {data}\n\n').encode())
-		self.wfile.flush()
-
-	def stream_cast(self, name):
-		"""Send the screen recording so far as one reset frame, then each new frame as it arrives."""
-		srv = self.server
-		self.send_response(200)
-		self.send_header('Content-Type', 'text/event-stream')
-		self.send_header('Cache-Control', 'no-store')
-		self.end_headers()
-		path = None
-		while path is None:
-			path = next((p for p in srv.app.casts(name) if p.exists()), None)
-			if path is None:
-				if srv.app.finished(name):
-					return self.sse('{}', 'done')
-				self.wfile.write(b': waiting\n\n')
-				self.wfile.flush()
-				srv.last_request = time.time()
-				time.sleep(1)
-		# The runner moves the recording when the builder stops. The open file stays readable.
-		with path.open('rb') as f:
-			buffer = b''
-
-			def complete_lines():
-				nonlocal buffer
-				buffer += f.read()
-				*lines, buffer = buffer.split(b'\n')
-				return [line for line in lines if line.strip()]
-
-			lines = complete_lines()
-			while not lines:
-				if srv.app.finished(name):
-					return self.sse('{}', 'done')
-				time.sleep(0.2)
-				lines = complete_lines()
-			header, frames = json.loads(lines[0]), [json.loads(line) for line in lines[1:]]
-			output = [f for f in frames if f[1] == 'o']
-			self.sse(json.dumps({'cols': header.get('width'), 'rows': header.get('height'),
-			                     'time': output[-1][0] if output else 0, 'init': ''.join(f[2] for f in output)}))
-			last_data = last_ping = time.time()
-			while True:
-				lines = complete_lines()
-				for line in lines:
-					frame = json.loads(line)
-					if frame[1] == 'o':
-						self.sse(json.dumps([frame[0], 'o', frame[2]]))
-				now = time.time()
-				srv.last_request = now
-				if lines:
-					last_data = now
-					continue
-				if srv.app.finished(name) and now - last_data > 3:
-					return self.sse('{}', 'done')
-				if now - last_ping > 15:
-					self.wfile.write(b': ping\n\n')
-					self.wfile.flush()
-					last_ping = now
-				time.sleep(0.2)
-
-
-def pid_alive(pid):
-	try:
-		os.kill(pid, 0)
-	except ProcessLookupError:
-		return False
-	except PermissionError:
-		return True
-	return True
-
-
-def stop_when_idle(server, parent_pid):
-	while True:
-		time.sleep(30)
-		if time.time() - server.last_request < IDLE_STOP_S:
-			continue
-		done = read_status(server.run_dir).get('phase') == 'done'
-		if done or (parent_pid and not pid_alive(parent_pid)):
-			server.shutdown()
-			return
-
-
-def serve(app, parent_pid=None, port=0):
-	"""Serve app until it is idle. Writes the address to <run-dir>/live-url.txt."""
-	server = LiveServer(('127.0.0.1', port), app)
-	(app.run_dir / 'live-url.txt').write_text(f'http://127.0.0.1:{server.server_address[1]}\n')
-	threading.Thread(target=stop_when_idle, args=(server, parent_pid), daemon=True).start()
-	server.serve_forever()
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--run-dir', required=True)
-	ap.add_argument('--parent-pid', type=int)
-	ap.add_argument('--port', type=int, default=0)
-	args = ap.parse_args()
-	serve(BuilderApp(Path(args.run_dir).resolve()), args.parent_pid, args.port)
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/merge_verdicts.py

@@ -1,122 +0,0 @@
-#!/usr/bin/env python3
-"""Merge the blind judge panel into verdicts.json.
-
-  merge_verdicts.py <run-dir>
-
-Reads judge/blind-map.json and judge/verdicts-*.json. Each judge file has one entry for each
-builder letter:
-
-  {"A": {"summary": "...", "goalMet": true, "goalReason": "...", "verdict": "pass",
-         "checks": [{"id": "4", "ok": true, "text": "...", "evidence": [{"node": "...", "path": "...", "value": "..."}]}]}}
-
-For each criterion id, the merged check takes the value that all judges or a majority of the judges
-gave. A split is listed under disagreements. Without a majority, the check is not confirmed (ok null).
-The summary and the findings outside the criteria come from the first judge.
-"""
-
-import json
-import sys
-from collections import Counter
-from pathlib import Path
-
-sys.path.insert(0, str(Path(__file__).resolve().parent))
-from build_report import load_case, parse_criteria  # noqa: E402
-
-VERSION = 2
-VOTE_TEXT = {True: 'met', False: 'not met', None: 'not confirmed'}
-
-
-def majority(values):
-	"""The value that more than half of the votes have, or None when no value has a majority."""
-	value, count = Counter(values).most_common(1)[0]
-	return value if count * 2 > len(values) else None
-
-
-def first_check(judge, cid):
-	return next((c for c in judge.get('checks') or [] if str(c.get('id')) == cid), None)
-
-
-def merge_quality(judges):
-	"""The quality checks by majority, in the order of the first judge that answered each check."""
-	ids = list(dict.fromkeys(str(q.get('id')) for j in judges for q in j.get('quality') or [] if q.get('id')))
-	merged = []
-	for qid in ids:
-		found = [next((q for q in j.get('quality') or [] if str(q.get('id')) == qid), None) for j in judges]
-		votes = [q.get('ok') if q else None for q in found]
-		ok = majority(votes)
-		pick = next((q for q, v in zip(found, votes) if q and v == ok), None) or next(q for q in found if q)
-		item = {'id': qid, 'ok': ok, 'text': pick.get('text', ''), 'votes': votes}
-		if len(set(votes)) > 1:
-			item['note'] = f'{Counter(votes)[ok]} of {len(judges)} judges.' if ok is not None else 'Judges disagree.'
-		merged.append(item)
-	return merged
-
-
-def merge_builder(judges, criteria_ids):
-	n = len(judges)
-	lead = judges[0]
-	checks, disagreements = [], []
-	for cid in criteria_ids:
-		found = [first_check(j, cid) for j in judges]
-		votes = [c.get('ok') if c else None for c in found]
-		if all(c is None for c in found):
-			continue
-		ok = majority(votes)
-		pick = next((c for c, v in zip(found, votes) if c and v == ok), None) or next(c for c in found if c)
-		check = {k: v for k, v in pick.items() if k != 'ok'}
-		check.update(id=cid, ok=ok, votes=votes)
-		if len(set(votes)) > 1:
-			counts = ', '.join(f'{count} {VOTE_TEXT[v]}' for v, count in Counter(votes).items())
-			disagreements.append({'id': cid, 'votes': votes, 'texts': [c.get('text') if c else None for c in found]})
-			note = f'Judges disagree ({counts}).' if ok is None else f'{Counter(votes)[ok]} of {n} judges.'
-			check['note'] = note
-		checks.append(check)
-	checks += [dict(c) for c in lead.get('checks') or [] if str(c.get('id')) not in criteria_ids]
-
-	goal_votes = [j.get('goalMet') for j in judges]
-	goal = majority(goal_votes)
-	if goal is None:
-		goal = False if goal_votes.count(False) * 2 >= n else None
-	if len(set(goal_votes)) > 1:
-		reasons = [j.get('goalReason') for j in judges if j.get('goalMet') is False and j.get('goalReason')]
-		disagreements.append({'id': 'goal', 'votes': goal_votes, 'texts': reasons})
-	verdicts = [j.get('verdict') for j in judges if j.get('verdict')]
-	return {
-		'quality': merge_quality(judges),
-		'version': VERSION,
-		'judges': n,
-		'verdict': majority(verdicts) if verdicts else None,
-		'summary': lead.get('summary', ''),
-		'goalMet': goal,
-		'goalReason': next((j.get('goalReason') for j in judges if j.get('goalMet') is goal and j.get('goalReason')), ''),
-		'checks': checks,
-		'disagreements': disagreements,
-	}
-
-
-def main():
-	run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
-	judge_dir = run_dir / 'judge'
-	blind_map = json.loads((judge_dir / 'blind-map.json').read_text())
-	files = sorted(judge_dir.glob('verdicts-*.json'))
-	if not files:
-		sys.exit(f'No judge/verdicts-*.json in {run_dir}.')
-	panel = [json.loads(f.read_text()) for f in files]
-	groups = parse_criteria(load_case(run_dir).get('expected') or '') or []
-	criteria_ids = [num for _, items in groups for num, _ in items]
-
-	merged = {}
-	for letter, builder in blind_map.items():
-		judges = [p[letter] for p in panel if letter in p]
-		if not judges:
-			print(f'{builder}: no judge gave a verdict for builder {letter}')
-			continue
-		merged[builder] = merge_builder(judges, criteria_ids)
-		split = len(merged[builder]['disagreements'])
-		print(f'{builder} (builder {letter}): {len(judges)} judges, {split} disagreement{"" if split == 1 else "s"}')
-	(run_dir / 'verdicts.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False))
-	print(run_dir / 'verdicts.json')
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/n8n_api.py

@@ -1,123 +0,0 @@
-"""Config loader and small REST client for a local n8n instance. Standard library only."""
-
-import http.cookiejar
-import json
-import os
-import stat
-import urllib.error
-import urllib.request
-import uuid
-from functools import cache
-from pathlib import Path
-
-REPO_ROOT = Path(__file__).resolve().parents[4]
-DEFAULT_BASE_URL = 'http://localhost:5678'
-TOKEN_FILE = Path.home() / '.config' / 'n8n-builder-compare' / 'mcp-token'
-
-
-def read_env_file(path):
-	"""Return KEY=VALUE pairs from a dotenv file. Values are never printed."""
-	values = {}
-	p = Path(path)
-	if not p.exists():
-		return values
-	for line in p.read_text().splitlines():
-		line = line.strip()
-		if line.startswith('export '):
-			line = line[7:]
-		if not line or line.startswith('#') or '=' not in line:
-			continue
-		key, value = line.split('=', 1)
-		value = value.strip()
-		if value[:1] in ('"', "'"):
-			value = value[1:].split(value[0], 1)[0]
-		else:
-			value = value.split(' #', 1)[0].strip()
-		values[key.strip()] = value
-	return values
-
-
-def env_files():
-	"""Config files from lowest to highest priority."""
-	custom = os.environ.get('AI_SURFACES_ENV')
-	return [REPO_ROOT / '.env.local', Path(custom) if custom else REPO_ROOT / '.env.ai-surfaces']
-
-
-@cache
-def config():
-	"""Merged settings: .env.local, then .env.ai-surfaces, then the process environment."""
-	merged = {}
-	for path in env_files():
-		merged.update({k: v for k, v in read_env_file(path).items() if v != ''})
-	merged.update({k: v for k, v in os.environ.items() if v != ''})
-	return merged
-
-
-def setting(*names, default=None):
-	"""Return the first name that has a value."""
-	values = config()
-	for name in names:
-		if values.get(name):
-			return values[name]
-	return default
-
-
-class N8nClient:
-	def __init__(self):
-		self.base_url = setting('N8N_BASE_URL', default=DEFAULT_BASE_URL).rstrip('/')
-		self.email = setting('N8N_EVAL_EMAIL', default='nathan@n8n.io')
-		self.password = setting('N8N_EVAL_PASSWORD', default='PlaywrightTest123')
-		# The auth cookie is bound to the browser-id header that was sent at login.
-		self.browser_id = str(uuid.uuid4())
-		self.cookies = http.cookiejar.CookieJar()
-		self.opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(self.cookies))
-
-	def headers(self, extra=None):
-		h = {'content-type': 'application/json', 'browser-id': self.browser_id}
-		h.update(extra or {})
-		return h
-
-	def request(self, method, path, body=None, timeout=60):
-		data = json.dumps(body).encode() if body is not None else None
-		req = urllib.request.Request(self.base_url + path, method=method, data=data, headers=self.headers())
-		try:
-			with self.opener.open(req, timeout=timeout) as res:
-				raw = res.read()
-		except urllib.error.HTTPError as e:
-			detail = e.read()[:500].decode(errors='replace')
-			raise RuntimeError(f'{method} {path} -> {e.code}: {detail}') from None
-		parsed = json.loads(raw) if raw else None
-		if isinstance(parsed, dict) and set(parsed) == {'data'}:
-			return parsed['data']
-		return parsed
-
-	def open_stream(self, path):
-		req = urllib.request.Request(
-			self.base_url + path, headers=self.headers({'accept': 'text/event-stream'})
-		)
-		return self.opener.open(req, timeout=600)
-
-	def login(self):
-		self.request('POST', '/rest/login', {'emailOrLdapLoginId': self.email, 'password': self.password})
-		return self
-
-	def workflow(self, workflow_id):
-		return self.request('GET', f'/rest/workflows/{workflow_id}')
-
-
-def save_token(token):
-	TOKEN_FILE.parent.mkdir(parents=True, exist_ok=True)
-	TOKEN_FILE.write_text(token)
-	os.chmod(TOKEN_FILE, stat.S_IRUSR | stat.S_IWUSR)
-
-
-def load_token():
-	"""N8N_MCP_TOKEN from the config wins over the cached token."""
-	token = setting('N8N_MCP_TOKEN')
-	if token:
-		return token
-	return TOKEN_FILE.read_text().strip() if TOKEN_FILE.exists() else None
-
-
-def is_full_token(value):
-	return isinstance(value, str) and value.count('.') == 2 and '*' not in value

.agents/skills/testmax-n8n-ai/scripts/n8n_browser.mjs

@@ -1,50 +0,0 @@
-// Shared browser helpers: settings, Playwright launch, and n8n sign-in.
-// Uses the Playwright copy in the repo test package. Falls back to the local Chrome install.
-import { createRequire } from 'node:module';
-import { existsSync, readFileSync } from 'node:fs';
-import { dirname, join, resolve } from 'node:path';
-import { fileURLToPath } from 'node:url';
-
-const here = dirname(fileURLToPath(import.meta.url));
-const repo = resolve(here, '../../../..');
-const require = createRequire(join(repo, 'packages/quality/testing/playwright/package.json'));
-const { chromium, expect, selectors } = require('@playwright/test');
-export { expect, selectors };
-// pngjs, as Playwright bundles it. Image diffs use it, so the skills need no extra package.
-export const loadPng = () =>
-	createRequire(require.resolve('@playwright/test'))('playwright-core/lib/utilsBundle').PNG;
-
-// Same order as n8n_api.config(): .env.local, then .env.ai-surfaces, then the environment.
-const settings = {};
-for (const file of [
-	join(repo, '.env.local'),
-	process.env.AI_SURFACES_ENV ?? join(repo, '.env.ai-surfaces'),
-]) {
-	if (!existsSync(file)) continue;
-	for (const line of readFileSync(file, 'utf8').split('\n')) {
-		const match = line.match(/^\s*(?:export\s+)?([A-Z0-9_]+)\s*=\s*(.*)$/);
-		if (!match) continue;
-		const value = match[2].trim().replace(/^(['"])(.*)\1$/, '$2');
-		if (value) settings[match[1]] = value;
-	}
-}
-export const setting = (name, fallback) => process.env[name] || settings[name] || fallback;
-
-export const baseUrl = setting('N8N_BASE_URL', 'http://localhost:5678').replace(/\/$/, '');
-
-export async function launchBrowser() {
-	const launch = existsSync(chromium.executablePath()) ? {} : { channel: 'chrome' };
-	return await chromium.launch({ headless: true, ...launch });
-}
-
-export async function signIn(page) {
-	await page.goto(`${baseUrl}/signin`);
-	await page
-		.getByRole('textbox', { name: 'Email' })
-		.fill(setting('N8N_EVAL_EMAIL', 'nathan@n8n.io'));
-	await page
-		.getByRole('textbox', { name: 'Password' })
-		.fill(setting('N8N_EVAL_PASSWORD', 'PlaywrightTest123'));
-	await page.getByRole('button', { name: 'Sign in', exact: true }).click();
-	await page.waitForURL((url) => !url.pathname.startsWith('/signin'), { timeout: 30_000 });
-}

.agents/skills/testmax-n8n-ai/scripts/prepare.py

@@ -1,109 +0,0 @@
-#!/usr/bin/env python3
-"""Check the local setup and make the n8n MCP token available.
-
-Prints a JSON status object. It never prints a token or a key.
-
-  prepare.py            check everything, reuse the cached MCP token
-  prepare.py --rotate   create a new MCP key (this breaks other clients that use the old key)
-"""
-
-import argparse
-import json
-import shutil
-import subprocess
-import sys
-import urllib.request
-
-from n8n_api import TOKEN_FILE, N8nClient, env_files, is_full_token, load_token, save_token, setting
-
-
-def mcp_handshake(base_url, token):
-	body = json.dumps({
-		'jsonrpc': '2.0', 'id': 1, 'method': 'initialize',
-		'params': {'protocolVersion': '2025-03-26', 'capabilities': {},
-		           'clientInfo': {'name': 'testmax-n8n-ai', 'version': '1'}},
-	}).encode()
-	req = urllib.request.Request(base_url + '/mcp-server/http', data=body, method='POST', headers={
-		'content-type': 'application/json',
-		'accept': 'application/json, text/event-stream',
-		'authorization': f'Bearer {token}',
-	})
-	try:
-		with urllib.request.urlopen(req, timeout=20) as res:
-			return res.status == 200
-	except Exception:
-		return False
-
-
-def docker_ready():
-	if not shutil.which('docker'):
-		return False
-	return subprocess.run(['docker', 'info'], capture_output=True).returncode == 0
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--rotate', action='store_true')
-	args = ap.parse_args()
-
-	status = {
-		'configFiles': [str(p) for p in env_files() if p.exists()],
-		'tokenFile': str(TOKEN_FILE),
-		'builders': setting('COMPARE_BUILDERS', default='iai,claude,codex'),
-		'keys': {
-			'claude': bool(setting('CLAUDE_API_KEY', 'ANTHROPIC_API_KEY')),
-			'codex': bool(setting('CODEX_API_KEY', 'OPENAI_API_KEY')),
-			'langsmith': bool(setting('LANGSMITH_API_KEY')),
-		},
-	}
-	client = N8nClient()
-	status['baseUrl'] = client.base_url
-	try:
-		client.login()
-		status['login'] = True
-	except Exception as e:
-		status['login'] = False
-		status['error'] = str(e)
-		print(json.dumps(status, indent=2))
-		sys.exit(1)
-
-	try:
-		settings = client.request('GET', '/rest/instance-ai/settings')
-		status['instanceAiEnabled'] = bool(settings.get('enabled'))
-	except Exception:
-		status['instanceAiEnabled'] = False
-
-	client.request('PATCH', '/rest/mcp/settings', {'mcpAccessEnabled': True})
-	status['mcpAccessEnabled'] = True
-
-	token = None
-	if args.rotate:
-		if setting('N8N_MCP_TOKEN'):
-			# The clients would keep the configured token, which the rotation makes invalid.
-			sys.exit('Remove N8N_MCP_TOKEN from the config before you rotate the MCP key, then run prepare.py --rotate again.')
-		token = client.request('POST', '/rest/mcp/api-key/rotate')['apiKey']
-		save_token(token)
-	else:
-		current = client.request('GET', '/rest/mcp/api-key')['apiKey']
-		if is_full_token(current):
-			# A new key was created for this user, so the full value is visible once.
-			save_token(current)
-		token = load_token()
-	status['mcpTokenSource'] = 'N8N_MCP_TOKEN' if setting('N8N_MCP_TOKEN') else 'cache'
-
-	status['mcpTokenAvailable'] = bool(token)
-	status['mcpTokenWorks'] = bool(token) and mcp_handshake(client.base_url, token)
-	if not status['mcpTokenWorks']:
-		status['mcpTokenHelp'] = (
-			'The MCP token is missing or invalid. Set N8N_MCP_TOKEN in .env.ai-surfaces, or run '
-			'prepare.py --rotate after the user agrees. Rotation invalidates the old key in every other MCP client.'
-		)
-
-	status['docker'] = docker_ready()
-	status['claudeOnHost'] = bool(shutil.which('claude'))
-	status['codexOnHost'] = bool(shutil.which('codex'))
-	print(json.dumps(status, indent=2))
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/record_iai.mjs

@@ -1,82 +0,0 @@
-#!/usr/bin/env node
-// Record a video of an Instance AI thread in the n8n UI while the run continues.
-//
-//   node record_iai.mjs <out-dir> [--live]
-//
-// --live also saves the screen to <out-dir>/live.png every few seconds, for the live page.
-//
-// Protocol on stdin and stdout, one line each:
-//   prints "ready" after sign-in
-//   reads  "open <thread-url>" and opens the thread
-//   reads  "stop", or stdin closes: keeps the last screen for a moment, then saves <out-dir>/screen.webm
-//                                   and the last screen as <out-dir>/screen-poster.png
-// The sign-in happens in a separate context, so the video starts at the thread.
-import { copyFileSync, mkdtempSync, renameSync, rmSync } from 'node:fs';
-import { tmpdir } from 'node:os';
-import { join, resolve } from 'node:path';
-import { createInterface } from 'node:readline';
-
-import { launchBrowser, signIn } from './n8n_browser.mjs';
-
-const outDir = resolve(process.argv[2] ?? '.');
-const live = process.argv.includes('--live');
-const SIZE = { width: 1600, height: 1000 };
-const LIVE_SHOT_MS = 2500;
-
-const browser = await launchBrowser();
-const login = await browser.newContext();
-const loginPage = await login.newPage();
-await signIn(loginPage);
-const storageState = await login.storageState();
-await login.close();
-
-const videoDir = mkdtempSync(join(tmpdir(), 'cwb-video-'));
-const context = await browser.newContext({
-	viewport: SIZE,
-	storageState,
-	recordVideo: { dir: videoDir, size: SIZE },
-});
-const page = await context.newPage();
-console.log('ready');
-
-let liveTimer;
-let shooting = false;
-async function saveLiveShot() {
-	if (shooting) return;
-	shooting = true;
-	// The live page can read the file at any time, so it gets only complete images.
-	const partial = join(outDir, 'live.tmp.png');
-	try {
-		await page.screenshot({ path: partial });
-		renameSync(partial, join(outDir, 'live.png'));
-	} catch {
-		// The page can be in the middle of a navigation. The next interval tries again.
-	} finally {
-		shooting = false;
-	}
-}
-
-const lines = createInterface({ input: process.stdin });
-for await (const line of lines) {
-	if (line.startsWith('open ')) {
-		await page.goto(line.slice(5).trim(), { waitUntil: 'domcontentloaded' });
-		console.log('opened');
-		if (live && !liveTimer) liveTimer = setInterval(saveLiveShot, LIVE_SHOT_MS);
-	} else if (line.trim() === 'stop') {
-		break;
-	}
-}
-
-clearInterval(liveTimer);
-await page.waitForTimeout(3000);
-if (live) await saveLiveShot();
-// The video starts on a blank page, so the report shows the last screen until playback starts.
-await page.screenshot({ path: join(outDir, 'screen-poster.png') }).catch(() => {});
-const video = page.video();
-await context.close();
-if (video) {
-	copyFileSync(await video.path(), join(outDir, 'screen.webm'));
-	console.log('saved screen.webm');
-}
-await browser.close();
-rmSync(videoDir, { recursive: true, force: true });

.agents/skills/testmax-n8n-ai/scripts/request_recorder.py

@@ -1,192 +0,0 @@
-"""Record the request bodies that a CLI sends to its model API. Standard library only.
-
-The recorder is a local HTTP proxy. The CLI sends its model requests to the recorder, and the
-recorder sends them to the real API without changes. It writes the request bodies to a JSONL file.
-It never writes headers, so the API key stays out of the file.
-
-Each row has the request settings without the conversation input. The tools and the instructions
-are in a row only when they differ from the previous request of the same thread, because the CLI
-sends them every turn.
-The tools field also holds the tools of additional_tools input items.
-"""
-
-import gzip
-import http.client
-import json
-import threading
-import time
-import zlib
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from urllib.parse import urlparse
-
-# Headers that apply to one connection only. The recorder sets its own values.
-HOP_HEADERS = {'connection', 'keep-alive', 'proxy-connection', 'proxy-authorization', 'te', 'trailer',
-               'transfer-encoding', 'upgrade', 'host', 'content-length', 'accept-encoding'}
-CHANGING_KEYS = ('tools', 'instructions')
-
-
-def decode_body(data, encoding):
-	encoding = (encoding or '').lower()
-	if encoding == 'gzip':
-		return gzip.decompress(data)
-	if encoding == 'deflate':
-		return zlib.decompress(data)
-	if encoding == 'zstd':
-		from compression import zstd
-		return zstd.decompress(data)
-	return data
-
-
-class RequestRecorder:
-	"""Start with start(). Point the CLI at base_url(host). Call stop() when the CLI is done."""
-
-	def __init__(self, upstream, out_file, bind='127.0.0.1'):
-		self.upstream = urlparse(upstream)
-		self.out_file = out_file
-		self.lock = threading.Lock()
-		self.idle = threading.Condition(self.lock)
-		self.active = 0
-		self.last = {}
-		self.count = 0
-		self.started = time.time()
-		self.server = ThreadingHTTPServer((bind, 0), self.handler())
-		self.server.daemon_threads = True
-		self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
-
-	@property
-	def port(self):
-		return self.server.server_address[1]
-
-	def base_url(self, host):
-		return f'http://{host}:{self.port}{self.upstream.path.rstrip("/")}'
-
-	def start(self):
-		self.out_file.parent.mkdir(parents=True, exist_ok=True)
-		self.out_file.write_text('')
-		self.thread.start()
-		return self
-
-	def stop(self, timeout=30):
-		"""Stop the server. Wait until the open requests are written, so count is final."""
-		self.server.shutdown()
-		with self.idle:
-			self.idle.wait_for(lambda: self.active == 0, timeout=timeout)
-		self.server.server_close()
-
-	def begin(self):
-		with self.lock:
-			self.active += 1
-
-	def end(self):
-		with self.idle:
-			self.active -= 1
-			self.idle.notify_all()
-
-	def record(self, method, path, body, encoding, status, error_text):
-		row = {'t': round(time.time() - self.started, 3), 'method': method, 'path': path, 'status': status}
-		try:
-			payload = json.loads(decode_body(body, encoding)) if body else None
-		except (ValueError, OSError, ImportError):
-			payload, row['bodyBytes'] = None, len(body)
-		with self.lock:
-			if isinstance(payload, dict):
-				items = payload.get('input') if isinstance(payload.get('input'), list) else []
-				if items:
-					row['inputItems'] = len(items)
-				# Codex sends its tools as an additional_tools input item, not in the top-level tools field.
-				extra = [t for i in items if isinstance(i, dict) and i.get('type') == 'additional_tools'
-				         for t in i.get('tools') or []]
-				if extra:
-					payload = {**payload, 'tools': (payload.get('tools') or []) + extra}
-				# Side tasks, such as a title request, run on their own thread with other tools.
-				metadata = payload.get('client_metadata') if isinstance(payload.get('client_metadata'), dict) else {}
-				thread = metadata.get('thread_id') or payload.get('prompt_cache_key') or ''
-				for key, value in payload.items():
-					if key == 'input':
-						continue
-					if key in CHANGING_KEYS:
-						if self.last.get((thread, key)) == value:
-							continue
-						self.last[(thread, key)] = value
-					row[key] = value
-			if error_text:
-				row['error'] = error_text
-			self.count += 1
-			with self.out_file.open('a') as f:
-				f.write(json.dumps(row, ensure_ascii=False) + '\n')
-
-	def handler(self):
-		recorder = self
-
-		class Handler(BaseHTTPRequestHandler):
-			protocol_version = 'HTTP/1.1'
-
-			def log_message(self, *args):
-				pass
-
-			def read_body(self):
-				if self.headers.get('Content-Length'):
-					return self.rfile.read(int(self.headers['Content-Length']))
-				if 'chunked' not in (self.headers.get('Transfer-Encoding') or '').lower():
-					return b''
-				parts = []
-				while True:
-					size = int(self.rfile.readline().split(b';')[0].strip() or b'0', 16)
-					if not size:
-						self.rfile.readline()
-						return b''.join(parts)
-					parts.append(self.rfile.read(size))
-					self.rfile.readline()
-
-			def forward(self):
-				recorder.begin()
-				try:
-					self.forward_and_record()
-				finally:
-					recorder.end()
-
-			def forward_and_record(self):
-				body = self.read_body()
-				up = recorder.upstream
-				conn_class = http.client.HTTPSConnection if up.scheme == 'https' else http.client.HTTPConnection
-				conn = conn_class(up.hostname, up.port, timeout=900)
-				headers = {k: v for k, v in self.headers.items() if k.lower() not in HOP_HEADERS}
-				headers['Host'] = up.netloc
-				headers['Content-Length'] = str(len(body))
-				status, error_text, replied = 502, None, False
-				try:
-					conn.request(self.command, self.path, body=body, headers=headers)
-					resp = conn.getresponse()
-					status, replied = resp.status, True
-					self.send_response(resp.status, resp.reason)
-					for key, value in resp.getheaders():
-						if key.lower() not in HOP_HEADERS:
-							self.send_header(key, value)
-					self.send_header('Transfer-Encoding', 'chunked')
-					self.end_headers()
-					kept = b''
-					while chunk := resp.read1(65536):
-						if status >= 400 and len(kept) < 4000:
-							kept += chunk
-						self.wfile.write(b'%x\r\n%s\r\n' % (len(chunk), chunk))
-						self.wfile.flush()
-					self.wfile.write(b'0\r\n\r\n')
-					self.wfile.flush()
-					if kept:
-						error_text = kept[:4000].decode('utf-8', 'replace')
-				except (OSError, http.client.HTTPException) as e:
-					error_text = f'recorder: {type(e).__name__}'
-					if not replied:
-						try:
-							self.send_error(502, 'Recorder could not reach the API')
-						except OSError:
-							pass
-					self.close_connection = True
-				finally:
-					conn.close()
-					recorder.record(self.command, self.path, body, self.headers.get('Content-Encoding'),
-					                status, error_text)
-
-			do_GET = do_POST = do_PUT = do_DELETE = do_PATCH = forward
-
-		return Handler

.agents/skills/testmax-n8n-ai/scripts/run_cli_builder.py

@@ -1,613 +0,0 @@
-#!/usr/bin/env python3
-"""Run one prompt through Claude Code or Codex with only the n8n MCP server.
-
-  run_cli_builder.py --builder claude --out <run-dir>/claude --prompt-file <run-dir>/prompt.txt
-  run_cli_builder.py --builder codex  --out <run-dir>/codex  --prompt-file <run-dir>/prompt.txt
-
-Isolation modes:
-  container  (default) Docker container. The only mount is the output directory.
-             The builder cannot see host files, skills, memory or settings.
-  clean      Claude only. A clean process with an empty config directory, no built-in
-             tools and only the n8n MCP server. It needs an API key, not a subscription login.
-
-Screen modes:
-  interactive  (default in a container) The normal terminal screen of the CLI runs in tmux.
-               screen.cast records the exact screen, and a terminal window can watch it live.
-               The trace comes from the session file of the CLI.
-  headless     `claude -p` or `codex exec` with a JSON event stream. Clean isolation uses this mode.
-
-Secrets: the model key and the MCP token go through environment variables only.
-They are never written to a file or put on a command line.
-
-Codex sends its model requests through a local recorder (request_recorder.py), which writes the
-request bodies, but not the headers, to requests.jsonl. Set COMPARE_RECORD_REQUESTS=0 to turn it off.
-"""
-
-import argparse
-import hashlib
-import json
-import os
-import re
-import shutil
-import subprocess
-import sys
-import threading
-import time
-from datetime import datetime, timezone
-from pathlib import Path
-from urllib.parse import urlparse
-
-from live import enabled as live_enabled
-from live import open_log_window, open_terminal
-from n8n_api import DEFAULT_BASE_URL, load_token, setting
-from request_recorder import RequestRecorder
-
-SKILL_DIR = Path(__file__).resolve().parents[1]
-IMAGE = 'n8n-builder-sandbox'
-# The builder-specific name comes first, so the CLI can use a different key than
-# the ANTHROPIC_API_KEY or OPENAI_API_KEY that the instance reads from .env.local.
-KEY_SETTINGS = {'claude': ['CLAUDE_API_KEY', 'ANTHROPIC_API_KEY'], 'codex': ['CODEX_API_KEY', 'OPENAI_API_KEY']}
-# Codex features that add tools or downloads beyond the n8n MCP server. shell_snapshot is off
-# because it writes the environment, with the API key, to a file.
-CODEX_FEATURES_OFF = ['apps', 'browser_use', 'browser_use_external', 'computer_use', 'image_generation',
-                      'in_app_browser', 'plugins', 'remote_plugin', 'shell_snapshot']
-SCREEN = {'cols': 120, 'rows': 36}
-OPENAI_BASE_URL = 'https://api.openai.com/v1'
-# Interactive mode: the variable that the CLI reads the model key from.
-SANDBOX_KEY = {'claude': 'CLAUDE_SANDBOX_KEY', 'codex': 'CODEX_SANDBOX_KEY'}
-IDLE_LIMIT_S = 600
-
-
-def now_iso():
-	return datetime.now(timezone.utc).isoformat()
-
-
-def ensure_image(rebuild):
-	# The label holds a hash of the Dockerfile, so an edit to the Dockerfile rebuilds the image.
-	digest = hashlib.sha256((SKILL_DIR / 'sandbox' / 'Dockerfile').read_bytes()).hexdigest()[:16]
-	found = subprocess.run(['docker', 'image', 'inspect', '--format',
-	                        '{{index .Config.Labels "dockerfile-sha"}}', IMAGE], capture_output=True, text=True)
-	if found.returncode == 0 and found.stdout.strip() == digest and not rebuild:
-		return
-	print(f'building docker image {IMAGE}', flush=True)
-	subprocess.run(['docker', 'build', '--label', f'dockerfile-sha={digest}', '-t', IMAGE,
-	                str(SKILL_DIR / 'sandbox')], check=True)
-
-
-def mcp_url(base_url, isolation):
-	if isolation != 'container':
-		return f'{base_url}/mcp-server/http'
-	parsed = urlparse(base_url)
-	port = parsed.port or (443 if parsed.scheme == 'https' else 80)
-	host = 'host.docker.internal' if parsed.hostname in ('localhost', '127.0.0.1') else parsed.hostname
-	return f'{parsed.scheme}://{host}:{port}/mcp-server/http'
-
-
-def claude_tool_args(tools):
-	# 'default' gives the normal Claude Code tools, all allowed without a question, like Codex in the
-	# container. Only a container may use it, because Bash and Write can change the files they see.
-	if tools == 'default':
-		return ['--permission-mode', 'bypassPermissions']
-	# dontAsk allows the pre-approved n8n tools and denies the rest without a prompt.
-	return ['--tools', '', '--allowedTools', 'mcp__n8n', '--permission-mode', 'dontAsk']
-
-
-def claude_args(prompt, model, config_path, bare, tools='none'):
-	# --bare also replaces the system prompt with a minimal one. A container already isolates the CLI,
-	# so only the clean mode, which runs on the host, uses it.
-	args = [
-		'claude', '-p', prompt, *(['--bare'] if bare else []),
-		'--strict-mcp-config', '--mcp-config', config_path, *claude_tool_args(tools),
-		'--disable-slash-commands', '--output-format', 'stream-json', '--verbose',
-	]
-	if model:
-		args += ['--model', model]
-	return args
-
-
-def claude_interactive_args(model, tools='none'):
-	args = [
-		'claude', '--strict-mcp-config', '--mcp-config', '/out/mcp.json', *claude_tool_args(tools),
-		'--disable-slash-commands', '--settings', '/out/.screen/claude-settings.json',
-	]
-	if model:
-		args += ['--model', model]
-	return args
-
-
-def codex_args(prompt, model):
-	args = ['codex', 'exec', '--json', '--skip-git-repo-check', '--sandbox', 'read-only']
-	if model:
-		args += ['--model', model]
-	return args + [prompt]
-
-
-def codex_interactive_args(model):
-	# The background app server needs `ps`, which the slim image does not have.
-	args = ['codex', '--no-daemon']
-	if model:
-		args += ['--model', model]
-	return args
-
-
-def write_codex_config(home, url, key_env=None, api_base=OPENAI_BASE_URL):
-	"""Write config.toml. With key_env, a provider reads the key from that variable, so the
-	interactive screen does not ask for a sign-in and no key is stored in auth.json."""
-	codex_home = home / '.codex'
-	codex_home.mkdir(parents=True, exist_ok=True)
-	lines = ['approval_policy = "never"', 'sandbox_mode = "read-only"']
-	if key_env:
-		lines.append('model_provider = "sandbox-openai"')
-	lines += ['', '[features]'] + [f'{name} = false' for name in CODEX_FEATURES_OFF]
-	if key_env:
-		lines += ['', '[model_providers.sandbox-openai]', 'name = "OpenAI"',
-		          f'base_url = "{api_base}"', f'env_key = "{key_env}"', 'wire_api = "responses"']
-	lines += ['', '[projects."/out/work"]', 'trust_level = "trusted"',
-	          '', '[mcp_servers.n8n]', f'url = "{url}"', 'bearer_token_env_var = "N8N_MCP_TOKEN"', '']
-	(codex_home / 'config.toml').write_text('\n'.join(lines))
-
-
-def write_claude_interactive_config(out, home, tools):
-	# Skip the first-run screens: theme, onboarding and the folder trust question.
-	(home / '.claude' / '.claude.json').write_text(json.dumps({
-		'hasCompletedOnboarding': True, 'theme': 'dark',
-		'projects': {'/out/work': {'hasTrustDialogAccepted': True, 'hasCompletedProjectOnboarding': True}},
-	}))
-	# With ANTHROPIC_API_KEY the screen asks to confirm the key and shows part of it.
-	# apiKeyHelper reads the key from the environment without that question.
-	settings = {'apiKeyHelper': f'printenv {SANDBOX_KEY["claude"]}'}
-	if tools == 'default':
-		# The bypass-mode warning screen also has a ❯ marker, so the prompt would answer "No, exit".
-		settings['skipDangerousModePermissionPrompt'] = True
-	(out / '.screen' / 'claude-settings.json').write_text(json.dumps(settings))
-
-
-def live_line(builder, event):
-	"""One short terminal line for an interesting stream event, or None."""
-	if builder == 'claude':
-		if event.get('subtype') == 'permission_denied':
-			return f'denied {event.get("tool_name")}'
-		if event.get('type') == 'assistant':
-			for block in (event.get('message') or {}).get('content') or []:
-				if block.get('type') == 'tool_use':
-					return f'tool {block.get("name")}'
-				if block.get('type') == 'text' and block.get('text', '').strip():
-					return 'text ' + block['text'].strip().splitlines()[0][:120]
-		if event.get('type') == 'result':
-			return f'result {event.get("subtype")} turns={event.get("num_turns")}'
-	else:
-		item = event.get('item') or {}
-		if event.get('type') == 'item.started' and item.get('type') == 'mcp_tool_call':
-			return f'tool {item.get("server")}.{item.get("tool")}'
-		if event.get('type') == 'item.completed' and item.get('type') == 'agent_message':
-			return 'text ' + (item.get('text') or '').strip()[:120]
-		if event.get('type') in ('turn.completed', 'turn.failed', 'error'):
-			return event['type']
-	return None
-
-
-def session_lines(builder, row):
-	"""Short terminal lines for one session-file row, and the end state: None, 'done' or 'error'."""
-	lines = []
-	if builder == 'claude':
-		if row.get('type') != 'assistant':
-			return lines, None
-		msg = row.get('message') or {}
-		for block in msg.get('content') or []:
-			if block.get('type') == 'tool_use':
-				lines.append(f'tool {block.get("name")}')
-			elif block.get('type') == 'text' and block.get('text', '').strip():
-				lines.append('text ' + block['text'].strip().splitlines()[0][:120])
-		if row.get('isApiErrorMessage'):
-			return lines, 'error'
-		return lines, 'done' if msg.get('stop_reason') == 'end_turn' else None
-	payload = row.get('payload') or {}
-	if row.get('type') != 'event_msg':
-		return lines, None
-	kind, item = payload.get('type'), payload.get('item') or {}
-	if kind == 'item_completed' and item.get('type') == 'McpToolCall':
-		lines.append(f'tool {item.get("server")}.{item.get("tool")}')
-	elif kind == 'item_completed' and item.get('type') == 'AgentMessage':
-		text = ' '.join(c.get('text', '') for c in item.get('content') or [] if isinstance(c, dict))
-		lines.append('text ' + text.strip()[:120])
-	elif kind == 'error':
-		lines.append(f'error {str(payload.get("message"))[:200]}')
-	return lines, {'task_complete': 'done', 'turn_aborted': 'error'}.get(kind)
-
-
-BOLD, DIM, CYAN, GREEN, RED, RESET = '\033[1m', '\033[2m', '\033[36m', '\033[32m', '\033[31m', '\033[0m'
-LABELS = {'claude': 'Claude Code + n8n MCP', 'codex': 'Codex + n8n MCP'}
-
-
-def short(value, limit=400):
-	"""Readable one-line text for a tool input or result."""
-	if isinstance(value, dict) and isinstance(value.get('content'), list):
-		value = value['content']
-	if isinstance(value, list) and all(isinstance(v, dict) and 'text' in v for v in value):
-		value = ' '.join(v['text'] for v in value)
-	text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
-	text = ' '.join(text.split())
-	return text if len(text) <= limit else text[:limit] + ' …'
-
-
-def log_entries(builder, event):
-	"""Detailed, colored lines for live.log, the file that the headless live window follows."""
-	lines = []
-	if builder == 'claude':
-		# A permission_denied event has a string message, not an API message.
-		if event.get('subtype') == 'permission_denied':
-			lines.append(f'{RED}denied {event.get("tool_name")}{RESET}')
-		message = event.get('message')
-		content = message.get('content') if isinstance(message, dict) else None
-		for block in content if isinstance(content, list) else []:
-			kind = block.get('type')
-			if kind == 'thinking' and block.get('thinking', '').strip():
-				lines.append(f'{DIM}thinking: {short(block["thinking"], 600)}{RESET}')
-			elif kind == 'text' and block.get('text', '').strip():
-				lines.append(block['text'].strip())
-			elif kind == 'tool_use':
-				name = block.get('name', '').removeprefix('mcp__n8n__')
-				lines.append(f'{BOLD}{CYAN}→ {name}{RESET} {short(block.get("input"))}')
-			elif kind == 'tool_result':
-				color = RED if block.get('is_error') else DIM
-				lines.append(f'{color}← {short(block.get("content"))}{RESET}')
-		if event.get('type') == 'result':
-			cost = event.get('total_cost_usd')
-			lines.append(f'{GREEN}done: {event.get("subtype")}, {event.get("num_turns")} turns'
-			             + (f', ${cost:.2f} (Claude Code estimate)' if cost is not None else '') + RESET)
-	else:
-		item, etype = event.get('item') or {}, event.get('type')
-		kind = item.get('type')
-		if etype == 'item.started' and kind == 'mcp_tool_call':
-			lines.append(f'{BOLD}{CYAN}→ {item.get("tool")}{RESET} {short(item.get("arguments"))}')
-		elif etype == 'item.completed' and kind == 'mcp_tool_call':
-			if item.get('error'):
-				lines.append(f'{RED}← {short(item["error"])}{RESET}')
-			else:
-				lines.append(f'{DIM}← {short(item.get("result"))}{RESET}')
-		elif etype == 'item.completed' and kind == 'reasoning' and item.get('text'):
-			lines.append(f'{DIM}thinking: {short(item["text"], 600)}{RESET}')
-		elif etype == 'item.completed' and kind == 'agent_message' and item.get('text'):
-			lines.append(item['text'].strip())
-		elif etype == 'turn.completed':
-			lines.append(f'{GREEN}done: turn completed{RESET}')
-		elif etype in ('turn.failed', 'error'):
-			lines.append(f'{RED}{etype}: {short(event.get("error") or event.get("message"))}{RESET}')
-	return lines
-
-
-WORKFLOW_ID = re.compile(r'(?:workflowId|WORKFLOW_ID)\\*"?\s*[:=]\s*\\*"?([A-Za-z0-9]{10,})')
-
-
-def session_candidates(builder, home):
-	if builder == 'claude':
-		# Subagent transcripts are in deeper folders. The main session is at projects/<cwd>/<id>.jsonl.
-		root = home / '.claude' / 'projects'
-		return list(root.glob('*/*.jsonl')) if root.exists() else []
-	root = home / '.codex' / 'sessions'
-	return list(root.rglob('rollout-*.jsonl')) if root.exists() else []
-
-
-def find_session_file(builder, home, session_id=None):
-	matches = session_candidates(builder, home)
-	if session_id:
-		matches = [p for p in matches if session_id in p.name] or matches
-	matches.sort(key=lambda p: p.stat().st_mtime)
-	return matches[-1] if matches else None
-
-
-class SessionReader:
-	"""Reads the rows that the CLI appends to its session file."""
-
-	def __init__(self, builder, home):
-		self.builder, self.home, self.path, self.offset = builder, home, None, 0
-
-	def rows(self):
-		self.path = self.path or find_session_file(self.builder, self.home)
-		if not self.path:
-			return []
-		with self.path.open('rb') as f:
-			f.seek(self.offset)
-			data = f.read()
-		end = data.rfind(b'\n') + 1
-		self.offset += end
-		rows = []
-		for line in data[:end].splitlines():
-			try:
-				rows.append(json.loads(line))
-			except json.JSONDecodeError:
-				continue
-		return rows
-
-
-def docker_command(container, out, env_names):
-	cmd = [
-		'docker', 'run', '--rm', '-i', '--name', container,
-		'--user', f'{os.getuid()}:{os.getgid()}',
-		'--cap-drop', 'ALL', '--security-opt', 'no-new-privileges',
-		'--read-only', '--tmpfs', '/tmp',
-		'--add-host', 'host.docker.internal:host-gateway',
-		'-v', f'{out}:/out', '-w', '/out/work',
-		'-e', 'HOME=/out/home',
-		'-e', 'CLAUDE_CONFIG_DIR=/out/home/.claude',
-		'-e', 'CODEX_HOME=/out/home/.codex',
-		'-e', 'CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1',
-		'-e', 'COLORTERM=truecolor',
-	]
-	# `-e NAME` without a value copies the value from this process, so secrets stay off argv.
-	for name in env_names:
-		cmd += ['-e', name]
-	return cmd + [IMAGE]
-
-
-def check_no_secrets(out, secrets):
-	"""Warn if a secret value ended up in a file of the run directory. Never prints the value."""
-	needles = [s.encode() for s in secrets if s and len(s) >= 12]
-	for path in out.rglob('*'):
-		if not path.is_file() or path.stat().st_size > 50_000_000:
-			continue
-		data = path.read_bytes()
-		if any(n in data for n in needles):
-			print(f'WARNING: a secret value is in {path}. Delete the file before you share the run.', flush=True)
-
-
-def run_headless(args, out, prompt, cmd, run_env, cwd, container):
-	stream_path = out / 'stream.jsonl'
-	session_id, workflow_ids, result, errors_in_row = None, [], None, 0
-	live_log = (out / 'live.log').open('w', buffering=1)
-	live_log.write(f'{BOLD}{LABELS[args.builder]}{RESET}  model: {args.model or "CLI default"}, '
-	               f'isolation: {args.isolation}\n{DIM}prompt: {short(prompt, 600)}{RESET}\n\n')
-	open_log_window(LABELS[args.builder], out / 'live.log')
-	# The Codex stream has no timestamps, so stream.times keeps the arrival time of each line.
-	with stream_path.open('w') as stream, (out / 'stream.times').open('w') as times, \
-			(out / 'stderr.log').open('w') as err, live_log:
-		proc = subprocess.Popen(cmd, env=run_env, cwd=cwd, stdin=subprocess.DEVNULL,
-		                        stdout=subprocess.PIPE, stderr=err, text=True)
-		watchdog = threading.Timer(args.timeout_min * 60, proc.kill)
-		watchdog.start()
-		for line in proc.stdout:
-			try:
-				event = json.loads(line)
-			except json.JSONDecodeError:
-				continue
-			stream.write(line)
-			stream.flush()
-			times.write(f'{time.time():.3f}\n')
-			if event.get('type') == 'system' and event.get('subtype') == 'init':
-				session_id = event.get('session_id')
-			if event.get('type') == 'thread.started':
-				session_id = event.get('thread_id')
-			if event.get('type') == 'result':
-				result = event
-			for match in WORKFLOW_ID.findall(line):
-				if match not in workflow_ids:
-					workflow_ids.append(match)
-			text = live_line(args.builder, event)
-			if text:
-				print(f'[{args.builder}] {text}', flush=True)
-			for entry in log_entries(args.builder, event):
-				live_log.write(entry + '\n\n')
-			errors_in_row = errors_in_row + 1 if event.get('type') == 'error' else 0
-			if errors_in_row >= 10:
-				print(f'[{args.builder}] stopped after 10 errors in a row: {event.get("message", "")[:200]}', flush=True)
-				proc.kill()
-				break
-		proc.wait()
-		if not watchdog.is_alive():
-			print('timeout: killed the builder', flush=True)
-		watchdog.cancel()
-		if args.isolation == 'container':
-			subprocess.run(['docker', 'kill', container], capture_output=True)
-		live_log.write(f'{BOLD}=== finished, exit code {proc.returncode}. You can close this window. ==={RESET}\n')
-	return {'exitCode': proc.returncode, 'sessionId': session_id, 'workflowIds': workflow_ids,
-	        'reportedCostUsd': (result or {}).get('total_cost_usd')}
-
-
-def run_interactive(args, out, home, prompt, run_env, container):
-	screen_dir = out / '.screen'
-	argv = (claude_interactive_args(args.model, args.claude_tools) if args.builder == 'claude'
-	        else codex_interactive_args(args.model))
-	# Claude Code shows its input box before the MCP servers connect, so it waits a few seconds more.
-	ready = {'claude': ('❯', 5000), 'codex': ('›', 2000)}[args.builder]
-	(screen_dir / 'command.json').write_text(json.dumps(
-		{'argv': argv, **SCREEN, 'input': prompt, 'readyText': ready[0], 'settleMs': ready[1]}))
-	cmd = docker_command(container, out, [n for n in run_env if n in (SANDBOX_KEY[args.builder], 'N8N_MCP_TOKEN')])
-	cmd += ['node', '/out/.screen/screen.mjs', 'run', '/out/.screen']
-
-	reader, workflow_ids, state, timed_out = SessionReader(args.builder, home), [], None, False
-	with (out / 'stderr.log').open('w') as err:
-		proc = subprocess.Popen(cmd, env=run_env, stdin=subprocess.DEVNULL, stdout=err, stderr=err)
-		for _ in range(120):
-			if (screen_dir / 'screen.cast').exists() or proc.poll() is not None:
-				break
-			time.sleep(0.25)
-		watch = f'docker exec -it {container} tmux attach -r -t run'
-		if live_enabled() and not open_terminal(LABELS[args.builder], watch, SCREEN['cols'], SCREEN['rows']):
-			print(f'watch {LABELS[args.builder]}: {watch}', flush=True)
-		deadline, last_growth = time.time() + args.timeout_min * 60, time.time()
-		while proc.poll() is None and not state:
-			rows = reader.rows()
-			if rows:
-				last_growth = time.time()
-			for row in rows:
-				lines, row_state = session_lines(args.builder, row)
-				for line in lines:
-					print(f'[{args.builder}] {line}', flush=True)
-				for match in WORKFLOW_ID.findall(json.dumps(row)):
-					if match not in workflow_ids:
-						workflow_ids.append(match)
-				state = state or row_state
-			if time.time() > deadline:
-				print('timeout: stopped the builder', flush=True)
-				timed_out = True
-				break
-			if reader.path and time.time() - last_growth > IDLE_LIMIT_S:
-				print(f'no progress for {IDLE_LIMIT_S // 60} minutes: stopped the builder', flush=True)
-				timed_out = True
-				break
-			time.sleep(1)
-		(screen_dir / 'stop').write_text('')
-		try:
-			proc.wait(timeout=30)
-		except subprocess.TimeoutExpired:
-			subprocess.run(['docker', 'kill', container], capture_output=True)
-			proc.wait()
-	for row in reader.rows():
-		for match in WORKFLOW_ID.findall(json.dumps(row)):
-			if match not in workflow_ids:
-				workflow_ids.append(match)
-	if (screen_dir / 'screen.cast').exists():
-		shutil.move(screen_dir / 'screen.cast', out / 'screen.cast')
-	# After a finished turn the driver stops tmux, so the exit code of the CLI means nothing.
-	exited = screen_dir / 'exited'
-	if state == 'done' and not timed_out:
-		code = 0
-	else:
-		code = int(exited.read_text() or 1) if exited.exists() else 1
-		code = code or 1
-	session = reader.path
-	return {'exitCode': code, 'sessionId': session.stem.split('rollout-')[-1] if session else None,
-	        'workflowIds': workflow_ids, 'reportedCostUsd': None}
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--builder', choices=['claude', 'codex'], required=True)
-	ap.add_argument('--out', required=True)
-	ap.add_argument('--prompt-file', required=True)
-	ap.add_argument('--isolation', choices=['container', 'clean'],
-	                default=setting('COMPARE_ISOLATION', default='container'))
-	ap.add_argument('--mode', choices=['interactive', 'headless'], default=setting('COMPARE_CLI_MODE'),
-	                help='Default: interactive in a container, headless in clean isolation.')
-	ap.add_argument('--model', help='Model id. Default: CLAUDE_MODEL or CODEX_MODEL, else the CLI default.')
-	ap.add_argument('--key-env', help='Name of the setting with the model API key.')
-	ap.add_argument('--claude-tools', choices=['none', 'default'],
-	                default=setting('COMPARE_CLAUDE_TOOLS', default='none'),
-	                help='none: only the n8n tools. default: also the built-in Claude Code tools, all allowed.')
-	ap.add_argument('--rebuild-image', action='store_true')
-	ap.add_argument('--timeout-min', type=float, default=30)
-	args = ap.parse_args()
-
-	if args.claude_tools not in ('none', 'default'):
-		sys.exit(f'COMPARE_CLAUDE_TOOLS must be none or default, not {args.claude_tools!r}.')
-	if args.builder == 'codex' and args.isolation != 'container':
-		sys.exit('Codex runs only in a container. Its shell tool can read the host disk.')
-	if args.isolation != 'container':
-		args.mode = 'headless'
-		if args.builder == 'claude' and args.claude_tools != 'none':
-			print('[claude] clean isolation runs on the host, so COMPARE_CLAUDE_TOOLS=default is ignored.', flush=True)
-			args.claude_tools = 'none'
-	args.mode = args.mode or 'interactive'
-
-	prefix = args.builder.upper()
-	args.model = args.model or setting(f'{prefix}_MODEL')
-	# Gateway-style ids such as anthropic/claude-opus-5-5 are not valid for the CLIs.
-	provider = {'claude': 'anthropic/', 'codex': 'openai/'}[args.builder]
-	if args.model and args.model.startswith(provider):
-		args.model = args.model.removeprefix(provider)
-	key_names = [args.key_env] if args.key_env else KEY_SETTINGS[args.builder]
-	api_key = setting(*key_names)
-	if not api_key:
-		sys.exit(f'Set {" or ".join(key_names)} in .env.ai-surfaces, .env.local or the environment.')
-	token = load_token()
-	if not token:
-		sys.exit('No MCP token. Run prepare.py first.')
-
-	out = Path(args.out).resolve()
-	if out.exists():
-		shutil.rmtree(out)
-	work, home = out / 'work', out / 'home'
-	work.mkdir(parents=True)
-	(home / '.claude').mkdir(parents=True)
-	prompt = Path(args.prompt_file).read_text().strip()
-	base_url = setting('N8N_BASE_URL', default=DEFAULT_BASE_URL).rstrip('/')
-	url = mcp_url(base_url, args.isolation)
-	container = f'cwb-{args.builder}-{int(time.time())}'
-	interactive = args.mode == 'interactive'
-
-	# Codex does not save the tools it sends to the model, so a local proxy records the request bodies.
-	recorder = None
-	if args.builder == 'codex' and setting('COMPARE_RECORD_REQUESTS', default='1') != '0':
-		recorder = RequestRecorder(OPENAI_BASE_URL, out / 'requests.jsonl',
-		                           setting('COMPARE_RECORDER_BIND', default='127.0.0.1')).start()
-	api_base = recorder.base_url('host.docker.internal') if recorder else OPENAI_BASE_URL
-
-	child_env = {'N8N_MCP_TOKEN': token}
-	if args.builder == 'claude':
-		(out / 'mcp.json').write_text(json.dumps({'mcpServers': {'n8n': {
-			'type': 'http', 'url': url, 'headers': {'Authorization': 'Bearer ${N8N_MCP_TOKEN}'},
-		}}}))
-	if interactive:
-		(out / '.screen').mkdir()
-		shutil.copy(SKILL_DIR / 'sandbox' / 'screen.mjs', out / '.screen' / 'screen.mjs')
-		child_env[SANDBOX_KEY[args.builder]] = api_key
-		if args.builder == 'claude':
-			write_claude_interactive_config(out, home, args.claude_tools)
-		else:
-			write_codex_config(home, url, SANDBOX_KEY['codex'], api_base)
-	elif args.builder == 'claude':
-		child_env['ANTHROPIC_API_KEY'] = api_key
-	else:
-		child_env['OPENAI_API_KEY'] = api_key
-		child_env['CODEX_API_KEY'] = api_key
-		write_codex_config(home, url, 'CODEX_API_KEY' if recorder else None, api_base)
-
-	started_at = now_iso()
-	if args.isolation == 'container':
-		ensure_image(args.rebuild_image)
-		run_env = {**os.environ, **child_env}
-		try:
-			if interactive:
-				result = run_interactive(args, out, home, prompt, run_env, container)
-			else:
-				cmd = docker_command(container, out, child_env)
-				cmd += (claude_args(prompt, args.model, '/out/mcp.json', False, args.claude_tools)
-				        if args.builder == 'claude' else codex_args(prompt, args.model))
-				result = run_headless(args, out, prompt, cmd, run_env, None, container)
-		finally:
-			if recorder:
-				recorder.stop()
-		if recorder and not recorder.count:
-			print(f'[{args.builder}] the request recorder saw no requests, so the report has no Codex tools.', flush=True)
-	else:
-		if not shutil.which('claude'):
-			sys.exit('claude is not on PATH.')
-		run_env = {
-			'PATH': os.environ.get('PATH', '/usr/bin:/bin'),
-			'HOME': str(home),
-			'CLAUDE_CONFIG_DIR': str(home / '.claude'),
-			'DISABLE_AUTOUPDATER': '1',
-			'DISABLE_TELEMETRY': '1',
-			**child_env,
-		}
-		cmd = claude_args(prompt, args.model, str(out / 'mcp.json'), True)
-		result = run_headless(args, out, prompt, cmd, run_env, str(work), container)
-
-	session_file = find_session_file(args.builder, home, result['sessionId'])
-	meta = {
-		'kind': args.builder,
-		'isolation': args.isolation,
-		'mode': args.mode,
-		'model': args.model,
-		**({'claudeTools': args.claude_tools} if args.builder == 'claude' else {}),
-		'startedAt': started_at,
-		'finishedAt': now_iso(),
-		'exitCode': result['exitCode'],
-		'sessionId': result['sessionId'],
-		'sessionFile': str(session_file.relative_to(out)) if session_file else None,
-		'workflowIds': result['workflowIds'],
-		'workflowId': result['workflowIds'][-1] if result['workflowIds'] else None,
-		'reportedCostUsd': result['reportedCostUsd'],
-		'screen': 'screen.cast' if (out / 'screen.cast').exists() else None,
-		'requests': 'requests.jsonl' if recorder and recorder.count else None,
-	}
-	(out / 'run.json').write_text(json.dumps(meta, indent=2))
-	shutil.rmtree(home / '.codex' / 'shell_snapshots', ignore_errors=True)
-	check_no_secrets(out, [api_key, token])
-	print(json.dumps({k: meta[k] for k in ('exitCode', 'sessionId', 'workflowId', 'sessionFile', 'screen')}, indent=2))
-	if meta['exitCode']:
-		print(f'see {out / "stderr.log"}', flush=True)
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/run_instance_ai.py

@@ -1,294 +0,0 @@
-#!/usr/bin/env python3
-"""Run one prompt through Instance AI on the local n8n instance.
-
-The script sends the prompt to a new thread, answers every confirmation card with
-a fixed policy, waits for the thread to go idle, and exports the trace.
-
-  run_instance_ai.py --out <run-dir>/iai --prompt-file <run-dir>/prompt.txt
-
-Approval policy (the same choices for every run, so runs are comparable):
-  - approval, plan review, web search, continue: approve
-  - text input: approve with "Use sensible defaults and continue."
-  - questions: pick the first option, or skip a free-text question
-  - domain access: allow once
-  - workflow or credential setup cards: defer (the builder must not wait for real credentials)
-  - local gateway resources and MCP connect cards: deny
-"""
-
-import argparse
-import json
-import os
-import queue
-import re
-import shutil
-import subprocess
-import sys
-import threading
-import time
-import uuid
-from datetime import datetime, timezone
-from pathlib import Path
-
-from live import enabled as live_enabled
-from live import open_url, recording
-from n8n_api import N8nClient
-
-DEFAULT_TEXT_ANSWER = 'Use sensible defaults and continue.'
-HERE = Path(__file__).resolve().parent
-
-
-class Recorder:
-	"""Video of the thread in the n8n UI, made by record_iai.mjs."""
-
-	def __init__(self, out):
-		self.proc, self.lines = None, queue.Queue()
-		node = shutil.which('node')
-		if not recording() or not node:
-			return
-		live_flag = ['--live'] if live_enabled() else []
-		self.proc = subprocess.Popen([node, str(HERE / 'record_iai.mjs'), str(out.resolve()), *live_flag], cwd=HERE,
-		                             stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-		                             stderr=(out / 'record.log').open('w'), text=True)
-		threading.Thread(target=self._read, daemon=True).start()
-
-	def _read(self):
-		for line in self.proc.stdout:
-			self.lines.put(line.strip())
-
-	def wait_for(self, word, timeout):
-		deadline = time.time() + timeout
-		while self.proc and time.time() < deadline:
-			try:
-				if self.lines.get(timeout=min(max(deadline - time.time(), 0.1), 1)) == word:
-					return True
-			except queue.Empty:
-				if self.proc.poll() is not None:
-					return False
-		return False
-
-	def send(self, line):
-		if self.proc and self.proc.poll() is None:
-			self.proc.stdin.write(line + '\n')
-			self.proc.stdin.flush()
-
-	def stop(self):
-		if not self.proc:
-			return
-		self.send('stop')
-		self.proc.stdin.close()
-		try:
-			self.proc.wait(timeout=120)
-		except subprocess.TimeoutExpired:
-			self.proc.kill()
-
-
-def now_iso():
-	return datetime.now(timezone.utc).isoformat()
-
-
-def local_time_zone():
-	zone = os.environ.get('TZ')
-	if zone:
-		return zone
-	link = os.path.realpath('/etc/localtime')
-	return link.split('zoneinfo/', 1)[1] if 'zoneinfo/' in link else 'UTC'
-
-
-def answer_questions(questions):
-	answers = []
-	for q in questions or []:
-		options = q.get('options') or []
-		if q.get('type') in ('single', 'multi') and options:
-			answers.append({'questionId': q['id'], 'selectedOptions': [options[0]]})
-		elif q.get('required'):
-			answers.append({'questionId': q['id'], 'selectedOptions': [], 'customText': DEFAULT_TEXT_ANSWER})
-		else:
-			answers.append({'questionId': q['id'], 'selectedOptions': [], 'skipped': True})
-	return {'kind': 'questions', 'answers': answers}
-
-
-def decide(payload):
-	"""Return (confirm body, short label) for one confirmation-request payload."""
-	input_type = payload.get('inputType') or 'approval'
-	if payload.get('credentialDestination'):
-		origin = payload['credentialDestination'].get('origin')
-		return {'kind': 'credentialDestination', 'origin': origin, 'approved': False}, 'defer credential destination'
-	if payload.get('setupRequests') or payload.get('credentialRequests') or payload.get('credentialFlow'):
-		return {'kind': 'approval', 'approved': False}, 'defer setup'
-	if payload.get('domainAccess'):
-		return {'kind': 'domainAccessApprove', 'domainAccessAction': 'allow_once'}, 'allow domain once'
-	if payload.get('mcpConnectRequest'):
-		return {'kind': 'mcpConnect', 'approved': False}, 'deny MCP connect'
-	if input_type == 'resource-decision':
-		return {'kind': 'resourceDecision', 'resourceDecision': 'denyOnce'}, 'deny local resource'
-	if input_type == 'questions':
-		return answer_questions(payload.get('questions')), 'answer questions'
-	if input_type == 'text':
-		return {'kind': 'approval', 'approved': True, 'userInput': DEFAULT_TEXT_ANSWER}, 'answer text'
-	return {'kind': 'approval', 'approved': True}, f'approve {input_type}'
-
-
-def read_stream(client, thread_id, out_queue, stop):
-	try:
-		res = client.open_stream(f'/rest/instance-ai/events/{thread_id}')
-	except Exception as e:
-		out_queue.put(('error', str(e)))
-		return
-	out_queue.put(('open', None))
-	data_lines, event_name = [], None
-	while not stop.is_set():
-		raw = res.readline()
-		if not raw:
-			out_queue.put(('closed', None))
-			return
-		line = raw.decode(errors='replace').rstrip('\r\n')
-		if line == '':
-			if data_lines:
-				try:
-					out_queue.put(('event', (event_name, json.loads('\n'.join(data_lines)))))
-				except json.JSONDecodeError:
-					pass
-			data_lines, event_name = [], None
-		elif line.startswith('data:'):
-			data_lines.append(line[5:].lstrip())
-		elif line.startswith('event:'):
-			event_name = line[6:].strip()
-
-
-def collect_workflow_ids(value, found):
-	text = json.dumps(value)
-	for match in re.finditer(r'"workflowId"\s*:\s*"([A-Za-z0-9]{10,})"', text):
-		if match.group(1) not in found:
-			found.append(match.group(1))
-
-
-def main():
-	ap = argparse.ArgumentParser()
-	ap.add_argument('--out', required=True)
-	ap.add_argument('--prompt-file', required=True)
-	ap.add_argument('--timeout-min', type=float, default=30)
-	args = ap.parse_args()
-
-	out = Path(args.out)
-	out.mkdir(parents=True, exist_ok=True)
-	prompt = Path(args.prompt_file).read_text().strip()
-
-	# The recorder signs in while the event stream opens, so the video starts soon after the prompt.
-	recorder = Recorder(out)
-	client = N8nClient().login()
-	thread_id = str(uuid.uuid4())
-	events_file = (out / 'events.jsonl').open('w')
-	q, stop = queue.Queue(), threading.Event()
-	threading.Thread(target=read_stream, args=(client, thread_id, q, stop), daemon=True).start()
-
-	kind, detail = q.get(timeout=30)
-	if kind != 'open':
-		sys.exit(f'Event stream did not open: {detail}')
-
-	project = client.request('GET', '/rest/projects/personal')
-	client.request('POST', '/rest/instance-ai/threads',
-	               {'threadId': thread_id, 'projectId': project['id'], 'source': 'assistant_page'})
-
-	if recorder.proc and not recorder.wait_for('ready', 60):
-		print(f'recording failed to start, see {out / "record.log"}', flush=True)
-		recorder.proc.kill()
-		recorder.proc = None
-
-	started_at = now_iso()
-	chat = client.request('POST', f'/rest/instance-ai/chat/{thread_id}',
-	                      {'message': prompt, 'timeZone': local_time_zone()})
-	print(f'thread {thread_id} run {chat.get("runId") if isinstance(chat, dict) else chat}', flush=True)
-	thread_url = f'{client.base_url}/instance-ai/{thread_id}'
-	# The UI shows the user message only after the chat request, so the recorder opens the thread now.
-	recorder.send(f'open {thread_url}')
-	print(f'watch: {thread_url}', flush=True)
-	open_url(thread_url)
-
-	workflow_ids, decisions, langsmith_run_ids = [], [], []
-	saw_finish, idle_since = False, None
-	deadline = time.time() + args.timeout_min * 60
-	while time.time() < deadline:
-		try:
-			kind, detail = q.get(timeout=5)
-		except queue.Empty:
-			kind, detail = 'tick', None
-
-		if kind == 'event':
-			_, event = detail
-			events_file.write(json.dumps(event) + '\n')
-			events_file.flush()
-			etype, payload = event.get('type'), event.get('payload') or {}
-			if etype == 'run-start':
-				if payload.get('langsmithRunId'):
-					langsmith_run_ids.append(payload['langsmithRunId'])
-				idle_since = None
-			elif etype == 'run-finish':
-				saw_finish = True
-				print(f'run-finish {payload.get("status", "")}', flush=True)
-			elif etype == 'tool-call':
-				print(f'tool {payload.get("toolName")}', flush=True)
-			elif etype in ('tool-result', 'confirmation-request'):
-				collect_workflow_ids(payload, workflow_ids)
-			if etype == 'confirmation-request':
-				body, label = decide(payload)
-				try:
-					client.request('POST', f'/rest/instance-ai/confirm/{payload["requestId"]}', body)
-					ok = True
-				except RuntimeError as e:
-					ok = False
-					label += f' (failed: {e})'
-				decisions.append({'at': now_iso(), 'toolName': payload.get('toolName'),
-				                  'message': payload.get('message'), 'decision': label, 'ok': ok})
-				print(f'confirm: {label} [{payload.get("toolName")}]', flush=True)
-		elif kind in ('closed', 'error'):
-			print(f'event stream {kind}: {detail or ""}', flush=True)
-
-		if saw_finish:
-			status = client.request('GET', f'/rest/instance-ai/threads/{thread_id}/status')
-			running = [t for t in status.get('backgroundTasks', []) if t.get('status') == 'running']
-			if status.get('hasActiveRun') or status.get('isSuspended') or running:
-				idle_since = None
-			elif idle_since is None:
-				idle_since = time.time()
-			elif time.time() - idle_since > 10:
-				break
-	else:
-		print('timeout: the thread did not go idle', flush=True)
-
-	finished_at = now_iso()
-	stop.set()
-	events_file.close()
-	recorder.stop()
-
-	trace = {}
-	for key, path in (
-		('rich', f'/threads/{thread_id}/messages?limit=200'),
-		('raw', f'/threads/{thread_id}/messages?limit=200&raw=true'),
-		('thread', f'/threads/{thread_id}'),
-		('debug', f'/debug/threads/{thread_id}/runs'),
-	):
-		try:
-			trace[key] = client.request('GET', '/rest/instance-ai' + path)
-		except RuntimeError as e:
-			trace[key] = None
-			print(f'export {key} failed: {str(e)[:200]}', flush=True)
-	(out / 'trace.json').write_text(json.dumps(trace, indent=1))
-
-	meta = {
-		'kind': 'instance-ai',
-		'threadId': thread_id,
-		'startedAt': started_at,
-		'finishedAt': finished_at,
-		'workflowIds': workflow_ids,
-		'workflowId': workflow_ids[-1] if workflow_ids else None,
-		'langsmithRunIds': langsmith_run_ids,
-		'decisions': decisions,
-		'threadUrl': thread_url,
-		'screen': 'screen.webm' if (out / 'screen.webm').exists() else None,
-	}
-	(out / 'run.json').write_text(json.dumps(meta, indent=2))
-	print(json.dumps({k: meta[k] for k in ('threadId', 'workflowId', 'startedAt', 'finishedAt')}, indent=2))
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/run_scenarios.py

@@ -1,172 +0,0 @@
-#!/usr/bin/env python3
-"""Run each workflow of a run directory once for each scenario, with pinned data.
-
-  run_scenarios.py <run-dir>
-
-Reads <run-dir>/scenarios.json (optional, see SKILL.md) and <builder>/workflow.json.
-sdk.mjs makes the pin data: triggers, service nodes and AI root nodes are pinned, and logic
-nodes (Set, IF, Code, Aggregate, ...) run for real. The n8n MCP tool test_workflow runs the
-workflow, and get_workflow_execution reads the result of each node.
-Writes <builder>/exec.json. MCP runs a workflow only when its MCP access is on, so the script
-turns it on through the REST API when necessary, records that in exec.json, and turns it off
-again after the scenarios. With several triggers, each scenario starts from the trigger that
-sdk.mjs picks (scenarios.json "triggerNode", or the trigger that reaches the "reach" types).
-"""
-
-import json
-import shutil
-import subprocess
-import sys
-import urllib.request
-from pathlib import Path
-
-from n8n_api import N8nClient, load_token
-
-HERE = Path(__file__).resolve().parent
-TIMEOUT_SECONDS = 90
-ITEMS_PER_NODE = 3
-
-
-class Mcp:
-	def __init__(self, base_url, token):
-		self.url, self.token = base_url + '/mcp-server/http', token
-
-	def call(self, tool, arguments):
-		body = json.dumps({'jsonrpc': '2.0', 'id': 1, 'method': 'tools/call',
-		                   'params': {'name': tool, 'arguments': arguments}}).encode()
-		req = urllib.request.Request(self.url, data=body, method='POST', headers={
-			'content-type': 'application/json',
-			'accept': 'application/json, text/event-stream',
-			'authorization': f'Bearer {self.token}',
-		})
-		with urllib.request.urlopen(req, timeout=TIMEOUT_SECONDS + 60) as res:
-			raw = res.read().decode()
-		lines = [line[5:].strip() for line in raw.splitlines() if line.startswith('data:')]
-		reply = json.loads(lines[-1] if lines else raw)
-		if 'error' in reply:
-			raise RuntimeError(f'{tool}: {reply["error"].get("message")}')
-		result = reply['result']
-		return result.get('structuredContent') or json.loads(result['content'][0]['text'])
-
-
-def matches(node_type, pattern):
-	short = node_type.rsplit('.', 1)[-1]
-	return node_type == pattern or short.lower() == pattern.lower()
-
-
-def read_run(run_data, types):
-	"""Per node: did it run, its error, and its first output items (all outputs)."""
-	nodes = {}
-	for name, runs in (run_data or {}).items():
-		run = runs[-1] if runs else {}
-		items = []
-		for output in ((run.get('data') or {}).get('main') or []):
-			items.extend(item.get('json') for item in (output or [])[:ITEMS_PER_NODE])
-		error = run.get('error')
-		nodes[name] = {
-			'type': types.get(name),
-			'status': run.get('executionStatus'),
-			'error': (error.get('message') or error.get('description')) if isinstance(error, dict) else error,
-			'items': items[:ITEMS_PER_NODE * 2],
-		}
-	return nodes
-
-
-def expectations(expect, nodes):
-	"""Check reach and skip: node types that must run, and node types that must not run."""
-	results = []
-	ran = [n for n in nodes.values() if n['status'] == 'success' and not n['error']]
-	for pattern in expect.get('reach') or []:
-		results.append({'kind': 'reach', 'type': pattern, 'ok': any(matches(n['type'] or '', pattern) for n in ran)})
-	for pattern in expect.get('skip') or []:
-		results.append({'kind': 'skip', 'type': pattern,
-		                'ok': not any(matches(n['type'] or '', pattern) for n in nodes.values())})
-	return results
-
-
-def set_mcp_access(client, workflow_id, on):
-	"""Set MCP access for the workflow. Returns True when the script changed the setting."""
-	wf = client.workflow(workflow_id)
-	settings = wf.get('settings') or {}
-	if bool(settings.get('availableInMCP')) == on:
-		return False
-	client.request('PATCH', f'/rest/workflows/{workflow_id}',
-	               {'versionId': wf['versionId'], 'settings': {**settings, 'availableInMCP': on}})
-	return True
-
-
-def run_builder(sub, scenarios_file, client, mcp, node):
-	meta = json.loads((sub / 'run.json').read_text())
-	workflow_id = meta.get('workflowId')
-	wf_file = sub / 'workflow.json'
-	out = {'workflowId': workflow_id, 'scenarios': []}
-	if not workflow_id or not wf_file.exists():
-		out['skipped'] = 'No workflow.'
-		return out
-	workflow = json.loads(wf_file.read_text())
-	types = {n['name']: n['type'] for n in workflow.get('nodes', [])}
-	cmd = [node, str(HERE / 'sdk.mjs'), 'pin-plan', str(wf_file)] + ([str(scenarios_file)] if scenarios_file else [])
-	plan = json.loads(subprocess.run(cmd, cwd=HERE, capture_output=True, text=True, check=True).stdout)
-	out.update(pinned=plan['pinned'], executed=plan['executed'])
-	try:
-		out['mcpAccessTurnedOn'] = set_mcp_access(client, workflow_id, True)
-	except Exception as e:
-		out['skipped'] = f'Could not turn on MCP access: {e}'
-		return out
-	try:
-		run_plan(plan, workflow_id, types, mcp, sub, out)
-	finally:
-		if out['mcpAccessTurnedOn']:
-			try:
-				set_mcp_access(client, workflow_id, False)
-			except Exception as e:
-				out['mcpAccessRestoreError'] = str(e)
-				print(f'[scenarios] {sub.name}: could not turn MCP access off again: {e}', flush=True)
-	return out
-
-
-def run_plan(plan, workflow_id, types, mcp, sub, out):
-	for scenario in plan['scenarios']:
-		entry = {'name': scenario['name'], 'expect': scenario['expect'], 'provenance': scenario['provenance']}
-		args = {'workflowId': workflow_id, 'pinData': scenario['pinData'], 'timeout': TIMEOUT_SECONDS}
-		if scenario.get('triggerNode'):
-			args['triggerNodeName'] = entry['triggerNode'] = scenario['triggerNode']
-		try:
-			result = mcp.call('test_workflow', args)
-			entry.update(status=result.get('status'), error=result.get('error'), executionId=result.get('executionId'))
-			if result.get('executionId'):
-				execution = mcp.call('get_workflow_execution', {'workflowId': workflow_id,
-				                                                'executionId': result['executionId'], 'includeData': True})
-				run_data = ((execution.get('data') or {}).get('resultData') or {}).get('runData')
-				entry['nodes'] = read_run(run_data, types)
-				entry['checks'] = expectations(scenario['expect'], entry['nodes'])
-		except Exception as e:
-			entry.update(status='error', error=f'{type(e).__name__}: {e}')
-		print(f'[scenarios] {sub.name}: {scenario["name"]} -> {entry.get("status")}'
-		      + (f' ({entry["error"]})' if entry.get('error') else ''), flush=True)
-		out['scenarios'].append(entry)
-
-
-def main():
-	run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.').resolve()
-	node = shutil.which('node')
-	token = load_token()
-	if not node or not token:
-		sys.exit('Scenarios need node on PATH and a working MCP token.')
-	client = N8nClient().login()
-	mcp = Mcp(client.base_url, token)
-	scenarios_file = run_dir / 'scenarios.json'
-	scenarios_file = scenarios_file if scenarios_file.exists() else None
-	for meta_file in sorted(run_dir.glob('*/run.json')):
-		sub = meta_file.parent
-		try:
-			out = run_builder(sub, scenarios_file, client, mcp, node)
-		except Exception as e:
-			out = {'skipped': f'{type(e).__name__}: {e}', 'scenarios': []}
-		(sub / 'exec.json').write_text(json.dumps(out, indent=1))
-		if out.get('skipped'):
-			print(f'[scenarios] {sub.name}: skipped, {out["skipped"]}', flush=True)
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/run_wire.mjs

@@ -1,313 +0,0 @@
-#!/usr/bin/env node
-// Run each workflow of a run directory with the real node code, and record its service requests.
-//
-//   node run_wire.mjs <run-dir>
-//
-// wire.mjs runs the workflow in this process. Only the trigger is pinned, and the AI root
-// nodes too, except in the real-AI scenarios. Every other node runs, and the HTTP requests of
-// the service nodes get answers in the documented shape of each API. A node that runs must be on
-// the allow-list (runsInProcess), else the scenarios that run it are skipped. Three kinds of scenario:
-//   contract  the scenarios of scenarios.json, with the AI output pinned as in run_scenarios.py
-//   edge      the edge cases of the trigger (TRIGGER_KITS in sdk.mjs) and scenarios.json "edge"
-//   ai        scenarios.json "questions": the AI nodes call their real model with each question
-// Writes <builder>/wire.json.
-import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
-import { join, resolve } from 'node:path';
-
-import { LOGIC_TYPES, TRIGGER_KITS, graph, isAiNode, isTrigger, pinPlan, typeMatches } from './sdk.mjs';
-import { MODEL_CREDENTIALS, fillSetup, isWrite, loadEngine, modelKey, nodeCredentialTypes, runWorkflow, startNode } from './wire.mjs';
-
-const ITEMS_PER_NODE = 3;
-const BODY_CHARS = 1500;
-// This process holds the settings and has no sandbox. The eval hook answers only the requests that
-// go through the n8n request helpers. Database, SSH, FTP and SMTP clients, files and a shell bypass
-// it, so only the node types below run here.
-// These nodes run code from the builder. The pinned-data runs run them in the n8n task runner.
-const CODE_NODE_TYPES = new Set([
-	'n8n-nodes-base.code',
-	'n8n-nodes-base.function',
-	'n8n-nodes-base.functionItem',
-	'n8n-nodes-base.aiTransform',
-	'@n8n/n8n-nodes-langchain.code',
-	'@n8n/n8n-nodes-langchain.toolCode',
-]);
-// Telegram is here because its credential has no authenticate block.
-const IN_PROCESS_TYPES = new Set([
-	...[...LOGIC_TYPES].filter((type) => !CODE_NODE_TYPES.has(type)),
-	'n8n-nodes-base.httpRequest',
-	'n8n-nodes-base.wait',
-	'n8n-nodes-base.telegram',
-]);
-// Each model node has a key in MODEL_CREDENTIALS. The other nodes call only the model or the helpers.
-const IN_PROCESS_AI_TYPES = new Set(
-	[
-		'agent', 'chainLlm', 'chainSummarization', 'informationExtractor', 'textClassifier', 'sentimentAnalysis',
-		'openAi', 'anthropic', 'googleGemini',
-		'lmChatOpenAi', 'lmChatAnthropic', 'lmChatGoogleGemini', 'lmChatMistralCloud', 'lmChatGroq',
-		'outputParserStructured', 'outputParserAutofixing', 'outputParserItemList',
-		'memoryBufferWindow', 'toolCalculator', 'toolThink', 'toolHttpRequest',
-	].map((type) => `@n8n/n8n-nodes-langchain.${type}`),
-);
-const readJson = (file) => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
-
-/** Read a path such as `messages[0].from` from a value. */
-export function readPath(value, path) {
-	const keys = Array.isArray(path) ? path : String(path).match(/[^.[\]]+/g) ?? [];
-	return keys.reduce((v, k) => (v == null ? undefined : v[/^\d+$/.test(k) ? Number(k) : k]), value);
-}
-
-const requestText = (r) => JSON.stringify({ url: r.url, qs: r.qs ?? undefined, body: r.body ?? undefined });
-
-function nodeSummary(runData, types) {
-	const nodes = {};
-	for (const [name, runs] of Object.entries(runData ?? {})) {
-		const run = runs.at(-1) ?? {};
-		const outputs = (run.data?.main ?? []).map((o) => o ?? []);
-		const error = run.error;
-		nodes[name] = {
-			type: types[name],
-			status: run.executionStatus,
-			error: error ? error.message ?? error.description ?? String(error) : null,
-			runs: runs.length,
-			outputCounts: outputs.map((o) => o.length),
-			items: outputs.flat().slice(0, ITEMS_PER_NODE).map((i) => i.json),
-		};
-	}
-	return nodes;
-}
-
-const trim = (value) => {
-	const text = value === undefined ? undefined : JSON.stringify(value);
-	return text && text.length > BODY_CHARS ? `${text.slice(0, BODY_CHARS)}…` : value;
-};
-const trimRequest = (r) => ({ ...r, body: trim(r.body), response: trim(r.response) });
-
-/**
- * Resolve a marker of an expectation:
- *   @trigger.<path>          a value of the first trigger item, such as @trigger.messages[0].from
- *   @response.<type>.<path>  a value of the response to the last write request of a node type,
- *                            such as @response.notion.id (the id of the page that Notion created)
- * Other strings stay as they are.
- */
-function resolveMarker(value, triggerItem, requests) {
-	if (typeof value !== 'string') return value;
-	if (value.startsWith('@trigger.')) return readPath(triggerItem, value.slice('@trigger.'.length));
-	const response = value.match(/^@response\.([^.]+)\.(.+)$/);
-	if (response) {
-		const last = requests.filter((r) => isWrite(r) && typeMatches(r.nodeType, response[1])).at(-1);
-		return last ? readPath(last.response, response[2]) : undefined;
-	}
-	return value;
-}
-
-/** Check the expectations of a scenario against the nodes that ran and the requests that they sent. */
-export function evaluate(scenario, nodes, requests, trigger, kit) {
-	const checks = [];
-	const expect = scenario.expect ?? {};
-	const ran = Object.values(nodes).filter((n) => n.status === 'success' && !n.error);
-	for (const pattern of expect.reach ?? []) {
-		checks.push({ kind: 'reach', type: pattern, ok: ran.some((n) => typeMatches(n.type ?? '', pattern)) });
-	}
-	for (const pattern of expect.skip ?? []) {
-		checks.push({ kind: 'skip', type: pattern, ok: !Object.values(nodes).some((n) => typeMatches(n.type ?? '', pattern)) });
-	}
-	for (const want of expect.requests ?? []) {
-		// An array in contains means any one of its markers.
-		const contains = (want.contains ?? []).map((c) => (Array.isArray(c) ? c : [c]).map((m) => ({ marker: m, value: resolveMarker(m, trigger[0], requests) })));
-		const sent = requests.filter((r) => typeMatches(r.nodeType, want.type) && (!want.method || r.method === want.method.toUpperCase()));
-		const has = (r, any) => any.some((c) => c.value !== undefined && c.value !== null && requestText(r).includes(String(c.value)));
-		const matching = sent.filter((r) => contains.every((any) => has(r, any)));
-		const ok = want.count === undefined ? matching.length > 0 : matching.length === want.count;
-		const show = (c) => (c.marker === c.value ? `"${c.value}"` : `${c.marker} ("${c.value ?? 'not found'}")`);
-		const details = sent.length
-			? sent.map((r) => {
-					const missing = contains.filter((any) => !has(r, any)).map((any) => any.map(show).join(' or '));
-					if (missing.length) return `${r.node}: ${r.method} ${r.url} does not carry ${missing.join(', ')}`;
-					const body = r.body === undefined ? '' : JSON.stringify(r.body);
-					return `${r.node}: ${r.method} ${r.url}${body ? ` with \`${body.length > 300 ? `${body.slice(0, 300)}…` : body}\`` : ''}`;
-				})
-			: [`No ${want.method ?? ''} request from a ${want.type} node.`.replace('  ', ' ')];
-		checks.push({ kind: 'request', type: want.type, contains: contains.map((any) => any.map((c) => c.marker).join(' or ')), count: want.count, ok, details });
-	}
-	if (expect.noWrites) {
-		const writes = requests.filter(isWrite);
-		checks.push({ kind: 'noWrites', ok: writes.length === 0, details: writes.map((r) => `${r.node}: ${r.method} ${r.url}`) });
-	}
-	if (scenario.derive === 'each-item' && kit) {
-		const ids = trigger.map((item) => String(readPath(item, kit.identity)));
-		const carriers = requests.filter((r) => isWrite(r) && ids.some((id) => requestText(r).includes(id)));
-		const byNode = new Map();
-		for (const r of carriers) byNode.set(r.node, [...(byNode.get(r.node) ?? []), r]);
-		const details = [];
-		let ok = byNode.size > 0;
-		for (const [node, list] of byNode) {
-			const reached = ids.filter((id) => list.some((r) => requestText(r).includes(id)));
-			const missed = ids.filter((id) => !reached.includes(id));
-			if (missed.length) ok = false;
-			details.push(`${node}: ${list.length} request${list.length === 1 ? '' : 's'}, for ${reached.join(', ') || 'nobody'}${missed.length ? `, nothing for ${missed.join(', ')}` : ''}`);
-		}
-		if (!byNode.size) details.push(`No outgoing request names a sender (${ids.join(', ')}).`);
-		checks.push({ kind: 'eachItem', ids, ok, details });
-	}
-	return checks;
-}
-
-/** The scenarios of one workflow: contract, edge and AI, with their pin data and case rows. */
-function buildScenarios(workflow, cfg) {
-	const g = graph(workflow);
-	const triggerNode = g.triggers.find((t) => isTrigger(t.type));
-	const kit = triggerNode ? TRIGGER_KITS[triggerNode.type] : undefined;
-	const contract = cfg.scenarios?.length ? cfg.scenarios : null;
-	const edges = [...(kit?.edges ?? []), ...(cfg.edge ?? [])];
-	// The triggerNode of pinPlan stays: a name from scenarios.json can be missing in this workflow.
-	const merge = (s, entry, kind) => ({ ...s, ...entry, triggerNode: s.triggerNode, kind });
-	const list = [
-		...(contract ? pinPlan(workflow, cfg, contract).scenarios.map((s, i) => merge(s, contract[i], 'contract')) : pinPlan(workflow, cfg).scenarios.map((s) => merge(s, {}, 'contract'))),
-		...pinPlan(workflow, cfg, edges).scenarios.map((s, i) => merge(s, edges[i], 'edge')),
-		...pinPlan(workflow, cfg, cfg.questions ?? []).scenarios.map((s, i) => merge(s, cfg.questions[i], 'ai')),
-	];
-	const keep = (name) => {
-		const node = g.byName.get(name);
-		return node && (isTrigger(node.type) || isAiNode(g, node));
-	};
-	return {
-		g,
-		scenarios: list.map((s) => {
-			const pinData = Object.fromEntries(
-				Object.entries(s.pinData).filter(([name]) => keep(name) && !(s.kind === 'ai' && isAiNode(g, g.byName.get(name)))),
-			);
-			return { ...s, pinData };
-		}),
-	};
-}
-
-/** Model sub-nodes and vendor model nodes, and whether a key exists for each, for the real-AI scenarios. */
-function modelReadiness(g) {
-	const subnodes = g.edges.filter((e) => e.type === 'ai_languageModel').map((e) => g.byName.get(e.from));
-	const vendors = g.nodes.filter((n) => isAiNode(g, n) && !g.aiInputs(n.name, 'ai_languageModel').length);
-	return [...new Set([...subnodes, ...vendors])].map((node) => {
-		const types = nodeCredentialTypes(node).filter((t) => MODEL_CREDENTIALS[t]);
-		const type = types.find((t) => modelKey(t)) ?? types[0];
-		const p = node.parameters ?? {};
-		return { node: node.name, credentialType: type ?? null, model: p.modelName ?? p.model?.value ?? p.model ?? p.modelId?.value ?? null, ready: Boolean(type && modelKey(type)) };
-	});
-}
-
-const isHttpCredential = (name, { credentialType, parentTypes }) =>
-	credentialType(name).authenticate !== undefined || [name, ...parentTypes(name)].some((p) => p === 'oAuth1Api' || p === 'oAuth2Api');
-
-/**
- * True for a node on the allow-list: the types above, declarative nodes, and nodes whose credentials
- * all authenticate HTTP requests. A tool variant (slackTool) counts as its base node.
- */
-export function runsInProcess(node) {
-	const base = node.type.startsWith('n8n-nodes-base.') && node.type.endsWith('Tool') ? node.type.slice(0, -'Tool'.length) : node.type;
-	if (IN_PROCESS_TYPES.has(base) || IN_PROCESS_AI_TYPES.has(base)) return true;
-	if (!base.startsWith('n8n-nodes-base.')) return false;
-	const engine = loadEngine();
-	try {
-		const { description } = engine.nodeTypes.getByNameAndVersion(base, node.typeVersion);
-		if (description.requestDefaults !== undefined) return true;
-		const credentials = (description.credentials ?? []).map((c) => c.name);
-		return credentials.length > 0 && credentials.every((name) => isHttpCredential(name, engine));
-	} catch {
-		return false;
-	}
-}
-
-/**
- * The names and types of the nodes that run from the start trigger and are not on the allow-list.
- * Pinned nodes do not run. A disabled node passes its input on, so the walk goes through it.
- */
-function outsideAllowList(workflow, g, start, pinData) {
-	const seen = new Set([start]);
-	const running = new Set();
-	// A Set visits the entries that the loop adds.
-	for (const name of seen) {
-		for (const targets of workflow.connections?.[name]?.main ?? []) {
-			for (const { node } of targets ?? []) {
-				if (seen.has(node)) continue;
-				seen.add(node);
-				if (g.byName.has(node) && !Object.hasOwn(pinData, node)) running.add(node);
-			}
-		}
-	}
-	for (const name of running) {
-		for (const e of g.edges) if (e.to === name && e.type.startsWith('ai_')) running.add(e.from);
-	}
-	return [...running].map((name) => g.byName.get(name)).filter((n) => !runsInProcess(n)).map((n) => `${n.name} (${n.type})`);
-}
-
-async function runBuilder(dir, cfg) {
-	const workflow = readJson(join(dir, 'workflow.json'));
-	const { g, scenarios } = buildScenarios(workflow, cfg);
-	const canvas = readJson(join(dir, 'canvas-issues.json'));
-	const { workflow: filledWorkflow, filled } = fillSetup(workflow, canvas);
-	const types = Object.fromEntries((workflow.nodes ?? []).map((n) => [n.name, n.type]));
-	const models = modelReadiness(g);
-	const out = { filled, models, scenarios: [] };
-	for (const scenario of scenarios) {
-		const entry = { name: scenario.name, kind: scenario.kind, why: scenario.why, expect: scenario.expect ?? {}, derive: scenario.derive, pinned: Object.keys(scenario.pinData) };
-		if (scenario.kind === 'ai' && !models.length) {
-			out.scenarios.push({ ...entry, status: 'skipped', error: 'The workflow has no model sub-node.' });
-			continue;
-		}
-		const start = startNode(workflow, scenario.pinData, scenario.triggerNode);
-		const outside = start ? outsideAllowList(workflow, g, start.name, scenario.pinData) : [];
-		if (outside.length) {
-			out.scenarios.push({ ...entry, status: 'skipped', error: `Nodes outside the in-process allow-list: ${outside.join(', ')}.` });
-			continue;
-		}
-		if (scenario.kind === 'ai' && models.some((m) => !m.ready)) {
-			const missing = models.filter((m) => !m.ready).map((m) => `${m.node} (${m.credentialType ?? 'unknown credential'}: set ${MODEL_CREDENTIALS[m.credentialType]?.keys[0] ?? 'a supported model'})`);
-			out.scenarios.push({ ...entry, status: 'skipped', error: `No model key for ${missing.join(', ')}.` });
-			continue;
-		}
-		const rows = (node) => Object.entries(scenario.byType ?? {}).find(([pattern]) => typeMatches(node.type, pattern))?.[1];
-		const trigger = start ? scenario.pinData[start.name].map((i) => i.json) : [];
-		const kit = start ? TRIGGER_KITS[start.type] : undefined;
-		const result = await runWorkflow(filledWorkflow, { pinData: scenario.pinData, triggerNode: scenario.triggerNode, rows, realModels: scenario.kind === 'ai' });
-		const nodes = nodeSummary(result.runData, types);
-		const errors = Object.entries(nodes).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
-		Object.assign(entry, {
-			status: result.status,
-			error: result.error,
-			ms: result.ms,
-			nodes,
-			requests: result.requests.map(trimRequest),
-			blocked: result.blocked,
-			checks: evaluate(scenario, nodes, result.requests, trigger, kit),
-		});
-		if (scenario.kind === 'ai') {
-			entry.aiOutputs = Object.fromEntries(Object.entries(nodes).filter(([name]) => isAiNode(g, g.byName.get(name))).map(([name, n]) => [name, n.items]));
-		}
-		console.log(`[wire] ${dir.split('/').pop()}: ${scenario.kind} "${scenario.name}" -> ${result.status}${errors.length ? ` (${errors[0]})` : result.error ? ` (${result.error})` : ''}`);
-		out.scenarios.push(entry);
-	}
-	return out;
-}
-
-if (process.argv[1] && resolve(process.argv[1]) === new URL(import.meta.url).pathname) {
-	const runDir = resolve(process.argv[2] ?? '.');
-	const loaded = loadEngine();
-	const cfg = readJson(join(runDir, 'scenarios.json')) ?? {};
-	for (const entry of readdirSync(runDir, { withFileTypes: true })) {
-		const dir = join(runDir, entry.name);
-		if (!entry.isDirectory() || !existsSync(join(dir, 'workflow.json'))) continue;
-		let out;
-		if (loaded.error) out = { skipped: loaded.error, scenarios: [] };
-		else {
-			try {
-				out = await runBuilder(dir, cfg);
-			} catch (error) {
-				out = { skipped: `${error?.name ?? 'Error'}: ${error?.message ?? error}`, scenarios: [] };
-			}
-		}
-		// A failed real model request can echo its URL, with the key, in the error message.
-		const keys = Object.keys(MODEL_CREDENTIALS).map(modelKey).filter(Boolean);
-		writeFileSync(join(dir, 'wire.json'), keys.reduce((text, key) => text.replaceAll(key, '[redacted]'), JSON.stringify(out, null, 1)));
-		if (out.skipped) console.log(`[wire] ${entry.name}: skipped, ${out.skipped}`);
-	}
-	// Model SDKs can keep sockets open.
-	process.exit(0);
-}

.agents/skills/testmax-n8n-ai/scripts/sdk.mjs

@@ -1,600 +0,0 @@
-#!/usr/bin/env node
-// Shared workflow knowledge for derive_checks.mjs and run_scenarios.py: the workflow SDK of
-// the repository, graph helpers, expression references, JSON Schema samples and pin data plans.
-//
-//   node sdk.mjs pin-plan <workflow.json> [scenarios.json]    print the pin data of each scenario
-import { createRequire } from 'node:module';
-import { existsSync, readFileSync } from 'node:fs';
-import { dirname, join, resolve } from 'node:path';
-import { fileURLToPath } from 'node:url';
-
-const here = dirname(fileURLToPath(import.meta.url));
-export const repo = resolve(here, '../../../..');
-const require = createRequire(import.meta.url);
-const SDK_DIST = join(repo, 'packages/@n8n/workflow-sdk/dist/index.js');
-// The same directories that the n8n node catalog gives to setSchemaBaseDirs().
-const NODE_DEFINITION_DIRS = [
-	join(repo, 'packages/nodes-base/dist/node-definitions/'),
-	join(repo, 'packages/@n8n/nodes-langchain/dist/node-definitions/'),
-];
-
-let loaded;
-/** The workflow SDK with the node parameter schemas set, or null when the package is not built. */
-export function loadSdk() {
-	if (loaded !== undefined) return loaded;
-	if (!existsSync(SDK_DIST)) return (loaded = null);
-	const sdk = require(SDK_DIST);
-	const schemaDirs = NODE_DEFINITION_DIRS.filter((dir) => existsSync(dir));
-	// Without schema directories validateNodeConfig reports every node as valid.
-	sdk.setSchemaBaseDirs(schemaDirs);
-	return (loaded = { sdk, schemaDirs, missingDirs: NODE_DEFINITION_DIRS.filter((d) => !existsSync(d)) });
-}
-
-// --- Node kinds ----------------------------------------------------------------
-
-const BASE = 'n8n-nodes-base.';
-const LC = '@n8n/n8n-nodes-langchain.';
-
-/** Nodes that run for real in a scenario. They need no credentials and call no service. */
-export const LOGIC_TYPES = new Set(
-	[
-		'set', 'if', 'switch', 'filter', 'merge', 'aggregate', 'splitOut', 'code', 'function', 'functionItem',
-		'noOp', 'splitInBatches', 'itemLists', 'sort', 'limit', 'removeDuplicates', 'summarize', 'dateTime',
-		'renameKeys', 'compareDatasets', 'markdown', 'html', 'xml', 'crypto', 'executionData', 'stopAndError',
-		'respondToWebhook', 'convertToFile', 'extractFromFile', 'debugHelper',
-	].map((t) => BASE + t),
-);
-
-/** Nodes that end up with one item, so `.first()` on them reads all of their output. */
-export const SINGLE_ITEM_TYPES = new Set(['aggregate', 'summarize'].map((t) => BASE + t));
-
-const TRIGGER_TYPES = new Set(['webhook', 'cron', 'interval', 'emailReadImap', 'start'].map((t) => BASE + t));
-
-export const isSticky = (type) => type === `${BASE}stickyNote`;
-export const isTrigger = (type) => TRIGGER_TYPES.has(type) || type.toLowerCase().includes('trigger');
-export const isManualTrigger = (type) => type === `${BASE}manualTrigger`;
-export const shortType = (type) => type.slice(type.lastIndexOf('.') + 1);
-
-/** True when type matches a full type name, or its part after the last dot (case-insensitive). */
-export function typeMatches(type, pattern) {
-	return type === pattern || shortType(type).toLowerCase() === pattern.toLowerCase();
-}
-
-const AI_ROOT_PLACEHOLDER = {
-	[`${LC}agent`]: (v) => ({ output: v }),
-	[`${LC}openAiAssistant`]: (v) => ({ output: v }),
-	[`${LC}informationExtractor`]: (v) => ({ output: v }),
-	[`${LC}chainLlm`]: (v) => ({ text: v }),
-	[`${LC}chainRetrievalQa`]: (v) => ({ response: v }),
-	[`${LC}chainSummarization`]: (v) => ({ output: { output_text: v } }),
-	[`${LC}sentimentAnalysis`]: (v) => ({ sentimentAnalysis: { category: v } }),
-	[`${LC}googleGemini`]: (v) => ({ content: { parts: [{ text: v }], role: 'model' }, finishReason: 'STOP', index: 0 }),
-};
-
-/** Vendor nodes that call a model themselves, without a model sub-node. */
-const AI_VENDOR_TYPES = new Set(['googleGemini', 'openAi', 'anthropic'].map((t) => LC + t));
-
-// --- Graph ---------------------------------------------------------------------
-
-/** Nodes, edges and lookups of a workflow. Sticky notes and disabled nodes are left out. */
-export function graph(workflow) {
-	const nodes = (workflow.nodes ?? []).filter((n) => !isSticky(n.type) && !n.disabled);
-	const byName = new Map(nodes.map((n) => [n.name, n]));
-	const edges = [];
-	for (const [from, outputs] of Object.entries(workflow.connections ?? {})) {
-		for (const [type, branches] of Object.entries(outputs ?? {})) {
-			(branches ?? []).forEach((targets, output) =>
-				(targets ?? []).forEach((t) => {
-					if (byName.has(from) && byName.has(t.node))
-						edges.push({ from, to: t.node, type, output, input: t.index ?? 0 });
-				}),
-			);
-		}
-	}
-	const subnodes = new Set(edges.filter((e) => e.type.startsWith('ai_')).map((e) => e.from));
-	const mainParents = (name) => [...new Set(edges.filter((e) => e.type === 'main' && e.to === name).map((e) => e.from))];
-	const mainChildren = (name) => edges.filter((e) => e.type === 'main' && e.from === name);
-	const aiInputs = (name, type) => edges.filter((e) => e.to === name && e.type === type).map((e) => byName.get(e.from));
-	const triggers = nodes.filter((n) => isTrigger(n.type) && !subnodes.has(n.name));
-	return { nodes, byName, edges, subnodes, mainParents, mainChildren, aiInputs, triggers };
-}
-
-/** Names of the nodes that a trigger reaches. AI sub-nodes count when they serve a reached node. */
-export function reachable(g) {
-	const seen = new Set(g.triggers.map((t) => t.name));
-	const queue = [...seen];
-	while (queue.length) {
-		const name = queue.shift();
-		for (const e of g.edges) {
-			if (e.type === 'main' && e.from === name && !seen.has(e.to)) {
-				seen.add(e.to);
-				queue.push(e.to);
-			}
-		}
-	}
-	// A sub-node can serve another sub-node, for example a model inside an auto-fixing parser.
-	let grew = true;
-	while (grew) {
-		grew = false;
-		for (const e of g.edges) {
-			if (e.type.startsWith('ai_') && seen.has(e.to) && !seen.has(e.from)) {
-				seen.add(e.from);
-				grew = true;
-			}
-		}
-	}
-	return seen;
-}
-
-/** Names of the nodes downstream of name, through main connections. */
-export function downstream(g, name) {
-	const seen = new Set();
-	const queue = [name];
-	while (queue.length) {
-		for (const e of g.mainChildren(queue.shift())) {
-			if (!seen.has(e.to)) {
-				seen.add(e.to);
-				queue.push(e.to);
-			}
-		}
-	}
-	return seen;
-}
-
-// --- Parameters and expressions --------------------------------------------------
-
-/** Every string in a parameter tree, with its dotted path. */
-export function strings(value, path = '') {
-	if (typeof value === 'string') return [{ path, value }];
-	if (Array.isArray(value)) return value.flatMap((v, i) => strings(v, `${path}[${i}]`));
-	if (value && typeof value === 'object')
-		return Object.entries(value).flatMap(([k, v]) => strings(v, path ? `${path}.${k}` : k));
-	return [];
-}
-
-const CODE_PARAMS = new Set(['jsCode', 'pythonCode', 'functionCode', 'code']);
-
-/** The expression text of a parameter value, or null. Code fields count, because they use the same references. */
-export function expressionText(path, value) {
-	if (value.startsWith('=')) return value.slice(1);
-	const key = path.split('.').pop().replace(/\[\d+\]$/, '');
-	return CODE_PARAMS.has(key) ? value : null;
-}
-
-const IDENT = /^[A-Za-z_$][\w$]*/;
-
-/** Read a property path such as `.a?.b['c'][0]` from text at index i. Returns { path, end, call }. */
-function readPath(text, i) {
-	const path = [];
-	let dynamic = false;
-	for (;;) {
-		if (text.startsWith('?.', i)) i += 2;
-		else if (text[i] === '.') i += 1;
-		else if (text[i] !== '[') break;
-		if (text[i] === '[') {
-			const close = text.indexOf(']', i);
-			if (close < 0) break;
-			const inner = text.slice(i + 1, close).trim();
-			if (/^\d+$/.test(inner)) path.push(Number(inner));
-			else if (/^(['"`])(.*)\1$/.test(inner)) path.push(inner.slice(1, -1));
-			else {
-				dynamic = true;
-				i = close + 1;
-				break;
-			}
-			i = close + 1;
-			continue;
-		}
-		const match = text.slice(i).match(IDENT);
-		if (!match) break;
-		path.push(match[0]);
-		i += match[0].length;
-	}
-	const call = /^\s*\(/.test(text.slice(i));
-	// A trailing method call, such as `.toUpperCase()`, is not a field.
-	if (call && path.length) path.pop();
-	return { path, end: i, call, dynamic };
-}
-
-const ACCESSORS = [
-	{ re: /^\.item\b/, kind: 'item' },
-	{ re: /^\.first\(\s*\)/, kind: 'first' },
-	{ re: /^\.last\(\s*\)/, kind: 'last' },
-	{ re: /^\.all\(\s*\)\[\s*\d+\s*\]/, kind: 'all-index' },
-	{ re: /^\.all\(\s*\)/, kind: 'all' },
-	{ re: /^\.itemMatching\([^)]*\)/, kind: 'item' },
-	{ re: /^\.pairedItem\([^)]*\)/, kind: 'item' },
-];
-
-/**
- * References in expression text: { node, kind, accessor, path, raw }.
- * node is null for `$json` and `$input`, which read the main input of the node.
- * path is null when the reference does not read item JSON, for example `$('A').params`.
- */
-export function references(text) {
-	const refs = [];
-	const named = /\$\(\s*(['"`])((?:\\.|(?!\1).)+)\1\s*\)|\$node\[\s*(['"`])((?:\\.|(?!\3).)+)\3\s*\]|\$items\(\s*(['"`])((?:\\.|(?!\5).)+)\5/g;
-	for (const m of text.matchAll(named)) {
-		const node = (m[2] ?? m[4] ?? m[6]).replace(/\\(.)/g, '$1');
-		let i = m.index + m[0].length;
-		let accessor = m[4] !== undefined ? 'node' : m[6] !== undefined ? 'items' : null;
-		if (accessor === null) {
-			const rest = text.slice(i);
-			const hit = ACCESSORS.find((a) => a.re.test(rest));
-			if (hit) {
-				accessor = hit.kind;
-				i += rest.match(hit.re)[0].length;
-			}
-		}
-		let path = null;
-		if (text.startsWith('.json', i) && accessor !== 'all' && accessor !== 'items') {
-			const read = readPath(text, i + '.json'.length);
-			path = read.dynamic ? null : read.path;
-		}
-		refs.push({ node, kind: 'named', accessor, path, raw: text.slice(m.index, i + 40).split(/[}\n]/)[0] });
-	}
-	for (const m of text.matchAll(/\$json\b/g)) {
-		const read = readPath(text, m.index + '$json'.length);
-		refs.push({ node: null, kind: 'json', accessor: 'item', path: read.dynamic ? null : read.path,
-			raw: text.slice(m.index, read.end) });
-	}
-	for (const m of text.matchAll(/\$input\.(item|first\(\s*\)|last\(\s*\)|all\(\s*\)\[\s*\d+\s*\])\.json\b/g)) {
-		const read = readPath(text, m.index + m[0].length);
-		const accessor = m[1].startsWith('first') ? 'first' : m[1].startsWith('last') ? 'last' : m[1] === 'item' ? 'item' : 'all-index';
-		refs.push({ node: null, kind: 'input', accessor, path: read.dynamic ? null : read.path,
-			raw: text.slice(m.index, read.end) });
-	}
-	return refs;
-}
-
-/** Every reference in a workflow: { node (the node that holds it), param, ref, code }. */
-export function workflowReferences(g) {
-	const out = [];
-	for (const node of g.nodes) {
-		for (const { path, value } of strings(node.parameters ?? {})) {
-			const text = expressionText(path, value);
-			if (text === null) continue;
-			const code = !value.startsWith('=');
-			for (const ref of references(text)) out.push({ node: node.name, param: path, ref, code });
-		}
-	}
-	return out;
-}
-
-/**
- * Look up a path in a value. Returns 'found', 'missing' or 'unknown'.
- * 'unknown' means that a null or an opaque value hides the rest of the path. With sample, the
- * value comes from a schema, which shows the shape of one array item and not the count, so an
- * index past the end of an array is 'unknown'. In real output, that index is 'missing'.
- */
-export function lookup(value, path, { sample = false } = {}) {
-	let current = value;
-	for (const key of path) {
-		if (current === null || current === undefined) return current === null ? 'unknown' : 'missing';
-		if (Array.isArray(current)) {
-			if (typeof key === 'number') {
-				if (key >= current.length) return sample ? 'unknown' : 'missing';
-				current = current[key];
-			} else if (key === 'length') return 'found';
-			else return 'missing';
-			continue;
-		}
-		if (typeof current === 'string') return key === 'length' ? 'found' : 'missing';
-		if (typeof current !== 'object') return 'missing';
-		if (!(key in current)) return 'missing';
-		current = current[key];
-	}
-	return current === undefined ? 'missing' : 'found';
-}
-
-export const showPath = (path) =>
-	path.map((k, i) => (typeof k === 'number' ? `[${k}]` : /^[A-Za-z_$][\w$]*$/.test(k) ? (i ? `.${k}` : k) : `['${k}']`)).join('');
-
-// --- JSON Schema samples ---------------------------------------------------------
-
-/** A deterministic sample value for a JSON Schema. key names the field, for readable strings. */
-export function sample(schema, key = 'value', depth = 0) {
-	if (!schema || typeof schema !== 'object') return null;
-	if (schema.example !== undefined) return schema.example;
-	if (schema.const !== undefined) return schema.const;
-	if (Array.isArray(schema.enum) && schema.enum.length) return schema.enum[0];
-	if (schema.default !== undefined) return schema.default;
-	const variant = schema.anyOf ?? schema.oneOf;
-	if (Array.isArray(variant) && variant.length) return sample(variant.find((v) => v.type !== 'null') ?? variant[0], key, depth);
-	let type = Array.isArray(schema.type) ? schema.type.find((t) => t !== 'null') : schema.type;
-	if (!type) type = schema.properties ? 'object' : schema.items ? 'array' : 'string';
-	switch (type) {
-		case 'object': {
-			if (depth > 6) return {};
-			return Object.fromEntries(
-				Object.entries(schema.properties ?? {}).map(([k, v]) => [k, sample(v, k, depth + 1)]),
-			);
-		}
-		case 'array':
-			return schema.items && depth <= 6 ? [sample(schema.items, key, depth + 1)] : [];
-		case 'integer':
-		case 'number':
-			return 1;
-		case 'boolean':
-			return true;
-		case 'null':
-			return null;
-		default: {
-			const k = String(key);
-			if (schema.format === 'date-time' || /(^|_)(date|time|at)$|At$/.test(k)) return '2026-01-01T09:00:00.000Z';
-			if (schema.format === 'email' || /email/i.test(k)) return 'ada@example.com';
-			if (schema.format === 'uri' || /url|link|href/i.test(k)) return `https://example.com/${k}`;
-			if (/(^|_)id$|Id$/.test(k)) return `${k}-1`;
-			return `sample ${k}`;
-		}
-	}
-}
-
-/** Set every boolean in a value to flag. */
-function setBooleans(value, flag) {
-	if (Array.isArray(value)) return value.map((v) => setBooleans(v, flag));
-	if (value && typeof value === 'object')
-		return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, setBooleans(v, flag)]));
-	return typeof value === 'boolean' ? flag : value;
-}
-
-function deepMerge(base, extra) {
-	if (!base || typeof base !== 'object' || Array.isArray(base) || !extra || typeof extra !== 'object') return extra;
-	const out = { ...base };
-	for (const [k, v] of Object.entries(extra)) out[k] = deepMerge(base[k], v);
-	return out;
-}
-
-// --- Pin data ------------------------------------------------------------------
-
-/**
- * Trigger output for a scenario, in the shape that the trigger code emits.
- * WhatsApp: WhatsAppTrigger.node.ts emits `{ ...change.value, field }` for each change,
- * and change.value is the Cloud API webhook payload of a text message.
- */
-export const TRIGGER_FIXTURES = {
-	[`${BASE}whatsAppTrigger`]: [{
-		messaging_product: 'whatsapp',
-		metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
-		contacts: [{ profile: { name: 'Ada Lovelace' }, wa_id: '4915112345678' }],
-		messages: [{ from: '4915112345678', id: 'wamid.HBgLNDkxNTExMjM0NTY3OBUCABIYFjNFQjA', timestamp: '1767258000',
-			text: { body: 'What are your opening hours?' }, type: 'text' }],
-		field: 'messages',
-	}],
-	[`${BASE}webhook`]: [{ headers: { 'content-type': 'application/json' }, params: {}, query: {},
-		body: { message: 'Hello' }, webhookUrl: 'http://localhost:5678/webhook/test', executionMode: 'test' }],
-	[`${LC}chatTrigger`]: [{ sessionId: 'session-1', action: 'sendMessage', chatInput: 'Hello' }],
-	[`${BASE}manualTrigger`]: [{}],
-	[`${BASE}scheduleTrigger`]: [{ timestamp: '2026-01-01T09:00:00.000+00:00', 'Readable date': 'January 1st 2026, 9:00:00 am',
-		'Day of week': 'Thursday', Year: '2026', Month: 'January', 'Day of month': '01', Hour: '09', Minute: '00',
-		Second: '00', Timezone: 'UTC' }],
-	[`${BASE}telegramTrigger`]: [{ update_id: 1, message: { message_id: 1, date: 1767258000, text: 'Hello',
-		chat: { id: 1001, type: 'private', first_name: 'Ada' }, from: { id: 1001, is_bot: false, first_name: 'Ada' } } }],
-	[`${BASE}formTrigger`]: [{ submittedAt: '2026-01-01T09:00:00.000Z', formMode: 'test' }],
-};
-
-const whatsAppMessage = (from, name, body, id) => ({
-	messaging_product: 'whatsapp',
-	metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
-	contacts: [{ profile: { name }, wa_id: from }],
-	messages: [{ from, id, timestamp: '1767258000', text: { body }, type: 'text' }],
-	field: 'messages',
-});
-
-/**
- * Per trigger type: identity is the path of the value that names the sender in one item,
- * withMessage puts a scenario text into the fixture, and edges are the edge cases that every
- * workflow with this trigger must survive. derive 'each-item' means that each trigger item
- * must get its own outgoing request that names its sender.
- */
-export const TRIGGER_KITS = {
-	[`${BASE}whatsAppTrigger`]: {
-		identity: ['messages', 0, 'from'],
-		withMessage: (items, text) => items.map((i) => ({ ...i, messages: (i.messages ?? []).map((m) => ({ ...m, text: { body: text } })) })),
-		edges: [
-			{
-				name: 'Two messages arrive in one webhook call',
-				why: 'WhatsAppTrigger.node.ts emits one item for each change, and Meta can batch several changes in one call',
-				derive: 'each-item',
-				ai: { booleans: true },
-				trigger: [
-					whatsAppMessage('4915112345678', 'Ada Lovelace', 'What are your opening hours?', 'wamid.A1'),
-					whatsAppMessage('4917699887766', 'Alan Turing', 'Do you ship abroad?', 'wamid.B2'),
-				],
-			},
-			{
-				name: 'A delivery status update arrives, not a customer message',
-				why: 'the messages update also delivers sent, delivered and read statuses, with no messages array',
-				expect: { noWrites: true },
-				trigger: [{
-					messaging_product: 'whatsapp',
-					metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
-					statuses: [{ id: 'wamid.HBgLNDkxNTExMjM0NTY3OBUCABEYEjQ', status: 'delivered', timestamp: '1767258060', recipient_id: '4915112345678' }],
-					field: 'messages',
-				}],
-			},
-		],
-	},
-};
-
-/** The trigger items of a scenario: its own trigger, the case trigger, or the fixture with the scenario message. */
-export function triggerItems(node, scenario = {}, scenarios = {}) {
-	const explicit = scenario.trigger ?? scenarios.trigger;
-	if (explicit) return { items: Array.isArray(explicit) ? explicit : [explicit], source: 'case' };
-	const fixture = TRIGGER_FIXTURES[node.type];
-	if (!fixture) return null;
-	const kit = TRIGGER_KITS[node.type];
-	if (scenario.message !== undefined && kit) return { items: kit.withMessage(fixture, scenario.message), source: 'case' };
-	return { items: fixture, source: 'fixture' };
-}
-
-export const isAiRootNode = (g, node) => node.type.startsWith(LC) && !g.subnodes.has(node.name) && g.aiInputs(node.name, 'ai_languageModel').length > 0;
-
-/** A node whose output comes from a model: an AI root node or a vendor node such as Google Gemini. */
-export const isAiNode = (g, node) => Boolean(node) && (isAiRootNode(g, node) || (AI_VENDOR_TYPES.has(node.type) && !g.subnodes.has(node.name)));
-
-function parserSchema(parser) {
-	const p = parser?.parameters ?? {};
-	const parse = (text) => {
-		try {
-			return typeof text === 'string' ? JSON.parse(text) : text && typeof text === 'object' ? text : null;
-		} catch {
-			return null;
-		}
-	};
-	const schemaType = p.schemaType ?? 'fromJson';
-	if (schemaType === 'manual' && parse(p.inputSchema)) return { kind: 'schema', value: parse(p.inputSchema) };
-	if (schemaType !== 'manual' && parse(p.jsonSchemaExample)) return { kind: 'example', value: parse(p.jsonSchemaExample) };
-	if (parse(p.jsonSchema)) return { kind: 'schema', value: parse(p.jsonSchema) };
-	return null;
-}
-
-/** JSON objects that a prompt shows as examples, in order. Objects with a boolean come first. */
-function promptExamples(node) {
-	const found = [];
-	for (const { value } of strings(node.parameters ?? {})) {
-		for (let start = value.indexOf('{'); start >= 0; start = value.indexOf('{', start + 1)) {
-			let depth = 0;
-			for (let i = start; i < value.length; i++) {
-				if (value[i] === '{') depth++;
-				else if (value[i] === '}' && --depth === 0) {
-					try {
-						const parsed = JSON.parse(value.slice(start, i + 1));
-						if (parsed && typeof parsed === 'object' && !Array.isArray(parsed) && Object.keys(parsed).length)
-							found.push(parsed);
-					} catch {}
-					break;
-				}
-			}
-		}
-	}
-	return found.sort((a, b) => Number(hasBoolean(b)) - Number(hasBoolean(a)));
-}
-
-const hasBoolean = (value) => Object.values(value).some((v) => typeof v === 'boolean');
-
-/** Apply the ai settings of a scenario to a model result: every boolean, then explicit fields. */
-function applyAi(value, ai) {
-	let out = typeof ai.booleans === 'boolean' ? setBooleans(value, ai.booleans) : value;
-	if (ai.set && out && typeof out === 'object') out = deepMerge(out, ai.set);
-	return out;
-}
-
-/**
- * The output of an AI root node for a scenario. Returns { json, provenance }.
- * With a structured output parser, the result follows the parser schema. Without one, the
- * model returns text: ai.text when the scenario sets it, else the first JSON example in the
- * prompt of the node (the contract that the builder declared), else a plain sentence.
- */
-function aiRootOutput(g, node, ai = {}) {
-	const parserNode = g.aiInputs(node.name, 'ai_outputParser')[0];
-	const parser = parserNode ? parserSchema(parserNode) : null;
-	if (parser) {
-		const parsed = parser.kind === 'schema' ? sample(parser.value, 'output') : parser.value;
-		return { json: { output: applyAi(parsed, ai) }, provenance: 'output-parser' };
-	}
-	const wrap = AI_ROOT_PLACEHOLDER[node.type] ?? ((v) => ({ output: v }));
-	if (ai.text !== undefined) return { json: wrap(ai.text), provenance: 'case' };
-	const example = promptExamples(node)[0];
-	if (example) return { json: wrap(JSON.stringify(applyAi(example, ai))), provenance: 'prompt-example' };
-	return { json: wrap('Sample answer from the model.'), provenance: AI_ROOT_PLACEHOLDER[node.type] ? 'ai-root' : 'placeholder' };
-}
-
-/**
- * The pin data of each scenario. Logic nodes run for real. Triggers, service nodes and AI root
- * nodes get pin data, with the provenance of each node, so a check knows how far to trust it:
- *   fixture, case       the shape of real data (trigger code, or the case file)
- *   node-schema         the __schema__ file of the node
- *   output-parser       the schema of the attached structured output parser
- *   prompt-example      a JSON example in the prompt of an AI root node without a parser
- *   ai-root             the known output envelope of the AI root node
- *   placeholder, empty  nothing describes the output, so the shape is a guess or one empty item
- */
-export function pinPlan(workflow, scenarios = {}, only = null) {
-	const g = graph(workflow);
-	const sdk = loadSdk()?.sdk;
-	const list = only ?? (scenarios.scenarios?.length ? scenarios.scenarios : defaultScenarios(g));
-	const plan = { pinned: {}, executed: [], subnodes: [...g.subnodes], scenarios: [] };
-	for (const node of g.nodes) {
-		if (!g.subnodes.has(node.name) && LOGIC_TYPES.has(node.type)) plan.executed.push(node.name);
-	}
-	for (const scenario of list) {
-		const pinData = {};
-		const provenance = {};
-		const byType = mergeByType(scenarios.pinByType, scenario.pinByType);
-		for (const node of g.nodes) {
-			if (g.subnodes.has(node.name) || LOGIC_TYPES.has(node.type)) continue;
-			let items;
-			let source;
-			const caseItems = Object.entries(byType).find(([pattern]) => typeMatches(node.type, pattern))?.[1];
-			const explicit = scenario.trigger ?? scenarios.trigger;
-			const trigger = isTrigger(node.type) && (explicit || !caseItems) ? triggerItems(node, scenario, scenarios) : null;
-			if (trigger) {
-				items = trigger.items;
-				source = trigger.source;
-			} else if (caseItems) {
-				items = caseItems;
-				source = 'case';
-			} else if (isAiRootNode(g, node) || (isAiNode(g, node) && AI_ROOT_PLACEHOLDER[node.type])) {
-				const out = aiRootOutput(g, node, scenario.ai);
-				items = [out.json];
-				source = out.provenance;
-			} else {
-				const schema = sdk?.discoverOutputSchemaForNode(node.type, node.typeVersion, node.parameters ?? {});
-				items = [schema ? sample(schema, 'item') : {}];
-				source = schema ? 'node-schema' : 'empty';
-			}
-			pinData[node.name] = (Array.isArray(items) ? items : [items]).map((json) => ({ json }));
-			provenance[node.name] = source;
-		}
-		plan.scenarios.push({ name: scenario.name, expect: scenario.expect ?? {}, pinData, provenance, byType, triggerNode: scenarioTrigger(g, scenario) });
-		Object.assign(plan.pinned, provenance);
-	}
-	return plan;
-}
-
-/**
- * The trigger that a scenario starts from, or null when the workflow has one trigger. Without
- * scenario.triggerNode, it is the trigger whose downstream nodes match the most expect.reach types.
- */
-function scenarioTrigger(g, scenario) {
-	if (g.triggers.length < 2) return null;
-	if (scenario.triggerNode && g.triggers.some((t) => t.name === scenario.triggerNode)) return scenario.triggerNode;
-	const reach = scenario.expect?.reach ?? [];
-	const score = (trigger) => {
-		const types = [...downstream(g, trigger.name)].map((name) => g.byName.get(name)?.type ?? '');
-		return reach.filter((pattern) => types.some((type) => typeMatches(type, pattern))).length;
-	};
-	return g.triggers.reduce((best, t) => (score(t) > score(best) ? t : best)).name;
-}
-
-/** Case pin data by node type. A scenario entry replaces a case entry for the same short type. */
-function mergeByType(base = {}, extra = {}) {
-	const out = { ...base };
-	for (const [pattern, items] of Object.entries(extra ?? {})) {
-		for (const key of Object.keys(out)) if (shortType(key).toLowerCase() === shortType(pattern).toLowerCase()) delete out[key];
-		out[pattern] = items;
-	}
-	return out;
-}
-
-/** Without a case file: one scenario for each value of the booleans that an AI node returns. */
-function defaultScenarios(g) {
-	const hasAi = g.nodes.some((n) => g.aiInputs(n.name, 'ai_languageModel').length && !g.subnodes.has(n.name));
-	return hasAi
-		? [{ name: 'AI flags true', ai: { booleans: true } }, { name: 'AI flags false', ai: { booleans: false } }]
-		: [{ name: 'Default run' }];
-}
-
-// --- CLI -----------------------------------------------------------------------
-
-if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
-	const [command, workflowFile, scenariosFile] = process.argv.slice(2);
-	if (command !== 'pin-plan' || !workflowFile) {
-		console.error('usage: node sdk.mjs pin-plan <workflow.json> [scenarios.json]');
-		process.exit(2);
-	}
-	const workflow = JSON.parse(readFileSync(workflowFile, 'utf8'));
-	const scenarios = scenariosFile && existsSync(scenariosFile) ? JSON.parse(readFileSync(scenariosFile, 'utf8')) : {};
-	process.stdout.write(JSON.stringify(pinPlan(workflow, scenarios)));
-}

.agents/skills/testmax-n8n-ai/scripts/summarize_runs.py

@@ -1,215 +0,0 @@
-#!/usr/bin/env python3
-"""Print the evidence of each run for the judge step.
-
-  summarize_runs.py <run-dir>            print the summary with the builder names
-  summarize_runs.py <run-dir> --blind    write judge/summary-blind.txt and judge/blind-map.json
-
-The summary has the expected outcome criteria with their ids, and for each builder: the final
-agent message, every node with all its parameters and settings, the connections, and the facts
-from auto-checks.json. Nothing is truncated, so the judge can cite exact values.
-
-In blind mode the builders are called A, B and C in a random order, and builder and model names
-are removed from the final messages. blind-map.json maps each letter back to its builder.
-"""
-
-import argparse
-import json
-import random
-import re
-import sys
-from pathlib import Path
-
-sys.path.insert(0, str(Path(__file__).resolve().parent))
-from build_report import load_case, parse_criteria  # noqa: E402
-
-NODE_SETTINGS = ('disabled', 'onError', 'retryOnFail', 'maxTries', 'waitBetweenTries', 'executeOnce',
-                 'alwaysOutputData', 'notesInFlow', 'notes')
-BUILDER_NAMES = re.compile(
-	r'\b(Instance AI|n8n Assistant|Claude Code|Claude|Anthropic|Codex|OpenAI|ChatGPT|GPT-[\w.-]+|Sonnet|Opus|Haiku)\b',
-	re.IGNORECASE)
-MARK = {True: '✓', False: '✗', None: '?'}
-REPO = Path(__file__).resolve().parents[4]
-LLM_CHECKS_DIR = REPO / 'packages/@n8n/instance-ai/evaluations/binaryChecks/checks'
-# The LLM binary checks of the eval suite that the judges answer as quality checks.
-QUALITY_CHECKS = ('valid-data-flow', 'correct-node-operations', 'handles-multiple-items', 'descriptive-node-names',
-                  'response-describes-changes-accurately')
-
-
-def quality_checks():
-	"""[(name, description, rules)] read from the check sources, so the rules stay those of the eval suite."""
-	out = []
-	for file in QUALITY_CHECKS:
-		path = LLM_CHECKS_DIR / f'{file}.ts'
-		if not path.exists():
-			continue
-		text = path.read_text()
-		name = re.search(r"name:\s*'([^']+)'", text)
-		desc = re.search(r"description:\s*(['\"])(.*?)\1,", text, re.S)
-		rules = re.search(r'systemPrompt:\s*`((?:\\`|[^`])*)`', text)
-		if name and rules:
-			out.append((name.group(1), desc.group(2) if desc else '', rules.group(1).replace('\\`', '`').strip()))
-	return out
-
-
-def quality_text():
-	checks = quality_checks()
-	if not checks:
-		return ''
-	lines = ['QUALITY CHECKS (from the Instance AI eval suite). For each builder, answer each check once in "quality",',
-	         'with the check name as the id. The final message is the narration of the agent. Each build starts',
-	         'from an empty workflow, so the START workflow is empty.']
-	for name, desc, rules in checks:
-		lines += [f'--- [{name}] {desc}', rules]
-	return '\n'.join(lines) + '\n'
-
-
-def session_final_message(sub, meta):
-	"""Final agent message from the session file of an interactive CLI run."""
-	if not meta.get('sessionFile'):
-		return None
-	text = None
-	for line in (sub / meta['sessionFile']).read_text().splitlines():
-		row = json.loads(line)
-		payload = row.get('payload') or {}
-		if row.get('type') == 'event_msg' and payload.get('type') == 'task_complete':
-			text = payload.get('last_agent_message') or text
-		if row.get('type') == 'assistant':
-			blocks = (row.get('message') or {}).get('content') or []
-			texts = [b['text'] for b in blocks if isinstance(b, dict) and b.get('type') == 'text' and b.get('text', '').strip()]
-			text = '\n'.join(texts) or text
-	return text
-
-
-def final_message(sub, meta):
-	kind = meta.get('kind')
-	if kind == 'instance-ai':
-		trace = json.loads((sub / 'trace.json').read_text())
-		rich = trace['rich'].get('data', trace['rich'])
-		assistant = [m for m in rich['messages'] if m.get('role') == 'assistant']
-		return (assistant[-1].get('agentTree') or {}).get('textContent') or assistant[-1].get('content') if assistant else None
-	if not (sub / 'stream.jsonl').exists():
-		return session_final_message(sub, meta)
-	text = None
-	for line in (sub / 'stream.jsonl').read_text().splitlines():
-		event = json.loads(line)
-		if event.get('type') == 'result':
-			text = event.get('result')
-		item = event.get('item') or {}
-		if event.get('type') == 'item.completed' and item.get('type') == 'agent_message':
-			text = item.get('text')
-	return text
-
-
-def connections(workflow):
-	lines = []
-	for source, outputs in (workflow.get('connections') or {}).items():
-		for kind, branches in outputs.items():
-			for index, targets in enumerate(branches or []):
-				for target in targets or []:
-					branch = f'[{index}]' if len(branches) > 1 else ''
-					# Merge and other multi-input nodes need the input index to show which side an edge feeds.
-					target_index = target.get('index') or 0
-					inlet = f'[in {target_index}]' if target_index else ''
-					via = '' if kind == 'main' else f' ({kind})'
-					lines.append(f'  {source}{branch} -> {target["node"]}{inlet}{via}')
-	return lines
-
-
-def criteria_text(expected):
-	"""The expected outcome with the id of each criterion, as the judge must cite it."""
-	groups = parse_criteria(expected or '')
-	if not groups:
-		return (expected or '(none)').strip()
-	lines = []
-	for heading, items in groups:
-		if heading:
-			lines.append(f'{heading}:')
-		lines.extend(f'  [{num}] {text}' for num, text in items)
-	return '\n'.join(lines)
-
-
-def auto_check_lines(sub):
-	path = sub / 'auto-checks.json'
-	if not path.exists():
-		return ['  (none: derive_checks.mjs did not run)']
-	lines = []
-	for group in json.loads(path.read_text()).get('groups', []):
-		score = group.get('score')
-		suffix = f' · score {score["passed"]}/{score["total"]}' if score else ''
-		lines.append(f'  {group["title"]}{suffix} — {group.get("note", "")}')
-		for c in group.get('checks', []):
-			mark = 'i' if c.get('level') == 'info' else MARK.get(c.get('ok'), '?')
-			lines.append(f'    [{mark}] {c["id"]} {c["text"]}')
-			lines.extend(f'        · {d}' for d in c.get('details') or [])
-	return lines
-
-
-def run_section(sub, meta, label, blind=False):
-	out = [f'===== {label} workflow={meta.get("workflowId")} =====']
-	try:
-		message = (final_message(sub, meta) or '(none)').strip()
-		out.append('FINAL MESSAGE:\n' + (scrub(message, meta) if blind else message))
-	except Exception as e:
-		out.append(f'FINAL MESSAGE: unreadable ({e})')
-	wf_file = sub / 'workflow.json'
-	if not wf_file.exists():
-		out.append('WORKFLOW: none\n')
-		return '\n'.join(out)
-	workflow = json.loads(wf_file.read_text())
-	out.append(f'WORKFLOW: {workflow.get("name")}  nodes={len(workflow.get("nodes", []))}')
-	out.append(f'SETTINGS: {json.dumps(workflow.get("settings") or {}, ensure_ascii=False)}')
-	for node in workflow.get('nodes', []):
-		creds = ', '.join((node.get('credentials') or {}).keys())
-		out.append(f'- {node["name"]} | {node["type"]} v{node.get("typeVersion")}' + (f' | creds: {creds}' if creds else ''))
-		settings = {k: node[k] for k in NODE_SETTINGS if k in node}
-		if settings:
-			out.append(f'    (settings) {json.dumps(settings, ensure_ascii=False)}')
-		for key, value in (node.get('parameters') or {}).items():
-			text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
-			out.append(f'    {key}: {text}')
-	out.append('CONNECTIONS:')
-	out.append('\n'.join(connections(workflow)) or '  (none)')
-	out.append('AUTO-CHECK FACTS (deterministic, from the node definitions, the n8n editor, pinned-data runs, '
-	           'in-process runs with recorded API requests, the eval binary checks and the final-message claims):')
-	out.extend(auto_check_lines(sub))
-	return '\n'.join(out) + '\n'
-
-
-def scrub(text, meta):
-	"""Remove builder and model names from a final message. The workflow itself stays as built."""
-	for name in filter(None, [meta.get('model'), meta.get('kind')]):
-		text = re.sub(re.escape(name), 'the builder', text, flags=re.IGNORECASE)
-	return BUILDER_NAMES.sub('the builder', text)
-
-
-def main():
-	ap = argparse.ArgumentParser(description='Print the evidence of each run for the judge step.')
-	ap.add_argument('run_dir', nargs='?', default='.')
-	ap.add_argument('--blind', action='store_true', help='Write a summary with the builders called A, B and C.')
-	ap.add_argument('--seed', type=int, help='Seed of the blind order, for a reproducible test.')
-	args = ap.parse_args()
-	run_dir = Path(args.run_dir)
-	case = load_case(run_dir)
-	runs = [(f.parent, json.loads(f.read_text())) for f in sorted(run_dir.glob('*/run.json')) if f.parent.name != 'report']
-	header = ('EXPECTED OUTCOME CRITERIA (cite each criterion by its id in brackets):\n'
-	          f'{criteria_text(case.get("expected"))}\n\n{quality_text()}')
-
-	if not args.blind:
-		print(header)
-		for sub, meta in runs:
-			print(run_section(sub, meta, f'{sub.name} ({meta.get("kind")})'))
-		return
-
-	order = runs[:]
-	random.Random(args.seed).shuffle(order)
-	letters = [chr(ord('A') + i) for i in range(len(order))]
-	sections = [run_section(sub, meta, f'Builder {letter}', blind=True) for letter, (sub, meta) in zip(letters, order)]
-	judge_dir = run_dir / 'judge'
-	judge_dir.mkdir(exist_ok=True)
-	(judge_dir / 'summary-blind.txt').write_text(header + '\n' + '\n'.join(sections))
-	(judge_dir / 'blind-map.json').write_text(json.dumps({letter: sub.name for letter, (sub, _) in zip(letters, order)}, indent=2))
-	print(judge_dir / 'summary-blind.txt')
-
-
-if __name__ == '__main__':
-	main()

.agents/skills/testmax-n8n-ai/scripts/wire.mjs

@@ -1,459 +0,0 @@
-// Run a workflow in this process with the real node code, and answer its HTTP requests here.
-//
-// The n8n execution engine and the nodes load from the compiled dist folders of the repository,
-// as in the execution evaluator of ai-workflow-builder.ee. n8n-core calls additionalData
-// .evalLlmMockHandler before it authenticates or sends a request (the Instance AI eval hook),
-// so every service request reaches respond() below and no request leaves the machine.
-// Credentials come from buildEvalMockCredentials, the same placeholders that the eval uses.
-// AI root nodes either get pinned data, or call their real model with a key from the settings.
-import { createRequire } from 'node:module';
-import { existsSync, readFileSync } from 'node:fs';
-import { join } from 'node:path';
-
-import { repo } from './sdk.mjs';
-
-const coreRequire = createRequire(join(repo, 'packages/core/package.json'));
-const PACKAGES = [
-	['n8n-nodes-base', join(repo, 'packages/nodes-base')],
-	['@n8n/n8n-nodes-langchain', join(repo, 'packages/@n8n/nodes-langchain')],
-];
-const REQUIRED_DIST = [
-	'packages/core/dist/execution-engine/index.js',
-	'packages/nodes-base/dist/known/nodes.json',
-	'packages/@n8n/nodes-langchain/dist/known/nodes.json',
-];
-
-// --- Settings ------------------------------------------------------------------
-
-// Same order as n8n_api.config(): .env.local, then .env.ai-surfaces, then the environment.
-const fileSettings = {};
-for (const file of [join(repo, '.env.local'), process.env.AI_SURFACES_ENV ?? join(repo, '.env.ai-surfaces')]) {
-	if (!existsSync(file)) continue;
-	for (const line of readFileSync(file, 'utf8').split('\n')) {
-		const match = line.match(/^\s*(?:export\s+)?([A-Z0-9_]+)\s*=\s*(.*)$/);
-		const value = match?.[2].trim().replace(/^(['"])(.*)\1$/, '$2');
-		if (value) fileSettings[match[1]] = value;
-	}
-}
-export const setting = (name) => process.env[name] || fileSettings[name] || '';
-
-/** Model credential types, the settings that can hold their key, and the hosts of their API. */
-export const MODEL_CREDENTIALS = {
-	googlePalmApi: { keys: ['GOOGLE_GENERATIVE_AI_API_KEY', 'GEMINI_API_KEY', 'GOOGLE_API_KEY'], field: 'apiKey', hosts: ['generativelanguage.googleapis.com'] },
-	openAiApi: { keys: ['OPENAI_API_KEY'], field: 'apiKey', hosts: ['api.openai.com'] },
-	anthropicApi: { keys: ['ANTHROPIC_API_KEY'], field: 'apiKey', hosts: ['api.anthropic.com'] },
-	mistralCloudApi: { keys: ['MISTRAL_API_KEY'], field: 'apiKey', hosts: ['api.mistral.ai'] },
-	groqApi: { keys: ['GROQ_API_KEY'], field: 'apiKey', hosts: ['api.groq.com'] },
-};
-export const modelKey = (type) => MODEL_CREDENTIALS[type]?.keys.map(setting).find(Boolean) ?? '';
-
-// --- Engine --------------------------------------------------------------------
-
-let engine;
-
-/** The execution engine, node types and credential types. Returns { error } when a dist folder is missing. */
-export function loadEngine() {
-	if (engine) return engine;
-	const missing = REQUIRED_DIST.filter((p) => !existsSync(join(repo, p)));
-	if (missing.length) return { error: `Build these first: ${missing.join(', ')}.` };
-	coreRequire('reflect-metadata');
-	const workflowLib = coreRequire('n8n-workflow');
-	const core = coreRequire(join(repo, 'packages/core/dist/execution-engine/index.js'));
-	const nodes = new Map();
-	const credentials = new Map();
-	for (const [prefix, dir] of PACKAGES) {
-		const read = (file) => JSON.parse(readFileSync(join(dir, 'dist/known', file), 'utf8'));
-		for (const [name, info] of Object.entries(read('nodes.json'))) nodes.set(`${prefix}.${name}`, { ...info, dir });
-		for (const [name, info] of Object.entries(read('credentials.json'))) credentials.set(name, { ...info, dir });
-	}
-	const instances = new Map();
-	const instance = (index, name) => {
-		const key = `${index === nodes ? 'n' : 'c'}:${name}`;
-		if (!instances.has(key)) {
-			const info = index.get(name);
-			if (!info) throw new Error(`Unknown type ${name}`);
-			const mod = createRequire(join(info.dir, 'package.json'))(join(info.dir, info.sourcePath));
-			instances.set(key, new mod[info.className]());
-		}
-		return instances.get(key);
-	};
-	const nodeTypes = {
-		getByName: (type) => instance(nodes, type),
-		getByNameAndVersion: (type, version) => {
-			const loaded = instance(nodes, type);
-			const versioned = 'nodeVersions' in loaded ? workflowLib.NodeHelpers.getVersionedNodeType(loaded, version) : loaded;
-			// The same rule as shouldAssignExecuteMethod in packages/cli/src/utils.ts. Without it a
-			// declarative node with a webhook (WhatsApp send and wait) gets no input data for $json.
-			const declarative = versioned.description?.requestDefaults !== undefined;
-			if (!versioned.execute && !versioned.supplyData && !versioned.poll && !versioned.trigger && (!versioned.webhook || declarative) && (!versioned.methods || declarative)) {
-				versioned.execute = async function () {
-					return (await new core.RoutingNode(this, versioned).runNode()) ?? [];
-				};
-			}
-			return versioned;
-		},
-		getKnownTypes: () => ({}),
-	};
-	const credentialType = (name) => instance(credentials, name);
-	const parentTypes = (name) => {
-		try {
-			return (credentialType(name).extends ?? []).flatMap((p) => [p, ...parentTypes(p)]);
-		} catch {
-			return [];
-		}
-	};
-	const credentialProperties = (name) => [...(credentialType(name).extends ?? []).flatMap(credentialProperties), ...(credentialType(name).properties ?? [])];
-	engine = { workflowLib, core, nodeTypes, nodes, credentialType, parentTypes, credentialProperties };
-	return engine;
-}
-
-/** Credential types that a node can use, from its node type description. */
-export function nodeCredentialTypes(node) {
-	try {
-		return (loadEngine().nodeTypes.getByNameAndVersion(node.type, node.typeVersion).description.credentials ?? []).map((c) => c.name);
-	} catch {
-		return [];
-	}
-}
-
-/** Apply the generic authenticate rule of a credential type, such as `qs.key` of Google Gemini. */
-function applyGenericAuth(credentialType, credentials, request) {
-	const auth = credentialType.authenticate;
-	if (!auth || typeof auth !== 'object' || auth.type !== 'generic') return request;
-	const fill = (v) => (typeof v === 'string' ? v.replace(/^=/, '').replace(/\{\{\s*\$credentials\.(\w+)\s*\}\}/g, (_m, k) => credentials[k] ?? '') : v);
-	for (const part of ['headers', 'qs', 'body']) {
-		const props = auth.properties?.[part];
-		if (props) request[part] = { ...(request[part] ?? {}), ...Object.fromEntries(Object.entries(props).map(([k, v]) => [k, fill(v)])) };
-	}
-	return request;
-}
-
-function credentialsHelper(realModels) {
-	const { core, parentTypes, credentialProperties, credentialType } = loadEngine();
-	return {
-		getParentTypes: parentTypes,
-		getDecrypted: async (_data, _details, type) => {
-			const mocked = core.buildEvalMockCredentials(credentialProperties(type));
-			const model = MODEL_CREDENTIALS[type];
-			if (realModels && model) mocked[model.field] = modelKey(type);
-			return mocked;
-		},
-		// Vendor model nodes send their request through the n8n helpers, so the real key must be applied here.
-		authenticate: async (credentials, type, request) => (realModels && MODEL_CREDENTIALS[type] ? applyGenericAuth(credentialType(type), credentials, request) : request),
-		preAuthentication: async () => undefined,
-		runPreAuthentication: async () => undefined,
-		getCredentials: async () => ({}),
-		getCredentialsProperties: (type) => credentialProperties(type),
-		updateCredentials: async () => {},
-		updateCredentialsOauthTokenData: async () => {},
-		isCredentialUsableByNode: () => true,
-	};
-}
-
-// --- Setup values ----------------------------------------------------------------
-
-/** Values that a node accepts for a parameter that the builder left for setup. */
-const SETUP_VALUES = {
-	'googleSheets.sheetName': 'gid=0',
-};
-
-const isEmptyValue = (v) => v === undefined || v === null || v === '';
-
-/**
- * A copy of the workflow with placeholder values in the parameters that the builder left for
- * the user: empty resource locators, and empty parameters that the editor reports as required.
- * Returns { workflow, filled }.
- */
-export function fillSetup(workflow, canvas) {
-	const copy = structuredClone(workflow);
-	const filled = [];
-	const required = new Map();
-	for (const node of canvas?.nodes ?? []) {
-		for (const [param, messages] of Object.entries(node.issues?.parameters ?? {})) {
-			if (messages.some((m) => /is required/i.test(m))) required.set(`${node.name}|${param}`, true);
-		}
-	}
-	for (const node of copy.nodes ?? []) {
-		const short = node.type.slice(node.type.lastIndexOf('.') + 1);
-		for (const [param, value] of Object.entries(node.parameters ?? {})) {
-			const placeholder = SETUP_VALUES[`${short}.${param}`] ?? `mock-${param}`;
-			if (value && typeof value === 'object' && value.__rl && isEmptyValue(value.value)) {
-				node.parameters[param] = { ...value, value: placeholder };
-				filled.push(`${node.name} › ${param}`);
-			} else if (isEmptyValue(value) && required.has(`${node.name}|${param}`)) {
-				node.parameters[param] = placeholder;
-				filled.push(`${node.name} › ${param}`);
-			}
-		}
-	}
-	return { workflow: copy, filled };
-}
-
-// --- Wire responses --------------------------------------------------------------
-
-const WRITE_METHODS = new Set(['POST', 'PUT', 'PATCH', 'DELETE']);
-export const isWrite = (request) => WRITE_METHODS.has(request.method);
-
-/** Rows for a read node: [{ Question, Answer }, ...] become the header row and the value rows. */
-function sheetValues(rows) {
-	const headers = [...new Set(rows.flatMap((r) => Object.keys(r ?? {})))];
-	if (!headers.length) return [['Question', 'Answer']];
-	return [headers, ...rows.map((r) => headers.map((h) => (r?.[h] === undefined ? '' : String(r[h]))))];
-}
-
-/** Page properties as Notion returns them: each with an id, a type, and plain_text in rich text. */
-function notionProperties(sent = {}) {
-	const out = {};
-	for (const [name, value] of Object.entries(sent ?? {})) {
-		const type = value?.type ?? Object.keys(value ?? {}).find((k) => k !== 'id' && k !== 'type') ?? 'rich_text';
-		let content = value?.[type];
-		if (Array.isArray(content)) content = content.map((t) => ({ type: 'text', ...t, plain_text: t?.plain_text ?? t?.text?.content ?? '' }));
-		out[name] = { id: `prop-${Object.keys(out).length}`, type, [type]: content ?? null };
-	}
-	if (!Object.values(out).some((p) => p.type === 'title')) out.Name = { id: 'title', type: 'title', title: [] };
-	return out;
-}
-
-const bodyOf = (request) => {
-	if (typeof request.body !== 'string') return request.body ?? {};
-	try {
-		return JSON.parse(request.body);
-	} catch {
-		return {};
-	}
-};
-
-/**
- * Responses in the shape that each API documents. A fixture matches on host, method and path.
- * rows(node) gives the case rows for a read node (scenarios.json pinByType).
- */
-const FIXTURES = [
-	{
-		service: 'Google Sheets',
-		host: /(^|\.)sheets\.googleapis\.com$/,
-		method: 'GET',
-		path: /^\/v4\/spreadsheets\/[^/]+$/,
-		respond: () => ({ spreadsheetId: 'mock-sheet', properties: { title: 'FAQ' }, sheets: [{ properties: { sheetId: 0, title: 'Sheet1', index: 0, gridProperties: { rowCount: 100, columnCount: 26 } } }] }),
-	},
-	{
-		service: 'Google Sheets',
-		host: /(^|\.)sheets\.googleapis\.com$/,
-		method: 'GET',
-		path: /^\/v4\/spreadsheets\/[^/]+\/values\//,
-		respond: (_r, ctx, node) => ({ range: 'Sheet1!A1:Z100', majorDimension: 'ROWS', values: sheetValues(ctx.rows(node) ?? []) }),
-	},
-	{
-		service: 'Google Sheets',
-		host: /(^|\.)sheets\.googleapis\.com$/,
-		method: 'POST',
-		path: /:append$|values:batchUpdate$|:batchUpdate$/,
-		respond: () => ({ spreadsheetId: 'mock-sheet', updates: { updatedRange: 'Sheet1!A2:B2', updatedRows: 1 }, replies: [] }),
-	},
-	{
-		service: 'Notion',
-		host: /(^|\.)api\.notion\.com$/,
-		method: 'GET',
-		path: /^\/v1\/(databases|data_sources)\/[^/]+$/,
-		respond: (r) => ({
-			object: r.url.includes('data_sources') ? 'data_source' : 'database',
-			id: 'mock-database',
-			title: [{ type: 'text', plain_text: 'Support tickets', text: { content: 'Support tickets' } }],
-			properties: { Name: { id: 'title', name: 'Name', type: 'title', title: {} } },
-			data_sources: [{ id: 'mock-data-source', name: 'Support tickets' }],
-		}),
-	},
-	{
-		service: 'Notion',
-		host: /(^|\.)api\.notion\.com$/,
-		method: 'POST',
-		path: /^\/v1\/pages$/,
-		respond: (r) => {
-			const body = bodyOf(r);
-			return { object: 'page', id: '59833787-2cf9-4fdf-8782-e53db20768a5', created_time: '2026-01-01T09:00:00.000Z', last_edited_time: '2026-01-01T09:00:00.000Z', archived: false, in_trash: false, parent: body.parent ?? {}, properties: notionProperties(body.properties), url: 'https://www.notion.so/Ticket-598337872cf94fdf8782e53db20768a5', public_url: null };
-		},
-	},
-	{
-		service: 'Notion',
-		host: /(^|\.)api\.notion\.com$/,
-		method: 'PATCH',
-		path: /^\/v1\/blocks\/[^/]+\/children$/,
-		respond: (r) => ({ object: 'list', results: (bodyOf(r).children ?? []).map((c, i) => ({ object: 'block', id: `mock-block-${i}`, ...c })), next_cursor: null, has_more: false }),
-	},
-	{
-		service: 'Slack',
-		host: /(^|\.)slack\.com$/,
-		method: 'POST',
-		path: /^\/api\/chat\.postMessage$/,
-		respond: (r) => {
-			const body = bodyOf(r);
-			return { ok: true, channel: body.channel ?? 'C0000000000', ts: '1767258000.000100', message: { type: 'message', text: body.text ?? '', ts: '1767258000.000100', bot_id: 'B0000000000' } };
-		},
-	},
-	{
-		service: 'WhatsApp Cloud API',
-		host: /(^|\.)graph\.facebook\.com$/,
-		method: 'POST',
-		path: /^\/v\d+(\.\d+)?\/[^/]+\/messages$/,
-		respond: (r) => {
-			const to = bodyOf(r).to ?? '';
-			return { messaging_product: 'whatsapp', contacts: [{ input: to, wa_id: to }], messages: [{ id: 'wamid.HBgLMTU1NTAwMDExMTEVAgARGBI5QTNDQTVCM0Q0Q0Q2RTY3RTcA' }] };
-		},
-	},
-	{
-		service: 'Telegram Bot API',
-		host: /(^|\.)api\.telegram\.org$/,
-		method: 'POST',
-		path: /\/sendMessage$/,
-		respond: (r) => ({ ok: true, result: { message_id: 2, date: 1767258000, chat: { id: bodyOf(r).chat_id ?? 1001, type: 'private' }, text: bodyOf(r).text ?? '' } }),
-	},
-	{
-		service: 'Gmail',
-		host: /(^|\.)gmail\.googleapis\.com$|(^|\.)www\.googleapis\.com$/,
-		method: 'POST',
-		path: /\/messages\/send$/,
-		respond: () => ({ id: 'mock-message', threadId: 'mock-thread', labelIds: ['SENT'] }),
-	},
-];
-
-/**
- * A request handler for evalLlmMockHandler. Every request is recorded in ctx.requests with the
- * fixture that answered it. A request with no fixture gets a generic answer: a read gets an
- * empty object, a write gets its own body back with an id. A request to a host in passHosts
- * goes to the network: these are the model APIs of the real-AI scenarios.
- */
-export function wireHandler(ctx, passHosts = new Set()) {
-	return async (request, node) => {
-		const url = new URL(request.url ?? '', 'https://unknown.invalid');
-		if (request.baseURL && !/^https?:/.test(request.url ?? '')) {
-			const base = new URL(request.baseURL);
-			url.host = base.host;
-			url.pathname = `${base.pathname.replace(/\/$/, '')}/${(request.url ?? '').replace(/^\//, '')}`;
-		}
-		const method = String(request.method ?? 'GET').toUpperCase();
-		if (passHosts.has(url.hostname)) {
-			ctx.requests.push({ node: node.name, nodeType: node.type, method, url: url.toString(), body: request.body, answeredBy: 'real model' });
-			return undefined;
-		}
-		const fixture = FIXTURES.find((f) => f.host.test(url.hostname) && f.method === method && f.path.test(url.pathname));
-		const record = { node: node.name, nodeType: node.type, method, url: url.toString(), qs: request.qs, body: request.body, answeredBy: fixture?.service ?? 'generic' };
-		let body;
-		try {
-			body = fixture ? fixture.respond({ ...request, url: url.toString() }, ctx, node) : method === 'GET' ? {} : { id: 'mock-1', ...(bodyOf(request) ?? {}) };
-		} catch (error) {
-			body = { error: String(error) };
-		}
-		ctx.requests.push({ ...record, response: body });
-		return { statusCode: 200, headers: { 'content-type': 'application/json' }, body };
-	};
-}
-
-// --- Network guard ---------------------------------------------------------------
-
-/**
- * Replace global fetch while a run is active. Model SDKs use fetch directly, so this is the
- * only path that could reach the network. Only the hosts in allow pass, the rest is blocked.
- */
-function guardFetch(allow, blocked) {
-	const original = globalThis.fetch;
-	globalThis.fetch = async (input, init) => {
-		const url = new URL(typeof input === 'string' ? input : input.url ?? String(input));
-		if (allow.has(url.hostname)) return await original(input, init);
-		blocked.push(`${init?.method ?? 'GET'} ${url.origin}${url.pathname}`);
-		throw new Error(`Blocked network request to ${url.hostname} during a test run.`);
-	};
-	return () => {
-		globalThis.fetch = original;
-	};
-}
-
-// --- Code nodes ------------------------------------------------------------------
-
-/**
- * additionalData.startRunnerTask. The n8n server sends Code node jobs to a sandboxed task runner.
- * This process has no sandbox, so it never runs builder code. run_wire.mjs skips workflows with a
- * Code node. This refusal catches a node type that it does not know.
- */
-async function startRunnerTask(_data, jobType) {
-	return { ok: false, error: { message: `${jobType} code does not run in the in-process run.` } };
-}
-
-// --- Run -------------------------------------------------------------------------
-
-const CANCEL_GRACE_MS = 15_000;
-
-/** The value of promise, or null after ms. */
-async function within(promise, ms) {
-	let timer;
-	try {
-		return await Promise.race([promise, new Promise((resolve) => (timer = setTimeout(() => resolve(null), ms)))]);
-	} finally {
-		clearTimeout(timer);
-	}
-}
-
-/** The pinned trigger that a run starts from: triggerNode, or else the first pinned trigger. */
-export const startNode = (workflow, pinData, triggerNode) =>
-	workflow.nodes.find((n) => (!triggerNode || n.name === triggerNode) && Object.hasOwn(pinData, n.name) && !n.disabled && Object.keys(workflow.connections ?? {}).includes(n.name) && /trigger|webhook/i.test(n.type));
-
-/**
- * Run a workflow once. pinData holds the trigger and any pinned nodes. triggerNode names the
- * trigger to start from. With realModels, model sub-nodes get the real key and fetch can reach
- * their API hosts. Returns { status, error, runData, requests, blocked, ms }.
- */
-export async function runWorkflow(workflow, { pinData, triggerNode, rows = () => undefined, realModels = false, timeoutMs = 90_000 }) {
-	const { workflowLib, core, nodeTypes } = loadEngine();
-	const started = Date.now();
-	const ctx = { requests: [], rows };
-	const blocked = [];
-	const allow = new Set(realModels ? Object.values(MODEL_CREDENTIALS).flatMap((m) => m.hosts) : []);
-	const restore = guardFetch(allow, blocked);
-	let stillRunning = false;
-	try {
-		const instance = new workflowLib.Workflow({ id: 'wire-run', name: workflow.name, nodes: workflow.nodes, connections: workflow.connections, nodeTypes, active: false, settings: workflow.settings ?? {} });
-		const start = startNode(workflow, pinData, triggerNode);
-		if (!start) return { status: 'error', error: 'No pinned trigger to start from.', runData: {}, requests: [], blocked, ms: 0 };
-		const hooks = new core.ExecutionLifecycleHooks('manual', 'wire-run', workflow);
-		const finished = new Promise((resolve) => hooks.addHandler('workflowExecuteAfter', (run) => resolve(run)));
-		const values = {
-			executionId: 'wire-run',
-			hooks,
-			credentialsHelper: credentialsHelper(realModels),
-			evalLlmMockHandler: wireHandler(ctx, allow),
-			startRunnerTask,
-			getRunnerStatus: () => ({ available: false }),
-			variables: {},
-			restApiUrl: 'http://localhost:5678/rest',
-			instanceBaseUrl: 'http://localhost:5678',
-			webhookBaseUrl: 'http://localhost:5678/webhook',
-			webhookTestBaseUrl: 'http://localhost:5678/webhook-test',
-			webhookWaitingBaseUrl: 'http://localhost:5678/webhook-waiting',
-			formWaitingBaseUrl: 'http://localhost:5678/form-waiting',
-			userId: 'wire-run-user',
-		};
-		// Unset fields stay undefined. Methods that n8n calls for logs and UI updates do nothing.
-		const additionalData = new Proxy({}, {
-			get: (_t, prop) => (prop in values ? values[prop] : typeof prop === 'string' && /^(log|send|set|on|emit|track)/.test(prop) ? () => undefined : undefined),
-		});
-		const runExecutionData = workflowLib.createRunExecutionData({
-			executionData: { nodeExecutionStack: [{ node: start, data: { main: [pinData[start.name]] }, source: null }] },
-			resultData: { pinData },
-		});
-		const execute = new core.WorkflowExecute(additionalData, 'manual', runExecutionData);
-		const execution = execute.processRunExecutionData(instance);
-		const run = await within(execution.then(() => finished), timeoutMs);
-		if (!run) {
-			// Cancel aborts the running node. Wait for the engine to stop, so no request runs after the guard is gone.
-			execution.cancel();
-			stillRunning = !(await within(execution.then(() => true, () => true), CANCEL_GRACE_MS));
-			const error = `No result after ${timeoutMs / 1000} s.${stillRunning ? ' The run did not stop after the cancel.' : ''}`;
-			return { status: 'timeout', error, runData: {}, requests: ctx.requests, blocked, ms: Date.now() - started };
-		}
-		const error = run.data?.resultData?.error;
-		return { status: run.status, error: error ? error.message ?? String(error) : null, runData: run.data?.resultData?.runData ?? {}, requests: ctx.requests, blocked, ms: Date.now() - started };
-	} catch (error) {
-		return { status: 'error', error: `${error?.name ?? 'Error'}: ${error?.message ?? error}`, runData: {}, requests: ctx.requests, blocked, ms: Date.now() - started };
-	} finally {
-		// A run that did not stop keeps the guard. The process exits when all runs are done.
-		if (!stillRunning) restore();
-	}
-}

.aikido

@@ -8,7 +8,7 @@ exclude:
     # Machine-generated instance-AI expectation traces. Contain ephemeral
     # credential reference IDs (nanoids) that re-trip the secrets scanner
     # on every regeneration. No secret material.
-    - packages/quality/testing/playwright/expectations
+    - packages/testing/playwright/expectations
 
     # Test code repo-wide. Fixtures routinely contain synthetic PEM blocks,
     # basic-auth strings, and deliberately fake keys (including the secret

.boundaries-baseline.json

@@ -1,3 +1,3 @@
 {
-  "issues": 110
+  "issues": 163
 }

.claude/plugins/n8n/skills/agent-design-language

@@ -1 +0,0 @@
-../../../../.agents/skills/agent-design-language

.claude/plugins/n8n/skills/env-var-docs

@@ -1 +0,0 @@
-../../../../.agents/skills/env-var-docs

.claude/plugins/n8n/skills/testmax-assistant-ui

@@ -1 +0,0 @@
-../../../../.agents/skills/testmax-assistant-ui

.claude/plugins/n8n/skills/testmax-n8n-ai

@@ -1 +0,0 @@
-../../../../.agents/skills/testmax-n8n-ai

.devcontainer/codespaces/Dockerfile

@@ -4,7 +4,6 @@ ARG NODE_VERSION=24
 FROM node:${NODE_VERSION}-bookworm
 
 ARG PLAYWRIGHT_VERSION=1.62.1
-ARG OPENCODE_VERSION=1.18.25
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
     tmux sudo less jq ripgrep postgresql-client \
@@ -15,7 +14,7 @@ RUN npx -y playwright@${PLAYWRIGHT_VERSION} install-deps chromium
 
 # Agent harnesses. Self-update is off: the image is the update mechanism
 ENV DISABLE_AUTOUPDATER=1
-RUN npm install -g @anthropic-ai/claude-code opencode-ai@${OPENCODE_VERSION}
+RUN npm install -g @anthropic-ai/claude-code opencode-ai
 
 COPY codespaces-env.sh /usr/local/lib/codespaces-env.sh
 COPY codespaces-secrets.sh /etc/profile.d/codespaces-secrets.sh
@@ -38,7 +37,4 @@ ENV PNPM_HOME=/home/node/.local/share/pnpm
 ENV PATH=$PNPM_HOME/bin:$PATH
 
 USER node
-# The store must sit on the same filesystem as the worktrees (/workspaces is its own
-# mount), or pnpm copies node_modules into every worktree instead of hardlinking.
-# The directory is created on the first install: Codespaces mounts /workspaces at runtime.
-RUN mkdir -p $PNPM_HOME && pnpm config set store-dir /workspaces/.pnpm-store --global
+RUN mkdir -p $PNPM_HOME ~/.pnpm-store && pnpm config set store-dir ~/.pnpm-store --global

.devcontainer/codespaces/README.md

@@ -2,7 +2,7 @@
 
 Run long-lived, human-steered agent sessions (Claude Code / OpenCode) on a
 GitHub Codespace instead of your laptop: start a task, close the lid, steer it
-from anywhere, resume tomorrow.
+from anywhere with a terminal, resume tomorrow.
 
 This is a separate devcontainer config from the laptop one in
 `.devcontainer/` — it ships both agent CLIs, `tmux` for session persistence,
@@ -11,73 +11,26 @@ Playwright system deps, and Docker-in-Docker (for testcontainers and
 
 ## One-time setup (~5 min)
 
-1. Give the GitHub CLI the codespace scope:
+1. Add provider keys at [github.com/settings/codespaces](https://github.com/settings/codespaces).
+   Add `ANTHROPIC_API_KEY` for Claude Code. Add `OPENROUTER_API_KEY` for OpenCode.
+   Give both secrets access to `n8n-io/n8n`.
+   (Alternative for Max subscriptions: `CLAUDE_CODE_OAUTH_TOKEN` from `claude setup-token`.)
+2. Give the GitHub CLI the codespace scope:
 
    ```bash
    gh auth refresh -h github.com -s codespace
    ```
 
-2. Add provider keys at [github.com/settings/codespaces](https://github.com/settings/codespaces)
-   if your client needs them. Give each secret access to `n8n-io/n8n`.
-   - `ANTHROPIC_API_KEY` for Claude Code in the terminal (`pnpm session`).
-     Alternative for Max subscriptions: `CLAUDE_CODE_OAUTH_TOKEN` from
-     `claude setup-token`. The desktop app does not need either secret.
-   - `OPENROUTER_API_KEY` for OpenCode.
-
-## Choose a client
-
-All clients run the agent, its tools, and builds in the Codespace. Your laptop
-shows the conversation only.
-
-| Client | Command | Use it when |
-|---|---|---|
-| Claude Code desktop app | `pnpm session ssh-config` once | You want the full desktop experience. |
-| Claude Code in VS Code | Open the codespace in VS Code or the browser | You already work in VS Code. |
-| Claude Code in a terminal | `pnpm session` | You want a plain terminal, or you are on a remote machine. |
-| OpenCode | `pnpm session:opencode` | You use OpenCode. See [Local OpenCode clients](#local-opencode-clients). |
-
-### Claude Code desktop app
-
-1. From a local checkout, add an SSH host for your codespace:
-
-   ```bash
-   pnpm session ssh-config
-   ```
-
-   The command creates or starts the codespace. It writes the host
-   `n8n-codespace` to `~/.ssh/n8n-codespace.conf` and includes that file from
-   `~/.ssh/config`.
-2. In the desktop app, add an SSH connection to the host `n8n-codespace`.
-3. Open the folder `/workspaces/n8n`, or a worktree under `/workspaces`.
-
-The desktop app installs its own Claude Code on the codespace and signs in
-with your desktop account. Sessions continue when you close the app. They stop
-when the codespace stops.
-
-- **Run `pnpm session ssh-config` again after you recreate the codespace.**
-- **If the connection times out**, the codespace is probably starting. Run
-  `pnpm session ssh-config`, then connect again.
-- **Test the connection** with `ssh n8n-codespace`.
-
-### Claude Code in VS Code
-
-The dev container installs the Claude Code extension. Open the codespace in
-VS Code or the browser, then open the Claude Code panel. Sign in from the
-extension if you did not add `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN`.
-
-### Claude Code in a terminal
+## Daily flow
 
 ```bash
 pnpm session                       # attach Claude Code (creates everything on first run)
 pnpm session:shell                 # open a shell in the default checkout
-pnpm session:opencode              # connect the local OpenCode TUI
-pnpm session:opencode fix-flaky    # use a separate remote worktree
-pnpm session:opencode --web        # open OpenCode in the local browser
-pnpm session:opencode --legacy     # use the remote TUI in tmux
+pnpm session:opencode              # attach OpenCode in auto mode
+pnpm session:opencode fix-flaky    # OpenCode in a separate worktree
 pnpm session fix-flaky             # Claude Code in a separate worktree
 pnpm session ls                    # what's running
 pnpm session tunnel                # forward n8n ports (default 5678, 8080); Ctrl-C to stop
-pnpm session ssh-config            # add the n8n-codespace SSH host for the desktop app
 pnpm session stop                  # end of day: billing stops, disk survives
 pnpm session rm                    # delete the codespace
 ```
@@ -85,18 +38,15 @@ pnpm session rm                    # delete the codespace
 The dev container supports Codespaces with 2, 4, or 8 cores. The session commands
 create an 8-core Codespace by default.
 
-- **Detach from tmux** with `Ctrl-b d` — the agent keeps working without you.
-  Local OpenCode connections use the exit controls described below.
-- **Scroll in tmux** with the mouse wheel (tmux mouse mode is on). To use the
+- **Detach** with `Ctrl-b d` — the agent keeps working without you.
+- **Scroll** with the mouse wheel (tmux mouse mode is on). To use the
   terminal's own text selection, hold **Shift** and drag.
 - **Reattach** by running the same session command from any machine.
 - Each named session gets its own worktree (`/workspaces/wt-<name>`, branch
-  `session/<name>`), so parallel agents never touch each other's tree. A new
-  branch starts from the latest `origin/master`. Builds
+  `session/<name>`), so parallel agents never touch each other's tree. Builds
   in fresh worktrees are cache-hits via a shared turbo cache.
 - First codespace creation takes ~20 min uncached (image + full build). After
-  that, sessions attach instantly; new worktrees cost a `pnpm install` (~10 s: the
-  store lives on `/workspaces`, so `node_modules` is hardlinked, not copied).
+  that, sessions attach instantly; new worktrees cost a `pnpm install` (~1–2 min).
 
 ## Configure the local OpenCode harness
 
@@ -111,111 +61,6 @@ The command verifies the release checksum. It caches the bundle under
 `~/.config/opencode/plugins`. Run the command again after the lock changes.
 Restart OpenCode after the command completes.
 
-## Local OpenCode clients
-
-Run these commands from a local checkout. The TUI runs on your laptop. The
-OpenCode server, repository, tools, and builds run in the Codespace. Browser
-mode opens the remote web interface through a local connection.
-
-```bash
-pnpm session:opencode fix-flaky              # resume that workspace's latest conversation
-pnpm session:opencode fix-flaky --web        # open it in a browser
-pnpm session:opencode fix-flaky --new        # start a new conversation in that worktree
-pnpm session:opencode --web --port 4100      # override the default browser port
-pnpm session:opencode --help
-```
-
-For the TUI, install the same OpenCode version as the remote server. The
-`OPENCODE_VERSION` argument in the [Dockerfile](Dockerfile) pins the version
-for new images:
-
-```bash
-npm install -g opencode-ai@<version>
-```
-
-Use npm, not pnpm. The package gets its binary from a `postinstall` script.
-pnpm does not run that script by default.
-
-The launcher checks both versions. It reports a mismatch with both version
-numbers before it opens the TUI. An existing Codespace can have a different
-version. Install that version locally, or use `--web`. Browser mode does not
-need a local OpenCode install.
-The launcher sends its bootstrap code over SSH, so an existing Codespace does
-not need a rebuild to use this connection method.
-The remote server loads the shared OpenCode harness that the Codespace
-post-start command installs. The local harness setup above is not required for
-these commands.
-
-The command prepares the worktree, starts or reuses one server, opens an SSH
-tunnel, and connects the client. Each workspace opens its own conversation: the
-most recently updated one in that worktree. A conversation moves to the front
-when it receives a message. Switching to a conversation without sending a
-message does not move it. Switching to another workspace's conversation does
-not change what this workspace opens next. `--new` starts a new conversation
-instead. It preserves the worktree and the old conversation.
-
-- **Exit the TUI** with `/exit` or its quit shortcut. The launcher closes its
-  tunnel. The remote server stays running.
-- **Disconnect browser mode** with `Ctrl-C` in the launcher terminal. Closing
-  the browser tab does not close the tunnel. Keep the launcher running while
-  you use the browser.
-- **Reconnect** with the same command after a network interruption. After a
-  Codespace stop, the command restarts the server and opens the workspace's
-  most recently updated conversation.
-  A stop terminates running tools. It does not resume interrupted commands.
-- **Browser mode uses local port 4096 by default.** Use `--port` to override it.
-  TUI mode selects an available port unless you specify one.
-  A fixed browser port preserves the browser origin across runs.
-  An occupied fixed port causes an error. It does not stop the existing listener.
-- **Use model and permission controls in the client.** The new connection uses
-  the remote OpenCode configuration. It does not force the legacy `--auto` mode.
-  For the old CLI flags, use `pnpm session:opencode fix-flaky --legacy --model <model>`.
-
-Both server and tunnel bind to `127.0.0.1`. The server uses a generated password.
-The TUI receives it through its environment. Browser mode uses a local proxy
-that adds authentication and rejects requests from other browser origins. The
-proxy supports streamed responses, attachments, and terminal WebSockets. The
-password does not appear in the URL or terminal output. Other processes on
-your laptop can access the local browser proxy while it runs. Do not forward
-this proxy or the OpenCode server port to other machines.
-
-The server enables only OpenRouter. It reads `OPENROUTER_API_KEY` when it starts.
-It enables OpenCode code mode by default to reduce the initial tool context.
-Browser mode opens the workspace's most recently updated conversation directly.
-The web UI stores opened projects in browser storage. If a new-session page
-shows **New project**, open `/workspaces/n8n` there once. Keep the same browser
-port when you reconnect to preserve this selection.
-
-The server runs in the detached tmux session `n8n-opencode-server`. Its log is
-`/workspaces/.n8n-opencode/server.log`. That directory also holds the server
-credentials. It is readable only by its owner.
-An unhealthy server produces an error without stopping active work. Inspect
-the log through `pnpm session:shell`. To restart it after checking active work,
-run `tmux kill-session -t '=n8n-opencode-server'` in that shell. Then reconnect.
-Restart the server after changing provider secrets, server configuration, or
-the pinned harness release.
-
-### Limits
-
-- The launcher supports macOS and Linux. On Windows, run it in WSL. If the
-  browser does not open automatically, open the printed URL yourself.
-- Local clipboard and attachment controls depend on the client, terminal,
-  file type, and model. A laptop file path does not copy a file to the Codespace.
-  Use a supported attachment control or copy the file with `gh codespace cp`.
-- Tools and local MCP processes run on the Codespace. Laptop configuration,
-  browser sessions, and files do not sync automatically.
-- Workspace names identify worktrees. Two agents that use the same name share
-  files. Separate conversations alone do not isolate edits.
-- The server survives a client disconnect while the Codespace stays running.
-  Codespaces idle timeouts still apply. An open tunnel is not a guarantee that
-  the Codespace will stay awake. Use `pnpm session stop` to stop compute billing.
-- Existing tmux OpenCode conversations are not migrated automatically. Use
-  `--legacy` to return to them.
-
-For upstream behavior, see the [OpenCode CLI](https://opencode.ai/docs/cli/),
-[server](https://opencode.ai/docs/server/), and [web](https://opencode.ai/docs/web/)
-documentation.
-
 ## PR previews (a running instance of someone else's PR)
 
 A preview is a codespace that serves one pull request, so a reviewer can use the
@@ -377,15 +222,11 @@ session rarely needs a cold `pnpm install` or a full `pnpm build`. Both are slow
 Claude sessions and the headless OpenCode worker get the `flaky` MCP server automatically: Currents
 flaky/quarantine data, the `qa_*` BigQuery dataset, Sentry RCA, live Linear,
 and repo investigation. The worker keeps the token in its environment and puts
-only an environment reference in the OpenCode config. Forks have no secrets and
+only an environment reference in the OpenCode config. Claude login registers
+the same server without writing the token to disk. Forks have no secrets and
 skip it. Tell the agent to call `get_flaky_context` first — it returns the rules
 the tools assume.
 
-`post-start.mjs` registers the server for Claude Code on each container start.
-Its `headersHelper` reads the token from the secrets file on each connect, so
-the server works in every client. The token is not copied into the Claude Code
-MCP configuration.
-
 ## Quality and security skills (Claude plugins)
 
 Claude sessions can also load the private skills from the
@@ -447,8 +288,7 @@ to pull the skills into context.
 
 ## Viewing the dev UI locally
 
-`pnpm session tunnel` works with every client. With the terminal client, use
-two terminal windows:
+Two terminal windows:
 
 ```bash
 pnpm session          # window 1: attach the agent session
@@ -478,7 +318,7 @@ partial mapping breaks it.
 | Event | Running processes | Disk (checkout, worktrees, chat history) |
 |---|---|---|
 | Detach / close laptop / network drop | ✅ keep running | ✅ |
-| Stop, or idle timeout (2 h for a codespace that `pnpm session` creates, max 4 h) | ❌ killed | ✅ |
+| Stop, or idle timeout (default 30 min, max 4 h) | ❌ killed | ✅ |
 | Delete (`pnpm session rm`) | ❌ | ❌ (push your branches first) |
 
 After a stop, `pnpm session <name>` restarts the codespace (~30–60 s); run
@@ -498,10 +338,10 @@ After a stop, `pnpm session <name>` restarts the codespace (~30–60 s); run
   agent injects them into VS Code sessions only; they're delivered
   base64-encoded to `/workspaces/.codespaces/shared/.env-secrets`. The image
   sources `/usr/local/lib/codespaces-env.sh` in login shells (profile.d), in
-  interactive shells (bashrc), and in the `pnpm session` prelude. Desktop app
-  sessions do not source it. `git`, `gh`, `scripts/codespace-env.mjs`, and the
-  `flaky` MCP entry read the file when they run, so they work everywhere. New
-  code that needs a secret must do the same.
+  interactive shells (bashrc), and in the `pnpm session` prelude. If Claude
+  Code shows `Missing environment variables: FLAKY_MCP_TOKEN`, the shell that
+  started Claude did not source the file. Run
+  `. /usr/local/lib/codespaces-env.sh` and start Claude again.
 - **Do not read `CODESPACE_NAME` or `GITHUB_USER` from the process env** — use
   `scripts/codespace-env.mjs`. Codespaces gives these variables to VS Code
   sessions only. Other processes read them from `codespaces-env.sh`, and a
@@ -510,7 +350,7 @@ After a stop, `pnpm session <name>` restarts the codespace (~30–60 s); run
   polled correctly as its owner while `dev:up` in the same session saw an empty
   box name, printed the localhost URL, and did not share the port. The helper
   reads `/workspaces/.codespaces/shared`, which is always correct.
-- **You cannot paste images into a terminal Claude session.** Image paste reads
+- **You cannot paste images into a remote Claude session.** Image paste reads
   the clipboard of the machine where `claude` runs — the codespace, not your
   laptop. Drag the file into the VS Code explorer (or
   `gh codespace cp shot.png remote:/workspaces/n8n/`) and give Claude the

.devcontainer/codespaces/cloud-session.test.mjs

@@ -27,14 +27,6 @@ if (args[0] === 'codespace' && args[1] === 'list') {
 `,
 	);
 	chmodSync(mockGh, 0o755);
-	for (const [name, script] of Object.entries({
-		infocmp: '[ "$1" = "$SUPPORTED_TERM" ]',
-		tmux: 'printf "%s" "$TERM"',
-	})) {
-		const executable = join(fixtureDir, name);
-		writeFileSync(executable, `#!/bin/sh\n${script}\n`);
-		chmodSync(executable, 0o755);
-	}
 });
 
 after(() => rmSync(fixtureDir, { recursive: true, force: true }));
@@ -66,59 +58,19 @@ function remoteCommand(args) {
 	return invocationArgs[6];
 }
 
-for (const { name, supportedTerm, expectedTerm } of [
-	{
-		name: 'uses xterm-256color when the remote terminal definition is missing',
-		supportedTerm: 'xterm-256color',
-		expectedTerm: 'xterm-256color',
-	},
-	{
-		name: 'keeps the terminal type when the remote definition is available',
-		supportedTerm: 'xterm-ghostty',
-		expectedTerm: 'xterm-ghostty',
-	},
-]) {
-	test(name, () => {
-		const result = spawnSync('/bin/sh', ['-c', remoteCommand(['--opencode', '--legacy'])], {
-			encoding: 'utf8',
-			env: {
-				...process.env,
-				PATH: `${fixtureDir}:${process.env.PATH}`,
-				TERM: 'xterm-ghostty',
-				SUPPORTED_TERM: supportedTerm,
-			},
-		});
-
-		assert.equal(result.status, 0, result.stderr);
-		assert.equal(result.stdout, expectedTerm);
-	});
-}
-
 test('starts a named OpenCode session in a worktree', () => {
-	const command = remoteCommand(['--opencode', 'fix-flaky', '--legacy', '--model', 'test']);
+	const command = remoteCommand(['--opencode', 'fix-flaky', '--model', 'test']);
 
 	assert.match(command, /tmux new -As fix-flaky-opencode/);
 	assert.match(command, /unset AGENT_WORKER_TOKEN N8N_DEQUEUE_URL SLACK_BOT_TOKEN/);
 	assert.match(command, /OPENCODE_CONFIG_CONTENT/);
 	assert.match(command, /export N8N_AGENT_RUNTIME=sandbox; unset N8N_AGENT_PROFILE/);
-	assert.match(
-		command,
-		/git -C \/workspaces\/n8n fetch origin master; git -C \/workspaces\/n8n worktree add --no-track -b "session\/fix-flaky" "\/workspaces\/wt-fix-flaky" origin\/master/,
-	);
+	assert.match(command, /git -C \/workspaces\/n8n worktree add "\/workspaces\/wt-fix-flaky"/);
 	assert.match(command, /cd "\/workspaces\/wt-fix-flaky" && opencode --auto --model test/);
 	assert.doesNotMatch(command, /unset OPENROUTER_API_KEY/);
 	assert.doesNotMatch(command, /claude plugin/);
 });
 
-test('forwards legacy flags without a workspace name to the main checkout', () => {
-	for (const flags of [['--help'], ['--model', 'test']]) {
-		const command = remoteCommand(['--opencode', '--legacy', ...flags]);
-		assert.match(command, /tmux new -As agent-opencode/);
-		assert.ok(command.includes(`cd /workspaces/n8n && opencode --auto ${flags.join(' ')}`));
-		assert.doesNotMatch(command, /git .*worktree add/);
-	}
-});
-
 test('keeps removed credentials out of the login shell', () => {
 	const command = remoteCommand(['--shell']);
 	const profile = readFileSync(new URL('./codespaces-secrets.sh', import.meta.url), 'utf8');

.devcontainer/codespaces/codespaces-secrets.sh

@@ -12,6 +12,14 @@
 	'{"hasCompletedOnboarding":true,"theme":"dark","projects":{"/workspaces/n8n":{"hasTrustDialogAccepted":true}}}' \
 	>"$HOME/.claude.json"
 
+# Register the Flaky MCP server for Claude Code. The config keeps a literal
+# ${FLAKY_MCP_TOKEN}; Claude Code expands it at connect time, so the token is
+# not written to disk. Forks have no repo secrets and skip this.
+if [ -n "$FLAKY_MCP_URL" ] && ! grep -qs '"flaky"' "$HOME/.claude.json" && command -v claude >/dev/null 2>&1; then
+	claude mcp add --scope user --transport http flaky "$FLAKY_MCP_URL" \
+		--header 'Authorization: Bearer ${FLAKY_MCP_TOKEN}' >/dev/null 2>&1 || true
+fi
+
 # Register the credential helper in the user config, because Codespaces
 # regenerates the managed /etc/gitconfig. When the env token is missing, the
 # system helper exits 0 without output, and git falls through to ours.

.devcontainer/codespaces/devcontainer.json

@@ -18,13 +18,9 @@
 		"ghcr.io/devcontainers/features/docker-in-docker:2": {}
 	},
 	"forwardPorts": [8080, 5678],
+	// Grant the Codespace token read access to the private skills and harness repositories.
+	// Each user authorizes this access once when they create the Codespace.
 	"customizations": {
-		// Claude Code in VS Code or the browser editor. It runs in the codespace.
-		"vscode": {
-			"extensions": ["anthropic.claude-code"]
-		},
-		// Grant the Codespace token read access to the private skills and harness repositories.
-		// Each user authorizes this access once when they create the Codespace.
 		"codespaces": {
 			"repositories": {
 				"n8n-io/n8n-agent-skills": { "permissions": { "contents": "read" } },

.devcontainer/codespaces/fake-bin.mjs

@@ -1,36 +0,0 @@
-// Fake executables for tests. Each shim is a Node script. It receives `args`, `root`,
-// `file(name)` under the root, and `log(event)`, which appends to calls.jsonl in the root.
-import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
-import { tmpdir } from 'node:os';
-import { join } from 'node:path';
-
-const COMMON = `
-const fs = require('node:fs');
-const path = require('node:path');
-const args = process.argv.slice(2);
-const root = process.env.TEST_ROOT;
-const file = (name) => path.join(root, name);
-const log = (event) => fs.appendFileSync(file('calls.jsonl'), JSON.stringify(event) + '\\n');
-`;
-
-export function fakeBinaries(prefix) {
-	const root = mkdtempSync(join(tmpdir(), prefix));
-	const binDir = join(root, 'bin');
-	mkdirSync(binDir);
-	const logFile = join(root, 'calls.jsonl');
-	writeFileSync(logFile, '');
-	return {
-		root,
-		env: { PATH: `${binDir}:${process.env.PATH}`, TEST_ROOT: root },
-		bin: (name, body) =>
-			writeFileSync(join(binDir, name), `#!${process.execPath}\n${COMMON}\n${body}`, {
-				mode: 0o755,
-			}),
-		calls: () =>
-			readFileSync(logFile, 'utf8')
-				.trim()
-				.split('\n')
-				.filter(Boolean)
-				.map((line) => JSON.parse(line)),
-	};
-}

.devcontainer/codespaces/opencode-client.test.mjs

@@ -1,224 +0,0 @@
-import assert from 'node:assert/strict';
-import { spawn } from 'node:child_process';
-import { rmSync, readFileSync } from 'node:fs';
-import { test } from 'node:test';
-import { setTimeout as delay } from 'node:timers/promises';
-import { fileURLToPath } from 'node:url';
-
-import { fakeBinaries } from './fake-bin.mjs';
-import { parseOpenCodeArgs } from '../../scripts/cloud-session-opencode.mjs';
-
-const script = fileURLToPath(new URL('../../scripts/cloud-session.mjs', import.meta.url));
-const secret = 'a'.repeat(64);
-
-function fixture(t, env = {}) {
-	const { root, bin, calls, env: binEnv } = fakeBinaries('opencode-client-');
-	t.after(() => rmSync(root, { recursive: true, force: true }));
-	bin(
-		'gh',
-		`
-log({ command: 'gh', args });
-if (args[0] === 'codespace' && args[1] === 'list') {
-  console.log(JSON.stringify([
-    { name: 'dev-box', state: 'Available' },
-  ]));
-} else if (args.includes('-N')) {
-  if (process.env.TEST_TUNNEL_FAIL) process.exit(1);
-  const mapping = args[args.indexOf('-L') + 1];
-  const port = +mapping.split(':')[1];
-  const ssh = require('node:child_process').spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { stdio: 'ignore' });
-  log({ event: 'ssh-child', pid: ssh.pid });
-  const server = require('node:http').createServer((req, res) => {
-    const expected = 'Basic ' + Buffer.from('opencode:' + '${secret}').toString('base64');
-    if (process.env.TEST_UNAUTHORIZED || req.headers.authorization !== expected) { res.writeHead(401).end(); return; }
-    res.setHeader('content-type', 'application/json');
-    res.end(JSON.stringify({ healthy: true, version: process.env.TEST_SERVER_VERSION || '1.2.3' }));
-  });
-  server.listen(port, '127.0.0.1');
-  if (process.env.TEST_TUNNEL_DROP) setTimeout(() => process.exit(1), 500);
-  process.on('SIGTERM', () => { log({ event: 'tunnel-stopped' }); server.closeAllConnections(); server.close(() => process.exit(0)); });
-} else if (args.at(-1).includes('node --input-type=module')) {
-  let source = '';
-  process.stdin.on('data', chunk => source += chunk);
-  process.stdin.on('end', () => {
-    log({ event: 'bootstrap', hasSource: source.includes('prepareOpenCode') });
-    if (process.env.TEST_BOOTSTRAP_FAIL) process.exit(1);
-    const parts = args.at(-1).split(' ');
-    const name = parts.at(-3);
-    const fresh = parts.at(-2) === 'true';
-    const web = parts.at(-1) === 'true';
-    console.log(JSON.stringify({ port: 4242, password: '${secret}',
-      ...(fresh && !web ? {} : { sessionID: 'ses_saved' }),
-      directory: name === 'agent' ? '/workspaces/n8n' : '/workspaces/wt-' + name }));
-  });
-}
-`,
-	);
-	bin(
-		'opencode',
-		`
-if (args[0] === '--version') {
-  if (process.env.TEST_NO_CLIENT) process.exit(1);
-  console.log(process.env.TEST_CLIENT_VERSION || '1.2.3');
-} else {
-  log({ command: 'opencode', args, password: process.env.OPENCODE_SERVER_PASSWORD });
-  if (process.env.TEST_CLIENT_WAIT) setInterval(() => {}, 1000);
-}
-`,
-	);
-	for (const opener of ['open', 'xdg-open']) bin(opener, `log({ command: 'browser', args });`);
-	return {
-		calls,
-		start(args) {
-			const child = spawn(process.execPath, [script, '--opencode', ...args], {
-				env: { ...process.env, ...binEnv, ...env },
-				stdio: ['ignore', 'pipe', 'pipe'],
-			});
-			let output = '';
-			child.stdout.on('data', (chunk) => {
-				output += chunk;
-			});
-			child.stderr.on('data', (chunk) => {
-				output += chunk;
-			});
-			const done = new Promise((resolve) => child.on('exit', (code) => resolve({ code, output })));
-			t.after(() => child.kill('SIGKILL'));
-			return { child, done };
-		},
-	};
-}
-
-async function waitFor(check) {
-	for (let attempt = 0; attempt < 100; attempt++) {
-		const result = check();
-		if (result) return result;
-		await delay(50);
-	}
-	throw new Error('Fixture did not become ready.');
-}
-
-// An orphaned fixture process can linger as a zombie where PID 1 does not reap
-// it, and kill(pid, 0) still succeeds for one. Read the process state where the
-// kernel exposes it; without /proc only the signal error can tell.
-function stopped(pid) {
-	try {
-		process.kill(pid, 0);
-	} catch {
-		return true;
-	}
-	try {
-		return readFileSync(`/proc/${pid}/stat`, 'utf8').split(' ')[2] === 'Z';
-	} catch {
-		return false;
-	}
-}
-
-test('parses options before or after the workspace and rejects unsupported flags', () => {
-	assert.equal(parseOpenCodeArgs(['--web']).port, 4096);
-	assert.equal(parseOpenCodeArgs([]).port, 0);
-	assert.equal(parseOpenCodeArgs(['--port', '4100', '--web']).port, 4100);
-	assert.deepEqual(parseOpenCodeArgs(['--web', 'fix-flaky', '--new', '--port', '4100']), {
-		name: 'fix-flaky',
-		web: true,
-		fresh: true,
-		port: 4100,
-	});
-	for (const args of [
-		['--port'],
-		['--port', '0'],
-		['--port', '65536'],
-		['--model', 'test'],
-		['a/b'],
-		['one', 'two'],
-	]) {
-		assert.throws(() => parseOpenCodeArgs(args));
-	}
-});
-
-test(
-	'attaches the local TUI to the selected workspace and closes the whole tunnel',
-	{ timeout: 15000 },
-	async (t) => {
-		const f = fixture(t);
-		const result = await f.start(['fix-flaky']).done;
-		assert.equal(result.code, 0, result.output);
-		const calls = f.calls();
-		assert.ok(calls.find((call) => call.event === 'bootstrap').hasSource);
-		assert.ok(
-			calls
-				.filter((call) => call.command === 'gh' && call.args[1] === 'ssh')
-				.every((call) => call.args[3] === 'dev-box'),
-		);
-		const tunnel = calls.find((call) => call.command === 'gh' && call.args.includes('-N'));
-		assert.match(tunnel.args.at(-1), /^127\.0\.0\.1:\d+:127\.0\.0\.1:4242$/);
-		const client = calls.find((call) => call.command === 'opencode');
-		assert.deepEqual(client.args.slice(2), [
-			'--dir',
-			'/workspaces/wt-fix-flaky',
-			'--session',
-			'ses_saved',
-		]);
-		assert.equal(client.password, secret);
-		assert.ok(!result.output.includes(secret));
-		assert.ok(calls.some((call) => call.event === 'tunnel-stopped'));
-		const pid = calls.find((call) => call.event === 'ssh-child').pid;
-		await waitFor(() => stopped(pid));
-	},
-);
-
-test('starts a new conversation for --new instead of continuing', { timeout: 15000 }, async (t) => {
-	const f = fixture(t);
-	const result = await f.start(['fix-flaky', '--new']).done;
-	assert.equal(result.code, 0, result.output);
-	const client = f.calls().find((call) => call.command === 'opencode');
-	assert.deepEqual(client.args.slice(2), ['--dir', '/workspaces/wt-fix-flaky']);
-});
-
-for (const signal of ['SIGINT', 'SIGHUP']) {
-	test(
-		`opens the newest web conversation and cleans up on ${signal}`,
-		{ timeout: 15000 },
-		async (t) => {
-			const f = fixture(t, { TEST_NO_CLIENT: '1' });
-			const run = f.start(['--web', 'fix-flaky']);
-			const browser = await waitFor(() => f.calls().find((call) => call.command === 'browser'));
-			const url = new URL(browser.args[0]);
-			assert.equal(
-				url.pathname,
-				`/${Buffer.from('/workspaces/wt-fix-flaky').toString('base64url')}/session/ses_saved`,
-			);
-			assert.equal(url.password, '');
-			const response = await fetch(`${url.origin}/global/health`);
-			assert.equal(response.status, 200);
-			await response.text();
-			run.child.kill(signal);
-			const result = await run.done;
-			assert.equal(result.code, 0, result.output);
-			assert.ok(f.calls().some((call) => call.event === 'tunnel-stopped'));
-			await assert.rejects(fetch(url.origin));
-		},
-	);
-}
-
-for (const [label, env, message] of [
-	[
-		'a version mismatch',
-		{ TEST_SERVER_VERSION: '1.14.22' },
-		/versions differ.*npm install -g opencode-ai@1\.14\.22/,
-	],
-	['an authentication failure', { TEST_UNAUTHORIZED: '1' }, /health check failed \(401\)/],
-	['a tunnel startup failure', { TEST_TUNNEL_FAIL: '1' }, /SSH tunnel closed/],
-	['a remote preparation failure', { TEST_BOOTSTRAP_FAIL: '1' }, /Could not prepare/],
-	['a missing local client', { TEST_NO_CLIENT: '1' }, /Install OpenCode locally/],
-	['a connection loss', { TEST_CLIENT_WAIT: '1', TEST_TUNNEL_DROP: '1' }, /SSH connection closed/],
-]) {
-	test(`reports ${label} without printing credentials`, { timeout: 15000 }, async (t) => {
-		const f = fixture(t, env);
-		const result = await f.start([]).done;
-		assert.equal(result.code, 1, result.output);
-		assert.match(result.output, message);
-		assert.ok(!result.output.includes(secret));
-		if (label !== 'a connection loss')
-			assert.ok(!f.calls().some((call) => call.command === 'opencode'));
-	});
-}

.devcontainer/codespaces/opencode-proxy.test.mjs

@@ -1,147 +0,0 @@
-import assert from 'node:assert/strict';
-import { createServer, request } from 'node:http';
-import { connect } from 'node:net';
-import { test } from 'node:test';
-
-import { openCodeProxy } from '../../scripts/cloud-session-opencode-proxy.mjs';
-
-test('proxies uploads and event streams with authentication and rejects other origins', async (t) => {
-	const requests = [];
-	const backend = createServer(async (req, res) => {
-		requests.push(req.headers);
-		res.writeHead(200, { 'content-type': 'text/event-stream' });
-		for await (const chunk of req) res.write(chunk);
-		res.end();
-	});
-	await new Promise((resolve) => backend.listen(0, '127.0.0.1', resolve));
-	t.after(() => {
-		backend.closeAllConnections();
-		backend.close();
-	});
-	const proxy = await openCodeProxy({ password: 'test-secret' });
-	proxy.targetPort = backend.address().port;
-	t.after(proxy.close);
-	const response = await fetch(`${proxy.origin}/session`, {
-		method: 'POST',
-		body: 'data: attached image\n\n',
-		headers: { origin: proxy.origin },
-	});
-	assert.equal(response.status, 200);
-	assert.equal(await response.text(), 'data: attached image\n\n');
-	assert.equal(
-		requests[0].authorization,
-		`Basic ${Buffer.from('opencode:test-secret').toString('base64')}`,
-	);
-	assert.ok(!proxy.origin.includes('test-secret'));
-	for (const headers of [
-		{ origin: 'https://example.com' },
-		{ host: 'example.com' },
-		{ 'sec-fetch-site': 'cross-site' },
-	]) {
-		const status = await new Promise((resolve, reject) => {
-			const req = request(proxy.origin, { headers }, (response) => {
-				response.resume();
-				resolve(response.statusCode);
-			});
-			req.on('error', reject);
-			req.end();
-		});
-		assert.equal(status, 403);
-	}
-	assert.equal(requests.length, 1);
-});
-
-test('forwards WebSocket upgrades and closes their sockets', async (t) => {
-	const peers = new Set();
-	const backend = createServer();
-	backend.on('upgrade', (req, socket) => {
-		assert.equal(
-			req.headers.authorization,
-			`Basic ${Buffer.from('opencode:test-secret').toString('base64')}`,
-		);
-		peers.add(socket);
-		socket.on('close', () => peers.delete(socket));
-		socket.write(
-			'HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n',
-		);
-		socket.pipe(socket);
-	});
-	await new Promise((resolve) => backend.listen(0, '127.0.0.1', resolve));
-	t.after(() => {
-		for (const peer of peers) peer.destroy();
-		backend.close();
-	});
-	const proxy = await openCodeProxy({ password: 'test-secret' });
-	proxy.targetPort = backend.address().port;
-	t.after(proxy.close);
-	const socket = await new Promise((resolve, reject) => {
-		const req = request(`${proxy.origin}/pty/test/connect`, {
-			headers: { origin: proxy.origin, connection: 'Upgrade', upgrade: 'websocket' },
-		});
-		req.on('upgrade', (_response, peer) => resolve(peer));
-		req.on('error', reject);
-		req.end();
-	});
-	t.after(() => socket.destroy());
-	const echoed = new Promise((resolve) => socket.once('data', (data) => resolve(data.toString())));
-	socket.write('terminal input');
-	assert.equal(await echoed, 'terminal input');
-	const closed = new Promise((resolve) => socket.once('close', resolve));
-	proxy.close();
-	await closed;
-});
-
-test(
-	'closes the browser response when the upstream stream disconnects',
-	{ timeout: 5000 },
-	async (t) => {
-		const backend = createServer((_req, res) => {
-			res.writeHead(200, { 'content-type': 'text/event-stream' });
-			res.write('data: partial\n\n');
-			setTimeout(() => res.destroy(), 50);
-		});
-		await new Promise((resolve) => backend.listen(0, '127.0.0.1', resolve));
-		t.after(() => {
-			backend.closeAllConnections();
-			backend.close();
-		});
-		const proxy = await openCodeProxy({ password: 'test' });
-		proxy.targetPort = backend.address().port;
-		t.after(proxy.close);
-		const signal = AbortSignal.timeout(3000);
-		const response = await fetch(proxy.origin, { signal });
-		await assert.rejects(response.text());
-		assert.equal(
-			signal.aborted,
-			false,
-			'The proxy must close the response before the client times out.',
-		);
-	},
-);
-
-test('rejects an occupied browser port and reports a missing or closed upstream', async (t) => {
-	const proxy = await openCodeProxy({ password: 'test' });
-	t.after(proxy.close);
-	await assert.rejects(openCodeProxy({ password: 'test', port: +new URL(proxy.origin).port }), {
-		code: 'EADDRINUSE',
-	});
-	const waiting = await fetch(proxy.origin);
-	assert.equal(waiting.status, 503);
-	await waiting.text();
-	proxy.targetPort = 1;
-	const response = await fetch(proxy.origin);
-	assert.equal(response.status, 502);
-	await response.text();
-	const rejection = await new Promise((resolve) => {
-		const socket = connect(+new URL(proxy.origin).port, '127.0.0.1', () => {
-			socket.write(
-				'GET /pty/test/connect HTTP/1.1\r\nHost: example.com\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n',
-			);
-		});
-		socket.once('data', (data) => {
-			socket.destroy();
-			resolve(data.toString());
-		});
-	});
-	assert.match(rejection, /403 Forbidden/);
-});

.devcontainer/codespaces/opencode-server.mjs

@@ -1,228 +0,0 @@
-#!/usr/bin/env node
-// The laptop sends this file over SSH as one standalone module. Do not import sibling files.
-// It also works in an older Codespace checkout.
-import { execFileSync, spawnSync } from 'node:child_process';
-import { randomBytes } from 'node:crypto';
-import { existsSync, mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs';
-import { createServer } from 'node:net';
-import { join } from 'node:path';
-import { setTimeout as delay } from 'node:timers/promises';
-
-const quote = (value) => `'${value.replaceAll("'", "'\\''")}'`;
-
-function readJson(file) {
-	try {
-		return JSON.parse(readFileSync(file, 'utf8'));
-	} catch (error) {
-		if (error.code === 'ENOENT') return undefined;
-		throw new Error(`Cannot read ${file}. Restore or remove this state file, then retry.`);
-	}
-}
-
-function saveJson(file, value) {
-	writeFileSync(`${file}.tmp`, JSON.stringify(value), { mode: 0o600 });
-	renameSync(`${file}.tmp`, file);
-}
-
-function run(command, args, options = {}) {
-	const result = spawnSync(command, args, {
-		stdio: ['ignore', 'inherit', 'inherit'],
-		...options,
-	});
-	if (result.error || result.status !== 0) {
-		throw new Error(`${command} failed. Check the output above, then retry.`);
-	}
-}
-
-async function freePort() {
-	const server = createServer();
-	await new Promise((resolve, reject) => {
-		server.once('error', reject);
-		server.listen(0, '127.0.0.1', resolve);
-	});
-	const { port } = server.address();
-	await new Promise((resolve) => server.close(resolve));
-	return port;
-}
-
-async function request(server, path, directory, options = {}, timeout = 60_000) {
-	return await fetch(`http://127.0.0.1:${server.port}${path}`, {
-		...options,
-		headers: {
-			authorization: `Basic ${Buffer.from(`opencode:${server.password}`).toString('base64')}`,
-			'x-opencode-directory': encodeURIComponent(directory),
-			'content-type': 'application/json',
-		},
-		signal: AbortSignal.timeout(timeout),
-	});
-}
-
-async function health(server, directory) {
-	try {
-		const response = await request(server, '/global/health', directory, {}, 3000);
-		if (!response.ok) return undefined;
-		const result = await response.json();
-		return result.healthy === true && typeof result.version === 'string' ? result : undefined;
-	} catch {
-		return undefined;
-	}
-}
-
-const serverFile = (stateDir) => join(stateDir, 'server.json');
-
-async function runningServer({ stateDir, mainDirectory }) {
-	const previous = readJson(serverFile(stateDir));
-	return previous && (await health(previous, mainDirectory)) ? previous : undefined;
-}
-
-async function startServer({ stateDir, mainDirectory, workspaces }) {
-	const tmuxSession = 'n8n-opencode-server';
-	if (spawnSync('tmux', ['has-session', '-t', `=${tmuxSession}`]).status === 0) {
-		throw new Error(
-			`OpenCode is still starting or is unhealthy. Check ${stateDir}/server.log, then retry. The running server was not stopped.`,
-		);
-	}
-
-	const server = { port: await freePort(), password: randomBytes(32).toString('hex') };
-	const launcher = join(stateDir, 'serve.sh');
-	// Read secrets when the server starts. Do not store provider keys in the launcher.
-	writeFileSync(
-		launcher,
-		[
-			'#!/bin/bash',
-			'. /usr/local/lib/codespaces-env.sh 2>/dev/null || true',
-			'unset AGENT_WORKER_TOKEN N8N_DEQUEUE_URL SLACK_BOT_TOKEN',
-			`export TURBO_CACHE_DIR=${quote(join(workspaces, '.turbo-cache'))}`,
-			`[ -d "$TURBO_CACHE_DIR" ] || cp -r ${quote(join(mainDirectory, '.turbo/cache'))} "$TURBO_CACHE_DIR" 2>/dev/null || mkdir -p "$TURBO_CACHE_DIR"`,
-			`export OPENCODE_SERVER_USERNAME=opencode OPENCODE_SERVER_PASSWORD=${quote(server.password)}`,
-			'export OPENCODE_EXPERIMENTAL_CODE_MODE=true',
-			`export OPENCODE_CONFIG_CONTENT=${quote(
-				JSON.stringify({
-					enabled_providers: ['openrouter'],
-					provider: { openrouter: { options: { apiKey: '{env:OPENROUTER_API_KEY}' } } },
-				}),
-			)}`,
-			// Interactive sessions use the harness sandbox runtime without a profile.
-			'export N8N_AGENT_RUNTIME=sandbox; unset N8N_AGENT_PROFILE',
-			`cd ${quote(mainDirectory)} || exit 1`,
-			`exec opencode serve --hostname 127.0.0.1 --port ${server.port} >> ${quote(join(stateDir, 'server.log'))} 2>&1`,
-			'',
-		].join('\n'),
-		{ mode: 0o600 },
-	);
-	saveJson(serverFile(stateDir), server);
-	run('tmux', ['new-session', '-d', '-s', tmuxSession, `bash ${quote(launcher)}`]);
-	console.error('Starting the OpenCode server…');
-	for (let attempt = 0; attempt < 60; attempt++) {
-		if (await health(server, mainDirectory)) return server;
-		await delay(500);
-	}
-	throw new Error(`OpenCode did not become ready. Check ${stateDir}/server.log.`);
-}
-
-function prepareWorkspace({ name, directory, mainDirectory, stateDir }) {
-	if (!existsSync(directory)) {
-		console.error(`Creating worktree ${directory}…`);
-		const branch = `session/${name}`;
-		const exists =
-			spawnSync('git', [
-				'-C',
-				mainDirectory,
-				'show-ref',
-				'--verify',
-				'--quiet',
-				`refs/heads/${branch}`,
-			]).status === 0;
-		if (!exists)
-			spawnSync('git', ['-C', mainDirectory, 'fetch', 'origin', 'master'], {
-				stdio: ['ignore', 2, 2],
-			});
-		run(
-			'git',
-			[
-				'-C',
-				mainDirectory,
-				'worktree',
-				'add',
-				...(exists ? [] : ['--no-track', '-b', branch]),
-				directory,
-				exists ? branch : 'origin/master',
-			],
-			{ stdio: ['ignore', 2, 2] },
-		);
-	}
-	// A failed install must be retried even if it left a node_modules directory.
-	const installed = join(stateDir, `${name}.installed`);
-	const installing = join(stateDir, `${name}.installing`);
-	if (
-		!existsSync(join(directory, 'node_modules')) ||
-		existsSync(installing) ||
-		(name !== 'agent' && !existsSync(installed))
-	) {
-		console.error(`Installing dependencies in ${directory}…`);
-		writeFileSync(installing, '', { mode: 0o600 });
-		run('pnpm', ['install'], { cwd: directory, stdio: ['ignore', 2, 2] });
-		writeFileSync(installed, '', { mode: 0o600 });
-		rmSync(installing);
-	}
-}
-
-// OpenCode's own resume is scoped to the repository project, so all worktrees
-// share one pool. Scope the selection to the workspace directory instead, so
-// each worktree resumes its own most recently updated root conversation.
-async function resolveSession({ name, fresh, directory, server }) {
-	if (!fresh) {
-		const query = new URLSearchParams({ directory, roots: 'true' });
-		const response = await request(server, `/session?${query}`, directory);
-		if (!response.ok) throw new Error(`Cannot list OpenCode sessions (${response.status}).`);
-		const sessions = await response.json();
-		const latest = Array.isArray(sessions)
-			? sessions.find((entry) => typeof entry?.id === 'string' && entry.id.length > 0)
-			: undefined;
-		if (latest) return latest.id;
-	}
-	const response = await request(server, '/session', directory, {
-		method: 'POST',
-		body: JSON.stringify({ title: `n8n: ${name}` }),
-	});
-	if (!response.ok) throw new Error(`Cannot create OpenCode session (${response.status}).`);
-	const session = await response.json();
-	if (typeof session.id !== 'string' || session.directory !== directory)
-		throw new Error('OpenCode returned an invalid session.');
-	return session.id;
-}
-
-export async function prepareOpenCode({
-	name = 'agent',
-	fresh = false,
-	web = false,
-	workspaces = '/workspaces',
-} = {}) {
-	const stateDir = join(workspaces, '.n8n-opencode');
-	const mainDirectory = join(workspaces, 'n8n');
-	if (!/^\w[\w-]*$/.test(name)) throw new Error('Invalid OpenCode workspace name.');
-	mkdirSync(stateDir, { recursive: true, mode: 0o700 });
-	const directory = name === 'agent' ? mainDirectory : join(workspaces, `wt-${name}`);
-	let server = await runningServer({ stateDir, mainDirectory });
-	// On a cold start, fail before the worktree install when OpenCode is missing on the Codespace.
-	if (!server) execFileSync('opencode', ['--version'], { stdio: ['ignore', 'pipe', 'inherit'] });
-	prepareWorkspace({ name, directory, mainDirectory, stateDir });
-	server ??= await startServer({ stateDir, mainDirectory, workspaces });
-	const state = { ...server, directory };
-	// A fresh TUI conversation needs no record. The TUI creates it on the first message.
-	if (fresh && !web) return state;
-	// Only the parent process reads stdout. Never send this record to terminal output.
-	return { ...state, sessionID: await resolveSession({ name, fresh, directory, server }) };
-}
-
-if (process.argv[1] === '-') {
-	try {
-		const [name, fresh, web] = process.argv.slice(2);
-		console.log(
-			JSON.stringify(await prepareOpenCode({ name, fresh: fresh === 'true', web: web === 'true' })),
-		);
-	} catch (error) {
-		console.error(error.message);
-		process.exitCode = 1;
-	}
-}

.devcontainer/codespaces/opencode-server.test.mjs

@@ -1,275 +0,0 @@
-import assert from 'node:assert/strict';
-import { mkdirSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs';
-import { join } from 'node:path';
-import { test } from 'node:test';
-import { setTimeout as delay } from 'node:timers/promises';
-
-import { fakeBinaries } from './fake-bin.mjs';
-import { prepareOpenCode } from './opencode-server.mjs';
-
-// An orphaned fixture process can linger as a zombie where PID 1 does not reap
-// it, and kill(pid, 0) still succeeds for one. Read the process state where the
-// kernel exposes it; without /proc only the signal error can tell.
-function stopped(pid) {
-	try {
-		process.kill(pid, 0);
-	} catch {
-		return true;
-	}
-	try {
-		return readFileSync(`/proc/${pid}/stat`, 'utf8').split(' ')[2] === 'Z';
-	} catch {
-		return false;
-	}
-}
-
-function fixture(t) {
-	const fake = fakeBinaries('opencode-server-');
-	const dir = fake.root;
-	mkdirSync(join(dir, 'n8n', 'node_modules'), { recursive: true });
-	const savedEnv = { ...process.env };
-	// prepareOpenCode spawns the shims in this process. Log every call before the shim body runs.
-	Object.assign(process.env, fake.env, {
-		AGENT_WORKER_TOKEN: 'test-worker',
-		N8N_DEQUEUE_URL: 'test-queue',
-		SLACK_BOT_TOKEN: 'test-slack',
-	});
-	const bin = (name, body) =>
-		fake.bin(name, `log({ command: ${JSON.stringify(name)}, args });\n${body}`);
-	bin(
-		'git',
-		`
-if (args.includes('show-ref')) process.exit(fs.existsSync(file('branch-exists')) ? 0 : 1);
-if (args.includes('fetch')) process.exit(0);
-fs.mkdirSync(args.at(-2), { recursive: true });
-`,
-	);
-	bin(
-		'pnpm',
-		`
-fs.mkdirSync(path.join(process.cwd(), 'node_modules'), {recursive:true});
-if (process.env.TEST_INSTALL_FAIL) process.exit(1);
-`,
-	);
-	bin(
-		'tmux',
-		`
-if (args[0] === 'has-session') {
-  const pid = +fs.readFileSync(file('pid'), 'utf8');
-  let gone = false;
-  try { process.kill(pid, 0); } catch { gone = true; }
-  if (!gone) {
-    try { gone = fs.readFileSync('/proc/' + pid + '/stat', 'utf8').split(' ')[2] === 'Z'; } catch {}
-  }
-  if (gone) process.exit(1);
-} else {
-  const child = require('node:child_process').spawn('bash', ['-c', args.at(-1)], { detached:true, stdio:'ignore' });
-  fs.writeFileSync(file('pid'), String(child.pid)); child.unref();
-}
-`,
-	);
-	bin(
-		'opencode',
-		`
-if (args[0] === '--version') { console.log('1.2.3'); process.exit(0); }
-fs.writeFileSync(file('server-env.json'), JSON.stringify({
-  worker: !!process.env.AGENT_WORKER_TOKEN, queue: !!process.env.N8N_DEQUEUE_URL, slack: !!process.env.SLACK_BOT_TOKEN,
-  cache: process.env.TURBO_CACHE_DIR, config: JSON.parse(process.env.OPENCODE_CONFIG_CONTENT),
-  runtime: process.env.N8N_AGENT_RUNTIME, profile: process.env.N8N_AGENT_PROFILE ?? null,
-}));
-let sessions = fs.existsSync(file('sessions.json')) ? JSON.parse(fs.readFileSync(file('sessions.json'), 'utf8')) : {};
-const server = require('node:http').createServer(async (req, res) => {
-  res.setHeader('content-type', 'application/json');
-  const expected = 'Basic ' + Buffer.from('opencode:' + process.env.OPENCODE_SERVER_PASSWORD).toString('base64');
-  if (req.headers.authorization !== expected) { res.writeHead(401).end('{}'); return; }
-  const directory = decodeURIComponent(req.headers['x-opencode-directory']);
-  if (req.url === '/global/health') { res.end(JSON.stringify({ healthy:true, version:'1.2.3' })); return; }
-  if (req.method === 'POST' && req.url === '/session') {
-    if (fs.existsSync(file('reject-create'))) { res.writeHead(503).end('{}'); return; }
-    let body = '';
-    for await (const chunk of req) body += chunk;
-    const id = 'ses_' + (Object.keys(sessions).length + 1);
-    sessions[id] = { id, directory, title: JSON.parse(body).title };
-    fs.writeFileSync(file('sessions.json'), JSON.stringify(sessions));
-    res.end(JSON.stringify(sessions[id])); return;
-  }
-  if (fs.existsSync(file('reject-session'))) { res.writeHead(503).end('{}'); return; }
-  const url = new URL(req.url, 'http://localhost');
-  if (url.pathname === '/session') {
-    // Newest first, like the real server. Filter on the query, not the header,
-    // so the test fails when the launcher omits either parameter. The
-    // child-session file models a subagent conversation that the roots query
-    // flag must exclude.
-    const directory = url.searchParams.get('directory');
-    const roots = url.searchParams.get('roots') === 'true';
-    const stored = Object.values(sessions);
-    if (fs.existsSync(file('child-session')))
-      stored.push({ id: 'ses_child', directory, title: 'child', parentID: 'ses_1' });
-    const list = stored
-      .filter(s => s.directory === directory && (!roots || !s.parentID))
-      .reverse();
-    if (fs.existsSync(file('extra-session'))) list.unshift({ id: 'ses_extra', directory, title: 'TUI session' });
-    res.end(JSON.stringify(list)); return;
-  }
-  const session = sessions[req.url.split('/').at(-1)];
-  if (!session) { res.writeHead(404).end('{}'); return; }
-  res.end(JSON.stringify(session));
-});
-server.listen(+args[args.indexOf('--port') + 1], '127.0.0.1');
-`,
-	);
-	const stop = async () => {
-		try {
-			const pid = +readFileSync(join(dir, 'pid'), 'utf8');
-			process.kill(-pid, 'SIGTERM');
-			for (let count = 0; count < 100; count++) {
-				if (stopped(pid)) return;
-				await delay(20);
-			}
-			throw new Error('Fixture server did not stop.');
-		} catch (error) {
-			if (!['ENOENT', 'ESRCH'].includes(error.code)) throw error;
-		}
-	};
-	t.after(async () => {
-		await stop();
-		for (const key of Object.keys(process.env)) if (!(key in savedEnv)) delete process.env[key];
-		Object.assign(process.env, savedEnv);
-		rmSync(dir, { recursive: true, force: true });
-	});
-	return {
-		dir,
-		stop,
-		prepare: (options = {}) => prepareOpenCode({ workspaces: dir, ...options }),
-		commands: fake.calls,
-	};
-}
-
-test(
-	'reuses the server, serves each workspace its newest conversation, and resumes after restart',
-	{ timeout: 15000 },
-	async (t) => {
-		const f = fixture(t);
-		const first = await f.prepare({ name: 'fix-flaky' });
-		assert.equal(first.directory, join(f.dir, 'wt-fix-flaky'));
-		const git = f.commands().filter((entry) => entry.command === 'git');
-		assert.deepEqual(
-			git.map((entry) => entry.args.slice(2)),
-			[
-				['show-ref', '--verify', '--quiet', 'refs/heads/session/fix-flaky'],
-				['fetch', 'origin', 'master'],
-				[
-					'worktree',
-					'add',
-					'--no-track',
-					'-b',
-					'session/fix-flaky',
-					first.directory,
-					'origin/master',
-				],
-			],
-		);
-		assert.equal(first.sessionID, 'ses_1');
-		assert.deepEqual(await f.prepare({ name: 'fix-flaky' }), first);
-		const second = await f.prepare({ name: 'another-task' });
-		assert.equal(second.port, first.port);
-		assert.equal(second.sessionID, 'ses_2');
-		const webbed = await f.prepare({ name: 'fix-flaky', web: true });
-		assert.equal(webbed.sessionID, 'ses_1');
-		const created = JSON.parse(readFileSync(join(f.dir, 'sessions.json'), 'utf8'));
-		assert.equal(created.ses_1.title, 'n8n: fix-flaky');
-		assert.equal(created.ses_1.directory, join(f.dir, 'wt-fix-flaky'));
-		assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, 'ses_1');
-		assert.equal((await f.prepare({ name: 'another-task', web: true })).sessionID, 'ses_2');
-		writeFileSync(join(f.dir, 'extra-session'), '');
-		assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, 'ses_extra');
-		rmSync(join(f.dir, 'extra-session'));
-		writeFileSync(join(f.dir, 'child-session'), '');
-		assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, 'ses_1');
-		rmSync(join(f.dir, 'child-session'));
-		const fresh = await f.prepare({ name: 'fix-flaky', web: true, fresh: true });
-		assert.notEqual(fresh.sessionID, 'ses_1');
-		assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, fresh.sessionID);
-		const env = JSON.parse(readFileSync(join(f.dir, 'server-env.json'), 'utf8'));
-		assert.deepEqual([env.worker, env.queue, env.slack], [false, false, false]);
-		assert.equal(env.cache, join(f.dir, '.turbo-cache'));
-		assert.equal(env.config.provider.openrouter.options.apiKey, '{env:OPENROUTER_API_KEY}');
-		assert.deepEqual(env.config.enabled_providers, ['openrouter']);
-		assert.deepEqual([env.runtime, env.profile], ['sandbox', null]);
-		assert.equal(statSync(join(f.dir, '.n8n-opencode')).mode & 0o777, 0o700);
-		for (const file of ['serve.sh', 'server.json']) {
-			assert.equal(statSync(join(f.dir, '.n8n-opencode', file)).mode & 0o777, 0o600);
-		}
-		assert.ok(!f.commands().some((entry) => JSON.stringify(entry.args).includes(first.password)));
-		assert.equal(f.commands().filter((entry) => entry.command === 'pnpm').length, 2);
-		await f.stop();
-		const restarted = await f.prepare({ name: 'fix-flaky', web: true });
-		assert.equal(restarted.sessionID, fresh.sessionID);
-		assert.notEqual(restarted.password, first.password);
-		rmSync(first.directory, { recursive: true });
-		writeFileSync(join(f.dir, 'branch-exists'), '');
-		assert.equal((await f.prepare({ name: 'fix-flaky' })).directory, join(f.dir, 'wt-fix-flaky'));
-		const worktree = f
-			.commands()
-			.filter((entry) => entry.command === 'git' && entry.args.includes('add'))
-			.at(-1);
-		assert.deepEqual(worktree.args, [
-			'-C',
-			join(f.dir, 'n8n'),
-			'worktree',
-			'add',
-			first.directory,
-			'session/fix-flaky',
-		]);
-	},
-);
-
-test(
-	'retries a failed worktree install before starting the server',
-	{ timeout: 10000 },
-	async (t) => {
-		const f = fixture(t);
-		process.env.TEST_INSTALL_FAIL = '1';
-		await assert.rejects(f.prepare({ name: 'retry' }), /pnpm failed/);
-		assert.ok(!f.commands().some((entry) => entry.command === 'tmux'));
-		delete process.env.TEST_INSTALL_FAIL;
-		assert.equal((await f.prepare({ name: 'retry' })).directory, join(f.dir, 'wt-retry'));
-		assert.equal(f.commands().filter((entry) => entry.command === 'pnpm').length, 2);
-	},
-);
-
-test(
-	'retries a failed main checkout install even when node_modules exists',
-	{ timeout: 10000 },
-	async (t) => {
-		const f = fixture(t);
-		rmSync(join(f.dir, 'n8n', 'node_modules'), { recursive: true });
-		process.env.TEST_INSTALL_FAIL = '1';
-		await assert.rejects(f.prepare(), /pnpm failed/);
-		delete process.env.TEST_INSTALL_FAIL;
-		assert.equal((await f.prepare()).directory, join(f.dir, 'n8n'));
-		assert.equal(f.commands().filter((entry) => entry.command === 'pnpm').length, 2);
-	},
-);
-
-test(
-	'reports session API errors and creates conversations only when none exist',
-	{ timeout: 10000 },
-	async (t) => {
-		const f = fixture(t);
-		const first = await f.prepare({ web: true });
-		assert.equal(first.sessionID, 'ses_1');
-		writeFileSync(join(f.dir, 'reject-session'), '');
-		await assert.rejects(f.prepare({ web: true }), /Cannot list OpenCode sessions \(503\)/);
-		assert.ok(JSON.parse(readFileSync(join(f.dir, 'sessions.json'), 'utf8')).ses_1);
-		rmSync(join(f.dir, 'reject-session'));
-		assert.equal((await f.prepare({ web: true })).sessionID, first.sessionID);
-		writeFileSync(join(f.dir, 'reject-create'), '');
-		await assert.rejects(
-			f.prepare({ name: 'fresh-task', web: true }),
-			/Cannot create OpenCode session \(503\)/,
-		);
-		rmSync(join(f.dir, 'reject-create'));
-		assert.equal((await f.prepare({ name: 'fresh-task', web: true })).sessionID, 'ses_2');
-	},
-);

.devcontainer/codespaces/post-start.mjs

@@ -1,11 +1,10 @@
 #!/usr/bin/env node
 // Runs on each Codespace start. Installs the skills and harness, then starts the worker.
 import { execFileSync } from 'node:child_process';
-import { readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { rmSync, writeFileSync } from 'node:fs';
 import { homedir } from 'node:os';
 import { join } from 'node:path';
 import { installAgentHarness } from '../../scripts/agent-harness.mjs';
-import { codespaceSecret } from '../../scripts/codespace-env.mjs';
 import { MARKETPLACE, PLUGINS } from './plugins.mjs';
 
 const STATUS_FILE = '/tmp/post-start-status.json';
@@ -50,42 +49,6 @@ function addMarketplace() {
 	return add('marketplace add (retry)');
 }
 
-// Claude Code reads the Flaky token when it connects. An env reference such as
-// ${FLAKY_MCP_TOKEN} fails in processes that did not load the secrets file, for
-// example the desktop app's SSH server. The helper reads the file on each connect,
-// so it also gets a rotated token. The token is not copied into ~/.claude.json.
-const FLAKY_HEADERS_HELPER =
-	'. /usr/local/lib/codespaces-env.sh; printf %s "{\\"Authorization\\":\\"Bearer $FLAKY_MCP_TOKEN\\"}"';
-
-function registerFlakyMcp() {
-	// Forks have no repository secrets.
-	const url = codespaceSecret('FLAKY_MCP_URL');
-	if (!url || !codespaceSecret('FLAKY_MCP_TOKEN')) return;
-
-	let current;
-	try {
-		current = JSON.parse(readFileSync(join(homedir(), '.claude.json'), 'utf8')).mcpServers?.flaky;
-	} catch {
-		// No readable config yet: register the server below.
-	}
-	if (current?.url === url && current.headersHelper === FLAKY_HEADERS_HELPER && !current.headers) {
-		return;
-	}
-
-	if (current) tryRun('flaky mcp remove', 'claude', ['mcp', 'remove', '--scope', 'user', 'flaky']);
-	const config = { type: 'http', url, headersHelper: FLAKY_HEADERS_HELPER };
-	tryRun('flaky mcp add', 'claude', [
-		'mcp',
-		'add-json',
-		'--scope',
-		'user',
-		'flaky',
-		JSON.stringify(config),
-	]);
-}
-
-registerFlakyMcp();
-
 tryRun('skills repo reachable', 'git', ['ls-remote', `https://github.com/${MARKETPLACE}`, 'HEAD']);
 
 const installed = [];

.devcontainer/codespaces/ssh-config.test.mjs

@@ -1,32 +0,0 @@
-import assert from 'node:assert/strict';
-import { test } from 'node:test';
-
-import { aliasHostConfig, withInclude } from '../../scripts/cloud-session-ssh-config.mjs';
-
-const GH_CONFIG = `Host cs.ominous-doodle.master
-	User node
-	ProxyCommand gh cs ssh -c ominous-doodle --stdio
-	UserKnownHostsFile=/dev/null
-`;
-
-test('replaces the gh host name with the fixed alias', () => {
-	assert.equal(
-		aliasHostConfig(GH_CONFIG),
-		GH_CONFIG.replace('cs.ominous-doodle.master', 'n8n-codespace'),
-	);
-});
-
-test('rejects gh output without exactly one proxied host', () => {
-	assert.throws(() => aliasHostConfig(''));
-	assert.throws(() => aliasHostConfig(`${GH_CONFIG}Host other\n`));
-	assert.throws(() => aliasHostConfig('Host cs.x\n\tUser node\n'));
-});
-
-test('adds the Include line at the top once', () => {
-	const include = 'Include ~/.ssh/n8n-codespace.conf';
-	assert.equal(withInclude(''), `${include}\n`);
-
-	const updated = withInclude('Host github.com\n\tUser git\n');
-	assert.equal(updated, `${include}\n\nHost github.com\n\tUser git\n`);
-	assert.equal(withInclude(updated), undefined);
-});

.dockerignore

@@ -26,7 +26,7 @@
 !docker/images/runners
 !docker/images/runners/n8n-task-runners.json
 
-# === benchmark image (packages/quality/efficiency/scale/benchmark/Dockerfile) ===
+# === benchmark image (packages/@n8n/benchmark/Dockerfile) ===
 !package.json
 !pnpm-lock.yaml
 !pnpm-workspace.yaml
@@ -34,10 +34,8 @@
 !patches/**
 !scripts
 !scripts/**
-!packages/quality
-!packages/quality/efficiency
-!packages/quality/efficiency/scale
-!packages/quality/efficiency/scale/benchmark
-!packages/quality/efficiency/scale/benchmark/**
+!packages/@n8n/benchmark
+!packages/@n8n/benchmark/**
 !packages/@n8n/typescript-config
 !packages/@n8n/typescript-config/**
+

.env.ai-surfaces.example

@@ -1,75 +0,0 @@
-# Example env for comparing AI workflow builders (Instance AI, Claude Code + n8n MCP,
-# Codex + n8n MCP) on a local instance.
-#
-# Copy to `.env.ai-surfaces` (gitignored) and fill in the values that you need.
-# Never commit keys. The testmax-n8n-ai skill reads the files in this
-# order, from low to high priority: .env.local, .env.ai-surfaces, the shell
-# environment. Empty values are ignored, so keys that are already in .env.local
-# work without a change. Set AI_SURFACES_ENV to use a file in a different place.
-#
-# See .agents/skills/testmax-n8n-ai/SKILL.md for how the values are used.
-
-# --- n8n ---------------------------------------------------------------------
-# Empty values use .env.local, then the defaults: http://localhost:5678,
-# nathan@n8n.io and PlaywrightTest123.
-N8N_BASE_URL=
-N8N_EVAL_EMAIL=
-N8N_EVAL_PASSWORD=
-# Optional. The MCP key of the user above (Settings > MCP). When set, the skill
-# does not need the cached token and never asks to rotate the key.
-N8N_MCP_TOKEN=
-
-# --- Run defaults --------------------------------------------------------------
-COMPARE_BUILDERS=iai,claude,codex
-# container (Docker) or clean (Claude only, no Docker)
-COMPARE_ISOLATION=container
-COMPARE_RUNS_DIR=/tmp/builder-compare
-# 1 runs the builders at the same time. 0 runs them one at a time.
-COMPARE_PARALLEL=1
-# 1 opens a local page that shows the runs live and then the report. 0 turns it off.
-COMPARE_LIVE=1
-# 1 also opens the Instance AI thread in the browser and a terminal window for
-# each CLI builder.
-COMPARE_LIVE_WINDOWS=0
-# 1 records a video of the Instance AI thread for the replay on the page.
-COMPARE_RECORD=1
-# interactive records the real terminal screen of Claude Code and Codex.
-# headless uses `claude -p` and `codex exec` and has no screen recording.
-COMPARE_CLI_MODE=interactive
-
-# --- Claude Code + n8n MCP -----------------------------------------------------
-# Empty model uses the Claude Code default.
-CLAUDE_MODEL=
-# none: Claude Code has only the n8n tools. default: it also has its built-in tools,
-# such as Bash, Read, Write and WebFetch, and uses all of them without a question,
-# like Codex in the same container. Clean isolation always uses none.
-COMPARE_CLAUDE_TOOLS=none
-# 1 runs each created workflow with pinned data through the n8n MCP server, once
-# for each scenario in scenarios.json. 0 turns it off.
-COMPARE_SCENARIOS=1
-# 1 runs each created workflow in process with recorded API fixtures, for the
-# scenarios, edge cases and questions in scenarios.json. 0 turns it off.
-COMPARE_WIRE=1
-# 1 runs the deterministic binary checks of the Instance AI eval suite.
-COMPARE_BINARY_CHECKS=1
-# Model keys for the real AI runs of the questions in scenarios.json. A workflow
-# model without a key skips those runs. Set the key of each model provider that
-# the builders can pick: GOOGLE_GENERATIVE_AI_API_KEY, OPENAI_API_KEY,
-# ANTHROPIC_API_KEY, MISTRAL_API_KEY or GROQ_API_KEY.
-GOOGLE_GENERATIVE_AI_API_KEY=
-# Falls back to ANTHROPIC_API_KEY.
-CLAUDE_API_KEY=
-
-# --- Codex + n8n MCP -----------------------------------------------------------
-# Empty model uses the Codex default.
-CODEX_MODEL=
-# Falls back to OPENAI_API_KEY.
-CODEX_API_KEY=
-
-# --- Instance AI cost (LangSmith) ----------------------------------------------
-# The n8n instance must trace to the same project (LANGSMITH_TRACING=true).
-LANGSMITH_API_KEY=
-# Use the same region as the instance. Empty uses https://api.smith.langchain.com.
-# For the EU region, set https://eu.api.smith.langchain.com.
-LANGSMITH_ENDPOINT=
-LANGSMITH_PROJECT=instance-ai

.githooks/check-pnpm.mjs

@@ -1,36 +0,0 @@
-#!/usr/bin/env node
-// Git hook helper (post-merge, post-rewrite, post-checkout): warn when the
-// installed pnpm major differs from the one pinned in package.json.
-// Usage: node .githooks/check-pnpm.mjs [checkout-type]
-// For post-checkout, pass lefthook's {3}: "0" is a file checkout, which we skip.
-import { spawnSync } from 'node:child_process';
-import { readFileSync } from 'node:fs';
-import { homedir } from 'node:os';
-import { fileURLToPath } from 'node:url';
-
-if (process.argv[2] === '0') process.exit(0);
-
-const packageJsonPath = fileURLToPath(new URL('../package.json', import.meta.url));
-const { packageManager = '' } = JSON.parse(readFileSync(packageJsonPath, 'utf8'));
-const wanted = /^pnpm@(\d+)/.exec(packageManager)?.[1];
-
-if (!wanted) process.exit(0);
-
-// Run from the home dir so pnpm reports the installed version,
-// not the version pinned by this repo.
-// `shell` lets Windows resolve pnpm.cmd. The command has no user input.
-const result = spawnSync('pnpm --version', { cwd: homedir(), shell: true, encoding: 'utf8' });
-const current = result.status === 0 ? result.stdout.trim().split('.')[0] : '';
-if (wanted === current) process.exit(0);
-
-const installCommand =
-	process.platform === 'win32'
-		? 'powershell -c "irm https://get.pnpm.io/install.ps1 | iex"'
-		: 'curl -fsSL https://get.pnpm.io/install.sh | sh -';
-
-console.log(`
-⚠ This repo now pins pnpm ${wanted} (you have ${current || 'none'}).
-  pnpm should auto-download it on the next install command.
-  If that fails (some pnpm 10 versions can't install v12), reinstall:
-    ${installCommand}
-`);

.githooks/check-pnpm.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+# .githooks/check-pnpm — symlink or call from post-merge, post-rewrite, post-checkout
+
+WANTED=$(sed -n 's/.*"packageManager": *"pnpm@\([0-9]*\).*/\1/p' package.json)
+CURRENT=$(cd ~ && pnpm --version 2>/dev/null | cut -d. -f1)
+[ -z "$WANTED" ] || [ "$WANTED" = "$CURRENT" ] && exit 0
+echo ""
+echo "⚠ This repo now pins pnpm $WANTED (you have ${CURRENT:-none})."
+echo "  pnpm should auto-download it on the next install command."
+echo "  If that fails (some pnpm 10 versions can't install v12), reinstall:"
+echo "    curl -fsSL https://get.pnpm.io/install.sh | sh -"
+echo ""
+exit 0

.github/CI-TELEMETRY.md

@@ -81,12 +81,11 @@ return 'blacksmith';
 
 | Telemetry | Source | Metrics |
 |-----------|--------|---------|
-| Playwright perf/benchmark | `packages/quality/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
-| Accessibility buckets | `packages/quality/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
+| Playwright perf/benchmark | `packages/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
+| Accessibility buckets | `packages/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
 | Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
 | Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
-| Dive image efficiency | `packages/quality/efficiency/images/image-efficiency.spec.ts` | Layer content size, estimated waste, efficiency score, and layer count through the shared Playwright reporter |
-| Container stack | `packages/quality/environments/containers/telemetry.ts` | E2E stack startup times per service |
+| Container stack | `packages/testing/containers/telemetry.ts` | E2E stack startup times per service |
 
 ## Secrets
 

.github/CODEOWNERS

@@ -0,0 +1,11 @@
+# Temporary: kept side by side with the top-level OWNERS file while the
+# "Required Reviews" enforcement is on trial. Do not add entries here;
+# add them to OWNERS. See "Transition from CODEOWNERS" in WORKFLOWS.md.
+packages/@n8n/db/src/migrations/ @n8n-io/migrations-review
+/OWNERS @n8n-io/qa-dx
+.github/workflows @n8n-io/qa-dx
+.github/scripts @n8n-io/qa-dx
+.github/actions @n8n-io/qa-dx
+.github/poutine-rules @n8n-io/qa-dx
+docker/get-n8n.sh @n8n-io/qa-dx
+.code-health-baseline.json @n8n-io/qa-dx

.github/WORKFLOWS.md

@@ -10,6 +10,7 @@ Complete reference for n8n's `.github/` folder.
 .github/
 ├── WORKFLOWS.md                          # This document
 ├── CI-TELEMETRY.md                       # Telemetry & metrics guide
+├── CODEOWNERS                            # Temporary, side by side with OWNERS during the trial
 ├── pull_request_template.md              # PR description template
 ├── pull_request_title_conventions.md     # Title format rules (Angular)
 ├── actionlint.yml                        # Workflow linter config
@@ -177,7 +178,7 @@ These only run if specific files changed:
 | `docker/images/n8n-base/Dockerfile`                                    | `build-base-image.yml`      | any        |
 | `**/package.json`, `**/turbo.json`                                     | `build-windows.yml`         | master     |
 | `packages/@n8n/ai-workflow-builder.ee/evaluations/programmatic/python/**` | `test-evals-python.yml`  | any        |
-| `packages/quality/efficiency/scale/benchmark/**`                       | `build-benchmark-image.yml` | master     |
+| `packages/@n8n/benchmark/**`                                           | `build-benchmark-image.yml` | master     |
 | `packages/cli/src/public-api/**/*.yml`, `packages/cli/src/public-api/**/*.yaml`, `packages/cli/src/public-api/**/*.css`, `packages/cli/src/public-api/v1/openapi-gen/**/*.ts`, `packages/cli/scripts/build.mjs`, `packages/cli/package.json` | `util-publish-api-schema.yml` | master   |
 | `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts`, `packages/@n8n/agents/src/**` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
 | `docker/get-n8n.sh`, `docker/get-n8n-compose.yml`, `docker/test-get-n8n.sh` | `test-get-n8n.yml`          | any        |
@@ -236,6 +237,14 @@ parallelism). See the `--build-via-mcp` section in
 | PR merged to `release/*`           | `release-publish.yml`          |
 | PR closed with `codespace-preview` | `util-codespace-preview.yml`   |
 
+### Manual Triggers (PR Comments)
+
+| Command            | Workflow                     | Permissions         |
+|--------------------|------------------------------|---------------------|
+| `/test-workflows`  | `test-workflows-callable.yml`| admin/write/maintain|
+
+**Why:** Re-run tests without pushing commits. Useful for flaky test investigation.
+
 ### Label Triggers
 
 | Label                 | Workflow                     | Effect                                             |
@@ -411,9 +420,7 @@ Run-workflow dropdown, which only a user with write access can pick.
 |-----------------------------|---------------------------------------------------------|
 | `util-codespace-preview.yml`| Wake, re-serve or delete a PR preview instance by hand   |
 | `util-data-tooling.yml`     | SQLite/PostgreSQL export/import validation (manual)     |
-| `test-image-efficiency.yml` | Measure n8n and runners image size and layer waste with Dive |
 | `util-probe-registry.yml`   | Diagnose slow npm metadata fetches (temporary)          |
-| `sec-sync-bundle-branches.yml` | Sync both bundle branches on demand, including before a bundle cut |
 
 ---
 
@@ -447,6 +454,10 @@ test-sbom-nightly.yml
 
 test-workflows-nightly.yml  (manual dispatch only — nightly schedule disabled, DEVP-544)
     └──────────────────────────▶  test-workflows-callable.yml
+
+PR Comment Dispatchers (triggered by /command in PR comments):
+test-workflows-pr-comment.yml
+    └──────────────────────────▶  test-workflows-callable.yml
 ```
 
 ---
@@ -564,15 +575,14 @@ out: `npm deprecate n8n@X.Y.Z "Failed release, use X.Y.(Z+1)"`.
 
 ## ci-master.yml
 
-Runs on push to `master`:
+Runs on push to `master` or `1.x`:
 
 ```
-Push to master
-├─ build-and-format (Blacksmith: build, then format check; populate master cache)
-├─ unit-test (matrix: Node 24.18.1, 26.5.1)
+Push to master/1.x
+├─ build-github (populate cache)
+├─ unit-test (matrix: Node 22.23.2, 24.18.1)
 │   └─ Coverage only on 24.18.1
 ├─ lint
-├─ (performance: CodSpeed benchmarks, paused until the plugin supports Vitest 5)
 ├─ verify-single-instance-npm (advisory; packages changed by this push)
 └─ notify-on-failure (Slack #alerts-build)
 ```
@@ -587,7 +597,7 @@ Push to master
 | Daily 03:00               | `sec-sync-bundle-branches.yml`    | Merge the base into `bundle/*` |
 | Daily 00:00               | `docker-build-push.yml`           | Nightly Docker images    |
 | Daily 00:00               | `test-db.yml`                     | Database compatibility   |
-| Daily 00:00               | `test-e2e-performance-reusable.yml`| Browser performance and idle memory baselines |
+| Daily 00:00               | `test-e2e-performance-reusable.yml`| Performance E2E         |
 | Daily 00:00               | `release-storybook.yml`       | Storybook deploy         |
 | Daily 00:00               | `release-chromatic.yml`       | Visual regression        |
 | Daily 00:00               | `util-check-docs-urls.yml`        | Doc link validation      |
@@ -600,7 +610,6 @@ Push to master
 | Daily 08:00               | `build-v3-nightly.yml`            | Nightly v3 Docker images |
 | Monday 00:00              | `util-update-node-popularity.yml` | Node usage stats         |
 | Monday 02:00              | `test-e2e-coverage-weekly.yml`    | Weekly E2E coverage      |
-| Wednesday 04:00           | `util-update-get-n8n-versions.yml` | get-n8n stable versions |
 | Saturday 22:00            | `test-evals-ai.yml`               | AI workflow evals        |
 | 1st of month 04:00        | `util-refresh-cubic-schema.yml`   | Refresh vendored cubic schema |
 
@@ -840,8 +849,9 @@ See **[CI-TELEMETRY.md](CI-TELEMETRY.md)** for:
 
 ## OWNERS
 
-Team ownership lives in the top-level `OWNERS` file. The scripts that consume
-it live in `.github/scripts/owners/`. Line format:
+Team ownership lives in the top-level `OWNERS` file (this replaces the
+GitHub-native `CODEOWNERS` file; see the transition note below). The scripts
+that consume it live in `.github/scripts/owners/`. Line format:
 
 ```
 <pattern> <@org/team> [required]
@@ -873,10 +883,8 @@ The file drives four workflows:
 ### Required reviews
 
 An entry with the `required` option makes team approval mandatory: when a PR
-changes a non-test file whose winning entry carries `required`, a member of each
-listed team must approve the PR. Test files match the shared patterns in
-`test-files.mjs` and do not trigger required reviews.
-`ci-owners-required-reviews.yml` evaluates this on
+changes a file whose winning entry carries `required`, a member of each listed
+team must approve the PR. `ci-owners-required-reviews.yml` evaluates this on
 PR changes and review events, and reports a commit status
 named **Required Reviews** on the head SHA. A missing approval reports
 `pending` ("Waiting for approval from: …"), not `failure`, so an unreviewed PR
@@ -906,13 +914,6 @@ so only `master` is trusted input. A PR cannot lift its own review
 requirement. A retarget re-evaluates the PR, so a verdict computed against the
 old base does not carry over.
 
-Members of the `large-scale-changes` GitHub team can make the **Required
-Reviews** status succeed without required OWNERS team approvals. A team member
-must apply the `large-scale-change` label. The gate checks the actor from the
-label event and verifies their current team membership. The label alone is not
-sufficient. Label changes re-evaluate the status. Removing the label removes
-the exemption.
-
 Every path that writes the status runs in the base repository context, because
 a fork-context run has no secrets and a read-only token. PR changes arrive
 through `pull_request_target`, which is safe here because no step checks out
@@ -925,6 +926,16 @@ contribution whose runs still wait for approval gets no review-event
 re-evaluation until a maintainer approves the runs; `workflow_dispatch` with
 the PR number is the manual fallback.
 
+### Transition from CODEOWNERS
+
+During a trial period, `.github/CODEOWNERS` stays in place next to OWNERS:
+GitHub's native code-owner enforcement keeps gating merges while the
+"Required Reviews" status runs side by side. The two must agree — CODEOWNERS
+holds exactly the `required` entries of OWNERS (plus the OWNERS file itself)
+and must not gain new entries; new ownership goes into OWNERS. After the
+trial, delete `.github/CODEOWNERS`, remove "Require review from Code Owners"
+from the master ruleset, and delete this section (tracked in DEVP-887).
+
 ---
 
 ## Runner Selection
@@ -948,7 +959,7 @@ the PR number is the manual fallback.
 
 **`blacksmith-4vcpu-ubuntu-2204`** - Unit tests (parallelized), linting (parallel file processing), typechecking (CPU-intensive), E2E test shards
 
-**`blacksmith-8vcpu-ubuntu-2204`** - Heavy parallel workloads. The PR `install-and-build` job uses it with the default runner provider. Most PR jobs wait for that job, and a cold build keeps all 4 vCPUs of a smaller runner busy.
+**`blacksmith-8vcpu-ubuntu-2204`** - Heavy parallel workloads
 
 ### Runner Provider Toggle
 
@@ -1141,28 +1152,40 @@ open the PR there. That PR **must stay a single-parent squash** — the publish
 posts to `#alerts-build` when that gate fails on a PR opened *from* `bundle/2.x` or
 `bundle/1.x` (link only, no PR title, since the branch is embargoed).
 
-`sec-sync-bundle-branches.yml` keeps those branches current nightly at 03:00 UTC and on
-`workflow_dispatch`, including before a bundle cut. It **merges the public base into** the
-bundle branch via
-[`scripts/sync-bundle-branch.mjs`](scripts/sync-bundle-branch.mjs) and pushes without forcing.
-Every push is verified to carry exactly the tree a merge of the two sides would produce (`git
-merge-tree`); a mismatch, or a conflict marker, fails the run instead of pushing.
+`sec-sync-bundle-branches.yml` keeps those branches current, daily plus whenever a PR is
+merged into one (and on `workflow_dispatch`). It **merges the base into** the bundle branch
+via [`scripts/sync-bundle-branch.mjs`](scripts/sync-bundle-branch.mjs) and pushes without
+forcing. Every push is verified to carry exactly the tree a merge of the two sides would
+produce (`git merge-tree`); a mismatch, or a conflict marker, fails the run instead of pushing.
 
-**A bundle branch is only ever built on published history.** The sync fetches `master` or `1.x`
-directly from `https://github.com/n8n-io/n8n.git`. It creates a missing bundle branch at that
-exact public SHA and merges that public SHA into an existing bundle branch. It never reads the
-private base branch. This matters because the `chore: Bundle/*` squash on private `master` is
-private-only until publication. The mirror can replace that commit with the public copy after
-the bundle branch is deleted. The next bundle branch must not inherit the replaced commit.
+**A bundle branch is only ever built on published history.** Before it creates or merges
+anything, the sync fetches the same-named branch from `https://github.com/n8n-io/n8n.git`
+(anonymously — its token is scoped to the private repo) and checks that the private base tip is
+contained in it. This matters because the `chore: Bundle/*` squash on private `master` is
+*private-only*: the mirror above discards it in favour of the public cherry-pick of the same
+changes. A sync in that window would root the branch on a commit that is about to disappear,
+leaving it carrying two commits for one set of fixes — and every fix PR cut from it inherits the
+dead one. The window is real: this workflow's daily cron and the mirror's hourly cron both fire
+at `:00` with nothing ordering them, which is why the check lives in the script rather than in
+step ordering.
+
+An unpublished cut is a **skip**, not a failure: the mirror discards that commit every hour
+regardless, so the run exits green with the reason in the log and the next one proceeds. A
+missing branch also self-heals, because the mirror re-dispatches this workflow while one is
+absent. A base ahead of public for **any other reason** fails the run — the mirror is stuck and
+will not clear it on its own, so fix that first (remove the commit, or dispatch **Security: Sync
+from Public** with `force`) and re-run. To sync a bundle branch immediately, dispatch the mirror
+and then this workflow. The blocking commits are listed in the run log; the failure annotation
+carries only a count, since a subject hints at the fix.
 
 Deleting a bundle branch takes its open PRs with it: GitHub moves each one onto the deleted
-branch's own base, and re-creating the branch does not move them back. The next nightly or manual
-sync re-creates the branch and then calls
+branch's own base, and re-creating the branch does not move them back. So the sync then calls
 [`sec-sync-retarget-prs.yml`](workflows/sec-sync-retarget-prs.yml), which moves every open PR
 on `master` onto `bundle/2.x` and every one on `1.x` onto `bundle/1.x`. It skips a bundle
 branch that does not exist, and skips PRs whose *head* is `bundle/*` — those are the
 `chore: Bundle/*` cut PRs, which target the base on purpose. It is also dispatchable on its
-own. The retarget runs inside the bundle sync after a missing branch is recreated.
+own. `sec-sync-public-to-private.yml` only *dispatches* the bundle sync when it finds a branch
+missing; the retarget runs inside that dispatched run, once the branch exists.
 
 **`bundle/*` is append-only — never rebase it, never force-push it.** These branches receive
 PRs, and rewriting a branch that receives PRs orphans the copies of its commits that the open
@@ -1175,19 +1198,21 @@ itself. To refresh a fix branch, use GitHub's **Update branch** button or
 sibling PR's merge base untouched, which is why only a rewrite breaks this.
 
 The costs of merging are deliberate and paid for: a merge commit per run, and fixes that have
-already been published staying in the branch's log. Neither reaches anything downstream,
-because a bundle publishes as one squashed commit taken from the tree rather than the history —
-the `chore: Bundle/*` PR's **diff** stays
+already been published staying in the branch's log (the old rebase dropped them as empty
+commits). Neither reaches anything downstream, because a bundle publishes as one squashed
+commit taken from the tree rather than the history — the `chore: Bundle/*` PR's **diff** stays
 exactly the pending fixes even when its commit list does not. For a list of what a bundle
-actually carries, read the fix PRs merged into the branch since the last cut, not `base..bundle`.
+actually carries, read the fix PRs merged into the branch since the last cut, not
+`base..bundle`. A lower cadence than the base's is fine too: a base push never re-triggered
+CI on the fix PRs, so syncing more often bought them nothing.
 
 There is **one job per bundle branch**. A conflict is detected from the merge tree before the
 working tree is touched, so the branch is left exactly as it was, that job **fails** (no
 green runs hiding a stalled branch) and `#alerts-security` gets a run link — while the other
-branch still syncs. Recovery is deliberate: merge the public base into the branch locally,
-resolve, push, then re-run the workflow — and that resolution then lives in the merge commit
-instead of being re-litigated on every later run. The sync never resolves a conflict itself,
-unlike `util-sync-master-to-3x.yml`.
+branch still syncs. Recovery is deliberate: merge the base into the branch locally, resolve,
+push, then re-run the workflow — and that resolution then lives in the merge commit instead of
+being re-litigated on every later run. The sync never resolves a conflict itself, unlike
+`util-sync-master-to-3x.yml`.
 
 See **[`../AGENTS.md`](../AGENTS.md)** ("Security Fix Hygiene") for the naming rules that
 keep the vulnerability out of public branch names, commits, and test descriptions.
@@ -1257,6 +1282,12 @@ Adding a new channel requires inviting the bot first; the first run otherwise fa
 
 ## Future Vision
 
+### Redundancy Review
+
+Comment trigger (`/test-workflows`) is a workaround.
+
+Long-term: Main CI should be reliable enough to not need these.
+
 ### Workflow Testability
 
 - Tools like `act` for local testing

.github/actions/ci-filter/__tests__/ci-filter.test.ts

@@ -58,15 +58,6 @@ describe('matchGlob', () => {
 		);
 	});
 
-	it('backend module pattern matches packaged module files', () => {
-		assert.ok(
-			matchGlob(
-				'packages/modules/insights/backend/src/insights.service.ts',
-				'packages/modules/*/backend/**',
-			),
-		);
-	});
-
 	it('* matches single-level only', () => {
 		assert.ok(matchGlob('README.md', '*.md'));
 		assert.ok(!matchGlob('docs/README.md', '*.md'));
@@ -251,7 +242,7 @@ describe('runtime filter', () => {
 		'!**/CHANGELOG.md',
 		'!**/*.test.ts',
 		'!**/*.spec.ts',
-		'!packages/quality/testing/playwright/**',
+		'!packages/testing/playwright/**',
 		'!packages/frontend/@n8n/storybook/**',
 		'!scripts/agent-setup.mjs',
 		'!scripts/backend-module/**',
@@ -274,7 +265,7 @@ describe('runtime filter', () => {
 
 	it('does not trigger on playwright tests', () => {
 		assert.equal(
-			evaluateFilter(['packages/quality/testing/playwright/tests/x.spec.ts'], runtimePatterns),
+			evaluateFilter(['packages/testing/playwright/tests/x.spec.ts'], runtimePatterns),
 			false,
 		);
 	});

.github/actions/ci-filter/ci-filter.mjs

@@ -143,8 +143,8 @@ export function getAddedFiles(baseRef, headRef = 'HEAD') {
 }
 
 /**
- * Fetch the base ref, then deepen only if its merge base with headRef is not
- * reliably reachable from the checkout.
+ * Deepen the shallow clone until the merge base between the fetched base ref
+ * (FETCH_HEAD) and headRef is reliably reachable.
  *
  * A single fixed deepen is not enough for stale PRs whose divergence point is
  * older than the shallow boundary. We fetch with an exponentially growing
@@ -159,12 +159,7 @@ export function getAddedFiles(baseRef, headRef = 'HEAD') {
 function fetchUntilMergeBase(baseRef, headRef) {
 	let step = Number(process.env.CI_FILTER_DEEPEN_STEP) || 200;
 	const maxDeepen = Number(process.env.CI_FILTER_MAX_DEEPEN) || 20_000;
-	// A PR merge checkout already has the base commit as its first parent.
-	const hasBaseCommit = headRef === 'HEAD^2' && hasCommit('HEAD^1');
-	const depth = isShallow() && !hasBaseCommit ? `--depth=${step} ` : '';
-	execSync(`git fetch --no-tags --prune --filter=blob:none ${depth}origin ${baseRef}`, {
-		stdio: 'pipe',
-	});
+	deepenFetch(baseRef, step, maxDeepen);
 
 	while (!hasReliableMergeBase(headRef)) {
 		if (!isShallow()) {
@@ -172,17 +167,8 @@ function fetchUntilMergeBase(baseRef, headRef) {
 				`No merge base between FETCH_HEAD and ${headRef} after fetching the full history of "${baseRef}" (unrelated histories).`,
 			);
 		}
-		deepenFetch(baseRef, step, maxDeepen);
 		step *= 2;
-	}
-}
-
-function hasCommit(ref) {
-	try {
-		execSync(`git cat-file -e ${ref}^{commit}`, { stdio: 'pipe' });
-		return true;
-	} catch {
-		return false;
+		deepenFetch(baseRef, step, maxDeepen);
 	}
 }
 

.github/actions/setup-nodejs/action.yml

@@ -171,7 +171,7 @@ runs:
 
     - name: Restore pnpm metadata cache
       if: ${{ inputs.install-command != '' }}
-      uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+      uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
       with:
         path: ${{ steps.pnpm-metadata-cache.outputs.path }}
         key: pnpm-metadata-v1-${{ runner.os }}-${{ runner.arch }}-${{ steps.pnpm-metadata-cache.outputs.version }}-${{ hashFiles(inputs.cache-dependency-path) }}

.github/docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   postgres:
-    image: ${TEST_IMAGE_POSTGRES:?set it from packages/quality/environments/containers/postgres-versions.json}
+    image: ${TEST_IMAGE_POSTGRES:?set it from packages/testing/containers/postgres-versions.json}
     restart: always
     environment:
       - POSTGRES_DB=n8n

.github/pull_request_template.md

@@ -1,15 +1,8 @@
 ## Summary
 
 <!--
-Start with 1 to 3 sentences of background for a new reader. Then describe what the PR does, in prose, not as a list of edits.
-Add photos or videos for UI changes.
--->
-
-## Why
-
-<!--
-State the problem or the constraint that makes the current code unfit. Do not repeat the Summary.
-Add a rejected alternative only when a reviewer would ask about it, in one or two sentences.
+Describe what the PR does.
+Photos and videos are recommended.
 -->
 
 ## How to test

.github/scripts/db-test-matrix.mjs

@@ -7,16 +7,11 @@ import { fileURLToPath } from 'node:url';
 
 const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
 
-export const POSTGRES_VERSIONS_PATH = 'packages/quality/environments/containers/postgres-versions.json';
+export const POSTGRES_VERSIONS_PATH = 'packages/testing/containers/postgres-versions.json';
 
 /** The matrix wall is bounded by its slowest leg, so a larger runner buys nothing. */
 const RUNNER = 'blacksmith-4vcpu-ubuntu-2204';
 
-// Each leg runs one suite on its own runner, so use every CPU. The CI default
-// (half the CPUs) is for jobs that run many suites at once. Every test file
-// gets its own database, so more workers do not share state.
-const MAX_WORKERS = '--maxWorkers=100%';
-
 /**
  * @typedef {Object} PostgresVersions
  * @property {string} primary
@@ -91,7 +86,7 @@ export function buildMatrix(versions, scope = 'full') {
 		{
 			name: 'SQLite Pooled',
 			runner: RUNNER,
-			'test-cmd': `pnpm test:sqlite ${MAX_WORKERS}`,
+			'test-cmd': 'pnpm test:sqlite',
 			'migration-cmd': 'pnpm test:sqlite:migrations',
 			'schema-check-cmd': 'pnpm --filter=@n8n/db schema:check:sqlite',
 			TEST_IMAGE_POSTGRES: undefined,
@@ -100,8 +95,8 @@ export function buildMatrix(versions, scope = 'full') {
 		...postgresLegs.map(({ major, image }) => ({
 			name: `Postgres ${major}`,
 			runner: RUNNER,
-			'test-cmd': `pnpm test:postgres:integration:tc ${MAX_WORKERS}`,
-			'migration-cmd': `pnpm test:postgres:migrations:tc ${MAX_WORKERS}`,
+			'test-cmd': 'pnpm test:postgres:integration:tc',
+			'migration-cmd': 'pnpm test:postgres:migrations:tc',
 			'schema-check-cmd': image === primary ? 'pnpm --filter=@n8n/db schema:check:postgres' : '',
 			TEST_IMAGE_POSTGRES: image,
 			collectCoverage: scope === 'pr' && image === primary ? 'true' : 'false',

.github/scripts/github-helpers.mjs

@@ -490,23 +490,6 @@ export async function getPrReviews(pullRequestNumber) {
 	});
 }
 
-/**
- * Returns all events for a PR, including label changes and their actors.
- *
- * @param { number } pullRequestNumber
- * @returns { Promise<Array<{ event: string, actor: { login: string } | null, label?: { name?: string } | null, created_at?: string }>> }
- * */
-export async function getPrEvents(pullRequestNumber) {
-	const { octokit, owner, repo } = initGithub();
-
-	return await octokit.paginate(octokit.rest.issues.listEvents, {
-		owner,
-		repo,
-		issue_number: pullRequestNumber,
-		per_page: 100,
-	});
-}
-
 /**
  * Test whether a user is an active member of an org team.
  *

.github/scripts/github-helpers.test.mjs

@@ -20,7 +20,6 @@ mock.module('@actions/github', {
 
 const {
 	findCommentByMarker,
-	getPrEvents,
 	initGithub,
 	postOrUpdateComment,
 	setOctokit,
@@ -30,33 +29,6 @@ const {
 
 const ORIGINAL_ENV = { ...process.env };
 
-describe('getPrEvents', () => {
-	beforeEach(() => {
-		process.env.GITHUB_REPOSITORY = 'n8n-io/n8n';
-	});
-
-	afterEach(() => {
-		setOctokit(null);
-		process.env = { ...ORIGINAL_ENV };
-	});
-
-	it('paginates issue events for the pull request', async () => {
-		const events = [{ event: 'labeled', actor: { login: 'ana' } }];
-		const listEvents = {};
-		const paginate = mock.fn(async () => events);
-		setOctokit({ paginate, rest: { issues: { listEvents } } });
-
-		assert.equal(await getPrEvents(42), events);
-		assert.equal(paginate.mock.calls[0].arguments[0], listEvents);
-		assert.deepEqual(paginate.mock.calls[0].arguments[1], {
-			owner: 'n8n-io',
-			repo: 'n8n',
-			issue_number: 42,
-			per_page: 100,
-		});
-	});
-});
-
 describe('findCommentByMarker', () => {
 	beforeEach(() => {
 		process.env.GITHUB_TOKEN = 'token';

.github/scripts/glob.test.mjs

@@ -30,9 +30,9 @@ describe('matchesGlob', () => {
 	});
 
 	it('matches a prefix pattern', () => {
-		assert.equal(matchesGlob('packages/quality/testing/playwright/a.ts', 'packages/quality/**'), true);
-		assert.equal(matchesGlob('packages/quality', 'packages/quality/**'), false);
-		assert.equal(matchesGlob('packages/cli/a.ts', 'packages/quality/**'), false);
+		assert.equal(matchesGlob('packages/testing/playwright/a.ts', 'packages/testing/**'), true);
+		assert.equal(matchesGlob('packages/testing', 'packages/testing/**'), false);
+		assert.equal(matchesGlob('packages/cli/a.ts', 'packages/testing/**'), false);
 	});
 
 	it('matches a literal path only exactly', () => {
@@ -55,8 +55,8 @@ describe('matchesGlob', () => {
 			'packages/cli/__snapshots__/a.ts.snap',
 			'packages/nodes-base/nodes/Foo/test/fixtures/a.json',
 			'packages/frontend/editor-ui/src/__mocks__/a.ts',
-			'packages/quality/testing/playwright/a.ts',
-			'packages/quality',
+			'packages/testing/playwright/a.ts',
+			'packages/testing',
 			'.github/scripts/glob.test.mjs',
 			'.github/workflows/ci.yml',
 			'a/.hidden/b.spec.js',

.github/scripts/owners/owners.mjs

@@ -1,8 +1,6 @@
 import { readFileSync, statSync } from "node:fs";
 import { join } from "node:path";
 
-import { isTestFile } from "../test-files.mjs";
-
 /**
  * @typedef OwnersEntry
  * @property { string } pattern
@@ -238,7 +236,7 @@ export function assignOwnership(files, entries) {
 
 /**
  * Determine which teams must approve the changeset: a team is required when
- * its `required` entry wins (last-match) for a changed non-test file.
+ * its `required` entry wins (last-match) for a changed file.
  *
  * @param { Set<string> } files
  * @param { OwnersEntry[] } entries
@@ -249,8 +247,6 @@ export function resolveRequiredTeams(files, entries) {
 	const teamToFiles = new Map();
 
 	for (const file of [...files].sort()) {
-		if (isTestFile(file)) continue;
-
 		const entry = findOwningEntry(file, entries);
 		if (!entry?.required) continue;
 

.github/scripts/owners/owners.test.mjs

@@ -293,22 +293,6 @@ describe('resolveRequiredTeams', () => {
 		assert.equal(result.size, 0);
 	});
 
-	it('ignores test files whose winning entry is required', () => {
-		const result = resolveRequiredTeams(
-			new Set([
-				'.github/scripts/check.test.mjs',
-				'.github/workflows/__tests__/check.ts',
-				'.github/workflows/ci.yml',
-			]),
-			[
-				entry({ pattern: '.github/scripts/', team: '@n8n-io/qa-dx', required: true }),
-				entry({ pattern: '.github/workflows/', team: '@n8n-io/qa-dx', required: true }),
-			],
-		);
-
-		assert.deepEqual(result.get('@n8n-io/qa-dx'), ['.github/workflows/ci.yml']);
-	});
-
 	it('a later non-required entry overrides an earlier required one', () => {
 		const result = resolveRequiredTeams(new Set(['db/migrations/1-init.ts']), [
 			entry({ pattern: 'db/', team: '@n8n-io/migrations-review', required: true }),

.github/scripts/owners/required-reviews.mjs

@@ -1,16 +1,15 @@
 /**
  * Required-review enforcement for OWNERS entries marked `required`.
  *
- * Computes the teams whose approval the non-test changes need (per OWNERS)
+ * Computes the teams whose approval the changeset needs (per OWNERS)
  * and reports the verdict as a "Required Reviews" commit status on the PR
  * head SHA. A ruleset that lists this status as a required check blocks the
  * merge until a member of each required team has approved the PR. Merge-queue
  * runs do not reach this script: the workflow reports success on the queue
  * head directly, because entering the queue already required a green status.
  *
- * Every base branch is enforced unless the PR matches a trusted exemption.
- * Exemptions include configured routes and an authorized large-scale-change
- * label. An exempt PR reports success without approval evaluation.
+ * Every base branch is enforced unless the PR matches a route in
+ * REQUIRED_REVIEW_EXEMPTIONS; an exempt PR reports success without evaluation.
  *
  * The job itself always succeeds when the evaluation runs; the commit status
  * carries the verdict. The status is set to pending before the evaluation
@@ -21,19 +20,15 @@
 import {
 	getChangedFiles,
 	getEventFromGithubEventPath,
-	getPrEvents,
 	getPrReviews,
 	getPullRequestById,
 	isTeamMember,
 	listOpenPullRequestsByHead,
-	readPrLabels,
 	setCommitStatus,
 } from '../github-helpers.mjs';
 import { parseOwnersFile, resolveRequiredTeams, teamHandleToSlug } from './owners.mjs';
 
 export const STATUS_CONTEXT = 'Required Reviews';
-export const LARGE_SCALE_CHANGE_LABEL = 'large-scale-change';
-export const LARGE_SCALE_CHANGES_TEAM = 'large-scale-changes';
 
 /**
  * PR routes that skip the evaluation. Each entry is `<head> -> <base>` or
@@ -165,32 +160,6 @@ export function collectApprovers(reviews) {
 	);
 }
 
-/**
- * Return the team member who applied the active large-scale-change label.
- * The current membership check makes the exemption expire when the actor
- * leaves the team.
- *
- * @param { number } pullRequestNumber
- * @param {{ labels?: Array<string | { name: string }> }} pullRequest
- * @returns { Promise<string | undefined> }
- */
-export async function findLargeScaleChangeExemption(pullRequestNumber, pullRequest) {
-	if (!readPrLabels(pullRequest).includes(LARGE_SCALE_CHANGE_LABEL)) return undefined;
-
-	const labelEvents = (await getPrEvents(pullRequestNumber)).filter(
-		(event) =>
-			['labeled', 'unlabeled'].includes(event.event) &&
-			event.label?.name === LARGE_SCALE_CHANGE_LABEL,
-	);
-	const latestEvent = labelEvents.at(-1);
-	if (latestEvent?.event !== 'labeled') return undefined;
-
-	const actor = latestEvent.actor?.login;
-	if (!actor) return undefined;
-
-	return (await isTeamMember(LARGE_SCALE_CHANGES_TEAM, actor)) ? actor : undefined;
-}
-
 /**
  * @param { string[] } missingTeams Team handles without an approving member.
  * @param { number } requiredCount Total number of required teams.
@@ -289,20 +258,9 @@ export async function run() {
 	let status;
 	try {
 		const exemption = findExemption(pullRequest);
-		const largeScaleChangeActor = exemption
-			? undefined
-			: await findLargeScaleChangeExemption(pullRequestNumber, pullRequest);
 		if (exemption) {
 			console.log(`PR #${pullRequestNumber} matches exempt route "${exemption.source}"; skipping the evaluation.`);
 			status = { state: 'success', description: `Exempt route: ${exemption.source}` };
-		} else if (largeScaleChangeActor) {
-			console.log(
-				`PR #${pullRequestNumber} has an authorized ${LARGE_SCALE_CHANGE_LABEL} label from ${largeScaleChangeActor}; skipping the evaluation.`,
-			);
-			status = {
-				state: 'success',
-				description: `Large-scale change exemption by @${largeScaleChangeActor}`,
-			};
 		} else {
 			status = await evaluateRequiredReviews(pullRequestNumber);
 		}

.github/scripts/owners/required-reviews.test.mjs

@@ -15,8 +15,6 @@ let getPullRequestByIdImpl = async () => ({});
 let getChangedFilesImpl = async () => new Set();
 /** @type {(pullRequestNumber: number) => Promise<any[]>} */
 let getPrReviewsImpl = async () => [];
-/** @type {(pullRequestNumber: number) => Promise<any[]>} */
-let getPrEventsImpl = async () => [];
 /** @type {(teamSlug: string, username: string) => Promise<boolean>} */
 let isTeamMemberImpl = async () => false;
 /** @type {(sha: string, status: any) => Promise<void>} */
@@ -29,14 +27,11 @@ mock.module('../github-helpers.mjs', {
 		getEventFromGithubEventPath: () => eventImpl(),
 		getPullRequestById: (n) => getPullRequestByIdImpl(n),
 		getChangedFiles: (n) => getChangedFilesImpl(n),
-		getPrEvents: (n) => getPrEventsImpl(n),
 		getPrReviews: (n) => getPrReviewsImpl(n),
 		isTeamMember: (slug, username) => isTeamMemberImpl(slug, username),
 		setCommitStatus: (sha, status) => setCommitStatusImpl(sha, status),
 		listOpenPullRequestsByHead: (headOwner, headBranch) =>
 			listOpenPullRequestsByHeadImpl(headOwner, headBranch),
-		readPrLabels: (pullRequest) =>
-			(pullRequest.labels ?? []).map((label) => (typeof label === 'string' ? label : label.name)),
 	},
 });
 
@@ -56,12 +51,9 @@ mock.module('./owners.mjs', {
 
 const {
 	REQUIRED_REVIEW_EXEMPTIONS,
-	LARGE_SCALE_CHANGE_LABEL,
-	LARGE_SCALE_CHANGES_TEAM,
 	STATUS_CONTEXT,
 	buildStatus,
 	collectApprovers,
-	findLargeScaleChangeExemption,
 	findExemption,
 	latestReviewStates,
 	matchesBranchPattern,
@@ -254,116 +246,6 @@ describe('collectApprovers', () => {
 	});
 });
 
-describe('findLargeScaleChangeExemption', () => {
-	beforeEach(() => {
-		getPrEventsImpl = async () => [];
-		isTeamMemberImpl = async () => false;
-	});
-
-	it('returns the team member who applied the active label', async () => {
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'ana' } },
-		];
-		isTeamMemberImpl = async (slug, username) =>
-			slug === LARGE_SCALE_CHANGES_TEAM && username === 'ana';
-
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			'ana',
-		);
-	});
-
-	it('rejects a label applied by someone outside the team', async () => {
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'outsider' } },
-		];
-
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			undefined,
-		);
-	});
-
-	it('rejects the exemption when the label event is not available', async () => {
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			undefined,
-		);
-	});
-
-	it('rejects the exemption when the label event has no actor', async () => {
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: null },
-		];
-		const isTeamMember = mock.fn(async () => true);
-		isTeamMemberImpl = isTeamMember;
-
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			undefined,
-		);
-		assert.equal(isTeamMember.mock.calls.length, 0);
-	});
-
-	it('ignores events for other labels', async () => {
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'ana' } },
-			{ event: 'labeled', label: { name: 'bug' }, actor: { login: 'outsider' } },
-			{ event: 'unlabeled', label: { name: 'bug' }, actor: { login: 'outsider' } },
-		];
-		isTeamMemberImpl = async (slug, username) =>
-			slug === LARGE_SCALE_CHANGES_TEAM && username === 'ana';
-
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			'ana',
-		);
-	});
-
-	it('uses the actor from the latest label application', async () => {
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'ana' } },
-			{
-				event: 'unlabeled',
-				label: { name: LARGE_SCALE_CHANGE_LABEL },
-				actor: { login: 'outsider' },
-			},
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'outsider' } },
-		];
-		isTeamMemberImpl = async (slug, username) => username === 'ana';
-
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			undefined,
-		);
-	});
-
-	it('rejects the exemption when the latest event removed the label', async () => {
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'ana' } },
-			{
-				event: 'unlabeled',
-				label: { name: LARGE_SCALE_CHANGE_LABEL },
-				actor: { login: 'outsider' },
-			},
-		];
-		isTeamMemberImpl = async (slug, username) => username === 'ana';
-
-		assert.equal(
-			await findLargeScaleChangeExemption(42, { labels: [{ name: LARGE_SCALE_CHANGE_LABEL }] }),
-			undefined,
-		);
-	});
-
-	it('does not inspect events when the label is absent', async () => {
-		const getPrEvents = mock.fn(async () => []);
-		getPrEventsImpl = getPrEvents;
-
-		assert.equal(await findLargeScaleChangeExemption(42, { labels: [] }), undefined);
-		assert.equal(getPrEvents.mock.calls.length, 0);
-	});
-});
-
 describe('buildStatus', () => {
 	it('succeeds when nothing is required', () => {
 		const status = buildStatus([], 0);
@@ -401,7 +283,6 @@ describe('run', () => {
 		});
 		getChangedFilesImpl = async () => new Set(['a.ts']);
 		getPrReviewsImpl = async () => [];
-		getPrEventsImpl = async () => [];
 		isTeamMemberImpl = async () => false;
 		parseOwnersFileImpl = () => [];
 		resolveRequiredTeamsImpl = () => new Map();
@@ -550,44 +431,6 @@ describe('run', () => {
 		assert.equal(status.description, 'Exempt route: sync/master-to-3x -> 3.x');
 	});
 
-	it('reports success without evaluating an authorized large-scale change', async () => {
-		getPullRequestByIdImpl = async () => ({
-			...pullRequestFor('feature/x', 'master'),
-			labels: [{ name: LARGE_SCALE_CHANGE_LABEL }],
-		});
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'ana' } },
-		];
-		isTeamMemberImpl = async (slug, username) =>
-			slug === LARGE_SCALE_CHANGES_TEAM && username === 'ana';
-		const getChangedFiles = mock.fn(async () => new Set(['a.ts']));
-		getChangedFilesImpl = getChangedFiles;
-
-		await run();
-
-		assert.equal(getChangedFiles.mock.calls.length, 0);
-		const [, status] = setCommitStatus.mock.calls.at(-1).arguments;
-		assert.equal(status.state, 'success');
-		assert.equal(status.description, 'Large-scale change exemption by @ana');
-	});
-
-	it('evaluates a large-scale label applied outside the team', async () => {
-		getPullRequestByIdImpl = async () => ({
-			...pullRequestFor('feature/x', 'master'),
-			labels: [{ name: LARGE_SCALE_CHANGE_LABEL }],
-		});
-		getPrEventsImpl = async () => [
-			{ event: 'labeled', label: { name: LARGE_SCALE_CHANGE_LABEL }, actor: { login: 'outsider' } },
-		];
-		resolveRequiredTeamsImpl = () => new Map([['@n8n-io/qa-dx', ['a.ts']]]);
-
-		await run();
-
-		const [, status] = setCommitStatus.mock.calls.at(-1).arguments;
-		assert.equal(status.state, 'pending');
-		assert.match(status.description, /Waiting for approval from: qa-dx/);
-	});
-
 	it('does not exempt a fork branch that is named like an exempt route', async () => {
 		getPullRequestByIdImpl = async () => pullRequestFor('sync/master-to-3x', '3.x', 'someone/n8n');
 		resolveRequiredTeamsImpl = () => new Map([['@n8n-io/qa-dx', ['a.ts']]]);

.github/scripts/owners/review-recommendations.mjs

@@ -31,38 +31,6 @@ import {
  */
 
 const BOT_MARKER = '<!-- pr-recommendations -->';
-const REQUIRED_FILES_DISPLAY_LIMIT = 5;
-
-/**
- * @param { string } value
- * @returns { string }
- */
-function escapeHtml(value) {
-	return value
-		.replaceAll('&', '&amp;')
-		.replaceAll('<', '&lt;')
-		.replaceAll('>', '&gt;')
-		.replaceAll('|', '&#124;');
-}
-
-/**
- * @param { string[] } files
- * @returns { string }
- */
-function formatRequiredFiles(files) {
-	const displayedFiles = files
-		.slice(0, REQUIRED_FILES_DISPLAY_LIMIT)
-		.map((file) => `<code>${escapeHtml(file)}</code>`);
-	const omittedFileCount = files.length - displayedFiles.length;
-
-	if (omittedFileCount > 0) {
-		displayedFiles.push(
-			`<em>${omittedFileCount} more file${omittedFileCount === 1 ? '' : 's'}</em>`,
-		);
-	}
-
-	return displayedFiles.join('<br>');
-}
 
 function createEmptyLineStats() {
 	return {
@@ -242,11 +210,9 @@ export function buildRequiredReviewsSection(requiredTeamFiles) {
 		'',
 		'Some changed files have a `required` owner in `OWNERS`. A member of each of these teams must approve this PR before it can merge:',
 		'',
-		'| Team | Files owned | Owned files |',
-		'| --- | ---: | --- |',
-		...[...requiredTeamFiles].map(
-			([team, files]) => `| ${team} | ${files.length} | ${formatRequiredFiles(files)} |`,
-		),
+		'| Team | Files |',
+		'| --- | ---: |',
+		...[...requiredTeamFiles].map(([team, files]) => `| ${team} | ${files.length} |`),
 		'',
 		`Request a review from the team${plural ? 's' : ''} — GitHub assigns reviewers according to the team's review settings. The \`Auto-assign reviewers\` label does this for all owning teams.`,
 	].join('\n');

.github/scripts/owners/review-recommendations.test.mjs

@@ -217,7 +217,7 @@ describe('computeLineStats', () => {
 			['*.snap extension', 'src/foo.snap'],
 			['fixtures/ directory', 'src/fixtures/data.json'],
 			['__mocks__/ directory', 'src/__mocks__/axios.ts'],
-			['packages/quality/**', 'packages/quality/testing/playwright/spec.ts'],
+			['packages/testing/**', 'packages/testing/playwright/spec.ts'],
 			['dot-directory *.test.mjs', '.github/scripts/owners/owners.test.mjs'],
 		]) {
 			it(`classifies ${label} as testFiles`, () => {
@@ -322,7 +322,7 @@ describe('buildRequiredReviewsSection', () => {
 		assert.equal(buildRequiredReviewsSection(new Map()), null);
 	});
 
-	it('lists each required team with its file count and owned files', () => {
+	it('lists each required team with its file count', () => {
 		const section = buildRequiredReviewsSection(
 			new Map([
 				['@n8n-io/qa-dx', ['a.yml', 'b.yml']],
@@ -331,30 +331,8 @@ describe('buildRequiredReviewsSection', () => {
 		);
 
 		assert.match(section, /### Required reviews/);
-		assert.match(
-			section,
-			/\| @n8n-io\/qa-dx \| 2 \| <code>a\.yml<\/code><br><code>b\.yml<\/code> \|/,
-		);
-		assert.match(section, /\| @n8n-io\/migrations-review \| 1 \| <code>m\.ts<\/code> \|/);
-	});
-
-	it('shows no more than five files for each team', () => {
-		const section = buildRequiredReviewsSection(
-			new Map([['@n8n-io/qa-dx', ['a.yml', 'b.yml', 'c.yml', 'd.yml', 'e.yml', 'f.yml', 'g.yml']]]),
-		);
-
-		assert.match(section, /<code>a\.yml<\/code>/);
-		assert.match(section, /<code>e\.yml<\/code>/);
-		assert.doesNotMatch(section, /<code>f\.yml<\/code>/);
-		assert.match(section, /<em>2 more files<\/em>/);
-	});
-
-	it('escapes file paths before adding them to the table', () => {
-		const section = buildRequiredReviewsSection(
-			new Map([['@n8n-io/qa-dx', ['packages/a|b/<script>.ts']]]),
-		);
-
-		assert.match(section, /<code>packages\/a&#124;b\/&lt;script&gt;\.ts<\/code>/);
+		assert.match(section, /\| @n8n-io\/qa-dx \| 2 \|/);
+		assert.match(section, /\| @n8n-io\/migrations-review \| 1 \|/);
 	});
 
 	it('prompts to request review from the team, plural when several teams are required', () => {
@@ -469,7 +447,7 @@ describe('run', () => {
 
 		const body = postOrUpdateComment.mock.calls[0].arguments[1];
 		assert.match(body, /### Required reviews/);
-		assert.match(body, /\| @n8n-io\/qa-dx \| 1 \| <code>\.github\/workflows\/ci\.yml<\/code> \|/);
+		assert.match(body, /\| @n8n-io\/qa-dx \| 1 \|/);
 		assert.match(body, /Request a review from the team —/);
 	});
 });