Compare commits

..

8 Commits

Author SHA1 Message Date
n8n-assistant[bot]
970da944b9
🚀 Release 2.34.2 (#35693)
Co-authored-by: n8n-release-helper[bot] <265227495+n8n-release-helper[bot]@users.noreply.github.com>
2026-08-06 07:16:53 +00:00
n8n-assistant[bot]
38b6dca1b5
fix(editor): Show agent tool credentials above configuration (backport to release-candidate/2.34.x) (#35612)
Co-authored-by: Michael Drury <me@michaeldrury.co.uk>
2026-08-05 12:32:46 +00:00
n8n-assistant[bot]
ed4298acfd
fix(editor): Focus Markdown editor input before toolbar (backport to release-candidate/2.34.x) (#35605)
Co-authored-by: Rob Hough <robhough180@gmail.com>
2026-08-05 11:56:20 +00:00
n8n-assistant[bot]
e2279d07cd
🚀 Release 2.34.1 (#35574)
Co-authored-by: Matsuuu <16068444+Matsuuu@users.noreply.github.com>
2026-08-05 10:50:10 +03:00
n8n-assistant[bot]
f2a28826ab
fix(editor): Fix janky spacing between agent capability chips with long labels (backport to release-candidate/2.34.x) (#35571)
Co-authored-by: Rob Hough <robhough180@gmail.com>
2026-08-05 07:42:04 +00:00
Matsu
170cfc7870
chore: Bundle/2.x (backport to 2.34.x) (#35569) 2026-08-05 09:30:47 +03:00
n8n-assistant[bot]
a93a2c8fb0
fix(core): Recover unresponsive task runners (backport to release-candidate/2.34.x) (#35527)
Co-authored-by: Lorent Lempereur <looorent@users.noreply.github.com>
Co-authored-by: Lorent Lempereur <lorent.lempereur@n8n.io>
2026-08-04 15:36:55 +00:00
n8n-assistant[bot]
9eb299b857
fix(editor): Realign SchemaItem text to optical baseline (no-changelog) (backport to release-candidate/2.34.x) (#35531)
Co-authored-by: Rob Hough <robhough180@gmail.com>
2026-08-04 15:07:36 +00:00
15764 changed files with 553598 additions and 1479862 deletions

View File

@ -1,104 +0,0 @@
# Review rules
Rules the AI reviewer (cubic) enforces on pull requests. One rule per file,
grouped by the agent that loads it. `cubic.yaml` links these files via
`file_paths`; the prose in `cubic.yaml` stays thin so the rules are reviewable
as normal markdown.
These files hold *what to look for*. How to review — the bar a comment must
clear, when to stay silent, what never to say — lives once in
`custom_instructions` in `cubic.yaml`, because it reaches every agent.
## Layout
| Directory | Agent | Scope |
|-------------|----------|-----------------------------------------------------|
| `security/` | Security | backend packages, nodes, and the frontend render surface |
| `backend/` | Backend | `cli`, `@n8n/db`, `core`, `workflow`, node packages |
| `db-migrations/` | DB migrations | `@n8n/db` migrations + their tests in `cli` |
| `frontend/` | Frontend | `packages/frontend` |
| `qa-dx/` | QA & DX | `.github`, `docker`, `scripts`, `patches`, `packages/quality`, the lint/test/TS config packages, baselines |
| `testing/` | Backend + Frontend | any package with a test suite |
A directory maps to one agent unless, like `testing/`, the policy is identical
across domains — then it is one file listed in several agents' `file_paths`,
never a copy per directory. Security and QA & DX deliberately don't link
`testing/`: coverage nagging on a credential fix or a Dockerfile is noise those
agents shouldn't be able to produce.
All five slots are used, so a new domain now merges into an existing agent. QA &
DX covers the build, test, and CI surface — the same paths `OWNERS`
assigns to `@n8n-io/qa-dx`. Code-quality rules that happen to apply broadly
(error classes, `any`, lazy imports) are backend rules, not QA & DX ones. DB
migrations is split out of Backend because a migration is permanent and runs
unattended on every instance, so it is judged against a different bar.
## Limits that bite
cubic fails silently on all three of these, which is why `pnpm check:cubic-config`
enforces them in CI:
- **5 enabled agents per repository.** Rules past the fifth never run and cubic
says nothing. All five are in use.
- **10,000 characters per agent**, counting the `description`, every linked
file, and `custom_instructions` — concatenated in the listed order.
Everything past the limit is dropped from the review prompt. The shared block
is prepended to every agent, so a line added there is spent against all of
their ceilings, not one; `check:cubic-config` prints the own/shared split.
- **Repo-relative file paths only.** Globs, directories, parent-directory
traversal, and absolute paths are all rejected — list each file explicitly.
The schema caps `file_paths` at 10 entries per agent.
## Adding a rule
1. Pick the level of reach first. Guidance about *how* to review goes in
`custom_instructions`, not here. A policy that is word-for-word the same in
two domains becomes one file linked by both agents. Everything else goes in
the directory of the agent that owns it.
2. Open with a one-line "Applies to:" so the reviewer skips it on unrelated
files — a backend PR still loads the node rules, since include globs are
per-agent.
3. Add its path to every `file_paths` that should load it in `cubic.yaml`.
Keep each agent's list ordered by how often the class it covers actually
shows up — the ceiling truncates from the end, so the last file listed is the
one that silently disappears first.
4. Run `pnpm check:cubic-config`. It validates `cubic.yaml` against cubic's
published JSON schema, then fails on a missing path, an over-budget agent, or
a rule file nobody links, and warns at 80% of the ceiling.
The schema is vendored at `.github/scripts/quality/cubic-config.schema.json` so
the check needs no network. `util-refresh-cubic-schema.yml` re-pulls it on the 1st
of each month and opens a PR when it changed — review that diff for new cubic
options worth adopting. To refresh by hand:
`node .github/scripts/quality/check-cubic-config.mjs --refresh`.
Don't restate something CI already fails on — see the "Don't repeat what CI
already fails on" section in `cubic.yaml`. Write what static analysis cannot
see.
## Working with cubic on a PR
Tag `@cubic-dev-ai` in a PR comment (GitHub autocomplete won't offer it — type it).
Replying to one of cubic's own comments needs no tag, but a bare reply never
authorises code changes.
| Ask | Comment |
|-----|---------|
| Re-review everything | `@cubic-dev-ai review this PR` |
| Review only what changed since its last pass | `@cubic-dev-ai incremental review` |
| Deeper pass with the stronger models | `@cubic-dev-ai ultrareview` — or `ultrareview: focus on <topic>` |
| Fix a finding on this branch | `@cubic-dev-ai fix this issue in this branch` |
| One-off context for this run only | `@cubic-dev-ai review this and use <url>` |
| Ask about a finding | reply in the thread |
Two things worth knowing:
- Ultrareview is billed at 3× a normal review, which is why it is manual rather
than automatic here.
- A PR over 10,000 changed lines, or carrying one of the ignored automation
labels, gets no automatic review. `@cubic-dev-ai review this PR` overrides
that when you want it.
Disagreeing in a thread is not wasted: cubic turns feedback into team-scoped
learnings, so a well-argued "this is fine because X" shapes later reviews. Vague
replies ("done", "thanks") don't.

View File

@ -1,59 +0,0 @@
# Controller request bodies must use a DTO
Applies to: `*.controller.ts` outside `packages/cli/src/public-api` (the public
API has its own lint rules).
Controller endpoints that accept a request body must use the `@Body` decorator
with a DTO class. TypeScript types alone provide no runtime validation — only
`@Body` with a Zod-based DTO (extending `Z.class`) validates incoming data.
This fails quietly rather than loudly: `controller.registry.ts` validates only
when the parameter type exposes `safeParse`, and otherwise pushes nothing into
the handler's arguments at all.
Only flag when there is positive evidence the developer intended to accept a
body but didn't use the pattern:
1. A parameter named `payload`, `body`, or `data` without `@Body`
2. Direct `req.body` access in a controller method
3. `@Body` with a type not ending in `Dto` — suggests unawareness of the pattern
Do NOT flag:
- `@Body` with a type ending in `Dto` — the developer knows the pattern
- POST/PUT/PATCH endpoints with no body parameter; they may legitimately have no body
- Webhook controllers
Violation:
```typescript
@Post('/')
async create(req: AuthenticatedRequest, payload: CreateUser) {
// payload is undefined - missing @Body decorator
}
@Post('/')
async create(req: AuthenticatedRequest) {
const data = req.body; // No runtime validation
}
@Post('/')
async create(@Body payload: CreateUser) {
// Type doesn't end in Dto - likely missing Zod validation
}
```
Allowed:
```typescript
@Post('/')
async create(@Body payload: CreateUserDto) { ... }
@Post('/activate')
async activate(req: AuthenticatedRequest) {
// Legitimately no body needed
}
```
Use a DTO from `@n8n/api-types` or a local `dto/` directory. DTOs must extend
`Z.class` from `zod-class` for runtime validation.

View File

@ -1,15 +0,0 @@
# Throw a typed error, not a plain Error
Applies to: backend packages (`cli`, `@n8n/db`, `core`, `workflow`) and the node packages.
The `no-plain-errors` lint rule is switched off repo-wide, so nothing catches
this. Flag `throw new Error(...)` in new code and pick by cause:
- `UserError` — the user caused it: invalid input, unauthorized action,
business-rule violation
- `OperationalError` — transient and expected: a failing network request, a DB
timeout; something to handle gracefully
- `UnexpectedError` — a bug: logic mistake, unhandled case, failed assertion
Do not flag `ApplicationError`; the `no-application-error` ESLint rule already
fails the build for it.

View File

@ -1,10 +0,0 @@
# `any` where a real type is available
Applies to: `packages/cli`, `packages/workflow`, `packages/nodes-base`,
`packages/@n8n/nodes-langchain`.
`@typescript-eslint/no-explicit-any` is downgraded to a warning in these
packages, and `pnpm lint` runs `--quiet`, so warnings never surface.
Flag a new `any` there when `unknown` plus a type guard, or an existing
interface, would do. Do not flag `any` in test code.

View File

@ -1,24 +0,0 @@
# Hand-rolled delays the linter cannot see
Applies to: backend packages (`cli`, `@n8n/db`, `core`, `workflow`) and the node packages.
`no-restricted-sleep-definition` catches only helpers literally named `sleep` or
`sleepWithAbort`, so it misses the same thing under another name.
Flag a delay re-implemented as `wait`, `delay`, `pause`, `waitFor`, … whose
entire body is a promise wrapping `setTimeout`, and an inline awaited
`new Promise((resolve) => setTimeout(resolve, ms))`.
Use `sleep(ms, abortSignal?)` from `@n8n/utils/sleep`. It takes the abort signal,
so an abort-aware local waiter is a violation too.
Do NOT flag:
- Wrappers that do more than wait: racing another promise, rejecting with a
timeout error, retry backoff, debounce/throttle
- Bare `setTimeout` used for scheduling rather than awaited as a delay
- Fake timer helpers such as `vi.advanceTimersByTime`
- Helpers literally named `sleep`/`sleepWithAbort`, and `sleep` imported from
`n8n-workflow` — ESLint already fails the build for both
- `packages/@n8n/node-cli/**` and `packages/@n8n/typeorm/**`, which cannot depend
on `@n8n/utils`

View File

@ -1,11 +0,0 @@
# Lazy-load heavy or native modules
Applies to: backend packages (`cli`, `@n8n/db`, `core`, `workflow`) and the node packages.
A top-level `import` of a module used only on a specific code path loads it into
every process at startup, raising baseline memory. Native modules (e.g.
`isolated-vm`) can crash instances that lack the binary. Large parsers (e.g.
`jsdom`, ~16 MB heap) waste memory when the path is rarely hit.
Fix: `await import()` at point of use. For barrel files, use `export type`
instead of a value re-export when consumers only need the type.

View File

@ -1,14 +0,0 @@
# License enforcement
Applies to: `packages/cli`, `*.ee.ts` files, licensed modules.
Nothing fails the build when a paid feature ships ungated, so this is review-only.
- Endpoints in `*.ee.ts` reaching licensed features without `@Licensed` — a
scope decorator is a permission check, not a license check
- `@Licensed` not matching the feature (`LICENSE_FEATURES` in `@n8n/constants`)
- Bypassed quota checks, or missing `FeatureNotLicensedError` on an unlicensed
path
- Custom licensing middleware where the `@Licensed` decorator fits
- Decorator order: route decorator → `@Licensed` → scope decorators
- Enterprise code reachable outside `*.ee.ts` files or licensed modules

View File

@ -1,32 +0,0 @@
# Migration conventions and tests
Applies to: `packages/@n8n/db/src/migrations/**`, `packages/cli/test/migration/**`.
`.agents/skills/db-migrations/SKILL.md` is the standard; prefer it over your
priors. Read a recent migration for the local idiom.
Flag:
- a hand-written table prefix or quoted identifier. Use `escape.tableName()` and
`escape.columnName()`;
- `queryRunner.query()` where `runQuery()` belongs, or `console.log` where the
context `logger` belongs;
- a value import of an entity, or a cross-package import;
- a timestamp that is not above every existing migration;
- a hand-edited generated index file;
- an 80-line `up()` that needs named private methods.
Precedent beats generic style opinion, but it is not a defence. An older migration
doing it wrong makes the finding informational.
## Tests
A data migration needs an integration test. Reading cannot verify its assumptions
about row shape, JSON and NULLs. Say so when there is none. A good test sets up
with `initDbUpToMigration`, runs with `runSingleMigration`, and covers both
engines, `down()`, and dirty rows. A happy-path test, on a migration that claims
to handle bad data, is worth a comment. For a schema-only migration the DSL calls
are enough, so a missing test is informational.
Regenerate the migration index files and the schema docs in `docs/generated/` when
the schema changes. Check the changed file list before you call one stale.

View File

@ -1,31 +0,0 @@
# Postgres and SQLite compatibility
Applies to: `packages/@n8n/db/src/migrations/**`.
A migration in `common/` runs on both engines. Judge them separately. A change
that is safe on Postgres can lose data on SQLite.
## The CASCADE trap on SQLite
Six DSL helpers recreate the table on SQLite: `addColumns`, `dropColumns`,
`addNotNull`, `dropNotNull`, `addEnumCheck`, `dropEnumCheck`. TypeORM copies the
rows to a temp table, drops the original and renames. If another table has an
incoming `ON DELETE CASCADE` foreign key, that `DROP` **deletes its rows**.
This is the most valuable check on any SQLite migration. The
`{ recreatesOnSqlite: true }` argument is an acknowledgement, not a fix. Check the
target table for incoming CASCADE foreign keys. If it has them, the migration
needs a `sqlite/` subclass with `withFKsDisabled = true as const`, or raw
`ALTER TABLE ADD COLUMN` when every new column is nullable or has a default. Say
so if you cannot check.
## Everything else
In a `common/` migration, flag Postgres-only SQL — `ALTER COLUMN … TYPE`,
expression indexes, `gen_random_uuid()` — raw SQL that assumes one engine's
boolean literal or quoting, and `INSERT OR REPLACE` where `ON CONFLICT DO NOTHING`
was meant. `varchar(N)` enforcement, transactional DDL, JSON and timestamp
handling, and NULL in unique constraints all differ.
A dialect migration belongs in `postgresdb/` or `sqlite/`. Read the DSL source
before you claim how a helper behaves.

View File

@ -1,27 +0,0 @@
# Data safety
Applies to: `packages/@n8n/db/src/migrations/**`.
Think of a database last upgraded two years ago. It holds dirty rows, unexpected
NULLs, retired enum values and malformed JSON.
Flag:
- a drop in the same release the code stopped writing the column. Expand-contract
must finish first;
- rows deleted with no fallback copy;
- a table copy with no row-count check;
- a backfill that overwrites the old value with no way back, in a
`ReversibleMigration`;
- `JSON.parse` with no `try`/`catch`;
- iteration with no `Array.isArray`, or a property read that assumes the property
exists;
- a backfill that cannot be retried after a partial failure;
- ordering that is assumed, not enforced;
- a `down()` that is empty, or does not restore the previous state.
`IrreversibleMigration` is correct only when `up()` destroys data that `down()`
would need, and the reason is stated. It is not a way to skip a tedious `down()`.
Do not flag a hazard the code handles. A guarded parse, a batched update or a
`withFKsDisabled` subclass is the fix.

View File

@ -1,41 +0,0 @@
# Necessity and schema design
Applies to: `packages/@n8n/db/src/migrations/**`.
## Should it exist?
Flag:
- a column or table that duplicates one that exists;
- state that application code or a read-time value could hold instead;
- a backfill that lazy computation would avoid, or a denormalised copy with no
read pattern that needs it;
- a drop before expand-contract is complete;
- two unrelated changes in one file;
- an edit to an already-merged migration. Add a new one instead.
## Is the shape right?
Use the narrowest type that fits the value:
- a numeric type for numbers and byte counts, never `varchar`;
- native `uuid`, not `varchar(36)`;
- `timestampTimezone()` or `timestampNoTimezone()`. Plain `.timestamp()` is
deprecated;
- `json` for structured data, `text` for unbounded user strings;
- never `double` for a version field.
Also flag:
- a reference column with no foreign key, a foreign key with no `onDelete`, or
`ON DELETE SET NULL` on a `NOT NULL` column;
- a missing primary key, or one unlike the adjacent tables;
- a default that is wrong for existing rows;
- an enum-like string with no CHECK constraint;
- a standalone index on a boolean, or one that repeats a unique constraint. A
partial index with a `WHERE` clause is correct. Do not flag it;
- an ID type that does not match the column it joins to.
Read the entity in `packages/@n8n/db/src/entities/`. Its type, nullability,
default, `@Index` and FK must agree with the migration. If you cannot read it, say
parity is unverified rather than guess.

View File

@ -1,25 +0,0 @@
# Performance and scale
Applies to: `packages/@n8n/db/src/migrations/**`.
`execution_entity` and `workflow_entity` hold millions of rows. Never assume a
table is small.
Flag:
- an unbounded `SELECT` into Node. Use `runInBatches`;
- row-by-row updates where one `UPDATE … FROM` does the work;
- a dataset in memory that grows with the table;
- a table rewrite on a hot table: `ALTER … TYPE`, `SET NOT NULL`, SQLite
recreation;
- an index built on a huge table;
- a `LIKE '%…%'` scan over a JSON column;
- a large blob column added to a hot row.
Every index slows every write, so an index with no query to serve is a finding.
But grep for the table and column first. Without that grep, say the query pattern
is unverified.
A nullable column with no backfill is cheap on Postgres. Do not comment on it.
On SQLite it is cheap only when added with raw `ALTER TABLE ADD COLUMN`. The DSL
`addColumns` recreates the whole table. On a hot table, flag that.

View File

@ -1,21 +0,0 @@
# Wrapping a third-party component
Applies to: `packages/frontend`.
`@n8n/design-system` wraps reka-ui, Element Plus, TipTap and CodeMirror. The
recurring defect is assuming the wrapper passes something through, or that a
library default is what you want.
- **Attributes are forwarded only if the wrapper forwards them.** `N8nSelect`
binds just `$props` and `on*` listeners to `ElSelect`, so an `aria-label`,
`data-test-id` or native attribute set on it lands on the outer `<div>` and
never reaches the control. Check the wrapper's `v-bind`, and prefer its own
prop where one exists.
- **Library defaults are behaviour.** A dialog focus trap focusing the first
tabbable child opens a filterable select on mount. Flag a handler
suppressing a library behaviour unconditionally when one state needs it,
e.g. a blanket `@interact-outside` `preventDefault`.
- **Vendor class names are contracts.** A `:global()` selector against one
stops matching silently when the library renames it.
- An accessible name computing to empty for an icon-only control, or a
`<button>` with no explicit `type`, which is `submit` inside a form.

View File

@ -1,42 +0,0 @@
# Design system
Applies to: `packages/frontend`.
Stylelint checks the *grammar* of a custom-property name, at warning severity.
Nothing checks that a token exists or is the right one, and `var(--gone)`
resolves to nothing rather than failing. Wrong tokens are the most common
shipped frontend regression in this repo.
Three layers, narrowest first; reaching past the narrowest that fits is the
defect:
1. component-scoped: `--button--*`, `--callout--*`, `--input--*`, `--tag--*`,
`--node--*`, `--canvas-node--*`, `--sticky--*`
2. semantic: `--background--*`, `--text-color--*`, `--border-color`
3. primitives, in px: `--spacing--` 5xs 2, 4xs 4, 3xs 6, 2xs 8, xs 12, sm 16,
md 20, lg 24, xl 32, 2xl 48, 3xl 64, 4xl 128, 5xl 256; `--radius--` 4xs 2,
3xs 4, 2xs 6, xs 8, sm 12, md 16, lg 20, xl 24, 2xl 32, `--radius--full`
Flag, strongest first:
- A `var()` naming a token that no longer exists, e.g. a legacy `--color--*`
kept through a refactor. It renders as nothing, silently.
- A hard-coded px, rem, hex, `rgb()`, ms or `z-index` literal where a token
covers it. Name the token.
- A generic token where a component-scoped one exists, e.g. a `--color--*`
background on an input that has `--input--color--background`.
- A px target stated in the diff or a comment that the chosen token misses.
The scales share suffixes at different values: `--radius--xs` is 8px,
`--spacing--xs` is 12px. "Design calls for 8px" beside `--radius--lg` (20px)
is a real defect.
Soft warning on a plain token swap: ask the intent, since a deliberate change
and a mistake look identical.
Flag a hand-rolled control duplicating an `N8n*` component (icon button,
tooltip, dialog, select), which arrives missing hover, focus and disabled
states, and a NEW direct `element-plus` or `reka-ui` import in `editor-ui`.
No lint rule covers those and many files predate the convention, so judge
only the import being added.
Do NOT comment on Storybook story titles, args, or doc cross-references.

View File

@ -1,18 +0,0 @@
# i18n
Applies to: `packages/frontend`.
`baseText(key, { interpolate })` takes an untyped record and vue-i18n leaves
an unmatched placeholder in the output. Nothing checks those names against
`en.json`.
- An `interpolate` key not matching the `{placeholder}` in that string,
including a case difference such as `docUrl` against `{docURL}`. The user
sees the raw placeholder or nothing.
- `i18n.baseText` passed as a bare function reference. It is a class method
reading `this`, so detaching it returns the key instead of the translation.
Pass the `i18n` object.
- Punctuation, separators or brackets hardcoded around a translated string in
a template. Composition differs per locale; put them in the string.
Do NOT flag a missing non-`en` translation, or wording itself unless it
contradicts copy the same PR adds.

View File

@ -1,43 +0,0 @@
# Vue correctness
Applies to: `packages/frontend`.
These fail silently: `vue-tsc` and ESLint pass, the app runs, the value or the
DOM is wrong.
## List identity
The editor patches long-lived lists in place, so `:key` decides whether Vue
reuses the wrong row. Flag an index `:key` on a list that can be reordered,
filtered or shortened, where revealed fields, inputs and refs then follow the
position instead of the item; a key omitting part of what identifies the
instance, e.g. a session key without the execution id, so switching
executions reuses the old component's state; and a key regenerated every
render. Index keys are fine on an append-only list.
## Reactivity
- A `ref` or `computed` read without `.value` outside a template. Templates
unwrap, script does not, so you hold the wrapper: `activeNode?.type` reads
`undefined` instead of the type, without throwing.
- A reactive object spread, cloned or deep-merged. `{ ...store }`,
`structuredClone` and recursive merge helpers detach it, and a merge into a
shared defaults object leaks one call into the next.
- An `injectStrict` helper such as `injectNDVStore` called outside `setup()`.
## State written after an await
Navigation swaps the active workflow document, and dialogs, threads and NDV
panels unmount mid-request. Flag a continuation writing to a store,
`localStorage` or the route without re-checking what it captured before the
`await`: navigating first and persisting afterwards what the destination was
meant to read, clearing a selection the user has since changed, toasting for a
closed dialog. Ask for a revalidated reference or a request sequence number,
never a lock.
## Payload shape
Backend responses and `/rest/settings` omit optional sections entirely. Flag a
new getter or computed walking more than one level into a response without
optional chaining, e.g. `settings.value.license.planName`. It throws during
boot and takes the view with it.

View File

@ -1,32 +0,0 @@
# Base images and pinning
Applies to: `docker/images/**/Dockerfile*`.
## A tag and its digest change together
Base images are pinned as `tag@sha256:...`. The tag is documentation; the digest
is what actually resolves. Flag a diff that edits one without the other — a
bumped tag with a stale digest silently keeps building the old image, and a
bumped digest under an old tag makes the Dockerfile lie about what it runs.
```dockerfile
ARG BUILDER_IMAGE=node:26.7.0-alpine3.24@sha256:aadf41...
ARG RUNTIME_IMAGE=n8nio/base:26.7.0@sha256:336873...
```
Also flag a base image reference that loses its digest entirely, and a
`FROM`/`ARG` that introduces a floating tag such as `latest`, `alpine`, or a
major-only version.
## Node version consistency
`ARG NODE_VERSION`, the builder tag, and the runtime base tag describe the same
Node release. Flag a bump that moves one and leaves another behind.
## The runtime base only changes here
A separate workflow builds `n8nio/base`; a new base reaches the n8n image only
when `RUNTIME_IMAGE` changes in this file. Flag a PR that expects a base change
to arrive on its own, and flag a runtime stage that installs a compiler or
build dependency — the runtime base ships without one deliberately, and build
tooling belongs in a builder stage that is discarded.

View File

@ -1,46 +0,0 @@
# Native module builds in the Docker images
Applies to: `docker/images/**/Dockerfile*`.
n8n compiles `sqlite3`, `isolated-vm`, and `@confluentinc/kafka-javascript`
inside the image. Each line of that setup exists because a specific build broke.
Flag a change that undoes one.
## Never `npm rebuild` a native module in a pnpm tree
A pnpm `node_modules` reaches the same package through several symlinks, and
`npm rebuild` runs one install script per link, concurrently, in the same store
directory. They collide on `build/node_gyp_bins` and on each other's make
output. This produced an intermittent Docker build failure twice, for sqlite3
and for isolated-vm.
Call node-gyp once, directly:
```dockerfile
node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js rebuild --release
```
Flag `npm rebuild <pkg>` or `npm install` in a Dockerfile stage that builds a
native module.
## Use npm's bundled node-gyp, not `npx node-gyp`
`npx node-gyp` resolves a version at build time, so the toolchain drifts from
the pinned image digest and builds stop being reproducible. The path above
pins node-gyp to whatever the base image ships. Flag `npx node-gyp`.
## Do not reintroduce prebuilds
`node-gyp-build` prefers `prebuilds/` over `build/Release`, and reads
`/etc/alpine-release` to detect musl. The hardened Alpine base has no such
file, so the loader picks the glibc prebuild and the module crashes at runtime.
The `rm -rf prebuilds` steps are load-bearing — flag their removal, and flag a
new native module added without one.
## Keep the native compile above `COPY ./compiled`
`isolated-vm` builds in its own stage whose only cache input is the module
source, specifically so an application change does not trigger a recompile.
Flag a `COPY` of application code moved above a native build step, or a native
build folded into the stage that copies `./compiled` — it silently turns a
cached layer into a multi-minute compile on every build.

View File

@ -1,45 +0,0 @@
# Ratchets and allowlists
Applies to: baseline files, `packages/cli/eslint.config.mjs`, lint configs.
Several checks are ratchets: they permit what already exists and fail only on
growth. That design has one blind spot — adding the new violation to the
baseline is indistinguishable, to the tool, from fixing it. Only review catches
it.
Flag a diff that **adds** entries to any of these, and ask for the fix instead:
| File | Ratchet |
|------|---------|
| `.code-health-baseline.json` | `@n8n/code-health` violations |
| `.boundaries-baseline.json` | `turbo boundaries` issue count |
| `packages/quality/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
| `.code-health-baseline.json` | the `typeorm-persistence-boundary` baseline, which only shrinks as business logic moves queries into repositories |
| `packages/cli/eslint.config.mjs` | the public-API allowlists captioned "NEVER add to this list" |
| `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
Removals are the healthy direction and need no comment.
Two specifics:
- `.boundaries-baseline.json` holds a single integer, not a fingerprinted list.
A fixed issue and a new one at the same count cancel out and the ratchet stays
green. Treat any change to that number as worth explaining.
- Regenerating a baseline wholesale, rather than appending to it, hides growth
inside a large diff. If a PR rewrites a baseline, ask what the net change to
the violation count is.
## Downgrading a rule is the same move
Flag a lint rule moved from `'error'` to `'warn'` or `'off'`, and a new
`eslint-disable` for one of the guarded rules. Most packages run
`eslint . --quiet`, so a warning never fails CI — downgrading to `'warn'` is
functionally deleting the rule while appearing to keep it.
In a package `eslint.config.mjs` the `lint-config-layering` code-health rule
catches this for you: a package-wide downgrade fails Static Analysis unless it
is already in the baseline. What it cannot judge is a downgrade dressed as a
scoped block. Check that a new `files` glob names the paths that actually need
the exception, and that a rule retired in `@n8n/eslint-config/src/configs/base.ts`
comes with the count of packages that had already stopped enforcing it
(`node scripts/lint-parity/majority.mjs`).

View File

@ -1,46 +0,0 @@
# GitHub Actions safety
Applies to: `.github/workflows/**`, `.github/actions/**`, `.poutine.yml`.
Poutine and Zizmor already fail CI for the classic hazards — `pull_request_target`
(also an error-level `@n8n/code-health` rule), actions unpinned from a full commit
SHA, untrusted checkout execution, self-hosted runner exposure. Do not repeat
them. What follows is what the scanners cannot see.
## Silencing a scanner is the change to review
`.poutine.yml` carries two allowlists — `github_action_from_unverified_creator_used`
(by purl) and `untrusted_checkout_exec` (by path). The file says "Add new
entries only after security review". Nothing enforces that.
Flag any addition to `.poutine.yml`'s `skip:` block, any new `zizmor: ignore`
comment, and any new entry in `.github/poutine-rules/`'s exceptions. Ask what
was reviewed and why the finding is a false positive rather than a real one.
The entry needs a comment giving the reason, matching the ones already there.
## A gate that cannot fail is not a gate
Required checks are assembled by the `ci-filter` action in `mode: validate`,
fed from a `needs:` list. Flag:
- `continue-on-error: true` on a job that appears in a required-checks `needs:`
list, or on the final validate step. It makes the gate green regardless of
outcome. `continue-on-error` on a nightly or a notification step is fine.
- A job removed from a required-checks `needs:` list without the PR saying why.
- `if:` conditions on a required job that can silently skip it. A skipped
required job and a passing one are hard to tell apart in the merge queue.
## Path filters decide whether a test runs at all
Jobs gated on a `ci-filter` filter only run when a matching file changed, so a
filter that is too narrow means the test quietly stops covering new code. When
a PR adds a directory that an existing filter was meant to cover — a new package
under `packages/quality/`, a new script under `.github/scripts/` — check the
filter still matches it.
## Least privilege
Every workflow declares a top-level `permissions:` block; without one it runs
with the repository's broad default token. Flag a new workflow that omits it,
and a job that widens `permissions` beyond what its steps use. A job calling a
reusable workflow must grant at least what that workflow declares.

View File

@ -1,18 +0,0 @@
# Authentication and access control
Applies to: `packages/cli`, `packages/@n8n/db`.
## Authentication and sessions
- JWT validation or expiration weakened; cookie flags (HttpOnly, Secure, SameSite) missing or disabled; MFA bypassed; SSO/SAML/OIDC validation gaps; OAuth 2.0 without PKCE or a weak redirect check
## Authorization
`@n8n/code-health`'s `endpoint-scope-coverage` rule is disabled, so an undecorated route ships unless a reviewer catches it.
- Every authenticated route needs `@GlobalScope` or `@ProjectScope`, unless it declares `skipAuth`, `allowUnauthenticated`, or `apiKeyAuth`
- A decorator proves the caller may do this somewhere, not that the row is theirs. Ownership belongs in the query — a `find…ForUser` finder, or the project filter alongside the id — never a comparison afterwards
- An endpoint querying a repository directly while its siblings use a project-aware service — the public API reaching past it is the usual shape
- Middleware treating an authenticated session as sufficient (`if (req.user) return next()`), skipping the ownership check behind it
- A scope parameter defaulting to the weakest value, or a role map gaining write or execute: mutating call sites must ask for the stronger scope
- Subworkflow execution bypassing caller policies

View File

@ -1,17 +0,0 @@
# Code execution and sandboxing
Applies to: `packages/cli`, `packages/core`, `packages/workflow`, Code and Merge nodes.
Every sandbox n8n has shipped was eventually escaped, so the bar is reachability: if sandboxed code can reach it, assume it can climb out.
Flag changes that:
- Give a sandboxed context new access to Node.js builtins, `process.env`, dynamic `require`/`import`, the filesystem, or the network
- Add a global, helper, or bridge callable from inside the sandbox that hands back a host-realm value — one returned object, thrown error, or callback is enough to walk `constructor` back out
- Weaken a prototype sanitizer, a blocklist, or the allowed globals
- Weaken file access restriction enforcement, or let unsanitized input reach a file path or an n8n internal directory
Two guards that did not hold before:
- `typeof value === 'string'` before a blocklist check: `['__proto__']` reports `'object'` yet coerces to `__proto__` as a key. Compare `String(value)`
- An AST check covering computed member expressions only — the same name also arrives as a call argument or a template literal, neither a string literal

View File

@ -1,13 +0,0 @@
# Credentials and secrets
Applies to: all backend packages.
Flag:
- Credential values reaching logs, error messages, response headers, or a REST response — a decrypted field returned "because the UI needs it" is the shape
- Hardcoded secrets, API keys, or tokens
- Credential encryption weakened, or `N8N_ENCRYPTION_KEY` reused for token signing, OAuth state, or anything but credential encryption
- OAuth state/CSRF validation bypassed, or webhook requests keeping auth cookies
- Credential access ignoring scope boundaries (instance/project/user), or a credential resolved by name and type with no ownership filter
A caught integration error interpolated into a message is a common way credential material reaches a status field or an API response.

View File

@ -1,34 +0,0 @@
# Node input safety
Applies to: `packages/nodes-base`, `packages/@n8n/nodes-langchain`. Skip this
file for other packages.
## Prototype pollution via node parameters
ESLint does not cover this: `no-prototype-builtins` is only a warning here and `pnpm lint` runs with `--quiet`.
Flag NEW code where a value from `this.getNodeParameter(...)` — directly or via a variable, a callback param, or a helper — is a computed key BUILDING A NESTED OR CONTAINER STRUCTURE on a plain object (`IDataObject` or object literal, not a `Map` or `Object.create(null)`):
1. Container creation: `obj[key] = {}` or `obj[key] = []`
2. Nested write where the untrusted value is a key: `obj[k1][k2] = value`
3. The same via `obj[key] ??= {}` / `obj[key] ||= []`
The carrier is usually a shared accumulator; a deep merge of user options into a config object is the same defect.
Do NOT flag:
- Reads: `const x = obj[key]`, `if (obj[key] === undefined)` — reads don't pollute
- Single-level writes of a concrete (non-object) value, e.g. `item.json[field] = value` — a primitive assigned to `__proto__` is a no-op. This is the common, benign case in nodes; flagging it is noise
- Keys that are literals, or validated by `isSafeObjectProperty(key)`
- Writes routed through `setSafeObjectProperty(...)`
Use `setSafeObjectProperty` / `isSafeObjectProperty` from `n8n-workflow`, or a `Map`, and coerce with `String(key)` before any check.
## Injection through node parameters
Flag NEW code where a parameter reaches a sink without escaping:
- SQL values concatenated or expression-interpolated instead of bound. An expression is interpolation, not parameterisation
- SQL identifiers — table, column, `ORDER BY`, a cast suffix after `name:`. They cannot be bound, so they need an allowlist; numeric fields a numeric cast
- A SQL dialect reaching JavaScript (AlaSQL in the Merge node) gaining a new construct that can call a function
- Command execution with unsanitized shell arguments

View File

@ -1,15 +0,0 @@
# Outbound requests
Applies to: `packages/cli`, `packages/core`, `packages/nodes-base`,
`packages/@n8n/nodes-langchain`.
A URL is user-controlled unless the code shows otherwise. The recurring finding is not a weak allowlist but a working check the new path never calls: `n8n-workflow` exports `assertUrlAllowed`, `assertCredentialAllowsUrl` and `getCredentialAllowedDomains`, and a request surface reaching none of them is the finding.
Flag NEW code where:
- A request built from user input reaches an HTTP client unchecked
- A later hop inherits the first hop's approval — pagination `next` links, redirects, `.well-known` discovery, webhook callbacks. Per-hop checks need the allowlist `getCredentialAllowedDomains` returns
- Credential material (`Authorization`, cookies, signed headers) reaches a host the check did not cover
- The URL arrives in a field that reads like configuration but the user writes: `requestTokenUrl`, `accessTokenUrl`, OAuth `serverUrl`, SAML `metadataUrl`, a region or host fragment reaching an SDK endpoint
- A URL is checked by regex rather than `new URL()`; regexes miss `@`-userinfo, `#` fragments and IPv6 literals
- Certificate validation is dropped (`rejectUnauthorized: false`)

View File

@ -1,15 +0,0 @@
# Rendering untrusted content
Applies to: `packages/frontend`, and backend paths serving HTML or user bytes.
Workflow names, node parameters, and execution data are author-controlled
strings, rendered in a session with full REST access.
Flag NEW code that:
- Passes user-derived content to `v-html`, `innerHTML`, `insertAdjacentHTML`, or a render function concatenating markup, with no sanitizer in front of it
- Widens a sanitizer allowlist — tags, attributes, `on*` handlers, `srcdoc` — or drops a sanitize call from an existing render path
- Puts a user-supplied value into `href`, `src`, or a `window.open` target without rejecting non-`http(s)` schemes. `javascript:` and `data:` survive naive checks; parse with `new URL()`, never a regex with the `m` flag, which makes `^` match any line start
- Renders markdown or HTML returned by a node, an AI response, or a webhook body as trusted markup
- Serves user-controlled bytes from the n8n origin without `Content-Disposition: attachment` or an equivalent inline-render guard
- Loosens an iframe `sandbox` attribute or a CSP directive on a view or response carrying user content

View File

@ -1,21 +0,0 @@
# Test coverage
Applies to: any package with a test suite.
Flag new behaviour that ships with no test — a service method, controller
route, node operation, store action, or composable. Coverage does not need to
be complete; core functionality and the critical path do.
Also flag a new or edited test whose assertion cannot tell the fixed behaviour
from the broken one, since the suite going green is then the only signal
anyone gets: a name stating the opposite of what is asserted, an assertion
that holds whether or not the invariant it names does, or a case duplicating a
sibling on the same path but for a mock the code never reads.
Do NOT flag:
- Exports, types, configuration objects, metadata, version files
- A percentage — the number is not the bar, the critical path is
- Edge cases a human reviewer is better placed to ask for
- A skipped test — `no-skipped-tests` already fails the build
- A change breaking an existing test elsewhere; the suite runs on every PR

View File

@ -1,24 +0,0 @@
---
name: n8n:agent-design-language
description: Design or review Agent Builder interfaces. Use for Agent configuration patterns, modal flows, responsive behavior, status treatments, and other Agent-only UI work.
---
# Agent design language
Use this skill only for interface work in
`packages/frontend/editor-ui/src/features/agents`.
Before you change code:
1. Read `packages/frontend/editor-ui/src/features/agents/AGENTS.md`.
2. Read `packages/frontend/editor-ui/src/features/agents/agents-design-language.md`
completely.
3. Use [ui-design](../ui-design/SKILL.md) for visual implementation rules.
4. Use [content-design](../content-design/SKILL.md) when you add or change UI
text.
Keep Agent-specific components in the Agent feature. Do not change the Design
System unless the user explicitly asks for this change.
Update the design language document when you establish a reusable Agent
interface pattern. Add or update tests for the behavior that you change.

View File

@ -113,7 +113,7 @@ Run when `AutoReject` is `null`. Full rules for each in `reference/checks.md`:
## Step 4 — Identify the responsible team
Run `node .github/scripts/owners/owners.mjs` against the changed file list and map the winning GitHub team to a Linear team. Full mapping table, sub-agent fallback procedure, and label rules: see `reference/teams.md`.
Run `node .github/scripts/owners.mjs` against the changed file list and map the winning GitHub team to a Linear team. Full mapping table, sub-agent fallback procedure, and label rules: see `reference/teams.md`.
## Step 5 — Extract the Linear ticket

View File

@ -16,7 +16,7 @@ Owner resolution, the GitHub-team → Linear-team → label mapping, and the Lin
## Identifying the owning team
Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parses `OWNERS` with last-match-wins semantics and returns allocations sorted by file count with a `share` percentage. Using the script keeps this skill consistent with whatever CI uses.
Use the canonical owners script at `.github/scripts/owners.mjs`. It parses `.github/OWNERS` with last-match-wins semantics and returns allocations sorted by file count with a `share` percentage. Using the script keeps this skill consistent with whatever CI uses.
1. Write the PR's changed file paths (from the `files` list) to a temp file, one per line:
```bash
@ -24,7 +24,7 @@ Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parse
```
2. Run the script:
```bash
node .github/scripts/owners/owners.mjs /tmp/pr-<number>-files.txt
node .github/scripts/owners.mjs /tmp/pr-<number>-files.txt
```
3. The script prints JSON of the form:
```json
@ -38,10 +38,10 @@ Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parse
```
Allocations are already sorted by `fileCount` descending — take the first entry as the winning team.
4. Clean up: `rm /tmp/pr-<number>-files.txt`.
5. Strip the `@n8n-io/` prefix from `allocations[0].team` — the GitHub team slug is `nodes`, `iam`, `ai`, etc. If `allocations` is empty (no file matched any rule, which is possible only if `OWNERS` lost its catch-all), fall back to `catalysts`.
5. Strip the `@n8n-io/` prefix from `allocations[0].team` — the GitHub team slug is `nodes`, `iam`, `ai`, etc. If `allocations` is empty (no file matched any rule, which is possible only if `.github/OWNERS` lost its catch-all), fall back to `catalysts`.
6. Map the GitHub team slug to its Linear team name and PR label using the table below. The `team` field in the JSON output is the **Linear team name**. If the resolved GitHub team slug has no entry in the table, fall back to `Engineering`.
**Sub-agent fallback**: if `node` execution is denied by the sandbox, read `OWNERS` directly and apply last-match-wins by hand. All the active rules fit on one screen.
**Sub-agent fallback**: if `node` execution is denied by the sandbox, read `.github/OWNERS` directly and apply last-match-wins by hand. All the active rules fit on one screen.
## GitHub team → Linear team → GitHub label
@ -54,7 +54,7 @@ Use the canonical owners script at `.github/scripts/owners/owners.mjs`. It parse
| `design` | Design | `team:design` |
| `iam` | Identity & Access | `team:identity` |
| `ligo` | Lifecycle & Governance | `team:lifecycle` |
| `ai-assistant` | AI Assistant | `team:instance-ai` |
| `instance-ai` | AI Assistant | `team:instance-ai` |
| `frontend` | Adore | `team:adore` |
| `qa-dx` | Developer Platform | `team:qa-dx` |
| `migrations-review` | Catalysts | `team:cats` |

View File

@ -111,10 +111,6 @@ When suggesting new keys, follow the existing hierarchy. Browse nearby keys in
### Language and grammar
**ASD-STE100 Simplified Technical English.** Use short sentences, the active
voice, and one instruction for each sentence. Use one approved word for each
meaning.
**US English.** Always. No exceptions.
- Do: "categorizing", "color", "analyze"
- Don't: "categorising", "colour", "analyse"

View File

@ -13,10 +13,6 @@ Use this skill when you need quick reminders on critical patterns.
## Critical Rules (Must Follow)
**Technical writing (comments, PRs, issues, docs):**
- Write in ASD-STE100 Simplified Technical English: short sentences, the
active voice, one instruction for each sentence
**TypeScript:**
- Never `any` → use `unknown`
- Prefer `satisfies` over `as` (except tests)
@ -40,14 +36,10 @@ throw new UnexpectedError('message', { extra: { context } });
- Dependency injection via `@n8n/di`
- Config via `@n8n/config`
- Zod schemas for validation
- Pagination args: use `offset` + `limit` in controllers and services; translate to TypeORM `skip`/`take` only inside repositories
**Testing:**
- Vitest (unit), Playwright (E2E)
- Mock external dependencies
- Keep filesystem tests in a test-owned temporary directory
- Set `N8N_USER_FOLDER` before importing settings code
- Trace branches activated by mocks and isolate every reachable mutation
- Work from package directory: `pushd packages/cli && pnpm test`
**Database:**
@ -72,8 +64,7 @@ pnpm lint # Before commit
| Package | Purpose |
|---------|---------|
| `packages/cli` | Backend API |
| `packages/frontend/editor-ui` | Vue 3 frontend shell |
| `packages/modules/<name>/frontend` | Frontend feature modules. Guide: `packages/@n8n/module-cli/frontend-module-guide.md` |
| `packages/frontend/editor-ui` | Vue 3 frontend |
| `packages/@n8n/api-types` | Shared types |
| `packages/@n8n/db` | TypeORM entities |
| `packages/workflow` | Core interfaces |

View File

@ -1,377 +0,0 @@
---
name: n8n:create-agent-builder-eval
description: >-
Authors and calibrates Instance AI evaluations that build standalone n8n
Agents through Agent Builder. Use when a change under
packages/cli/src/modules/agents affects build-agent routing, Agent setup,
model or credential selection, tools, MCP servers, integrations, skills,
tasks, testing, or user-facing build responses. Requires LangTracer access
before authoring so each finished case can be published.
---
# Create an Agent Builder eval
Use the shared Instance AI eval harness. Agent cases use an Agent-specific
authoring directory, dataset, and LangTracer suite.
## Required LangTracer preflight
Run this check before sourcing, drafting, or writing an eval. Run it from
`packages/@n8n/instance-ai`:
```bash
pnpm exec dotenvx run -f ../../../.env.eval -- \
sh -c 'test -n "${LANGTRACER_URL:-}" && test -n "${LANGTRACER_API_KEY:-}"'
```
If the check fails, stop before creating an eval file. Ask the user to:
1. Generate a key on the
[LangTracer API page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
2. Add these variables to the repository root `.env.eval` file:
```env
LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
LANGTRACER_API_KEY=<generated-key>
```
3. Confirm when the environment is ready.
Do not ask the user to paste the key into chat. Do not print or inspect its
value. Rerun the check after the user confirms. Continue only when it passes.
## Non-negotiable routing
- Author the case at
`packages/@n8n/instance-ai/evaluations/data/agents/<slug>.json`.
- Set `"datasets": ["agents"]`.
- Ask the driver for the target suite before you do anything else. See
"Ask for the target suite first" below. Do not assume `agents`.
- Do not commit the case JSON. LangTracer is the durable source of truth.
- Commit changes to this skill, the harness, and CI when applicable.
The disk runner loads both `data/agents/` and `data/workflows/`. A misplaced
Agent case can therefore pass locally. That does not make the location correct.
## Decide what the case proves
Write the smallest user request that exercises the changed behavior.
- Use `processExpectations` for the Instance AI conversation and final response.
- Use `outcomeExpectations` for the created Agent artifact and its configuration.
- Use `executionScenarios` only when the built Agent must run to prove the behavior.
- Declare only the credentials that the build must see.
The harness captures the Agent configuration and authored skills. It supplies
them to the expectation judge. A scenario-less Agent case is valid when process
or outcome expectations can prove the behavior.
Use the substitution test for every expectation. A correct alternative build
must pass. A build that misses the requested behavior must fail.
## Write expectations about the Agent, not the builder's internals
Agent Builder runs as a delegated sub-agent architecture today. That design
can change. Do not write an expectation that depends on it. Follow
"Keep expectations free of internal mechanics" in
[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md).
Do not write:
- "The sub-agent runs/tests the Agent before it responds."
- "Instance AI delegates to `build-agent`" or "calls `build-agent` once."
- "The builder calls `call_agent` to verify the Agent."
- "The sub-agent reads the skill, then sets the model."
Write what a user or a reviewer can observe:
| ❌ mechanical | ✅ intent |
|---|---|
| "The sub-agent tests the Agent before it reports" | "The final response says the Agent was tested only if a test ran, and it reports the result of that test honestly" |
| "`build-agent` is called with the Slack credential" | "The Agent uses the Slack credential the user has" |
| "The sub-agent adds a tool for the lookup" | "The Agent can look up the order status when a user asks for it" |
| "The builder sets `model` after the catalog lookup" | "The Agent uses a model that the connected credential supports" |
Put each one where it belongs:
- The Agent's configuration, tools, skills, and behavior go in
`outcomeExpectations`.
- What the user is told, asked, or shown goes in `processExpectations`.
- The final response is the place to check claims ("says it is ready",
"lists the tools it added"). Do not check which component made the claim.
The calibration steps below ask you to confirm that `build-agent` ran. That is
a check for you, to be sure the case reached Agent Builder. Do not add it to the
case.
For multi-turn, seeded, or capability-gap cases, follow the case-shape and
calibration rules in [create-instance-ai-eval](../create-instance-ai-eval/SKILL.md).
This skill overrides its workflow directory and suite guidance for Agent cases.
The sections below add the Agent-specific parts. Do not skip the general rules
they point to.
## Ask for the target suite first
Follow "Ask for the target suite first" in
[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md). Do it after the
LangTracer preflight and before you source or draft anything. If the request
already names a suite, state it and do not ask. Otherwise ask, in autonomous
mode too, in the same message as the autonomy question.
For Agent cases, recommend
[Instance AI capabilities — agents](https://lang-tracer.n8n-maintenance.workers.dev/suites/10)
(slug `agents`). Offer `baseline` as well: the Instance AI (INS) team monitors
it and it runs nightly. Get the full list from `list_suites`. Use the suite the
driver picks in every push command, in place of `agents` in the examples below.
Keep the `agents` dataset and the `data/agents/` directory. They do not depend
on the suite.
## Set the autonomy level first
Follow "Set the autonomy level first" in
[create-instance-ai-eval](../create-instance-ai-eval/SKILL.md). Ask the driver
for *autonomous* or *checkpoint* mode before you source or draft anything. The
four gates are the same: selection, shape and expectations, calibration, push.
## Source the case from a real thread
The best Agent cases come from a real conversation. Discover the conversation
with LangTracer. Author a synthetic case from what you learn.
1. List candidates: `list_conversations` with `usedAgentBuilder: true`. Do not
use `source: "agent-builder"`. That pool is a different product, has
unscrubbed payloads, and has no Agent snapshots.
2. List what the thread did to the Agent: `list_conversation_agent_snapshots`.
`target-resolved` is the state that a turn opened on. `config-updated` is the
state after the builder changed it.
3. Pick the shape from the snapshot sequence (next section).
4. Fetch the state to seed with `get_conversation_agent_snapshot`. Seed from the
`target-resolved` row of the turn under test.
5. Read the raw turn before you write any expectation. Follow
"First reproduce, then reclassify" in the general skill.
Two snapshot patterns to look for:
- A `target-resolved` hash that differs from the previous `config-updated` hash.
The user edited the Agent in the UI between turns. This is the best seed for a
repair or update case.
- A run of turns that each end in `config-updated`. The user added one rule per
turn. Use it for a "new rule does not erase old rules" case.
Credentials arrive as `[redacted]`. Instructions and skill bodies do not. Scrub
the prose before it enters a case. See the `agents` section in
[case-shapes.md](../create-instance-ai-eval/case-shapes.md) (the `agents` seed section).
## Pick the shape: create or update
| Shape | Question it answers | How to write it |
|---|---|---|
| **Create** | Does the builder make the right Agent from a fresh request? | One user turn. No seed. Assert on the Agent artifact. |
| **Update** | Does the builder change only what the user asked for? | `seed.agents` with the Agent before the change. One live turn with the change. Assert on the new behavior and that the untouched parts survive. |
| **Repair** | Does the builder find the cause of a failure and fix it? | Seed the broken Agent. The live turn reports the failure. Assert on the cause found and the fix applied. |
| **Incremental rules** | Do earlier rules survive when a new rule lands? | Seed the Agent after several rules. Add one conflicting or overlapping rule. Assert that the old rules are still in force. |
Rules that apply to all four:
- An update that rebuilds the Agent from scratch is a failure. Assert on a
specific untouched instruction, tool, or skill that must still be there.
- Write the conversation in English, even when the source thread is not.
- Trim to the fewest turns that reproduce the behavior. Do not copy a long
thread.
- Put only the live turn in `conversation`. Put earlier turns in `seed.messages`.
## Confirm the precondition fired
A pass can mean the builder did the right thing. It can also mean the situation
never happened. Check before you trust a green.
- For "the builder did not do X", confirm that it reached the turn where X was
possible. A builder that never edited the Agent passes "did not delete the old
rule".
- For a repair case, confirm that the seeded Agent really has the defect. Read
the seeded config in the rendered artifact.
- For an update case, compare the seeded config with the final config. Confirm
that the requested change is the only difference you expected.
- For a "say you cannot" case, confirm that the tool or data was really missing
in the run.
## A red is a finding
Follow "A red is signal" in the general skill. For Agent cases, add this:
1. Classify the red as a product gap, a harness limitation, or non-determinism.
2. Keep a product-gap red. Start the case `description` with
`Capability-gap finding:`.
3. **Propose a Linear ticket for the Agent team.** The Linear team is named
`Agent`. Propose the ticket. Do not create it unless the driver says go.
Follow "Capability gap → propose a Linear ticket" in the general skill. Also
check whether the gap already has a ticket.
- Write the ticket in English. Use Simplified Technical English, as
[AGENTS.md](../../../AGENTS.md) requires. Translate the user's words. Do
not paste them.
- This repository is public. Keep customer names, company names, and real
URLs out of the ticket. Describe the use case in neutral words.
- Link the LangTracer case and the source thread. Do not copy raw user
messages or agent instructions from the thread.
- Reproduce the gap first. This is a default, not a hard rule. Propose a
ticket only after a case is red on a real build. Quote the failing
expectation and the judge reason. Follow "Reproduce first" in the general
skill.
- The driver may approve an exception, for example when the harness cannot
reach the mechanism after about three attempts. Then the ticket must say
"Not reproduced in the eval. Based on trace analysis."
4. **Check the builder's own guidance for the gap.** Read the files that steer
Agent Builder:
`packages/cli/src/modules/agents/builder/agents-builder-prompts.ts`,
`packages/cli/src/modules/agents/builder/prompts/`, and
`packages/cli/src/modules/agents/builder/skills/`. Also read the Instance AI
orchestrator prompt in `packages/@n8n/instance-ai/src/agent/system-prompt.ts`.
If none of them covers the failing situation, add that to the ticket as a
proposed prompt or skill change.
5. Propose the smallest added line first. Do not delete existing guidance to fix a
gap. A measured case: adding one line fixed a routing gap, and deleting two
lines did not.
6. Every prompt or skill change needs a before and after measurement. The new
case is the "before" run. Run it again after the change. Use `--iterations 5`.
When the failure is in the Agent's runtime (for example a channel that cannot
report an error), the builder cannot see it. The right fix is often a builder
instruction to say so and stop guessing. Write that as a `processExpectations`
item on the final response.
## Publish and link
Follow "Share links, never bare ids" and "Link the pushed case to its source" in
the general skill. For a sourced Agent case:
- Report the case, the suite, and the source thread as links. Use the
`https://lang-tracer.n8n-maintenance.workers.dev/test-cases/<id>` form.
- Call `update_test_case` with `sourceThreadId`, `expectedBehavior`, and
`failurePattern`.
- Add a capability tag with `add_case_tags`.
- Push a case whose *build* is wrong with `--set-kind capability_gap` into a
suite of that kind. If no such Agent suite exists, tell the driver. Do not put
it in a `regression` suite.
## Draft the case
Start with this shape:
```json
{
"description": "The Agent Builder behavior this case guards.",
"conversation": [
{ "role": "user", "text": "Build me an Agent that ..." }
],
"complexity": "simple",
"tags": ["agent", "agent-build", "<capability>"],
"credentials": [{ "type": "<credentialType>", "name": "<display name>" }],
"processExpectations": [
"The final response ..."
],
"outcomeExpectations": [
"A standalone Agent is the deliverable; any workflow created exists only as a tool the Agent calls.",
"The Agent ..."
],
"datasets": ["agents"]
}
```
Keep the prompt in the user's voice. Do not tell Instance AI which internal
tools or configuration fields to use unless that choice is the behavior under
test.
## Validate and run locally
Read [local-setup.md](local-setup.md) when the machine does not already have an
eval instance and environment file.
From `packages/@n8n/instance-ai`:
```bash
pnpm exec tsx -e "import { loadAgentEvalTestCasesWithFiles } from './evaluations/data/agents/index.ts'; const matches = loadAgentEvalTestCasesWithFiles('<slug>'); if (matches.length !== 1) throw new Error('Expected exactly one Agent eval case, found ' + matches.length); console.log(matches[0].fileSlug)"
pnpm eval:instance-ai \
--base-url http://localhost:5680 \
--filter <slug> \
--tier agents \
--concurrency 1 \
--keep-workflows \
--verbose
```
Use `eval:instance-ai` for a new disk case. `eval:agents` reads the published
LangTracer suite and is for running cases that are already there.
Inspect the transcript, the rendered Agent artifact, and each judge reason.
Do not accept a green result when a conditional expectation never occurred.
Do not weaken an expectation to hide a real Agent Builder defect.
## CI coverage
The Instance AI PR gate checks the files changed by the PR. A change under
`packages/cli/src/modules/agents/` selects the `Instance AI capabilities — agents`
suite through its `agents` slug. It also selects the `agents` dataset and uses
an absolute pass gate. The run uses a suite-scoped LangSmith cohort. It does not
write to the workflow dataset or compare against the workflow baseline. Other
Instance AI changes select the `baseline` suite and its `pr` dataset.
The gate runs when a PR opens, reopens, or becomes ready for review. It does not
run for each new push. Use the PR gate's manual dispatch after a later push.
## Credential behavior
Declared credentials are real n8n credential records with placeholder data.
The eval thread limits the builder to those credential IDs.
Agent Builder model catalog requests return deterministic fake models during an
eval. They do not decrypt the placeholder model credential or call its provider.
Production model catalog requests remain live.
This mock covers catalog lookup only. A builder `call_agent` action and an Agent
`executionScenario` run the target Agent model. They need a working provider
credential such as `EVAL_OPENAI_API_KEY`. A build-only case does not need one.
## Calibrate and publish
1. Run the case once with `--concurrency 1`.
2. Confirm that Instance AI called `build-agent`.
3. Confirm that the expected Agent artifact was captured.
4. Classify each red as a product gap, harness limitation, or non-determinism.
5. Use `--iterations 5` before adding a case to a gating tier.
6. Preview the LangTracer change.
7. Push it to the suite the driver chose (`<suite>` below).
```bash
pnpm exec dotenvx run -f ../../../.env.eval -- \
pnpm eval:langtracer-push --suite <suite> --dry-run --changed
pnpm exec dotenvx run -f ../../../.env.eval -- \
pnpm eval:langtracer-push --suite <suite> --changed
```
The push needs `LANGTRACER_URL` and `LANGTRACER_API_KEY`. Generate a key on the
[LangTracer API page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
Report the case and suite as clickable LangTracer links. Delete the local JSON
after a successful push.
## Completion checklist
- The LangTracer environment preflight passed before authoring.
- The driver chose the target suite before any work, including in autonomous mode.
- The autonomy level was set and stated before any work.
- A sourced case links to its source thread, and the source turn was read raw.
- Each conditional expectation has proof that its precondition fired.
- A kept product-gap red has a proposed Linear ticket and a check of the builder guidance.
- The case is in `data/agents/` and uses the `agents` dataset.
- The strict loader accepts it.
- A local run captures a standalone Agent.
- The transcript proves each process expectation was exercised.
- The rendered Agent artifact proves each outcome expectation.
- No expectation names a sub-agent, an internal tool, or a delegation step.
Each one passes for a correct build from a different architecture.
- Repeated runs are stable enough for the selected tier.
- The case is pushed to the suite the driver chose, and the report links to it.
- Agent Builder PR changes select the `agents` suite in CI. A case pushed to
another suite does not run in that lane.
- The local case JSON is not committed.

View File

@ -1,110 +0,0 @@
# Local Agent Builder eval setup
## Required LangTracer access
Generate a key on the
[LangTracer API keys page](https://lang-tracer.n8n-maintenance.workers.dev/account?section=api).
Add it to `.env.eval` at the repository root:
```env
LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
LANGTRACER_API_KEY=<generated-key>
```
Do not paste the key into chat or put it in a command. The main skill checks
that both variables exist before it creates an eval.
## Remaining environment
Create `.env.eval` at the repository root. Use one Instance AI model provider
and one sandbox provider.
```env
N8N_INSTANCE_AI_MODEL=anthropic/claude-sonnet-4-6
N8N_INSTANCE_AI_MODEL_API_KEY=<model-key>
ANTHROPIC_API_KEY=<same-model-key>
N8N_AI_ASSISTANT_BASE_URL=
N8N_INSTANCE_AI_SANDBOX_ENABLED=true
N8N_INSTANCE_AI_SANDBOX_PROVIDER=daytona
DAYTONA_API_URL=https://app.daytona.io/api
DAYTONA_API_KEY=<daytona-key>
N8N_EVAL_EMAIL=nathan@n8n.io
N8N_EVAL_PASSWORD=PlaywrightTest123
```
Use the n8n sandbox service instead of Daytona when it is available. Set
`N8N_INSTANCE_AI_SANDBOX_PROVIDER=n8n-sandbox`,
`N8N_SANDBOX_SERVICE_URL`, and, when required,
`N8N_SANDBOX_SERVICE_API_KEY`.
LangSmith is optional. Unset `LANGSMITH_API_KEY` and `LANGSMITH_TRACING` for a
throwaway direct run. This keeps the run out of the shared LangSmith dataset.
## Build after code changes
From the repository root:
```bash
(
pnpm --filter n8n... build > build.log 2>&1
build_status=$?
tail -n 20 build.log
exit "$build_status"
)
```
Restart the instance after each build. A running process keeps the old code in
memory.
## Start an isolated instance
Create a temporary user directory:
```bash
mktemp -d /tmp/n8n-agent-eval.XXXXXX
```
Use the printed path as `N8N_USER_FOLDER`. Start the built CLI from
`packages/cli`:
```bash
N8N_PORT=5680 \
N8N_RUNNERS_BROKER_PORT=5681 \
N8N_USER_FOLDER=<printed-temp-directory> \
E2E_TESTS=true \
N8N_ENABLED_MODULES=instance-ai,agents \
N8N_AI_ENABLED=true \
pnpm exec dotenvx run -f ../../.env.eval -- pnpm start
```
Use different ports when 5680 or 5681 is busy.
## Seed the owner
Run this once after the isolated instance starts:
```bash
curl -sf -X POST http://localhost:5680/rest/e2e/reset \
-H 'Content-Type: application/json' \
-d '{"owner":{"email":"nathan@n8n.io","password":"PlaywrightTest123","firstName":"Eval","lastName":"Owner"},"admin":{"email":"admin@n8n.io","password":"PlaywrightTest123","firstName":"Admin","lastName":"User"},"members":[],"chat":{"email":"chat@n8n.io","password":"PlaywrightTest123","firstName":"Chat","lastName":"User"}}'
```
## Run a local disk case
From `packages/@n8n/instance-ai`:
```bash
pnpm exec dotenvx run -f ../../../.env.eval -- \
env -u LANGSMITH_API_KEY -u LANGSMITH_TRACING \
pnpm eval:instance-ai \
--base-url http://localhost:5680 \
--filter <slug> \
--tier agents \
--concurrency 1 \
--verbose
```
The run writes `eval-results.json`, an HTML report, and a PR-comment preview.
The warnings about disabled run-debug capture are benign.

View File

@ -1,21 +1,19 @@
---
name: n8n:create-instance-ai-eval
description: >-
Authors a new Instance AI workflow or Agent eval case — written locally as
JSON, calibrated against a real build, then pushed to the LangTracer suite CI
runs — build cases, behaviour/process cases, credential cases, and seeded
Authors a new Instance AI workflow eval case — written locally as JSON,
calibrated against a real build, then pushed to the LangTracer suite CI runs
— build cases, behaviour/process cases, credential cases, and seeded
(mid-conversation) cases — with intent-driven expectations. Use when adding
or changing an Instance AI eval, or debugging why one is flaky.
or changing an Instance AI workflow eval, or debugging why one is flaky.
---
# Create an Instance AI eval
# Create an Instance AI workflow eval
Each eval is **one JSON case**. Author workflow cases in
`packages/@n8n/instance-ai/evaluations/data/workflows/`. Author standalone Agent
cases in `packages/@n8n/instance-ai/evaluations/data/agents/` and follow the
[`create-agent-builder-eval` skill](../create-agent-builder-eval/SKILL.md).
The disk loader auto-discovers `*.json` in both directories. A LangTracer suite
is the durable home. Cases validate against
Each eval is **one JSON case** — authored locally as a file in
`packages/@n8n/instance-ai/evaluations/data/workflows/` (the disk loader
auto-discovers `*.json`, no registration step), with a LangTracer suite as its
durable home. Cases validate against
[`harness/schema.ts`](../../../packages/@n8n/instance-ai/evaluations/harness/schema.ts)
(`.strict()` — unknown keys fail at load). The eval
[README](../../../packages/@n8n/instance-ai/evaluations/README.md) is the
@ -25,48 +23,17 @@ exhaustive field reference; this skill is the opinionated *how*.
> approach.** Author the file locally (uncommitted), calibrate it against a real
> build, then **push it to a lang-tracer suite** with `eval:langtracer-push`
> (see [Push to a lang-tracer suite](#push-to-a-lang-tracer-suite)) —
> the suite the driver chose (see
> [Ask for the target suite first](#ask-for-the-target-suite-first)) —
> `baseline` for the consolidated corpus n8n CI runs, or a dedicated
> `--suite baseline` for the consolidated corpus n8n CI runs, or a dedicated
> capability suite like `agents`.
> The suite is the home for the case; the eval CLI reads it back via
> `--source langtracer`. You still write the JSON file — it's just the input to
> the push, not a committed artifact.
>
> **Seeded cases.** An `inline` seed pushes with the case — the case-write API
> stores it verbatim, so the suite is its home like any other case. Only a
> `seed.mode: "replay"` case is refused (listed under `skipped:`): it's
> reconstructed from a LangSmith trace at run time, so it dies when that trace is
> pruned and has no durable home. Don't commit a replay case either — derive a
> synthetic case from it. See [`case-shapes.md`](case-shapes.md).
## Ask for the target suite first
**Before you source, draft, or run anything, ask the driver which LangTracer
suite receives every eval you create in this session.** Ask this in both
autonomy levels, including autonomous mode. Never pick the suite yourself and
never default silently. A case pushed to the wrong suite runs in the wrong CI
lane and alerts the wrong team.
Ask the suite question and the autonomy question together, in one message.
Recommend `baseline`:
- `baseline` (recommended): the general corpus. The Instance AI (INS) team
monitors it, and it runs nightly.
- Any other suite, such as `agents` or `node-gauntlet`: call `list_suites` and
offer the live list. Do not use a hardcoded list, because suites are added
and retired.
Rules:
- The driver's answer applies to all cases in the session. Do not ask again per
case.
- If the driver already named a suite in the request, state it back in one line
and continue. Do not ask again.
- Check that the suite kind matches the case. Push a case with a wrong build
(`capability_gap`) only into a suite that allows that kind. If the suite does
not allow it, tell the driver and ask again.
- Put the chosen suite slug in the final decision log, as a link.
> **Exception — seeded cases.** The case-write API has no `seed` field, so seeded
> cases are never pushed. A `seed.mode: "replay"` case is a local throwaway (don't
> commit it either — it dies when its trace is pruned); a `seed.mode: "inline"`
> case isn't transient and has no suite home, so it's the one sanctioned
> exception — it lives as committed JSON. See [`case-shapes.md`](case-shapes.md).
## Set the autonomy level first
@ -85,7 +52,7 @@ trim), **calibration** (classify each red and resolve keep/loosen/drop), and
| Level | Who decides when to stop | Behaviour |
|---|---|---|
| **autonomous** | agent | Runs all four gates start-to-finish; reports a **decision log** at the end for the driver to review — with the pushed case, its suite, and the source thread as **links** ([Share links, never bare ids](#share-links-never-bare-ids)); includes the observation id when the driver opted to record one; and includes a Linear ticket **proposal** for any kept capability-gap red ([Capability gap → propose a Linear ticket](#capability-gap--propose-a-linear-ticket)). |
| **autonomous** | agent | Runs all four gates start-to-finish; reports a **decision log** at the end for the driver to review. |
| **checkpoint** | driver, per gate | Stops at each gate with a compact **proposal + recommendation**; driver says "go" or redirects. At the **calibration** gate, hands the driver a link to the just-built thread on the live instance plus login credentials so they can review the real conversation and workflow themselves before confirming (below). |
**Calibration is special-cased at both levels.** A calibration verdict that
@ -95,15 +62,6 @@ any loosening that would let a known-bad build pass — is **surfaced explicitly
committed. It's the one call where a quiet mistake corrupts the suite, so it
never fully auto-commits.
**Recording a source observation is optional, not a fifth gate.** After a real
thread passes selection, offer to save why it was selected and what the developer
observed with LangTracer's `create_observation` tool (see
[`sourcing-cases.md`](sourcing-cases.md#optional-record-the-selection-as-an-observation)).
If the driver declines, has not stated a preference in autonomous mode, or the
write fails, continue with the eval. Never block drafting, calibration, or push
on an observation. `create_observation` and `update_observation` write only to
LangTracer; do not add a LangSmith feedback or sync step.
**Checkpoint calibration — review the real thread on the instance.** Because the
calibration verdict is trust-critical, in checkpoint mode you don't ask the
driver to trust your reading of the run. You built the case against a live
@ -123,39 +81,6 @@ keep/loosen/drop, and the review link. The driver logs in, reads the thread and
the workflow, and confirms or redirects before you write the verdict back into
the case `description`.
## Share links, never bare ids
Every lang-tracer entity has a shareable web page, but the CLI and the MCP hand
you **numeric ids** — `eval:langtracer-push` prints `+ created <slug> (#621)`,
`get_eval_run` returns a run number, `list_conversations` returns thread ids.
An id is unclickable: the driver has to go find it. **Whenever you name a case,
suite, thread, cluster, or run in anything a human reads** — a checkpoint
proposal, the end-of-run decision log, a PR description, a Linear ticket, a Slack
message — render it as a link, keeping the id in the label:
```
pushed as [#621](https://lang-tracer.n8n-maintenance.workers.dev/test-cases/621)
```
Build links off the **web base** (`LANGTRACER_URL`, in production
`https://lang-tracer.n8n-maintenance.workers.dev`). Never off the API bases —
`${LANGTRACER_URL}/api/v1` and `/api/mcp` are machine endpoints, and a link into
either 404s for the driver or dumps JSON.
| Entity | URL | Where the id comes from |
|---|---|---|
| Test case | `<base>/test-cases/<id>` | push output `(#<id>)`; `create_test_case` / `search_test_cases` |
| Suite | `<base>/suites/<suiteId>` | push header `Suite "<slug>" (#<id>)`; `list_suites` |
| Source conversation | `<base>/conversations/<threadId>` | `list_conversations` / `get_conversation` |
| Cluster report | `<base>/clusters/<id>` | `list_cluster_runs` / `get_latest_cluster_run` |
| Eval run (sweep) | `<base>/results?sweep=<runId>` | `list_eval_runs` / `get_eval_run` (`runId` *is* `sweeps.id`, the "run #N") |
Two links that are **not** lang-tracer and don't take this base: the built thread
(`<base-url>/assistant/<threadId>`) and workflow (`<base-url>/workflow/<id>`) live
on the **n8n instance** the eval ran against. When both are relevant — reviewing a
calibration red, writing a capability-gap ticket — give both, labelled, so nobody
has to guess which host a link points at.
## Where the best cases come from
The strongest cases encode a **real** failure, not an invented premise. Two
@ -170,7 +95,7 @@ discover → verify → encode workflow.
## Pick the case shape first
The corpus is five archetypes. Decide which you're writing before you draft — it
The corpus is four archetypes. Decide which you're writing before you draft — it
determines the fields, the grading, and how you validate. They compose (a seeded
case can still assert outcome), but the primary shape drives the work.
@ -179,26 +104,11 @@ case can still assert outcome), but the primary shape drives the work.
| **Build** (default) | Does the workflow the agent builds actually *work*? | `outcomeExpectations` + `executionScenarios` |
| **Behaviour / process** | Does the agent *converse* correctly (ask the right clarifying question, not re-ask, honour a correction, respect plan approval)? | `processExpectations` + multi-turn director script; often **build-only** |
| **Credential** | Does the build behave correctly given a specific credential view? | `credentials[]` |
| **Seeded** | Start mid-thread, with prior work already in place, and drive the turn under test | `seed` (authored `mode: "inline"`; `"replay"` for a local check) |
| **Context** | Does the agent still work once the conversation is long — does it reuse what it already read, and keep what matters after observational memory compacts the history? | `processExpectations` + a long `seed`; `requiresMemoryCompaction` for the compaction ones |
| **Seeded** | Reproduce a conversation mid-thread and drive the turn under test | `seed` (`mode: "inline"` or `"replay"`) |
**Build** is documented in full below. The other four, the director-script
**Build** is documented in full below. The other three, the director-script
vocabulary, and the seeding modes are in [`case-shapes.md`](case-shapes.md).
The judge also sees **token ground truth** for workflow-build cases: per-turn
input and output tokens inline in each transcript turn header, a build-wide
total, a cache read/write split, and the opening step's cost. So an expectation
may reference cost or consumption directly ("does not re-read the same node's
schema in turn 2"). The numbers cover the orchestrator's own LLM steps only. A
delegated Agent build runs in a sub-agent whose steps are not in the snapshot,
so do not write cost expectations on an Agent case. Those numbers come from run-debug snapshots, so they need
`N8N_INSTANCE_AI_RUN_DEBUG_ENABLED=true` on the instance under test; without it
the judge reads `(no run debug captured)` and cost expectations are ungradeable.
The judge also sees the thread's **observation rows** — what observational
memory kept after it compacted — so a memory case can grade the summary itself.
That block is a separate REST read and needs no flag.
## Core principle (all shapes)
**Write expectations from intent, then calibrate against a real build.** Decide
@ -254,9 +164,8 @@ the combinatorial bulk. A case that never builds tests nothing.
## Workflow
These steps map to the four gates from [Set the autonomy level first](#set-the-autonomy-level-first):
sourcing (before step 1) is the **selection** gate, including the optional offer
to record a source observation; steps 1–2 are the **shape +
expectations** gate; steps 5–6 are the **calibration** gate; steps 7–8 are the
sourcing (before step 1) is the **selection** gate; steps 1–2 are the **shape +
expectations** gate; steps 5–6 are the **calibration** gate; step 7 is the
**push** gate. In *autonomous* mode you flow through all of them and summarize in
a decision log; in *checkpoint* mode you pause at each with a proposal, and at
calibration you hand the driver the thread link + login to review the real build
@ -291,26 +200,17 @@ calibration you hand the driver the thread link + login to review the real build
build has a real gap, or because the harness can't exercise it, **that red is
the result — keep it and surface why** (see "A red is signal", below). Never
delete a scenario, weaken an assertion, or drop to build-only just to make the
run green. And when the case comes back **green**, that is a result to earn,
not to accept: confirm the precondition actually fired, then re-derive it from
the raw thread before calling the case a regression guard (see
[First reproduce, then reclassify](#first-reproduce-then-reclassify)).
run green.
7. **Push to the suite — do NOT commit the JSON.** Once calibrated, push the case
into its curated lang-tracer suite with `eval:langtracer-push` (see
[Push to a lang-tracer suite](#push-to-a-lang-tracer-suite)); the suite is the
case's home, not the repo. Leave the `data/{workflows,agents}/*.json` file uncommitted
case's home, not the repo. Leave the `data/workflows/*.json` file uncommitted
(or delete it once it's in the suite). Committing new case JSONs into the repo
is no longer the approach. (An `inline` seed pushes with the case; only a
`replay` case is refused — it's a local throwaway; see
[`case-shapes.md`](case-shapes.md).) For a sourced case,
is no longer the approach. (Exception: seeded cases can't be pushed — an
`inline`-seeded case stays committed JSON, a `replay` case is
a local throwaway; see [`case-shapes.md`](case-shapes.md).) For a sourced case,
finish by **linking it to its source thread/finding** over the MCP — see
[Link the pushed case to its source](#link-the-pushed-case-to-its-source-provenance-step--always-do-this).
8. **Hand back links, and a ticket proposal if the case found a gap.** Report the
pushed case as `<base>/test-cases/<id>`, not `#<id>` ([Share links, never bare
ids](#share-links-never-bare-ids)), and if calibration kept a real
capability-gap red, propose a Linear ticket for it ([Capability gap → propose a
Linear ticket](#capability-gap--propose-a-linear-ticket)) rather than leaving the
gap as a red case nobody owns.
`--iterations N` is available to measure flakiness (pass@k / pass^k) — reach for
it when you suspect a case is non-deterministic or before promoting it to a
@ -332,80 +232,13 @@ and verify X actually materialised — the direct-loop `eval-results.json` does
persist per-expectation judge reasoning, so pass/fail alone can't tell you which
reading you got.
**The negative form is the easiest to fool yourself with.** An assertion phrased
as "the agent did NOT call `X` with a bad argument" passes when the agent called
`X` correctly *and* when **it never called `X` at all**. Those are opposite
results and the judge reports the same green. So for any assertion about tool
misuse, confirm the tool was actually invoked before believing the pass: parse
`testCases[].transcriptPerRun[][].steps[]` for the call. Note the transcript
groups multi-action tools under a bare `toolName` (`nodes`, `workflows`,
`credentials`), so read `args.action` to get the real one — filtering on
`nodes[explore-resources]` finds nothing and looks like a clean pass. Measured on
this corpus: a batch of five tool-misuse cases scored 100% on its first
calibration run, and three of them were passing vacuously because the tool under
test was never called.
**A sourced failure that does not reproduce is not yet a regression guard — first
re-derive the precondition.** Behaviour does drift across versions, and a case
that flips from *capability-gap* (red) to *regression guard* (green, catches a
re-introduction) is a legitimate and valuable outcome. But reach it by
elimination, not by default: a green far more often means *your case never set up
the situation* than *the builder improved*. See below.
## First reproduce, then reclassify
A case built from a real failure that comes back green is the most common
outcome of a first calibration run, and "the build must have improved" is the
most common wrong conclusion. The usual cause is that you authored from a
*summary* of the thread — the theme label, the observation description, your own
one-line note — and the trigger you assumed is not the trigger that fired. Go
back to the raw thread before you downgrade anything.
**Find the turn, not the topic.** Locate the exact tool call that failed, then
read the assistant text immediately before it. The agent usually states its
intent in the open, and that sentence is the precondition. Then ask what *state*
made that call necessary — not what the conversation was about.
Worked example from this corpus. Sourced finding: "the agent invents
`nodes[explore-resources]` method names." Assumed trigger: the user swaps model
provider. A case built on a clean provider swap came back green — the agent set
the model id directly and never called the tool at all. The raw thread said it
plainly:
> "The **Groq Chat Model** has an invalid model (`llama3-8b-8192` isn't offered
> by your Groq credential). **Let me list valid models and fix it.**"
The precondition was never the swap. It was *an existing model id that the
provider rejects at runtime, with a credential already connected* — that is the
state that makes enumerating models necessary. Rebuilt on it, the same case
reproduced the failure on the first run, with the agent inventing two method
names in a row.
Three moves turn a non-reproducing case into a reproducing one. Try them in
order before settling for a guard:
1. **Fix the precondition.** Rebuild the seed and the live turn to recreate the
state the source thread was in, not the subject it was discussing.
2. **Move the assertion to the first call.** A mechanism where the agent
*self-corrects* grades green on the end state and is still a real defect — the
wasted round-trip and the guessed schema are the finding. Grading first-call
correctness turned a mechanism previously dismissed as "self-corrects, weak
eval" into a gap that reproduced in 2 of 2 runs.
3. **Keep what the attempt actually caught.** A reproduction run often reds on a
*different* real defect than the one you targeted. That is still a
capability-gap finding — keep the red, retarget the description, and say
plainly in it that the originally targeted mechanism did not reproduce.
**Know when to stop.** Some mechanisms are structurally unreachable in this
harness, and no amount of re-deriving fixes that. The clearest example: a seed
restores a **fresh** workspace file at the start of the graded turn, so the
agent's first `old_str` always comes from a file it just read. Failures that need
*accumulated drift* across many turns — the `str_replace` byte-fidelity family,
the largest agent-caused tool-call failure in production at 15% of threads — do
not reproduce even in a multi-turn chain of edits, each followed by a build. Cap
the effort at about three attempts, then write the negative result into the case
`description` as the finding it is, and ask whether the real defect belongs in a
ticket rather than an eval.
**A sourced failure that no longer reproduces is still worth keeping — it's now a
regression guard.** When you encode a real failure and calibration shows the
current build handling it correctly (behaviour drifts across versions), the case
doesn't lose value: it flips from *capability-gap* (currently red) to *regression
guard* (currently green, catches a re-introduction). Keep it — but only after the
non-vacuous check above proves it *would* turn red on the bad behaviour, else the
"guard" guards nothing.
## A red is signal — surface it, don't work around it
@ -450,9 +283,7 @@ failures are excluded do the three categories below apply:
something the user asked for (a miswired branch, a missing retry, wrong field
keys). This is exactly what the eval is for. **Keep it red.** Don't loosen the
assertion or drop the scenario; a currently-red gap is the capability signal
today, and a re-introduction guard once the builder improves. Then **propose a
Linear ticket** for the gap — see [Capability gap → propose a Linear
ticket](#capability-gap--propose-a-linear-ticket).
today, and a re-introduction guard once the builder improves.
- **Harness limitation** — the build is correct but the mock/execution layer
can't exercise the path (see "Known harness limitations", below). **Keep the
scenario and say so in its `description`** — that this red is harness-caused,
@ -489,72 +320,6 @@ classification is stated in the open, never silently committed — misreading a
harness red as a real gap (or the reverse) is the one calibration mistake that
quietly corrupts the suite.
### Capability gap → propose a Linear ticket
A kept capability-gap red is a **product bug you just characterised better than
any bug report would**. But a red case in a suite doesn't assign itself to anyone:
without a ticket the gap sits in CI as permanent noise, and the next person to
read the run assumes someone already owns it. So once a red is classified as a
real gap (and the driver has confirmed it, per the autonomy level), **propose a
Linear ticket for it.**
**Reproduce first. This is a default, not a hard rule.** Propose a ticket only
after a case has reproduced the gap. That means the case is red on a real build,
the precondition was confirmed to fire, and you re-read the raw thread (see
[First reproduce, then reclassify](#first-reproduce-then-reclassify)). A gap that
comes from trace analysis alone is a hypothesis. A ticket for it costs the owning
team time. The driver may approve an exception, for example when the harness
cannot reach the mechanism after about three attempts. In that case the ticket
must say that the eval did not reproduce the gap. Never file an unreproduced gap
without the driver's approval.
**Propose, don't create.** Per [AGENTS.md](../../../AGENTS.md), never open a
Linear ticket unasked. Put the draft in front of the driver — interactively in
checkpoint mode, in the decision log in autonomous mode — with a title, a team,
and the body, and let them say go. Skip the proposal in two cases:
- **The gap already has a ticket.** Check the case's linked issues on its page,
and run `get_linear_ticket_context <TEAM-N>` on any candidate identifier the
driver or the source thread mentions, before you draft a duplicate.
- **The red isn't a capability gap.** A `Harness note:` red is a
lang-tracer/harness issue, and genuine non-determinism is a case-hygiene chore.
Neither belongs in the builder's queue.
The draft body should carry what makes the gap actionable, all of it already in
hand from calibration:
- **The eval case**, as a link — `<base>/test-cases/<id>` (see [Share links, never
bare ids](#share-links-never-bare-ids)). This is the reproducer; it's the most
valuable line in the ticket.
- **What failed, verbatim** — the failing `outcomeExpectation` /
`processExpectation` or scenario name, plus the judge's stated reason. Not a
paraphrase: the exact text is what the fixer will grep for.
- **What the build did instead** — the specific defect (miswired branch, wrong
field key, missing gate), and links to the real evidence: the source conversation
(`<base>/conversations/<threadId>`) and the built thread + workflow on the eval
instance (`<base-url>/assistant/<threadId>`, `<base-url>/workflow/<id>`).
- **Blast radius, if you know it** — the cluster theme or the number of real
conversations behind the gap (`<base>/clusters/<id>`) is what turns "one red
case" into a prioritisable bug.
**File it from the case page so the link is made.** The `<base>/test-cases/<id>`
page has a *Create Linear issue* dialog that creates the ticket **and** links it to
the case; that link is what makes `get_linear_ticket_context <TEAM-N>` later return
the case, its scenarios, the source conversation, and its analysis in one call. The
case↔ticket link is only writable from that UI — lang-tracer's MCP and `/api/v1`
are read-only for it — so if the ticket gets created some other way (a Linear MCP,
if your harness has one, or Linear directly), say plainly that it isn't linked, ask
the driver to link it on the case page, and meanwhile put the identifier + URL in
the case `description` via `update_test_case` so the provenance isn't lost.
Then extend the description prefix with the ticket, so the corpus stays greppable
in both directions: `Capability-gap finding: current build reds because <X> — a
real builder bug (flips to a regression guard once fixed). Tracked in
[<TEAM-123>](<ticket url>)`. And when the *build itself* is wrong — not just a
scenario red under a correct build — push it with `--set-kind capability_gap` into
a suite of that kind (see
[Push to a lang-tracer suite](#push-to-a-lang-tracer-suite)).
## Example
Minimal build case:
@ -655,16 +420,9 @@ nodes get LLM-generated pin data). So:
static data** (`removeItemsSeenInPreviousExecutions`, `$getWorkflowStaticData`)
is **not** seedable, because static data starts empty every run, so such a
scenario reds vacuously (it sees everything as "new"). To get a seedable
change-detection scenario, say "keep the state in a data table" in the prompt;
otherwise
change-detection scenario, steer the build toward a Data Table; otherwise
accept the static-data red as a harness limit and carry the logic in
`outcomeExpectations`. Note the agent may *choose* static-data dedup on its own.
- **Message an Agent runs the called Agent with its real model.** The harness
mocks the Agent's tools, MCP servers and web search, and records those calls
under the calling node. It turns off the Agent's memory, vector stores and chat
integrations. Only OpenAI models are supported today, and only when
`EVAL_OPENAI_API_KEY` is set (CI sets it): put the Agent on an `openai/*` model
and declare an `openAiApi` credential.
- Don't assert exact counts that depend on mock generation ("exactly 7 posts").
Say "fewer than the original 10".
@ -758,9 +516,8 @@ Both are natural-language assertions graded by the same Sonnet judge, and each
They run everywhere, including prebuilt/MCP runs (no transcript needed).
- **`processExpectations`** — **how the agent behaved during the build**, judged
from the transcript. Assert clarifying questions asked (or not re-asked),
user-visible behaviour, plan/approval handling, batching, honouring a
correction, ordering. Do not assert internal mechanics such as sub-agents or
tool names (see "Keep expectations free of internal mechanics"). They need a transcript, so they're **skipped in prebuilt/MCP runs**.
tool-call behaviour, plan/approval handling, batching, honouring a correction,
ordering. They need a transcript, so they're **skipped in prebuilt/MCP runs**.
Rule of thumb: an assertion about *the artifact* is an outcome expectation; an
assertion about *the conversation or the agent's choices along the way* is a
@ -798,54 +555,6 @@ case, where the source and channel were **left unspecified**):
Put intent the conversation only *implied* (a preferred but unstated channel) in
`processExpectations`, not `outcomeExpectations`.
## Keep expectations free of internal mechanics
An expectation describes what the user or the artifact sees. It does not
describe how Instance AI is built inside. The internal architecture changes:
sub-agents, tool names, delegation, planning steps, and run ordering all move.
An expectation that names them goes red when the design changes, although the
behavior is still correct. It also stops proving the behavior: a build that
solves the problem by another route fails it.
**Do not name these in an expectation:**
- Sub-agents, delegation, or hand-off ("the sub-agent runs ...", "the
orchestrator delegates ...").
- Internal tool names or call counts ("calls `build-agent`", "calls
`search-nodes` twice").
- Internal stages, task lists, or step order that the user cannot see.
- Which component produced a result.
**Do name these:**
- What the user sees in the conversation: the question asked, the plan shown
for approval, the claim in the final response.
- What the artifact contains: the workflow or Agent, its configuration, and
how it behaves.
- A user-visible sequence, only when the user asked for it ("asks before it
builds").
Rewrite test. Delete the internal term. State the result the term produced.
| ❌ mechanical | ✅ intent |
|---|---|
| "The sub-agent runs the workflow before it reports success" | "The final response reports only results that the build verified; it does not claim an untested step works" |
| "The orchestrator calls `ask-user` before delegating" | "The agent asks for the missing Slack channel before it builds, and does not guess one" |
| "The builder calls `search-nodes` for the trigger" | "The workflow starts from a trigger that matches the requested schedule" |
Second test. Imagine the internals are replaced by one agent with different
tools. A correct build must still pass. If the expectation would fail or become
meaningless, rewrite it.
Some mechanics are a legitimate subject: a case whose purpose is to guard a
specific tool contract (for example, a regression in how a tool reports an
error). Keep these rare. State the reason in `description`. Do not mix them with
intent expectations in the same case.
**Calibration steps are not expectations.** Reading the transcript to confirm
that a build ran, or that a sub-agent was reached, is a good calibration check.
Keep it in your own run notes. Do not copy it into the case JSON.
## Robust design vs harness flakiness
Two different things — keep them apart:
@ -916,10 +625,7 @@ concluding whether the failure is your case, the build, or the harness.
```bash
cd packages/@n8n/instance-ai
pnpm exec tsx -e "import {loadWorkflowTestCasesWithFiles} from './evaluations/data/workflows/index.ts'; console.log(loadWorkflowTestCasesWithFiles('<slug>')[0].fileSlug)"
# For a standalone Agent case:
pnpm exec tsx -e "import {loadAgentEvalTestCasesWithFiles} from './evaluations/data/agents/index.ts'; console.log(loadAgentEvalTestCasesWithFiles('<slug>')[0].fileSlug)"
npx tsx -e "import {loadWorkflowTestCasesWithFiles} from './evaluations/data/workflows/index.ts'; console.log(loadWorkflowTestCasesWithFiles('<slug>')[0].fileSlug)"
```
## Push to a lang-tracer suite
@ -932,32 +638,24 @@ drifted, leaves the rest unchanged, and never prunes. It's the inverse of
```bash
cd packages/@n8n/instance-ai
# <suite> is the slug the driver chose (e.g. baseline)
# preview first — no writes:
pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite> --dry-run --changed
# preview first — no writes (use `npx dotenvx`; the bare `dotenvx` binary is usually not on PATH):
npx dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite baseline --dry-run --changed
# then push (drop --dry-run):
pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite> --changed
npx dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite baseline --changed
```
- **Selectors** (at least one required — no accidental push-all): positional
`<slugs...>` (exact file slugs), `--changed` (new/untracked + staged + modified
`data/{workflows,agents}/*.json`, ideal right after authoring an uncommitted case),
`data/workflows/*.json`, ideal right after authoring an uncommitted case),
`--filter`/`--tier` (with `--exclude` as a modifier).
- **Validation is selective.** Exact slugs and `--changed` read only the named
files, so an unrelated invalid file in `data/workflows` (a case authored on a
newer branch, a half-written draft) never blocks your push. `--filter` parses
only the files whose slug matches; `--tier` reads the tier from inside each
file, so it parses them all. An invalid file either one parses prints
`⚠ skipped invalid case file …` and the push continues without it. A file you
named still fails the push when it is invalid.
- **Multiple positional slugs? Skip pnpm — call the script directly.** `pnpm
eval:langtracer-push … slugA slugB` forwards the slugs as one joined argument
(`"slugA slugB"`), so no case file matches and nothing is pushed. Either use a
no-positional selector through pnpm (`--changed`), or run the script directly so
each slug is its own argv: `pnpm exec dotenvx run -f .env.eval -- pnpm exec tsx
each slug is its own argv: `npx dotenvx run -f .env.eval -- npx tsx
evaluations/cli/langtracer-push.ts --suite <slug> <slug1> <slug2> …`.
- **Env:** `LANGTRACER_URL` + `LANGTRACER_API_KEY` (an `lt_` bearer; one key works
for MCP + REST) — put them in `.env.eval` and run under `pnpm exec dotenvx`.
for MCP + REST) — put them in `.env.eval` and run under `npx dotenvx`.
- **Options:** `--set-kind regression|capability_gap` (default `regression`, must
match the suite's kind), `--contains-user-data` (default is `synthetic`). A case
whose **build is correct** (outcome expectations green) but that carries a
@ -969,18 +667,12 @@ pnpm exec dotenvx run -f .env.eval -- pnpm eval:langtracer-push --suite <suite>
lang-tracer #48), so scenario edits re-push like any other field. A lang-tracer
deployment predating that change silently ignores the key; if a pushed scenario
edit doesn't land, update the scenario in the lang-tracer UI.
- **Report what was pushed as links, not `#ids`.** The CLI prints `+ created
<slug> (#621)` / `~ updated <slug> (#621, rev 3)` and a suite header — that's the
id, and nothing more. Turn each one into `<base>/test-cases/<id>` (and the suite
into `<base>/suites/<suiteId>`) in whatever you hand the driver, so they can open
the case they just authored instead of hunting for it. See [Share links, never
bare ids](#share-links-never-bare-ids).
- **An `inline` seed pushes with the case:** the case-write API stores it
verbatim, so a seeded case lives in a suite like any other. Only a `replay`
case is refused — the push lists it under `skipped:`, because it's
reconstructed from a LangSmith trace at run time and dies when that trace is
pruned or deleted. Don't commit a replay case either; derive a durable
synthetic case as the artifact instead.
- **Seeded cases can't be pushed:** the case-write API has no `seed` field, so the
push lists any seeded case under `skipped:` and it never reaches the suite. A
`replay` case shouldn't be committed either — it dies when its trace is pruned
or deleted — so derive a durable synthetic case as the artifact instead. An
`inline`-seeded case isn't transient and has no suite home, so it's the one
exception to "don't commit the JSON" — it lives as a committed artifact.
### Link the pushed case to its source (provenance step — always do this)
@ -1027,9 +719,8 @@ in *checkpoint* mode calibration this is how the driver opens the built thread
## Other eval harnesses (not this skill)
Use the [`create-agent-builder-eval` skill](../create-agent-builder-eval/SKILL.md)
for standalone Agent cases. Three other harnesses have their own data dirs and
CLIs: **`eval:subagent`** (workflow-build compatibility corpus,
This skill is for `data/workflows/` cases. Three siblings exist with their own
data dirs and CLIs: **`eval:subagent`** (workflow-build compatibility corpus,
binary-check scored), **`eval:discovery`** (asserts first-hop tool/dispatch
routing, no n8n server), **`eval:pairwise`** (head-to-head build comparison vs
`ai-workflow-builder.ee`). Authoring them is out of scope here — see the README

View File

@ -1,7 +1,7 @@
# Case shapes beyond the plain build
The [SKILL](SKILL.md) covers the **build** archetype. This file covers the other
four — **behaviour/process**, **credential**, **seeded**, **context** — plus the
three — **behaviour/process**, **credential**, **seeded** — plus the
director-note vocabulary multi-turn cases rely on. Field-level docs live in the
eval [README](../../../packages/@n8n/instance-ai/evaluations/README.md); this is
the opinionated *how* and the traps.
@ -103,7 +103,6 @@ recognises this vocabulary:
| Dismiss a setup card / skip a value | `[When the setup card asks for the API base URL, dismiss it — the user hasn't decided yet.]` (proxy dismisses via `approve_or_reject(false)`) |
| Reject a plan that misses a requirement | `[When the agent shows its plan, reject it unless it sorts descending by count.]` |
| Iterate change-by-change, in order | `[Send each change below in order, waiting for the build after each; keep bundled changes in one message.]` |
| Run the workflow themselves and report back | `[When the agent asks you to test the workflow, run it yourself, then tell it you ran it.]` Set `allowUserExecution: true` on the case. The proxy then sets `runWorkflowId` to the intended saved workflow. The harness uses normal execution without mocks. This action supports credential-free manual or schedule workflows without pinned data. |
The `text` of a turn may be an **array of strings** (joined with newlines) so a
long director note stays readable in JSON. A note governs only what it covers;
@ -254,11 +253,10 @@ skill; these are illustrative, trimmed of the full calibrated wording):
}
```
All three are complete as written in one respect that earlier guidance got
wrong: none of them asserts anything about the credential's connection test.
Both routes a credential takes into a build now authenticate, so an expectation
acknowledging a connection-test failure reds every correct build — see
"Credential validity" below.
All three omit one detail for brevity that the real, calibrated versions
include: a `processExpectations` entry acknowledging the placeholder-token
connection-test failure as expected (see the note right below) — a full case
must include that or it will fail on a correct build for the wrong reason.
The wire shapes (verified live against both tools — `credentials.tool.ts`'s
`handleSetup` state machine and `workflows.tool.ts`'s setup-wizard equivalent):
@ -274,18 +272,26 @@ The wire shapes (verified live against both tools — `credentials.tool.ts`'s
**Every eval credential holds a placeholder token** unless you set the type's
`EVAL_*_ACCESS_TOKEN` env var (see "Credential cases" above). The parent
umbrella (TRUST-348) requires "no stored provider credentials in any phase," so
a real token is the wrong fix. Instead the harness resolves the *connection
test* as passing, the same way for both routes a credential can take into a
build:
a real token is the wrong fix. What differs is whether the *connection test*
against that placeholder is allowed to fail, and that depends on where the
credential came from:
- **Declared in `credentials[]`** — models a credential the user already has
connected, so it authenticates.
- **Created by the simulated user on an engaged setup card** — authenticates
too, since the product won't apply a card whose credential failed its test.
- **Declared in `credentials[]`** (pre-seeded, never touched by the user during
the build) — the product runs a real connection test and reports a genuine
"Invalid access token" failure. Phrase `processExpectations` to assert the
agent reports that honestly (doesn't claim success, doesn't go silent), not
that the token works:
Do **not** assert a connection-test failure for either; such an assertion reds
on every correct build. See "Credential validity" below for how to script a
card-created credential that deliberately does not authenticate.
```json
"Harness note: a connection-test failure (invalid access token) is expected here since the credential uses a placeholder token. The agent reported that failure honestly — it did not claim the Slack integration was fully working, and did not silently ignore or hide the failure."
```
- **Created by the simulated user on an engaged setup card** — the test resolves
as **passing** by default, because the product won't apply a card whose
credential failed one. Do **not** assert a connection-test failure for these;
such an assertion reds on every correct build. See "Credential validity"
below for how to script a card-created credential that deliberately does not
authenticate.
**`auto` is reachable but inert** — the product genuinely rebuilds the agent
and returns `needsBrowserSetup:true`, but this harness has no Computer Use
@ -293,24 +299,24 @@ tools attached, so the conversation stalls afterward (expected, not a bug).
Keep any case scripting `auto` a local smoke test, never part of the gated
suite — it will time out.
### Credential validity: a credential works by default
### Credential validity: a set-up credential works by default
The product will not apply a setup card whose credential fails its connection
test — the frontend's `isCredentialComplete` returns `isCredentialTestedOk`, so
Apply stays disabled until the test passes. **"The user completed the setup
card" therefore implies "the credential authenticates."** The same holds for a
credential declared in `credentials[]`: it stands for one the user connected
before the conversation started. Both carry a placeholder token that would fail
for real, so the harness resolves the test as successful for both — otherwise
every such case models a state a real user cannot reach.
card" therefore implies "the credential authenticates."** A seeded credential
carries a placeholder token and would fail for real, so the harness resolves
the connection test as successful for credentials it creates on an engaged
card. Without that, every such case would model a state a real user cannot
reach.
Mechanically: the harness registers the case's seeded credential ids on the
thread up front, the proxy adds the ones it sets up mid-run (the types it lists
in `workingCredentialTypes`), both go on the same `bypassCredentialTest` list of
the eval allowlist endpoint, and the credential adapter resolves their test as
successful without contacting the provider. The token is untouched — only the
test result is synthesized, and only for credentials that case created. Nothing
changes about "no stored provider credentials".
Mechanically: the proxy lists the types it set up in `workingCredentialTypes`,
the harness registers those credential ids on the thread
(`bypassCredentialTest` on the eval allowlist endpoint), and the credential
adapter resolves their test as successful without contacting the provider. The
token is untouched — only the test result is synthesized, and only for
credentials that case created. Nothing changes about "no stored provider
credentials".
**To script a credential that does NOT authenticate**, say so explicitly in the
direction, naming which one:
@ -324,8 +330,8 @@ direction, naming which one:
The proxy then omits that type, its test runs for real, and it fails. Note what
this models: not "the card was applied with a broken credential" (unreachable),
but a credential that stopped authenticating — expired, revoked, scope changed.
A card is currently the only way to get a failing credential: declaring one in
`credentials[]` gives you a working one.
For a credential that was already broken *before* the conversation, declare it
in `credentials[]` instead of setting it up on a card.
**Non-vacuity for these cases is deterministic, not judged.** A bypassed test is
deliberately indistinguishable from a real pass in everything the agent sees —
@ -339,19 +345,19 @@ assertion: two credentials with one scripted invalid should show exactly `1`.
## Seeded cases (start mid-conversation)
A seeded case puts prior history into the build thread *before* the live turn, so
the eval drives only the turn under test. Use it to set up the situation you want
to test — history up to some point, then a message that should trigger (or
correct) a behaviour.
A seeded case restores prior history into the build thread *before* the live
turn, so the eval drives only the turn under test. Use it to reproduce a real
situation — a conversation up to some point, then a message that should trigger
(or correct) a behaviour.
Pick the lightest mode that fits:
| Situation | Mode | Pairs with |
|---|---|---|
| Prior work already exists (a workflow to repair, an agent to change) | `seed.mode: "inline"` — prior messages + the workflows/agents they reference, in the case body | a normal `conversation` for the live turn |
| Reproduce a real conversation (common case) | `seed.mode: "replay"` — fetch + reconstruct its LangSmith trace at run time; nothing committed | supplies its own live turn (omit `conversation`) |
| Prior work already exists (a workflow to repair) | `seed.mode: "inline"` — prior messages + the workflows they reference, in the case body | a normal `conversation` for the live turn |
| Prelude is just "what was discussed" (no tool calls, no workflows) | `seed.mode: "inline"` with `{role, text}` shorthand messages | a normal `conversation` for the live turn |
| Shallow 2–3 turn prelude where the agent's live replies matter | none — a plain multi-turn `conversation` re-drives it live | — |
| Confirming a real failure locally, before authoring the case | `seed.mode: "replay"` — rebuilds a thread from its LangSmith trace at run time; nothing committed, expires with the trace | supplies its own live turn (omit `conversation`) |
Both modes are implemented and wired (`harness/conversation-seed.ts` +
`harness/langsmith-seed.ts`, threaded through the runner). The literals match
@ -359,17 +365,13 @@ lang-tracer's `metadata.seed` verbatim, so nothing translates between the repos.
### What the seed does — and does not — exercise
**The seeded portion is replayed, not re-run.** The prior messages are written into
the thread as they stand (marked `seeded: true` so the judge and checks can tell
them apart), and the workflows, data tables and agents they reference are **created
on the instance** — so when the live turn runs, the agent sees the workspace the case
says it should. Data tables are created **schema-only, no rows**: row values are the
most sensitive thing a table holds, and they stay off the eval instance.
One thing the restore can't reproduce: the **sandbox is empty**. The agent re-reads
state from the database rather than editing source it "wrote", so a seeded case is
harder than the real turn was — never grade one on cost, turn count or
`messageBudget`.
**The seeded portion is restored, not re-run.** The message log is written into
the thread verbatim (marked `seeded: true` so the judge and checks can tell it
apart), and the workflows and data tables the history references are **recreated
on the instance** — so when the live turn runs, the agent sees the same
workspace the original conversation left behind. Data tables are recreated
**schema-only, no rows** (row values are the most sensitive part of a trace and
are kept out of the eval instance).
The consequence to internalise: **nothing you assert can change what already
happened in the seeded turns** — the agent didn't produce them, it's only
@ -389,21 +391,18 @@ the issue is in a *later* turn. (A turn-0 issue can't be isolated by seeding: it
lands inside the seed, so you'd bake the bug into the prelude.) Two standing
costs keep it a last resort, not a default:
- **Data handling.** A replay stands someone's own conversation up on the eval
instance. The most sensitive parts are removed first — data-table row values are
kept out and redacted from the history, node credentials stripped — but that pass
can't be assumed exhaustive, so treat what comes back as potentially personal data
and follow your team's data-handling policy. Cleanup deletes the thread, workflows
and tables when the build finishes, though it is best-effort: a crashed run can
leave them on the instance. Scrubbing the workflow into an `inline` seed is how the
whole concern goes away for good.
- **Data handling.** It recreates a real conversation on the eval instance. The
most sensitive content is scrubbed first — data-table row values are kept out
and redacted from the restored history, node credentials stripped — but that
isn't guaranteed exhaustive, so treat reproduced content as if it may carry
user data and follow your team's data-handling policy.
- **Transience.** It depends on LangSmith trace retention (~14 days); the case
stops running once the source trace ages out (tag it `seeded`, keep it out of
`full`/`pr`).
If a plain prompt + director script can reproduce the situation, prefer that.
### `mode: "replay"` — rebuild a thread for a local run
### `mode: "replay"` — reproduce a real conversation
```json
"seed": { "mode": "replay", "threadId": "<thread-id>", "project": "instance-ai" }
@ -417,11 +416,13 @@ sent live. `project` defaults to `instance-ai`. Optional `endpoint` pins a
US-tenant source host during the US→EU migration; optional `liveTurnRunId` pins
which user turn goes live.
- **Cross-workspace, zero config.** A production thread can be replayed in a staging
- **Cross-workspace, zero config.** A prod thread can be reproduced in a staging
eval — the harness enumerates the workspaces your `LANGSMITH_API_KEY` can reach
and finds the one holding the thread. It only *reads* the source; the eval
writes its own traces/datasets to its own workspace. What it rebuilds still lands
on the eval instance, so the data-handling note above applies.
writes its own traces/datasets to its own workspace. Reproducing a real thread
recreates its conversation on the eval instance; the most sensitive content is
scrubbed first (see the data-handling note above), and it's still worth
handling per your team's data policy.
- **Continue past the live turn.** Add a `conversation` to keep driving after the
trace's last message replays (first authored turn = expected assistant reply as
proxy reference; subsequent `user` turns become follow-ups). Omit it to replay
@ -442,27 +443,27 @@ which user turn goes live.
seeds with **no workflow to inspect**. Reproduce the target workflow yourself
(a synthetic case whose `executionScenarios` precondition builds the stand-in),
or grade the live turn with `processExpectations` only.
- **Can't be pushed to a lang-tracer suite either.** `eval:langtracer-push`
refuses a `replay` case and lists it under `skipped:` — a suite is a durable
home and this seed isn't. Combined with the don't-commit rule above, a `replay`
case has **no durable home by design** — the durable artifact is always the
synthetic case you derive from it. (An `inline` seed has no such problem: it
pushes with the case and lives in the suite like any other.)
- **Can't be pushed to a lang-tracer suite either.** The case-write API has no
`seed` field, so `eval:langtracer-push` silently lists any seeded case under
`skipped:`. Combined with the don't-commit rule above, a `replay` case has **no
durable home by design** — the durable artifact is always the synthetic case you
derive from it. (An `inline` seed carries no thread dependency and can't be
pushed either, so — unlike a normal case — it's the one exception to the skill's
"push, don't commit the JSON" rule: it lives as a committed artifact.)
### `mode: "inline"` — durable synthetic fixture
For a **synthetic, sanitised** seed pinned in git (never a real user's
conversation): author the prior messages, plus the artifacts they reference, in
conversation): author the prior messages, plus the workflows they reference, in
the case body (schema in
[`harness/conversation-seed.ts`](../../../packages/@n8n/instance-ai/evaluations/harness/conversation-seed.ts)
— `messages` + optional `workflows`, `dataTables` and `agents`). Real
conversations belong in `replay`, which keeps their content out of the repo.
— `messages` + optional `workflows` + `dataTables`). Real conversations belong in
`replay`, which keeps their content out of the repo.
Two constraints that bite: an artifact `id` must be ≥8 characters (the id remap
Two constraints that bite: a workflow `id` must be ≥8 characters (the id remap
refuses shorter ones), and a seeded `build-workflow` tool call's
`output.workflowId` must match the seeded workflow's `id` — otherwise the remap
separates them and the agent can't find the workflow it should act on. The same
applies to a seeded `build-agent` result's `output.agentId`.
separates them and the agent can't find the workflow it should act on.
The seed sits in the case body, not a sibling file, so it travels with the case
whether it comes off disk, out of a LangTracer suite, or from a dispatched case
@ -486,349 +487,10 @@ write a message as `{role, text}` and the schema expands it to a full envelope:
`text` also takes an array of lines (newline-joined), like a `conversation` turn.
The expansion stamps `createdAt` itself — ascending, in the past — so a shorthand
message can't order *after* the live turn. Shorthand and full envelopes mix freely
in one array; a full envelope keeps its authored `createdAt` — **unless the authored
stamps don't already ascend and sit in the past**, in which case the whole sequence
is restamped onto ascending pre-live slots. A future stamp would sort a seeded turn
after the live turn, and a non-ascending sequence (a shorthand turn appended after
later-stamped envelopes, say) would present the history in an order the graded
transcript never had; restamping only the offending entry would reorder it against
the array the transcript is graded from. A near-miss (say
in one array; a full envelope keeps its authored `createdAt` — **unless any message
in the array is in the FUTURE**, in which case the whole sequence is restamped onto
ascending pre-live slots (a future stamp would sort a seeded turn after the live
turn, and moving only that one entry would reorder it against the array the
transcript is graded from). A near-miss (say
`text: 123`) is deliberately **not** expanded — it fails at load instead of
becoming a message the transcript builder would silently drop.
#### `agents` — "here's an agent you already built, now change it"
An n8n **Agent** is not a workflow, so it has its own slot: a project-scoped
resource with a config *plus authored skill bodies*. Declare it and the restore
creates it at its pinned id in the thread's project, with its skills, before the
live turn:
```json
"seed": {
"mode": "inline",
"messages": [ /* … the turn that built it … */ ],
"agents": [
{
"id": "AgEnT12345678901",
"config": {
"name": "Support Triage",
"model": "anthropic/claude-sonnet-4-5",
"instructions": "Triage inbound support tickets.",
"skills": [{ "type": "skill", "id": "skill_1" }]
},
"skills": {
"skill_1": {
"name": "Triage rules",
"description": "How tickets are sorted",
"instructions": "Label each ticket by severity…"
}
}
}
]
}
```
`config` and `skills` are the exact shapes `GET …/agents/v2/:id/config` and
`…/skills` return, so you can author a seed from an agent you built by hand: build
it on a dev instance, fetch both, scrub, paste. Things worth knowing:
- **The thread is bound to the seeded agent**, exactly as the conversation that
built it would have left it, so the live turn's `build-agent` call continues
that agent directly. Without the binding the call is rejected (`Unknown
agentRef`) and the model recovers from the agent id in its seeded history —
measured at 3/3 runs recovering correctly, but it burns a turn, and the
rejection message offers "create a new agent" as its first option.
- **Names are not uniquified and leftovers are not evicted**, unlike seeded
workflows. An agent is addressed by id, so a same-named copy can't misdirect the
live turn; the name is also woven through skill prose, where a rename would
rewrite instructions the case grades.
- **Grade the agent, not a workflow.** The agent's config + skills render into the
judge context, so `outcomeExpectations` cover them. Assert on the *change* the
live turn makes — and assert the untouched parts survive, which is how you catch
a rebuild-from-scratch masquerading as an edit.
- **Skill bodies are prose — scrub them like conversation content, not like
config.** A skill carries far more free text than a workflow does: instructions
and `references[]` are whole markdown documents, and they routinely name real
teams, customers, internal tools, ticket queues, Slack channels and escalation
contacts. Rewrite them into neutral equivalents (`Acme Corp`, `#support`) rather
than trimming, and reread the full body — a workflow-shaped scan of names and
ids will miss a paragraph.
- **Skill ids must match**: every `config.skills[].id` needs an entry in the
`skills` map, or the agent renders with a dangling reference.
- **Two agents can't share an addressing key.** Names are slugified to address the
agent (`Support Bot` and `support-bot` both become `support-bot`), so a seed
whose agent names differ only by case, spacing or punctuation is refused rather
than silently dropping one from the registry.
- **Credential ids are blanked on restore**, the agent counterpart of stripping a
seed workflow's node credentials. An id from the instance you authored on
addresses nothing here, so you can paste a fetched config as-is and the restore
empties them. The seeded agent therefore arrives unconfigured for credentials —
fine for grading its config and skills, but it is not runnable as seeded.
Declare what the live turn should see in the case's own `credentials[]`.
- **Requires the agents module.** A seeded agent restore fails loudly (as a
framework issue) on an instance where agents are disabled, rather than running
the case unseeded.
#### `folders` — "look at the ODW folder"
A folder case grades how the agent finds the contents of a folder the user names
(CONTEXT-86: the workflow list tool takes `folderPath` or `folderId`). Declare the
folders and place the seed workflows in them:
```json
"seed": {
"mode": "inline",
"folders": [{ "id": "odwFolder0001", "name": "ODW" }],
"workflows": [
{ "id": "odwSignal1Wf", "name": "Odds Watch - 1", "parentFolderId": "odwFolder0001", "nodes": [], "connections": {} },
{ "id": "rootWorkflow1", "name": "Voice Agent", "nodes": [], "connections": {} }
]
}
```
Things worth knowing:
- **Leave decoys at the root.** A folder case is only informative when the folder's
contents differ from what a name search would return. Keep workflows outside the
folder, and pick names a `query` filter for the folder name would not match.
- **Grade the mechanism and the result.** One expectation for the `folderPath` or
`folderId` call, one for reporting exactly the folder's workflows, one for not asking
the user what the folder holds. The miss path (no folder exists) is a separate case:
LangTracer case 699 grades it, and it needs no `folders` slot.
- **Names are verbatim and leftovers are evicted**, like seeded projects. The live turn
names the folder, so the created name has to match exactly; a root folder of the same
name that existed before the run started is deleted before the restore, with every
workflow and subfolder in it. There is no seed marker on the name, so a same-named
folder a human made on that instance goes too: use an eval instance, and a name a
real project would not use. A previous iteration's folder, created during the run,
is left alone.
- **Run it alone.** A folder case and its miss-path sibling (case 699) share the
project, so one invocation per case. With several iterations the previous folder can
still be live for a few seconds while it is judged; keep the name distinctive.
- **Nesting works.** A folder's own `parentFolderId` names another seed folder. Parents
are created first. The `folderPath` for a nested folder is `Parent/Child`, so a folder
name cannot contain `/`.
- **Rules are checked at case load**: ids of at least 8 characters, unique; every
`parentFolderId` names a declared folder; no cycles; trimmed names that pass n8n's
folder-name rules; at most 20 folders.
- **Needs a licensed instance** (`feat:folders`) with
`N8N_INSTANCE_AI_FOLDER_EXPLORATION_ENABLED=true`. An unlicensed instance fails the
restore with a hint rather than running the case without its folder. Locally, after
`/rest/e2e/reset`: `PATCH /rest/e2e/feature {"feature":"feat:folders","enabled":true}`.
- **Keep it on disk for now.** The LangTracer case-write API does not store
`seed.folders` or `workflows[].parentFolderId`, so the push refuses the case until
`n8n-io/lang-tracer` adds both.
#### Which opening shape — the agent is handed the workflow, or it has to find it
Two real conversations look the same in a case file but test different things, and
picking wrong makes the case harder than reality.
**Handed it.** The user is looking at a workflow and opens the assistant: "why is
this failing?", "add error handling". They never name it — the editor sends the
workflow along as a resource reference and the agent resolves it by **id**. Declare
that with `attach` on the opening turn:
```json
"conversation": [
{ "role": "user", "text": "why is this failing?", "attach": { "workflow": "wKk3RmT9xQ2bVn7L" } }
],
"seed": {
"mode": "inline",
"messages": [ … ],
"workflows": [ { "id": "wKk3RmT9xQ2bVn7L", "name": "Batch loop", … } ]
}
```
The id is the one the **seed declares**; the harness swaps in the per-run id, so you
track nothing. Only the opening turn may carry `attach` (an attachment is a hand-off,
not something a user re-sends), and it must name a workflow the inline seed declares —
both are refused at load rather than ignored.
**The opening often has no text at all** — the user opens the assistant on a workflow
and waits for it to speak first. Keep `"text": ""` when that's what happened; it's the
faithful shape, and openings with no user text jumped from 1% to 31% of the corpus when
the editor hand-off shipped, so it is not an edge case. Note that an empty text is
valid *only* alongside `attach`: the chat API rejects a message that is empty with
nothing attached ("Either message or attachments must be provided"), so the two stand
or fall together.
**Has to find it.** The user refers to the workflow in words: by name ("the Wait node
in *Generate leads* failed") or loosely ("the batch image workflow"). No `attach` —
finding it *is* part of what the case tests. This also works when the seeded history
already shows the agent building it, since the id is in its own record.
Get this wrong in the "handed it" direction — omit `attach` on a conversation that
really had one — and the agent has to guess from prose that deliberately names
nothing. It will list workflows and pick, or ask which one, and you will score a
clarification failure the real user never hit.
#### Seeded faults must really fail
`seed.priorRuns[].hints` steer the mock for the staged run's outbound calls and
nothing else. The builder's own runs are never mocked, and Code, Set, IF, Filter
and Merge run for real. So:
- **The fault must fail in n8n.** A Code node that returns a plain object does not
fail; n8n wraps it as one item. Return an array of plain values, reference a
node that does not exist (`$('Missing')`), or throw.
- **The fault fires before any external call.** The builder's rerun reaches a
fixture host and dies there (DNS, 401), so a fault behind an HTTP node is never
reached. Never grade the rerun itself; grade the fix with `outcomeExpectations`
and the harness's `executionScenarios`.
- **An empty branch needs a Filter that keeps nothing.** Split Out over an empty
array still emits a placeholder item.
- **Walk the item flow before asserting a count.** A Slack node fed three items
posts three times.
#### Before you ship a seeded case — three checks
1. **The defect still bites.** A seed whose workflow isn't broken any more makes the
case a silent no-op that passes forever.
2. **Run it once with the seed removed. It must fail.** Copy the case, delete `seed`,
run both. If the no-seed copy also passes, the seed isn't carrying the test. Name
the copy so it doesn't share a `--filter` substring with the real case
(`control-noseed-<slug>` works; `<slug>-noseed` would match both).
3. **The workflow's skeleton is untouched** if you scrubbed it from a real one: node
types, versions and connection topology byte-identical, only string leaves moved.
Don't grade a seeded case on cost, turn count, `messageBudget`, or "fixed it in one
build". A seeded thread starts with an empty sandbox, so the agent re-reads the
workflow from the database and re-derives SDK source a real resumed session would
still have on disk. The bias is *harder* than reality, so those numbers read worse
for a reason that has nothing to do with the builder.
---
## Context cases (long conversations, token cost, compaction)
A context case asks whether the agent still works once the conversation is
long. Two questions, two mechanics.
### The judge already has the token numbers
Every case's judge context carries token ground truth, so no new field is
needed to grade cost:
- Each transcript turn header reads
`Turn 2 (3 steps, 47446 tokens in [41220 cached], 812 tokens out)`.
- A **Total** block sums every LLM step across the whole build, main run and
resumes, plus a cache read/write split and a **Fixed overhead** line — the
opening step's input, which is instructions and tool schemas plus one user
message, so it reads as the floor cost of any turn on that instance. Use it
to separate "the prompt got bigger" from "this conversation got expensive".
So `processExpectations` may reference consumption directly. Write them as
statements about *behaviour the numbers reveal*, not as thresholds:
```json
"processExpectations": [
"The agent answers the second question without calling get_node_details on the Notion node again — it read that schema in turn 1 and the answer needs nothing new.",
"Turn 2's input tokens are mostly cache reads, which is what an unchanged prefix looks like. A turn that re-reads the history from scratch shows a cache-read share near zero."
]
```
Absolute token budgets rot: a prompt edit moves every number and the case goes
red for a reason that has nothing to do with the agent. Grade the *shape* —
what got re-read, what came from cache, which turn is the expensive one.
Two limits to know before you write a cost expectation:
- It needs `N8N_INSTANCE_AI_RUN_DEBUG_ENABLED=true` on the instance under test.
Without it the blocks render `(no run debug captured)` and the expectation is
unjudgeable — it fails or passes at random.
- The capture hooks **only the orchestrator's own stream**. A workflow build
runs inside that loop, so a workflow case's numbers are its real cost. A
delegated **Agent** build (the `build-agent` tool) runs on a separate stream
with no hooks, so its tokens are missing from every block. Don't write a cost
expectation on an `data/agents/` case.
### `requiresMemoryCompaction` — grade the post-compaction window
Observational memory compacts a long thread: an Observer summarises it into
observation rows, the summary is rendered into the system prompt, and the early
turns are masked out of the agent's window. A case that tests whether a decision
*survives* that has a setup problem — production compacts at 30k tokens of
visible message content, which is a conversation too long to hand-author.
Set the flag and the harness handles it:
```json
"requiresMemoryCompaction": true
```
Two things happen, both per-thread, so no other case in the run is affected:
1. The harness sends a low observer threshold with every turn of that thread
only: **1,000 tokens** of visible message content, the request field's floor.
The seed plus the live turns must cross it, or the observer never runs — a
short seed does not test memory, it reports **not judged** (below). Size the
seed generously; a few real turns of tool output cross 1,000 tokens easily.
The Observer's prompt, masking and cursor logic are the production ones at
any threshold.
2. After the build, the harness checks the premise actually held. If it did
not, every expectation on the case is reported **not judged** — an
`incomplete` verdict, excluded from scoring, not a red.
The premise check is the point of the flag. Uncompacted, the raw early turns
are still in the window, so the agent answers off them and every expectation
passes for free — a green that tests nothing.
The evidence is structural. `GET /rest/instance-ai/eval/threads/:threadId/memory`
returns the observation rows and the compaction cursor from their own tables. A
cursor means the observer ran and everything up to `lastObservedMessageId` is
masked out; the rows are what replaced it. Missing either, the case had nothing
to test and is reported not judged. Nothing reads the system prompt, so this
needs **no debug flag** and a prompt or SDK rename cannot quietly turn "never
compacted" into the answer.
The judge never sees the premise check — it grades the conversation, not whether
the harness configured the scenario. A misconfigured lane must not read as a
quality regression.
### Grade the summary, not just the reply
The judge *does* see the observation rows, as ground truth:
```
## Observational memory after compaction (ground truth — do not recount)
- [CRITICAL] User ruled OUT the native Slack node; will post through the HTTP
Request node instead, because the workspace is pinned to a legacy Slack API
version. Do not revisit.
- [CRITICAL] Alert fires when queue depth goes above 4700 items.
```
So write expectations at **both** layers:
```json
"processExpectations": [
"The observational-memory block retains the alert threshold as 4700 queued items. A summary that keeps the topic but loses the number is the specific failure this looks for.",
"In its reply to the final turn, the agent gives the alert threshold as 4700 queued items."
]
```
The first grades the memory system, the second grades the agent. Grading only
the reply is weaker than it looks: the agent can answer correctly from the
unmasked tail, or guess well, and the case goes green while memory dropped the
detail. The `[MARKER]` is the Observer's own priority label, so "retained as
CRITICAL" is checkable too.
### Writing the conversation
The shape that makes a compaction case real:
1. **State the anchors once, in a `user` turn near the start of the seed**, and
never restate them. Anything the agent repeats later lands *after* the
cursor and survives for the wrong reason.
2. **Make live turn 1 off-topic.** Its answer must not re-surface the anchors
into the post-cursor window. Compaction runs at the end of it.
3. **Ask for the anchors back in live turn 2.** That turn runs against the
compacted context.
4. **Force the antecedent** — a lost anchor must produce a *confident wrong*
answer, not a vague one. "Posting goes through HTTP Request, not the native
Slack node" works because the Slack node is the obvious guess. "The
threshold is 4700" works because a guess reaches for 5000. See
[negative expectations need a forced antecedent](SKILL.md#core-principle-all-shapes).
Tag these `context-consumption` and put them in a `context` dataset, so a
tier that needs run debug enabled can select them.

View File

@ -65,18 +65,11 @@ Workflows without the suffix — real ones, and anything the agent built — are
touched. So `--keep-workflows` is safe to use on a seeded case; the leftover is
cleaned up by the next run rather than contaminating it.
**Seeded agents are not evicted** — they get a fresh id per run but keep their
authored name, so `--keep-workflows` on an agent-seeding case leaves one behind
and they accumulate under the same name. That can't misdirect a later run (the
live turn is bound to its own agent by id), but it does clutter what the `agents`
tool lists. Delete them yourself when calibrating:
`DELETE /rest/projects/<projectId>/agents/v2/<agentId>`.
## Case source: disk vs langtracer
| Source | When to use it |
|---|---|
| **`disk`** (default) | **Preferred for local development** — authoring and calibrating the case in front of you: drop the JSON into `data/workflows/`, `--filter` it, iterate. Also the only home of the `agents` tier and of a `replay`-seeded case (reconstructed from a trace at run time, so no suite can hold it); since the corpus migration the directory holds only those, not the full suite. |
| **`disk`** (default) | **Preferred for local development** — authoring and calibrating the case in front of you: drop the JSON into `data/workflows/`, `--filter` it, iterate. Also the only home of the `agents` tier and the seeded carve-out cases; since the corpus migration the directory holds only those, not the full suite. |
| **`langtracer`** (`--source langtracer --suite baseline`) | Bigger runs (the full corpus or a whole tier), re-running specific cases that already live in the suite, and CI — which always runs this way. Needs `LANGTRACER_URL`/`LANGTRACER_API_KEY` in your env. |
## Configuration & secrets
@ -218,44 +211,6 @@ deliberately does not re-sync tier-only edits to an existing case. **Only
promote to `pr` after `--iterations 5+` shows it's reliably green** — a flaky
case in the gate poisons it.
## Calibrating a batch of new cases on the dispatchers
One case costs 10–20 minutes per iteration on a laptop, and docker lanes have
crashed it. A batch calibrates on the LangTracer dispatchers:
1. Author the JSON locally. Static checks only: a `--dry-run` push (schema) and
the similarity check against the verification suites.
2. Push the batch to its suite with the tag `calibration-pending`.
3. Dispatch one manual sweep for those case ids:
`gh workflow run eval-run.yml --repo n8n-io/lang-tracer -f image=n8nio/n8n:nightly -f case_ids=<ids> -f iterations=3 -f trigger=manual`.
`trigger=manual` keeps it out of the nightly baseline.
4. Read `list_eval_runs` → `get_eval_run`. Classify each red: **builder** (keep
it red, tag `capability-gap-finding`, propose a ticket), **harness** (fix it or
design around it; never leave it to charge the model), **authoring** (fix the
case).
5. Re-push the fixed cases and re-dispatch only those ids.
6. Swap `calibration-pending` for `calibrated` and stamp the evidence into the
description, e.g. "Calibration (Opus 4.8, N=3): sweep 174 3/4, sweep 176 4/4".
## Validating a harness change before merge
```bash
gh workflow run test-evals-instance-ai.yml --repo n8n-io/n8n \
-f branch=<your-branch> -f cache-sha=$(git rev-parse origin/<your-branch>) \
-f suite=<suite> -f filter=<slug-substrings> \
-f iterations=3 -f experiment-name=<change-name>
```
`cache-sha` is what puts a backend change under test: without it the workflow
restores the n8n image cached for master's head, and only the eval CLI runs from
your branch. A harness-only change works either way.
CI reads cases from LangTracer only. A case that exists only on disk goes into a
throwaway suite first (`eval:langtracer-push --suite <scratch-suite> <slug>`).
Never pass `experiment-name=instance-ai-baseline`; that name refreshes the shared
baseline. Results: the run log's summary table and the
`instance-ai-workflow-eval-results` artifact (`eval-results.json` + HTML report).
## Baselines & regression
When `LANGSMITH_API_KEY` is set, every run auto-compares against the most recent

View File

@ -1,24 +1,17 @@
# Sourcing cases from real conversations (LangTracer + LangSmith)
# Sourcing cases from real failures (LangTracer + LangSmith)
The strongest cases encode a **real** failure, not an invented premise. But the
case you keep is always one you **wrote**: a real thread tells you what to test,
and you author the case — prompt, workflow, seed and all — from what you learned.
That is what makes the committed fixture durable and free of anyone's personal
data. The [scrub recipe](#scrubbing-a-real-workflow-into-a-synthetic-seed) below is
that path, and it's the default.
The strongest cases encode a **real** failure, not an invented premise. Two
connections help you find one and confirm it — and neither is usually the
durable artifact. You author a synthetic case from what you learn (reach for
`seed.mode: "replay"` only per [`case-shapes.md`](case-shapes.md)).
Replaying a thread at run time (`seed.mode: "replay"`) is a **secondary mode**: a
local check that the failure is real, not something to commit. See
[Replaying a thread first](#replaying-a-thread-first-the-secondary-mode).
Two connections make either possible:
- **LangTracer — discover.** It analyses Instance AI conversations and clusters
them into **capability-gap themes** ("what fails, at scale"). Use it to find
high-frequency real failures worth encoding, instead of guessing a failure mode.
- **LangSmith — verify.** Eval runs and production conversations land in LangSmith
as traces. When a finding or a flaky result is ambiguous, read the trace to
confirm what happened — which tool calls fired, and their payloads.
- **LangTracer — discover.** It ingests real Instance AI conversations and
clusters them into **capability-gap themes** ("what fails, at scale"), and
stores each analysed conversation. Use it to find high-frequency real failures
worth encoding, instead of guessing a failure mode.
- **LangSmith — verify.** Eval runs and prod conversations land in LangSmith as
raw traces. When a finding or a flaky result is ambiguous, read the raw trace
to confirm exactly what happened — which tool calls fired, and their payloads.
## Connect the LangTracer MCP
@ -36,31 +29,14 @@ claude mcp add --scope local --transport http langtracer-hosted \
session start, so reconnect to pick it up. (The LangSmith MCP is usually already
connected.)
The MCP returns ids; the driver needs pages. Every thread, cluster, and case you
cite while sourcing has a web URL off the same base — `<base>/conversations/<threadId>`,
`<base>/clusters/<id>`, `<base>/test-cases/<id>` — so cite them as links, not bare
ids. Full table in
[Share links, never bare ids](SKILL.md#share-links-never-bare-ids).
## Discover → verify → encode
1. **Scan cluster themes** — `list_cluster_runs` / `get_latest_cluster_run`
return capability-gap themes with a `label`, `summary`, and `mechanism`: the
real, recurring failure modes.
2. **Pull the conversations.** Start from what LangTracer already detected.
`list_signal_rates` lists every code check and label rule with its rate over a
window: what fails most right now. Then `list_conversations` with `signals`,
one token per signal, ANDed — `check:has_nodes`,
`label:task_outcome=agent-gave-up`, `analysis:bad`, `keeper:*`, `review:*` — puts
the matching entries on each row, each with its evidence (the verdict comment,
the judge's words, Keeper's reason) and its version. On a candidate,
`get_conversation_signals` shows what every source said and which ones agreed:
a thread flagged by a code check, the judge and Keeper is a stronger source
than one flagged by a single model pass. A source marked `not_run` never looked
at the thread, so read its silence as unknown, not as clean. The older path
still works: `list_conversations` (e.g. `verdict:"bad"`, `analyzed:"yes"`) →
`get_conversation` (raw trace) + `get_conversation_analysis` (findings).
`get_linear_ticket_context` links a thread to its ticket. To target
2. **Pull the conversations** — `list_conversations` (e.g. `verdict:"bad"`,
`analyzed:"yes"`) → `get_conversation` (raw trace) + `get_conversation_analysis`
(findings). `get_linear_ticket_context` links a thread to its ticket. To target
**execution failures**, add `funnelDrop:"05"` (built + launched an execution
that never *succeeded*). **Caveat: the funnel drop is a *conversion* signal, not
a build-quality one** — most `funnelDrop:"05"` threads are healthy builds that
@ -72,287 +48,19 @@ ids. Full table in
(was `build-workflow` called on this turn?), which it reads reliably — but a
content-dependent claim ("invented ID", "missing node") can be wrong when it
couldn't see the built workflow. Confirm against the raw trace before building
a case around it. The same goes for a signal's `evidence`: it is what that
source saw, not the trace.
4. **Optionally record the selection as an observation.** Offer to save why this
thread was selected and what the developer observed. This is a best-effort
LangTracer note, not a gate: skip it when the developer declines or has not
stated a preference, and continue if the write fails. See
[Optional: record the selection as an observation](#optional-record-the-selection-as-an-observation).
5. **Encode a durable synthetic case** — turn the confirmed failure into an
a case around it.
4. **Encode a durable synthetic case** — turn the confirmed failure into an
authored case ([SKILL.md](SKILL.md), [`case-shapes.md`](case-shapes.md)). The
failure mode is the anchor; the conversation is yours to write, in the user's
voice. **Write down the precondition, not just the failure mode** — the state
the thread was in when the call went wrong, which is usually narrower than the
theme label suggests and is often stated outright in the assistant text just
before the failing call. Get this wrong and the case grades green because it
never set the situation up; see
[First reproduce, then reclassify](SKILL.md#first-reproduce-then-reclassify).
6. **Push it to a curated suite** (don't commit the JSON) with
voice.
5. **Push it to a curated suite** (don't commit the JSON) with
`eval:langtracer-push` — see
[Push to a lang-tracer suite](SKILL.md#push-to-a-lang-tracer-suite). An `inline`
seed rides along with the case. Exception: a `replay` case is refused and listed
under `skipped:` — it's reconstructed from a trace at run time, so it dies when
that trace is pruned and has no durable home; that's exactly why step 5 turns the
confirmed failure into a durable synthetic case.
## Optional: record the selection as an observation
After the raw trace confirms the finding, offer the developer one optional
record before drafting:
- **Why did you choose this thread?** Capture why it is useful eval coverage,
such as impact, recurrence, reproducibility, or a gap in the current suite.
- **What did you observe?** Capture the concrete behavior, the expected behavior,
and the run or turn that contains the strongest evidence.
If the developer opts in, call LangTracer's `create_observation` tool. Map the
record like this:
- `sourceThreadId`: the selected imported thread.
- `sourceRunId` and `anchorRunId`: the evidence run, when the finding is narrower
than the whole conversation.
- `kind`: `capability_gap` for a confirmed failure, `regression` for successful
behavior worth protecting, or `binary_check` when that is the intended output.
- `description`: why the developer selected this thread.
- `openCodeNote`: what the developer observed, in reviewer voice.
- `expectedBehavior`, `failurePattern`, and `proposedCheck`: the rule, the
observed deviation, and the check the eval should enforce.
- `severity` and `failureClass`: include them only when the evidence supports the
classification.
- `isTestCaseCandidate`: `true` because the developer selected it for eval
authoring.
This record is optional. Do not pause an autonomous run to request it when the
driver has not already opted in. In checkpoint mode, offer it once with the
selection proposal. A declined offer, unavailable MCP tool, or failed write must
not stop the eval workflow. Report a successful observation by id with the source
conversation link; otherwise omit it from the handoff.
Calibration can sharpen the note. If an observation exists, use
`update_observation` only for mutable details such as the expected behavior,
failure pattern, proposed check, severity, failure class, or reviewer note. Its
`kind` and source provenance are immutable, so classify the source evidence
before creation. Both observation tools are LangTracer-local. Do not create or
update LangSmith feedback for this record.
## Was the workflow handed over? (sourcing an `attach` opening)
Whether the opening turn carries `attach` is a **fact about the thread**, not a
judgement call — and guessing wrong makes the case harder than reality. Read it two
ways, depending on when the thread was imported:
- **Recorded.** Turn 0 of `get_conversation` carries
`resourceAttachments: [{ "type": "workflow", "id": "…" }]`. That's the editor
hand-off verbatim. Only kind and ids are stored, never the workflow's name.
- **Inferred**, for threads imported before n8n traced it. The tell is an opening turn
whose `userMessage` is **empty** — the editor's context block is stripped before the
trace, so a hand-off where the user typed nothing leaves a blank record. Corroborate
with an early `workflows[get]` on a workflow the user never named.
Then use the id to recover the workflow itself: find the tool call whose *input*
carries that `workflowId` (usually `workflows[get]`, `full: true`) — its **output** is
the workflow, nodes and connections and parameters, and that is what you scrub into
`seed.workflows`. The id alone is only a join key; it addresses the user's own
instance, so it is worthless by itself.
Two things to expect:
- **The agent may never have read it.** Then you have an id and no content, and the
workflow is yours to write — keep the topology plausible for the complaint.
- **The opening often has no text at all.** Keep it that way; see the empty-opening
note in [`case-shapes.md`](case-shapes.md).
## Scrubbing a real workflow into a synthetic seed
A seeded case carries a workflow inside it, and that file is committed and runs for
as long as the case lives. So the workflow has to be **written, not copied**: read
the real one, then author a stand-in that keeps its technical shape and none of its
personal data. One careful pass, with a person reading it, and the committed fixture
is clean from the start — nothing further down the line will clean it up for you.
### What to look for
Identifying data hides in more places than the obvious ones:
- **Parameters** — email addresses, phone numbers, channel names, folder paths,
spreadsheet / document / table ids, internal hostnames, and any token someone
pasted into a header or a query string. A token sitting in a parameter is not a
credential, so nothing strips it for you.
- **Free text inside the workflow** — sticky notes and node notes, string literals in
a Code node, sample rows in a Set node. Sample rows are often real records.
- **The workflow's own name**, which often carries a company or a person
("Acme lead sync", "Chase invoices — Dana"). This repo is public, so a real
customer's name must not appear anywhere in a case; use a placeholder like
`Acme Corp`.
- **The earlier messages you seed alongside it** — someone's own wording, their name,
their employer, links they pasted. Write those turns yourself, in a user's voice.
- **Recorded tool calls, if you keep any.** Measured on a real thread, this is where
the leaks actually were — the workflow was clean, while the outputs of
`workflows[list]` / `get` / `get-as-code` and `data-tables[list]` carried a live
credential id, the project id, the person's entire workflow inventory, table schemas
down to every column name, and a webhook id. Each `toolCallId` also carries the
source run id. Read the messages: a regex pass over that seed caught one category out
of about eight.
Provenance is the one thing worth keeping: note the source thread id in the case's
`description` so anyone can find the original later. An id points at the conversation
without carrying any of it.
### Which version of the workflow to start from
Take the workflow as it stood when the turn you're testing began, not the state it
ended the thread in.
- **The assistant built it.** `list_conversation_workflow_builds` lists each build
with a `seq` and `turnRunId`, so you can find the one just before your turn, and
`get_conversation_workflow_build` returns it. Its `contentStatus` says what you
actually get: `stored-json` is ready to work with; `compilable-source` means only
the builder's code was kept, which
[`parseSeedWorkflowCode`](../../../packages/@n8n/instance-ai/evaluations/harness/parse-seed-workflow.ts)
turns back into a workflow; `unrecoverable` and `unrecoverable-intermediate` mean
there is nothing to read. About one build in five has no stored JSON, so don't
assume. And don't reach for a **later** build instead — that one is the workflow
after the fix, so the case would test the repair rather than the problem.
- **The person already had it.** When the editor hands a workflow to the assistant,
the assistant looks it up by id — so the workflow you want is the *result* of the
`workflows[get]` call whose input mentions that id.
- **Nothing was stored.** Older threads pre-date this being kept at all, and the read
says so in a `note` rather than returning an empty list. An empty answer never means
"there was no workflow". Two ways out: replay the thread locally and take the
workflow the harness rebuilds from the trace (see below), or write the stand-in
yourself — the simplest set of nodes that can still show the problem. Hand-authored,
there's no original to compare against, so the before-you-ship checks are the only
safety net left.
### What to keep, and what to replace
Keep the technical shape, since that is what the case tests. Replace the values that
point at a real person, company or account.
| Keep exactly as it is | Safe to replace |
|---|---|
| node `type` and `typeVersion` | URLs, hostnames, webhook paths |
| how the nodes are wired (`connections`), and the node names it refers to | email addresses, people's names, channel names |
| parameter names, and the shape of an expression (`={{ $json.url }}`, `$('Prepare rows')`) | spreadsheet / document / data-table ids |
| settings that change behaviour: `batchSize`, `resource` / `operation` / `mode`, the shape of a condition | sticky notes, node notes, Code-node text, sample data |
**Swap like for like** — a URL stays a URL, an id keeps the same look and length.
That's not only about readability:
- Loading a seed checks every node for `id`, `name`, `type`, a numeric `typeVersion`,
a two-number `position` and a `parameters` object, and rejects the whole seed if one
is missing. So don't delete fields while you're in there.
- Ids get swapped for fresh ones by a plain search-and-replace across the seed.
Anything under 8 characters is rejected outright, and a short id that squeaks
through is exactly how you end up rewriting unrelated text that happens to contain
it.
On one node that comes out as: the host, the id and the key change; the expression,
the reference to another node and the parameter names don't.
```jsonc
// before
"url": "=https://acme-internal.example/v2/orders/{{ $('Get order').item.json.orderId }}",
"headerParameters": { "parameters": [{ "name": "X-Api-Key", "value": "sk_live_9f2c8b…" }] }
// after
"url": "=https://api.example.com/v2/orders/{{ $('Get order').item.json.orderId }}",
"headerParameters": { "parameters": [{ "name": "X-Api-Key", "value": "sk_test_placeholder" }] }
```
**Trim as you go.** The seed is one field on the case, with a 256KB ceiling —
`load_skill` bodies alone can account for most of it. Trimming has a rule of its own:
**keep the shape here too.** Filtering a list output changes what the agent believes
exists, and a `{ "note": "…" }` where a result belongs invents a shape no tool ever
returns. Shortening the text inside a block is fine; to get rid of a call, drop the
whole tool-call block. If you keep a build-workflow call, its `output.workflowId` has
to match a workflow the seed declares — see [`case-shapes.md`](case-shapes.md).
### Three easy mistakes
**Renaming a node.** A node's name is how the rest of the workflow points at it — the
wiring is keyed by name, and expressions call it by name (`$('Old Name')`). It can
also turn up in recorded builder code and in the prose. Miss one reference and the
workflow quietly stops working, which is the kind of over-cleaning that leaves the
case testing nothing. Node names are rarely identifying on their own, so the default
is to leave them be. If you do rename one, `grep -c 'Old Name' <case>.json` should
come back 0. (Workflow names are a different story: each run gets its own copy with a
`[seed …]` suffix, and a seed declaring two workflows with the same name is refused,
because there's no way to tell which one a mention refers to.)
**Replacing the value the case is about.** In
`http-keep-generic-credential-unknown-service`, the host `queue.fal.run` matters
precisely because n8n has no built-in credential for it — swap in a well-known host
and the case tests the opposite thing. Same with the invented column names in
`flags-unverified-sql-identifiers`. Expectations quote values too ("posts to
`#growth`", a particular model name), so a replacement has to be made in both places,
or the expectation loosened the way others already are for values that don't matter
("an unset or placeholder value is acceptable — it's something to fill in at setup,
not a build mistake"). Knowing what the case asserts is part of scrubbing it, which
is why this is a person's job and not a script's.
**Tidying up a node's version.** Split In Batches v2 lists its outputs as
`['loop', 'done']`; v3 lists them as `['done', 'loop']`. Same two outputs, opposite
meaning — so bumping the version quietly sends the loop the other way while the wiring
still looks untouched. Leave versions as the conversation had them.
### Check that only values changed
Compare the JSON you started from with the workflow you put in the case:
```bash
leaves() { jq -r '{nodes,connections} | paths(scalars) as $p | "\($p|map(tostring)|join("."))\t\(getpath($p)|tojson)"' | sort; }
diff <(leaves < original-workflow.json) <(jq '.seed.workflows[0]' <case>.json | leaves)
```
`leaves` prints one line per value in the file, so the diff is a plain list of what
you changed. Every line should be a value you meant to replace. A line mentioning
`type`, `typeVersion`, `connections` or a behaviour setting means the shape moved, not
just the values; a line on one side only means something was added or dropped. Keys
that aren't part of the graph (`settings`, `meta`, `pinData`) are ignored, so the
original compares cleanly against the trimmed version.
This is the third of
[Before you ship a seeded case](case-shapes.md#before-you-ship-a-seeded-case--three-checks).
The other two — the problem is still there, and the case fails without the seed — are
what catch a clean-up that kept the shape but lost the point.
### What the harness already handles
You don't need to do these by hand:
- **Credential references on nodes resolve against the case's `credentials[]` as the
seed loads.** A reference is kept only when exactly one seeded credential has that
type and display name; any other reference is dropped. On a `published: true`
workflow an unresolved reference fails the restore instead.
- **Data tables are columns only.** The seed format has no place for rows, and a
`rows` key is rejected rather than quietly removed — so table contents can't come
along by accident.
- **A seed workflow is only `id`, `name`, `nodes`, `connections` and an optional
`published` flag.** `published: true` activates the workflow on restore, the way the
user's publish left it. Pinned example data, instance metadata and settings never
travel.
- **Ids and workflow names are per run.** Ids are replaced with fresh ones and names
get a `[seed <8hex>]` suffix, with mentions updated in the prose — though not inside
node definitions or recorded tool calls.
Everything else in the seed is yours to check.
### Replaying a thread first (the secondary mode)
Scrubbing takes work, and sometimes you want to know the failure is real before you
put that work in. That's what `seed.mode: "replay"` is for: give it a thread id and
the harness rebuilds the conversation from its trace at run time, then drives the
turn you care about. Nothing about the thread lands in the repo — the case holds only
the id.
Treat it as a local check, not a case. The trace it depends on ages out in about two
weeks, so a committed replay case stops working; and it stands up someone's real
conversation on the eval instance, which is exactly what scrubbing exists to avoid.
So run it, confirm the failure, then scrub the workflow it hands you into a seeded
case that will still be there next quarter. Details and limits in
[`case-shapes.md`](case-shapes.md).
[Push to a lang-tracer suite](SKILL.md#push-to-a-lang-tracer-suite). Exception:
seeded cases (any `seed` mode) can't be pushed — the case-write API has no
`seed` field, so the push lists them under
`skipped:`. And a `replay` case shouldn't be committed either — it dies when
its trace is pruned or deleted — so it has no durable home; that's exactly why
step 4 turns the confirmed failure into a durable synthetic case.
## Sourcing a regression baseline (successful builds)

View File

@ -14,9 +14,6 @@ compatibility:
Create a Linear ticket or GitHub issue for: **$ARGUMENTS**
Write all titles and descriptions in ASD-STE100 Simplified Technical English:
use short sentences, the active voice, and one instruction for each sentence.
## Determine Target
Decide where the issue should be created based on user input:
@ -76,20 +73,17 @@ Structure the description using markdown headers. Use the appropriate template:
**For features / enhancements:**
```markdown
## Goal
[What this adds and why: the user problem it solves]
## Summary
[One-paragraph overview of what this adds or changes]
## Background
[Current state and the gap, plus the technical context needed to plan the work: relevant constraints, prior findings, and links to any related investigation.]
## Problem
[What limitation or gap exists today]
## Scope
[Concrete list of what changes. Name the files or areas to create or modify and any existing pattern to follow.]
## Acceptance criteria
[Testable outcomes, including automated tests.]
## Proposed solution
[How it should work — technical approach if known]
## Out of scope
[What this explicitly does not cover]
[Explicitly note what this does NOT cover, if helpful]
```
**For tech debt:**
@ -115,19 +109,13 @@ Structure the description using markdown headers. Use the appropriate template:
```markdown
## Goal
[What question(s) are we trying to answer]
[What question are we trying to answer]
## Context
[Why this investigation is needed now]
## Questions
1. [Specific question to resolve]
## Expected output
[What deliverable is expected: RFC, PoC, decision document, path matrix, etc.]
## Acceptance criteria
[How we know the spike is done: each question answered, deliverable produced]
[What deliverable is expected — RFC, PoC, decision document, etc.]
```
#### Attachments (Screenshots / Videos)

View File

@ -34,7 +34,6 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
- `benchmark` - Benchmark CLI changes
- `core` - Core/backend/private API
- `editor` - Editor UI changes
- `engine` - New workflow execution engine v2 (@n8n/engine package & engine-v2 module in cli)
- `* Node` - Specific node (e.g., `Slack Node`, `GitHub Node`)
### Summary Rules
@ -43,7 +42,7 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
- Capitalize first letter
- No period at the end
- No ticket IDs (e.g., N8N-1234)
- Add `(no-changelog)` suffix to exclude from changelog, unless the type already excludes it
- Add `(no-changelog)` suffix to exclude from changelog
## Steps
@ -78,24 +77,11 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
body structure, then populate each section with actual content before
creating the PR:
- **Summary**: describe what the PR does
- **Why**: explain why the change is needed (see Why below)
- **How to test**: describe how to test the changes. Include an example workflow if appropriate
- **Related tickets**: add the Linear ticket URL (`https://linear.app/n8n/issue/[TICKET-ID]`) and any GitHub issue links
- **Checklist**: keep as-is from the template
- Add a "🤖 PR Summary generated by AI" at the end of the body
Before you create the PR, proofread the body with these checks. Rewrite
each sentence that fails a check:
1. The Summary and the Why together are under the word budget. If not,
first remove the details that the reviewer reads in the diff, then
shorten the Why.
2. Each sentence names who does what: a component, a process, or a person
as the subject, with a plain technical verb.
3. Each domain term has a plain definition where it first appears.
4. Each sentence is true for the diff.
When a later commit changes a fact in the body, update the body.
```bash
gh pr create --draft --title "<type>(<scope>): <summary>" --body "$(cat <<'EOF'
<populated body based on pull_request_template.md>
@ -107,43 +93,9 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
Based on `.github/pull_request_template.md`:
Write for a reviewer who does not know this area of the code. The reviewer
must understand the change before they open the diff. Use ASD-STE100
Simplified Technical English: short sentences, active voice.
1. Keep the Summary and the Why together under about 200 words. A small PR
needs much less.
2. Define each domain term in plain words where it first appears. Do not use
a ticket ID or an internal name as an explanation.
3. Leave out details that the reviewer reads in the diff: error cases, status
codes, renamed fields, the order of calls, file lists, test names. Mention
one only when it changes behaviour that a reviewer could miss. Also leave
out what reviewers already know (framework behaviour).
4. Use plain technical verbs, not metaphors ("passes the value", not
"hands the value" or "lets the value through").
### Summary: what the PR does
1. Start with one to three sentences of background for a reader who is new to
the area. Then state what the PR does.
2. Write **prose** about behaviour, not a list of code edits. Name a class or
a function when it represents an essential/core concept and it helps
the reader to navigate the change.
3. If the PR changes no product code, say so in the first sentence
(for example: "This PR only adds tests.").
4. Do not justify the change here. The justification goes in the Why.
5. Include screenshots or videos for UI changes.
### Why: the reasoning
Always fill the `## Why` section. The Summary and the core of the
Why are what the reviewer needs most. Keep the core short:
1. Do not repeat the Summary.
2. State the problem or the constraint that makes the pre-existing code unfit.
3. If there are accepted gaps, or changes for existing code paths, state them
in a separate short paragraph.
Add rejected alternatives only when a reviewer would otherwise ask about them,
if any. Use one or two sentences for each alternative: state the concrete
reason it fails. Write an alternative that was never built in the conditional.
Together, the alternatives must be shorter than the core Why section.
### Summary Section
- Describe what the PR does
- Include screenshots/videos for UI changes
### How to test Section
- Explain how to test the changes
@ -191,7 +143,7 @@ feat(API)!: Remove deprecated v1 endpoints
### No changelog entry
```
feat(core): Experimental feature not enabled yet (no-changelog)
refactor(core): Simplify error handling (no-changelog)
```
### No scope (affects multiple areas)

View File

@ -110,7 +110,6 @@ Run through this before requesting review. Each item is a real, recurring review
- [ ] Identifiers go through **`escape.tableName(...)` / `escape.columnName(...)`**. Never hand-write `n8n_table` prefixes. — [Always escape identifiers](#always-escape-identifiers)
- [ ] **Match column type to value semantics.** Native `uuid` for UUIDs, `timestampTimezone()` for timestamps, a numeric type for numbers, `bool` for booleans, `json` for structured data. Never `varchar` as a catch-all. — [Column types](#column-types)
- [ ] **Pick the narrowest sane type within that category:** `int`/`smallint` not `bigint` when range allows; `text` not `varchar(255)` for unbounded strings; never `double` for version numbers. — [Column types](#column-types)
- [ ] **A generated `id` key is an integer** (`int.primary.autoGenerate2`). A UUID key is a UUIDv7. For a UUID or string key, write the reason in the PR description. A join table uses its foreign-key columns as the key, not an `id` column. — [Primary Keys](#primary-keys)
- [ ] **Default `notNull`**, relax only when justified. PK is implicitly NOT NULL. Migration's `notNull` matches the entity's nullability. — [NOT NULL and entity parity](#not-null-and-entity-parity)
- [ ] **Enum-like columns** carry `.withEnumCheck([...])` AND `.comment('explains values')`. Opaque IDs / unix timestamps / JSON shapes also get `.comment()`. — [Constrain enum-like strings](#constrain-enum-like-strings), [Add comments on columns](#add-comments-on-columns)
- [ ] **Every reference column has an explicit FK** with deliberate `onDelete`. Name FKs explicitly when SQLite recreate cycles risk duplicating them. Avoid polymorphic `(typeCol, idCol)` patterns. — [Foreign Key Constraints](#foreign-key-constraints), [General Design Guidance](#general-design-guidance)
@ -141,7 +140,7 @@ Rules that apply to every migration — schema or data, common or DB-specific. R
Migration files are named `{TIMESTAMP}-{DescriptiveName}.ts`. The timestamp must be strictly greater than every existing migration timestamp in this package (across `common/`, `postgresdb/`, and `sqlite/`). TypeORM runs unrecorded migrations in timestamp order, so inserting a value below the current max corrupts ordering on databases that have already executed the later migrations.
Use the generator — it picks a safe timestamp, writes the scaffold, and regenerates the migration index files (`sqlite/index.ts` and `postgresdb/index.ts` are gitignored build artifacts, generated from the files on disk by `scripts/generate-migration-index.mjs` — never edit or commit them):
Use the generator — it picks a safe timestamp, writes the scaffold, and registers the migration in the relevant `index.ts` files:
```sh
pnpm --filter=@n8n/db migration:new <Name> [--folder=common|postgresdb|sqlite]
@ -440,32 +439,11 @@ Don't set the default of a `status` column to a terminal value — `"running"` m
### Primary Keys
Every table needs a primary key. **Use an integer key by default.** Read the options below in order. Use the first option that fits. Do not skip an option only because you like a later one better.
Every table needs a primary key. Choose the type in this order:
1. **Integer (default)** — `column('id').int.primary.autoGenerate2`. Use this for every new table. Use a different option only when one of its conditions is true. Integer keys are small and make joins fast. `autoGenerate2` uses Postgres `IDENTITY`. Do not use `autoGenerate`. It uses `serial`, which is deprecated.
2. **UUID** — `column('id').uuid.primary`. Use this only when one of these conditions is true:
- The application must know the ID before the insert. For example, the client makes the ID.
- The ID is in a URL or a public API. Users must not see how many rows exist or guess other IDs.
**You must use UUIDv7.** Make the ID in application code with the `uuid` package: `import { v7 as uuidv7 } from 'uuid'`.
Why: a UUIDv7 starts with a timestamp. Thus, new rows go near the end of the index. A UUIDv4 is random, for example `randomUUID()` from `node:crypto`. New rows go to random positions in the index. This splits index pages and makes the index larger.
A UUIDv7 does not give an exact creation order. Two IDs made in the same millisecond can sort in the wrong order. Clock differences between hosts can also cause this. If you need the exact order, sort by a timestamp column.
Do **not** chain `.autoGenerate2` on `.uuid`. The DSL throws an error. The database default makes a UUIDv4. Also, `uuid_generate_v4()` needs the `uuid-ossp` extension. Some managed Postgres services, such as Supabase, do not have it. Use `.uuid`, not `.varchar(36)`.
3. **String** — `column('id').varchar(36).primary`. Use this only when one of these conditions is true:
- The entity already uses IDs in a format that is not a UUID. For example, the public API or the UI already shows nanoid IDs for this entity.
- The UI shows the ID to users. A UUID is too long to read, copy, or type.
Entity IDs are usually nanoids with 16 characters.
**Give a reason for every key that is not an integer.** Write the reason in the PR description. Reviewers will ask for it. These reasons are not sufficient:
- "Other tables use string IDs."
- "This module already does it this way."
- "We might need it later."
**These options apply to a generated `id` column only.** A join table that only links other tables does not need an `id` column. Use its foreign-key columns as a composite key (see below). A foreign-key column always has the same type as the key that it refers to.
1. **Integer** — `column('id').int.primary.autoGenerate2`. Preferred for new tables: compact, fast joins, no ordering surprises. `autoGenerate2` uses Postgres `IDENTITY` (preferred over the deprecated `serial`-based `autoGenerate`).
2. **UUID** — `column('id').uuid.primary`. Use when IDs are generated client-side, exposed in URLs, or need to be unguessable. Generate UUIDs in application code via `randomUUID()` from `node:crypto`; do **not** chain `.autoGenerate2` on `.uuid` (the DSL throws — `DEFAULT uuid_generate_v4()` fails on managed Postgres like Supabase because it needs the `uuid-ossp` extension in `public`). Use `.uuid` instead of `.varchar(36)`.
3. **String** — `column('id').varchar(36).primary` for IDs whose format isn't a UUID (e.g. nanoid-style IDs). Convention: nanoid length 16 for entity IDs.
**Keep ID-column types consistent across related tables.** Mixing `uuid` and `varchar(36)` for what is "the same kind of ID" creates JOIN footguns.
@ -718,7 +696,7 @@ If only Postgres needs the change, just put the file in `postgresdb/`; don't wri
### Postgres-version-aware UUID generation
`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, make a UUIDv7 in application code — see [Primary Keys](#primary-keys).
`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, generate at the application level with `randomUUID()` — see [Primary Keys](#primary-keys).
### SQLite doesn't enforce `varchar(N)` length

View File

@ -0,0 +1,33 @@
---
name: n8n:design-system
description: Guidelines on using Design System styles and components. Use when working on .vue files in packages/frontend. Triggers for tasks that include component architecture, styling, UI changes, or feature work.
---
# Design System
Comprehensive guide for building, styling, and using components in the frontend.
## When to Apply
Reference these guidelines when:
- Working on `.{vue|css|scss}` files in `packages/frontend`
- Adding new components to `packages/frontend/@n8n/design-system`
- Refactoring styles for Vue components
- Implementing new UI components or features
- Reviewing changes to UI
## Rules
- Follow guidelines in `packages/frontend/@n8n/design-system/src/styleguide/*.mdx`
- ALWAYS use CSS variables for styles from `packages/frontend/@n8n/design-system/src/css/_tokens.scss` or `packages/frontend/@n8n/design-system/src/css/_primitives.scss`. Use hard-coded values only when no suitable tokens.
- ALWAYS prefer using existing components from `packages/frontend/@n8n/design-system/src/components`. Prefer components that aren't marked `@deprecated`.
- Use `light-dark()` when alternating colors for ligh/dark mode
- When working with animations or transitions, ALWAYS prefer using mixins from `packages/frontend/@n8n/design-system/src/css/mixins/motion.scss`
- When reviewing animations, follow the guides in `rules/web-animation-guidelines.md`
- When reviewing UI changes or adding new components, follow `rules/web-interface-guidelines.md`
## Examples
- "Add a modal dialog for confirming workflow deletion" → Use `N8nDialog`
- "Add a dropdown to select workflow status" → Use `N8nDropdown` or `N8nSelect`
- "Add button with + icon to add new tiem" → Wrap `N8nButton` with `iconOnly` prop with `N8nTooltip` and wrap in `N8nTooltip`. Use `N8nIcon` and proper aria-label.
- "Add a destructive action button" → use `N8nButton` with `variant="destructive"`
- "Make background color white/black" → Use `var(--background--surface)` for white on light mode and "black" on dark mode
- "Animate the title in gracefully" -> Use `fade-in-up` mixin from `motion.scss` with `var(--duration--base)`

View File

@ -1,158 +0,0 @@
---
name: n8n:env-var-docs
description: >-
Writes and reviews the JSDoc on `@Env(...)` config properties. The docs team
extracts these comments from source into the public environment variable
reference, so they are user documentation, not code comments. Use when adding
or changing an `@Env` property, a config class, or reviewing a PR that touches
one.
---
# Environment variable docs
The JSDoc above every `@Env('...')` property is exported almost verbatim into
the [environment variables reference](https://docs.n8n.io/deploy/host-n8n/configure-n8n/basic-configuration/use-environment-variables)
on docs.n8n.io. The generated table has four columns: variable name,
description, data type, default. Your JSDoc is the description column. The
reader is a self-hosting operator, not an n8n developer.
Follow the [n8n docs style guide](https://docs.n8n.io/contribute/contribution-guide-for-n8n-docs/style-guide-for-n8n-docs)
for these comments. It overrides the ASD-STE100 rule in `AGENTS.md` for this
one case. The rules below are the parts of that guide that matter for a
one-paragraph description.
## Rules
- **Every `@Env` property has a JSDoc.** No exceptions. A missing comment
ships as an empty description cell.
- **Write for the operator.** Say what the setting does, when to change it,
and what happens when it's unset or wrong. Don't explain why the code is
written the way it is.
- **No code references.** No file names, class names, function names,
constants, license keys, or other internals the reader can't open. Values
the operator types into the variable are fine and often needed, for example
a credential type ID like `googleOAuth2Api` or a mode like `postgresdb`.
The test: would the operator paste this into a config file, or would they
need the source to know what it means? Details for developers go in a
regular `//` comment on the implementation, not in the JSDoc.
- **No attack scenarios.** Say what the setting protects, not how an
attacker would exploit it. This is a public page.
- **Don't repeat the default.** The default renders in its own column. Don't
write "Defaults to true" or "Default: 1 hour". Do say what the unset state
means when it's not obvious, for example "When unset, n8n uses the Redis
host".
- **State units and formats.** Say "in milliseconds", "in bytes", "a
comma-separated list", "a JSON object". The data type column only says
`Number` or `String`.
- **Use allowed values when the set is small.** "One of `sqlite`,
`postgresdb`." For a list, show one inline example in the sentence.
- **No JSDoc tags.** No `@example`, `@default`, `@see`, `@deprecated`. The
exporter copies the tag name into the docs. Put the example or the
deprecation notice in a sentence instead.
- **Product names:** "Gateway credits", not "n8n Connect" or "AI Gateway".
"Publish a workflow", not "activate". `n8n` is always lowercase.
- **Style:** present tense, active voice, second person. Use contractions.
No em dashes, use a comma or a new sentence. No "e.g." or "i.e.", write
"for example" and "that is". Spell out zero to nine, numerals from 10.
Values the operator types stay numerals, for example "Set to 0".
Sentence case. One idea per sentence. End every sentence with a period.
- **Plain words:** "use" not "utilize", "lets you" not "allows you to", cut
"powerful", "seamless", "simply", "it's important to note".
- **Length:** one to three sentences. If the setting needs more, the extra
belongs in a docs page the comment can't link to yet; keep the comment to
what fits a table cell.
## Examples
Bad, as found in the generated docs:
```ts
/**
* Instance-level enablement for n8n Connect (AI Gateway). Licensed instances
* are enabled by default; false opts out. Budget / free allowance still comes
* from the license (`quota:aiGatewayBudget`).
*/
```
Wrong product name, a restated default, and an internal license key. Good:
```ts
/**
* Whether Gateway credits are available on this instance. Set to false to
* turn them off on a licensed instance. The credit allowance comes from your
* license.
*/
```
Bad:
```ts
/**
* Time to live in milliseconds for Redis cache entries. Set to 0 to disable
* expiry. Default: 1 hour.
*/
```
The default is already in the table. Good:
```ts
/**
* Time to live in milliseconds for Redis cache entries. Set to 0 to keep
* entries until Redis evicts them.
*/
```
Bad:
```ts
/**
* Comma-separated list of credential types for which overwrites are skipped
* when the credential has been customized.
* @example `N8N_SKIP_CREDENTIAL_OVERWRITE=zohoOAuth2Api,salesforceOAuth2Api`
*/
```
The `@example` tag lands in the description as literal text. Good:
```ts
/**
* Comma-separated list of credential types to skip when applying credential
* overwrites, for example `zohoOAuth2Api,salesforceOAuth2Api`. n8n skips an
* overwrite only when a user has already changed one of its fields.
*/
```
Bad:
```ts
/**
* Bind OAuth credential callbacks to the browser that initiated the flow.
* When enabled, a session-scoped `n8n-oauth-binding` cookie is set on flow
* initiation and verified on callback. Defeats phishing attacks where an
* attacker injects an OAuth authorize URL into a victim's browser to land
* tokens on the attacker's credential.
*/
```
Attack narrative and an internal cookie name on a public page. Good:
```ts
/**
* Whether an OAuth credential connection must finish in the same browser
* that started it. When enabled, n8n rejects the callback if it arrives
* from a different browser.
*/
```
## Review checklist
When you review a PR that adds or changes an `@Env` property, check each
comment against this list and quote the fix in the review:
1. JSDoc present.
2. No code identifiers, file names, or internal keys.
3. No default restated.
4. Units, format, or allowed values stated.
5. No JSDoc tags.
6. No em dashes, Latin abbreviations, or filler words.
7. Reads correctly as a sentence in a table cell with no surrounding context.

View File

@ -20,4 +20,4 @@ Start with the relevant mode in [reference.md](reference.md):
- `Review` for auditing experiment changes.
- `Retire` for cleaning up completed or abandoned experiments.
When experiment work touches Vue components or user-facing copy, also follow `n8n:ui-design` and `n8n:content-design`.
When experiment work touches Vue components or user-facing copy, also follow `n8n:design-system` and `n8n:content-design`.

View File

@ -66,7 +66,7 @@ Use this guide for code under `packages/frontend/editor-ui/src/experiments/` and
- Wire experiments at the narrowest host surface that owns the decision.
- Keep experiment components inside the experiment folder unless they become generally reusable.
- All user-facing text in Vue code must use i18n. Follow `n8n:content-design` when writing or revising copy.
- Follow `n8n:ui-design` for Vue component structure, n8n design-system components, and CSS variables.
- Follow `n8n:design-system` for Vue component structure, n8n design-system components, and CSS variables.
- At the end of scaffold or wiring work, tell the user what remains manual, such as PostHog configuration, route guards, modal registration, or extra workflow payload files.
## Test

View File

@ -1,891 +0,0 @@
---
description: |
Manage stacked branches and pull requests with the gh-stack GitHub CLI extension. Use when the user wants to create, push, rebase, sync, navigate, or view stacks of dependent PRs. Triggers on tasks involving stacked diffs, dependent pull requests, branch chains, or incremental code review workflows.
metadata:
author: github
github-path: skills/gh-stack
github-ref: refs/tags/v0.1.0
github-repo: https://github.com/github/gh-stack
github-tree-sha: c95c8b5b4dd850f3fef007b304428f5684f2fb87
version: 0.0.9
name: n8n:gh-stack
---
# gh-stack
`gh stack` is a [GitHub CLI](https://cli.github.com/) extension for managing **stacked branches and pull requests**. A stack is an ordered list of branches where each branch builds on the one below it, rooted on a trunk branch (typically the repo's default branch). Each branch maps to one PR whose base is the branch below it, so reviewers see only the diff for that layer.
```
main (trunk)
└── auth-layer → PR #1 (base: main) - bottom (closest to trunk)
└── api-endpoints → PR #2 (base: auth-layer)
└── frontend → PR #3 (base: api-endpoints) - top (furthest from trunk)
```
The **bottom** of the stack is the branch closest to the trunk, and the **top** is the branch furthest from the trunk. Each branch inherits from the one below it. Navigation commands (`up`, `down`, `top`, `bottom`) follow this model: `up` moves away from trunk, `down` moves toward it.
## When to use this skill
Use this skill when the user wants to:
- Break a large change into a chain of small, reviewable PRs
- Create, rebase, push, or sync a stack of dependent branches
- Navigate between layers of a branch stack
- View the status of stacked PRs
- Tear down and rebuild a stack to remove, reorder, or rename branches
## Prerequisites
The GitHub CLI (`gh`) v2.0+ must be installed and authenticated. Install the extension with:
```bash
gh extension install github/gh-stack
```
Before using `gh stack`, configure git to prevent interactive prompts:
```bash
git config rerere.enabled true # remember conflict resolutions (skips prompt on init)
git config remote.pushDefault origin # if multiple remotes exist (skips remote picker)
```
## Agent rules
**All `gh stack` commands must be run non-interactively.** Every command invocation must include the flags and positional arguments needed to avoid prompts, TUIs, and interactive menus. If a command would prompt for input, it will hang indefinitely.
1. **Always supply branch names as positional arguments** to `init`, `add`, and `checkout`. Running these commands without arguments triggers interactive prompts. Branch names are used exactly as given — a name is never prefixed or transformed, so `gh stack add refactor/foo` creates a branch named `refactor/foo`.
2. **Always use `--auto` with `gh stack submit`** to auto-generate PR titles. Without `--auto`, `submit` prompts for a title for each new PR.
3. **Always use `--json` with `gh stack view`.** Without `--json`, the command launches an interactive TUI that cannot be operated by agents. There is no other appropriate flag — always pass `--json`.
4. **Handle multiple remotes.** If more than one remote is configured, pre-configure `git config remote.pushDefault origin`, or pass `--remote <name>` to the commands that accept it: `push`, `submit`, `sync`, `rebase`, and `link`. `checkout`, `modify`, and `trunk` resolve a remote but have **no `--remote` flag** — they rely on `remote.pushDefault`. With multiple remotes and no configured default, these commands exit with an error in non-interactive mode.
5. **Avoid branches shared across multiple stacks.** If a branch belongs to multiple stacks, commands exit with code 6. Check out a non-shared branch first.
6. **Plan your stack layers by dependency order before writing code.** Foundational changes (models, APIs, shared utilities) go in lower branches; dependent changes (UI, consumers) go in higher branches. Think through the dependency chain before running `gh stack init`.
7. **Use standard `git add` and `git commit` for staging and committing.** This gives you full control over which changes go into each branch. The `-Am` shortcut is available but should not be the default approach—stacked PRs are most effective when each branch contains a deliberate, logical set of changes.
8. **Navigate down the stack when you need to change a lower layer.** If you're working on a frontend branch and realize you need API changes, don't hack around it at the current layer. Navigate to the appropriate branch (`gh stack down`, `gh stack checkout`, or `gh stack bottom`), make and commit the changes there, run `gh stack rebase --upstack`, then navigate back up to continue.
9. **Use `gh stack link` for external tool workflows.** When branches are managed by an external tool (jj, Sapling, etc.), use `gh stack link branch-a branch-b`. `link` does not rely on local tracking state and is intended for API-driven PR and stack management. Provide at least two branches/PRs to create or update a stack, or a stack number followed by the new branches/PRs to append them to the top of an existing stack (e.g. `gh stack link 7 branch-c`).
10. **Use `gh stack merge --yes` to merge stacked PRs.** `gh pr merge` does not work with stacked PRs. In a non-interactive terminal `gh stack merge` runs without prompting and merges the entire stack (bottom to top) atomically; pass `--yes` to be explicit. Scope the merge by passing a pull request number (`gh stack merge 42 --yes` merges everything up to and including PR #42) or a stack number (`gh stack merge 7 --yes`, which needs no local checkout). Choose the method with `--squash`, `--rebase`, `--merge`, or `--merge-method <method>`; without one, the last-used method is used. The merge is all-or-nothing — if any PR can't be merged, none are, and the failure reason is reported. Only basic pull request state is checked before merging (open and not a draft); bypassing merge requirements is not supported for stacks. If the base branch uses a merge queue, the stack is added to the queue instead of merging directly: the queue chooses the merge method (any method you pass is ignored with a warning), and the pull requests are added to the queue together but merge as the queue processes them, so they may land in separate groups rather than all at once.
**Never do any of the following — each triggers an interactive prompt or TUI that will hang:**
- ❌ `gh stack view` or `gh stack view --short` — always use `gh stack view --json`
- ❌ `gh stack submit` without `--auto` — always use `gh stack submit --auto`
- ❌ `gh stack init` without branch arguments — always provide branch names
- ❌ `gh stack add` without a branch name — always provide a branch name
- ❌ `gh stack checkout` without an argument — always provide a PR number or branch name
- ❌ `gh stack checkout <pr-number>` when a different local stack already exists on those branches — this triggers an unbypassable conflict resolution prompt; use `gh stack unstack --local` first to remove the local tracking state (this keeps the stack on GitHub intact), then retry the checkout
## Thinking about stack structure
Each branch in a stack should represent a **discrete, logical unit of work** that can be reviewed independently. The changes within a branch should be cohesive—they belong together and make sense as a single PR.
### Dependency chain
Stacked branches form a dependency chain: each branch builds on the one below it. This means **foundational changes must go in lower (earlier) branches**, and code that depends on them goes in higher (later) branches.
**Plan your layers before writing code.** For example, a full-stack feature might be structured like this (use branch names relevant to your actual task, not these generic ones):
```
main (trunk)
└── data-models ← shared types, database schema
└── api-endpoints ← API routes that use the models
└── frontend-ui ← UI components that call the APIs
└── integration ← tests that exercise the full stack
```
This is illustrative — choose branch names and layer boundaries that reflect the specific work you're doing. The key principle is: if code in one layer depends on code in another, the dependency must be in the same branch or a lower one.
### Branch naming
Choose a clear, descriptive branch name for each layer that reflects the concern it contains (e.g., `auth`, `api-routes`, `frontend`). Branch names are used exactly as you provide them to `init` and `add` — nothing is prepended or transformed. Slashes are allowed and are treated as part of the name (e.g., `gh stack add refactor/foo` creates a branch named `refactor/foo`).
### Staging changes deliberately
The main reason to use `git add` and `git commit` directly is to control **which changes go into which branch**. When you have multiple files in your working tree, you can stage a subset for the current branch, commit them, then create a new branch and stage the rest there:
```bash
# You're on data-models with several new files in your working tree.
# Stage only the model files for this branch:
git add internal/models/user.go internal/models/session.go
git commit -m "Add user and session models"
git add db/migrations/001_create_users.sql
git commit -m "Add user table migration"
# Now create a new branch for the API layer and stage the API files there:
gh stack add api-routes # created & switched to the api-routes branch
git add internal/api/routes.go internal/api/handlers.go
git commit -m "Add user API routes"
```
This keeps each branch focused on one concern. Multiple commits per branch are fine — the key is that all commits in a branch relate to the same logical concern, and changes that belong to a different concern go in a different branch.
### When to create a new branch
Create a new branch (`gh stack add`) when you're starting a **different concern** that depends on what you've built so far. Signs it's time for a new branch:
- You're switching from backend to frontend work
- You're moving from core logic to tests or documentation
- The next set of changes has a different reviewer audience
- The current branch's PR is already large enough to review
### One stack, one story
Think of a stack from the reviewer's perspective: the stack of PRs should **tell a cohesive story** about a feature or project. A reviewer should be able to read the PRs in sequence and understand the progression of changes, with each PR being a small, logical piece of the whole.
**When to use a single stack:** All the branches are part of the same feature, project, or closely related effort. Even if the work spans multiple concerns (models, API, frontend), they're all building toward the same goal.
**When to create a separate stack:** The work is unrelated to your current stack — a different feature, a bug fix in an unrelated area, or an independent refactor. Don't mix unrelated work into a single stack just because you happen to be working on both. Start a new stack with `gh stack init` or switch to an existing stack with `gh stack checkout` for each distinct effort.
Small, incidental fixes (e.g., fixing a typo you noticed) can go in the current stack if they're trivial. But if a change grows into its own project, it deserves its own stack.
## Quick reference
| Task | Command |
|------|---------|
| Create a stack | `gh stack init auth` |
| Create a stack of multiple branches | `gh stack init auth api frontend` |
| Adopt existing branches | `gh stack init existing-branch-a existing-branch-b` |
| Set custom trunk | `gh stack init --base develop branch-a` |
| Add a branch to stack | `gh stack add api-routes` |
| Add branch + stage all + commit | `gh stack add -Am "message" api-routes` |
| Push branches to remote | `gh stack push` |
| Push to specific remote | `gh stack push --remote origin` |
| Push branches + create draft PRs | `gh stack submit --auto` |
| Create PRs as ready for review | `gh stack submit --auto --open` |
| Sync (fetch, rebase, push) | `gh stack sync` |
| Sync with specific remote | `gh stack sync --remote origin` |
| Sync and prune merged branches | `gh stack sync --prune` |
| Rebase entire stack | `gh stack rebase` |
| Rebase upstack only | `gh stack rebase --upstack` |
| Rebase without trunk | `gh stack rebase --no-trunk` |
| Continue after conflict | `gh stack rebase --continue` |
| Abort rebase | `gh stack rebase --abort` |
| View stack details (JSON) | `gh stack view --json` |
| Switch branches up/down in stack | `gh stack up [n]` / `gh stack down [n]` |
| Switch to top/bottom branch | `gh stack top` / `gh stack bottom` |
| Check out by stack number | `gh stack checkout 7` |
| Check out by PR | `gh stack checkout 42` |
| Check out by branch (local only) | `gh stack checkout feature-auth` |
| Tear down the current stack to restructure it | `gh stack unstack` |
| Tear down a specific stack by number | `gh stack unstack 7` |
| Merge the whole current stack | `gh stack merge --yes` |
| Merge a stack by number | `gh stack merge 7 --yes` |
| Merge up to a specific PR | `gh stack merge 42 --yes` |
| Merge with a specific method | `gh stack merge --yes --squash` |
---
## Workflows
### End-to-end: create a stack from scratch
```bash
# 1. Initialize a stack with the first branch
gh stack init auth
# → creates auth and checks it out
# 2. Write code for the first layer (auth)
cat > auth.go << 'EOF'
package auth
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// verify token
next.ServeHTTP(w, r)
})
}
EOF
# 3. Stage and commit using standard git commands
git add auth.go
git commit -m "Add auth middleware"
# You can make multiple commits on the same branch
cat > auth_test.go << 'EOF'
package auth
func TestMiddleware(t *testing.T) {
// test auth middleware
}
EOF
git add auth_test.go
git commit -m "Add auth middleware tests"
# 4. When you're ready for a new concern, add the next branch
gh stack add api-routes
# → creates api-routes
# 5. Write code for the API layer
cat > api.go << 'EOF'
package api
func RegisterRoutes(mux *http.ServeMux) {
mux.HandleFunc("/users", handleUsers)
}
EOF
git add api.go
git commit -m "Add API routes"
# 6. Add a third layer for frontend
gh stack add frontend
# → creates frontend
cat > frontend.go << 'EOF'
package frontend
func RenderDashboard(w http.ResponseWriter) {
// calls the API endpoints from the layer below
}
EOF
git add frontend.go
git commit -m "Add frontend dashboard"
# ── Stack complete: auth → api-routes → frontend ──
# 7. Push everything and create PRs (drafts by default)
gh stack submit --auto
# 8. Verify the stack
gh stack view --json
```
> **Shortcut:** If you prefer a faster flow, `gh stack add -Am "message" branch-name` combines staging, committing, and branch creation into one command. This is useful for single-commit layers but bypasses deliberate staging.
### Making mid-stack changes
This is a critical workflow for agents. When you're working on a higher layer and realize you need to change something in a lower layer (e.g., you're building frontend components but need to add an API endpoint), **navigate down to the correct branch, make the change there, and rebase**.
```bash
# You're on frontend but need to add an API endpoint
# 1. Navigate to the API branch
gh stack down
# or: gh stack checkout api-routes
# 2. Make the change where it belongs
cat > users_api.go << 'EOF'
package api
func handleGetUser(w http.ResponseWriter, r *http.Request) {
// new endpoint the frontend needs
}
EOF
git add users_api.go
git commit -m "Add get-user endpoint"
# 3. Rebase everything above to pick up the change
gh stack rebase --upstack
# 4. Navigate back to where you were working
gh stack top
# or: gh stack checkout frontend
# 5. Continue working — the API changes are now available
```
**Why this matters:** If you make API changes on the frontend branch, those changes will end up in the wrong PR. The API PR won't include them, and the frontend PR will have unrelated API diffs mixed in. Always put changes in the branch where they logically belong.
### Modify a mid-stack branch and sync
When you need to revisit a branch after the initial creation (e.g., responding to review feedback):
```bash
# 1. Navigate to the branch that needs changes
gh stack bottom
# or: gh stack checkout auth
# or: gh stack checkout 42 (by PR number)
# 2. Make changes and commit
cat > auth.go << 'EOF'
package auth
// updated implementation
EOF
git add auth.go
git commit -m "Fix auth token validation"
# 3. Rebase everything above this branch
gh stack rebase --upstack
# 4. Push the updated stack
gh stack push
```
### Routine sync after merges
```bash
# Single command: fetch, rebase, push, sync PR and stack state
gh stack sync
# Sync and automatically clean up local branches for merged PRs
gh stack sync --prune
```
> **Note for agents:** In non-interactive environments, the prune prompt is not shown. Use `--prune` explicitly to delete local branches for merged PRs.
> **Note for agents:** `sync` also mirrors the stack on GitHub locally. If PRs were added to the stack on github.com, their branches are pulled down and appended to the local stack automatically. If the local and remote stacks have **diverged** (you changed the local stack while the remote stack changed differently), sync can only prompt to resolve it in an interactive terminal — in non-interactive environments it aborts the sync (nothing is pushed or updated) and exits successfully with `ℹ Sync aborted`. Resolve a divergence by unstacking and recreating the stack.
### Squash-merge recovery
When a PR is squash-merged on GitHub, the original branch's commits no longer exist in the trunk history. `gh stack` detects this automatically and uses `git rebase --onto` to correctly replay remaining commits.
```bash
# After PR #1 (auth) is squash-merged on GitHub:
gh stack sync
# → fetches latest, detects the merge, fast-forwards trunk
# → rebases api-routes onto updated trunk (skips merged branch)
# → rebases frontend onto api-routes
# → pushes updated branches
# → reports: "Merged: #1"
# Verify the result
gh stack view --json
# → auth shows "isMerged": true, "state": "MERGED"
# → api-routes and frontend show updated heads
```
If `sync` hits a conflict during this process, it restores all branches to their pre-rebase state and exits with code 3. See [Handle rebase conflicts](#handle-rebase-conflicts-agent-workflow) for the resolution workflow.
### Handle rebase conflicts (agent workflow)
```bash
# 1. Start the rebase
gh stack rebase
# 2. If exit code 3 (conflict):
# - Parse stderr for conflicted file paths
# - Read those files to find <<<<<<< / ======= / >>>>>>> markers
# - Edit files to resolve conflicts
# - Stage resolved files:
git add path/to/resolved-file.go
# 3. Continue the rebase
gh stack rebase --continue
# 4. If another conflict occurs, repeat steps 2-3
# 5. If unable to resolve, abort to restore everything
gh stack rebase --abort
```
### Parsing `--json` output
```bash
# Get stack state as JSON
output=$(gh stack view --json)
# Check if any branch needs a rebase, and rebase if so
needs_rebase=$(echo "$output" | jq '[.branches[] | select(.needsRebase == true)] | length')
if [ "$needs_rebase" -gt 0 ]; then
echo "Branches need rebase, rebasing stack..."
gh stack rebase
fi
# Get all open PR URLs
echo "$output" | jq -r '.branches[] | select(.pr.state == "OPEN") | .pr.url'
# Find merged branches
echo "$output" | jq -r '.branches[] | select(.isMerged == true) | .name'
# Get the current branch
echo "$output" | jq -r '.currentBranch'
# Check if the stack is fully merged (all branches merged)
echo "$output" | jq '[.branches[] | .isMerged] | all'
```
### Restructure a stack (remove a branch, reorder, or rename)
Use `unstack` to tear down the stack, make structural changes, then re-init:
```bash
# 1. Remove the local tracking and the GitHub stack grouping (PRs are NOT deleted)
gh stack unstack
# 2. Make structural changes — e.g. delete a branch, reorder, rename
git branch -m old-branch-1 new-branch-1
# 3. Re-create the stack with the new structure
gh stack init --base main new-branch-1 new-branch-2 new-branch-3
```
---
## Commands
### Initialize a stack — `gh stack init`
Creates a new stack. **Always provide at least one branch name as a positional argument** — running without branch arguments triggers interactive prompts that agents cannot use.
```
gh stack init [flags] <branches...>
```
```bash
# Create a stack with a new branch
gh stack init auth
# → creates auth and checks it out
# Create a stack with new branches
gh stack init branch-a branch-b branch-c
# Use a different trunk branch
gh stack init --base develop branch-a branch-b
# Adopt existing branches into a stack (handled automatically if the branches exist)
gh stack init branch-a branch-b branch-c
```
| Flag | Description |
|------|-------------|
| `-b, --base <branch>` | Trunk branch (defaults to the repo's default branch) |
**Behavior:**
- Branch names are created exactly as given (slashes are allowed and kept as-is)
- Creates any branches that don't already exist (branching from the trunk branch)
- Existing branches are adopted automatically; missing branches are created from the trunk
- Checks out the last branch in the list
- Enables `git rerere` so conflict resolutions are remembered across rebases. On first run in a repo, this may trigger a confirmation prompt — pre-configure with `git config rerere.enabled true` to avoid it
---
### Add a branch — `gh stack add`
Add a new branch on top of the current stack. Must be run while on the topmost branch (or the trunk if the stack has no branches yet). **Always provide a branch name** — running without one triggers an interactive prompt.
```
gh stack add [flags] <branch>
```
**Recommended workflow — create the branch, then use standard git:**
```bash
# Create a new branch and switch to it
gh stack add api-routes
# Write code, stage deliberately, and commit
git add internal/api/routes.go internal/api/handlers.go
git commit -m "Add user API routes"
# Make more commits on the same branch as needed
git add internal/api/middleware.go
git commit -m "Add rate limiting middleware"
```
**Shortcut — stage, commit, and branch in one command:**
```bash
# Create a new branch, stage all changes, and commit
gh stack add -Am "Add API routes" api-routes
# Create a new branch, stage tracked files only, and commit
gh stack add -um "Fix auth bug" auth-fix
```
| Flag | Description |
|------|-------------|
| `-m, --message <string>` | Create a commit with this message |
| `-A, --all` | Stage all changes including untracked files (requires `-m`) |
| `-u, --update` | Stage tracked files only (requires `-m`) |
**Behavior notes:**
- `-A` and `-u` are mutually exclusive.
- When the current branch has no commits (e.g., right after `init`), `add -Am` commits directly on the current branch instead of creating a new one.
- **Branch names are used verbatim.** `gh stack add refactor/foo` creates a branch named `refactor/foo` — names are never prefixed or transformed. When `-m` is given without a branch name, the name is auto-generated from the commit message in date+slug format (e.g., `03-24-add_api_routes`).
- If called from a branch that is not the topmost in the stack, exits with code 5: `"can only add branches on top of the stack"`. Use `gh stack top` to switch first.
- **Uncommitted changes:** When using `gh stack add branch-name` without `-Am`, any uncommitted changes (staged or unstaged) in your working tree carry over to the new branch. This is standard git behavior — the working tree is not touched. Commit or stash changes on the current branch before running `add` if you want a clean starting point on the new branch.
---
### Push branches to remote — `gh stack push`
Push active stack branches to the remote.
```
gh stack push [flags]
```
```bash
# Push all branches
gh stack push
# Push to specific remote
gh stack push --remote upstream
```
| Flag | Description |
|------|-------------|
| `--remote <name>` | Remote to push to (use if multiple remotes exist) |
**Behavior:**
- Pushes all active (non-merged, non-queued) branches in one non-atomic multi-ref push with explicit per-branch `--force-with-lease` checks
- Some branches may update if another is rejected; fix the rejected branch and rerun the command
- Does **not** create or update pull requests — use `gh stack submit` for that
**Output (stderr):**
- `Pushed N branches` summary
---
### Submit branches and create PRs — `gh stack submit`
Push all stack branches and create PRs on GitHub. **Always pass `--auto`** — without it, `submit` prompts for a PR title for each new branch.
```bash
# Submit and auto-title new PRs (required for non-interactive use)
gh stack submit --auto
# Submit and create PRs as ready for review (not drafts)
gh stack submit --auto --open
```
| Flag | Description |
|------|-------------|
| `--auto` | Auto-generate PR titles without prompting (**required** for non-interactive use) |
| `--open` | Mark new and existing PRs as ready for review |
| `--remote <name>` | Remote to push to (use if multiple remotes exist) |
**Behavior:**
- Pushes each active (non-merged, non-queued) branch sequentially with explicit per-branch `--force-with-lease` checks; the overall submit is not atomic
- If a later branch push is rejected, earlier branch pushes and PR updates remain; fix the rejection and rerun the same command
- Creates a new PR for each branch that doesn't have one (base set to the first non-merged ancestor branch)
- After creating PRs, links them together as a **Stack** on GitHub (requires the repository to have stacks enabled)
- If every PR in the stack has already been merged, the stack is complete and can't be extended. `submit` automatically forks your unmerged branches into a **new** stack rooted at the trunk and creates it on GitHub, leaving the merged stack untouched.
- If stacks are not available (exit code 9), the repository does not have stacked PRs enabled. In interactive mode, `submit` offers to create regular (unstacked) PRs instead. In non-interactive mode, it exits with code 9.
- Syncs PR metadata for branches that already have PRs
**PR title auto-generation (`--auto`):**
- Single commit on branch → uses the commit subject as the PR title, commit body as PR body
- Multiple commits on branch → humanizes the branch name (hyphens/underscores → spaces) as the title
**Output (stderr):**
- `Created PR #N for <branch>` for each newly created PR
- `PR #N for <branch> is up to date` for existing PRs
- `Pushed and synced N branches` summary
---
### Link branches as a stack (no local tracking) — `gh stack link`
Link PRs into a stack on GitHub without creating any local tracking state. This is the recommended approach if you are managing stacked branches with other tools (jj, Sapling, git-town) and want to simply create GitHub Stacked PRs via an API.
```
gh stack link [flags] <stack-number | branch-or-pr> <branch-or-pr> [...]
```
```bash
# Link branches into a stack (pushes, creates PRs, creates stack)
gh stack link branch-a branch-b branch-c
# Use a different base branch and mark PRs as ready for review
gh stack link --base develop --open branch-a branch-b branch-c
# Link existing PRs by number
gh stack link 10 20 30
# Add branches to an existing stack of PRs
gh stack link 42 43 feature-auth feature-ui
# Append to the top of an existing stack by its stack number
# (7 is a stack number; only the new PRs/branches are listed)
gh stack link 7 48 feature-auth
```
When the first argument is a stack number, the remaining arguments are appended to the top of that stack, so you don't have to re-list its current PRs. Arguments already in the stack are skipped; arguments in a different stack are rejected. A numeric first argument is treated as a stack only when it matches an existing stack — otherwise it is a PR or branch.
| Flag | Description |
|------|---------|
| `--base <branch>` | Base branch for the bottom of the stack (defaults to the repository's default branch) |
| `--open` | Mark new and existing PRs as ready for review |
| `--remote <name>` | Remote to push to (use if multiple remotes exist) |
**Behavior:**
- Arguments are provided in stack order (bottom to top)
- Each argument can be a branch name or a PR number. Numeric arguments are tried as PR numbers first; if no PR with that number exists, the argument is treated as a branch name
- Branch arguments are pushed to the remote automatically (non-force, atomic)
- For branches without open PRs, new PRs are created with auto-generated titles and the correct base branch chaining (first branch uses `--base`, subsequent branches use the previous branch)
- Existing PRs whose base branch doesn't match the expected chain are corrected automatically
- If the PRs are not yet in a stack, a new stack is created. If some PRs are already in a stack, the stack is updated (additive only — existing PRs are never removed)
- Does **not** create or modify any local state
**Output (stderr):**
- `Pushing N branches to <remote>...`
- `Found PR #N for branch <name>` for branches with existing PRs
- `Created PR #N for <branch> (base: <base>)` for newly created PRs
- `Updated base branch for PR #N to <base>` when base branches are corrected
- `Created stack with N PRs` or `Updated stack to N PRs`
---
### Sync the stack — `gh stack sync`
Fetch, rebase, push, and sync PR state in a single command. This is the recommended command for routine synchronization.
```
gh stack sync [flags]
```
| Flag | Description |
|------|-------------|
| `--remote <name>` | Remote to fetch from and push to (use if multiple remotes exist) |
| `--prune` | Delete local branches for merged PRs |
**What it does (in order):**
1. **Fetch** latest changes from the remote
2. **Reconcile the remote stack** — mirror the GitHub stack locally. If PRs were added to the stack on GitHub, pull their branches down and append them to the local stack. If the local and remote stacks have diverged, aborts the sync in a non-interactive terminal. In an interactive terminal, offers prompts to resolve any divergence (replace local stack with remote version, delete stack on GitHub so it can be recreated, or cancel).
3. **Fast-forward trunk** to match remote (skips if already up to date, warns if diverged)
4. **Cascade rebase** all stack branches onto their updated parents (only if trunk moved). Handles merged PRs automatically. If a conflict is detected, **all branches are restored** to their pre-rebase state and the command exits with code 3 — see [Handle rebase conflicts](#handle-rebase-conflicts-agent-workflow) for the resolution workflow
5. **Push** all active branches atomically
6. **Sync PR state** from GitHub and report the status of each PR
7. **Sync the stack object** — link the open PRs into a stack on GitHub. If the PRs are not yet in a stack, a new stack is created; if some PRs are already in a stack, it is updated (additive only). This only happens when two or more PRs exist. Sync **never opens PRs** — use `gh stack submit` for that
8. **Prune** — in interactive terminals, prompts to delete local branches for merged PRs. Use `--prune` to skip the prompt. In non-interactive environments, pruning only happens when `--prune` is passed explicitly
**Output (stderr):**
- `✓ Fetched latest changes from origin`
- `Pulling N new branches from the remote stack ...` then `✓ Pulled N new branches into the stack from the remote` (when the remote stack is ahead)
- `⚠ Your local stack has diverged from the stack on GitHub` (with `Local:` / `Remote:` chains) when the stacks have diverged
- `ℹ Sync aborted — no changes were made` when a sync is cancelled
- `✓ Trunk main fast-forwarded to <sha>` or `✓ Trunk main is already up to date`
- `✓ Rebased <branch> onto <base>` per branch (if base moved)
- `✓ Pushed N branches`
- `✓ PR #N (<branch>) — Open` per branch
- `Merged: #N, #M` for merged branches
- `✓ Stack created on GitHub with N PRs` / `✓ Stack updated on GitHub with N PRs` / `✓ Linked to the existing stack on GitHub` (when two or more PRs exist)
- `✓ Pruned <branch> (merged)` per pruned branch (when pruning)
- `✓ Stack synced` when the stack object on GitHub was created/updated to match local, or `✓ Branches synced` when only the branches were synced (fewer than two PRs or stacked PRs unavailable)
---
### Rebase the stack — `gh stack rebase`
Pull from remote and cascade-rebase stack branches. Use this when `sync` reports a conflict or when you need finer control (e.g., rebase only part of the stack).
```
gh stack rebase [flags] [branch]
```
```bash
# Rebase the entire stack
gh stack rebase
# Rebase only branches from trunk to current branch
gh stack rebase --downstack
# Rebase only branches from current branch to top
gh stack rebase --upstack
# Rebase stack branches without pulling from or rebasing with trunk
gh stack rebase --no-trunk
# After resolving a conflict: stage files with `git add`, then:
gh stack rebase --continue
# Abort and restore all branches to pre-rebase state
gh stack rebase --abort
```
| Flag | Description |
|------|-------------|
| `--downstack` | Only rebase branches from trunk to the current branch |
| `--upstack` | Only rebase branches from the current branch to the top |
| `--no-trunk` | Skip trunk — only rebase stack branches onto each other (no fetch, no trunk rebase) |
| `--continue` | Continue after resolving conflicts |
| `--abort` | Abort and restore all branches |
| `--remote <name>` | Remote to fetch from (use if multiple remotes exist) |
| Argument | Description |
|----------|-------------|
| `[branch]` | Target branch (defaults to the current branch) |
**Conflict handling:** See [Handle rebase conflicts](#handle-rebase-conflicts-agent-workflow) in the Workflows section for the full resolution workflow.
**Merged PR detection:** If a branch's PR was merged on GitHub, the rebase automatically handles this using `--onto` mode and correctly replays commits on top of the merge target.
**Rerere (conflict memory):** `git rerere` is enabled by `init` so previously resolved conflicts are auto-resolved in future rebases.
**No-trunk mode:** Use `--no-trunk` to skip fetching from the remote and rebasing with the trunk branch. Only inter-branch rebases are performed (branch 2 onto branch 1, branch 3 onto branch 2, etc.). Useful when you only need to align stack branches with each other without pulling upstream changes.
---
### View the stack — `gh stack view`
Display the current stack's branches, PR status, and recent commits. **Always pass `--json`** — without it, this command launches an interactive TUI that agents cannot operate.
```bash
# Always use --json
gh stack view --json
```
| Flag | Description |
|------|-------------|
| `--json` | Output stack data as JSON to stdout (**required** for non-interactive use) |
**`--json` output format:**
```json
{
"trunk": "main",
"currentBranch": "api-routes",
"branches": [
{
"name": "auth",
"head": "abc1234...",
"base": "def5678...",
"isCurrent": false,
"isMerged": true,
"isQueued": false,
"needsRebase": false,
"pr": {
"number": 42,
"url": "https://github.com/owner/repo/pull/42",
"state": "MERGED"
}
},
{
"name": "api-routes",
"head": "789abcd...",
"base": "abc1234...",
"isCurrent": true,
"isMerged": false,
"isQueued": false,
"needsRebase": false,
"pr": {
"number": 43,
"url": "https://github.com/owner/repo/pull/43",
"state": "OPEN"
}
}
]
}
```
Fields per branch:
- `name` — branch name
- `head` — current HEAD SHA
- `base` — parent branch's HEAD SHA at last sync
- `isCurrent` — whether this is the checked-out branch
- `isMerged` — whether the PR has been merged
- `isQueued` — whether the PR is queued for merge (in a merge queue)
- `needsRebase` — whether the base branch is not an ancestor (non-linear history)
- `pr` — PR metadata (omitted if no PR exists). `state` is `"OPEN"`, `"MERGED"`, or `"QUEUED"`.
---
### Navigate the stack
Move between branches without remembering branch names. These commands are fully non-interactive.
```bash
gh stack up # Move up one branch (further from trunk)
gh stack up 3 # Move up three branches
gh stack down # Move down one branch (closer to trunk)
gh stack down 2 # Move down two branches
gh stack top # Jump to the top of the stack (furthest from trunk)
gh stack bottom # Jump to the bottom (first non-merged branch above trunk)
gh stack trunk # Jump to the trunk branch (e.g. main)
```
Navigation clamps to stack bounds. Merged branches are skipped when navigating from active branches.
---
### Check out a stack — `gh stack checkout`
Check out a stack by stack number, pull request number, PR URL, or branch name. **Always provide an argument** — running `gh stack checkout` without arguments triggers an interactive selection menu.
```
gh stack checkout <stack-number | pr-number | pr-url | branch>
```
```bash
# By stack number (the identifier shown in the GitHub stack UI)
gh stack checkout 7
# By PR number (pulls from GitHub)
gh stack checkout 42
# By PR URL
gh stack checkout https://github.com/owner/repo/pull/42
# By branch name (local only)
gh stack checkout feature-auth
```
A bare number is resolved as a **stack number first** (the identifier shown in the GitHub stack UI); if no stack has that number it is tried as a PR number, then a branch name. When a stack or PR number (or PR URL) is provided, the command fetches the stack on GitHub, pulls the branches, and sets up the stack locally. If the stack already exists locally and matches, it switches to the branch.
> **⚠️ Agent warning:** If the local and remote stacks have different branch compositions, this command triggers an interactive conflict-resolution prompt that cannot be bypassed with a flag. To avoid this: run `gh stack unstack --local` first to remove the conflicting local tracking state (this keeps the stack on GitHub intact), then retry `gh stack checkout <pr-number>`.
When a branch name is provided, the command resolves it against locally tracked stacks only. This is always safe for non-interactive use.
---
### Remove a stack — `gh stack unstack`
Tear down a stack so you can restructure it — remove a branch, reorder branches, rename branches, or make other large changes. After unstacking, use `gh stack init` to re-create the stack with the desired structure.
Unstacking only removes the stack grouping (on GitHub and/or locally); it never deletes the underlying pull requests or branches.
With no argument, the command targets the active stack — the one containing the currently checked out branch — unstacking it on GitHub and removing local tracking.
Provide a stack number to unstack a specific stack on GitHub. This works from anywhere in the repository, whether or not the stack is checked out locally — the number is unstacked directly through the GitHub API (like `gh stack link`, no local tracking required). If the stack is also tracked locally, its local tracking is removed as well.
```
gh stack unstack [<stack-number>] [flags]
```
```bash
# Tear down the current stack — removes local tracking and the GitHub grouping (PRs are NOT deleted), then rebuild
gh stack unstack
gh stack init --base main branch-2 branch-1 branch-3 # reordered
# Unstack a specific stack by its number, from anywhere in the repo
gh stack unstack 7
# Only remove local tracking (keep the stack on GitHub)
gh stack unstack --local
```
| Flag | Description |
|------|-------------|
| `--local` | Only remove the stack locally (keep it on GitHub); never contacts GitHub |
> **Note for agents:** `gh stack unstack <number>` is a remote-first API wrapper — it unstacks on GitHub by number from anywhere in the repo, tracked locally or not, and is safe for non-interactive use. `--local` never contacts GitHub; combining `--local` with a number that isn't tracked locally is an error. An unknown stack number returns a "not found on GitHub" error (exit code 2).
---
## Output conventions
- **Status messages** go to **stderr** with emoji prefixes: `✓` (success), `✗` (error), `⚠` (warning), `ℹ` (info).
- **Data output** (e.g., `view --json`) goes to **stdout**.
- When piping output, use `2>/dev/null` to suppress status messages if only data output is needed.
## Exit codes and error recovery
| Code | Meaning | Agent action |
|------|---------|-------------|
| 0 | Success | Proceed normally |
| 1 | Generic error | Read stderr for details; may indicate commit/push failure |
| 2 | Not in a stack | Run `gh stack init` to create a stack first |
| 3 | Rebase conflict | Parse stderr for conflicted file paths, resolve conflicts, run `gh stack rebase --continue` |
| 4 | GitHub API failure | Check `gh auth status`, retry the command |
| 5 | Invalid arguments | Fix the command invocation (check flags and arguments) |
| 6 | Disambiguation required | A branch belongs to multiple stacks. Run `gh stack checkout <specific-branch>` to switch to a non-shared branch first |
| 7 | Rebase already in progress | Run `gh stack rebase --continue` (after resolving conflicts) or `gh stack rebase --abort` to start over |
| 8 | Stack is locked | Another `gh stack` process is writing the stack file. Wait and retry — the lock times out after 5 seconds |
| 9 | Stacked PRs unavailable | The repository does not have stacked PRs enabled. Tell the user that stacks must be enabled on the repository first |
| 10 | Modify recovery required | A `gh stack modify` session was interrupted. This skill does not use `modify`, so agents should not produce this; if the repo is left in this state, run `gh stack modify --abort` to restore the pre-modify state |
## Known limitations
1. **Stacks are strictly linear.** Branching stacks (multiple children on a single parent) are not supported. Each branch has exactly one parent and at most one child. If you need parallel workstreams, use separate stacks.
2. **Stack disambiguation cannot be bypassed.** If the current branch is the trunk of multiple stacks, commands error with code 6. Check out a non-shared branch first.
3. **Multiple remotes require `--remote` or config.** If more than one remote is configured, set `remote.pushDefault` in git config, or pass `--remote <name>` to the commands that accept it (`push`, `submit`, `sync`, `rebase`, `link`). `checkout`, `modify`, and `trunk` have no `--remote` flag and rely on `remote.pushDefault`.
4. **Remote stack checkout requires a stack or PR number.** `checkout` with a branch name only works with locally tracked stacks. Use a stack number or PR number (e.g. `gh stack checkout 7` or `gh stack checkout 123`) to pull a stack from GitHub.
5. **PR title and body are auto-generated.** There is no flag to set a custom PR title or body during `submit`. The title and body are generated from commit messages plus a footer. Use `gh pr edit` to modify PR title and body after creation.

View File

@ -13,11 +13,11 @@ the same commands from the repo via `pnpm nathan`.
## Prerequisites
- **No tunnel setup needed.** Nathan replies asynchronously through a short-lived
Cloudflare Quick Tunnel that the script opens for you (`npx cloudflared`). A
`deploy` to a **new** `test-<name>` also polls that instance URL directly, so it
still reports success even if the tunnel drops. (Redeploying a name that's
*already up* skips the poll — it can't tell the old instance from the new — so
it falls back to the tunnel; prefer a fresh name when you need the reliable report.)
public tunnel that the script opens for you (`npx localtunnel`). A `deploy` to a
**new** `test-<name>` also polls that instance URL directly, so it still reports
success even if the tunnel drops. (Redeploying a name that's *already up* skips
the poll — it can't tell the old instance from the new — so it falls back to the
tunnel; prefer a fresh name when you need the reliable report.)
- **A token in `~/.n8n/dev/nathan-token`.** If a command reports no token, **ask the
user for one** — point them at the form
(`https://internal.users.n8n.cloud/form/d6d34a2f-4899-4ee8-afc8-f8c41a8a243d`),

View File

@ -21,7 +21,8 @@ check the code, not license to drop a team default.
## Non-negotiable rules
- New endpoints are `@PublicApiController` classes under `v1/controllers/`, one
`*.public.controller.ts` per feature.
`*.public.controller.ts` per feature. A controller is a class — never
`export =` (the legacy tuple style; `require-public-api-controller` flags it).
- Public API and internal REST are separate HTTP surfaces. A public controller
never calls an internal controller/endpoint; both reuse the same service.
- Controllers and handlers delegate to a service — never import a repository or
@ -30,6 +31,7 @@ check the code, not license to drop a team default.
`@ApiResponse(Dto)`.
- Register each controller via a side-effect import in `v1/controllers/index.ts`
(`public-api-controllers.test.ts` fails otherwise).
- Don't add business logic to legacy `express-openapi-validator` (EOV) handlers.
- Migrating a legacy endpoint must not change its public contract.
These are `n8n-local-rules` ESLint rules (see `packages/cli/eslint.config.mjs`)
@ -39,14 +41,8 @@ add to it.
## Team defaults
- Write code that acts as its own documentation. The schema, the decorator, and the test should make the rule clear on their own without a comment.
- List endpoints: cursor-based pagination (internal API uses both cursor- and
page-based — don't copy an internal endpoint's model).
- Pagination args are always `offset` and `limit` — on service methods, handler
calls, and repository methods you add. Never `skip`/`take` (TypeORM names).
Translate to `skip`/`take` only inside a repository, at the TypeORM `find`
call. The public query string is still `cursor` + `limit`; `offset` is the
decoded cursor field passed into the service, never a client-facing param.
- Updates: full-object `PUT`, not `PATCH`. A successful `GET` body should be
acceptable as a `PUT` body for the same resource (round-trip), aside from
server-managed/immutable fields.
@ -80,18 +76,13 @@ Open these — they are the source of truth, not this skill:
cursor) or `workflows.public.controller.ts` (`@Param` + `@ProjectScope`), and
`index.ts` for the barrel.
- Decorators in `packages/@n8n/decorators/src/controller/`:
`public-api-controller.ts`, `api-key-scope.ts`, `api-response.ts`,
`api-error-response.ts`, `api-summary.ts`, `api-description.ts`, `api-tags.ts`,
`route.ts`, `scoped.ts`, `args.ts`, `licensed.ts`.
- The OpenAPI generator (reads the decorators above, no hand-written YAML
needed for a controller route): `v1/openapi-gen/generate.ts`,
`v1/openapi-gen/decorator-routes.ts`.
`public-api-controller.ts`, `api-key-scope.ts`, `api-response.ts`, `route.ts`,
`scoped.ts`, `args.ts`, `licensed.ts`.
- Pagination helpers: `v1/shared/services/pagination.service.ts`
(`decodeCursor`, `encodeNextCursor`).
- DTOs: `packages/@n8n/api-types/src/dto/`.
- Gating tests: `v1/__tests__/public-api-controllers.test.ts`,
`v1/__tests__/scope-parity.test.ts`,
`v1/openapi-gen/__tests__/generated-spec-drift.test.ts`.
`v1/__tests__/scope-parity.test.ts`.
- The internal controller for this resource and its neighboring functional tests.
## Declaring a controller
@ -107,25 +98,17 @@ model; reuse only what applies. Decorators, all from `@n8n/decorators`:
| `@Get/@Post/@Put/@Patch/@Delete('/path')` | Route method. |
| `@ApiKeyScope('res:action')` | API-key grant check. |
| `@ProjectScope/@GlobalScope('res:action')` | User RBAC check. |
| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO or a binary body — throws. For a binary response body (returned as `{ body, headers? }`), use `@ApiResponse(status, { mediaType, description?, headers? })` — see [Binary response bodies](reference.md#binary-response-bodies). |
| `@ApiErrorResponse(status)` | Declares an additional documented non-2xx status (e.g. `404`, `409`). Stack multiple for more than one. `400`/`401`/`403` are added automatically (body/query present, always, and `@ApiKeyScope` present, respectively) — don't declare those yourself. |
| `@ApiSummary(text)` / `@ApiDescription(text)` / `@ApiTags([...])` | OpenAPI summary/description/tags. `@ApiTags` sorts alphabetically regardless of the order you pass. All optional but expected on every real route. |
| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. `@Body` is JSON by default; `@Body({ mediaType: 'multipart/form-data', uploadLimits })` takes a `multipart/form-data` body instead — see [Request body media types](reference.md#request-body-media-types). |
| `@Licensed('feat')` | Gates the route on a single `BooleanLicenseFeature`; `PublicApiControllerRegistry` runs its own license middleware (after auth/`@ApiKeyScope`/`@ProjectScope`|`@GlobalScope`, before the handler) and 403s unlicensed requests. Only takes one feature — if the gate is an any-of/all-of combination (e.g. `LicenseState.isProvisioningLicensed()`, which is `feat:saml` OR `feat:oidc`), `@Licensed` can't express that; check manually in the handler instead, same as the internal `provisioning.controller.ee.ts`/`role-mapping-rule.controller.ee.ts` do today (throwing `ForbiddenError` on failure). |
| `@ApiResponse(Dto)` | Output DTO; registry `.parse()`s + strips the return value. |
| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. |
| `@Licensed('feat')` | Gate an EE-only endpoint. |
## Authorization (easy to get wrong)
- `@ApiKeyScope` (what the API key is granted) and `@ProjectScope`/`@GlobalScope`
(what the user may do) are independent. Use both when the model needs both.
- Name every path param `{resource}Id` (e.g. `workflowId`, `credentialId`,
`projectId`, …) — never a generic `:id` / `{id}`. This is the Public API's
naming convention: it keeps the API self-documenting and gives typed SDK
codegen a real argument name instead of `id`. `@ProjectScope` also reads
`req.params` as-is and does not remap `id` — it resolves authorization by
exact key name (`workflowId`, `credentialId`, `projectId`, `dataTableId`,
…), so a generic `id` on a `@ProjectScope` route often fails outright; a
`@GlobalScope` or unscoped route won't fail the same way, but still follow
the convention.
- `@ProjectScope` reads `req.params` as-is and does not remap `id` — name the path
param what the resolver expects (`workflowId`, `credentialId`, `projectId`,
`dataTableId`, …). A generic `id` often fails.
- `@ApiKeyScope` takes a string, `{ anyOf: [...] }`, or `{ allOf: [...] }` — never
a bare array. The scope must exist in the permissions registry
(`API_KEY_RESOURCES` in `@n8n/permissions`); `scope-parity.test.ts` fails on an
@ -137,29 +120,17 @@ model; reuse only what applies. Decorators, all from `@n8n/decorators`:
on `@ApiResponse` stripping to hide fields.
- Treat the output DTO as an allowlist. Re-check nested relations, ownership
fields, tokens, and encrypted values.
- An output DTO restricts which fields you return, not which values they may hold.
The registry parses the handler's return value against it, so a value the schema
rejects becomes a `500`. Keep the schema loose enough for anything an existing
row may contain.
- Build the response from the relations the route loaded, not from the entity type.
TypeORM relations are opt-in, so two routes over the same entity can return
different shapes.
- Make input DTOs strict so unknown/partial fields aren't silently accepted:
`Z.class(shape, { strict: true })`.
- Make input DTOs strict so unknown/partial fields aren't silently accepted.
- Secrets: never return a real secret; use the resource's sentinel/placeholder
(or omit). See [Updates and write-only secrets](reference.md#updates-and-write-only-secrets).
## List endpoints (cursor pagination)
Copy the cursor flow from `tags.public.controller.ts`. The input DTO takes
`limit: publicApiPaginationSchema.limit` plus `cursor: z.string().optional()` —
pick `limit` off the schema, never spread the whole `publicApiPaginationSchema`
(it also exports `offset`, which must never be a Public API query param). Use
`decodeCursor` / `encodeNextCursor` from the shared pagination service; the
Copy the cursor flow from `tags.public.controller.ts`. Use `publicApiPaginationSchema`
plus `decodeCursor` / `encodeNextCursor` from the shared pagination service; the
cursor is opaque; return `{ data, nextCursor }` (never a bare array) with
`nextCursor: null` on the last page; an invalid cursor is a `400`. Preserve an
existing endpoint's cursor semantics as-is — but an `offset` param is a
defect to remove, not a contract to preserve. Detail:
existing endpoint's pagination as-is. Detail:
[List endpoints and cursor pagination](reference.md#list-endpoints-and-cursor-pagination).
## Wiring checklist
@ -168,16 +139,8 @@ defect to remove, not a contract to preserve. Detail:
`v1/controllers/index.ts`.
2. Public DTO in `@n8n/api-types` + export from the barrel (`src/dto/`).
3. `@ApiKeyScope` value exists in the permissions registry.
4. Don't hand-write the OpenAPI path or `x-required-scope` for a controller
route — the generator (`v1/openapi-gen/generate.ts`) builds it from your
decorators (`@ApiSummary`/`@ApiDescription`/`@ApiTags`/`@ApiKeyScope`/
`@ApiResponse`/`@ApiErrorResponse`). Run the full `pnpm build` and commit
the regenerated `handlers/<feature>/spec/paths/*.generated.yml` fragment(s)
and `openapi.decorator-routes.generated.yml` —
`generated-spec-drift.test.ts` fails CI if they're stale. `pnpm run
build:data` alone is **not** enough after touching a controller: it runs
the generator against the already-compiled `dist/`, so a new/changed
controller silently doesn't show up unless `tsc` ran first.
4. OpenAPI path with `x-required-scope` matching `@ApiKeyScope`; do **not** set
`x-eov-operation-*` for controller routes; keep top-level `tags:` sorted A→Z.
5. Add the route to `packages/nodes-base/nodes/N8n/n8n-api-coverage.json`.
6. Tests.
@ -198,5 +161,4 @@ existing tests.
- [Test-before-save endpoints](reference.md#test-before-save-endpoints)
- [Errors](reference.md#errors)
- [Testing matrix](reference.md#testing-matrix)
- [Verifying a migration](reference.md#verifying-a-migration)
- [CI and merging](reference.md#ci-and-merging)
- [Migrating legacy EOV endpoints](reference.md#migrating-legacy-eov-endpoints)

View File

@ -13,29 +13,18 @@ Copy the working flow from `v1/controllers/tags.public.controller.ts` (and
`workflows.public.controller.ts` for a `@Param` list) rather than pasting a
snippet here — a copy would drift. The moving parts:
- Input DTO takes `limit: publicApiPaginationSchema.limit` plus an opaque
`cursor: z.string().optional()` — cherry-pick `limit` but never spread the whole
`publicApiPaginationSchema`. That schema also exports `offset`, used by
internal-API-style page params elsewhere; a Public API list DTO must never
expose it as a query param. See `ListTagsQueryDto` for the shape to copy.
- Input DTO composes `publicApiPaginationSchema` from `@n8n/api-types` (a `limit`
plus an opaque `cursor`).
- `decodeCursor` / `encodeNextCursor` live in
`v1/shared/services/pagination.service.ts`. Decode the incoming cursor to
`{ offset, limit }`, guard the decoded shape, and pass `offset`/`limit` to the
service — never `{ skip, take }`. `offset` is an internal implementation
detail of the cursor here, never a client-facing query param. TypeORM's
`skip`/`take` stay inside the repository, at the `find` call.
service.
- Treat the cursor as opaque; never hand-encode a token.
- Return an envelope `{ data, nextCursor }` — never a bare array.
- `encodeNextCursor(...)` returns `null` when there is no further page; surface
that as `nextCursor: null`.
- An invalid/undecodable cursor is a `400` via the existing bad-request error.
- For an existing list endpoint, keep its current *cursor* semantics unchanged.
A leaked `offset` query param (DTO spreading `publicApiPaginationSchema`
instead of picking `limit`) is a defect to remove, not a contract to
preserve — decorator-routed DTOs validate via a plain `z.object()`, which
silently strips unknown query keys rather than rejecting them, so removing
`offset` from the DTO makes it inert rather than erroring for existing
callers.
- For an existing list endpoint, keep its current pagination semantics unchanged.
The output DTO wraps the list as `{ data, nextCursor }` and is declared with
`@ApiResponse(...)` so the registry strips undeclared fields.
@ -100,105 +89,27 @@ scope, RBAC denial. Add whichever apply, matching the nearest existing tests:
`PUT` with the exact sentinel keeps the secret; any other value replaces it;
the sentinel is never persisted as a real secret.
- Migration: path, method, status codes, scope, and response contract are
unchanged. Tests alone cannot show this — see
[Verifying a migration](#verifying-a-migration).
unchanged.
## Request body media types
## Migrating legacy EOV endpoints
`@Body` defaults to `application/json`; the body is already parsed by the
app-wide `bodyParser` before the registry sees it. Declaring
`@Body({ mediaType: 'multipart/form-data', uploadLimits })` instead takes a
`multipart/form-data` body:
Legacy `express-openapi-validator` endpoints live under
`v1/handlers/`, wired through `openapi.yml` with `x-eov-operation-*` and request
types in `packages/cli/src/public-api/types.ts`. Treat these as migration targets,
not templates.
- `uploadLimits: () => MultipartUploadLimits` is a thunk, read once the route
handles its first request (not at startup) — read live config inside it
(e.g. `Container.get(GlobalConfig)`), don't inline a literal.
- The route's `@Body` DTO validates text fields merged with uploaded files,
not `req.body` alone. A text field and a file share the same flat object;
a file field uses `publicApiUploadedFileSchema` (`@n8n/api-types`) — a
multer-file-shaped schema that documents itself as `{ type: 'string',
format: 'binary' }`. Several files under the same field name become an
array.
- An unknown field (one the DTO doesn't declare, on a `{ strict: true }` DTO)
fails with `Unexpected form field "<name>"` — not the default "unrecognized
keys" wording.
- Multer parsing errors map to these statuses: `413` for a size or count limit,
`500` for `LIMIT_UNEXPECTED_FILE`, `400` for any other multer error, and `500`
for anything else (a malformed body). The `500` is not masked, because it is a
`ResponseError`. A missing boundary gives `400` with `multipart file(s) required`.
- The body is parsed **after** every auth/scope/license/quota gate and
**before** controller/route middlewares — a caller those gates would reject
never has their (possibly huge) body read off the socket, and a middleware
that reads `req.body` sees it already parsed.
- `/discover` shows no request schema for a multipart route (same as a legacy
multipart route today) — a client can't assume a JSON schema it never gets.
- The generator documents the body under the `multipart/form-data` content
key (not `application/json`) and adds `413` alongside `415` to the route's
documented responses automatically.
Everything above lives in `packages/cli/src/public-api/media-types/`, one
handler per media type (`REQUEST_BODY_HANDLERS` in
`media-types/request-body/index.ts`). Adding a further media type (e.g.
`application/octet-stream`) means: add it to `RequestBodyMediaOptions` in
`@n8n/decorators`'s `controller/types.ts`, write a handler implementing
`RequestBodyHandler` in `media-types/request-body/`, and register it in
`REQUEST_BODY_HANDLERS` — the registry, resolver, generator and `/discover`
need no change, since they all read the handler, not the media type.
## Binary response bodies
`@ApiResponse(status, { mediaType, description?, headers? })` declares a
success body that the controller method returns as `{ body, headers? }`, for
example an `application/gzip` stream. The framework treats options whose
`mediaType` is in `BINARY_RESPONSE_MEDIA_TYPES` as a binary response.
`mediaType` accepts only those types.
- `body` is a `Buffer` or a `Readable`. The method returns the result and never
writes to `res`.
- `headers` holds the response headers. Every header declared in `headers` must
be in the result. A missing header fails the request with a `500`. Matching is
case-insensitive.
- The registry checks the headers, then sets the declared status, the headers,
and `Content-Type: <mediaType>`. A header in the result cannot replace the
`Content-Type`.
- A stream's first chunk is read before any header is set. So a stream that
fails before its first chunk gives the normal JSON error response, with no
binary `Content-Type`. A method that throws before it returns does the same.
- After the first chunk is sent, an error ends the response. A client that
disconnects is not an error, and the framework destroys the stream.
- A method that returns no binary body fails with a `500`.
- The generator documents the body as `{ type: string, format: binary }`
under the `mediaType` content key, with the description and headers.
The description defaults to `Operation successful.`.
The runtime part is `sendBinaryResponse` in
`packages/cli/src/public-api/media-types/binary-response.ts`.
## Verifying a migration
Tests are written against the new code, so they can't show that the old behavior
survived. These checks can:
- Diff live responses against master. Run the route on an instance per branch and
compare status, body, and error message for the same requests. Reading the old
YAML beside the new Zod schema doesn't find the differences.
- Check whether a field is absent or `null`. `activeVersion` omitted is not the
same response as `activeVersion: null`; use a conditional spread to omit it.
- Check query-param coercion at the edges (`""`, `"0"`, `"false"`, absent). A Zod
DTO and the old validator don't coerce identically.
- Request a route the PR didn't migrate. A shared DTO change can stop the spec
bundle compiling at startup, which turns every legacy route into a `500` without
failing a test.
## CI and merging
Two failures that a migration hits outside the code itself:
- Merge master in before merging. A generator change on master leaves every
branch's committed `*.generated.yml` stale, and `generated-spec-drift.test.ts`
then fails only on the merge commit, so the PR itself stays green and
`MERGEABLE`. `gh pr checks` hides `merge_group` runs; look for the run on
`gh-readonly-queue/master/pr-<number>-<sha>`.
- Ask a maintainer for a `/size-limit-override` early. Generated YAML counts
toward the 1,000-line PR size limit, so a single-route migration can exceed it
on generator output alone.
- Prefer migrating to `@PublicApiController` over extending the handler.
- Preserve the public contract: path, method, scopes, status codes, response
shape, and pagination.
- Move HTTP concerns into the controller and business orchestration into the
shared service; keep the controller a thin HTTP boundary.
- A route must be served by either the EOV handler or a controller, not both —
`scope-parity.test.ts` rejects a duplicated route. Remove the legacy wiring
(its `x-eov-operation-*` entry and handler middleware) only after the new
controller is registered and the spec + tests are updated.
- As a legacy file drops repository access / the `export =` tuple, remove its
entry from the `off` allowlists for `no-repository-in-public-api-handler` and
`require-public-api-controller` in `packages/cli/eslint.config.mjs` (shrink-only
— never extend them).
- For complex legacy-only, multipart, or non-standard endpoints, study the
nearest existing handler first.

View File

@ -32,7 +32,7 @@ Based on the affected area, pick the test layer and pattern:
| CLI / API | Vitest integration | setupTestServer + supertest | `packages/cli/test/integration/` |
| Config | Vitest unit | GlobalConfig + Container | `packages/@n8n/config/src/configs/__tests__/` |
| Editor UI | Vitest | Vue Test Utils + Pinia | `packages/frontend/editor-ui/src/**/__tests__/` |
| E2E / Canvas | Playwright | Test containers + composables | `packages/quality/testing/playwright/` |
| E2E / Canvas | Playwright | Test containers + composables | `packages/testing/playwright/` |
## Step 3: Locate Source Files
@ -73,12 +73,6 @@ Write a regression test that:
- Asserts the CORRECT behavior (test will fail on current code)
- Also includes a "happy path" test to prove the setup works
Complete the test safety gate in `/AGENTS.md` before you run the test:
- Trace side effects from imports, constructors, hooks, and mocked branches
- Keep filesystem access in a test-owned temporary directory
- Set `N8N_USER_FOLDER` before importing modules that resolve it
- When a mock changes a state check, inspect and isolate the branch that it activates
## Step 8: Run and Score
Run the test from the package directory (e.g., `cd packages/nodes-base && pnpm test <file>`).

View File

@ -1,39 +0,0 @@
---
name: n8n:ui-design
description: Guidelines on designing and building UI. Use when working in editor-ui or design-system packages. Triggers for tasks that include refactoring components, styling changes, or feature work.
---
# UI Design
Comprehensive guide for building, styling, and using components in the frontend.
## References
- When styling components, use `packages/frontend/@n8n/design-system/src/styleguide/*.mdx`
- For animations, use `rules/web-animation-guidelines.md`
- When reviewing UI changes, use `rules/web-interface-guidelines.md`
## Best practices
- ALWAYS use CSS variables from `_tokens.scss` or `_primitives.scss` in
`packages/frontend/@n8n/design-system/src/css/`. Look up every length, color,
font-size and duration before you write it as a literal. Each spacing token
carries its px value in a comment: `--spacing--5xs: 0.125rem; /** 2px **/`.
Grep that comment to find the token for a px value. The grep gives one hit for
each token family, so pick the family that matches the property: `--spacing--*`
for margin and padding, `--radius--*` for border-radius, `--height--*` for
height. The styleguide
`packages/frontend/@n8n/design-system/src/styleguide/spacing-size.mdx` lists the
same scale as a table. The spacing scale starts at 2px, so a small value is not
a reason to hardcode. Hardcode only if the lookup finds no token. Give the
reason in a comment.
- ALWAYS prefer using existing components from `packages/frontend/@n8n/design-system/src/components`. Prefer components that aren't marked `@deprecated`.
- If you need to add hover/active alpha behavior to solid components, prefer `color-mix()` with explicit percentages.
- When working with animations or transitions, ALWAYS prefer using mixins from `packages/frontend/@n8n/design-system/src/css/mixins/motion.scss`
## Components
Use existing `design-system` components over creating custom implementations:
- "Add a modal dialog for confirming workflow deletion" → Use `N8nDialog`
- "Add a dropdown to select workflow status" → Use `N8nDropdown` or `N8nSelect`
- "Add button with + icon to add new tiem" → Wrap `N8nButton` with `iconOnly` prop with `N8nTooltip` and wrap in `N8nTooltip`. Use `N8nIcon` and proper aria-label.
- "Add a destructive action button" → use `N8nButton` with `variant="destructive"`
- "Make background color white/black" → Use `var(--background--surface)` for white on light mode and "black" on dark mode
- "Animate the title in gracefully" -> Use `fade-in-up` mixin from `motion.scss` with `var(--duration--base)`

View File

@ -8,7 +8,7 @@ exclude:
# Machine-generated instance-AI expectation traces. Contain ephemeral
# credential reference IDs (nanoids) that re-trip the secrets scanner
# on every regeneration. No secret material.
- packages/quality/testing/playwright/expectations
- packages/testing/playwright/expectations
# Test code repo-wide. Fixtures routinely contain synthetic PEM blocks,
# basic-auth strings, and deliberately fake keys (including the secret

View File

@ -1,3 +1,3 @@
{
"issues": 110
"issues": 169
}

View File

@ -1 +0,0 @@
../../../../.agents/skills/agent-design-language

View File

@ -1 +0,0 @@
../../../../.agents/skills/create-agent-builder-eval

View File

@ -0,0 +1 @@
../../../../.agents/skills/design-system

View File

@ -1 +0,0 @@
../../../../.agents/skills/env-var-docs

View File

@ -1 +0,0 @@
../../../../.agents/skills/gh-stack

View File

@ -1 +0,0 @@
../../../../.agents/skills/ui-design

File diff suppressed because it is too large Load Diff

View File

@ -1,44 +0,0 @@
ARG NODE_VERSION=24
# Debian, not Alpine: Playwright browsers and the agent CLIs need glibc
FROM node:${NODE_VERSION}-bookworm
ARG PLAYWRIGHT_VERSION=1.62.1
ARG OPENCODE_VERSION=1.18.25
RUN apt-get update && apt-get install -y --no-install-recommends \
tmux sudo less jq ripgrep postgresql-client \
&& rm -rf /var/lib/apt/lists/*
# Chromium system libs baked into the image; browser binaries install per-user in onCreate
RUN npx -y playwright@${PLAYWRIGHT_VERSION} install-deps chromium
# Agent harnesses. Self-update is off: the image is the update mechanism
ENV DISABLE_AUTOUPDATER=1
RUN npm install -g @anthropic-ai/claude-code opencode-ai@${OPENCODE_VERSION}
COPY codespaces-env.sh /usr/local/lib/codespaces-env.sh
COPY codespaces-secrets.sh /etc/profile.d/codespaces-secrets.sh
# git and gh must survive token rotation: these read the token on each call
COPY gitcredential-refresh.sh /usr/local/bin/gitcredential-refresh.sh
COPY gh-shim.sh /usr/local/bin/gh
RUN chmod +x /usr/local/bin/gitcredential-refresh.sh /usr/local/bin/gh
# VS Code terminals load secrets here because they do not read profile.d.
RUN echo '[ "${N8N_SKIP_CODESPACE_SECRETS:-}" = "1" ] || . /usr/local/lib/codespaces-env.sh' >> /etc/bash.bashrc
COPY tmux.conf /etc/tmux.conf
RUN echo node ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/node && chmod 0440 /etc/sudoers.d/node
RUN mkdir -p /workspaces && chown node:node /workspaces
RUN corepack enable
ENV PNPM_HOME=/home/node/.local/share/pnpm
ENV PATH=$PNPM_HOME/bin:$PATH
USER node
# The store must sit on the same filesystem as the worktrees (/workspaces is its own
# mount), or pnpm copies node_modules into every worktree instead of hardlinking.
# The directory is created on the first install: Codespaces mounts /workspaces at runtime.
RUN mkdir -p $PNPM_HOME && pnpm config set store-dir /workspaces/.pnpm-store --global

View File

@ -1,542 +0,0 @@
# Cloud agent sessions (Codespaces)
Run long-lived, human-steered agent sessions (Claude Code / OpenCode) on a
GitHub Codespace instead of your laptop: start a task, close the lid, steer it
from anywhere, resume tomorrow.
This is a separate devcontainer config from the laptop one in
`.devcontainer/` — it ships both agent CLIs, `tmux` for session persistence,
Playwright system deps, and Docker-in-Docker (for testcontainers and
`pnpm --filter n8n-containers services`), on the same Postgres sidecar setup.
## One-time setup (~5 min)
1. Give the GitHub CLI the codespace scope:
```bash
gh auth refresh -h github.com -s codespace
```
2. Add provider keys at [github.com/settings/codespaces](https://github.com/settings/codespaces)
if your client needs them. Give each secret access to `n8n-io/n8n`.
- `ANTHROPIC_API_KEY` for Claude Code in the terminal (`pnpm session`).
Alternative for Max subscriptions: `CLAUDE_CODE_OAUTH_TOKEN` from
`claude setup-token`. The desktop app does not need either secret.
- `OPENROUTER_API_KEY` for OpenCode.
## Choose a client
All clients run the agent, its tools, and builds in the Codespace. Your laptop
shows the conversation only.
| Client | Command | Use it when |
|---|---|---|
| Claude Code desktop app | `pnpm session ssh-config` once | You want the full desktop experience. |
| Claude Code in VS Code | Open the codespace in VS Code or the browser | You already work in VS Code. |
| Claude Code in a terminal | `pnpm session` | You want a plain terminal, or you are on a remote machine. |
| OpenCode | `pnpm session:opencode` | You use OpenCode. See [Local OpenCode clients](#local-opencode-clients). |
### Claude Code desktop app
1. From a local checkout, add an SSH host for your codespace:
```bash
pnpm session ssh-config
```
The command creates or starts the codespace. It writes the host
`n8n-codespace` to `~/.ssh/n8n-codespace.conf` and includes that file from
`~/.ssh/config`.
2. In the desktop app, add an SSH connection to the host `n8n-codespace`.
3. Open the folder `/workspaces/n8n`, or a worktree under `/workspaces`.
The desktop app installs its own Claude Code on the codespace and signs in
with your desktop account. Sessions continue when you close the app. They stop
when the codespace stops.
- **Run `pnpm session ssh-config` again after you recreate the codespace.**
- **If the connection times out**, the codespace is probably starting. Run
`pnpm session ssh-config`, then connect again.
- **Test the connection** with `ssh n8n-codespace`.
### Claude Code in VS Code
The dev container installs the Claude Code extension. Open the codespace in
VS Code or the browser, then open the Claude Code panel. Sign in from the
extension if you did not add `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN`.
### Claude Code in a terminal
```bash
pnpm session # attach Claude Code (creates everything on first run)
pnpm session:shell # open a shell in the default checkout
pnpm session:opencode # connect the local OpenCode TUI
pnpm session:opencode fix-flaky # use a separate remote worktree
pnpm session:opencode --web # open OpenCode in the local browser
pnpm session:opencode --legacy # use the remote TUI in tmux
pnpm session fix-flaky # Claude Code in a separate worktree
pnpm session ls # what's running
pnpm session tunnel # forward n8n ports (default 5678, 8080); Ctrl-C to stop
pnpm session ssh-config # add the n8n-codespace SSH host for the desktop app
pnpm session stop # end of day: billing stops, disk survives
pnpm session rm # delete the codespace
```
The dev container supports Codespaces with 2, 4, or 8 cores. The session commands
create an 8-core Codespace by default.
- **Detach from tmux** with `Ctrl-b d` — the agent keeps working without you.
Local OpenCode connections use the exit controls described below.
- **Scroll in tmux** with the mouse wheel (tmux mouse mode is on). To use the
terminal's own text selection, hold **Shift** and drag.
- **Reattach** by running the same session command from any machine.
- Each named session gets its own worktree (`/workspaces/wt-<name>`, branch
`session/<name>`), so parallel agents never touch each other's tree. A new
branch starts from the latest `origin/master`. Builds
in fresh worktrees are cache-hits via a shared turbo cache.
- First codespace creation takes ~20 min uncached (image + full build). After
that, sessions attach instantly; new worktrees cost a `pnpm install` (~10 s: the
store lives on `/workspaces`, so `node_modules` is hardlinked, not copied).
## Configure the local OpenCode harness
Install the pinned shared harness:
```bash
pnpm agent:harness
```
The command verifies the release checksum. It caches the bundle under
`~/.cache/n8n-agent-harness`. It then links the bundle plugin into
`~/.config/opencode/plugins`. Run the command again after the lock changes.
Restart OpenCode after the command completes.
## Local OpenCode clients
Run these commands from a local checkout. The TUI runs on your laptop. The
OpenCode server, repository, tools, and builds run in the Codespace. Browser
mode opens the remote web interface through a local connection.
```bash
pnpm session:opencode fix-flaky # resume that workspace's latest conversation
pnpm session:opencode fix-flaky --web # open it in a browser
pnpm session:opencode fix-flaky --new # start a new conversation in that worktree
pnpm session:opencode --web --port 4100 # override the default browser port
pnpm session:opencode --help
```
For the TUI, install the same OpenCode version as the remote server. The
`OPENCODE_VERSION` argument in the [Dockerfile](Dockerfile) pins the version
for new images:
```bash
npm install -g opencode-ai@<version>
```
Use npm, not pnpm. The package gets its binary from a `postinstall` script.
pnpm does not run that script by default.
The launcher checks both versions. It reports a mismatch with both version
numbers before it opens the TUI. An existing Codespace can have a different
version. Install that version locally, or use `--web`. Browser mode does not
need a local OpenCode install.
The launcher sends its bootstrap code over SSH, so an existing Codespace does
not need a rebuild to use this connection method.
The remote server loads the shared OpenCode harness that the Codespace
post-start command installs. The local harness setup above is not required for
these commands.
The command prepares the worktree, starts or reuses one server, opens an SSH
tunnel, and connects the client. Each workspace opens its own conversation: the
most recently updated one in that worktree. A conversation moves to the front
when it receives a message. Switching to a conversation without sending a
message does not move it. Switching to another workspace's conversation does
not change what this workspace opens next. `--new` starts a new conversation
instead. It preserves the worktree and the old conversation.
- **Exit the TUI** with `/exit` or its quit shortcut. The launcher closes its
tunnel. The remote server stays running.
- **Disconnect browser mode** with `Ctrl-C` in the launcher terminal. Closing
the browser tab does not close the tunnel. Keep the launcher running while
you use the browser.
- **Reconnect** with the same command after a network interruption. After a
Codespace stop, the command restarts the server and opens the workspace's
most recently updated conversation.
A stop terminates running tools. It does not resume interrupted commands.
- **Browser mode uses local port 4096 by default.** Use `--port` to override it.
TUI mode selects an available port unless you specify one.
A fixed browser port preserves the browser origin across runs.
An occupied fixed port causes an error. It does not stop the existing listener.
- **Use model and permission controls in the client.** The new connection uses
the remote OpenCode configuration. It does not force the legacy `--auto` mode.
For the old CLI flags, use `pnpm session:opencode fix-flaky --legacy --model <model>`.
Both server and tunnel bind to `127.0.0.1`. The server uses a generated password.
The TUI receives it through its environment. Browser mode uses a local proxy
that adds authentication and rejects requests from other browser origins. The
proxy supports streamed responses, attachments, and terminal WebSockets. The
password does not appear in the URL or terminal output. Other processes on
your laptop can access the local browser proxy while it runs. Do not forward
this proxy or the OpenCode server port to other machines.
The server enables only OpenRouter. It reads `OPENROUTER_API_KEY` when it starts.
It enables OpenCode code mode by default to reduce the initial tool context.
Browser mode opens the workspace's most recently updated conversation directly.
The web UI stores opened projects in browser storage. If a new-session page
shows **New project**, open `/workspaces/n8n` there once. Keep the same browser
port when you reconnect to preserve this selection.
The server runs in the detached tmux session `n8n-opencode-server`. Its log is
`/workspaces/.n8n-opencode/server.log`. That directory also holds the server
credentials. It is readable only by its owner.
An unhealthy server produces an error without stopping active work. Inspect
the log through `pnpm session:shell`. To restart it after checking active work,
run `tmux kill-session -t '=n8n-opencode-server'` in that shell. Then reconnect.
Restart the server after changing provider secrets, server configuration, or
the pinned harness release.
### Limits
- The launcher supports macOS and Linux. On Windows, run it in WSL. If the
browser does not open automatically, open the printed URL yourself.
- Local clipboard and attachment controls depend on the client, terminal,
file type, and model. A laptop file path does not copy a file to the Codespace.
Use a supported attachment control or copy the file with `gh codespace cp`.
- Tools and local MCP processes run on the Codespace. Laptop configuration,
browser sessions, and files do not sync automatically.
- Workspace names identify worktrees. Two agents that use the same name share
files. Separate conversations alone do not isolate edits.
- The server survives a client disconnect while the Codespace stays running.
Codespaces idle timeouts still apply. An open tunnel is not a guarantee that
the Codespace will stay awake. Use `pnpm session stop` to stop compute billing.
- Existing tmux OpenCode conversations are not migrated automatically. Use
`--legacy` to return to them.
For upstream behavior, see the [OpenCode CLI](https://opencode.ai/docs/cli/),
[server](https://opencode.ai/docs/server/), and [web](https://opencode.ai/docs/web/)
documentation.
## PR previews (a running instance of someone else's PR)
A preview is a codespace that serves one pull request, so a reviewer can use the
PR instead of reading it. It is a different box from a `pnpm session`: it uses
`.devcontainer/preview/devcontainer.json`, and its display name is
`preview/pr-<number>`.
**From GitHub:** for a PR targeting master, add the `codespace-preview` label to the PR.
[util-codespace-preview.yml](../../.github/workflows/util-codespace-preview.yml)
creates the box, serves the PR head, shares port 5678 with the org, and comments
the URL on the PR. A later push serves the new head in the same box. Remove the
label, or close the PR, to delete the box. The same workflow runs by hand from
the Actions tab: give it a PR number and `up`, `refresh` or `down`.
**From your laptop:** `pnpm preview up <pr>` does the same thing, plus
`pnpm preview refresh <pr>`, `pnpm preview down <pr>` and `pnpm preview ls`. It
needs `gh` with the codespace scope, the same as `pnpm session`.
- **Watch it come up on the PR.** The comment appears before the box work starts
and updates about once a minute with a checklist of the phases, so you can see
which step a slow preview is on. `pnpm preview up <pr>` prints the same phases as
plain progress — the markers the comment reads are `--json` only.
- **Sign in with one click** at `<url>/preview-signin`. It logs you in as the
seeded owner and sends you to the editor. The credentials are
`preview@n8n.io` / `PreviewInstance1`. They are not secrets: the boundary is
the org-visible forwarded port, which needs a GitHub sign-in and n8n org
membership.
- **Configure the instance with `preview:*` labels.** `preview:enterprise` serves
it with a licence, so enterprise features such as SSO and source control are
present. `preview:debug` sets `N8N_LOG_LEVEL=debug`. Adding or removing one
re-serves the box; it never creates or deletes one. From a laptop the labels
apply the same way — `pnpm preview refresh <pr>` reads them from the PR. The
toggles are defined in `scripts/codespace-preview/preview-labels.mjs`; add new ones there.
- **Configure the instance from a webhook.** A preview also reads extra
environment from an n8n webhook, so a value can change without a commit. It
needs the `CODESPACE_ENV_URL`, `CODESPACE_ENV_USER` and `CODESPACE_ENV_PASSWORD`
codespace secrets. Every key the webhook returns becomes an environment
variable, so editing that workflow runs code in the box. Without the secrets
the preview serves as usual. See [WORKFLOWS.md](../../.github/WORKFLOWS.md).
- **A preview sleeps after 2 hours** of no use and GitHub deletes it after 24
hours. A box that slept serves nothing and its port is private again, so wake
it with `pnpm preview up <pr>` or a manual run of
[util-codespace-preview.yml](../../.github/workflows/util-codespace-preview.yml)
with `up`. Removing and adding the label works too, but it deletes the box and
builds a new one.
- **A PR from a fork gets no preview.** A codespace's token is scoped to
`n8n-io/n8n`, so it cannot check out a fork head.
- **A PR that predates this tooling has no `scripts/codespace-preview/preview-serve.mjs`.** The
serve step says so and stops; rebase the PR on master and retry.
## Agent worker (drive a session from n8n)
`agent-worker.mjs` lets an n8n workflow drive an OpenCode session on the
codespace. This is how Slack (the Flaky bot) steers a session that runs here.
Each turn runs `openrouter/openai/gpt-5.6-sol` through
`opencode run --format json --auto`. The session state remains on disk. The
returned session ID continues a conversation across turns and a Codespace
restart.
**The worker polls outward. Nothing inbound is exposed.** GitHub sets every
forwarded port to private on start. It gives no API to make a port public. So
you cannot reach a codespace from outside reliably. The worker calls out
instead. It asks n8n for a turn addressed to this box's owner (`$GITHUB_USER`).
It runs the turn. It sends the result to the turn's resume URL. It uses no
tunnel, no open port, and no domain.
The post-start command installs the pinned OpenCode harness before it starts the
worker. It verifies a valid cached bundle before reuse. The worker does not start
if the harness is unavailable. `/tmp/post-start-status.json` contains the harness
and worker status.
The worker needs three
secrets and uses three optional secrets. Add them at
[github.com/settings/codespaces](https://github.com/settings/codespaces), the
same way as `ANTHROPIC_API_KEY`:
- `AGENT_WORKER_TOKEN` — the shared bearer token. The worker sends it on each poll.
- `N8N_DEQUEUE_URL` — the n8n webhook that returns a pending turn.
- `OPENROUTER_API_KEY` — the model provider key for Sol.
- `SLACK_BOT_TOKEN` — optional bot token for progress messages. It needs `chat:write` only.
- `FLAKY_MCP_URL` and `FLAKY_MCP_TOKEN` — optional Flaky MCP connection for OpenCode.
The worker explicitly reloads `/usr/local/lib/codespaces-env.sh` when its tmux
session starts. An existing tmux server can otherwise retain an environment
that did not load the Codespaces secrets.
The dequeue payload can include `slack.channel` and `slack.thread_ts`. The
worker posts one placeholder in that thread. It coalesces completed tool calls.
It updates the message at most once every 1.5 seconds. It does not send reasoning
text. The worker replaces the placeholder with the final answer.
If the Slack API fails, the turn still completes through the n8n resume URL.
The worker does not export its dequeue or Slack credentials to OpenCode. It uses
the harness `sandbox` runtime and `slack` profile. The profile supplies the
atomic-turn instruction.
Interactive sessions remove all worker-only credentials before they start.
Interactive OpenCode uses the global harness plugin with the `sandbox` runtime.
It does not set a profile. It keeps the dynamic OpenRouter configuration.
An idle worker starts with a 3-second poll interval. After each empty dequeue,
it doubles the interval and limits it to 30 seconds. Work resets the interval
to 3 seconds. Queued turns run without a delay between them.
The log is at `/tmp/agent-worker.log`. The tmux session is `agent-worker`. To
watch it, run `tmux attach -t agent-worker`. The worker does not start if a
required secret or `$GITHUB_USER` is missing.
A turn stops after about 25 minutes (`TURN_TIMEOUT_MS`). This limit is below the
n8n Wait limit. So the worker reports a clear message before n8n reports a
generic timeout. Keep the worker limit below the n8n limit if you change either.
**A turn is atomic, and the harness tells the session so.** The turn ends on the
session's final message, and its children end with it: a background `Bash` task
is killed, `Monitor` events never arrive, `PushNotification` has nowhere to go,
and `ScheduleWakeup` never fires. The session also gets no turn of its own to
report back in — the turn's resume URL continues one waiting n8n execution and is
then spent, so nothing on the box can post to the thread unprompted. A session
that backgrounds a build and signs off with "I'll verify once it finishes" is
therefore describing something that cannot happen. The harness `slack` profile
states this contract. This is only the n8n/Slack path: an
interactive session (`pnpm session`, tmux) is long-lived, so background work,
monitors and notifications behave normally there.
### Build and run the app in a session
The prebuild already installed the dependencies and warmed the build. So a
session rarely needs a cold `pnpm install` or a full `pnpm build`. Both are slow
(often 10–20 minutes cold). Both can outlast a turn's limit.
- **Bring the app up with one command: `pnpm dev:up`.** It installs missing
dependencies, starts the backend, waits for health, and prints the URL. Add
`--build` only when a frontend change must appear (see below).
- **Open the app** at `https://<codespace-name>-5678.app.github.dev`. `dev:up`
makes that port visible to the org, thus any n8n member who is signed into
GitHub can open it. You do not need a tunnel. GitHub makes every forwarded port
private again at each container start, so `dev:up` shares it again on each run.
To see the current state, run `gh codespace ports`. The share command needs `gh`
with the codespace scope (see the one-time setup above). If it fails, `dev:up`
starts the app, prints the reason, and gives you the command to try again. A
private port opens for you only, in a browser that is signed in to GitHub. An
anonymous or server caller gets a 302. That is why the worker polls outward.
- **`pnpm dev` no longer exists.** Use `pnpm dev:be` for the backend (on 5678).
Use `pnpm dev:fe:editor` for the editor UI with hot reload (on 8080).
- **`dev:be` serves the editor from the `dist` build.** So a frontend edit does
not hot-reload there. Run `pnpm dev:up --build` (or `pnpm build`) and restart
to show it. For live frontend hot reload, use `pnpm dev:fe:editor`. That path
needs `pnpm session tunnel 5678 8080` from your laptop. Its API base is set to
`localhost:5678`, so the `-8080.app.github.dev` URL does not work on its own.
- Run `pnpm install` only when the dependencies change. `pnpm build` reuses the
turbo cache and is fast when warm.
- To clear stale build outputs after a branch switch, run `pnpm reset`. Add
`--full` if that does not clear it.
- Run a long build in the foreground and give it its own turn. Backgrounding it
does not help: it is killed when the turn ends (see above). Do not chain an
install and a full build behind other work in one turn either — that is what
runs into the 25-minute limit.
## Flaky tools (MCP)
Claude sessions and the headless OpenCode worker get the `flaky` MCP server automatically: Currents
flaky/quarantine data, the `qa_*` BigQuery dataset, Sentry RCA, live Linear,
and repo investigation. The worker keeps the token in its environment and puts
only an environment reference in the OpenCode config. Forks have no secrets and
skip it. Tell the agent to call `get_flaky_context` first — it returns the rules
the tools assume.
`post-start.mjs` registers the server for Claude Code on each container start.
Its `headersHelper` reads the token from the secrets file on each connect, so
the server works in every client. The token is not copied into the Claude Code
MCP configuration.
## Quality and security skills (Claude plugins)
Claude sessions can also load the private skills from the
`n8n-io/n8n-agent-skills` repository. `post-start.mjs` installs both plugins on
each container start, so every session gets the skills with no per-session step:
- `quality` — bug insights, defect attribution, flaky test investigation,
mutation and property testing, PR council, and more.
- `security` — security code review, adversarial review of security-fix PRs,
regression test generation, and Security Hub report triage.
Together they add roughly 5k always-on tokens to every session. Drop a plugin
from `PLUGINS` in `plugins.mjs` if that budget matters more than the skills.
`post-start.mjs` and the `pnpm session` prelude in `scripts/cloud-session.mjs`
both read that list, so a session that races the container start still gets
every plugin.
The private marketplace uses the codespace's own GitHub auth — no extra token.
`devcontainer.json` grants the codespace read access to
`n8n-io/n8n-agent-skills` via `customizations.codespaces.repositories`, and
**each user authorizes that access once when they create the codespace** (GitHub
prompts for it, then remembers). Both repos are in the same org, which is what
lets this work.
The codespace authenticates git over HTTPS and has no SSH key, so `post-start.mjs`
sets `CLAUDE_CODE_PLUGIN_PREFER_HTTPS=1` — Claude Code's plugin loader otherwise
clones `owner/repo` shorthand over SSH and the private clone fails.
If a user does not authorize the grant, the clone fails and the skills step is
skipped (the worker still starts). Existing codespaces created before this change
need a recreate to get the prompt.
### When the skills are missing
`/tmp/post-start-status.json` lists what installed and what did not, and
`/tmp/post-start.log` has the detail. Reading the log:
- A failed `skills repo reachable` line means the grant was not authorized.
- A `marketplace add` failure mentioning `File exists` is a clone that died
partway through `~/.claude/plugins/marketplaces/n8n-io-n8n-agent-skills`, the
path the loader stages into before renaming it to the cache. This has been
seen once as a transient failure, so the script removes that path and retries
the add once.
- Any other `marketplace add` failure after a reachable repo means a
loader-auth problem.
A failure that survives the retry needs a human — the container still starts and
the worker still runs, only the skills are missing.
Both `marketplace add` and `plugin install` are idempotent, so re-running the
script by hand is safe:
```bash
node /workspaces/n8n/.devcontainer/codespaces/post-start.mjs
```
Verify with `claude plugin list`, then restart the session (or `/reload-plugins`)
to pull the skills into context.
## Viewing the dev UI locally
`pnpm session tunnel` works with every client. With the terminal client, use
two terminal windows:
```bash
pnpm session # window 1: attach the agent session
pnpm session tunnel # window 2: forward 5678 + 8080 to localhost
```
Attaching lands you in the agent (Claude), not a shell — open a shell in a new
tmux window with `Ctrl-b c` (or ask the agent), then pick a dev server and open
the matching port:
- `pnpm dev:up` — backend on **5678** (editor served from the `dist` build).
Open http://localhost:5678.
- `pnpm dev:fe:editor` — editor UI with hot reload on **8080**. Open
http://localhost:8080. It needs the backend on 5678 too, so run `pnpm dev:up`
alongside it.
The tunnel prints nothing while forwarding — that's normal. `Connection
refused` means nothing is listening on that port in the codespace yet; it
starts serving as soon as the dev server is up, no restart needed. Pass ports
to override the defaults (`pnpm session tunnel 5678 8080 5679`), but always
forward the pair together with matching numbers: the Vite dev UI points its
API base at `localhost:5678` (the `N8N_PORT` default), so an asymmetric or
partial mapping breaks it.
## What survives what
| Event | Running processes | Disk (checkout, worktrees, chat history) |
|---|---|---|
| Detach / close laptop / network drop | ✅ keep running | ✅ |
| Stop, or idle timeout (2 h for a codespace that `pnpm session` creates, max 4 h) | ❌ killed | ✅ |
| Delete (`pnpm session rm`) | ❌ | ❌ (push your branches first) |
After a stop, `pnpm session <name>` restarts the codespace (~30–60 s); run
`claude --resume` (or `--continue`) inside to restore the conversation from disk.
## Gotchas (learned the hard way)
- **Codespaces clones the repo to `/workspaces/n8n`**, not `/workspaces` —
`workspaceFolder` here differs from the laptop config on purpose.
- **A failing `onCreateCommand` is fatal**: Codespaces discards the container
and drops you into a minimal recovery container (no node, no CLIs). If your
codespace has nothing installed, check
`/workspaces/.codespaces/.persistedshare/creation.log`.
- **`gh codespace ssh` needs sshd inside the container** — provided by the
`sshd` devcontainer feature, don't remove it.
- **User secrets aren't visible in ssh shells by default**: the codespace
agent injects them into VS Code sessions only; they're delivered
base64-encoded to `/workspaces/.codespaces/shared/.env-secrets`. The image
sources `/usr/local/lib/codespaces-env.sh` in login shells (profile.d), in
interactive shells (bashrc), and in the `pnpm session` prelude. Desktop app
sessions do not source it. `git`, `gh`, `scripts/codespace-env.mjs`, and the
`flaky` MCP entry read the file when they run, so they work everywhere. New
code that needs a secret must do the same.
- **Do not read `CODESPACE_NAME` or `GITHUB_USER` from the process env** — use
`scripts/codespace-env.mjs`. Codespaces gives these variables to VS Code
sessions only. Other processes read them from `codespaces-env.sh`, and a
process that tmux starts can get an empty copy: tmux keeps the environment of
its own start, and `update-environment` does not refresh these keys. A worker
polled correctly as its owner while `dev:up` in the same session saw an empty
box name, printed the localhost URL, and did not share the port. The helper
reads `/workspaces/.codespaces/shared`, which is always correct.
- **You cannot paste images into a terminal Claude session.** Image paste reads
the clipboard of the machine where `claude` runs — the codespace, not your
laptop. Drag the file into the VS Code explorer (or
`gh codespace cp shot.png remote:/workspaces/n8n/`) and give Claude the
path. The file stays on disk and survives detach and `--resume`.
- **`git push` / `gh` return 401 in tmux and long sessions** — same root
cause as the secrets gotcha, plus rotation: Codespaces refreshes the
on-disk `GITHUB_TOKEN` every few minutes, so a login-time snapshot goes
stale. The image fixes both: a credential helper reads the current token
on each `git push` (`gitcredential-refresh.sh`), and a shim at
`/usr/local/bin/gh` does the same for `gh`. The token is scoped to
`n8n-io/n8n`: fork-based flows do not work, push branches directly. Do
not add SSH keys as a workaround — they have no per-repo granularity.
- Codespaces created by org members on this repo are **org-owned and
org-billed** (organization ownership + a monthly Codespaces budget are
enabled for n8n-io). Codespaces created before that change, or by
non-members, bill to the personal account (free tier: 120 core-hours/month).
If creation unexpectedly falls back to personal billing, the org budget is
exhausted for the period.
## Costs
Compute bills only while the codespace runs: ~$0.72/hr for the 8-core box,
~nothing stopped. Usage draws from a shared monthly org budget, so
`pnpm session stop` when you leave; the idle timeout is the backstop, not the
plan.
Previews draw from the same budget on a 2-core box. Remove the
`codespace-preview` label when the review is done instead of waiting for the
24-hour retention.

View File

@ -1,416 +0,0 @@
#!/usr/bin/env node
// GitHub keeps Codespaces ports private, so inbound delivery is not available.
import { spawn } from 'node:child_process';
import { resolve as resolvePath, sep } from 'node:path';
import { setTimeout as sleep } from 'node:timers/promises';
import { pathToFileURL } from 'node:url';
import { codespaceEnv } from '../../scripts/codespace-env.mjs';
const DEQUEUE_URL = process.env.N8N_DEQUEUE_URL;
const TOKEN = process.env.AGENT_WORKER_TOKEN;
const SLACK_TOKEN = process.env.SLACK_BOT_TOKEN;
// tmux can retain empty identity values, but the Codespaces files stay current.
const GITHUB_USER = codespaceEnv('GITHUB_USER');
const BOX_ID = codespaceEnv('CODESPACE_NAME');
const ROOT = '/workspaces';
const INITIAL_POLL_INTERVAL_MS = 3000;
const MAX_POLL_INTERVAL_MS = 30_000;
const SLACK_UPDATE_INTERVAL_MS = 1500;
const SLACK_TEXT_LIMIT = 3900;
export function openCodeConfig(environment) {
const config = {
provider: { openrouter: { options: { apiKey: '{env:OPENROUTER_API_KEY}' } } },
};
if (environment.FLAKY_MCP_URL && environment.FLAKY_MCP_TOKEN) {
config.mcp = {
flaky: {
type: 'remote',
url: environment.FLAKY_MCP_URL,
enabled: true,
oauth: false,
headers: { Authorization: 'Bearer {env:FLAKY_MCP_TOKEN}' },
},
};
}
return config;
}
function posNum(name, fallback) {
const raw = process.env[name];
if (raw === undefined) return fallback;
const n = Number(raw);
if (Number.isFinite(n) && n > 0) return n;
console.error(`${name} is not a positive number ("${raw}"); using ${fallback}.`);
return fallback;
}
function nextIdlePollInterval(interval) {
return Math.min(interval * 2, MAX_POLL_INTERVAL_MS);
}
export async function pollOnce(
interval,
{ dequeueTurn = dequeue, handleTurn = handle, wait = sleep, logError = console.error } = {},
) {
let turn;
try {
turn = await dequeueTurn();
} catch (error) {
logError(`poll error: ${error.message}`);
await wait(interval);
return nextIdlePollInterval(interval);
}
if (!turn) {
await wait(interval);
return nextIdlePollInterval(interval);
}
try {
await handleTurn(turn);
return INITIAL_POLL_INTERVAL_MS;
} catch (error) {
logError(`poll error: ${error.message}`);
await wait(INITIAL_POLL_INTERVAL_MS);
return nextIdlePollInterval(INITIAL_POLL_INTERVAL_MS);
}
}
// This limit expires before n8n's Wait node so that the user receives a specific error.
const TURN_TIMEOUT_MS = posNum('TURN_TIMEOUT_MS', 25 * 60_000);
function turnTimeoutMessage(timeout) {
const duration = timeout % 60_000 === 0 ? `${timeout / 60_000}-minute` : `${timeout}-millisecond`;
return `The turn passed the ${duration} limit and stopped. It may have been in a build. Do a smaller step, or run a long build in its own turn.`;
}
export function openCodeEnvironment(environment) {
const childEnvironment = { ...environment };
delete childEnvironment.AGENT_WORKER_TOKEN;
delete childEnvironment.N8N_DEQUEUE_URL;
delete childEnvironment.SLACK_BOT_TOKEN;
childEnvironment.OPENCODE_CONFIG_CONTENT = JSON.stringify(openCodeConfig(childEnvironment));
childEnvironment.N8N_AGENT_RUNTIME = 'sandbox';
childEnvironment.N8N_AGENT_PROFILE = 'slack';
return childEnvironment;
}
// OpenCode does not need the credentials that control the broker.
const TURN_ENV = openCodeEnvironment(process.env);
if (BOX_ID) TURN_ENV.CODESPACE_NAME = BOX_ID;
if (GITHUB_USER) TURN_ENV.GITHUB_USER = GITHUB_USER;
function safeCwd(cwd) {
const safeCwd = resolvePath(typeof cwd === 'string' && cwd ? cwd : `${ROOT}/n8n`);
if (safeCwd !== ROOT && !safeCwd.startsWith(ROOT + sep))
throw new Error(`cwd must be under ${ROOT}`);
return safeCwd;
}
function stopProcessTree(child, signal) {
if (!child.pid) return;
try {
process.kill(-child.pid, signal);
} catch {
child.kill(signal);
}
}
function eventError(event) {
return event.error?.data?.message ?? event.error?.message ?? event.error?.name;
}
export function runOpenCode(
{ message, sessionId, cwd },
onEvent,
onSession,
{
spawnProcess = spawn,
stopProcess = stopProcessTree,
timeout = TURN_TIMEOUT_MS,
killDelay = 5000,
} = {},
) {
const directory = safeCwd(cwd);
const args = ['run', '--model', 'openrouter/openai/gpt-5.6-sol', '--format', 'json', '--auto'];
if (sessionId) args.push('--session', sessionId);
return new Promise((resolve, reject) => {
const child = spawnProcess('opencode', args, {
cwd: directory,
detached: true,
env: TURN_ENV,
stdio: ['pipe', 'pipe', 'pipe'],
});
let activeSessionId = sessionId ?? '';
let buffer = '';
let stderr = '';
let error = '';
const text = [];
let timedOut = false;
let forceTimer;
const timer = setTimeout(() => {
timedOut = true;
stopProcess(child, 'SIGTERM');
forceTimer = setTimeout(() => stopProcess(child, 'SIGKILL'), killDelay);
forceTimer.unref?.();
}, timeout);
const consume = (line) => {
if (!line.trim()) return;
let event;
try {
event = JSON.parse(line);
} catch {
console.error(`OpenCode output ignored: ${line.slice(0, 200)}`);
return;
}
if (!activeSessionId && typeof event.sessionID === 'string') {
activeSessionId = event.sessionID;
onSession(activeSessionId);
}
if (event.sessionID !== activeSessionId) return;
onEvent(event);
if (event.type === 'text' && typeof event.part?.text === 'string') text.push(event.part.text);
if (event.type === 'error') error = eventError(event) || 'OpenCode failed';
};
child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => {
buffer += chunk;
const lines = buffer.split(/\r?\n/);
buffer = lines.pop() ?? '';
for (const line of lines) consume(line);
});
child.stderr.setEncoding('utf8');
child.stderr.on('data', (chunk) => (stderr = `${stderr}${chunk}`.slice(-4000)));
child.once('error', (processError) => {
clearTimeout(timer);
if (forceTimer) clearTimeout(forceTimer);
reject(processError);
});
child.once('close', (code) => {
clearTimeout(timer);
if (forceTimer) clearTimeout(forceTimer);
consume(buffer);
if (timedOut) {
reject(new Error(turnTimeoutMessage(timeout)));
return;
}
if (code !== 0 || error) {
const failureMessage = error || stderr.trim() || `OpenCode exited with ${code}`;
reject(new Error(failureMessage));
return;
}
if (!activeSessionId) {
reject(new Error('OpenCode did not return a session id'));
return;
}
resolve({ result: text.join('\n').trim(), session_id: activeSessionId });
});
child.stdin.end(message);
});
}
async function post(url, body) {
return fetch(url, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
signal: AbortSignal.timeout(20_000),
});
}
async function dequeue() {
const res = await post(DEQUEUE_URL, { githubUser: GITHUB_USER, boxId: BOX_ID, token: TOKEN });
if (!res.ok) throw new Error(`dequeue HTTP ${res.status}`);
const text = await res.text();
if (!text.trim()) return null;
const turn = JSON.parse(text);
return turn?.turnId ? turn : null;
}
async function slackApi(method, body) {
const res = await fetch(`https://slack.com/api/${method}`, {
method: 'POST',
headers: { authorization: `Bearer ${SLACK_TOKEN}`, 'content-type': 'application/json' },
body: JSON.stringify(body),
signal: AbortSignal.timeout(20_000),
});
const result = await res.json();
if (!res.ok || !result.ok) throw new Error(result.error || `HTTP ${res.status}`);
return result;
}
function progressText(tools) {
const progress = [...tools.values()].slice(-6).map(({ status, title }) => {
if (status === 'error') return `Failed: ${title}`;
return `Done: ${title}`;
});
return (progress.length ? progress.join('\n') : 'Flaky is working…').slice(0, SLACK_TEXT_LIMIT);
}
function finalSlackText(text, sessionId, boxId) {
const metadata = [sessionId && `⟳session:${sessionId}`, boxId && `⟳box:${boxId}`]
.filter(Boolean)
.join(' ');
const suffix = metadata ? `\n\n${metadata}` : '';
const body = text || 'Flaky completed the turn';
return `${body.slice(0, SLACK_TEXT_LIMIT - suffix.length)}${suffix}`;
}
const NO_SLACK_PROGRESS = { event() {}, async finish() {} };
export async function startSlackProgress(
turn,
{
callSlack = SLACK_TOKEN ? slackApi : undefined,
updateInterval = SLACK_UPDATE_INTERVAL_MS,
} = {},
) {
const channel = turn.slack?.channel;
const threadTs = turn.slack?.thread_ts;
if (!callSlack || typeof channel !== 'string' || typeof threadTs !== 'string')
return NO_SLACK_PROGRESS;
let message;
try {
message = await callSlack('chat.postMessage', {
channel,
thread_ts: threadTs,
text: 'Flaky is working…',
});
} catch (error) {
console.error(`turn ${turn.turnId}: Slack placeholder failed: ${error.message}`);
return NO_SLACK_PROGRESS;
}
const tools = new Map();
let timer;
let inFlight;
let pending;
let lastUpdate = 0;
let stopped = false;
const update = (text) =>
callSlack('chat.update', { channel, ts: message.ts, text }).catch((error) =>
console.error(`turn ${turn.turnId}: Slack update failed: ${error.message}`),
);
const flush = () => {
if (stopped || timer || inFlight || pending === undefined) return;
const delay = Math.max(0, lastUpdate + updateInterval - Date.now());
timer = setTimeout(() => {
timer = undefined;
const text = pending;
pending = undefined;
lastUpdate = Date.now();
inFlight = update(text).finally(() => {
inFlight = undefined;
flush();
});
}, delay);
};
const schedule = () => {
pending = progressText(tools);
flush();
};
return {
event(event) {
if (!event.part?.id) return;
if (event.type === 'tool_use') {
tools.set(event.part.id, {
status: event.part.state?.status,
title: event.part.state?.title || event.part.tool,
});
schedule();
}
},
async finish(text, sessionId, boxId) {
stopped = true;
pending = undefined;
if (timer) {
clearTimeout(timer);
timer = undefined;
}
if (inFlight) await inFlight;
const delay = Math.max(0, lastUpdate + updateInterval - Date.now());
if (delay) await sleep(delay);
await update(finalSlackText(text, sessionId, boxId));
},
};
}
async function handle(turn) {
console.log(
`${new Date().toISOString()} turn ${turn.turnId} by ${turn.author ?? 'unknown'}: ${turn.sessionId ? 'resume' : 'new'}`,
);
let result;
let activeSessionId = turn.sessionId ?? '';
const progress = await startSlackProgress(turn);
try {
const r = await runOpenCode(turn, progress.event, (sessionId) => (activeSessionId = sessionId));
result = {
turnId: turn.turnId,
status: 'done',
output: r.result,
sessionId: r.session_id,
boxId: BOX_ID,
};
} catch (error) {
result = {
turnId: turn.turnId,
status: 'error',
output: error.message,
sessionId: activeSessionId,
boxId: BOX_ID,
};
}
await progress.finish(result.output, result.sessionId, result.boxId);
// The resume POST is the delivery contract. Retry transient failures.
for (let attempt = 1; attempt <= 3; attempt++) {
try {
const res = await post(turn.resumeUrl, result);
if (res.ok) return;
console.error(`turn ${turn.turnId}: result POST got HTTP ${res.status} (attempt ${attempt})`);
} catch (error) {
console.error(
`turn ${turn.turnId}: result POST failed (attempt ${attempt}): ${error.message}`,
);
}
await sleep(2000 * attempt);
}
console.error(`turn ${turn.turnId}: result not delivered after 3 attempts`);
}
async function main() {
for (const [key, value] of Object.entries({
N8N_DEQUEUE_URL: DEQUEUE_URL,
AGENT_WORKER_TOKEN: TOKEN,
OPENROUTER_API_KEY: process.env.OPENROUTER_API_KEY,
GITHUB_USER,
})) {
if (!value) {
console.error(`Refusing to start: ${key} is not set.`);
process.exit(1);
}
}
if (!BOX_ID)
console.error(
'CODESPACE_NAME did not resolve — box pinning disabled; turns route by githubUser only.',
);
if (!SLACK_TOKEN)
console.error(
'SLACK_BOT_TOKEN is not set. Turns will complete without Slack progress updates.',
);
console.log(
`agent-worker polling as ${GITHUB_USER} every ${INITIAL_POLL_INTERVAL_MS}-${MAX_POLL_INTERVAL_MS}ms`,
);
let pollInterval = INITIAL_POLL_INTERVAL_MS;
for (;;) pollInterval = await pollOnce(pollInterval);
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) await main();

View File

@ -1,383 +0,0 @@
import assert from 'node:assert/strict';
import { EventEmitter } from 'node:events';
import { readFileSync } from 'node:fs';
import { PassThrough } from 'node:stream';
import { test } from 'node:test';
import { setTimeout as sleep } from 'node:timers/promises';
import {
openCodeConfig,
openCodeEnvironment,
pollOnce,
runOpenCode,
startSlackProgress,
} from './agent-worker.mjs';
const turn = {
turnId: 'turn-1',
slack: { channel: 'C123', thread_ts: '123.456' },
};
test('reloads Codespaces secrets before the worker starts', () => {
const postStart = readFileSync(new URL('./post-start.mjs', import.meta.url), 'utf8');
assert.match(
postStart,
/bash -lc "\. \/usr\/local\/lib\/codespaces-env\.sh; .*node \/workspaces\/n8n\/\.devcontainer\/codespaces\/agent-worker\.mjs/,
);
assert.match(postStart, /installAgentHarness\(\)/);
assert.match(postStart, /harness\.status === 'active' &&\s+tryRun\('worker start'/);
assert.match(postStart, /JSON\.stringify\(\{ installed, failed, harness, workerStarted \}/);
});
function slackRecorder() {
const calls = [];
return {
calls,
async call(method, body) {
calls.push({ method, body });
return method === 'chat.postMessage' ? { ok: true, ts: '456.789' } : { ok: true };
},
};
}
async function waitFor(check) {
for (let attempt = 0; attempt < 100; attempt++) {
if (check()) return;
await sleep(5);
}
throw new Error('Condition was not met');
}
function childProcess() {
const child = new EventEmitter();
child.pid = 123;
child.stdin = new PassThrough();
child.stdout = new PassThrough();
child.stderr = new PassThrough();
child.kill = () => {};
return child;
}
function completedChild(lines) {
const child = childProcess();
setImmediate(() => {
for (const line of lines) child.stdout.write(`${line}\n`);
child.stdout.end();
child.stderr.end();
child.emit('close', 0);
});
return child;
}
test('backs off idle polls and resets when work arrives', async () => {
let interval = 3000;
const waits = [];
const turns = [null, null, null, null, null, { turnId: '1' }, { turnId: '2' }];
const handled = [];
for (let attempt = 0; attempt < 5; attempt++) {
interval = await pollOnce(interval, {
dequeueTurn: async () => turns.shift(),
handleTurn: async (turn) => handled.push(turn.turnId),
wait: async (delay) => waits.push(delay),
});
}
assert.deepEqual(waits, [3000, 6000, 12_000, 24_000, 30_000]);
assert.equal(interval, 30_000);
interval = await pollOnce(interval, {
dequeueTurn: async () => turns.shift(),
handleTurn: async (turn) => handled.push(turn.turnId),
wait: async (delay) => waits.push(delay),
});
interval = await pollOnce(interval, {
dequeueTurn: async () => turns.shift(),
handleTurn: async (turn) => handled.push(turn.turnId),
wait: async (delay) => waits.push(delay),
});
assert.equal(interval, 3000);
assert.deepEqual(handled, ['1', '2']);
assert.equal(waits.length, 5);
});
test('backs off after a dequeue error', async () => {
const waits = [];
const errors = [];
const interval = await pollOnce(3000, {
dequeueTurn: async () => {
throw new Error('unavailable');
},
wait: async (delay) => waits.push(delay),
logError: (error) => errors.push(error),
});
assert.equal(interval, 6000);
assert.deepEqual(waits, [3000]);
assert.deepEqual(errors, ['poll error: unavailable']);
});
test('resets the interval before handling work', async () => {
const waits = [];
const errors = [];
const interval = await pollOnce(30_000, {
dequeueTurn: async () => ({ turnId: '1' }),
handleTurn: async () => {
throw new Error('failed');
},
wait: async (delay) => waits.push(delay),
logError: (error) => errors.push(error),
});
assert.equal(interval, 6000);
assert.deepEqual(waits, [3000]);
assert.deepEqual(errors, ['poll error: failed']);
});
test('streams tool progress but excludes reasoning', async () => {
const slack = slackRecorder();
const progress = await startSlackProgress(turn, {
callSlack: slack.call,
updateInterval: 50,
});
progress.event({
type: 'reasoning',
sessionID: 'ses_1',
part: { id: 'prt_reason', type: 'reasoning', text: 'private reasoning' },
});
progress.event({
type: 'tool_use',
sessionID: 'ses_1',
part: {
id: 'prt_tool',
type: 'tool',
tool: 'read',
state: { status: 'completed', title: 'Read worker source' },
},
});
await sleep(100);
const streamed = slack.calls.find((call) => call.method === 'chat.update')?.body.text;
assert.match(streamed, /Done: Read worker source/);
assert.doesNotMatch(streamed, /private reasoning/);
});
test('coalesces Slack updates within the configured interval', async () => {
const slack = slackRecorder();
const progress = await startSlackProgress(turn, {
callSlack: slack.call,
updateInterval: 50,
});
for (const [id, title] of ['One', 'Two', 'Three'].entries()) {
progress.event({
type: 'tool_use',
sessionID: 'ses_1',
part: {
id: `prt_${id}`,
type: 'tool',
tool: 'read',
state: { status: 'completed', title },
},
});
}
await sleep(100);
assert.equal(slack.calls.filter((call) => call.method === 'chat.update').length, 1);
progress.event({
type: 'tool_use',
sessionID: 'ses_1',
part: {
id: 'prt_4',
type: 'tool',
tool: 'read',
state: { status: 'completed', title: 'Four' },
},
});
assert.equal(slack.calls.filter((call) => call.method === 'chat.update').length, 1);
await sleep(100);
assert.equal(slack.calls.filter((call) => call.method === 'chat.update').length, 2);
});
test('replaces progress with the final answer', async () => {
const calls = [];
let releaseProgress;
const callSlack = async (method, body) => {
calls.push({ method, body });
if (method === 'chat.postMessage') return { ok: true, ts: '456.789' };
if (calls.filter((call) => call.method === 'chat.update').length === 1) {
return await new Promise((resolve) => (releaseProgress = () => resolve({ ok: true })));
}
return { ok: true };
};
const progress = await startSlackProgress(turn, {
callSlack,
updateInterval: 0,
});
progress.event({
type: 'tool_use',
sessionID: 'ses_1',
part: {
id: 'prt_tool',
type: 'tool',
tool: 'read',
state: { status: 'completed', title: 'Read worker source' },
},
});
await waitFor(() => calls.filter((call) => call.method === 'chat.update').length === 1);
for (const title of ['Second tool', 'Third tool', 'Fourth tool']) {
progress.event({
type: 'tool_use',
sessionID: 'ses_1',
part: {
id: title,
type: 'tool',
tool: 'read',
state: { status: 'completed', title },
},
});
}
const finish = progress.finish('Final answer', 'ses_1', 'box-1');
await sleep(10);
assert.equal(calls.filter((call) => call.method === 'chat.update').length, 1);
releaseProgress();
await finish;
assert.deepEqual(calls.at(-1), {
method: 'chat.update',
body: {
channel: 'C123',
ts: '456.789',
text: 'Final answer\n\n⟳session:ses_1 ⟳box:box-1',
},
});
});
test('streams OpenCode CLI events and resumes an OpenCode session', async () => {
let invocation;
let prompt = '';
const events = [];
const result = await runOpenCode(
{
message: 'Test message',
sessionId: 'ses_existing',
cwd: '/workspaces/n8n',
author: 'Tester',
},
(event) => events.push(event),
() => {},
{
spawnProcess(command, args, options) {
invocation = { command, args, options };
const child = completedChild([
JSON.stringify({ type: 'step_start', sessionID: 'ses_existing', part: {} }),
JSON.stringify({
type: 'tool_use',
sessionID: 'ses_existing',
part: { id: 'prt_1', type: 'tool', tool: 'read', state: { status: 'completed' } },
}),
JSON.stringify({
type: 'text',
sessionID: 'ses_existing',
part: { id: 'prt_2', type: 'text', text: 'Completed response' },
}),
]);
child.stdin.on('data', (chunk) => (prompt += chunk));
return child;
},
},
);
assert.equal(result.result, 'Completed response');
assert.equal(result.session_id, 'ses_existing');
assert.deepEqual(invocation.args, [
'run',
'--model',
'openrouter/openai/gpt-5.6-sol',
'--format',
'json',
'--auto',
'--session',
'ses_existing',
]);
assert.equal(invocation.options.detached, true);
assert.equal(prompt, 'Test message');
assert.deepEqual(
events.map((event) => event.type),
['step_start', 'tool_use', 'text'],
);
});
test('stops the OpenCode process group at the turn limit', async () => {
const signals = [];
let child;
const run = runOpenCode(
{ message: 'Test message', cwd: '/workspaces/n8n', author: 'Tester' },
() => {},
() => {},
{
spawnProcess() {
child = childProcess();
return child;
},
stopProcess(_child, signal) {
signals.push(signal);
if (signal === 'SIGTERM') setImmediate(() => child.emit('close', null));
},
killDelay: 5,
timeout: 1,
},
);
await assert.rejects(run, /The turn passed the 1-millisecond limit/);
assert.deepEqual(signals, ['SIGTERM']);
});
test('keeps broker credentials out of the OpenCode process', () => {
const environment = openCodeEnvironment({
AGENT_WORKER_TOKEN: 'worker',
N8N_DEQUEUE_URL: 'https://example.com',
SLACK_BOT_TOKEN: 'slack',
ANTHROPIC_API_KEY: 'model',
OPENROUTER_API_KEY: 'openrouter',
GITHUB_TOKEN: 'github',
});
assert.deepEqual(JSON.parse(environment.OPENCODE_CONFIG_CONTENT), {
provider: { openrouter: { options: { apiKey: '{env:OPENROUTER_API_KEY}' } } },
});
delete environment.OPENCODE_CONFIG_CONTENT;
assert.deepEqual(environment, {
ANTHROPIC_API_KEY: 'model',
OPENROUTER_API_KEY: 'openrouter',
GITHUB_TOKEN: 'github',
N8N_AGENT_RUNTIME: 'sandbox',
N8N_AGENT_PROFILE: 'slack',
});
});
test('configures Flaky MCP without serializing its token', () => {
const token = 'flaky-secret';
const environment = openCodeEnvironment({
FLAKY_MCP_URL: 'https://example.com/mcp',
FLAKY_MCP_TOKEN: token,
OPENROUTER_API_KEY: 'openrouter',
});
assert.deepEqual(JSON.parse(environment.OPENCODE_CONFIG_CONTENT), {
provider: { openrouter: { options: { apiKey: '{env:OPENROUTER_API_KEY}' } } },
mcp: {
flaky: {
type: 'remote',
url: 'https://example.com/mcp',
enabled: true,
oauth: false,
headers: { Authorization: 'Bearer {env:FLAKY_MCP_TOKEN}' },
},
},
});
assert.equal(environment.FLAKY_MCP_TOKEN, token);
assert.equal(environment.OPENCODE_CONFIG_CONTENT.includes(token), false);
});
test('omits Flaky MCP when either setting is missing', () => {
assert.equal(openCodeConfig({ FLAKY_MCP_URL: 'https://example.com/mcp' }).mcp, undefined);
assert.equal(openCodeConfig({ FLAKY_MCP_TOKEN: 'flaky-secret' }).mcp, undefined);
});

View File

@ -1,150 +0,0 @@
import assert from 'node:assert/strict';
import { spawnSync } from 'node:child_process';
import { chmodSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { after, before, test } from 'node:test';
import { fileURLToPath } from 'node:url';
const repoRoot = fileURLToPath(new URL('../../', import.meta.url));
const sessionScript = join(repoRoot, 'scripts/cloud-session.mjs');
let fixtureDir;
let invocation = 0;
before(() => {
fixtureDir = mkdtempSync(join(tmpdir(), 'cloud-session-'));
const mockGh = join(fixtureDir, 'gh');
writeFileSync(
mockGh,
`#!/usr/bin/env node
const { writeFileSync } = require('node:fs');
const args = process.argv.slice(2);
if (args[0] === 'codespace' && args[1] === 'list') {
process.stdout.write('[{"name":"test-codespace","state":"Available"}]\\n');
} else {
writeFileSync(process.env.GH_CAPTURE, JSON.stringify(args));
}
`,
);
chmodSync(mockGh, 0o755);
for (const [name, script] of Object.entries({
infocmp: '[ "$1" = "$SUPPORTED_TERM" ]',
tmux: 'printf "%s" "$TERM"',
})) {
const executable = join(fixtureDir, name);
writeFileSync(executable, `#!/bin/sh\n${script}\n`);
chmodSync(executable, 0o755);
}
});
after(() => rmSync(fixtureDir, { recursive: true, force: true }));
function runSession(args) {
const capture = join(fixtureDir, `invocation-${invocation++}.json`);
const result = spawnSync(process.execPath, [sessionScript, ...args], {
encoding: 'utf8',
env: {
...process.env,
GH_CAPTURE: capture,
PATH: `${fixtureDir}:${process.env.PATH}`,
},
});
assert.equal(result.status, 0, result.stderr || result.stdout);
return JSON.parse(readFileSync(capture, 'utf8'));
}
function remoteCommand(args) {
const invocationArgs = runSession(args);
assert.deepEqual(invocationArgs.slice(0, 6), [
'codespace',
'ssh',
'-c',
'test-codespace',
'--',
'-t',
]);
return invocationArgs[6];
}
for (const { name, supportedTerm, expectedTerm } of [
{
name: 'uses xterm-256color when the remote terminal definition is missing',
supportedTerm: 'xterm-256color',
expectedTerm: 'xterm-256color',
},
{
name: 'keeps the terminal type when the remote definition is available',
supportedTerm: 'xterm-ghostty',
expectedTerm: 'xterm-ghostty',
},
]) {
test(name, () => {
const result = spawnSync('/bin/sh', ['-c', remoteCommand(['--opencode', '--legacy'])], {
encoding: 'utf8',
env: {
...process.env,
PATH: `${fixtureDir}:${process.env.PATH}`,
TERM: 'xterm-ghostty',
SUPPORTED_TERM: supportedTerm,
},
});
assert.equal(result.status, 0, result.stderr);
assert.equal(result.stdout, expectedTerm);
});
}
test('starts a named OpenCode session in a worktree', () => {
const command = remoteCommand(['--opencode', 'fix-flaky', '--legacy', '--model', 'test']);
assert.match(command, /tmux new -As fix-flaky-opencode/);
assert.match(command, /unset AGENT_WORKER_TOKEN N8N_DEQUEUE_URL SLACK_BOT_TOKEN/);
assert.match(command, /OPENCODE_CONFIG_CONTENT/);
assert.match(command, /export N8N_AGENT_RUNTIME=sandbox; unset N8N_AGENT_PROFILE/);
assert.match(
command,
/git -C \/workspaces\/n8n fetch origin master; git -C \/workspaces\/n8n worktree add --no-track -b "session\/fix-flaky" "\/workspaces\/wt-fix-flaky" origin\/master/,
);
assert.match(command, /cd "\/workspaces\/wt-fix-flaky" && opencode --auto --model test/);
assert.doesNotMatch(command, /unset OPENROUTER_API_KEY/);
assert.doesNotMatch(command, /claude plugin/);
});
test('forwards legacy flags without a workspace name to the main checkout', () => {
for (const flags of [['--help'], ['--model', 'test']]) {
const command = remoteCommand(['--opencode', '--legacy', ...flags]);
assert.match(command, /tmux new -As agent-opencode/);
assert.ok(command.includes(`cd /workspaces/n8n && opencode --auto ${flags.join(' ')}`));
assert.doesNotMatch(command, /git .*worktree add/);
}
});
test('keeps removed credentials out of the login shell', () => {
const command = remoteCommand(['--shell']);
const profile = readFileSync(new URL('./codespaces-secrets.sh', import.meta.url), 'utf8');
const dockerfile = readFileSync(new URL('./Dockerfile', import.meta.url), 'utf8');
assert.match(command, /tmux new -As agent-shell/);
assert.match(
command,
/unset AGENT_WORKER_TOKEN N8N_DEQUEUE_URL SLACK_BOT_TOKEN; unset OPENROUTER_API_KEY/,
);
assert.match(command, /export N8N_SKIP_CODESPACE_SECRETS=1; exec "\$\{SHELL:-\/bin\/bash\}" -l/);
assert.ok(profile.includes('[ "${N8N_SKIP_CODESPACE_SECRETS:-}" = "1" ] && return'));
assert.ok(
dockerfile.includes(
'RUN echo \'[ "${N8N_SKIP_CODESPACE_SECRETS:-}" = "1" ] || . /usr/local/lib/codespaces-env.sh\'',
),
);
assert.doesNotMatch(command, /claude plugin|OPENCODE_CONFIG_CONTENT/);
});
test('keeps the existing Claude session behavior', () => {
const command = remoteCommand(['agent', '--model', 'test']);
assert.match(command, /tmux new -As agent/);
assert.match(command, /unset OPENROUTER_API_KEY/);
assert.match(command, /claude plugin marketplace add/);
assert.match(command, /cd \/workspaces\/n8n && claude --model test/);
assert.doesNotMatch(command, /OPENCODE_CONFIG_CONTENT|N8N_SKIP_CODESPACE_SECRETS/);
});

View File

@ -1,9 +0,0 @@
# Codespaces delivers secrets base64-encoded in a shared env file. It injects
# them into VS Code sessions only. Other shells must source this file to get
# them. Tokens rotate every few minutes: read at time of use, not at login.
if [ -f /workspaces/.codespaces/shared/.env-secrets ]; then
while IFS='=' read -r key val; do
case "$key" in '' | *[!A-Za-z0-9_]*) continue ;; esac
dec=$(printf %s "$val" | base64 -d 2>/dev/null) && export "$key=$dec"
done </workspaces/.codespaces/shared/.env-secrets
fi

View File

@ -1,21 +0,0 @@
# Interactive shell sessions set this guard after they remove worker credentials.
[ "${N8N_SKIP_CODESPACE_SECRETS:-}" = "1" ] && return
# Login-shell setup for ssh/tmux sessions: export secrets, then do the
# one-time registrations.
. /usr/local/lib/codespaces-env.sh
# Claude Code keeps its first-run state in ~/.claude.json. Write the defaults
# so a new codespace does not show the theme and trust prompts. The env
# secrets supply the login.
[ -f "$HOME/.claude.json" ] || printf '%s\n' \
'{"hasCompletedOnboarding":true,"theme":"dark","projects":{"/workspaces/n8n":{"hasTrustDialogAccepted":true}}}' \
>"$HOME/.claude.json"
# Register the credential helper in the user config, because Codespaces
# regenerates the managed /etc/gitconfig. When the env token is missing, the
# system helper exits 0 without output, and git falls through to ours.
if command -v git >/dev/null 2>&1 &&
! git config --global --get-all credential.helper 2>/dev/null | grep -qs gitcredential-refresh; then
git config --global --add credential.helper /usr/local/bin/gitcredential-refresh.sh
fi

View File

@ -1,68 +0,0 @@
{
"name": "n8n (Codespaces agent sessions)",
"dockerComposeFile": "docker-compose.yml",
"service": "n8n",
// Codespaces clones the repo into /workspaces/<repo>, unlike the laptop config
"workspaceFolder": "/workspaces/n8n",
"hostRequirements": {
"cpus": 2,
"memory": "8gb",
"storage": "32gb"
},
"features": {
"ghcr.io/devcontainers/features/github-cli:1": {},
// gh codespace ssh needs an SSH server inside the container
"ghcr.io/devcontainers/features/sshd:1": {},
// Docker for testcontainers and `pnpm --filter n8n-containers services`;
// needs `privileged: true` on the n8n service (compose-based config)
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
},
"forwardPorts": [8080, 5678],
"customizations": {
// Claude Code in VS Code or the browser editor. It runs in the codespace.
"vscode": {
"extensions": ["anthropic.claude-code"]
},
// Grant the Codespace token read access to the private skills and harness repositories.
// Each user authorizes this access once when they create the Codespace.
"codespaces": {
"repositories": {
"n8n-io/n8n-agent-skills": { "permissions": { "contents": "read" } },
"n8n-io/cat-bot": { "permissions": { "contents": "read" } }
}
}
},
// Runs in prebuilds, so a fresh codespace starts with deps installed and the build warm
"onCreateCommand": "corepack install && pnpm install && (pnpm build > /tmp/build.log 2>&1 || tail -n 50 /tmp/build.log) && pnpm --filter=n8n-playwright exec playwright install chromium",
// On each container start: install the private skills marketplace into Claude
// (best-effort) and start the poll worker. `bash -lc` loads the secrets shim,
// so the worker gets its token. Logs: /tmp/post-start.log, /tmp/agent-worker.log.
"postStartCommand": "bash -lc 'node /workspaces/n8n/.devcontainer/codespaces/post-start.mjs >> /tmp/post-start.log 2>&1' || true",
"secrets": {
"ANTHROPIC_API_KEY": {
"description": "Anthropic API key for Claude Code / OpenCode sessions"
},
"OPENROUTER_API_KEY": {
"description": "OpenRouter API key for Flaky's OpenCode sessions"
},
"AGENT_WORKER_TOKEN": {
"description": "The shared bearer token. The poll worker sends it to n8n on each dequeue. See .devcontainer/codespaces/agent-worker.mjs."
},
"N8N_DEQUEUE_URL": {
"description": "n8n webhook the poll worker calls to claim a pending turn"
},
"SLACK_BOT_TOKEN": {
"description": "Slack bot token for agent progress messages. It needs the chat:write scope."
},
"CLAUDE_CODE_OAUTH_TOKEN": {
"description": "Alternative to an API key: token from `claude setup-token` (Max subscription)"
},
// Both set at the repository level — team members get them automatically
"FLAKY_MCP_TOKEN": {
"description": "Bearer token for the Flaky (QA assistant) MCP endpoint"
},
"FLAKY_MCP_URL": {
"description": "URL of the Flaky MCP endpoint on the internal n8n instance"
}
}
}

View File

@ -1,34 +0,0 @@
volumes:
postgres-data:
services:
postgres:
image: postgres:18-alpine
restart: unless-stopped
volumes:
- postgres-data:/var/lib/postgresql
environment:
- POSTGRES_DB=n8n
- POSTGRES_PASSWORD=password
n8n:
build:
context: .
dockerfile: Dockerfile
# Required by the docker-in-docker devcontainer feature
privileged: true
volumes:
- ../..:/workspaces:cached
command: sleep infinity
depends_on:
- postgres
environment:
DB_POSTGRESDB_HOST: postgres
DB_TYPE: postgresdb
DB_POSTGRESDB_PASSWORD: password
# Prompt-cache TTL in seconds. 1 hour keeps the cache warm between turns.
ANTHROPIC_PROMPT_CACHE_TTL: '3600'
# Do not add CODESPACE_NAME here. Codespaces does not give it to compose, so
# this file can set an empty value only. An empty value is worse than no
# value, because tmux keeps it for each new pane. The consumers read the name
# with scripts/codespace-env.mjs.

View File

@ -1,36 +0,0 @@
// Fake executables for tests. Each shim is a Node script. It receives `args`, `root`,
// `file(name)` under the root, and `log(event)`, which appends to calls.jsonl in the root.
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const COMMON = `
const fs = require('node:fs');
const path = require('node:path');
const args = process.argv.slice(2);
const root = process.env.TEST_ROOT;
const file = (name) => path.join(root, name);
const log = (event) => fs.appendFileSync(file('calls.jsonl'), JSON.stringify(event) + '\\n');
`;
export function fakeBinaries(prefix) {
const root = mkdtempSync(join(tmpdir(), prefix));
const binDir = join(root, 'bin');
mkdirSync(binDir);
const logFile = join(root, 'calls.jsonl');
writeFileSync(logFile, '');
return {
root,
env: { PATH: `${binDir}:${process.env.PATH}`, TEST_ROOT: root },
bin: (name, body) =>
writeFileSync(join(binDir, name), `#!${process.execPath}\n${COMMON}\n${body}`, {
mode: 0o755,
}),
calls: () =>
readFileSync(logFile, 'utf8')
.trim()
.split('\n')
.filter(Boolean)
.map((line) => JSON.parse(line)),
};
}

View File

@ -1,7 +0,0 @@
#!/bin/sh
# gh reads its token from the env only. Long sessions hold a stale snapshot of
# the rotated token. Read the current token on each call, then run the real gh
# (the github-cli feature installs it at /usr/bin/gh).
. /usr/local/lib/codespaces-env.sh
[ -n "$GITHUB_TOKEN" ] && export GH_TOKEN="$GITHUB_TOKEN"
exec /usr/bin/gh "$@"

View File

@ -1,6 +0,0 @@
#!/bin/sh
# The stock helper reads $GITHUB_TOKEN from the env. Non-login shells do not
# have it, and the token rotates. Read the current token, then delegate.
[ "$1" = get ] || exit 0
. /usr/local/lib/codespaces-env.sh
exec /.codespaces/bin/gitcredential_github.sh "$@"

View File

@ -1,224 +0,0 @@
import assert from 'node:assert/strict';
import { spawn } from 'node:child_process';
import { rmSync, readFileSync } from 'node:fs';
import { test } from 'node:test';
import { setTimeout as delay } from 'node:timers/promises';
import { fileURLToPath } from 'node:url';
import { fakeBinaries } from './fake-bin.mjs';
import { parseOpenCodeArgs } from '../../scripts/cloud-session-opencode.mjs';
const script = fileURLToPath(new URL('../../scripts/cloud-session.mjs', import.meta.url));
const secret = 'a'.repeat(64);
function fixture(t, env = {}) {
const { root, bin, calls, env: binEnv } = fakeBinaries('opencode-client-');
t.after(() => rmSync(root, { recursive: true, force: true }));
bin(
'gh',
`
log({ command: 'gh', args });
if (args[0] === 'codespace' && args[1] === 'list') {
console.log(JSON.stringify([
{ name: 'dev-box', state: 'Available' },
]));
} else if (args.includes('-N')) {
if (process.env.TEST_TUNNEL_FAIL) process.exit(1);
const mapping = args[args.indexOf('-L') + 1];
const port = +mapping.split(':')[1];
const ssh = require('node:child_process').spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { stdio: 'ignore' });
log({ event: 'ssh-child', pid: ssh.pid });
const server = require('node:http').createServer((req, res) => {
const expected = 'Basic ' + Buffer.from('opencode:' + '${secret}').toString('base64');
if (process.env.TEST_UNAUTHORIZED || req.headers.authorization !== expected) { res.writeHead(401).end(); return; }
res.setHeader('content-type', 'application/json');
res.end(JSON.stringify({ healthy: true, version: process.env.TEST_SERVER_VERSION || '1.2.3' }));
});
server.listen(port, '127.0.0.1');
if (process.env.TEST_TUNNEL_DROP) setTimeout(() => process.exit(1), 500);
process.on('SIGTERM', () => { log({ event: 'tunnel-stopped' }); server.closeAllConnections(); server.close(() => process.exit(0)); });
} else if (args.at(-1).includes('node --input-type=module')) {
let source = '';
process.stdin.on('data', chunk => source += chunk);
process.stdin.on('end', () => {
log({ event: 'bootstrap', hasSource: source.includes('prepareOpenCode') });
if (process.env.TEST_BOOTSTRAP_FAIL) process.exit(1);
const parts = args.at(-1).split(' ');
const name = parts.at(-3);
const fresh = parts.at(-2) === 'true';
const web = parts.at(-1) === 'true';
console.log(JSON.stringify({ port: 4242, password: '${secret}',
...(fresh && !web ? {} : { sessionID: 'ses_saved' }),
directory: name === 'agent' ? '/workspaces/n8n' : '/workspaces/wt-' + name }));
});
}
`,
);
bin(
'opencode',
`
if (args[0] === '--version') {
if (process.env.TEST_NO_CLIENT) process.exit(1);
console.log(process.env.TEST_CLIENT_VERSION || '1.2.3');
} else {
log({ command: 'opencode', args, password: process.env.OPENCODE_SERVER_PASSWORD });
if (process.env.TEST_CLIENT_WAIT) setInterval(() => {}, 1000);
}
`,
);
for (const opener of ['open', 'xdg-open']) bin(opener, `log({ command: 'browser', args });`);
return {
calls,
start(args) {
const child = spawn(process.execPath, [script, '--opencode', ...args], {
env: { ...process.env, ...binEnv, ...env },
stdio: ['ignore', 'pipe', 'pipe'],
});
let output = '';
child.stdout.on('data', (chunk) => {
output += chunk;
});
child.stderr.on('data', (chunk) => {
output += chunk;
});
const done = new Promise((resolve) => child.on('exit', (code) => resolve({ code, output })));
t.after(() => child.kill('SIGKILL'));
return { child, done };
},
};
}
async function waitFor(check) {
for (let attempt = 0; attempt < 100; attempt++) {
const result = check();
if (result) return result;
await delay(50);
}
throw new Error('Fixture did not become ready.');
}
// An orphaned fixture process can linger as a zombie where PID 1 does not reap
// it, and kill(pid, 0) still succeeds for one. Read the process state where the
// kernel exposes it; without /proc only the signal error can tell.
function stopped(pid) {
try {
process.kill(pid, 0);
} catch {
return true;
}
try {
return readFileSync(`/proc/${pid}/stat`, 'utf8').split(' ')[2] === 'Z';
} catch {
return false;
}
}
test('parses options before or after the workspace and rejects unsupported flags', () => {
assert.equal(parseOpenCodeArgs(['--web']).port, 4096);
assert.equal(parseOpenCodeArgs([]).port, 0);
assert.equal(parseOpenCodeArgs(['--port', '4100', '--web']).port, 4100);
assert.deepEqual(parseOpenCodeArgs(['--web', 'fix-flaky', '--new', '--port', '4100']), {
name: 'fix-flaky',
web: true,
fresh: true,
port: 4100,
});
for (const args of [
['--port'],
['--port', '0'],
['--port', '65536'],
['--model', 'test'],
['a/b'],
['one', 'two'],
]) {
assert.throws(() => parseOpenCodeArgs(args));
}
});
test(
'attaches the local TUI to the selected workspace and closes the whole tunnel',
{ timeout: 15000 },
async (t) => {
const f = fixture(t);
const result = await f.start(['fix-flaky']).done;
assert.equal(result.code, 0, result.output);
const calls = f.calls();
assert.ok(calls.find((call) => call.event === 'bootstrap').hasSource);
assert.ok(
calls
.filter((call) => call.command === 'gh' && call.args[1] === 'ssh')
.every((call) => call.args[3] === 'dev-box'),
);
const tunnel = calls.find((call) => call.command === 'gh' && call.args.includes('-N'));
assert.match(tunnel.args.at(-1), /^127\.0\.0\.1:\d+:127\.0\.0\.1:4242$/);
const client = calls.find((call) => call.command === 'opencode');
assert.deepEqual(client.args.slice(2), [
'--dir',
'/workspaces/wt-fix-flaky',
'--session',
'ses_saved',
]);
assert.equal(client.password, secret);
assert.ok(!result.output.includes(secret));
assert.ok(calls.some((call) => call.event === 'tunnel-stopped'));
const pid = calls.find((call) => call.event === 'ssh-child').pid;
await waitFor(() => stopped(pid));
},
);
test('starts a new conversation for --new instead of continuing', { timeout: 15000 }, async (t) => {
const f = fixture(t);
const result = await f.start(['fix-flaky', '--new']).done;
assert.equal(result.code, 0, result.output);
const client = f.calls().find((call) => call.command === 'opencode');
assert.deepEqual(client.args.slice(2), ['--dir', '/workspaces/wt-fix-flaky']);
});
for (const signal of ['SIGINT', 'SIGHUP']) {
test(
`opens the newest web conversation and cleans up on ${signal}`,
{ timeout: 15000 },
async (t) => {
const f = fixture(t, { TEST_NO_CLIENT: '1' });
const run = f.start(['--web', 'fix-flaky']);
const browser = await waitFor(() => f.calls().find((call) => call.command === 'browser'));
const url = new URL(browser.args[0]);
assert.equal(
url.pathname,
`/${Buffer.from('/workspaces/wt-fix-flaky').toString('base64url')}/session/ses_saved`,
);
assert.equal(url.password, '');
const response = await fetch(`${url.origin}/global/health`);
assert.equal(response.status, 200);
await response.text();
run.child.kill(signal);
const result = await run.done;
assert.equal(result.code, 0, result.output);
assert.ok(f.calls().some((call) => call.event === 'tunnel-stopped'));
await assert.rejects(fetch(url.origin));
},
);
}
for (const [label, env, message] of [
[
'a version mismatch',
{ TEST_SERVER_VERSION: '1.14.22' },
/versions differ.*npm install -g opencode-ai@1\.14\.22/,
],
['an authentication failure', { TEST_UNAUTHORIZED: '1' }, /health check failed \(401\)/],
['a tunnel startup failure', { TEST_TUNNEL_FAIL: '1' }, /SSH tunnel closed/],
['a remote preparation failure', { TEST_BOOTSTRAP_FAIL: '1' }, /Could not prepare/],
['a missing local client', { TEST_NO_CLIENT: '1' }, /Install OpenCode locally/],
['a connection loss', { TEST_CLIENT_WAIT: '1', TEST_TUNNEL_DROP: '1' }, /SSH connection closed/],
]) {
test(`reports ${label} without printing credentials`, { timeout: 15000 }, async (t) => {
const f = fixture(t, env);
const result = await f.start([]).done;
assert.equal(result.code, 1, result.output);
assert.match(result.output, message);
assert.ok(!result.output.includes(secret));
if (label !== 'a connection loss')
assert.ok(!f.calls().some((call) => call.command === 'opencode'));
});
}

View File

@ -1,147 +0,0 @@
import assert from 'node:assert/strict';
import { createServer, request } from 'node:http';
import { connect } from 'node:net';
import { test } from 'node:test';
import { openCodeProxy } from '../../scripts/cloud-session-opencode-proxy.mjs';
test('proxies uploads and event streams with authentication and rejects other origins', async (t) => {
const requests = [];
const backend = createServer(async (req, res) => {
requests.push(req.headers);
res.writeHead(200, { 'content-type': 'text/event-stream' });
for await (const chunk of req) res.write(chunk);
res.end();
});
await new Promise((resolve) => backend.listen(0, '127.0.0.1', resolve));
t.after(() => {
backend.closeAllConnections();
backend.close();
});
const proxy = await openCodeProxy({ password: 'test-secret' });
proxy.targetPort = backend.address().port;
t.after(proxy.close);
const response = await fetch(`${proxy.origin}/session`, {
method: 'POST',
body: 'data: attached image\n\n',
headers: { origin: proxy.origin },
});
assert.equal(response.status, 200);
assert.equal(await response.text(), 'data: attached image\n\n');
assert.equal(
requests[0].authorization,
`Basic ${Buffer.from('opencode:test-secret').toString('base64')}`,
);
assert.ok(!proxy.origin.includes('test-secret'));
for (const headers of [
{ origin: 'https://example.com' },
{ host: 'example.com' },
{ 'sec-fetch-site': 'cross-site' },
]) {
const status = await new Promise((resolve, reject) => {
const req = request(proxy.origin, { headers }, (response) => {
response.resume();
resolve(response.statusCode);
});
req.on('error', reject);
req.end();
});
assert.equal(status, 403);
}
assert.equal(requests.length, 1);
});
test('forwards WebSocket upgrades and closes their sockets', async (t) => {
const peers = new Set();
const backend = createServer();
backend.on('upgrade', (req, socket) => {
assert.equal(
req.headers.authorization,
`Basic ${Buffer.from('opencode:test-secret').toString('base64')}`,
);
peers.add(socket);
socket.on('close', () => peers.delete(socket));
socket.write(
'HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n',
);
socket.pipe(socket);
});
await new Promise((resolve) => backend.listen(0, '127.0.0.1', resolve));
t.after(() => {
for (const peer of peers) peer.destroy();
backend.close();
});
const proxy = await openCodeProxy({ password: 'test-secret' });
proxy.targetPort = backend.address().port;
t.after(proxy.close);
const socket = await new Promise((resolve, reject) => {
const req = request(`${proxy.origin}/pty/test/connect`, {
headers: { origin: proxy.origin, connection: 'Upgrade', upgrade: 'websocket' },
});
req.on('upgrade', (_response, peer) => resolve(peer));
req.on('error', reject);
req.end();
});
t.after(() => socket.destroy());
const echoed = new Promise((resolve) => socket.once('data', (data) => resolve(data.toString())));
socket.write('terminal input');
assert.equal(await echoed, 'terminal input');
const closed = new Promise((resolve) => socket.once('close', resolve));
proxy.close();
await closed;
});
test(
'closes the browser response when the upstream stream disconnects',
{ timeout: 5000 },
async (t) => {
const backend = createServer((_req, res) => {
res.writeHead(200, { 'content-type': 'text/event-stream' });
res.write('data: partial\n\n');
setTimeout(() => res.destroy(), 50);
});
await new Promise((resolve) => backend.listen(0, '127.0.0.1', resolve));
t.after(() => {
backend.closeAllConnections();
backend.close();
});
const proxy = await openCodeProxy({ password: 'test' });
proxy.targetPort = backend.address().port;
t.after(proxy.close);
const signal = AbortSignal.timeout(3000);
const response = await fetch(proxy.origin, { signal });
await assert.rejects(response.text());
assert.equal(
signal.aborted,
false,
'The proxy must close the response before the client times out.',
);
},
);
test('rejects an occupied browser port and reports a missing or closed upstream', async (t) => {
const proxy = await openCodeProxy({ password: 'test' });
t.after(proxy.close);
await assert.rejects(openCodeProxy({ password: 'test', port: +new URL(proxy.origin).port }), {
code: 'EADDRINUSE',
});
const waiting = await fetch(proxy.origin);
assert.equal(waiting.status, 503);
await waiting.text();
proxy.targetPort = 1;
const response = await fetch(proxy.origin);
assert.equal(response.status, 502);
await response.text();
const rejection = await new Promise((resolve) => {
const socket = connect(+new URL(proxy.origin).port, '127.0.0.1', () => {
socket.write(
'GET /pty/test/connect HTTP/1.1\r\nHost: example.com\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n',
);
});
socket.once('data', (data) => {
socket.destroy();
resolve(data.toString());
});
});
assert.match(rejection, /403 Forbidden/);
});

View File

@ -1,228 +0,0 @@
#!/usr/bin/env node
// The laptop sends this file over SSH as one standalone module. Do not import sibling files.
// It also works in an older Codespace checkout.
import { execFileSync, spawnSync } from 'node:child_process';
import { randomBytes } from 'node:crypto';
import { existsSync, mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs';
import { createServer } from 'node:net';
import { join } from 'node:path';
import { setTimeout as delay } from 'node:timers/promises';
const quote = (value) => `'${value.replaceAll("'", "'\\''")}'`;
function readJson(file) {
try {
return JSON.parse(readFileSync(file, 'utf8'));
} catch (error) {
if (error.code === 'ENOENT') return undefined;
throw new Error(`Cannot read ${file}. Restore or remove this state file, then retry.`);
}
}
function saveJson(file, value) {
writeFileSync(`${file}.tmp`, JSON.stringify(value), { mode: 0o600 });
renameSync(`${file}.tmp`, file);
}
function run(command, args, options = {}) {
const result = spawnSync(command, args, {
stdio: ['ignore', 'inherit', 'inherit'],
...options,
});
if (result.error || result.status !== 0) {
throw new Error(`${command} failed. Check the output above, then retry.`);
}
}
async function freePort() {
const server = createServer();
await new Promise((resolve, reject) => {
server.once('error', reject);
server.listen(0, '127.0.0.1', resolve);
});
const { port } = server.address();
await new Promise((resolve) => server.close(resolve));
return port;
}
async function request(server, path, directory, options = {}, timeout = 60_000) {
return await fetch(`http://127.0.0.1:${server.port}${path}`, {
...options,
headers: {
authorization: `Basic ${Buffer.from(`opencode:${server.password}`).toString('base64')}`,
'x-opencode-directory': encodeURIComponent(directory),
'content-type': 'application/json',
},
signal: AbortSignal.timeout(timeout),
});
}
async function health(server, directory) {
try {
const response = await request(server, '/global/health', directory, {}, 3000);
if (!response.ok) return undefined;
const result = await response.json();
return result.healthy === true && typeof result.version === 'string' ? result : undefined;
} catch {
return undefined;
}
}
const serverFile = (stateDir) => join(stateDir, 'server.json');
async function runningServer({ stateDir, mainDirectory }) {
const previous = readJson(serverFile(stateDir));
return previous && (await health(previous, mainDirectory)) ? previous : undefined;
}
async function startServer({ stateDir, mainDirectory, workspaces }) {
const tmuxSession = 'n8n-opencode-server';
if (spawnSync('tmux', ['has-session', '-t', `=${tmuxSession}`]).status === 0) {
throw new Error(
`OpenCode is still starting or is unhealthy. Check ${stateDir}/server.log, then retry. The running server was not stopped.`,
);
}
const server = { port: await freePort(), password: randomBytes(32).toString('hex') };
const launcher = join(stateDir, 'serve.sh');
// Read secrets when the server starts. Do not store provider keys in the launcher.
writeFileSync(
launcher,
[
'#!/bin/bash',
'. /usr/local/lib/codespaces-env.sh 2>/dev/null || true',
'unset AGENT_WORKER_TOKEN N8N_DEQUEUE_URL SLACK_BOT_TOKEN',
`export TURBO_CACHE_DIR=${quote(join(workspaces, '.turbo-cache'))}`,
`[ -d "$TURBO_CACHE_DIR" ] || cp -r ${quote(join(mainDirectory, '.turbo/cache'))} "$TURBO_CACHE_DIR" 2>/dev/null || mkdir -p "$TURBO_CACHE_DIR"`,
`export OPENCODE_SERVER_USERNAME=opencode OPENCODE_SERVER_PASSWORD=${quote(server.password)}`,
'export OPENCODE_EXPERIMENTAL_CODE_MODE=true',
`export OPENCODE_CONFIG_CONTENT=${quote(
JSON.stringify({
enabled_providers: ['openrouter'],
provider: { openrouter: { options: { apiKey: '{env:OPENROUTER_API_KEY}' } } },
}),
)}`,
// Interactive sessions use the harness sandbox runtime without a profile.
'export N8N_AGENT_RUNTIME=sandbox; unset N8N_AGENT_PROFILE',
`cd ${quote(mainDirectory)} || exit 1`,
`exec opencode serve --hostname 127.0.0.1 --port ${server.port} >> ${quote(join(stateDir, 'server.log'))} 2>&1`,
'',
].join('\n'),
{ mode: 0o600 },
);
saveJson(serverFile(stateDir), server);
run('tmux', ['new-session', '-d', '-s', tmuxSession, `bash ${quote(launcher)}`]);
console.error('Starting the OpenCode server…');
for (let attempt = 0; attempt < 60; attempt++) {
if (await health(server, mainDirectory)) return server;
await delay(500);
}
throw new Error(`OpenCode did not become ready. Check ${stateDir}/server.log.`);
}
function prepareWorkspace({ name, directory, mainDirectory, stateDir }) {
if (!existsSync(directory)) {
console.error(`Creating worktree ${directory}…`);
const branch = `session/${name}`;
const exists =
spawnSync('git', [
'-C',
mainDirectory,
'show-ref',
'--verify',
'--quiet',
`refs/heads/${branch}`,
]).status === 0;
if (!exists)
spawnSync('git', ['-C', mainDirectory, 'fetch', 'origin', 'master'], {
stdio: ['ignore', 2, 2],
});
run(
'git',
[
'-C',
mainDirectory,
'worktree',
'add',
...(exists ? [] : ['--no-track', '-b', branch]),
directory,
exists ? branch : 'origin/master',
],
{ stdio: ['ignore', 2, 2] },
);
}
// A failed install must be retried even if it left a node_modules directory.
const installed = join(stateDir, `${name}.installed`);
const installing = join(stateDir, `${name}.installing`);
if (
!existsSync(join(directory, 'node_modules')) ||
existsSync(installing) ||
(name !== 'agent' && !existsSync(installed))
) {
console.error(`Installing dependencies in ${directory}…`);
writeFileSync(installing, '', { mode: 0o600 });
run('pnpm', ['install'], { cwd: directory, stdio: ['ignore', 2, 2] });
writeFileSync(installed, '', { mode: 0o600 });
rmSync(installing);
}
}
// OpenCode's own resume is scoped to the repository project, so all worktrees
// share one pool. Scope the selection to the workspace directory instead, so
// each worktree resumes its own most recently updated root conversation.
async function resolveSession({ name, fresh, directory, server }) {
if (!fresh) {
const query = new URLSearchParams({ directory, roots: 'true' });
const response = await request(server, `/session?${query}`, directory);
if (!response.ok) throw new Error(`Cannot list OpenCode sessions (${response.status}).`);
const sessions = await response.json();
const latest = Array.isArray(sessions)
? sessions.find((entry) => typeof entry?.id === 'string' && entry.id.length > 0)
: undefined;
if (latest) return latest.id;
}
const response = await request(server, '/session', directory, {
method: 'POST',
body: JSON.stringify({ title: `n8n: ${name}` }),
});
if (!response.ok) throw new Error(`Cannot create OpenCode session (${response.status}).`);
const session = await response.json();
if (typeof session.id !== 'string' || session.directory !== directory)
throw new Error('OpenCode returned an invalid session.');
return session.id;
}
export async function prepareOpenCode({
name = 'agent',
fresh = false,
web = false,
workspaces = '/workspaces',
} = {}) {
const stateDir = join(workspaces, '.n8n-opencode');
const mainDirectory = join(workspaces, 'n8n');
if (!/^\w[\w-]*$/.test(name)) throw new Error('Invalid OpenCode workspace name.');
mkdirSync(stateDir, { recursive: true, mode: 0o700 });
const directory = name === 'agent' ? mainDirectory : join(workspaces, `wt-${name}`);
let server = await runningServer({ stateDir, mainDirectory });
// On a cold start, fail before the worktree install when OpenCode is missing on the Codespace.
if (!server) execFileSync('opencode', ['--version'], { stdio: ['ignore', 'pipe', 'inherit'] });
prepareWorkspace({ name, directory, mainDirectory, stateDir });
server ??= await startServer({ stateDir, mainDirectory, workspaces });
const state = { ...server, directory };
// A fresh TUI conversation needs no record. The TUI creates it on the first message.
if (fresh && !web) return state;
// Only the parent process reads stdout. Never send this record to terminal output.
return { ...state, sessionID: await resolveSession({ name, fresh, directory, server }) };
}
if (process.argv[1] === '-') {
try {
const [name, fresh, web] = process.argv.slice(2);
console.log(
JSON.stringify(await prepareOpenCode({ name, fresh: fresh === 'true', web: web === 'true' })),
);
} catch (error) {
console.error(error.message);
process.exitCode = 1;
}
}

View File

@ -1,275 +0,0 @@
import assert from 'node:assert/strict';
import { mkdirSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { test } from 'node:test';
import { setTimeout as delay } from 'node:timers/promises';
import { fakeBinaries } from './fake-bin.mjs';
import { prepareOpenCode } from './opencode-server.mjs';
// An orphaned fixture process can linger as a zombie where PID 1 does not reap
// it, and kill(pid, 0) still succeeds for one. Read the process state where the
// kernel exposes it; without /proc only the signal error can tell.
function stopped(pid) {
try {
process.kill(pid, 0);
} catch {
return true;
}
try {
return readFileSync(`/proc/${pid}/stat`, 'utf8').split(' ')[2] === 'Z';
} catch {
return false;
}
}
function fixture(t) {
const fake = fakeBinaries('opencode-server-');
const dir = fake.root;
mkdirSync(join(dir, 'n8n', 'node_modules'), { recursive: true });
const savedEnv = { ...process.env };
// prepareOpenCode spawns the shims in this process. Log every call before the shim body runs.
Object.assign(process.env, fake.env, {
AGENT_WORKER_TOKEN: 'test-worker',
N8N_DEQUEUE_URL: 'test-queue',
SLACK_BOT_TOKEN: 'test-slack',
});
const bin = (name, body) =>
fake.bin(name, `log({ command: ${JSON.stringify(name)}, args });\n${body}`);
bin(
'git',
`
if (args.includes('show-ref')) process.exit(fs.existsSync(file('branch-exists')) ? 0 : 1);
if (args.includes('fetch')) process.exit(0);
fs.mkdirSync(args.at(-2), { recursive: true });
`,
);
bin(
'pnpm',
`
fs.mkdirSync(path.join(process.cwd(), 'node_modules'), {recursive:true});
if (process.env.TEST_INSTALL_FAIL) process.exit(1);
`,
);
bin(
'tmux',
`
if (args[0] === 'has-session') {
const pid = +fs.readFileSync(file('pid'), 'utf8');
let gone = false;
try { process.kill(pid, 0); } catch { gone = true; }
if (!gone) {
try { gone = fs.readFileSync('/proc/' + pid + '/stat', 'utf8').split(' ')[2] === 'Z'; } catch {}
}
if (gone) process.exit(1);
} else {
const child = require('node:child_process').spawn('bash', ['-c', args.at(-1)], { detached:true, stdio:'ignore' });
fs.writeFileSync(file('pid'), String(child.pid)); child.unref();
}
`,
);
bin(
'opencode',
`
if (args[0] === '--version') { console.log('1.2.3'); process.exit(0); }
fs.writeFileSync(file('server-env.json'), JSON.stringify({
worker: !!process.env.AGENT_WORKER_TOKEN, queue: !!process.env.N8N_DEQUEUE_URL, slack: !!process.env.SLACK_BOT_TOKEN,
cache: process.env.TURBO_CACHE_DIR, config: JSON.parse(process.env.OPENCODE_CONFIG_CONTENT),
runtime: process.env.N8N_AGENT_RUNTIME, profile: process.env.N8N_AGENT_PROFILE ?? null,
}));
let sessions = fs.existsSync(file('sessions.json')) ? JSON.parse(fs.readFileSync(file('sessions.json'), 'utf8')) : {};
const server = require('node:http').createServer(async (req, res) => {
res.setHeader('content-type', 'application/json');
const expected = 'Basic ' + Buffer.from('opencode:' + process.env.OPENCODE_SERVER_PASSWORD).toString('base64');
if (req.headers.authorization !== expected) { res.writeHead(401).end('{}'); return; }
const directory = decodeURIComponent(req.headers['x-opencode-directory']);
if (req.url === '/global/health') { res.end(JSON.stringify({ healthy:true, version:'1.2.3' })); return; }
if (req.method === 'POST' && req.url === '/session') {
if (fs.existsSync(file('reject-create'))) { res.writeHead(503).end('{}'); return; }
let body = '';
for await (const chunk of req) body += chunk;
const id = 'ses_' + (Object.keys(sessions).length + 1);
sessions[id] = { id, directory, title: JSON.parse(body).title };
fs.writeFileSync(file('sessions.json'), JSON.stringify(sessions));
res.end(JSON.stringify(sessions[id])); return;
}
if (fs.existsSync(file('reject-session'))) { res.writeHead(503).end('{}'); return; }
const url = new URL(req.url, 'http://localhost');
if (url.pathname === '/session') {
// Newest first, like the real server. Filter on the query, not the header,
// so the test fails when the launcher omits either parameter. The
// child-session file models a subagent conversation that the roots query
// flag must exclude.
const directory = url.searchParams.get('directory');
const roots = url.searchParams.get('roots') === 'true';
const stored = Object.values(sessions);
if (fs.existsSync(file('child-session')))
stored.push({ id: 'ses_child', directory, title: 'child', parentID: 'ses_1' });
const list = stored
.filter(s => s.directory === directory && (!roots || !s.parentID))
.reverse();
if (fs.existsSync(file('extra-session'))) list.unshift({ id: 'ses_extra', directory, title: 'TUI session' });
res.end(JSON.stringify(list)); return;
}
const session = sessions[req.url.split('/').at(-1)];
if (!session) { res.writeHead(404).end('{}'); return; }
res.end(JSON.stringify(session));
});
server.listen(+args[args.indexOf('--port') + 1], '127.0.0.1');
`,
);
const stop = async () => {
try {
const pid = +readFileSync(join(dir, 'pid'), 'utf8');
process.kill(-pid, 'SIGTERM');
for (let count = 0; count < 100; count++) {
if (stopped(pid)) return;
await delay(20);
}
throw new Error('Fixture server did not stop.');
} catch (error) {
if (!['ENOENT', 'ESRCH'].includes(error.code)) throw error;
}
};
t.after(async () => {
await stop();
for (const key of Object.keys(process.env)) if (!(key in savedEnv)) delete process.env[key];
Object.assign(process.env, savedEnv);
rmSync(dir, { recursive: true, force: true });
});
return {
dir,
stop,
prepare: (options = {}) => prepareOpenCode({ workspaces: dir, ...options }),
commands: fake.calls,
};
}
test(
'reuses the server, serves each workspace its newest conversation, and resumes after restart',
{ timeout: 15000 },
async (t) => {
const f = fixture(t);
const first = await f.prepare({ name: 'fix-flaky' });
assert.equal(first.directory, join(f.dir, 'wt-fix-flaky'));
const git = f.commands().filter((entry) => entry.command === 'git');
assert.deepEqual(
git.map((entry) => entry.args.slice(2)),
[
['show-ref', '--verify', '--quiet', 'refs/heads/session/fix-flaky'],
['fetch', 'origin', 'master'],
[
'worktree',
'add',
'--no-track',
'-b',
'session/fix-flaky',
first.directory,
'origin/master',
],
],
);
assert.equal(first.sessionID, 'ses_1');
assert.deepEqual(await f.prepare({ name: 'fix-flaky' }), first);
const second = await f.prepare({ name: 'another-task' });
assert.equal(second.port, first.port);
assert.equal(second.sessionID, 'ses_2');
const webbed = await f.prepare({ name: 'fix-flaky', web: true });
assert.equal(webbed.sessionID, 'ses_1');
const created = JSON.parse(readFileSync(join(f.dir, 'sessions.json'), 'utf8'));
assert.equal(created.ses_1.title, 'n8n: fix-flaky');
assert.equal(created.ses_1.directory, join(f.dir, 'wt-fix-flaky'));
assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, 'ses_1');
assert.equal((await f.prepare({ name: 'another-task', web: true })).sessionID, 'ses_2');
writeFileSync(join(f.dir, 'extra-session'), '');
assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, 'ses_extra');
rmSync(join(f.dir, 'extra-session'));
writeFileSync(join(f.dir, 'child-session'), '');
assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, 'ses_1');
rmSync(join(f.dir, 'child-session'));
const fresh = await f.prepare({ name: 'fix-flaky', web: true, fresh: true });
assert.notEqual(fresh.sessionID, 'ses_1');
assert.equal((await f.prepare({ name: 'fix-flaky', web: true })).sessionID, fresh.sessionID);
const env = JSON.parse(readFileSync(join(f.dir, 'server-env.json'), 'utf8'));
assert.deepEqual([env.worker, env.queue, env.slack], [false, false, false]);
assert.equal(env.cache, join(f.dir, '.turbo-cache'));
assert.equal(env.config.provider.openrouter.options.apiKey, '{env:OPENROUTER_API_KEY}');
assert.deepEqual(env.config.enabled_providers, ['openrouter']);
assert.deepEqual([env.runtime, env.profile], ['sandbox', null]);
assert.equal(statSync(join(f.dir, '.n8n-opencode')).mode & 0o777, 0o700);
for (const file of ['serve.sh', 'server.json']) {
assert.equal(statSync(join(f.dir, '.n8n-opencode', file)).mode & 0o777, 0o600);
}
assert.ok(!f.commands().some((entry) => JSON.stringify(entry.args).includes(first.password)));
assert.equal(f.commands().filter((entry) => entry.command === 'pnpm').length, 2);
await f.stop();
const restarted = await f.prepare({ name: 'fix-flaky', web: true });
assert.equal(restarted.sessionID, fresh.sessionID);
assert.notEqual(restarted.password, first.password);
rmSync(first.directory, { recursive: true });
writeFileSync(join(f.dir, 'branch-exists'), '');
assert.equal((await f.prepare({ name: 'fix-flaky' })).directory, join(f.dir, 'wt-fix-flaky'));
const worktree = f
.commands()
.filter((entry) => entry.command === 'git' && entry.args.includes('add'))
.at(-1);
assert.deepEqual(worktree.args, [
'-C',
join(f.dir, 'n8n'),
'worktree',
'add',
first.directory,
'session/fix-flaky',
]);
},
);
test(
'retries a failed worktree install before starting the server',
{ timeout: 10000 },
async (t) => {
const f = fixture(t);
process.env.TEST_INSTALL_FAIL = '1';
await assert.rejects(f.prepare({ name: 'retry' }), /pnpm failed/);
assert.ok(!f.commands().some((entry) => entry.command === 'tmux'));
delete process.env.TEST_INSTALL_FAIL;
assert.equal((await f.prepare({ name: 'retry' })).directory, join(f.dir, 'wt-retry'));
assert.equal(f.commands().filter((entry) => entry.command === 'pnpm').length, 2);
},
);
test(
'retries a failed main checkout install even when node_modules exists',
{ timeout: 10000 },
async (t) => {
const f = fixture(t);
rmSync(join(f.dir, 'n8n', 'node_modules'), { recursive: true });
process.env.TEST_INSTALL_FAIL = '1';
await assert.rejects(f.prepare(), /pnpm failed/);
delete process.env.TEST_INSTALL_FAIL;
assert.equal((await f.prepare()).directory, join(f.dir, 'n8n'));
assert.equal(f.commands().filter((entry) => entry.command === 'pnpm').length, 2);
},
);
test(
'reports session API errors and creates conversations only when none exist',
{ timeout: 10000 },
async (t) => {
const f = fixture(t);
const first = await f.prepare({ web: true });
assert.equal(first.sessionID, 'ses_1');
writeFileSync(join(f.dir, 'reject-session'), '');
await assert.rejects(f.prepare({ web: true }), /Cannot list OpenCode sessions \(503\)/);
assert.ok(JSON.parse(readFileSync(join(f.dir, 'sessions.json'), 'utf8')).ses_1);
rmSync(join(f.dir, 'reject-session'));
assert.equal((await f.prepare({ web: true })).sessionID, first.sessionID);
writeFileSync(join(f.dir, 'reject-create'), '');
await assert.rejects(
f.prepare({ name: 'fresh-task', web: true }),
/Cannot create OpenCode session \(503\)/,
);
rmSync(join(f.dir, 'reject-create'));
assert.equal((await f.prepare({ name: 'fresh-task', web: true })).sessionID, 'ses_2');
},
);

View File

@ -1,5 +0,0 @@
// Shared by post-start.mjs (installs at container start) and scripts/cloud-session.mjs
// (re-installs in the session prelude). Both must agree or a session boots with a
// partial skill set.
export const MARKETPLACE = 'n8n-io/n8n-agent-skills';
export const PLUGINS = ['quality@n8n-agent-skills', 'security@n8n-agent-skills'];

View File

@ -1,131 +0,0 @@
#!/usr/bin/env node
// Runs on each Codespace start. Installs the skills and harness, then starts the worker.
import { execFileSync } from 'node:child_process';
import { readFileSync, rmSync, writeFileSync } from 'node:fs';
import { homedir } from 'node:os';
import { join } from 'node:path';
import { installAgentHarness } from '../../scripts/agent-harness.mjs';
import { codespaceSecret } from '../../scripts/codespace-env.mjs';
import { MARKETPLACE, PLUGINS } from './plugins.mjs';
const STATUS_FILE = '/tmp/post-start-status.json';
// Claude Code stages the marketplace clone here before renaming it to the cache dir.
// A clone that dies partway has been seen to leave the staging path in a state that
// fails the next clone ("cannot copy ... File exists"), so clear it before retrying.
const MARKETPLACE_STAGING = join(
homedir(),
'.claude/plugins/marketplaces',
MARKETPLACE.replace('/', '-'),
);
// The codespace clones over HTTPS and has no SSH key. The loader defaults to SSH
// for "owner/repo", so it must prefer HTTPS or the private clone fails.
process.env.CLAUDE_CODE_PLUGIN_PREFER_HTTPS = '1';
process.env.CLAUDE_CODE_PLUGIN_KEEP_MARKETPLACE_ON_FAILURE = '1';
function tryRun(label, cmd, args) {
try {
execFileSync(cmd, args, { stdio: 'inherit' });
return true;
} catch (error) {
console.error(`${label}: ${error.message}`);
return false;
}
}
// Both commands are idempotent and exit 0 when the marketplace or plugin is already
// present, so a healthy start re-runs them cheaply and only a real failure retries.
function addMarketplace() {
const add = (label) => tryRun(label, 'claude', ['plugin', 'marketplace', 'add', MARKETPLACE]);
if (add('marketplace add')) return true;
console.error(`marketplace add: clearing ${MARKETPLACE_STAGING} and retrying once`);
try {
rmSync(MARKETPLACE_STAGING, { recursive: true, force: true });
} catch (error) {
console.error(`marketplace staging cleanup: ${error.message}`);
return false;
}
return add('marketplace add (retry)');
}
// Claude Code reads the Flaky token when it connects. An env reference such as
// ${FLAKY_MCP_TOKEN} fails in processes that did not load the secrets file, for
// example the desktop app's SSH server. The helper reads the file on each connect,
// so it also gets a rotated token. The token is not copied into ~/.claude.json.
const FLAKY_HEADERS_HELPER =
'. /usr/local/lib/codespaces-env.sh; printf %s "{\\"Authorization\\":\\"Bearer $FLAKY_MCP_TOKEN\\"}"';
function registerFlakyMcp() {
// Forks have no repository secrets.
const url = codespaceSecret('FLAKY_MCP_URL');
if (!url || !codespaceSecret('FLAKY_MCP_TOKEN')) return;
let current;
try {
current = JSON.parse(readFileSync(join(homedir(), '.claude.json'), 'utf8')).mcpServers?.flaky;
} catch {
// No readable config yet: register the server below.
}
if (current?.url === url && current.headersHelper === FLAKY_HEADERS_HELPER && !current.headers) {
return;
}
if (current) tryRun('flaky mcp remove', 'claude', ['mcp', 'remove', '--scope', 'user', 'flaky']);
const config = { type: 'http', url, headersHelper: FLAKY_HEADERS_HELPER };
tryRun('flaky mcp add', 'claude', [
'mcp',
'add-json',
'--scope',
'user',
'flaky',
JSON.stringify(config),
]);
}
registerFlakyMcp();
tryRun('skills repo reachable', 'git', ['ls-remote', `https://github.com/${MARKETPLACE}`, 'HEAD']);
const installed = [];
const failed = [];
if (addMarketplace()) {
for (const plugin of PLUGINS) {
const ok = tryRun(`plugin install ${plugin}`, 'claude', ['plugin', 'install', plugin]);
(ok ? installed : failed).push(plugin);
}
} else {
failed.push(...PLUGINS);
}
// A skipped install is otherwise invisible until someone misses a skill mid-session.
if (failed.length > 0) {
console.error(`\n!! SKILLS NOT INSTALLED: ${failed.join(', ')}`);
console.error('!! Sessions start without them. Retry with:');
console.error('!! node /workspaces/n8n/.devcontainer/codespaces/post-start.mjs\n');
}
let harness;
try {
const result = installAgentHarness();
harness = { status: 'active', version: result.version, cacheHit: result.cacheHit };
} catch (error) {
harness = { status: 'unavailable', error: error.message };
console.error(`agent harness: ${error.message}`);
}
const workerStarted =
harness.status === 'active' &&
tryRun('worker start', 'tmux', [
'new-session',
'-d',
'-s',
'agent-worker',
'bash -lc ". /usr/local/lib/codespaces-env.sh; export CLAUDE_CODE_PLUGIN_PREFER_HTTPS=1 CLAUDE_CODE_PLUGIN_KEEP_MARKETPLACE_ON_FAILURE=1; node /workspaces/n8n/.devcontainer/codespaces/agent-worker.mjs >> /tmp/agent-worker.log 2>&1"',
]);
if (!workerStarted && harness.status !== 'active') {
console.error('worker start: skipped because the pinned agent harness is unavailable');
}
writeFileSync(STATUS_FILE, JSON.stringify({ installed, failed, harness, workerStarted }, null, 2));

View File

@ -1,32 +0,0 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { aliasHostConfig, withInclude } from '../../scripts/cloud-session-ssh-config.mjs';
const GH_CONFIG = `Host cs.ominous-doodle.master
User node
ProxyCommand gh cs ssh -c ominous-doodle --stdio
UserKnownHostsFile=/dev/null
`;
test('replaces the gh host name with the fixed alias', () => {
assert.equal(
aliasHostConfig(GH_CONFIG),
GH_CONFIG.replace('cs.ominous-doodle.master', 'n8n-codespace'),
);
});
test('rejects gh output without exactly one proxied host', () => {
assert.throws(() => aliasHostConfig(''));
assert.throws(() => aliasHostConfig(`${GH_CONFIG}Host other\n`));
assert.throws(() => aliasHostConfig('Host cs.x\n\tUser node\n'));
});
test('adds the Include line at the top once', () => {
const include = 'Include ~/.ssh/n8n-codespace.conf';
assert.equal(withInclude(''), `${include}\n`);
const updated = withInclude('Host github.com\n\tUser git\n');
assert.equal(updated, `${include}\n\nHost github.com\n\tUser git\n`);
assert.equal(withInclude(updated), undefined);
});

View File

@ -1,4 +0,0 @@
# Mouse scroll, a deep scrollback, and correct colors for agent sessions.
set -g mouse on
set -g history-limit 50000
set -g default-terminal "tmux-256color"

View File

@ -3,10 +3,10 @@ volumes:
services:
postgres:
image: postgres:18-alpine
image: postgres:16-alpine
restart: unless-stopped
volumes:
- postgres-data:/var/lib/postgresql
- postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=n8n
- POSTGRES_PASSWORD=password

View File

@ -1,32 +0,0 @@
ARG NODE_VERSION=24
# Debian to match the agent image's toolchain. No Playwright, no agent CLIs and no
# Docker here: a preview only builds the repo and serves it.
FROM node:${NODE_VERSION}-bookworm
# tmux holds the backend between `preview refresh` runs. The rest is for triage.
RUN apt-get update && apt-get install -y --no-install-recommends \
tmux sudo less jq \
&& rm -rf /var/lib/apt/lists/*
# Shared with the agent config, so there is one copy of each shim. The build context
# is .devcontainer/, set in devcontainer.json, which is why these paths start with
# codespaces/. git and gh must survive Codespaces' token rotation; the stock helpers
# read a snapshot of $GITHUB_TOKEN and go stale in ssh and tmux shells.
COPY codespaces/codespaces-env.sh /usr/local/lib/codespaces-env.sh
COPY codespaces/gitcredential-refresh.sh /usr/local/bin/gitcredential-refresh.sh
COPY codespaces/gh-shim.sh /usr/local/bin/gh
RUN chmod +x /usr/local/bin/gitcredential-refresh.sh /usr/local/bin/gh
# Interactive shells for a human debugging the box.
RUN echo '. /usr/local/lib/codespaces-env.sh' >> /etc/bash.bashrc
RUN echo node ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/node && chmod 0440 /etc/sudoers.d/node
RUN mkdir -p /workspaces && chown node:node /workspaces
RUN corepack enable
ENV PNPM_HOME=/home/node/.local/share/pnpm
ENV PATH=$PNPM_HOME/bin:$PATH
USER node
RUN mkdir -p $PNPM_HOME ~/.pnpm-store && pnpm config set store-dir ~/.pnpm-store --global

View File

@ -1,54 +0,0 @@
{
"name": "n8n (PR preview)",
// No compose: a preview needs no Postgres sidecar. n8n defaults to sqlite
// (packages/@n8n/config/src/configs/database.config.ts), which is what a
// throwaway instance wants. Context is .devcontainer/ so the Dockerfile can
// reuse the token-rotation shims in codespaces/ instead of copying them.
"build": {
"dockerfile": "Dockerfile",
"context": ".."
},
// Codespaces clones the repo into /workspaces/<repo>
"workspaceFolder": "/workspaces/n8n",
// The floor, so the 2-core machine stays available. This config exists mainly to
// ask for less than the agent one: the measured footprint is ~15gb, the
// editor-ui build peaks near 3gb, and scripts/agent-setup.mjs already builds the
// whole repo inside 6gb.
"hostRequirements": {
"cpus": 2,
"memory": "8gb",
"storage": "32gb"
},
"features": {
"ghcr.io/devcontainers/features/github-cli:1": {},
// gh codespace ssh needs an SSH server inside the container
"ghcr.io/devcontainers/features/sshd:1": {}
},
"forwardPorts": [5678],
// Runs in prebuilds, so a fresh preview starts with deps installed and the build
// warm. Concurrency and heap are capped for the 2-core box, matching the
// reasoning in scripts/agent-setup.mjs. A failing onCreateCommand is fatal —
// Codespaces discards the container — so the build only ever tails its log.
"onCreateCommand": "corepack install && pnpm install && (NODE_OPTIONS=--max-old-space-size=6144 pnpm exec turbo run build --concurrency=2 > /tmp/build.log 2>&1 || tail -n 50 /tmp/build.log)",
// Codespaces regenerates the managed /etc/gitconfig on each start, and the ssh
// command shells `pnpm preview` uses are neither login nor interactive, so
// profile.d and bashrc never fire for them. Register the helper here instead.
"postStartCommand": "git config --global --get-all credential.helper 2>/dev/null | grep -qs gitcredential-refresh || git config --global --add credential.helper /usr/local/bin/gitcredential-refresh.sh",
// Declarations only. All of these are set at the repository level, so Codespaces
// injects them into every preview box whether or not they are listed here.
// scripts/codespace-preview/preview-serve.mjs reads them with codespaceSecret().
"secrets": {
"N8N_LICENSE_ACTIVATION_KEY": {
"description": "Sandbox licence key. The preview:enterprise label decides whether it reaches n8n."
},
"CODESPACE_ENV_URL": {
"description": "n8n webhook that returns extra environment for a preview, as a flat JSON object"
},
"CODESPACE_ENV_USER": {
"description": "Basic auth user for CODESPACE_ENV_URL. Defaults to `preview`."
},
"CODESPACE_ENV_PASSWORD": {
"description": "Basic auth password for CODESPACE_ENV_URL"
}
}
}

View File

@ -1,81 +0,0 @@
/**
* One-click sign-in for preview instances. Loaded through EXTERNAL_HOOK_FILES by
* scripts/codespace-preview/preview-serve.mjs, so it is never part of a packaged n8n.
*
* The route performs an ordinary login on the visitor's behalf: it calls
* /rest/login with the preview instance's own seeded credentials and forwards the
* session cookie that endpoint returns. The visitor gets exactly the session a
* normal sign-in would give them, so this is a convenience over the login form
* rather than a different way in.
*
* Only enable this on a throwaway instance whose port is org-visible. It refuses to
* register unless N8N_PREVIEW_SIGNIN is exactly "1", so loading the file by
* accident does nothing.
*
* Calling /rest/login rather than AuthService keeps this working across changes to
* n8n's internals, and means it can never grant more than a real login would.
*
* Two things make the route reachable:
* - `n8n.ready` fires after configure(), so the SPA catch-all is already
* registered. N8N_ADDITIONAL_NON_UI_ROUTES must list this path, or
* historyApiHandler answers with index.html before the route is reached.
* - the handler is registered on server.app, which AbstractServer exposes.
*/
const ROUTE = '/preview-signin';
module.exports = {
n8n: {
ready: [
async function previewSignin(server) {
if (process.env.N8N_PREVIEW_SIGNIN !== '1') return;
const email = process.env.PREVIEW_OWNER_EMAIL;
const password = process.env.PREVIEW_OWNER_PASSWORD;
if (!email || !password) {
console.warn(
`[preview-signin] PREVIEW_OWNER_EMAIL or PREVIEW_OWNER_PASSWORD is unset — ${ROUTE} not registered`,
);
return;
}
const port = process.env.N8N_PORT ?? '5678';
server.app.get(ROUTE, async (_req, res) => {
const fallback = `Sign in as ${email} instead.`;
try {
const login = await fetch(`http://127.0.0.1:${port}/rest/login`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ emailOrLdapLoginId: email, password }),
signal: AbortSignal.timeout(15_000),
});
if (!login.ok) {
res
.status(502)
.type('text')
.send(`Preview sign-in failed: /rest/login returned ${login.status}. ${fallback}`);
return;
}
// Hand the browser exactly the cookies the login endpoint issued.
const cookies = login.headers.getSetCookie();
if (cookies.length === 0) {
res
.status(502)
.type('text')
.send(`Preview sign-in failed: no session cookie was issued. ${fallback}`);
return;
}
res.setHeader('set-cookie', cookies);
res.redirect('/');
} catch (error) {
res.status(502).type('text').send(`Preview sign-in failed: ${error.message}. ${fallback}`);
}
});
console.log(`[preview-signin] ${ROUTE} is live — it signs the visitor in as ${email}`);
},
],
},
};

View File

@ -26,7 +26,7 @@
!docker/images/runners
!docker/images/runners/n8n-task-runners.json
# === benchmark image (packages/quality/efficiency/scale/benchmark/Dockerfile) ===
# === benchmark image (packages/@n8n/benchmark/Dockerfile) ===
!package.json
!pnpm-lock.yaml
!pnpm-workspace.yaml
@ -34,10 +34,8 @@
!patches/**
!scripts
!scripts/**
!packages/quality
!packages/quality/efficiency
!packages/quality/efficiency/scale
!packages/quality/efficiency/scale/benchmark
!packages/quality/efficiency/scale/benchmark/**
!packages/@n8n/benchmark
!packages/@n8n/benchmark/**
!packages/@n8n/typescript-config
!packages/@n8n/typescript-config/**

2
.gitattributes vendored
View File

@ -1,3 +1 @@
*.sh text eol=lf
*.patch text eol=lf
docs/generated/**/* linguist-generated

View File

@ -1,36 +0,0 @@
#!/usr/bin/env node
// Git hook helper (post-merge, post-rewrite, post-checkout): warn when the
// installed pnpm major differs from the one pinned in package.json.
// Usage: node .githooks/check-pnpm.mjs [checkout-type]
// For post-checkout, pass lefthook's {3}: "0" is a file checkout, which we skip.
import { spawnSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
import { homedir } from 'node:os';
import { fileURLToPath } from 'node:url';
if (process.argv[2] === '0') process.exit(0);
const packageJsonPath = fileURLToPath(new URL('../package.json', import.meta.url));
const { packageManager = '' } = JSON.parse(readFileSync(packageJsonPath, 'utf8'));
const wanted = /^pnpm@(\d+)/.exec(packageManager)?.[1];
if (!wanted) process.exit(0);
// Run from the home dir so pnpm reports the installed version,
// not the version pinned by this repo.
// `shell` lets Windows resolve pnpm.cmd. The command has no user input.
const result = spawnSync('pnpm --version', { cwd: homedir(), shell: true, encoding: 'utf8' });
const current = result.status === 0 ? result.stdout.trim().split('.')[0] : '';
if (wanted === current) process.exit(0);
const installCommand =
process.platform === 'win32'
? 'powershell -c "irm https://get.pnpm.io/install.ps1 | iex"'
: 'curl -fsSL https://get.pnpm.io/install.sh | sh -';
console.log(`
⚠ This repo now pins pnpm ${wanted} (you have ${current || 'none'}).
pnpm should auto-download it on the next install command.
If that fails (some pnpm 10 versions can't install v12), reinstall:
${installCommand}
`);

View File

@ -19,39 +19,6 @@ All telemetry uses the same format:
}
```
Container startup payloads also include:
```json
{
"attempt_id": "uuid",
"correlation": {
"profile": "sqlite",
"shard": "3",
"worker": "2",
"retry": 1,
"restartReason": null
},
"stages": [
{
"name": "n8n-startup",
"source": "local",
"startedAt": "2026-03-16T12:00:00.000Z",
"elapsedMs": 4200,
"outcome": "failure",
"errorMessage": "readiness failed"
}
],
"failure_phase": "n8n-startup"
}
```
`attempt_id` identifies one stack startup. Join all stage records and attached
Playwright startup evidence with this value. `outcome` is `success`, `failure`,
or `cancelled`. Stage durations are also sent as `stack-startup-stage` metrics
with `attempt_id`, `stage`, `source`, and `outcome` dimensions. `profile` is
resolved from the configured profile or stack shape. Other unavailable
correlation values stay `null`.
## Standard Context Fields
```typescript
@ -81,12 +48,10 @@ return 'blacksmith';
| Telemetry | Source | Metrics |
|-----------|--------|---------|
| Playwright perf/benchmark | `packages/quality/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
| Accessibility buckets | `packages/quality/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
| Playwright perf/benchmark | `packages/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
| Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
| Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
| Dive image efficiency | `packages/quality/efficiency/images/image-efficiency.spec.ts` | Layer content size, estimated waste, efficiency score, and layer count through the shared Playwright reporter |
| Container stack | `packages/quality/environments/containers/telemetry.ts` | E2E stack startup times per service |
| Container stack | `packages/testing/containers/telemetry.ts` | E2E stack startup times per service |
## Secrets

2
.github/CLAUDE.md vendored
View File

@ -13,7 +13,7 @@ This folder contains n8n's GitHub Actions infrastructure.
| `workflows/` | GitHub Actions workflows |
| `actions/` | Reusable composite actions |
| `scripts/` | Release & Docker automation |
| `scripts/owners/` | Team review ownership scripts; the OWNERS file is at the repo root (`required` entries gate merges) |
| `CODEOWNERS` | Team review ownership |
### Workflow Naming

6
.github/CODEOWNERS vendored Normal file
View File

@ -0,0 +1,6 @@
packages/@n8n/db/src/migrations/ @n8n-io/migrations-review
.github/workflows @n8n-io/qa-dx
.github/scripts @n8n-io/qa-dx
.github/actions @n8n-io/qa-dx
.github/poutine-rules @n8n-io/qa-dx
docker/get-n8n.sh @n8n-io/qa-dx

View File

@ -124,9 +124,6 @@ of the breaking removal on `3.x`.
## How the daily sync works
The sync runs automation code from the triggering `master` SHA while its working checkout
stays on `3.x`.
[`util-sync-master-to-3x.yml`](./workflows/util-sync-master-to-3x.yml) runs daily and
**replays the `3.x`-only commits on top of `master`** (a rebase), then force-pushes `3.x`:
@ -136,26 +133,14 @@ stays on `3.x`.
original message and author. Nothing is ever squashed. Merge commits in the range (breaking
PRs merged into `3.x`) are flattened away, and a breaking commit that also landed on
`master` is dropped as empty.
3. Conflicts confined to **non-lockfile mechanical files** — bot-maintained content with a
deterministic resolution (`packages/frontend/editor-ui/data/node-popularity.json`,
`packages/@n8n/instance-ai/src/tools/nodes/credential-setupability.json`, and
`.github/test-metrics/e2e-impact-map.json`) — are **auto-resolved during the replay**,
by taking `master`'s side. The resolution is folded into the stalled commit, so this still
adds **no commit and no PR**. The list lives in `MECHANICAL_PATHS` in
[`sync-master-to-3x.mjs`](./scripts/sync-master-to-3x.mjs).
4. On a **real code conflict or any `pnpm-lock.yaml` conflict**, `3.x` is left **untouched**
and a **draft conflict PR** (labeled `automation:v3-sync`) carrying the conflict markers is
opened on `sync/master-to-3x`. Other mechanical files are already pre-resolved. The
lockfile stays unresolved because pnpm validation is not reliable while the merge index is
unresolved. The workflow also posts to the **`#alerts-v3-sync`** Slack channel. **Syncs
pause until that PR is merged**, so conflicts never pile up silently.
3. On a **conflict**, `3.x` is left **untouched** and a **draft conflict PR** (labeled
`automation:v3-sync`) carrying the conflict markers is opened on `sync/master-to-3x`, plus a
post to the **`#alerts-v3-sync`** Slack channel. **Syncs pause until that PR is merged** —
so conflicts never pile up silently.
Whatever route it takes, the sync verifies that the content it is about to push is **exactly
the tree a merge of `3.x` and `master` produces** (`git merge-tree`), and that no conflict
markers are present. When mechanical files had to be regenerated, the exactness check applies
to every path **except** those files, and a regenerated lockfile must additionally be
consistent with the manifests in the pushed tree. Any check failing fails the run instead of
rewriting `3.x`.
markers are present. Either check failing fails the run instead of rewriting `3.x`.
> **`3.x` is force-pushed daily.** Branch breaking-change PRs off **`master`** and target
> `3.x` — then your merge-base is a `master` commit that survives every rewrite and your
@ -167,8 +152,7 @@ rewriting `3.x`.
The conflict branch is `master` merged into `3.x` with the **conflict markers committed**, so
you see exactly what clashed — and the required checks stay red until they're gone, so the PR
can't be merged half-resolved. Non-lockfile mechanical files arrive **pre-resolved** (listed
in the PR under "Auto-resolved for you"), so only the code and lockfile conflicts need you:
can't be merged half-resolved:
```bash
git fetch origin sync/master-to-3x && git switch sync/master-to-3x
@ -176,20 +160,8 @@ git fetch origin sync/master-to-3x && git switch sync/master-to-3x
git push origin sync/master-to-3x
```
If the PR says the lockfile was **deferred**, resolve the other conflicts first. Then
regenerate it with `pnpm install --lockfile-only` and include the result in your fix commit.
Watch for the **"Deleted on one side, changed on the other"** section. Git leaves no markers
for a delete/modify, so the branch looks clean where it is not: the merge keeps `3.x`'s side
(its deletion, or its file when `master` deleted it). Confirm that is right, and check
whether `master`'s change has to be carried over by hand — when a breaking commit re-recorded
or renamed a file, `master`'s edit to the old one usually belongs on the replacement, and no
automation can find that for you. The PR names the `master` commit behind each conflicted
path so you can see what the change was.
Then **merge the PR with the normal merge button.** `master`'s commits arrive as-is and your
fix stays its own commit. **Never close a conflict PR unmerged** — closing resolves nothing and
the same conflict reopens on the next sync.
fix stays its own commit.
`3.x` never holds markers at its tip, so nightly images keep building; the merge commit that
carries them drops out of `3.x`'s history at the next sync (the replay takes the queue's
@ -198,16 +170,13 @@ commits only).
The next sync then makes `3.x` linear again. The plain replay stalls at that point (a fix
recorded around a merge commit leaves no patch to replay), so it replays a second time with
`3.x`'s side favoured, and your fix commit — which is in the queue — does the real work.
Stalls that favouring cannot settle on its own (e.g. modify/delete, when `master` touched a
file a breaking commit deletes) are resolved toward `3.x`'s side during the replay. Nothing
is squashed, and the tree guard proves the result is exactly the merge of `3.x` and
Nothing is squashed, and the tree guard proves the result is exactly the merge of `3.x` and
`master`.
**Who gets pinged.** The conflict is attributed to the authors of the `3.x` commits behind the
conflicted files, plus the `master` commits that touched the same files
(`.github/scripts/sync-conflict-owners.mjs`, mapped to GitHub accounts). Both sides are named
in the PR body and the `#alerts-v3-sync` message. **Nobody is requested as a reviewer** — the
resolver picks the PR up themselves.
conflicted files (`.github/scripts/sync-conflict-owners.mjs`, mapped to GitHub accounts).
Those authors are **requested as reviewers** on the conflict PR and listed in the
`#alerts-v3-sync` message.
## Trialing v3
@ -217,24 +186,8 @@ resolver picks the PR up themselves.
```bash
docker pull n8nio/n8n:v3-nightly # latest v3 nightly
docker pull n8nio/n8n:v3-nightly-20260625 # a specific build date
docker pull n8nio/n8n:v3-rc # latest release candidate
docker pull n8nio/n8n:v3-rc-20260625 # latest RC of that day
docker pull n8nio/n8n:v3-rc-20260625.2 # one exact RC, never overwritten
```
Every Monday's nightly is also retagged as a release candidate, and a maintainer can
publish extra RCs any day (`force_rc` on a manual run). Each publish claims the next
rolling number for the day — `v3-rc-<date>.1`, `.2`, … — and moves `v3-rc` and
`v3-rc-<date>` onto it, so:
- **`v3-rc-<date>.N`** — pin this to hold a build still. Immutable.
- **`v3-rc` / `v3-rc-<date>`** — track the newest RC overall / of that day. These move.
The retag covers the whole set — `n8nio/n8n`, `n8nio/runners` and
`n8nio/runners:v3-rc[-<date>.N]-distroless` — so pinning one RC across a stack gives
images built from one `3.x` commit, unlike `v3-nightly`, which moves daily and can be
mid-build when you pull. The same tags exist on GHCR (`ghcr.io/n8n-io/…`).
Use these to trial v3 in docker/kubernetes before release. Do **not** use them in
production.

234
.github/OWNERS vendored Normal file
View File

@ -0,0 +1,234 @@
# n8n CODEOWNERS
#
# Last-match-wins: specific rules MUST come AFTER general rules.
# Default catch-all (ensures every file gets at least one reviewer)
* @n8n-io/catalysts
# Catalysts
packages/core/ @n8n-io/catalysts
packages/workflow/ @n8n-io/catalysts
packages/workflow/styker.config.mjs @n8n-io/qa-dx
packages/@n8n/config/ @n8n-io/catalysts
packages/@n8n/backend-common/ @n8n-io/catalysts
packages/@n8n/backend-network/ @n8n-io/catalysts
packages/@n8n/backend-test-utils/ @n8n-io/catalysts
packages/@n8n/di/ @n8n-io/catalysts
packages/@n8n/errors/ @n8n-io/catalysts
packages/@n8n/constants/ @n8n-io/catalysts
packages/@n8n/utils/ @n8n-io/catalysts
packages/@n8n/api-types/ @n8n-io/catalysts
packages/@n8n/workflow-sdk/ @n8n-io/instance-ai
packages/@n8n/task-runner/ @n8n-io/catalysts
packages/@n8n/task-runner-python/ @n8n-io/catalysts
packages/@n8n/expression-runtime/ @n8n-io/catalysts
packages/@n8n/db/ @n8n-io/catalysts
packages/@n8n/json-schema-to-zod/ @n8n-io/catalysts
packages/@n8n/crdt/ @n8n-io/catalysts
packages/@n8n/extension-sdk/ @n8n-io/catalysts
packages/@n8n/eslint-config/ @n8n-io/qa-dx
packages/@n8n/typescript-config/ @n8n-io/qa-dx
packages/@n8n/db/src/migrations/ @n8n-io/migrations-review
# Top-level paths
scripts/ @n8n-io/qa-dx
patches/ @n8n-io/qa-dx
assets/ @n8n-io/adore
security/ @n8n-io/qa-dx
# @n8n/cli
packages/@n8n/cli/ @n8n-io/adore
packages/@n8n/cli/src/commands/credential/ @n8n-io/iam
packages/@n8n/cli/src/commands/user/ @n8n-io/iam
packages/@n8n/cli/src/commands/data-table/ @n8n-io/adore
packages/@n8n/cli/src/commands/tag/ @n8n-io/adore
packages/@n8n/cli/src/commands/project/ @n8n-io/ligo
packages/@n8n/cli/src/commands/source-control/ @n8n-io/ligo
packages/@n8n/cli/src/commands/variable/ @n8n-io/ligo
packages/@n8n/cli/src/commands/skill/ @n8n-io/ai
# packages/cli
packages/cli/ @n8n-io/catalysts
packages/cli/src/scaling/ @n8n-io/catalysts
packages/cli/src/concurrency/ @n8n-io/catalysts
packages/cli/src/execution-lifecycle/ @n8n-io/catalysts
packages/cli/src/executions/ @n8n-io/catalysts
packages/cli/src/task-runners/ @n8n-io/catalysts
packages/cli/src/webhooks/ @n8n-io/catalysts
packages/cli/src/push/ @n8n-io/catalysts
packages/cli/src/commands/ @n8n-io/catalysts
packages/cli/src/config/ @n8n-io/catalysts
packages/cli/src/eventbus/ @n8n-io/catalysts
packages/cli/src/events/ @n8n-io/catalysts
packages/cli/src/security-audit/ @n8n-io/catalysts
packages/cli/src/modules/workflow-index/ @n8n-io/catalysts
packages/cli/src/modules/breaking-changes/ @n8n-io/catalysts
packages/cli/src/modules/otel/ @n8n-io/ligo
packages/cli/src/auth/ @n8n-io/iam
packages/cli/src/credentials/ @n8n-io/iam
packages/cli/src/mfa/ @n8n-io/iam
packages/cli/src/oauth/ @n8n-io/iam
packages/cli/src/permissions.ee/ @n8n-io/iam
packages/cli/src/sso.ee/ @n8n-io/iam
packages/cli/src/user-management/ @n8n-io/iam
packages/cli/src/license/ @n8n-io/iam
packages/cli/src/modules/ldap.ee/ @n8n-io/iam
packages/cli/src/modules/log-streaming.ee/ @n8n-io/iam
packages/cli/src/modules/sso-oidc/ @n8n-io/iam
packages/cli/src/modules/sso-saml/ @n8n-io/iam
packages/cli/src/modules/provisioning.ee/ @n8n-io/iam
packages/cli/src/modules/dynamic-credentials.ee/ @n8n-io/iam
packages/cli/src/modules/redaction/ @n8n-io/iam
packages/cli/src/modules/instance-registry/ @n8n-io/iam
packages/cli/src/modules/token-exchange/ @n8n-io/iam
packages/cli/src/environments.ee/ @n8n-io/ligo
packages/cli/src/public-api/ @n8n-io/ligo
packages/cli/src/modules/source-control.ee/ @n8n-io/ligo
packages/cli/src/modules/external-secrets.ee/ @n8n-io/ligo
packages/cli/src/modules/insights/ @n8n-io/ligo
packages/cli/src/collaboration/ @n8n-io/catalysts
packages/cli/src/binary-data/ @n8n-io/catalysts
packages/cli/src/posthog/ @n8n-io/adore
packages/cli/src/modules/data-table/ @n8n-io/adore
packages/cli/src/evaluation.ee/ @n8n-io/ai
packages/cli/src/chat/ @n8n-io/ai
packages/cli/src/tool-generation/ @n8n-io/ai
packages/cli/src/modules/workflow-builder/ @n8n-io/ai
packages/cli/src/modules/mcp/ @n8n-io/ai
packages/cli/src/modules/quick-connect/ @n8n-io/ai
packages/cli/src/modules/chat-hub/ @n8n-io/ai
packages/cli/src/modules/instance-ai/ @n8n-io/instance-ai
packages/cli/src/modules/community-packages/ @n8n-io/nodes
# CLI controllers
packages/cli/src/controllers/auth.controller.ts @n8n-io/iam
packages/cli/src/controllers/invitation.controller.ts @n8n-io/iam
packages/cli/src/controllers/me.controller.ts @n8n-io/iam
packages/cli/src/controllers/mfa.controller.ts @n8n-io/iam
packages/cli/src/controllers/owner.controller.ts @n8n-io/iam
packages/cli/src/controllers/password-reset.controller.ts @n8n-io/iam
packages/cli/src/controllers/role.controller.ts @n8n-io/iam
packages/cli/src/controllers/users.controller.ts @n8n-io/iam
packages/cli/src/controllers/user-settings.controller.ts @n8n-io/iam
packages/cli/src/controllers/api-keys.controller.ts @n8n-io/iam
packages/cli/src/controllers/security-settings.controller.ts @n8n-io/iam
packages/cli/src/controllers/oauth/ @n8n-io/iam
packages/cli/src/controllers/ai.controller.ts @n8n-io/ai
packages/cli/src/controllers/annotation-tags.controller.ee.ts @n8n-io/ai
packages/cli/src/controllers/cta.controller.ts @n8n-io/adore
packages/cli/src/controllers/folder.controller.ts @n8n-io/adore
packages/cli/src/controllers/tags.controller.ts @n8n-io/adore
packages/cli/src/controllers/binary-data.controller.ts @n8n-io/adore
packages/cli/src/controllers/dynamic-templates.controller.ts @n8n-io/adore
packages/cli/src/controllers/posthog.controller.ts @n8n-io/adore
packages/cli/src/controllers/translation.controller.ts @n8n-io/adore
packages/cli/src/controllers/project.controller.ts @n8n-io/ligo
packages/cli/src/controllers/workflow-statistics.controller.ts @n8n-io/ligo
packages/cli/src/controllers/node-types.controller.ts @n8n-io/nodes
packages/cli/src/controllers/dynamic-node-parameters.controller.ts @n8n-io/nodes
packages/cli/src/controllers/e2e.controller.ts @n8n-io/qa-dx
# CLI services
packages/cli/src/services/jwt.service.ts @n8n-io/iam
packages/cli/src/services/user.service.ts @n8n-io/iam
packages/cli/src/services/role.service.ts @n8n-io/iam
packages/cli/src/services/role-cache.service.ts @n8n-io/iam
packages/cli/src/services/password.utility.ts @n8n-io/iam
packages/cli/src/services/public-api-key.service.ts @n8n-io/iam
packages/cli/src/services/security-settings.service.ts @n8n-io/iam
packages/cli/src/services/ssrf/ @n8n-io/catalysts
packages/cli/src/services/static-auth-service.ts @n8n-io/iam
packages/cli/src/services/access.service.ts @n8n-io/iam
packages/cli/src/services/ai.service.ts @n8n-io/ai
packages/cli/src/services/ai-usage.service.ts @n8n-io/ai
packages/cli/src/services/ai-workflow-builder.service.ts @n8n-io/ai
packages/cli/src/services/annotation-tag.service.ee.ts @n8n-io/ai
packages/cli/src/services/folder.service.ts @n8n-io/adore
packages/cli/src/services/tag.service.ts @n8n-io/adore
packages/cli/src/services/cta.service.ts @n8n-io/adore
packages/cli/src/services/dynamic-templates.service.ts @n8n-io/adore
packages/cli/src/services/frontend.service.ts @n8n-io/adore
packages/cli/src/services/banner.service.ts @n8n-io/adore
packages/cli/src/services/project.service.ee.ts @n8n-io/ligo
packages/cli/src/services/workflow-statistics.service.ts @n8n-io/ligo
packages/cli/src/services/export.service.ts @n8n-io/ligo
packages/cli/src/services/import.service.ts @n8n-io/ligo
packages/cli/src/services/ownership.service.ts @n8n-io/ligo
packages/cli/src/services/dynamic-node-parameters.service.ts @n8n-io/nodes
# Adore
packages/frontend/editor-ui/ @n8n-io/frontend
packages/frontend/editor-ui/src/features/ai/ @n8n-io/ai
packages/frontend/editor-ui/src/features/credentials/ @n8n-io/iam
packages/frontend/editor-ui/src/features/execution/ @n8n-io/ligo
packages/frontend/editor-ui/src/features/project-roles/ @n8n-io/iam
packages/frontend/editor-ui/src/features/integrations/ @n8n-io/ligo
packages/frontend/@n8n/design-system/ @n8n-io/design
packages/frontend/@n8n/stores/ @n8n-io/frontend
packages/frontend/@n8n/composables/ @n8n-io/frontend
packages/frontend/@n8n/rest-api-client/ @n8n-io/frontend
packages/frontend/@n8n/storybook/ @n8n-io/design
packages/frontend/@n8n/i18n/ @n8n-io/frontend
packages/@n8n/stylelint-config/ @n8n-io/qa-dx
# AI
packages/@n8n/instance-ai/ @n8n-io/instance-ai
packages/@n8n/nodes-langchain/ @n8n-io/ai
packages/@n8n/ai-utilities/ @n8n-io/ai
packages/@n8n/ai-node-sdk/ @n8n-io/ai
packages/@n8n/ai-workflow-builder.ee/ @n8n-io/ai
packages/@n8n/agents/ @n8n-io/ai
packages/frontend/@n8n/chat/ @n8n-io/ai
# Chat
packages/@n8n/chat-hub/ @n8n-io/ai
# Nodes
packages/@n8n/codemirror-lang/ @n8n-io/nodes
packages/@n8n/codemirror-lang-html/ @n8n-io/nodes
packages/@n8n/codemirror-lang-sql/ @n8n-io/nodes
packages/nodes-base/ @n8n-io/nodes
packages/@n8n/decorators/ @n8n-io/catalysts
packages/node-dev/ @n8n-io/nodes
packages/@n8n/create-node/ @n8n-io/nodes
packages/@n8n/node-cli/ @n8n-io/nodes
packages/@n8n/imap/ @n8n-io/iam
packages/@n8n/syslog-client/ @n8n-io/iam
packages/@n8n/scan-community-package/ @n8n-io/nodes
packages/@n8n/eslint-plugin-community-nodes/ @n8n-io/nodes
packages/@n8n/computer-use/ @n8n-io/nodes
packages/@n8n/local-gateway/ @n8n-io/nodes
packages/@n8n/mcp-browser/ @n8n-io/nodes
packages/@n8n/mcp-browser-extension/ @n8n-io/nodes
# IAM
packages/@n8n/permissions/ @n8n-io/iam
packages/@n8n/client-oauth2/ @n8n-io/iam
# LiGo
packages/extensions/insights/ @n8n-io/ligo
# CI/CD
.github/ @n8n-io/qa-dx
docker/ @n8n-io/qa-dx
# QA
packages/testing/ @n8n-io/qa-dx
packages/@n8n/benchmark/ @n8n-io/qa-dx
packages/@n8n/vitest-config/ @n8n-io/qa-dx

653
.github/WORKFLOWS.md vendored
View File

@ -10,6 +10,7 @@ Complete reference for n8n's `.github/` folder.
.github/
├── WORKFLOWS.md # This document
├── CI-TELEMETRY.md # Telemetry & metrics guide
├── CODEOWNERS # Team ownership for PR reviews
├── pull_request_template.md # PR description template
├── pull_request_title_conventions.md # Title format rules (Angular)
├── actionlint.yml # Workflow linter config
@ -20,7 +21,6 @@ Complete reference for n8n's `.github/` folder.
│ ├── config.yml # Routes to community/security
│ └── 01-bug.yml # Structured bug report form
├── scripts/ # Automation scripts
│ ├── owners/ # Owners scripts (the OWNERS file lives at the repo root)
│ ├── bump-versions.mjs # Calculate next version
│ ├── update-changelog.mjs # Generate CHANGELOG
│ ├── trim-fe-packageJson.js # Strip frontend devDeps
@ -71,6 +71,7 @@ Complete reference for n8n's `.github/` folder.
│ │ Schedule │───▶│ Nightly/Weekly Jobs │ ┌────────────┐ │
│ │ (cron) │ │ ├─ docker-build-push (nightly) │───▶│ Images │ │
│ └──────────┘ │ ├─ test-benchmark-nightly │───▶│ Metrics │ │
│ │ ├─ test-e2e-vm-expressions │ │
│ │ └─ test-e2e-coverage-weekly │ │
│ └──────────────────────────────────┘ │
│ │
@ -177,16 +178,17 @@ These only run if specific files changed:
| `docker/images/n8n-base/Dockerfile` | `build-base-image.yml` | any |
| `**/package.json`, `**/turbo.json` | `build-windows.yml` | master |
| `packages/@n8n/ai-workflow-builder.ee/evaluations/programmatic/python/**` | `test-evals-python.yml` | any |
| `packages/quality/efficiency/scale/benchmark/**` | `build-benchmark-image.yml` | master |
| `packages/cli/src/public-api/**/*.yml`, `packages/cli/src/public-api/**/*.yaml`, `packages/cli/src/public-api/**/*.css`, `packages/cli/src/public-api/v1/openapi-gen/**/*.ts`, `packages/cli/scripts/build.mjs`, `packages/cli/package.json` | `util-publish-api-schema.yml` | master |
| `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts`, `packages/@n8n/agents/src/**` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
| `docker/get-n8n.sh`, `docker/get-n8n-compose.yml`, `docker/test-get-n8n.sh` | `test-get-n8n.yml` | any |
| `packages/@n8n/benchmark/**` | `build-benchmark-image.yml` | master |
| `packages/cli/src/public-api/**/*.{css,yaml,yml}` | `util-publish-api-schema.yml` | master |
| `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
| `docker/get-n8n.sh`, `docker/test-get-n8n.sh` | `test-get-n8n.yml` | any |
### On PR Review
| Event | Workflow | Condition |
|----------------------------|-----------------------------|------------------------------------------------------|
| Review approved | `release-chromatic.yml` | + design files changed |
| Comment with `@claude` | `util-claude.yml` | mention in any comment |
| Any review | `util-notify-pr-status.yml` | not community-labeled |
**Why Instance AI evals fire once per PR state-change, not per push:** the
@ -209,7 +211,7 @@ the lab bench.** The gate deliberately exposes only PR re-runs. Anything that
isn't PR gating — baselines, model experiments, arbitrary branch runs — goes
through `test-evals-instance-ai.yml`'s own dispatch form ("Instance AI
Evals: Experiments"): full knob set (branch, filter, tier, suite,
iterations, experiment-name, model, model-url, model-key, reasoning-effort, supports-structured-outputs), no per-PR cancellation (dispatches run in parallel, e.g. concurrent
iterations, experiment-name, model), no per-PR cancellation (dispatches run in parallel, e.g. concurrent
model-comparison arms), and SHA-keyed docker cache hits on master. Evals never
run on fork PRs: the event trigger gates on `head.repo.fork`, and the `pr`
re-run path refuses fork PRs in `resolve` (dispatched runs carry secrets).
@ -230,190 +232,42 @@ parallelism). See the `--build-via-mcp` section in
### On PR Close/Merge
| Event | Workflow |
|------------------------------------|--------------------------------|
| PR closed (any) | `util-notify-pr-status.yml` |
| PR merged to `release/*` | `release-publish.yml` |
| PR closed with `codespace-preview` | `util-codespace-preview.yml` |
| Event | Workflow |
|----------------------------|-----------------------------|
| PR closed (any) | `util-notify-pr-status.yml` |
| PR merged to `release/*` | `release-publish.yml` |
### Label Triggers
### Manual Triggers (PR Comments)
| Label | Workflow | Effect |
|-----------------------|------------------------------|----------------------------------------------------|
| `codespace-preview` | `util-codespace-preview.yml` | Runs the PR in a Codespace, comments the URL |
| `preview:enterprise` | `util-codespace-preview.yml` | Re-serves the instance with an enterprise licence |
| `preview:debug` | `util-codespace-preview.yml` | Re-serves the instance with `N8N_LOG_LEVEL=debug` |
| Command | Workflow | Permissions |
|--------------------|------------------------------|---------------------|
| `/test-workflows` | `test-workflows-callable.yml`| admin/write/maintain|
**Why:** A reviewer gets a running instance of the PR without a Docker build or a
cloud deploy. The workflow calls `scripts/codespace-preview/preview.mjs`, which keeps one codespace
for each PR (display name `preview/pr-<number>`) and shares port 5678 with the
organization. A later push serves the new head in the same box. Removing the
label, or closing the PR, deletes the box.
Only a PR from a branch in this repository is eligible: a codespace token is
scoped to `n8n-io/n8n` and cannot check out a fork head.
#### Live progress on the PR
`up` and `refresh` take minutes, and an absent comment looks the same as a broken
preview. So the comment goes up before the box work starts, as a checklist of the
phases in `scripts/codespace-preview/preview-phases.mjs`, and is edited for each phase and once a
minute after that. The final URL replaces it in place.
A comment **edit sends no notification** — only a create does. That is what makes a
heartbeat on a sticky comment acceptable at all.
A `refresh` keeps the URL of the previous run in the checklist. The URL does not
change between runs, and taking a working link off the PR for several minutes is
worse than saying it is briefly down.
A cancelled job — including the 45-minute `timeout-minutes` — kills the script while
its checklist is up, so a last `if: cancelled()` step writes the outcome instead. It
writes **only over a checklist**: a run cancelled while it was queued never started
work, and must leave the previous run's URL alone.
#### Running it by hand
A box sleeps after 2 hours of no use, and GitHub makes every forwarded port
private again at each start. So a preview that slept reaches nobody until
something shares port 5678 again, and its backend is gone with the container.
**Actions → Util: Codespace Preview → Run workflow** does both without a commit
and without a label toggle:
| Input | Meaning |
|---|---|
| `pr_number` | The pull request to act on. |
| `operation` | `up` (default) creates the box if it is gone, starts it if it sleeps, serves the head and shares the port again. `refresh` re-serves the head in a box that already exists. `down` deletes the box. |
Prefer `up` unless you mean to delete: it covers create, wake and re-share.
The button needs write access, and it appears only once the trigger is on
`master` — GitHub lists dispatchable workflows from the default branch. A manual
run takes the branch picked in the dropdown, which is the branch GitHub read the
workflow from, so a branch can test a change to the preview scripts. A fork head
is still refused, by `preview.mjs` rather than by the job's `if`. There is no
`ls` operation: it needs no PR and posts no comment — run `pnpm preview ls`
locally.
#### Preview toggles
A `preview:*` label configures an instance that already exists, so adding or
removing one re-serves the box instead of creating or deleting it. It does
nothing on a PR without `codespace-preview`.
The vocabulary lives in `scripts/codespace-preview/preview-labels.mjs`, which both ends import:
`preview.mjs` turns the PR's labels into slugs, and `preview-serve.mjs` turns
those slugs into environment inside the box. Add a toggle there, in one place.
Two things cross the gap, and both are shape-checked rather than trusted: a
`preview:*` slug, and a phase key from `scripts/codespace-preview/preview-phases.mjs`. The `gh
codespace ssh` command is a shell string that appears in the box's process list, so
a value is never passed through it — `preview:enterprise` resolves to a licence key
inside the box, not on the runner.
`preview:enterprise` needs a **Codespaces** secret named
`N8N_LICENSE_ACTIVATION_KEY`, scoped to `n8n-io/n8n`. That is a Codespaces
secret, not an Actions secret, and it is unrelated to `CODESPACE_PREVIEW_TOKEN`.
Use the sandbox key: the preview sets tenant `1001` to match, and the default
tenant (`1`) rejects it. Without the secret the preview still serves, unlicensed,
and says so in the log.
Note what the label does and does not control. Codespaces injects the secret into
**every** preview box, so the label decides whether the licence reaches n8n, not
whether the key reaches the box. Anyone who can run PR-head code can read
`/workspaces/.codespaces/shared/.env-secrets`. Previews are limited to branches
in this repository, so that is the set of people who already have write access.
#### Preview environment from a webhook
A preview can also take environment from an n8n webhook we control, so a value
can change without a commit and a merge. `scripts/codespace-preview/preview-remote-env.mjs` fetches
it, and `preview-serve.mjs` hands the result to the backend.
It needs three **Codespaces** secrets on `n8n-io/n8n`, again not Actions secrets:
| Secret | Purpose |
| ------------------------ | ------------------------------------------------- |
| `CODESPACE_ENV_URL` | The webhook URL |
| `CODESPACE_ENV_USER` | Basic auth user. Optional, defaults to `preview`. |
| `CODESPACE_ENV_PASSWORD` | Basic auth password |
The webhook answers with a flat JSON object. Its keys become environment
variables and its values are used as-is, so a number or a boolean is stringified
and a nested object is dropped. The request carries the PR number and the head
SHA as query parameters, so one endpoint can answer per PR.
The fetch runs in the box, not on the runner. Codespaces secrets are unreadable
from Actions, so neither the password nor a returned value can reach a CI log.
The log prints key names only.
**Every key is passed through.** A response containing `NODE_OPTIONS`,
`EXTERNAL_HOOK_FILES` or `PATH` runs code inside the preview box, so whoever can
edit that workflow can run code there. The one exception is the preview's own
wiring — the sign-in hook and the owner credentials — which is applied last and
wins. The `.env-secrets` note above applies to these secrets too.
Nothing here is required. Without the secrets, an unreachable webhook or a
rejected password, the preview serves as usual and says so in the log. A wrong
password is not retried: it cannot fix itself.
#### The `CODESPACE_PREVIEW_TOKEN` secret
The job needs `CODESPACE_PREVIEW_TOKEN`, a **fine-grained** personal access token,
held in the `codespaces` environment. Set the resource owner to `n8n-io` and limit
repository access to `n8n-io/n8n`. Grant these repository permissions:
| Permission | Level | What it unlocks |
|---|---|---|
| Metadata | Read | Mandatory, selected for you |
| Codespaces | Read and write | Create, list and delete a box |
| Codespaces metadata | Read | `GET .../codespaces/machines`, which resolves the machine type |
| Codespaces lifecycle admin | Read and write | Start a stopped box, which is what `gh codespace ssh` does |
| Contents | Read | Read the repository |
| Pull requests | Read | `gh pr view`, to resolve the head ref and SHA |
`Codespaces metadata` is a **different permission** from `Codespaces`. Without it
the run fails with `HTTP 403: Resource not accessible by personal access token` on
the `machines` endpoint, after the org billing check has already printed a tick —
so the failure looks unrelated to permissions.
An organization owner may have to approve the token. It stays pending until then.
No other credential can do this:
- `GITHUB_TOKEN` has no Codespaces access.
- A GitHub App **installation** token cannot create a codespace at all. The
Codespaces API belongs to a user, not to an installation.
- A **classic** token is refused by org policy:
`` `n8n-io` forbids access via a personal access token (classic) ``. So no
combination of classic scopes works, whatever the API reference says about the
`codespace` scope.
The environment keeps the token away from every other workflow: only a job that
names `codespaces` can read it. **The environment must allow every branch.** A
`pull_request` run has the ref `refs/pull/<n>/merge`, which no deployment branch
policy matches, so a branch rule would block every preview. Add required
reviewers only if a click for each preview is acceptable.
The codespace belongs to whoever owns the token, and shows up in that account's
codespace list. Billing still goes to the organization, because the repository is
organization-owned and has a Codespaces budget. A service account is therefore
better than a person's account for quota attribution, though the token is scoped
to one repository either way.
The job checks out the base branch, never the PR head, so a PR cannot supply the
script that reads that token. A manual run checks out the branch chosen in the
Run-workflow dropdown, which only a user with write access can pick.
**Why:** Re-run tests without pushing commits. Useful for flaky test investigation.
### Other Manual Workflows
| Workflow | Purpose |
|-----------------------------|---------------------------------------------------------|
| `util-codespace-preview.yml`| Wake, re-serve or delete a PR preview instance by hand |
| `util-data-tooling.yml` | SQLite/PostgreSQL export/import validation (manual) |
| `test-image-efficiency.yml` | Measure n8n and runners image size and layer waste with Dive |
| `util-probe-registry.yml` | Diagnose slow npm metadata fetches (temporary) |
| `sec-sync-bundle-branches.yml` | Sync both bundle branches on demand, including before a bundle cut |
| Workflow | Purpose |
|---------------------------|---------------------------------------------------------|
| `util-claude-task.yml` | Run Claude Code to complete a task and create a PR |
| `util-data-tooling.yml` | SQLite/PostgreSQL export/import validation (manual) |
#### Claude Task Runner (`util-claude-task.yml`)
Runs Claude Code to complete a task, then creates a PR with the changes. Use for well-specced tasks or simple fixes. Can be triggered via GitHub UI or API.
Claude reads templates from `.github/claude-templates/` for task-specific guidance. Add new templates as needed for recurring task types.
**Inputs:**
- `task` - Description of what Claude should do
- `user_token` - GitHub PAT (PR will be authored by the token owner)
**Token requirements** (fine-grained PAT):
- Repository: `n8n-io/n8n`
- Contents: `Read and write`
- Pull requests: `Read and write`
**Governance:** If you provide your personal PAT, you cannot approve the resulting PR. For automated/bot use cases (e.g., dependabot-style updates via n8n workflows), an app token can be used instead.
---
@ -434,19 +288,22 @@ ci-pull-requests.yml
ci-master.yml
├──────────────────────────▶ test-unit-reusable.yml
├──────────────────────────▶ test-linting-reusable.yml
└──────────────────────────▶ test-single-instance-npm.yml
└──────────────────────────▶ test-linting-reusable.yml
release-publish.yml
├──────────────────────────▶ docker-build-push.yml
│ └──────────▶ security-trivy-scan-callable.yml
└──────────────────────────▶ sbom-generation-callable.yml
test-sbom-nightly.yml
└──────────────────────────▶ sbom-validation-callable.yml
test-workflows-nightly.yml (manual dispatch only — nightly schedule disabled, DEVP-544)
└──────────────────────────▶ test-workflows-callable.yml
test-e2e-vm-expressions-nightly.yml
└──────────────────────────▶ test-e2e-reusable.yml
PR Comment Dispatchers (triggered by /command in PR comments):
test-workflows-pr-comment.yml
└──────────────────────────▶ test-workflows-callable.yml
```
---
@ -487,10 +344,7 @@ test-workflows-nightly.yml (manual dispatch only — nightly schedule disabled,
│ │ ├─ ensure-provenance-fields.mjs ───▶ Add license fields │
│ │ └─ npm publish (tag: rc or latest) │
│ ├─ publish-to-docker-hub ────────▶ docker-build-push.yml │
│ │ ├─ Build the application once on amd64 │
│ │ ├─ Build/push images on native amd64 + arm64 runners │
│ │ └─ Merge manifests, then add release provenance │
│ │ └─ Release VEX/SBOM attestations follow provenance │
│ │ └─ Multi-arch: amd64 + arm64 │
│ ├─ create-github-release │
│ ├─ create-sentry-release (sourcemaps) │
│ ├─ generate-sbom ────────────────▶ sbom-generation-callable.yml │
@ -507,42 +361,12 @@ test-workflows-nightly.yml (manual dispatch only — nightly schedule disabled,
└────────────────────────────────────────────────────────────────────────────┘
```
### Recovering a failed release
If the pipeline publishes `n8n@X.Y.Z` to npm and then fails, that version is
burned — npm versions are immutable. Recovery depends on how far it got:
| Failure point | Recovery |
|---|---|
| Before `n8n` reached npm | **Re-run failed jobs** on the original run. Sub-packages publish before `n8n` and `pnpm publish -r` skips versions already on npm, so a retry is safe. |
| After `n8n` reached npm | Dispatch **`release-recreate-failed-release.yml`** with `failed-version: X.Y.Z`. |
`release-recreate-failed-release.yml` checks out `release/X.Y.Z`, bumps only the
root and `packages/cli` versions to `X.Y.(Z+1)`, pushes `release/X.Y.(Z+1)` from
the same commit, and opens an auto-merging `release-pr/X.Y.(Z+1)`. Merging it
fires `release-publish.yml` as normal — `release-publish.yml` has no
`workflow_dispatch`, so a release PR is the only way to re-drive it.
Only those two `package.json` files move: the root version drives every publish
output (git tag, Docker tags, GitHub Release, SBOM) and `packages/cli` drives
the runtime `N8N_VERSION`. Every other package keeps its version, so the publish
step skips the ones already on npm and publishes whichever ones the failed run
never reached.
It refuses to run unless `n8n@X.Y.Z` is on npm and `n8n@X.Y.(Z+1)` is not. That
check fails closed: if the registry can't be reached, the run stops rather than
guessing. Dispatch with `force: true` to skip it.
The burned version stays on npm. Deprecate it by hand once the re-release is
out: `npm deprecate n8n@X.Y.Z "Failed release, use X.Y.(Z+1)"`.
### Other Release Workflows
| Workflow | Trigger | Purpose |
|-------------------------------------------|-----------------|------------------------------------------------|
| `release-standalone-package.yml` | Manual dispatch | Release individual packages (@n8n/codemirror-lang, @n8n/create-node, etc.) |
| `release-create-patch-pr.yml` | Manual dispatch | Open a patch release PR for one track |
| `release-recreate-failed-release.yml` | Manual dispatch | Re-release a version whose publish failed after it reached npm |
| Workflow | Trigger | Purpose |
|----------------------------------|--------------------|------------------------------------------------|
| `release-standalone-package.yml` | Manual dispatch | Release individual packages (@n8n/codemirror-lang, @n8n/create-node, etc.) |
| `create-patch-release-branch.yml`| Manual dispatch | Cherry-pick commits for patch releases |
---
@ -564,16 +388,14 @@ out: `npm deprecate n8n@X.Y.Z "Failed release, use X.Y.(Z+1)"`.
## ci-master.yml
Runs on push to `master`:
Runs on push to `master` or `1.x`:
```
Push to master
├─ build-and-format (Blacksmith: build, then format check; populate master cache)
├─ unit-test (matrix: Node 24.18.1, 26.5.1)
│ └─ Coverage only on 24.18.1
Push to master/1.x
├─ build-github (populate cache)
├─ unit-test (matrix: Node 22.x, 24.16.0, 26.x)
│ └─ Coverage only on 24.16.0
├─ lint
├─ (performance: CodSpeed benchmarks, paused until the plugin supports Vitest 5)
├─ verify-single-instance-npm (advisory; packages changed by this push)
└─ notify-on-failure (Slack #alerts-build)
```
@ -583,56 +405,37 @@ Push to master
| Schedule (UTC) | Workflow | Purpose |
|---------------------------|-----------------------------------|--------------------------|
| Hourly :00 | `sec-sync-public-to-private.yml` | Mirror public → private |
| Daily 03:00 | `sec-sync-bundle-branches.yml` | Merge the base into `bundle/*` |
| Daily 00:00 | `docker-build-push.yml` | Nightly Docker images |
| Daily 00:00 | `test-db.yml` | Database compatibility |
| Daily 00:00 | `test-e2e-performance-reusable.yml`| Browser performance and idle memory baselines |
| Daily 00:00 | `test-e2e-performance-reusable.yml`| Performance E2E |
| Daily 00:00 | `release-storybook.yml` | Storybook deploy |
| Daily 00:00 | `release-chromatic.yml` | Visual regression |
| Daily 00:00 | `util-check-docs-urls.yml` | Doc link validation |
| Daily 01:30, 02:30, 03:30 | `test-benchmark-nightly.yml` | Performance benchmarks |
| Daily 02:00 | `test-get-n8n.yml` | get.n8n.io installer health |
| Daily 02:00 | `test-e2e-pc-nightly.yml` | E2E on the `-pc` image |
| Daily 04:00 | `test-sbom-nightly.yml` | Release and image SBOM license validation |
| Daily 04:00 | `test-e2e-vm-expressions-nightly.yml`| VM expression E2E |
| Daily 05:00 | `test-benchmark-destroy-nightly.yml`| Cleanup benchmark env |
| Daily 06:00 | `util-sync-master-to-3x.yml` | Replay 3.x onto master (v3) |
| Daily 08:00 | `build-v3-nightly.yml` | Nightly v3 Docker images |
| Monday 00:00 | `util-update-node-popularity.yml` | Node usage stats |
| Monday 02:00 | `test-e2e-coverage-weekly.yml` | Weekly E2E coverage |
| Wednesday 04:00 | `util-update-get-n8n-versions.yml` | get-n8n stable versions |
| Saturday 22:00 | `test-evals-ai.yml` | AI workflow evals |
| 1st of month 04:00 | `util-refresh-cubic-schema.yml` | Refresh vendored cubic schema |
---
## v3 development (master + 3.x)
The sync runs automation code from the triggering `master` SHA while its working checkout
stays on `3.x`.
During the v3 release window, `master` carries normal feature work (behind opt-in
flags) and the long-lived `3.x` branch carries breaking changes. `util-sync-master-to-3x.yml`
syncs daily by **replaying the `3.x`-only commits onto `master` and force-pushing `3.x`**, so a
clean sync adds no commit and nothing is squashed. What it pushes is always verified to be
exactly the tree a merge of `3.x` and `master` produces, and marker-free. Conflicts confined
to non-lockfile mechanical files (bot-maintained data files — see `MECHANICAL_PATHS` in
`sync-master-to-3x.mjs`) are auto-resolved during the replay. On a code or `pnpm-lock.yaml`
conflict, `3.x` is left untouched and a draft PR carrying the conflict markers (labeled
`automation:v3-sync`, with other mechanical files pre-resolved) is opened on
`sync/master-to-3x`. The lockfile is always left for the resolver. The PR names both ends of
the conflict — the breaking-commit authors and the `master` commits that touched the same
files — via `sync-conflict-owners.mjs`, posts to `#alerts-v3-sync`, and pauses further syncs
until it is resolved and merged normally. Delete/modify conflicts have no markers to carry,
so they are resolved toward `3.x` and listed as an explicit decision in the PR body.
exactly the tree a merge of `3.x` and `master` produces, and marker-free. On a real conflict
`3.x` is left untouched and a draft PR carrying the conflict markers (labeled
`automation:v3-sync`) is opened on `sync/master-to-3x`, requesting the breaking-commit authors
as reviewers via `sync-conflict-owners.mjs`, posting to `#alerts-v3-sync` and pausing further
syncs until it is resolved and merged normally.
`build-v3-nightly.yml` publishes `n8nio/n8n:v3-nightly[-<date>]` images from `3.x`
by calling `docker-build-push.yml` with `ref: 3.x` + `date_tag`. On Mondays it also
retags that run's n8n + runners manifests as a release candidate (by digest on GHCR, so
the RC is exactly what was built), giving a self-consistent set to trial. Any manual run
can promote too via the `force_rc` dispatch input, several times a day: each publish
claims the next free `v3-rc-<date>.N` as its immutable tag and moves the floating `v3-rc`
and `v3-rc-<date>` onto it. The counter is derived by probing the registry, and the job
is serialized on a `v3-rc-tagging` concurrency group so two runs can't claim one number.
by calling `docker-build-push.yml` with `ref: 3.x` + `date_tag`.
See **[`DEVELOPING_V3.md`](./DEVELOPING_V3.md)** for the full model.
@ -645,67 +448,17 @@ Composite actions in `.github/actions/`:
| Action | Purpose | Used By |
|--------------------------|----------------------------------------------|--------------------|
| `setup-nodejs` | pnpm + Node.js + Turbo cache + Docker (opt) | Most CI workflows |
| `run-workflow-script` | Run a `.github/scripts` module with no setup or install | Owners and PR quality checks |
| `docker-registry-login` | GHCR + DockerHub + DHI authentication | Docker workflows |
### setup-nodejs
```yaml
inputs:
node-version: # default: '26.5.1'
node-version: # default: '24.16.0'
enable-docker-cache: # default: 'false' (Blacksmith Buildx)
docker-cache-key: # required when enable-docker-cache is true
build-command: # default: 'pnpm build'
```
The pnpm version comes from the `packageManager` field in the root
`package.json`, through `resolve-pnpm-version.mjs`. There is no version input to
keep in sync: a bump in `package.json` moves the setup step, the cache key and
the version check together.
The action caches the pnpm executable in `~/setup-pnpm` by OS, architecture and
version. A cache miss runs `pnpm/setup` and retries once if the registry request
fails. The action verifies the version and saves the cache before the rest of
the job can fail.
Concurrent jobs can all miss a new key before the first save completes. Jobs
that start after the save use the cached executable. Windows keeps the standard
`pnpm/setup` path because its runner cannot activate the cached POSIX home path.
The existing `actions/setup-node` cache continues to store the pnpm package
store.
The Blacksmith layer cache lives on a sticky disk identified by
`docker-cache-key`, and commits are last-writer-wins. Splitting the key per
image would avoid that, but Blacksmith currently never populates a
newly created sticky disk - it stays at 0 bytes however many runs commit to it,
while the build reports a successful commit. Every job therefore shares the
`n8n-io/n8n` key, which is the only disk that actually retains layers. Revisit
once new-disk retention works.
### run-workflow-script
```yaml
inputs:
script: # path of the module, relative to the repository root
github-token: # token for the Octokit client, also exported as GITHUB_TOKEN
```
Runs one `.github/scripts` module through `actions/github-script`. That action
brings its own Node.js and an Octokit client, so the job needs no
`setup-nodejs` step and no dependency install. The action loads
`github-helpers.mjs`, hands the client to `setOctokit`, then imports the module
and awaits its exported `main()`.
Use it for a module that imports only node builtins and other `.github/scripts`
modules. `github-helpers.mjs` loads `@actions/github` and `semver` only when
they are present, so it works in both modes. A module that needs an npm
package (`semver`, `yaml`, `minimatch`, ...) keeps the `setup-nodejs` path with
the `.github/scripts` install command.
Pair it with a sparse checkout of `.github` when the module reads nothing else
from the tree. Cone mode always includes the root files, so `OWNERS` and
`package.json` stay available.
### docker-registry-login
```yaml
@ -715,18 +468,6 @@ inputs:
login-dhi: # default: 'false'
```
### External actions
Actions consumed from other n8n-io repositories, SHA-pinned like any third-party
action:
| Action | Purpose | Used By |
|-----------------------------------|-------------------------------------------------------------------------------|--------------------|
| `n8n-io/github-actions/cla-check` | CLA signature check: `CLA Check` commit status, in-place PR comment, `cla-signed` label | `ci-cla-check.yml` |
Behaviour changes belong in that repo; bumping the pin here is what picks them up.
A `/cla-check` comment on a PR re-runs the check without a push.
---
## Reusable Workflows
@ -739,14 +480,11 @@ Workflows with `workflow_call` trigger:
| `test-linting-reusable.yml` | `ref`, `nodeVersion` | ESLint |
| `test-e2e-reusable.yml` | `branch`, `test-mode`, `shards`, `runner` | Core E2E executor |
| `test-workflows-callable.yml` | `git_ref`, `compare_schemas` | Workflow tests |
| `docker-build-push.yml` | `n8n_version`, `release_type`, `push_enabled`, `ref`, `date_tag`, `create_attestations` | Docker build |
| `docker-build-push.yml` | `n8n_version`, `release_type`, `push_enabled`, `ref`, `date_tag` | Docker build |
| `sec-ci-reusable.yml` | `ref` | Security orchestrator |
| `sec-poutine-reusable.yml` | `ref` | Poutine scanner |
| `sec-sync-retarget-prs.yml` | none | Move bundle PRs back onto `bundle/*` |
| `security-trivy-scan-callable.yml` | `image_ref` | Trivy scan |
| `sbom-generation-callable.yml` | `n8n_version`, `release_tag_ref` | SBOM generation |
| `sbom-validation-callable.yml` | `sha` | Read-only SBOM validation |
| `test-single-instance-npm.yml` | `scope`, `base-ref`, `base-branch`, `blocking`, `timeout-minutes` | Dependency duplication |
---
@ -760,7 +498,6 @@ Scripts in `.github/scripts/`:
|-------------------------------|----------------------------|-------------------------|
| `bump-versions.mjs` | Calculate next version | `release-create-pr.yml` |
| `update-changelog.mjs` | Generate CHANGELOG | `release-create-pr.yml` |
| `prepare-rerelease.mjs` | Bump root + cli for a re-release | `release-recreate-failed-release.yml` |
| `trim-fe-packageJson.js` | Strip frontend devDeps | `release-publish.yml` |
| `ensure-provenance-fields.mjs`| Add license/author fields | `release-publish.yml` |
@ -770,10 +507,6 @@ Scripts in `.github/scripts/`:
|-------------------------|-------------------|------------------------|
| `docker/docker-config.mjs`| Build context | `docker-build-push.yml`|
| `docker/docker-tags.mjs` | Image tags | `docker-build-push.yml`|
| `docker/kafka-native-smoke-check.mjs`| Verify librdkafka binary loads in built image | `docker-build-smoke.yml`|
| `docker/assert-manifest-format.mjs`| Assert a merged manifest is an OCI image index with the expected platforms | `docker-build-push.yml`|
| `docker/should-smoke-build.mjs`| Narrow the `pnpm-workspace.yaml` smoke trigger to native dependency pins | `docker-build-smoke.yml`|
| `attest-image-sbom.mjs` | Generate, validate, and optionally attest image SBOMs | `docker-build-push.yml`, `test-sbom-nightly.yml` |
### Validation Scripts
@ -781,39 +514,6 @@ Scripts in `.github/scripts/`:
|-------------------------|-------------------|---------------------------|
| `validate-docs-links.js`| Check doc URLs | `util-check-docs-urls.yml`|
| `send-build-stats.mjs` | Build telemetry | `setup-nodejs` action |
| `resolve-pnpm-version.mjs` | Publish the pinned pnpm version and its executable cache key | `setup-nodejs` action |
| `nightly-sbom-context.mjs` | Resolve the source SHA and image tag for nightly SBOM validation | `test-sbom-nightly.yml` |
| `db-test-matrix.mjs` | DB test matrix from `postgres-versions.json` | `ci-pull-requests.yml` |
| `quality/check-cubic-config.mjs` | Validate `cubic.yaml` against the vendored cubic schema; enforce its silent agent/character limits. `--refresh` re-pulls the schema | `test-workflow-scripts-reusable.yml`, `util-refresh-cubic-schema.yml` |
| `glob.mjs` | Builtin-only glob matcher for changed-file paths (`**`, `*`, dot segments) | `quality/check-pr-size.mjs` |
| `probe-registry.mjs` | Registry path throughput probe (temporary) | `util-probe-registry.yml` |
### Preview Scripts
| Script | Purpose | Called By |
|---------------------------------|-------------------------------------------------------------------------|--------------------------------|
| `codespace-preview.mjs` | Map a `pull_request` event or a manual operation onto a preview operation, comment the result | `util-codespace-preview.yml` |
| `../../scripts/codespace-preview/preview.mjs` | One codespace for each PR: `up`, `refresh`, `down`, `ls`. `--json` for CI | `codespace-preview.mjs`, developers |
| `../../scripts/codespace-preview/preview-remote-env.mjs` | Fetch extra environment for a preview from the webhook, inside the box | `../../scripts/codespace-preview/preview-serve.mjs` |
| `../../scripts/codespace-preview/preview-phases.mjs` | The phase vocabulary and its one-line marker, so the runner, the box and the comment cannot drift | `codespace-preview.mjs`, `../../scripts/codespace-preview/preview.mjs`, `../../scripts/codespace-preview/preview-serve.mjs` |
`scripts/codespace-preview/preview.mjs` is also the developer entry point (`pnpm preview up <pr>`).
In `--json` mode stdout carries one line for each phase and then the report object,
and all human progress goes to stderr. So a workflow can follow a run that also
streams an in-box build log. The reader tells the two apart by the `url` field: the
report has one, a phase line never does.
### Branch Replay Scripts
Both keep a long-lived branch that is "base + its own commits" in sync by rebasing those
commits onto the base and force-pushing, sharing the merge-tree content guard that makes the
rewrite safe.
| Script | Purpose | Called By |
|----------------------------|----------------------------------------------------------------------|------------------------------------|
| `branch-replay.mjs` | Shared primitives: merge-tree, tree guard, marker scan | the two scripts below |
| `sync-master-to-3x.mjs` | master → `3.x`, rebased; auto-resolves mechanical files, opens a conflict PR | `util-sync-master-to-3x.yml` |
| `sync-bundle-branch.mjs` | base → `bundle/*` in n8n-private, merged; skips while the base is unpublished; fail-loud, never resolves conflicts | `sec-sync-bundle-branches.yml` |
### Slack Scripts
@ -838,92 +538,13 @@ See **[CI-TELEMETRY.md](CI-TELEMETRY.md)** for:
---
## OWNERS
## CODEOWNERS
Team ownership lives in the top-level `OWNERS` file. The scripts that consume
it live in `.github/scripts/owners/`. Line format:
Team ownership mappings in `CODEOWNERS`:
```
<pattern> <@org/team> [required]
```
Patterns are a catch-all (`*`), a directory prefix (`packages/x/`), or an
exact file path. Matching is last-match-wins, so specific rules must come
after general rules.
The format is strict, enforced by `node .github/scripts/owners/owners.mjs --check`:
- Tokens on a line come in a fixed order: pattern, one team, then options
such as `required`.
- Directory patterns end with `/` and must be existing directories; all other
patterns must be existing files. Duplicate patterns are rejected.
Team existence is not checked by `--check` (it needs an org read token, which
fork PRs do not have); a separate workflow covers it (DEVP-891).
The file drives four workflows:
| Workflow | Purpose |
|-----------------------------------|--------------------------------------------------------------------------|
| `ci-owners-validation.yml` | Validates OWNERS (syntax, dead paths) via `owners.mjs --check` |
| `ci-owners-review-recommendations.yml` | Advisory PR comment: reviewer teams, line stats, required reviews |
| `ci-owners-assign-reviewers.yml` | Opt-in reviewer auto-assignment (label-triggered) |
| `ci-owners-required-reviews.yml` | Enforces `required` entries via the "Required Reviews" commit status |
### Required reviews
An entry with the `required` option makes team approval mandatory: when a PR
changes a non-test file whose winning entry carries `required`, a member of each
listed team must approve the PR. Test files match the shared patterns in
`test-files.mjs` and do not trigger required reviews.
`ci-owners-required-reviews.yml` evaluates this on
PR changes and review events, and reports a commit status
named **Required Reviews** on the head SHA. A missing approval reports
`pending` ("Waiting for approval from: …"), not `failure`, so an unreviewed PR
does not show red CI; any non-success state blocks the merge equally. The
ruleset for a branch must list that status as a required check for the block
to take effect. Merge-queue runs report success on the queue head without
re-evaluating: a PR cannot enter the queue unless the status is green on its
head, and the queue does not change approvals.
The status is evaluated for a PR into any base branch, from a same-repo head
or a fork. Both matter because the ruleset that gates a PR is not always the
one on its base branch: GitHub applies the ruleset of a stack's target branch
to every PR in the stack, so a stacked PR into a feature branch is gated by
the `master` ruleset. A required status that no run ever writes leaves the PR
blocked on "Expected". Routes that skip the evaluation are listed in
`REQUIRED_REVIEW_EXEMPTIONS` in
`required-reviews.mjs`. An entry is `<head> -> <base>` or just `<base>`
(any head); `*` matches any run of characters. An exempt PR reports
`success` with the route in the description. Only heads in this repository
can match, so a fork branch with a matching name is still evaluated. Add a
route only when every commit it carries was already reviewed elsewhere, as
with `sync/master-to-3x -> 3.x`: its commits landed on `master` first.
The workflow reads OWNERS, its scripts and the exemption routes from `master`
only, never from the base branch or the PR: any writable branch can be a base,
so only `master` is trusted input. A PR cannot lift its own review
requirement. A retarget re-evaluates the PR, so a verdict computed against the
old base does not carry over.
Members of the `large-scale-changes` GitHub team can make the **Required
Reviews** status succeed without required OWNERS team approvals. A team member
must apply the `large-scale-change` label. The gate checks the actor from the
label event and verifies their current team membership. The label alone is not
sufficient. Label changes re-evaluate the status. Removing the label removes
the exemption.
Every path that writes the status runs in the base repository context, because
a fork-context run has no secrets and a read-only token. PR changes arrive
through `pull_request_target`, which is safe here because no step checks out
or runs PR code. Review events on a same-repo PR arrive through
`pull_request_review`. Review events on a fork PR arrive through the
`workflow_run` of `ci-pull-request-review.yml`, which runs on every submitted
or dismissed review; the owners workflow looks the PR up from that run's head
and skips same-repo heads, which the direct event already covers. A first
contribution whose runs still wait for approval gets no review-event
re-evaluation until a maintainer approves the runs; `workflow_dispatch` with
the PR number is the manual fallback.
| Path Pattern | Team |
|--------------------------------------------------------------|----------------------------|
| `packages/@n8n/db/src/migrations/` | @n8n-io/migrations-review |
---
@ -948,7 +569,7 @@ the PR number is the manual fallback.
**`blacksmith-4vcpu-ubuntu-2204`** - Unit tests (parallelized), linting (parallel file processing), typechecking (CPU-intensive), E2E test shards
**`blacksmith-8vcpu-ubuntu-2204`** - Heavy parallel workloads. The PR `install-and-build` job uses it with the default runner provider. Most PR jobs wait for that job, and a cold build keeps all 4 vCPUs of a smaller runner busy.
**`blacksmith-8vcpu-ubuntu-2204`** - Heavy parallel workloads
### Runner Provider Toggle
@ -1008,50 +629,10 @@ Supply chain security ensures artifacts haven't been tampered with. We provide t
### SBOM
There are two, with different subjects and different consumers. They are not duplicates.
| | Release SBOM | Image SBOM |
|---|---|---|
| **Job** | `generate-and-attach-sbom` (`sbom-generation-callable.yml`) | `sbom-attestation` (`docker-build-push.yml`) |
| **Scans** | the deployed npm closure in `compiled/` (`cdxgen -t pnpm`) | each pushed image, by digest (`syft`) |
| **Covers** | npm only | OS packages **and** npm, as laid down in the image |
| **Signing** | GitHub Attestation API, subject `./package.json` | `cosign attest`, subject = image digest |
| **Output** | `sbom-source.cdx.json`, `THIRD_PARTY_LICENSES.md`, `vex.openvex.json` on the GitHub Release | attestation in the registry beside the image |
| **Consumer** | humans — legal/license compliance; backs `/third-party-licenses` | machines — `cosign verify-attestation`, admission control |
Format is CycloneDX JSON 1.6 for both. Each pipeline pins the schema version, so a
scanner upgrade cannot change the shape of a signed artifact without a visible diff.
The two use different scanners on purpose. The release SBOM runs `cdxgen -t pnpm` over the
resolved pnpm closure with `FETCH_LICENSE=true`, because a lockfile scan has no package files
to read licenses from. The image SBOM runs `syft` over the pushed image, which resolves
licenses from the LICENSE files on disk and so needs no network at all.
The image job used to run `cdxgen -t docker --profile license-compliance`. That profile sets
`FETCH_LICENSE=true` and nothing else, so it made one sequential npm registry call per
component — roughly 3,700 per release, about half the job's runtime. syft resolves the same
licenses locally in a fraction of the time, and catalogues more of the image besides.
A/B any scanner change against the current output before shipping it. The gate only enforces
`pkg:npm/`, so a change can silently degrade PyPI or OS license coverage while CI stays green.
Compare the licenses resolved per component, not just the component counts.
`enrich-sbom.mjs --drop-phantom-npm` removes scan artefacts that would otherwise assert
components the image does not contain: nested test/fixture `package.json` and `exports`
subpaths. It reads the component's source path from either scanner's property name (`SrcFile`
for cdxgen, `syft:location:0:path` for syft) and treats syft's `version: "UNKNOWN"` the same as
a missing version.
Packages whose license cannot be resolved from disk go in
`scripts/licenses/license-overrides.json` with a verified `source` citation — the upstream
LICENSE file, not registry metadata.
`test-sbom-nightly.yml` runs at 04:00 UTC. It waits up to two hours for the current scheduled
Docker build to complete. It builds the production deployment closure at that run's SHA and
validates the release SBOM. It also resolves the four immutable SHA image tags from that
build and validates each image SBOM. The validation uses the same enrichment and SPDX gates
as a release. It does not publish, attest, or upload an artifact. A failure reports to the
Developer Platform Slack channel.
- **Runs on:** release-publish
- **Format:** CycloneDX JSON
- **Signing:** GitHub Attestation API
- **Attached to:** GitHub Release
### SLSA L3 Provenance
@ -1120,78 +701,6 @@ cosign verify-attestation --type openvex \
}
```
### Public ↔ private sync (bundle branches)
Embargoed security work happens in `n8n-io/n8n-private`. `sec-sync-public-to-private.yml`
runs hourly there (and on `workflow_dispatch` with `force` for conflict recovery),
mirroring public `master` and `1.x` into private with `reset --hard` +
`--force-with-lease` — skipping a branch when private is ahead, ignoring `chore: Bundle`
commits when judging "ahead". A skipped branch, or a hard failure, is reported to
`#alerts-build`: a non-`chore: Bundle` commit on private `master`/`1.x` leaves the mirror
stuck every hour until it is removed or a `force` dispatch overwrites it. Fixes are never
committed to private `master`/`1.x` directly: `ci-restrict-private-merges.yml` requires
PRs into them to come from the long-lived integration
branches `bundle/2.x` and `bundle/1.x` (a `bundle/2.x` merge is
backported to `bundle/1.x` by `util-backport-bundle.yml`). Once a bundle branch is merged
into private `master`/`1.x` as a `chore: Bundle/*` PR, `sec-publish-fix.yml` /
`sec-publish-fix-1x.yml` cherry-pick that commit onto a fresh branch in the public repo and
open the PR there. That PR **must stay a single-parent squash** — the publish step is a bare
`git cherry-pick` of `HEAD`, which aborts on a merge commit. A `chore: Bundle/*` PR whose
*Required Checks* go red holds back every fix batched into it, so `ci-pull-requests.yml`
posts to `#alerts-build` when that gate fails on a PR opened *from* `bundle/2.x` or
`bundle/1.x` (link only, no PR title, since the branch is embargoed).
`sec-sync-bundle-branches.yml` keeps those branches current nightly at 03:00 UTC and on
`workflow_dispatch`, including before a bundle cut. It **merges the public base into** the
bundle branch via
[`scripts/sync-bundle-branch.mjs`](scripts/sync-bundle-branch.mjs) and pushes without forcing.
Every push is verified to carry exactly the tree a merge of the two sides would produce (`git
merge-tree`); a mismatch, or a conflict marker, fails the run instead of pushing.
**A bundle branch is only ever built on published history.** The sync fetches `master` or `1.x`
directly from `https://github.com/n8n-io/n8n.git`. It creates a missing bundle branch at that
exact public SHA and merges that public SHA into an existing bundle branch. It never reads the
private base branch. This matters because the `chore: Bundle/*` squash on private `master` is
private-only until publication. The mirror can replace that commit with the public copy after
the bundle branch is deleted. The next bundle branch must not inherit the replaced commit.
Deleting a bundle branch takes its open PRs with it: GitHub moves each one onto the deleted
branch's own base, and re-creating the branch does not move them back. The next nightly or manual
sync re-creates the branch and then calls
[`sec-sync-retarget-prs.yml`](workflows/sec-sync-retarget-prs.yml), which moves every open PR
on `master` onto `bundle/2.x` and every one on `1.x` onto `bundle/1.x`. It skips a bundle
branch that does not exist, and skips PRs whose *head* is `bundle/*` — those are the
`chore: Bundle/*` cut PRs, which target the base on purpose. It is also dispatchable on its
own. The retarget runs inside the bundle sync after a missing branch is recreated.
**`bundle/*` is append-only — never rebase it, never force-push it.** These branches receive
PRs, and rewriting a branch that receives PRs orphans the copies of its commits that the open
PR branches already contain: every such PR's merge base regresses to an old base commit, so
GitHub shows it carrying everyone else's fixes, in the commit list *and* in the diff (which
can then trip required checks like *PR Size Limit*). It compounds — each refresh between
rewrites picks up another duplicate generation of the same fixes and starts conflicting with
itself. To refresh a fix branch, use GitHub's **Update branch** button or
`git merge origin/bundle/2.x`; squash-merging a fix *into* the bundle branch leaves every
sibling PR's merge base untouched, which is why only a rewrite breaks this.
The costs of merging are deliberate and paid for: a merge commit per run, and fixes that have
already been published staying in the branch's log. Neither reaches anything downstream,
because a bundle publishes as one squashed commit taken from the tree rather than the history —
the `chore: Bundle/*` PR's **diff** stays
exactly the pending fixes even when its commit list does not. For a list of what a bundle
actually carries, read the fix PRs merged into the branch since the last cut, not `base..bundle`.
There is **one job per bundle branch**. A conflict is detected from the merge tree before the
working tree is touched, so the branch is left exactly as it was, that job **fails** (no
green runs hiding a stalled branch) and `#alerts-security` gets a run link — while the other
branch still syncs. Recovery is deliberate: merge the public base into the branch locally,
resolve, push, then re-run the workflow — and that resolution then lives in the merge commit
instead of being re-litigated on every later run. The sync never resolves a conflict itself,
unlike `util-sync-master-to-3x.yml`.
See **[`../AGENTS.md`](../AGENTS.md)** ("Security Fix Hygiene") for the naming rules that
keep the vulnerability out of public branch names, commits, and test descriptions.
---
## Slack Notifications
@ -1228,7 +737,7 @@ If notify is a step inside an existing checked-out job, skip the `checkout` and
| `QBOT_SLACK_TOKEN` | QBot | Default — engineering / build / security |
| `RELEASE_HELPER_SLACK_TOKEN` | Release Helper | `#releases` (C036AELNMV0) |
Adding a new channel requires inviting the bot first; the first run otherwise fails loudly with `not_in_channel`. Private-repo workflows (`sec-publish-fix*.yml`, `sec-sync-public-to-private.yml`, and the bundle-PR alert in `ci-pull-requests.yml`) need `QBOT_SLACK_TOKEN` set in `n8n-io/n8n-private`; the scripts themselves are mirrored by `sec-sync-public-to-private.yml`.
Adding a new channel requires inviting the bot first; the first run otherwise fails loudly with `not_in_channel`. Private-repo workflows (`sec-publish-fix*.yml`) need `QBOT_SLACK_TOKEN` set in `n8n-io/n8n-private`; the scripts themselves are mirrored by `sec-sync-public-to-private.yml`.
---
@ -1245,7 +754,7 @@ Adding a new channel requires inviting the bot first; the first run otherwise fa
| Cloud/CDN | `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID` |
| GitHub Automation | `N8N_ASSISTANT_APP_ID`, `N8N_ASSISTANT_PRIVATE_KEY` |
| Benchmarking | `BENCHMARK_ARM_*`, `N8N_BENCHMARK_LICENSE_CERT` |
| AI/Evals | `EVALS_ANTHROPIC_KEY`, `EVALS_OPENAI_KEY`, `EVALS_OPENROUTER_KEY`, `EVALS_XAI_KEY`, `EVALS_BASETEN_KEY`, `EVALS_FIREWORKS_KEY`, `EVALS_TOGETHER_KEY`, `EVALS_DATABRICKS_KEY`, `EVALS_MODAL_KEY`, `EVALS_LYCEUM_KEY`, `EVALS_AZURE_FOUNDRY_KEY`, `EVALS_VERTEX_KEY`, `EVALS_VERTEX_PROJECT_ID`, `EVALS_VERTEX_LOCATION`, `EVALS_LANGSMITH_*` |
| AI/Evals | `ANTHROPIC_API_KEY`, `EVALS_LANGSMITH_*` |
### Scoping
@ -1257,6 +766,12 @@ Adding a new channel requires inviting the bot first; the first run otherwise fa
## Future Vision
### Redundancy Review
Comment trigger (`/test-workflows`) is a workaround.
Long-term: Main CI should be reliable enough to not need these.
### Workflow Testability
- Tools like `act` for local testing

View File

@ -5,6 +5,4 @@ self-hosted-runner:
- blacksmith-2vcpu-ubuntu-2204-arm
- blacksmith-8vcpu-ubuntu-2204
- blacksmith-8vcpu-ubuntu-2204-arm
- blacksmith-32vcpu-ubuntu-2204
- blacksmith-32vcpu-ubuntu-2204-arm
- ubuntu-slim

View File

@ -12,7 +12,7 @@ description: 'Builds n8nio/n8n + n8nio/runners test images and publishes them as
inputs:
build-variant:
description: 'standard, coverage, or pc. Coverage uses build:docker:coverage. pc builds n8n on the pointer-compressed Node base images.'
description: 'standard or coverage. Coverage uses build:docker:coverage.'
required: false
default: 'standard'
@ -21,10 +21,10 @@ runs:
steps:
- name: Check cache for existing image
id: cache-check
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
uses: actions/cache/restore@640a1c2554105b57832a23eea0b4672fc7a790d5 # v4.2.3
with:
key: n8n-docker-image-v2-${{ inputs.build-variant }}-${{ github.sha }}
path: /tmp/n8n-images
key: n8n-docker-image-${{ inputs.build-variant }}-${{ github.sha }}
path: /tmp/n8n-image.tar.zst
lookup-only: true
- name: Build Docker image
@ -33,32 +33,19 @@ runs:
with:
build-command: ${{ inputs.build-variant == 'coverage' && 'pnpm build:docker:coverage' || 'pnpm build:docker' }}
enable-docker-cache: true
docker-cache-key: n8n-io/n8n
env:
INCLUDE_TEST_CONTROLLER: 'true'
DOCKER_BUILD_TARBALL_DIR: /tmp/n8n-images
DOCKER_BUILD_PC: ${{ inputs.build-variant == 'pc' }}
- name: Assert pointer compression
if: steps.cache-check.outputs.cache-hit != 'true' && inputs.build-variant == 'pc'
shell: bash
run: |
docker load -i /tmp/n8n-images/n8n-pc.tar
docker run --rm --entrypoint node n8nio/n8n:local -e "
if (!process.config.variables.v8_enable_pointer_compression) {
throw new Error('pc variant image is not pointer-compressed');
}
console.log(process.version, 'pointer compression: on');
"
- name: Report tarball sizes
- name: Save image tarball
if: steps.cache-check.outputs.cache-hit != 'true'
shell: bash
run: ls -lh /tmp/n8n-images
run: |
docker save n8nio/n8n:local n8nio/runners:local | zstd -T0 -3 -o /tmp/n8n-image.tar.zst
ls -lh /tmp/n8n-image.tar.zst
- name: Publish image tarball to cache
if: steps.cache-check.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
uses: actions/cache/save@640a1c2554105b57832a23eea0b4672fc7a790d5 # v4.2.3
with:
key: n8n-docker-image-v2-${{ inputs.build-variant }}-${{ github.sha }}
path: /tmp/n8n-images
key: n8n-docker-image-${{ inputs.build-variant }}-${{ github.sha }}
path: /tmp/n8n-image.tar.zst

View File

@ -13,7 +13,6 @@ import {
getChangedFiles,
getAddedFiles,
getMergeBase,
resolveHeadRef,
} from '../ci-filter.mjs';
// --- matchGlob ---
@ -58,15 +57,6 @@ describe('matchGlob', () => {
);
});
it('backend module pattern matches packaged module files', () => {
assert.ok(
matchGlob(
'packages/modules/insights/backend/src/insights.service.ts',
'packages/modules/*/backend/**',
),
);
});
it('* matches single-level only', () => {
assert.ok(matchGlob('README.md', '*.md'));
assert.ok(!matchGlob('docs/README.md', '*.md'));
@ -251,7 +241,7 @@ describe('runtime filter', () => {
'!**/CHANGELOG.md',
'!**/*.test.ts',
'!**/*.spec.ts',
'!packages/quality/testing/playwright/**',
'!packages/testing/playwright/**',
'!packages/frontend/@n8n/storybook/**',
'!scripts/agent-setup.mjs',
'!scripts/backend-module/**',
@ -274,7 +264,7 @@ describe('runtime filter', () => {
it('does not trigger on playwright tests', () => {
assert.equal(
evaluateFilter(['packages/quality/testing/playwright/tests/x.spec.ts'], runtimePatterns),
evaluateFilter(['packages/testing/playwright/tests/x.spec.ts'], runtimePatterns),
false,
);
});
@ -405,100 +395,6 @@ describe('getChangedFiles', () => {
});
});
// --- resolveHeadRef ---
describe('resolveHeadRef', () => {
it('diffs the PR head (second parent of the test merge) on pull request events', () => {
assert.equal(resolveHeadRef('pull_request'), 'HEAD^2');
assert.equal(resolveHeadRef('pull_request_review'), 'HEAD^2');
});
it('diffs HEAD on every other event', () => {
assert.equal(resolveHeadRef('merge_group'), 'HEAD');
assert.equal(resolveHeadRef('workflow_dispatch'), 'HEAD');
assert.equal(resolveHeadRef(undefined), 'HEAD');
});
});
// --- getChangedFiles on a stacked PR's test-merge commit (shallow clone) ---
describe('getChangedFiles (stacked PR test merge)', () => {
const builderDir = mkdtempSync(join(tmpdir(), 'ci-filter-build-stack-'));
const remoteDir = mkdtempSync(join(tmpdir(), 'ci-filter-remote-stack-'));
const repoDir = mkdtempSync(join(tmpdir(), 'ci-filter-stack-'));
const originalCwd = process.cwd();
const originalStep = process.env.CI_FILTER_DEEPEN_STEP;
const git = (args: string[], cwd: string) =>
execFileSync('git', args, { cwd, stdio: 'pipe' }).toString().trim();
before(() => {
execFileSync('git', ['init', '--bare', '-b', 'main', remoteDir], { stdio: 'pipe' });
git(['init', '-b', 'main'], builderDir);
git(['config', 'user.email', 'test@test.local'], builderDir);
git(['config', 'user.name', 'test'], builderDir);
git(['remote', 'add', 'origin', remoteDir], builderDir);
writeFileSync(join(builderDir, 'shared.ts'), 'shared\n');
git(['add', '.'], builderDir);
git(['commit', '-m', 'root'], builderDir);
git(['push', 'origin', 'main'], builderDir);
// Bottom of the stack: parent PR branch off root
git(['checkout', '-b', 'parent-branch'], builderDir);
writeFileSync(join(builderDir, 'parent.ts'), 'parent\n');
git(['add', '.'], builderDir);
git(['commit', '-m', 'parent PR change'], builderDir);
git(['push', 'origin', 'parent-branch'], builderDir);
// Top of the stack: this PR branches off the parent PR
git(['checkout', '-b', 'pr-branch'], builderDir);
writeFileSync(join(builderDir, 'pr-only.ts'), 'pr\n');
git(['add', '.'], builderDir);
git(['commit', '-m', 'PR change'], builderDir);
// main drifts after the stack was created
git(['checkout', 'main'], builderDir);
writeFileSync(join(builderDir, 'shared.ts'), 'shared\ndrift\n');
git(['commit', '-am', 'main moves'], builderDir);
git(['push', 'origin', 'main'], builderDir);
// GitHub builds the parent PR's test merge against main, then builds
// this PR's test merge on top of it instead of on parent-branch.
git(['checkout', '--detach', 'main'], builderDir);
git(['merge', '--no-ff', '-m', 'Merge parent-branch into main', 'parent-branch'], builderDir);
git(['merge', '--no-ff', '-m', 'Merge pr-branch into parent merge', 'pr-branch'], builderDir);
git(['push', 'origin', 'HEAD:refs/pull/1/merge'], builderDir);
// Mirror actions/checkout: depth-1 fetch of the test-merge ref
git(['init'], repoDir);
git(['remote', 'add', 'origin', `file://${remoteDir}`], repoDir);
git(['fetch', '--depth=1', 'origin', '+refs/pull/1/merge:refs/remotes/pull/1/merge'], repoDir);
git(['checkout', '--force', 'refs/remotes/pull/1/merge'], repoDir);
process.env.CI_FILTER_DEEPEN_STEP = '1';
process.chdir(repoDir);
});
after(() => {
process.chdir(originalCwd);
if (originalStep === undefined) delete process.env.CI_FILTER_DEEPEN_STEP;
else process.env.CI_FILTER_DEEPEN_STEP = originalStep;
rmSync(builderDir, { recursive: true, force: true });
rmSync(remoteDir, { recursive: true, force: true });
rmSync(repoDir, { recursive: true, force: true });
});
it('diffing HEAD reports the base-branch drift carried by the parent test merge', () => {
assert.equal(git(['rev-parse', '--is-shallow-repository'], repoDir), 'true');
assert.deepEqual(getChangedFiles('parent-branch', 'HEAD').sort(), ['pr-only.ts', 'shared.ts']);
});
it('diffing HEAD^2 reports only the files this PR changes', () => {
assert.deepEqual(getChangedFiles('parent-branch', 'HEAD^2'), ['pr-only.ts']);
assert.deepEqual(getAddedFiles('parent-branch', 'HEAD^2'), ['pr-only.ts']);
assert.equal(getMergeBase('HEAD^2'), git(['rev-parse', 'origin/parent-branch'], repoDir));
});
});
// --- getChangedFiles deep merge-base (shallow clone, stale base) ---
describe('getChangedFiles (shallow clone, stale base)', () => {

View File

@ -34,7 +34,7 @@ outputs:
description: 'Resolved base ref used for the diff (filter mode only)'
value: ${{ steps.run.outputs.base-ref }}
merge-base:
description: 'Merge-base SHA between the base ref and the PR head (filter mode only)'
description: 'Merge-base SHA between FETCH_HEAD and HEAD (filter mode only)'
value: ${{ steps.run.outputs.merge-base }}
runs:

View File

@ -95,19 +95,7 @@ export function parseFilters(input) {
const SAFE_REF = /^[a-zA-Z0-9_./-]+$/;
/**
* Ref to diff against the base branch.
*
* On pull_request events checkout lands on GitHub's test-merge commit. For a
* stacked PR its first parent is the parent PR's test merge, which already
* carries every base-branch commit the stack lacks, so a diff against HEAD
* reports that drift as changed. The second parent is the PR head itself.
*/
export function resolveHeadRef(eventName = process.env.GITHUB_EVENT_NAME) {
return eventName === 'pull_request' || eventName === 'pull_request_review' ? 'HEAD^2' : 'HEAD';
}
export function getChangedFiles(baseRef, headRef = 'HEAD') {
export function getChangedFiles(baseRef) {
if (!SAFE_REF.test(baseRef)) {
throw new Error(`Unsafe base ref: "${baseRef}"`);
}
@ -116,8 +104,8 @@ export function getChangedFiles(baseRef, headRef = 'HEAD') {
// direction, so files added to base-branch after the PR diverged show up as
// "changed" — spuriously triggering path-filtered jobs. The merge base
// scopes the diff to PR-only changes.
fetchUntilMergeBase(baseRef, headRef);
const output = execSync(`git diff --name-only --no-renames --merge-base FETCH_HEAD ${headRef}`, {
fetchUntilMergeBase(baseRef);
const output = execSync('git diff --name-only --merge-base FETCH_HEAD HEAD', {
encoding: 'utf-8',
});
return output
@ -128,14 +116,13 @@ export function getChangedFiles(baseRef, headRef = 'HEAD') {
// Files added (not merely modified) in this PR. Same merge-base scoping as
// getChangedFiles; assumes the fetch it performs has already run.
export function getAddedFiles(baseRef, headRef = 'HEAD') {
export function getAddedFiles(baseRef) {
if (!SAFE_REF.test(baseRef)) {
throw new Error(`Unsafe base ref: "${baseRef}"`);
}
const output = execSync(
`git diff --name-only --no-renames --diff-filter=A --merge-base FETCH_HEAD ${headRef}`,
{ encoding: 'utf-8' },
);
const output = execSync('git diff --name-only --diff-filter=A --merge-base FETCH_HEAD HEAD', {
encoding: 'utf-8',
});
return output
.split('\n')
.map((f) => f.trim())
@ -143,8 +130,8 @@ export function getAddedFiles(baseRef, headRef = 'HEAD') {
}
/**
* Fetch the base ref, then deepen only if its merge base with headRef is not
* reliably reachable from the checkout.
* Deepen the shallow clone until the merge base between the fetched base ref
* (FETCH_HEAD) and HEAD is reliably reachable.
*
* A single fixed deepen is not enough for stale PRs whose divergence point is
* older than the shallow boundary. We fetch with an exponentially growing
@ -156,45 +143,31 @@ export function getAddedFiles(baseRef, headRef = 'HEAD') {
* git's signed int32 `--deepen` limit), switch to `--unshallow` instead of
* passing an ever-larger integer that git would reject.
*/
function fetchUntilMergeBase(baseRef, headRef) {
function fetchUntilMergeBase(baseRef) {
let step = Number(process.env.CI_FILTER_DEEPEN_STEP) || 200;
const maxDeepen = Number(process.env.CI_FILTER_MAX_DEEPEN) || 20_000;
// A PR merge checkout already has the base commit as its first parent.
const hasBaseCommit = headRef === 'HEAD^2' && hasCommit('HEAD^1');
const depth = isShallow() && !hasBaseCommit ? `--depth=${step} ` : '';
execSync(`git fetch --no-tags --prune --filter=blob:none ${depth}origin ${baseRef}`, {
stdio: 'pipe',
});
deepenFetch(baseRef, step, maxDeepen);
while (!hasReliableMergeBase(headRef)) {
while (!hasReliableMergeBase()) {
if (!isShallow()) {
throw new Error(
`No merge base between FETCH_HEAD and ${headRef} after fetching the full history of "${baseRef}" (unrelated histories).`,
`No merge base between FETCH_HEAD and HEAD after fetching the full history of "${baseRef}" (unrelated histories).`,
);
}
deepenFetch(baseRef, step, maxDeepen);
step *= 2;
}
}
function hasCommit(ref) {
try {
execSync(`git cat-file -e ${ref}^{commit}`, { stdio: 'pipe' });
return true;
} catch {
return false;
deepenFetch(baseRef, step, maxDeepen);
}
}
function deepenFetch(baseRef, step, maxDeepen) {
const flag = step > maxDeepen ? '--unshallow' : `--deepen=${step}`;
execSync(`git fetch --no-tags --prune --filter=blob:none ${flag} origin ${baseRef}`, {
stdio: 'pipe',
});
execSync(`git fetch --no-tags --prune ${flag} origin ${baseRef}`, { stdio: 'pipe' });
}
function isShallow() {
return execSync('git rev-parse --is-shallow-repository', { encoding: 'utf-8' }).trim() === 'true';
return (
execSync('git rev-parse --is-shallow-repository', { encoding: 'utf-8' }).trim() === 'true'
);
}
/**
@ -203,12 +176,12 @@ function isShallow() {
* sub-history is truncated; that result is unreliable, so we must deepen further
* before trusting it.
*/
function hasReliableMergeBase(headRef) {
function hasReliableMergeBase() {
let base;
try {
base = execSync(`git merge-base FETCH_HEAD ${headRef}`, { encoding: 'utf-8' }).trim();
base = execSync('git merge-base FETCH_HEAD HEAD', { encoding: 'utf-8' }).trim();
} catch {
return false; // no common ancestor reachable yet, or headRef still hidden by the shallow graft
return false; // no common ancestor reachable yet
}
if (!base) return false;
return !readShallowBoundaries().has(base);
@ -232,12 +205,12 @@ function readShallowBoundaries() {
}
/**
* Resolve the merge-base SHA between FETCH_HEAD and headRef.
* Resolve the merge-base SHA between FETCH_HEAD and HEAD.
* Used to give downstream tools (e.g. janitor's AST diff) a stable, PR-only
* comparison point that doesn't drift when the base branch moves forward.
*/
export function getMergeBase(headRef = 'HEAD') {
return execSync(`git merge-base FETCH_HEAD ${headRef}`, { encoding: 'utf-8' }).trim();
export function getMergeBase() {
return execSync('git merge-base FETCH_HEAD HEAD', { encoding: 'utf-8' }).trim();
}
// --- Filter evaluation ---
@ -314,12 +287,10 @@ export function runFilter() {
}
const filters = parseFilters(filtersInput);
const headRef = resolveHeadRef();
const changedFiles = getChangedFiles(baseRef, headRef);
const addedFiles = getAddedFiles(baseRef, headRef);
const mergeBase = getMergeBase(headRef);
const changedFiles = getChangedFiles(baseRef);
const addedFiles = getAddedFiles(baseRef);
const mergeBase = getMergeBase();
console.log(`Diffing ${headRef} against ${baseRef}`);
console.log(`Merge base: ${mergeBase}`);
console.log(`Changed files (${changedFiles.length}):`);
for (const f of changedFiles) {

Some files were not shown because too many files have changed in this diff Show More