Compare commits

..

814 Commits

Author SHA1 Message Date
Iván Ovejero
58f11bf0cf
fix(core): Stop dependency index rebuild looping after import (#40356) 2026-10-21 16:26:54 +00:00
Bernhard Wittmann
ea3bebc57f
feat(Workable Trigger Node): Add webhook signature verification (#41683) 2026-10-21 16:17:35 +00:00
Riqwan Thamir
3b96f7ebaf
feat: Add testmax skills to compare AI builders and test Assistant UI features (no-changelog) (#41524)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-21 16:17:03 +00:00
Irénée
95f163ab71
feat(core): Update existing credentials on package import (no-changelog) (#41199)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 15:57:13 +00:00
Uddish Verma
c86ca80379
feat(API): Add the settings API for Public API rate limits (no-changelog) (#41574)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:44:44 +00:00
Iván Ovejero
152d0a61bf
feat(core): Estimate run data held by each execution (no-changelog) (#41553) 2026-10-21 15:42:53 +00:00
Michael Drury
8dd4319781
fix(editor): Support agent test chat on HTTP origins (#41713) 2026-10-21 15:40:34 +00:00
Pierantonio Cangianiello
a56adb3c81
fix(core): Pair declarative node output items with their input items (#41645)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-21 15:15:53 +00:00
Mike Repeć
0c29642e21
feat(core): Compare a sample of native expression results with the engine (no-changelog) (#41512)
Co-authored-by: Lorent Lempereur <lorent.lempereur@n8n.io>
2026-10-21 15:14:53 +00:00
Florian Gößler
79b749757f
fix(core): Close chat attachment streams when the client disconnects (#41108)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-10-21 15:06:10 +00:00
Florian Gößler
da70de87b1
fix(core): Handle client disconnects in Respond to Webhook binary responses (#41110)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-10-21 15:01:44 +00:00
Ricardo Espinoza
7eed1237ee
feat(editor): Add skill download to the Skills settings page (no-changelog) (#41696) 2026-10-21 14:50:50 +00:00
Uddish Verma
61e5b01d8d
feat(API): Share Public API rate-limit counts across mains (no-changelog) (#41407)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 14:34:20 +00:00
Pierantonio Cangianiello
8dd5182add
fix(core): Pair output items before the error output split (#41639)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-21 14:31:01 +00:00
Kai
4341ac2b6b
feat(editor): Complete Promote and Apply from the preview pages (no-changelog) (#41425) 2026-10-21 14:26:31 +00:00
n8n-cat-bot[bot]
6e0d10e58c
refactor: Move project service reads into repositories (#41688)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-21 14:16:46 +00:00
Rob Hough
d8809f641c
fix(editor): Adapt Agent Builder panel widths to available space (#41562) 2026-10-21 14:10:57 +00:00
Benjamin Schroth
9ceb3945e7
fix(core): Show the root cause of model errors in AI Agent and Basic LLM Chain (#41388) 2026-10-21 13:51:13 +00:00
Benjamin Schroth
0b349760f8
feat(editor): Let agent owners make agents available in n8n Chat (no-changelog) (#41643)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 13:47:43 +00:00
Lucas Gabriel Hartmann
7ffa1b1715
fix(core): Don't close shared SSH tunnel clients that are still in use (#40114)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:46:28 +00:00
Francesca Sargent
98e98e5930
feat(core): Send telemetry when a user enables or disables MFA (no-changelog) (#41684)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:42:51 +00:00
n8n-cat-bot[bot]
1adcf7aa9f
feat(core): Show version mismatch warnings on all mains (#40097)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Veljko Terzic <veljko.terzic@n8n.io>
2026-10-21 13:38:54 +00:00
Jaakko Husso
a965a816d1
feat(core): Let a system agent supply its own sandbox workspace (no-changelog) (#40618)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:37:11 +00:00
Luigi Gubello
282fa323af
fix: Update certifi to 2026.10.19 in task runner (#41693)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Matsu <matias.huhta@n8n.io>
2026-10-21 13:36:38 +00:00
Yuliia Pominchuk
8708319d56
feat(core): Authenticate Webhook and MCP Trigger nodes through trusted sources (#41455) 2026-10-21 13:23:33 +00:00
Yuliia Pominchuk
25c86e0164
fix(editor): Say 'Can view' on projects a credential is used in but not shared with (#41690) 2026-10-21 13:22:55 +00:00
Jaakko Husso
f7cd6783c4
fix(core): Apply execution redaction to Assistant execution reads (#41565)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:17:10 +00:00
Kai
0eb0a5dd29
fix(core): Keep workflow tags when a selective Apply meets a tag difference (no-changelog) (#41420) 2026-10-21 13:15:53 +00:00
Robin Braumann
d1f33cc04a
feat(core): Add typed integration metadata to agent messages (no-changelog) (#41652)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-21 13:06:56 +00:00
Theodore Nastos
e6dc96de6e
refactor(editor): Share the project fetch and rename the contact admin dialog (no-changelog) (#41373) 2026-10-21 12:57:16 +00:00
Nour Alhadi Mahmoud
5a64eda765
feat(core): Show GitLab merge requests in the review inbox (no-changelog) (#41434) 2026-10-21 12:52:16 +00:00
Eugene
ea5b296d88
feat(core): Add persistence for Agent plan stops and job ownership (no-changelog) (#41269)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 12:40:07 +00:00
Raúl Gómez Morales
7ea0c14c52
fix(core): Remove the workflow default from Assistant routing (no-changelog) (#41435) 2026-10-21 12:38:58 +00:00
Luigi Gubello
b4bfdf99c7
chore: Update npm dependencies (#41634)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:33:07 +00:00
Kai
8ba4f696a6
feat(core): Check Apply blockers without applying (no-changelog) (#41346)
Co-authored-by: Sandra Zollner <sandra.zollner@n8n.io>
2026-10-21 12:19:35 +00:00
n8n-cat-bot[bot]
e228ee968e
fix: Handle null and undefined in manual console output (#41570)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-21 11:45:17 +00:00
Predrag Ristic
f15b08b42f
feat(core): Add background self-healing investigations (no-changelog) (#41020) 2026-10-21 11:34:46 +00:00
Milorad FIlipović
369bc86f77
feat(core): Scaffold the knowledge backend module with boundary interfaces (no-changelog) (#41517) 2026-10-21 11:26:23 +00:00
RomanDavydchuk
a91df9f6e7
feat: Implement source code and bundle management for apps (no-changelog) (#41274) 2026-10-21 11:25:32 +00:00
Yen Su
7cbdae8751
fix(Databricks Node): Update an existing Lakebase row when Insert or Update lacks a required column (no-changelog) (#41630) 2026-10-21 11:17:44 +00:00
yehorkardash
a67dba1d64
fix(Postgres Node): Prevent global pgtypes overwrite (#41641) 2026-10-21 11:12:37 +00:00
Leomihran
af111c0152
build: Update Electron to 41.10.6 (no-changelog) (#41619)
Co-authored-by: Léo Khatchadourian <315401086+Leomihran@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
Co-authored-by: Luigi Gubello <luigi.gubello@n8n.io>
2026-10-21 11:08:40 +00:00
Yen Su
6da54e1650
fix(Databricks Node): Send a JSON body for a Lakebase function call that has no arguments (#41632) 2026-10-21 11:05:45 +00:00
Michael Drury
dba1c53054
fix(editor): Use n8n Assistant in history and sandboxing copy (#41566) 2026-10-21 10:56:46 +00:00
Robin Braumann
d669f68a37
fix(core): Commit observation batches and cursor progress atomically (no-changelog) (#41609) 2026-10-21 10:52:05 +00:00
mfsiega
db8094ff09
feat(core): Export the migration report as CSV (#41534)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-21 10:45:12 +00:00
Robin Braumann
f54fc2ebfb
feat(core): Resolve Agent package import identities (no-changelog) (#40479) 2026-10-21 10:44:19 +00:00
Sam Wooler
800821d227
feat(editor): Add Environments group for source control API key scopes (#41537)
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-10-21 10:40:54 +00:00
Milorad FIlipović
a1edfdbd0f
test(editor): Fix the ProjectTabs preview badge test after the badge class rename (no-changelog) (#41650) 2026-10-21 10:33:48 +00:00
Sam Wooler
794ede87b5
feat(n8n Node): Add action to add a user to a project (#41253)
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-10-21 10:28:22 +00:00
Kai
9b85963d18
feat(editor): Add promote and apply badges to the project header (no-changelog) (#41336) 2026-10-21 10:18:27 +00:00
Robin Braumann
fb8ae3de68
perf(core): Deduplicate episodic candidates before embedding (#41618) 2026-10-21 09:50:37 +00:00
Florian Gößler
2dd1d15512
fix(core): Close upload body streams when an outbound request fails (#41078)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-10-21 09:50:17 +00:00
Kai
f830c2b113
feat(editor): Add promotion and apply preview pages with a resource tree and change cards (no-changelog) (#41333) 2026-10-21 09:42:11 +00:00
Sebastien Powell
c69581aedf
feat(editor): Add dialog props, update the layout, and migrate existing dialogs (#40690)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-21 09:15:45 +00:00
Sebastien Powell
d7fa8ada34
feat(editor): Add an All option to pagination and remove the custom datatable page-size controls (#39850)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-21 09:15:29 +00:00
Florian Gößler
e167fd5e09
fix(core): Close binary data streams when the client disconnects (#41077)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-10-21 09:15:23 +00:00
Charlie Kolb
65a2e21467
fix: Update logos in credential oauth confirmation screen and instance emails (#41629) 2026-10-21 09:05:12 +00:00
Jan Kalkan
4bb98c4305
fix(core): Check project credential:create before package import writes (no-changelog) (#41444)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:58:03 +00:00
n8n-cat-bot[bot]
fb08a75c50
fix: Handle IMAP transport errors during connection setup (#41248)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-21 08:57:46 +00:00
Yen Su
bed2ab9210
fix(Databricks AI Agent Node): Scrub MLflow trace request and response previews (#41247) 2026-10-21 08:50:00 +00:00
Guillaume Jacquart
953e15459f
feat(editor): Block Publish on restricted node types (no-changelog) (#41308)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 08:47:21 +00:00
n8n-cat-bot[bot]
4f3896adde
fix: Isolate LDAP clients and apply operation timeouts (#41611)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-21 08:41:27 +00:00
Kai
4990435136
feat(editor): Add the promotion preview model (no-changelog) (#41335) 2026-10-21 08:33:42 +00:00
Rob Hough
25010a29f5
fix(editor): Reduce Agent modal header spacing (#41585)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
2026-10-21 08:25:09 +00:00
n8n-assistant[bot]
90cd015a25
chore: Update e2e impact map (#41620)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-21 08:20:07 +00:00
Jaakko Husso
d497ba6959
feat(core): Add chat routes for system agents (no-changelog) (#41232)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:16:06 +00:00
Jaakko Husso
747eb1c3f1
feat(core): Add a system-agent layer to the Agents runtime (no-changelog) (#40617)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 08:16:06 +00:00
Rob Hough
1cafc2e121
feat(editor): Add and adopt the shared slider (no-changelog) (#41381)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
2026-10-21 08:13:27 +00:00
Rob Hough
990676c35b
fix(editor): Correct brand rollout styles and layout (no-changelog) (#41567)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
2026-10-21 08:10:04 +00:00
n8n-cat-bot[bot]
0a3b12f9fb
test: Stabilize RunData schema tests (no-changelog) (#41606)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-21 08:08:48 +00:00
Guillaume Jacquart
7e405aacfd
fix(core): Read type availability policies from memory at enforcement points (#41552)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 08:07:32 +00:00
Jan Kalkan
7b04efb48a
feat(editor): Revamp the Environments v2 settings page (no-changelog) (#41480)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 07:59:41 +00:00
Andreas Fitzek
b768f6c7ca
fix(core): Report SAML logins whose asserted email differs from the linked user (#41506)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-21 07:56:05 +00:00
Mike Repeć
10a1c3d4f0
test: Remove webhook isolate skip e2e spec (no-changelog) (#41560) 2026-10-21 07:54:53 +00:00
kevin
be2da292fa
fix(Summarize Node): Return null instead of NaN for an average with no numeric values (#41430) 2026-10-21 07:53:33 +00:00
Iván Ovejero
94b95bf461
ci: Promote the -pc image to v3 release candidates (#41575) 2026-10-21 07:41:09 +00:00
n8n-cat-bot[bot]
4d93b38892
fix: Preserve Salesforce instance URL on OAuth2 token refresh (#41415)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Andreas Fitzek <andreas.fitzek@n8n.io>
2026-10-21 07:37:24 +00:00
n8n-cat-bot[bot]
ef03659541
fix: Open Chat Hub OAuth popups before authorization requests (#41409)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Andreas Fitzek <andreas.fitzek@n8n.io>
2026-10-21 07:37:00 +00:00
Ricardo Espinoza
93fd95b4b1
refactor(core): Move skills into their own backend module (no-changelog) (#41478) 2026-10-21 07:06:25 +00:00
Bernhard Wittmann
1ebe2cf559
feat(editor): Set up Microsoft Teams again when its credential changes (no-changelog) (#41259) 2026-10-20 22:56:27 +00:00
RomanDavydchuk
bfbc56fe5b
feat: Scaffold the module and entities for apps (no-changelog) (#41049) 2026-10-20 18:18:00 +00:00
Pushkal Gupta
a5eada9be7
fix(editor): Localize the select placeholders in workflow settings (#41325)
Co-authored-by: Michael Drury <me@michaeldrury.co.uk>
2026-10-20 16:53:47 +00:00
Luka Zivkovic
aa4a35851e
fix(core): Use mocked execution for agent check previews (#41144) 2026-10-20 15:44:11 +00:00
Sandra Zollner
0d03e5de39
feat(core): Open a GitLab merge request for a branched promote (no-changelog) (#41208)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 15:43:40 +00:00
João Almeida
967112a8cd
fix(editor): Load and keep resource mapper fields inside a fixedCollection (#41209)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:39:38 +00:00
Alexander Malka
2ec85cacc8
feat(editor): Surface MCP to Claude trial users (no-changelog) (#40408) 2026-10-20 15:38:22 +00:00
Thanasis G
d3f7a232b0
feat(core): Report the whole deployment's footprint in telemetry (#40685)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 15:35:52 +00:00
Robin Braumann
630b973bba
feat: Extend Agent Builder telemetry (no-changelog) (#41564) 2026-10-20 15:27:18 +00:00
Michael Drury
964ae5127b
fix(editor): Delay agent setup checklist until building finishes (#41559) 2026-10-20 15:26:22 +00:00
Mayank SIngh
8ba05b1c33
fix(core): Set CORS methods for waiting webhooks (#39210)
Co-authored-by: Veljko Terzic <veljko.terzic@n8n.io>
2026-10-20 15:21:54 +00:00
Svetoslav Dekov
b1f6852959
feat(core): Add skills hub CRUD tools to n8n Assistant (no-changelog) (#41428)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:05:24 +00:00
Michael Drury
42f783a2cf
fix(core): Stream Slack agent replies in one message (#41470) 2026-10-20 15:00:34 +00:00
Michael Drury
f2bfe179f6
fix(core): Allow agent node tools to use global credentials (#40814) 2026-10-20 14:43:14 +00:00
Arvin A
4a5e4db695
feat(core): Add skill Assistant availability column (no-changelog) (#41448) 2026-10-20 14:33:26 +00:00
Kai
5253ec4799
fix(core): Block Apply of a root workflow that is in a folder on the instance (no-changelog) (#41382) 2026-10-20 14:13:52 +00:00
Florian Gößler
b2e7e9ffc6
fix(core): Replace the QuickJS runtime and WASM module after a native stack overflow (#40968)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 14:07:17 +00:00
Mike Repeć
32a9a7a99d
fix(core): Claim an idle worker's next queue job and its lock in one step (#41525) 2026-10-20 14:06:37 +00:00
Thanasis G
82362c6fe5
feat(Microsoft Foundry Chat Model Node): Use the Microsoft Foundry logo (#41426)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 14:03:42 +00:00
Jan Kalkan
0ef221184b
refactor: Replace promotion-connection CLI commands with promotion-environment (no-changelog) (#41453)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 14:01:20 +00:00
Benjamin Schroth
7f145b3008
feat(core): Add a migration that moves Tools Agent nodes below version 2 to version 2.2 (no-changelog) (#41496) 2026-10-20 13:57:50 +00:00
Jaakko Husso
6dd15242ef
fix(core): Parse node type names at the first dot (no-changelog) (#41546)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:51:36 +00:00
Robin Braumann
ead7c373f6
feat(editor): Add Agent Builder interaction telemetry (no-changelog) (#41543) 2026-10-20 13:42:36 +00:00
MHMD
36ea544ede
feat(core): Make granular credential sharing always on (#41376) 2026-10-20 13:37:53 +00:00
Kai
9aa6edbfe9
refactor(core): Move the serialized package schemas to @n8n/api-types (no-changelog) (#41319) 2026-10-20 13:37:25 +00:00
Jaakko Husso
db90867738
fix(editor): Scroll data table previews in the AI Assistant panel (#41522)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:36:16 +00:00
Riqwan Thamir
d5d659610a
perf(core): Always load the model catalog search tool in Instance AI (#41545)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 13:35:29 +00:00
Sandra Zollner
3bef0a5f7a
refactor(core): Replace promotion connections and configs with environments (no-changelog) (#41544)
Co-authored-by: Jan K. <jan.kalkan@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 13:22:20 +00:00
Thanasis G
1265d2f461
feat(core): Report the managed database and Redis service in telemetry (#40427)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 12:51:06 +00:00
Yen Su
1483a8fcb3
feat(Databricks Node): Add the Lakebase Execute Function operation (no-changelog) (#40547) 2026-10-20 12:37:48 +00:00
Stephen Wright
f930f2c530
fix(Microsoft SharePoint Node): Keep the filter and field options when Return All is on (#41205)
Co-authored-by: mayanksingh-27 <mayankdhapola2002@gmail.com>
2026-10-20 12:36:49 +00:00
Mayank SIngh
466f787818
fix(core): Skip empty cache hash writes (#40937) 2026-10-20 12:25:25 +00:00
Pushkal Gupta
6813be8507
fix(editor): Localize the binary data preview messages (#41323)
Co-authored-by: RomanDavydchuk <roman.davydchuk@n8n.io>
2026-10-20 12:14:06 +00:00
Michael Drury
ba9835ae55
fix(core): Keep AI Assistant image attachments visible in history (#41528) 2026-10-20 12:03:27 +00:00
Francesca Sargent
af65c1dffa
feat(core): Accept OAuth2 bearers on the instance MCP server through the trusted-source pipeline (no-changelog) (#41438)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 11:59:47 +00:00
Csaba Tuncsik
9f074c20c9
fix(Microsoft Teams Trigger Node): Renew the Graph subscription via lifecycle notifications (#41299) 2026-10-20 11:48:41 +00:00
mfsiega
e3310539a6
feat(engine): Run partial and pinned manual executions on engine v2 (#40453)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-20 11:45:15 +00:00
Stephen Wright
12e000b15c
fix(Kafka Trigger Node): Release each execution's run data when it completes (#41531) 2026-10-20 11:38:29 +00:00
Robin Braumann
ef1cb16061
fix(core): Include Agent runtime setup in the eval timeout (no-changelog) (#41523) 2026-10-20 11:19:28 +00:00
Kai
6f12141cd9
feat(core): Show all supported resources in the promotion and apply preview (no-changelog) (#41314) 2026-10-20 11:16:19 +00:00
James Martin
9f07c53e99
feat(core): Add an option to allow a child to not enforce its redaction policy on a caller (no-changelog) (#41193)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 11:16:07 +00:00
Florian Gößler
26dac9ae4a
fix(core): Let the folder table migration complete when its column already exists (#41006) 2026-10-20 11:15:51 +00:00
Alex Grozav
ff2000c925
feat(editor): Move migration rule filters into a Sort and Filter control (no-changelog) (#41393)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 11:13:21 +00:00
Michael Drury
744604c749
fix(editor): Correct Agent tool validation and home empty state (#41474) 2026-10-20 11:10:49 +00:00
Ali Elkhateeb
dee57b0568
docs(API): Record that a failed idempotency POST releases the key (#41505)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 11:08:54 +00:00
n8n-cat-bot[bot]
ac02213aa8
fix: Fall back to OpenID discovery after HTML metadata responses (#41063)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-20 11:02:30 +00:00
Alexander Gekov
8a35cf24b7
fix(Qwen Cloud Node): Route text-only DeepSeek models and add DeepSeek search alias (#41212)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 11:01:43 +00:00
n8n-cat-bot[bot]
4a74f027e6
fix: Add file-type tooltip to artifact download button (#41490)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Jaakko Husso <jaakko@n8n.io>
2026-10-20 10:55:26 +00:00
Charlie Kolb
1587770a24
feat: Replace brand assets (#41510) 2026-10-20 11:03:04 +00:00
Benjamin Schroth
2f8c127cb4
fix(editor): Select the agent's project in the n8n Chat picker (no-changelog) (#41454) 2026-10-20 10:43:15 +00:00
Charlie Kolb
4939f634b7
refactor(core): Move AI preferences into backend services (#41090) 2026-10-20 10:37:02 +00:00
Rob Hough
68afdd3a24
feat(editor): Align agent plan and message queue layouts (#40635)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
2026-10-20 10:33:20 +00:00
Robin Braumann
34d4b73c1f
fix(core): Unify Agent creation and correct creation telemetry (#41378) 2026-10-20 10:23:38 +00:00
n8n-cat-bot[bot]
2b2eb0f5e1
fix: Handle empty and invalid revision responses (#41114)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-20 10:07:08 +00:00
Robin Braumann
6b3c75299a
chore(core): Enforce the Agents and Instance AI boundary (#41240) 2026-10-20 10:06:12 +00:00
Robin Braumann
2c183300c3
refactor(core): Prepare Agent evaluations inside Agents (#41369) 2026-10-20 10:04:33 +00:00
Anne Aguirre
618fd7e272
fix(core): Ignore counts of tokens when provider omits input / output tokens (#41395) 2026-10-20 10:01:37 +00:00
Mike Repeć
2b65576b78
feat(core): Let workflow editors triage the migration findings of their workflows (#41093) 2026-10-20 09:54:19 +00:00
Florian Gößler
5e807659ae
fix(core): Detect invalid DateTimes from any luxon copy (no-changelog) (#40960)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-20 09:49:57 +00:00
Eugene
4530ba6b23
fix(core): Separate Agent foreground stops from background work (no-changelog) (#41268) 2026-10-20 09:45:54 +00:00
mfsiega
2dc24cfe45
feat(engine): Plan partial and pinned manual runs for engine v2 (#40457)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-20 09:44:18 +00:00
Florian Gößler
dbcfecf5b9
fix(core): Keep parentheses around && and || operands of ?? in expressions (#40962)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 09:35:06 +00:00
Robin Braumann
e5d3f0f9f4
fix(core): Enable Claude prompt caching for OpenRouter agents (no-changelog) (#40876) 2026-10-20 09:31:39 +00:00
Florian Gößler
6225e62469
fix(core): Run the own Error Trigger when no error workflow is selected (#40969)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 09:21:40 +00:00
Florian Gößler
917d035944
fix(core): Record a failing sub-node run at the sub-node's own run index (#40970)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 09:21:25 +00:00
Ilfat Mindubaev
6532d29df4
feat(editor): Show the model that answered in the log details header (#41390)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 09:21:06 +00:00
n8n-cat-bot[bot]
1526d6a416
fix: Reject task broker listen errors and use assigned test ports (#40925)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Florian Gößler <mail@floriangoessler.de>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 09:19:06 +00:00
Robin Braumann
0952989f95
refactor(core): Move Assistant policy out of Agent Builder (#41368) 2026-10-20 09:13:19 +00:00
n8n-release-tag-merge[bot]
02b548009b Merge tag 'n8n@2.45.0' 2026-10-20 09:10:38 +00:00
n8n-assistant[bot]
7c79fd09a0
🚀 Release 2.45.0 (#41509)
Co-authored-by: CharlieKolb <13814565+CharlieKolb@users.noreply.github.com>
2026-10-20 08:55:34 +00:00
Rob Hough
91496636d8
fix(editor): Center badge text when icons are absent (#41414)
Co-authored-by: Sebastien Powell <sebpowell@users.noreply.github.com>
2026-10-20 08:55:02 +00:00
Robin Braumann
aad59a59ae
refactor(core): Keep Agent draft lifecycle in Agents (#41367) 2026-10-20 08:40:11 +00:00
Robin Braumann
5eecf1463b
refactor(core): Read Agent context through an Agent-owned service (#41366) 2026-10-20 08:39:50 +00:00
n8n-assistant[bot]
e4df41ae0a
chore: Update e2e impact map (#41504)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-20 08:18:51 +00:00
Raúl Gómez Morales
9253eb91eb
test(core): Use the prefixed create tasks tool name in the skills service test (no-changelog) (#41501) 2026-10-20 08:15:17 +00:00
Robin Braumann
475ef36acc
feat(core): Add native tool deferral for agents (no-changelog) (#40727) 2026-10-20 07:52:44 +00:00
Robin Braumann
397adabdda
test(editor): Use a relative date for the Insights custom range (#41494) 2026-10-20 07:37:52 +00:00
Riqwan Thamir
373c57797e
fix(core): Restore Agent Builder behaviour lost with the build-agent sub-agent (no-changelog) (#41482)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-20 05:40:10 +00:00
Riqwan Thamir
7a334e3b15
refactor(core): Remove the build-agent sub-agent (no-changelog) (#41356)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 21:10:02 +00:00
Riqwan Thamir
e532aae35b
feat(core): Run Agent Builder tools in the Instance AI orchestrator (no-changelog) (#41355)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 21:09:54 +00:00
Riqwan Thamir
6e8064c6af
refactor(core): Prefix Instance AI and Agent Builder tool names (no-changelog) (#41354)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 20:58:37 +00:00
Luka Zivkovic
f73dc337f8
fix(core): Trace verification helpers and provider attempts (#40936)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 19:38:11 +00:00
Stephen Wright
967591041e
fix(core): Keep offline_access when a protected resource omits it (#41384) 2026-10-19 17:28:19 +00:00
srinivas-kannepalli
791c7bc418
fix(core): Allow gateway model IDs and forward credential headers in agents (#40470)
Co-authored-by: Michael Drury <michael.drury@n8n.io>
2026-10-19 16:20:58 +00:00
Irénée
d4bde146d6
fix: Align data table and variable editing on protected instances (#41258)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 15:43:43 +00:00
Jaakko Husso
16fb42d76e
feat(core): Add a system chat surface to the Agents runtime (no-changelog) (#41231)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 15:33:53 +00:00
Rick Gale
dd30fc447b
chore: Assign agent code to the Agent team (no-changelog) (#41394)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 15:28:29 +00:00
Stephen Wright
14d2d30563
fix(editor): Offer the managed OAuth choice when the app is supplied (#40284) 2026-10-19 15:01:55 +00:00
Jaakko Husso
ef30f75e7b
refactor(core): Move the Agents chat relay into its own service (no-changelog) (#41230)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:58:57 +00:00
Bernhard Wittmann
70f0131be5
fix(editor): Stop credential icon overlapping the credential name in the agent credential picker (#41262) 2026-10-19 14:58:31 +00:00
Yen Su
f527f748fd
feat(Databricks Node): Add the Lakebase Chat Memory sub-node (no-changelog) (#41055) 2026-10-19 14:57:01 +00:00
n8n-cat-bot[bot]
35874f421d
chore: Flaky test: InputPanel.test.ts (2 master or merge-queue commits in 72 h) (#41445)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-19 14:50:51 +00:00
Jaakko Husso
c94ff47cd8
feat(core): Add instance-scoped agents to the Agents (no-changelog) (#40616)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:37:16 +00:00
Svetoslav Dekov
804033af54
feat(core): Add skills hub CRUD tools to the MCP server (no-changelog) (#41398)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:35:54 +00:00
Mike Repeć
de4411b6b8
fix(core): Copy native evaluation results as JSON and hand other values to the engine (no-changelog) (#41082) 2026-10-19 14:28:55 +00:00
Bernhard Wittmann
51e53ad9b7
fix(editor): Match the credential dropdown width to its field and remove horizontal scroll (#41260) 2026-10-19 13:57:53 +00:00
Thanasis G
d6d24f267c
feat(core): Report the CPU and memory limits n8n was given in telemetry (#40421)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 13:50:59 +00:00
Michael Drury
d74f3f2cdd
fix(core): Keep cancelled Assistant activity settled (#40786) 2026-10-19 13:42:22 +00:00
n8n-cat-bot[bot]
b850125fbd
chore: Flaky test: AgentBuilderView.test.ts (2 master or merge-queue commits in 72 h) (#41427)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-19 12:57:09 +00:00
Riqwan Thamir
198127f9cd
feat(core): Support legacy tool names and extending runtime skill sources (no-changelog) (#41353)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 12:51:33 +00:00
Riqwan Thamir
5ad13f6606
feat(core): Record builder success metrics in Instance AI traces (no-changelog) (#40714)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 12:38:33 +00:00
Ricardo Espinoza
b27f132218
fix(core): Copy each agent draft into the skills hub only once (no-changelog) (#41350) 2026-10-19 12:36:57 +00:00
Rob Hough
20ea6717a0
fix(editor): Restore From AI button corner radii (#41176)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
Co-authored-by: Sebastien Powell <sebpowell@users.noreply.github.com>
2026-10-19 12:33:32 +00:00
Jaakko Husso
35e9ee9da5
feat(editor): Track the concise style experiment for Instance AI (no-changelog) (#41423)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 12:31:00 +00:00
Thanasis G
0c1a78793a
feat(Azure OpenAI Chat Model Node): Add an optional Project field to the Foundry credentials (#41181)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 12:25:17 +00:00
Jaakko Husso
f4f606754c
feat(core): Report prompt size on Instance AI credit claims (no-changelog) (#41402)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 12:18:44 +00:00
Eugene
4fd7a1bad7
fix(core): Apply partial Agent plan updates and report finish time (no-changelog) (#41267) 2026-10-19 12:16:51 +00:00
Ricardo Espinoza
086fcf9437
feat(core): Add SCIM user provisioning (no-changelog) (#40419) 2026-10-19 12:16:33 +00:00
Yash Rajeshbhai Darji
660849d93e
fix(Send Email Node): Send to every recipient when a recipient field is an array (#36467)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 12:13:03 +00:00
Bernhard Wittmann
117aa146ce
fix(editor): Show the agent's own icon on Microsoft Teams install cards (no-changelog) (#41273) 2026-10-19 12:01:36 +00:00
Svetoslav Dekov
8323dfd1e1
fix(core): Let preview chat agents make changes through their tools (#41403)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 11:50:31 +00:00
Mike Repeć
95b54e5390
fix(core): Decline the whole expression when the extension rewrite reaches an optional chain (no-changelog) (#41371) 2026-10-19 11:45:21 +00:00
yehorkardash
d27de60b95
feat: Don't save MCP connections without user approval (#39996)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-19 11:28:03 +00:00
Svetoslav Dekov
38a1b15646
feat(core): Let skills API callers report their telemetry source (no-changelog) (#41397)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 11:11:15 +00:00
Jan
d9d6c027f6
fix(editor): Center notification title with its icon in design-system (no-changelog) (#40323)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 11:04:35 +00:00
Sebastien Powell
28602ed56e
fix(editor): Place sticky notes and nodes at the right-click and clear the selection (#41365) 2026-10-19 10:56:37 +00:00
Jan
c084d996af
feat(editor): Extend N8nCopyInput with accessibility, redaction, loading and actions (#34942)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Sebastien Powell <sebpowell@users.noreply.github.com>
2026-10-19 10:54:01 +00:00
Stephen Wright
06ba85cbb9
fix(Gmail Node): Keep the References header on a draft created in a thread (#41392) 2026-10-19 10:23:26 +00:00
Tariq Shatat
b98640f0ab
fix(editor): Improve NDV credential options performance on large instances (#40198)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 10:18:17 +00:00
Charlie Kolb
04761593da
fix(editor): Support IPv6 connections to the dev server (no-changelog) (#41401) 2026-10-19 10:16:46 +00:00
Csaba Tuncsik
191ea98482
feat(Databricks Node): Show the Vector Store, Embeddings and AI Agent nodes in the node picker (#41206) 2026-10-19 10:08:35 +00:00
yehorkardash
9a025a188f
fix(editor): Load credential types on startup (#41400) 2026-10-19 09:53:26 +00:00
Shireen Missi
4ceb7dc33e
chore: Assign Nodes and Relay ownership in OWNERS (no-changelog) (#41073)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 09:50:10 +00:00
Toni Devine
275f7ebf68
feat(editor): Align the WhatsApp channel setup and settings with the … (#41283) 2026-10-19 09:48:19 +00:00
Mike Repeć
6e04287582
fix(core): Key queue job outcomes by job on shutdown resume (#40569) 2026-10-19 09:42:09 +00:00
Thanasis G
b2edcb5727
feat(core): Report the runtime n8n runs on in telemetry (#40420)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 09:32:53 +00:00
Stephen Wright
952cbf2bf2
feat(Databricks Node): Add the Lakebase Insert or Update operation (no-changelog) (#40645) 2026-10-19 09:29:04 +00:00
Bernhard Wittmann
5a02fc57aa
feat(core): Let Teams agents send messages from scheduled tasks (no-changelog) (#41244) 2026-10-19 09:14:29 +00:00
Tuukka Kantola
9eb7cb9849
refactor(editor): Apply brand icons to more product surfaces (no-changelog) (#41380)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 08:56:48 +00:00
Joco-95
eac2f46b0c
feat: Implements workflow previews above Assistant empty state experiment (#40913) 2026-10-19 08:52:09 +00:00
Stephen Wright
9d2df34333
feat(Databricks Node): Add the Lakebase Update operation (no-changelog) (#40546) 2026-10-19 08:48:54 +00:00
Anne Aguirre
4d292a6009
feat(core): Only show execution input and output for agent subworkflows in timeline (#41189) 2026-10-19 08:40:35 +00:00
Riqwan Thamir
10b1b30e72
chore(core): Remove leftover runtime skill references from Instance AI (#41351)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 08:39:52 +00:00
Lorent Lempereur
ffec16672c
feat(core): Run the idempotency-key cleanup as a durable system task (no-changelog) (#41009)
Co-authored-by: Veljko Terzic <veljko.terzic@n8n.io>
2026-10-19 08:25:25 +00:00
n8n-assistant[bot]
0544d7e597
chore: Update e2e impact map (#41387)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-19 08:21:51 +00:00
Veljko Terzic
d9cfa767ab
test(core): Add integration tests for the system task runner across mains (no-changelog) (#40878) 2026-10-19 08:20:18 +00:00
Kai
f6d70c140b
feat(editor): Show multiple workflows in the review detail view (#40695) 2026-10-19 07:54:56 +00:00
MHMD
862c7eceab
feat(core): Run the Form and Chat trigger redirect login against a trusted source (#41123) 2026-10-19 07:50:33 +00:00
James Martin
60d6231375
chore(core): Update redaction message to explain that it was redacted because of a child (no-changelog) (#41171)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 07:41:40 +00:00
MHMD
0936dfddc3
feat(core): Refuse revoked access tokens of the local server on the inbound pipeline (#41266) 2026-10-19 07:17:35 +00:00
Ilfat Mindubaev
b10c97ed45
feat(core): Surface the responding model name in LLM sub-node run data (#41161)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-19 06:44:22 +00:00
Lukas
e9723095d6
fix(OpenAI Node): Send max_completion_tokens when it analyzes an image (#38656) 2026-10-19 04:33:59 +00:00
n8n-assistant[bot]
b8a02f70dc
chore: Update node popularity data (#41359)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-19 00:42:27 +00:00
n8n-assistant[bot]
d2dc2f9ce5
chore: Update e2e impact map (#41339)
Some checks failed
CI: Master (Build, Test, Lint) / Build & format (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Lint (push) Has been cancelled
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Has been cancelled
Util: Update Node Popularity / update-popularity (push) Has been cancelled
Util: Update Credential Setupability / update-setupability (push) Has been cancelled
Util: Update Node Popularity / approve-and-automerge (push) Has been cancelled
Util: Update Credential Setupability / approve-and-automerge (push) Has been cancelled
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-18 08:18:22 +00:00
Ricardo Espinoza
84943adc6c
fix(editor): Show which reference blocks a skill save (no-changelog) (#41298) 2026-10-18 07:50:57 +00:00
Riqwan Thamir
ff975f1b9e
perf(core): Stop writing Instance AI skills to the sandbox (no-changelog) (#41220)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-17 14:02:07 +00:00
n8n-assistant[bot]
936349a9c7
chore: Update e2e impact map (#41317)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-17 08:18:42 +00:00
Olivier
842ec5b62d
fix(core): Turn off expression spans by default (#41211)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:23:07 +00:00
n8n-cat-bot[bot]
ec074519d9
feat: Add policy attachment read endpoints (no-changelog) (#41122)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 16:13:10 +00:00
Rúben Castro
6f3b397a1e
feat(editor): Mark community packages with blocked node types (no-changelog) (#41105)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Guillaume Jacquart <jacquart.guillaume@gmail.com>
2026-10-16 15:43:39 +00:00
n8n-cat-bot[bot]
21221adc78
fix: Validate attendee values (#40953)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-16 14:56:23 +00:00
Guillaume Jacquart
532e8ef930
fix(core): Save policy rules that name uninstalled packages (no-changelog) (#41210)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-16 14:48:56 +00:00
n8n-cat-bot[bot]
24621ad4ca
chore: Add review/lint rule to discourage open-ended fields in nodes (warn level) (#39959)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-16 14:47:15 +00:00
n8n-cat-bot[bot]
8ee5cf1845
chore: Flaky test: frontend.service.test.ts (3 master or merge-queue commits in 72 h) (#41279)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-16 14:44:01 +00:00
Stephen Wright
7f16040253
feat(Databricks Node): Add the Lakebase Delete operation (no-changelog) (#40543) 2026-10-16 14:42:03 +00:00
Csaba Tuncsik
ddc97694b5
fix(Microsoft Teams Node): Add the TeamMember.Read.All scope and correct the Trigger scope notice (#39819) 2026-10-16 14:27:44 +00:00
Rúben Castro
eaf47cfe39
fix(editor): Improve destination validation and surface save errors (#40897)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Guillaume Jacquart <jacquart.guillaume@gmail.com>
2026-10-16 14:14:55 +00:00
Ricardo Espinoza
986b82fa28
feat: Add telemetry events to the skills hub (no-changelog) (#41234) 2026-10-16 14:12:37 +00:00
Jan
76cce99626
fix(editor): Unify documentation links across settings pages (#40289)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-16 13:57:10 +00:00
Jaakko Husso
f34c093f08
feat(core): Add an optional model for n8n Assistant memory tasks (#41137)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 13:55:06 +00:00
Mike Repeć
158152496a
fix(core): Reuse connections in the log streaming webhook destination (#41261)
Co-authored-by: Theo Nastos <theodore.nastos@n8n.io>
2026-10-16 13:54:34 +00:00
Jan
1f223c85f8
fix(editor): Start the empty-state icon cards cycling on load and use them on the MCP and external secrets pages (#40232)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-16 13:53:32 +00:00
Ricardo Espinoza
6a015ecb18
fix(core): Check the delete permission when the agent page moves a skill (no-changelog) (#41222) 2026-10-16 13:52:18 +00:00
n8n-cat-bot[bot]
033ec05bff
chore: Bump handlebars 4.7.9 → 4.7.10 (patch) to clear 6 Aikido findings (#41277)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-16 13:46:42 +00:00
Theodore Nastos
dd79a610c5
feat(core): Enable node and credential type policies by default (#41022) 2026-10-16 13:44:56 +00:00
Martin Rechenberger
a63a8edb55
fix(editor): Hide the Community Plus enrollment modal on Cloud (no-changelog) (#41251) 2026-10-16 13:40:37 +00:00
symbiote15
f5dc06f753
fix(Postgres Node): Keep comma-containing values as one query parameter (#40240) 2026-10-16 13:28:55 +00:00
Jaakko Husso
b78913f75d
fix(core): Keep the workflow source file when n8n Assistant sends blank source (#41201)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 13:28:54 +00:00
Guillaume Jacquart
dfec6dee81
fix(editor): Allow copy and cut of a selection with a restricted node (no-changelog) (#41276)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 13:25:51 +00:00
Uddish Verma
f92b4c11d6
feat(API): Limit Public API requests for each API key (no-changelog) (#41008)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 13:19:34 +00:00
Luigi Gubello
5120d25cbc
fix(editor): Count missing input apart in the QuickJS shadow report (no-changelog) (#41214)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 13:15:19 +00:00
Bernhard Wittmann
a6719b8db7
fix(Email Trigger (IMAP) Node): Check the mailbox when the trigger activates (#41013) 2026-10-16 13:10:25 +00:00
Bernhard Wittmann
7001886432
fix(core): Give each new Microsoft Teams sign-in credential a unique name (no-changelog) (#41256) 2026-10-16 13:04:21 +00:00
Sam Wooler
a906d2fdfc
feat(API): Return the parent folder on workflow read endpoints (#40981)
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-10-16 12:48:25 +00:00
Anne Aguirre
683fd11259
fix(core): Pass reasoning effort on open router model node (#41179) 2026-10-16 12:43:58 +00:00
Mike Repeć
1b02863c00
fix(core): Hand node reference results without an item shape to the engine (no-changelog) (#41083) 2026-10-16 12:25:04 +00:00
n8n-cat-bot[bot]
d0cb582163
chore: Bump fast-uri 3.1.5 → 3.1.8 (patch) to clear 12 Aikido findings (#41255)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-16 12:21:08 +00:00
Anne Aguirre
ea98bd0600
fix(AI Agent Node): Replay Anthropic thinking turns with their tool arguments (#41241) 2026-10-16 12:09:01 +00:00
Predrag Ristic
1405d4a351
feat(core): Add self-healing results to Inbox (#40741) 2026-10-16 12:07:09 +00:00
Anne Aguirre
f84413516a
fix(core): Web search misconfigs in agents as UserError (#41257) 2026-10-16 12:00:35 +00:00
Sandra Zollner
333a010112
feat(core): Store promotion exports in n8n directory (#40863)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-16 12:00:28 +00:00
Ali Elkhateeb
b7b3d72737
feat(API): Honor Idempotency-Key on Public API POST requests (#40978)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-16 11:58:36 +00:00
n8n-cat-bot[bot]
5a3fc39972
fix: Skip dependency files and continue after custom node load (#41117)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-16 11:53:13 +00:00
Rúben Castro
d460eb28ca
fix(editor): Explain why copy, cut, duplicate and pin refuse a restricted node (no-changelog) (#40997)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-16 11:51:01 +00:00
Francesca Sargent
4a5fdd5b68
fix(editor): Return to the requested page after SSO sign-in (#41151) 2026-10-16 11:42:25 +00:00
MHMD
e7f0607bf1
fix(core): Allow switching a node back to a credential the user cannot use after a save (#41198) 2026-10-16 11:39:13 +00:00
Sandra Zollner
64d8a3ad19
feat(core): Add promotion run and review data model (no-changelog) (#40794)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-16 11:28:06 +00:00
James Martin
c70b5d49b3
fix(core): Redact a parent execution if any sub-workflows have redaction enabled (#41129)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 11:02:43 +00:00
Stephen Wright
c28d130c6c
feat(Databricks Node): Explain Lakebase setup failures (no-changelog) (#40523) 2026-10-16 10:49:36 +00:00
MHMD
ca9661c091
fix(editor): Keep the used-in rows on the credential sharing tab when personal-space sharing is disabled (#41207) 2026-10-16 10:40:31 +00:00
Emre Caglar
a94f5dc8b7
feat(editor): Move the check rule and delete actions in the agent check row (no-changelog) (#41170)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:26:51 +00:00
Emre Caglar
fffea4c0c0
fix(editor): Remove the warning badge from the needs-work agent avatar (no-changelog) (#41164)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:26:41 +00:00
Emre Caglar
14e633d481
fix(editor): Keep rounded corners on a single row in the agent checks list (no-changelog) (#41166)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:26:28 +00:00
Emre Caglar
575eb6fa43
fix(editor): Show the check conversation stripes in dark mode (no-changelog) (#41167)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:26:19 +00:00
Emre Caglar
3ea0a3da98
fix(editor): Join the check conversation box to the practice-run strip (no-changelog) (#41168)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:26:06 +00:00
Emre Caglar
fed8b1ec69
feat(editor): Use a segment control for the agent checks status filter (no-changelog) (#41169)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 10:09:00 +00:00
Jan Kalkan
eba25caa70
feat(core): Require credential:create scope for create-with-values package imports (no-changelog) (#41131)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:55:56 +00:00
Sam Wooler
fe3627914a
fix(API): Log errors before returning generic 500 in Public API (#41186)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:54:41 +00:00
Jaakko Husso
aba5258930
feat(core): Add the scope column to the agents table (no-changelog) (#41165)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:52:52 +00:00
Bernhard Wittmann
aadbb7c498
fix(editor): Let the Assistant chat finish the Microsoft Teams setup (no-changelog) (#41188) 2026-10-16 09:38:51 +00:00
Predrag Ristic
8bee1eeb20
refactor(core): Prepare the shared Inbox (#40739)
Co-authored-by: kgrhartlage <kai.hartlage@n8n.io>
2026-10-16 09:37:52 +00:00
Eugene
2eab299dfa
fix(core): Pass Agent stop signals to foreground workflow tools (no-changelog) (#40776) 2026-10-16 09:21:44 +00:00
João Almeida
fc9f9575e2
fix(editor): Keep workflow name in edit mode after Rename menu action (#41120)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:04:26 +00:00
João Almeida
116a81e06e
fix(editor): Keep pending navigation when workflows list query is unchanged (#41125)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:04:20 +00:00
João Almeida
b36f6c0baf
test: Fix flaky zoom after route changes e2e test (#41130)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:04:09 +00:00
Riqwan Thamir
fa88c3e251
feat(core): Write finish_setup credentials into node tools (no-changelog) (#41219)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-16 08:49:51 +00:00
Dominik
8120328661
fix(editor): Rename Admin Panel to Cloud console (#41187) 2026-10-16 08:38:56 +00:00
Ricardo Espinoza
1ee29b2875
fix(editor): Lock the skill scope for users who cannot delete the skill (no-changelog) (#41221) 2026-10-16 08:28:41 +00:00
Ricardo Espinoza
f553331100
fix(editor): Keep the builder skill modal open on a taken name (no-changelog) (#41227) 2026-10-16 08:28:19 +00:00
Ricardo Espinoza
1a6b6a65d4
feat(editor): Name the project in the skill "available to" options (no-changelog) (#41229) 2026-10-16 08:27:34 +00:00
Ricardo Espinoza
88a2426947
fix(core): Trim skill names and refuse a name of only spaces (no-changelog) (#41223) 2026-10-16 08:18:55 +00:00
n8n-assistant[bot]
f5febab84d
chore: Update e2e impact map (#41245)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-16 08:18:15 +00:00
Ricardo Espinoza
dde9871497
feat(editor): Link to the skills settings page from the agent skill picker (no-changelog) (#41228) 2026-10-16 08:18:13 +00:00
Bernhard Wittmann
cc83b5bd77
feat(core): Use the agent's icon in the Microsoft Teams app package (no-changelog) (#40322) 2026-10-16 08:14:01 +00:00
Guillaume Jacquart
1ea18e9ef9
feat(editor): Show policy-blocked items in the source control pull modal (#40604)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 08:04:25 +00:00
n8n-cat-bot[bot]
1ba56b6579
chore: Bug - Credentials sharing tab placeholder text misleading (#41239)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: kgrhartlage <kai.hartlage@n8n.io>
2026-10-16 07:47:39 +00:00
Jaakko Husso
baa38dbdc6
feat(core): Add owned-session queries to the Agents thread repositories (no-changelog) (#40615)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 07:45:16 +00:00
Jaakko Husso
601f6ac0da
feat(editor): Let the Agents chat host render host events by placement (no-changelog) (#40720)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 07:40:56 +00:00
OrdoAbChao
e29c3a4085
fix(Splunk Node): Preserve errors when continuing after failed items (#40368) 2026-10-16 07:35:28 +00:00
Jaakko Husso
447fc866bb
feat(core): Add hidden turns, turn options and a system kind to the Agents queue (no-changelog) (#40614)
Some checks failed
CI: Master (Build, Test, Lint) / Build & format (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Lint (push) Has been cancelled
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Has been cancelled
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:25:36 +00:00
Ricardo Espinoza
860f1273eb
feat(editor): Add the Skills page to Context settings (no-changelog) (#41031) 2026-10-15 15:35:20 +00:00
Jan Kalkan
deed9a2f32
feat(core): Report credentials created with partial package data separately (no-changelog) (#41156)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:29:25 +00:00
nvonahsen
a32c652c09
fix(editor): Prevent doubled base path when opening folders with N8N_PATH set (#33065)
Signed-off-by: Niclas von Ahsen <niclas.vonahsen@elementx.ai>
Signed-off-by: James Gee <1285296+geemanjs@users.noreply.github.com>
Co-authored-by: James Gee <1285296+geemanjs@users.noreply.github.com>
2026-10-15 15:25:39 +00:00
Ricardo Espinoza
1903d53bc0
fix(editor): Save only the OpenTelemetry status when enabling or disabling (#41142) 2026-10-15 14:43:53 +00:00
n8n-cat-bot[bot]
131e9c8679
fix: Keep free-floating sticky notes clear of workflow nodes (#41195)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-15 14:41:40 +00:00
Bernhard Wittmann
236220b248
fix(Email Trigger (IMAP) Node): Find new mail on servers with a broken IMAP4rev2 mode (#41018) 2026-10-15 14:33:49 +00:00
Toni Devine
af0202d840
feat(editor): Rework the WhatsApp channel setup to match the new design (#40923)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:33:41 +00:00
yehorkardash
adc6d35388
feat: Update Slack Bot to n8n rebrand (no-changelog) (#41203) 2026-10-15 14:25:48 +00:00
Svetoslav Dekov
b2b2a8641a
feat(editor): Attach skills hub skills from a picker in the agent builder (no-changelog) (#41104)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:14:18 +00:00
James Martin
952b1e73b2
fix(core): Display an error message when failing to connect to oauth credentials (#41058) 2026-10-15 14:03:47 +00:00
Anne Aguirre
f596162dc7
fix(core): Close agent after run when invoked from workflow (#41128) 2026-10-15 13:44:45 +00:00
MHMD
378847ae1e
refactor(core): Share the inbound admission steps between protected-resource surfaces (#41162) 2026-10-15 13:32:58 +00:00
n8n-cat-bot[bot]
9e3b52cbbb
fix: Restore command bar focus and standardize creation icons (#41185)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-15 12:58:16 +00:00
Checo
054288e488
fix(GitlabTrigger): use NodeApiError.httpCode and safe description access in webhook checkExists (#35486)
Signed-off-by: sergioperezcheco <checo520@outlook.com>
2026-10-15 12:53:34 +00:00
Mike Repeć
c839d6cd49
fix(core): Remove unused N8N_SCHEDULER_MIN_INTERVAL setting (#41194) 2026-10-15 12:29:35 +00:00
Nour Alhadi Mahmoud
154cd63bbc
feat(editor): Add GitLab promotion connector setup (no-changelog) (#41019) 2026-10-15 11:58:24 +00:00
Ricardo Espinoza
3c447b42a7
fix(core): Key each OAuth authorization request to its own session (#39930)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-15 11:43:45 +00:00
Yen Su
1659fa3737
feat(Databricks Node): Refuse the Vector Store metadata filter on non-standard endpoints (no-changelog) (#41047) 2026-10-15 11:18:08 +00:00
Luigi Gubello
9806798e89
chore: Update @langchain/langgraph to 1.4.21 (#40680)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 11:14:29 +00:00
Charlie Kolb
cdeb405948
refactor(core): Extract Workflow Builder session persistence into a backend module (#41075) 2026-10-15 10:56:13 +00:00
Dimitri Lavrenük
e45a65a9bf
fix(core): Improve base URL check for declarative nodes (#40778) 2026-10-15 10:51:15 +00:00
Charlie Kolb
b68bd3129b
refactor(core): Move auth registries and OAuth proxies into shared packages (#41088) 2026-10-15 10:32:13 +00:00
Lukas
e8ded61fe1
build: Update eslint-plugin-n8n-nodes-base to 2.0.0 for ESLint 10 (#38653)
Co-authored-by: Bernhard Wittmann <bernhard.wittmann@n8n.io>
2026-10-15 10:28:37 +00:00
Bernhard Wittmann
d046047dd0
feat(editor): Update the Microsoft Teams setup copy and give both flows a Done step (no-changelog) (#41010) 2026-10-15 10:10:06 +00:00
Srinjoy
f483ebd9d1
fix(MCP Trigger Node): Pass through MCP content arrays instead of double-wrapping tool results (#39131) 2026-10-15 10:08:42 +00:00
Luigi Gubello
fddef57734
chore: Update toml to 4.2.0 (#40672)
Some checks failed
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Build: Benchmark Image / build (push) Has been cancelled
Util: Publish API Schema / Cloudflare Pages (push) Has been cancelled
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:34:39 +00:00
Sebastien Powell
ba97e46afc
fix(editor): Cleanup legacy button variants (#41175) 2026-10-15 09:31:34 +00:00
Anne Aguirre
4cb7a6ee3d
fix(core): Clear agent runtime cache when credential is updated (#40961) 2026-10-15 09:26:22 +00:00
Luigi Gubello
8ed8607a52
chore: Update simple-git to 4.0.2 (#40675)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:24:16 +00:00
Yen Su
67c1981f2f
fix(core): Let LLM failed-attempt handling retry and honour Retry-After (#41002) 2026-10-15 09:13:44 +00:00
Luigi Gubello
a8880cb449
chore: Update pdfjs-dist to 6.3.289 (#40677)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:13:43 +00:00
Stephen Wright
d411a44be0
feat(Databricks Node): Add the Lakebase Insert operation (no-changelog) (#40520) 2026-10-15 09:12:26 +00:00
Ricardo Espinoza
0c0b38e3cb
feat(core): Turn the skills hub on with an environment variable instead of a PostHog flag (no-changelog) (#41143) 2026-10-15 08:59:25 +00:00
Mayank SIngh
315e0a5377
fix(Convert to File Node): Decode data URIs and validate Base64 input (#38554)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-10-15 08:57:21 +00:00
Gigi
efd4e21aad
fix(core): Evict idle mcp-browser sessions to prevent connection leak (#36850)
Signed-off-by: Luigi Colluto <luigicolluto2006@gmail.com>
2026-10-15 08:56:27 +00:00
Jaakko Husso
bab83634ce
perf(core): Reuse the prebaked runtime skills bundle in the sandbox (no-changelog) (#41027)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 08:35:34 +00:00
n8n-assistant[bot]
d6c9b9fba3
chore: Update e2e impact map (#41163)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-15 08:18:58 +00:00
Luigi Gubello
b36befcb75
chore: Update nodemailer to 10.0.14 (#40674)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 08:09:12 +00:00
Charlie Kolb
ed2500ffb2
refactor(core): Move feature provider proxies into backend services (#41087) 2026-10-15 08:00:25 +00:00
n8n-cat-bot[bot]
9c0b546a30
chore: Bug - Delete project confirmation dialog button misaligned (#41160)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-15 07:54:00 +00:00
Charlie Kolb
07595a0007
ci: Respect pnpm workspace exclusions in dependency verifier (#41159) 2026-10-15 07:48:25 +00:00
Nour Alhadi Mahmoud
1e67d0315d
feat(API): Add GitLab promotion repository and branch discovery (no-changelog) (#40788) 2026-10-15 07:15:40 +00:00
Charlie Kolb
338b0ca3fd
refactor(core): Move published workflow data into backend services (#41086) 2026-10-15 07:13:46 +00:00
Jaakko Husso
48edbef643
fix(core): Keep Observer detail lines instead of discarding the response (#41134)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:13:19 +00:00
Charlie Kolb
5d79c16f51
test(core): Stabilize transport timeout test (#41150) 2026-10-15 07:03:20 +00:00
Theodore Nastos
40094e363b
chore: Assign policy and OpenTelemetry code to Governance & Observability in OWNERS (no-changelog) (#41050) 2026-10-15 06:30:12 +00:00
Alex Antoniuk
e2529a8494
perf(editor): Load prettier only when the user formats code (#41014)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
2026-10-14 21:48:42 +00:00
Raúl Gómez Morales
50c4ced224
fix(core): Keep pending setup for nodes that n8n Assistant added in an earlier build (#41070) 2026-10-14 16:52:18 +00:00
Bernhard Wittmann
b7a5500d16
feat(editor): Align the Microsoft Teams channel settings with the new design (no-changelog) (#41054) 2026-10-14 16:52:09 +00:00
Mayank SIngh
f6b24d0ae9
fix(editor): Display custom evaluation metrics correctly (#40852) 2026-10-14 16:15:34 +00:00
Raúl Gómez Morales
3c7dcf902d
feat(ai-builder): Write LangTracer results from the routing evals (no-changelog) (#40954) 2026-10-14 15:42:42 +00:00
Mike Repeć
fbe743b9df
fix(core): Charge element content for comparing array methods in native evaluation (no-changelog) (#41079) 2026-10-14 15:40:38 +00:00
Andreas Fitzek
f5924004ca
feat(core): Accept OAuth2 bearers on the public API through the trusted-source pipeline (no-changelog) (#41127)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-14 15:34:39 +00:00
Mike Repeć
b270d48686
fix(core): Charge node reference calls against the native evaluation work budget (no-changelog) (#41080) 2026-10-14 15:19:55 +00:00
Mike Repeć
3277f8c3b5
test(core): Give node reference parity cases a slim fixture item (no-changelog) (#41081) 2026-10-14 15:19:46 +00:00
Yuliia Pominchuk
a57c8f3aba
fix(core): Redact the parent execution when a sub-workflow uses a credential the viewer cannot use (#41085)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 15:15:42 +00:00
Ricardo Espinoza
dcd312c5a9
feat(core): Copy inline agent skills into the skills hub at startup (no-changelog) (#41030)
Co-authored-by: Roman Makowiak <roman.makowiak@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:08:50 +00:00
Mike Repeć
a6d5b4de3f
test(core): Pin the quickjs nested -0 conversion in native evaluation parity (no-changelog) (#41092) 2026-10-14 15:04:02 +00:00
Temo
3968a60bf7
fix(ERPNext Node): Return each row once and tolerate empty Get Many options (#36860) 2026-10-14 14:51:43 +00:00
Theodore Nastos
17fb3c0e17
docs(API): Label type availability policy endpoints as Preview (no-changelog) (#41101) 2026-10-14 14:26:59 +00:00
Nikhil Kuriakose
b965e90307
test: Move Adore-owned specs to Knowledge and retire the Adore owner (#40933)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-14 14:18:23 +00:00
Bernhard Wittmann
38135e1f83
feat: Read files attached in Microsoft Teams channels and group chats (no-changelog) (#41001) 2026-10-14 13:51:02 +00:00
Anne Aguirre
616062487a
feat(editor): Show agent personalisation icon in subagent modal (#41094) 2026-10-14 13:47:36 +00:00
Mike Repeć
1950434a27
fix(core): Create an XML parser for each webhook request body (#41098) 2026-10-14 13:42:48 +00:00
Anne Aguirre
b3e8b7d9ce
fix(editor): Use the deployment name on the Az Foundry path (#41103) 2026-10-14 13:35:40 +00:00
Jaakko Husso
56eac665c3
fix(core): Make the Assistant check the n8n side before it names a failure cause (no-changelog) (#41097)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:26:37 +00:00
Luigi Gubello
8bc7161ab1
feat(editor): Compare the QuickJS expression engine with legacy behind a flag (no-changelog) (#40496)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:24:09 +00:00
Uddish Verma
57e26feda4
feat(API): Add the Public API rate limit environment variables (no-changelog) (#41003)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:21:34 +00:00
n8n-cat-bot[bot]
9c42a76b1a
test: Stabilize user select email-filtering tests (#41095)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-14 13:17:45 +00:00
Irénée
ae374e79ab
feat(core): Add a credential conflict policy to package import (no-changelog) (#40963)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:11:32 +00:00
n8n-cat-bot[bot]
ef8d61bda7
fix(API): Check scope version when replacing policy attachments (no-changelog) (#40964)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-14 13:06:15 +00:00
Ilfat Mindubaev
caeaff7230
feat(editor): Show the model that answered in the AI log header (#40988)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:58:13 +00:00
Johannes Nicolai
a3f47117cf
fix(OpenAI Node): Report token usage for image operations (#38463)
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-14 12:56:44 +00:00
Toni Devine
9a524e7c21
feat(core): Add WhatsApp credential check and webhook handshake status (#40921)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:48:31 +00:00
Toni Devine
adedbbf560
feat(core): Add typing indicator, media download and media retention … (#40920)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:48:20 +00:00
Bernhard Wittmann
4eb63d6f41
feat: Update Browser Use extension to n8n rebrand (no-changelog) (#41096) 2026-10-14 12:38:28 +00:00
Charlie Kolb
b6d39e2d30
refactor(core): Move JWT support into backend services (#41089) 2026-10-14 12:10:59 +00:00
Sam Wooler
7920e6cf6b
refactor(API): Retire the express-openapi-validator stack from the Public API (no-changelog) (#40801)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:07:18 +00:00
Charlie Kolb
76c43094cd
refactor(core): Extract Quick Connect into a backend package (#41076) 2026-10-14 12:04:16 +00:00
Jaakko Husso
898701b05b
fix(core): Skip resource mapper validation when the mapped value is null (#41069)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:00:20 +00:00
Kai
f63111202f
fix(core): Confirm project deletions before instance apply (#40941) 2026-10-14 11:50:52 +00:00
Mike Repeć
633eacaae6
fix(editor): Use an outline button for bulk notify in the migration report (no-changelog) (#41084) 2026-10-14 11:29:29 +00:00
José Braulio González Valido
1b0cdd5998
fix(ai-builder): Report eval helper model failures as infrastructure, not as bad JSON (no-changelog) (#40415)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 11:23:25 +00:00
Raúl Gómez Morales
578eea403d
fix(core): Stop asking to set up a node when n8n Assistant only adds a step next to it (#41067) 2026-10-14 11:20:43 +00:00
José Braulio González Valido
7c38974070
fix(ai-builder): Keep eval mock notes to data, not response layouts (no-changelog) (#40818)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:20:06 +00:00
Mike Repeć
d68ad7e97b
feat(core): Evaluate pure expression extensions in native expression evaluation (no-changelog) (#40865) 2026-10-14 10:49:37 +00:00
Declan Carroll
0650b9cce0
refactor: Move idle memory baselines to efficiency (#41037) 2026-10-14 10:32:11 +00:00
Mike Repeć
df7b3a19c6
fix(editor): Use outline buttons for secondary actions in the migration report (no-changelog) (#41072) 2026-10-14 10:28:04 +00:00
Mike Repeć
de42c9515d
feat(editor): Let admins notify workflow owners in bulk from the migration rule detail (#41060) 2026-10-14 10:26:05 +00:00
Jaakko Husso
d26f914e3d
fix(editor): Cancel the Assistant thread when the hand-off navigation fails (#41056)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:12:32 +00:00
Johannes Nicolai
8b5ea6d9e0
fix(editor): Stop auto-selecting the Evaluation Trigger over other triggers (#38883)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bernhard Wittmann <bernhard.wittmann@n8n.io>
2026-10-14 09:57:23 +00:00
Charlie Kolb
bfd4885026
ci: Automate get-n8n fallback version updates (#40973) 2026-10-14 09:53:55 +00:00
Stephen Wright
3800bce812
feat(Text Classifier Node): Add opt-in confidence scores to the output (#40743) 2026-10-14 09:42:05 +00:00
Mike Repeć
aae8eb63d7
docs: Add env-var-docs skill for JSDoc on @Env config properties (#41053) 2026-10-14 09:19:42 +00:00
Mutasem Aldmour
0b4c99f10d
fix(core): Pick output schemas by parameter value for Gmail and Notion (no-changelog) (#40757)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 09:11:49 +00:00
n8n-cat-bot[bot]
066480d8db
fix: Check provisioning license on Users settings page (#41048)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-14 08:59:23 +00:00
Florian Gareis
b8780d8fee
fix(HomeAssistant Node): Only sent Home Assistant attributes on upsert if set (#36298) 2026-10-14 08:58:22 +00:00
Anne Aguirre
e47940eecd
feat(editor): Publish agent modal and modal for warning about unpublished workflows (#40987)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-14 08:47:05 +00:00
Jaakko Husso
e3b2603d7b
chore(core): Remove activation-capped AI Assistant trial variant (no-changelog) (#41036)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:21:20 +00:00
Jaakko Husso
726a5829c8
fix(core): Use the module registry to decide if the Assistant has the Agents features (#41026)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:19:48 +00:00
n8n-assistant[bot]
54f1670791
chore: Update e2e impact map (#41051)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-14 08:18:31 +00:00
Jan
b3e8634c83
fix(editor): Apply the standard settings layout to the Context preferences page (#40716)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-14 08:02:30 +00:00
Rick Gale
1643a8598b
chore: Assign evaluation code to the AI Trust team (no-changelog) (#40918)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-14 08:01:17 +00:00
Anne Aguirre
9498088256
feat(core): Apply budget alerts and stops to sub-agent run (#40884)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-14 07:56:27 +00:00
Yuliia Pominchuk
da3491fc87
feat(editor): Link to the credential sharing tab when execution data is redacted (#40976) 2026-10-14 07:48:34 +00:00
Yen Su
d83f95b377
fix(Databricks Node): Mark the MLflow tool span ERROR when the tool returned an error string (no-changelog) (#40990) 2026-10-14 07:27:56 +00:00
n8n-assistant[bot]
46761a60a9
chore: Bundle/2.x (#41035)
Co-authored-by: Matsu <matias.huhta@n8n.io>
Co-authored-by: Bernhard Wittmann <bernhard.wittmann@n8n.io>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Elias Meire <elias@meire.dev>
Co-authored-by: n8n-assistant[bot] <n8n-assistant[bot]@users.noreply.github.com>
Co-authored-by: João Almeida <jg.almeida56@gmail.com>
Co-authored-by: Savelii <savelii.sychov@n8n.io>
Co-authored-by: Rick Gale <richardgale90@gmail.com>
Co-authored-by: Emilia <100027345+sovietspaceship@users.noreply.github.com>
Co-authored-by: oleg <me@olegivaniv.com>
Co-authored-by: RomanDavydchuk <roman.davydchuk@n8n.io>
Co-authored-by: Stephen Wright <sjw948@gmail.com>
Co-authored-by: Yuliia Pominchuk <31064937+yuliia-pominchuk@users.noreply.github.com>
Co-authored-by: Shireen Missi <94372015+ShireenMissi@users.noreply.github.com>
Co-authored-by: Miguel Ángel Moreno <1411774+miguelsaddress@users.noreply.github.com>
Co-authored-by: Anne Aguirre <anne.aguirre@n8n.io>
Co-authored-by: Dimitri Lavrenük <20122620+dlavrenuek@users.noreply.github.com>
Co-authored-by: Kai <kai.hartlage@n8n.io>
Co-authored-by: Luigi Gubello <27034386+luigigubello@users.noreply.github.com>
Co-authored-by: Danny Martini <danny@n8n.io>
Co-authored-by: MHMD <45968454+dou-mhmd@users.noreply.github.com>
Co-authored-by: Mutasem Aldmour <4711238+mutdmour@users.noreply.github.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Charlie Kolb <charlie@n8n.io>
2026-10-14 06:33:56 +00:00
Bernhard Wittmann
ccf42e3213
fix(ai-builder): Make browser credential-setup evals grade the right thing (no-changelog) (#40858) 2026-10-14 06:24:28 +00:00
Svetoslav Dekov
94bdf85a8a
feat(core): Switch agent skills to the skills hub (no-changelog) (#40860)
Co-authored-by: Ricardo Espinoza <ricardo@n8n.io>
Co-authored-by: Roman Makowiak <roman.makowiak@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:12:36 +00:00
José Braulio González Valido
29de1c63ae
fix(ai-builder): Remove the Context7 docs lookup from the eval mock (no-changelog) (#40021)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:30:08 +00:00
José Braulio González Valido
92f8eb51b3
chore(Message an Agent Node): Add output schemas (no-changelog) (#40821)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 21:29:33 +00:00
José Braulio González Valido
146d7804af
fix(ai-builder): Count eval runs that read another build's workflow as framework issues (no-changelog) (#40717)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:29:00 +00:00
José Braulio González Valido
88de3cac20
feat(ai-builder): Move eval judges to Sonnet 5.5 and add per-role eval LLM settings (no-changelog) (#40330)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:28:25 +00:00
Roman Makowiak
04f4afe3dc
feat(core): Add skills repository, service and REST API (no-changelog) (#40653)
Co-authored-by: Ricardo Espinoza <ricardo@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:28:55 +00:00
Nikhil Kuriakose
6924027e93
test: Move Adore-owned specs for Context features to Context (#40932)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 19:41:28 +00:00
Nikhil Kuriakose
ef183e632d
test: Move Adore-owned specs for the updates panel to Cloud Growth (#40931)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 19:41:27 +00:00
Nikhil Kuriakose
ed1c1e561a
test: Move Adore-owned specs for node features to NODES (#40930)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 19:41:27 +00:00
Nikhil Kuriakose
a3a014422f
test: Move Adore-owned specs for Catalysts features to Catalysts (#40929)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 19:41:26 +00:00
Ricardo Espinoza
f1fecf97e9
feat(core): Add skills hub tables (no-changelog) (#40464) 2026-10-13 19:03:23 +00:00
Andreas Fitzek
5da4d19e50
feat(core): Add the trusted sources management API (no-changelog) (#40975)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-13 18:24:34 +00:00
Uddish Verma
2019ea5593
feat(API): Expose the API key ID on the token grant (no-changelog) (#40986)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:16:50 +00:00
Guillaume Jacquart
4419fc4ba2
feat(core): Add the blocking project and opt-in state to policy violations (#40985)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 17:49:51 +00:00
yehorkardash
52aed7ac97
fix(Execute Command Node): Don't spawn console on Windows (#40909) 2026-10-13 16:07:56 +00:00
n8n-cat-bot[bot]
1d2c879fab
fix: Pass structured inputs to OpenAI Responses tools (#40638)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-13 16:07:14 +00:00
Rob Hough
bffe7d8392
feat(editor): Update UI with n8n rebrand (#40342)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
Co-authored-by: Sebastien Powell <sebpowell@users.noreply.github.com>
Co-authored-by: Tuukka Kantola <tuukka@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 16:06:30 +00:00
n8n-cat-bot[bot]
d245b17b89
fix: Allow skipping table initialization (#40437)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-13 16:03:37 +00:00
daniel-hai
8977833ee2
feat(Monday.com Node): Add V2 with full V1 parity as the default (#35515)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Stephen Wright <sjw948@gmail.com>
2026-10-13 15:53:55 +00:00
Alexander Gekov
acf5291d80
feat: Show Gemini models by name and list prioritized models first in model pickers (#40965)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-13 15:41:36 +00:00
Rob Hough
ebf078b64c
fix(editor): Restore the new session shortcut in agent chat (#40907)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
2026-10-13 15:29:20 +00:00
Charlie Kolb
9052f8ff3e
test(editor): Reduce publish timeline paging test work (#40993) 2026-10-13 15:05:18 +00:00
Kai
e0fe04084e
fix(core): Allow project promotion without an instance export (#40939) 2026-10-13 14:31:19 +00:00
Yen Su
e859abaf18
fix(Databricks Node): Bring the AI Agent panel to parity with AI Agent 2.2 (#40989) 2026-10-13 14:10:45 +00:00
Mike Repeć
a972eebe51
feat(editor): Let admins notify workflow owners from the migration rule detail (#40949) 2026-10-13 13:57:26 +00:00
yehorkardash
739c59e991
feat(core): Implement automated migrations for removed nodes (#40877) 2026-10-13 13:45:24 +00:00
Mike Repeć
de090cb2a0
feat(core): Evaluate primitive reads off $now and $today natively (no-changelog) (#40945) 2026-10-13 13:31:32 +00:00
MHMD
b97fe7c7d4
fix(core): Hide personal credentials from agents until they are fully supported (no-changelog) (#40789) 2026-10-13 13:31:17 +00:00
Riqwan Thamir
38e4aebfc8
perf(core): Reduce Agent Builder round trips in an initial build (no-changelog) (#40719)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-13 13:29:11 +00:00
Mike Repeć
d8a06666da
feat(core): Add an endpoint that notifies workflow owners about migration findings (#40952) 2026-10-13 13:07:59 +00:00
Bernhard Wittmann
60e0cb1e7b
feat(core): Publish the Microsoft Teams app to the organisation (no-changelog) (#40697)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-13 13:01:48 +00:00
Rup Sarmah
b09efb706c
fix(core): Handle every escaped closing bracket in an expression (#40487) 2026-10-13 12:58:10 +00:00
Tomi Turtiainen
f090f53480
refactor(core): Select CLI start trigger for workflow runs (#40375)
Co-authored-by: Danny Martini <danny@n8n.io>
2026-10-13 12:35:36 +00:00
Mike Repeć
347385b07b
feat(core): Treat notified migration findings as unresolved (#40951) 2026-10-13 12:19:30 +00:00
Guillaume Jacquart
bb45bca5ff
feat(editor): Show the policy reason when an agent tool call is refused (#40673)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 12:19:29 +00:00
Sam Wooler
3075b60576
fix(API): Apply projectId filter to executions for member API keys (#40887)
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-10-13 12:03:20 +00:00
Sam Wooler
4fcb628fca
refactor(API): Migrate n8n-packages export endpoint to PublicApiController (no-changelog) (#40756)
Some checks failed
Build: Benchmark Image / build (push) Has been cancelled
CI: Master (Build, Test, Lint) / Build & format (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Lint (push) Has been cancelled
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Has been cancelled
Util: Publish API Schema / Cloudflare Pages (push) Has been cancelled
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Has been cancelled
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:56:11 +00:00
Bernhard Wittmann
1ee379898b
feat(core): Add the recommended Microsoft Teams channel setup (no-changelog) (#40696)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-13 11:55:56 +00:00
Lorent Lempereur
366ed6ed0e
refactor(core): Declare system tasks with a typed schedule and target (no-changelog) (#40703) 2026-10-13 11:42:37 +00:00
Anne Aguirre
248c86355b
fix(core): Describe MCP servers only from user or registry text in Agent Builder (#40974) 2026-10-13 11:19:46 +00:00
Stephen Wright
e5866ac6c1
feat(Kafka Node): Make version 2 the default for new Kafka nodes and triggers (#40744) 2026-10-13 11:07:10 +00:00
Jaakko Husso
2bc3265e63
perf(ai-builder): Trust a matching prebaked bundle manifest in the sandbox (no-changelog) (#40820)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:04:18 +00:00
Lorent Lempereur
f6861fe494
feat(core): Enable the durable scheduler by default (#40891)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-13 10:47:35 +00:00
Anne Aguirre
632d8b9eb0
feat(editor): Update budget panel when agent executes (#40971)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-13 10:44:26 +00:00
Declan Carroll
5549374241
chore: Upgrade agent harness to 0.6.0 (no-changelog) (#40967) 2026-10-13 10:28:40 +00:00
Alex Grozav
cba16247b9
feat(editor): Open the migration workflow from its name (no-changelog) (#40895)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 10:24:15 +00:00
Alex Grozav
e528f948ce
fix(editor): Use full width for the migration rule header (no-changelog) (#40890)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 10:24:02 +00:00
James Gee
d3bc2754b4
feat(core): Add create-with-values credential missing mode to package import (#36767) 2026-10-13 10:21:33 +00:00
Jaakko Husso
ffda639b3f
fix(editor): Show follow-up run events in the Assistant after a reconnect (#40883)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 10:12:33 +00:00
Declan Carroll
97e5ae56e4
test: Add Dive checks and reusable Playwright CLI fixtures (#40831) 2026-10-13 10:02:00 +00:00
yehorkardash
bda9118168
feat: Overhaul MCP tool permissions for Agents (#39878)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-13 09:54:46 +00:00
Csaba Tuncsik
96eff1efb9
fix(core): Reflect failed publications in the active workflows endpoints (#40641) 2026-10-13 09:49:33 +00:00
Guillaume Jacquart
eb66e1bde0
docs(API): Mark type availability policy endpoints as beta (#40959)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 09:04:12 +00:00
Tomi Turtiainen
a8a028e2dc
docs: Clarify primary key selection order in migration skill (#40958)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-13 09:03:31 +00:00
Yen Su
44c8a312c3
feat(core): Append node tool attributions to agent replies and add Genie attribution and deep links to the Databricks node (#40765) 2026-10-13 08:42:57 +00:00
Declan Carroll
7bad46ee7e
chore: Upgrade agent harness to 0.5.1 (no-changelog) (#40956) 2026-10-13 08:34:04 +00:00
Veljko Terzic
45eea1bc35
feat(core): Remove the startup pass from workflow history compaction (#40886) 2026-10-13 08:02:51 +00:00
Stephen Wright
c8c8c3a81e
feat(Databricks Node): Add the Lakebase Get Many operation (no-changelog) (#40508) 2026-10-13 07:52:26 +00:00
Florian Gößler
e74f1c19da
fix(core): Log custom data truncation only above the 512-character limit (#40899)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-13 07:49:13 +00:00
Alex Grozav
f16c1375b1
feat(editor): Add bulk migration to the migration rule detail (no-changelog) (#40896)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 07:48:53 +00:00
n8n-release-tag-merge[bot]
5146e3960f Merge tag 'n8n@2.44.0' 2026-10-13 07:44:08 +00:00
Yen Su
14c77900df
fix(Databricks Node): Rename Genie resource labels to Genie Agent and the MCP tile to Databricks Genie One (#40760) 2026-10-13 07:21:37 +00:00
n8n-assistant[bot]
bae368caca
🚀 Release 2.44.0 (#40943)
Co-authored-by: n8n-release-helper[bot] <265227495+n8n-release-helper[bot]@users.noreply.github.com>
2026-10-13 07:12:07 +00:00
Mike Repeć
f3e2bd83b3
feat(core): Suspend in-flight production executions at worker shutdown (#40568) 2026-10-13 07:04:07 +00:00
Jaakko Husso
27fcb968fb
fix(core): Require workflow create scope for single node runs in n8n Assistant (#40908)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 06:40:58 +00:00
n8n-assistant[bot]
2562744611
chore: Update e2e impact map (#40942)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-13 06:40:48 +00:00
Pierantonio Cangianiello
8d93e31d1d
chore: Make the create-pr skill write shorter PR descriptions for new reviewers (#40811)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 05:25:35 +00:00
ADITYA KUMAR JHA
2596f3ac1c
fix(core): Match regex case-insensitively when ignore case is enabled (#33589)
Co-authored-by: Michael Kret <michael.k@radency.com>
2026-10-13 04:03:35 +00:00
Svetoslav Dekov
e2c4054cba
feat(editor): Replace Windsurf with Devin in the MCP client dialog (#40912)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 00:17:07 +00:00
Anne Aguirre
2ca4e55502
fix(core): Stabilise background child approval wait (#40834)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-12 21:18:01 +00:00
n8n-cat-bot[bot]
cea974e770
test: Stabilize MCP workflow select empty-state test (no-changelog) (#40926)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-12 18:39:14 +00:00
Luka Zivkovic
ce20d1dea4
fix: Load agent evals by default and share the rollout flag (no-changelog) (#40915) 2026-10-12 16:42:27 +00:00
Svetoslav Dekov
53d1fd5991
feat(editor): Show last updated date on the Preferences page (no-changelog) (#40869)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 15:16:43 +00:00
Veljko Terzic
776f4e1944
feat(core): Run the execution pruning hard delete as a durable system task (no-changelog) (#40661) 2026-10-12 15:01:55 +00:00
Mike Repeć
3ff9c83bda
fix(core): Validate the size argument of the chunk() expression extension (#40880) 2026-10-12 14:54:04 +00:00
Guillaume Jacquart
89733cdb9a
feat(core): Send telemetry when a policy blocks an action (no-changelog) (#40871)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 14:51:40 +00:00
Jaakko Husso
0209caafff
feat(editor): Let the Agents chat host render results of finished tool calls (no-changelog) (#40611)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 14:26:14 +00:00
Andreas Fitzek
0755255c35
feat(core): Add internal API for running the inbound authentication pipeline (no-changelog) (#40894) 2026-10-12 14:05:26 +00:00
James Martin
36ae8f74ce
chore(core): Seeds the default internal trusted source (no-changelog) (#40758)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 13:55:20 +00:00
Nour Alhadi Mahmoud
160c126c6e
feat(core): Add GitLab promotion providers (no-changelog) (#40749) 2026-10-12 13:55:19 +00:00
Mike Repeć
d3deffbfa5
fix(core): Match the Brazilian real symbol in the in-isolate currency regex (#40893) 2026-10-12 13:55:17 +00:00
Anne Aguirre
b542f66119
fix(editor): Remove redundant description for create workflow option in agent config (#40882) 2026-10-12 13:54:15 +00:00
n8n-cat-bot[bot]
f0a38cfe58
feat: Add Antigravity to the MCP client connection dialog (#40879)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Svetoslav Dekov <svetoslav.dekov@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 13:53:19 +00:00
Jan Kalkan
27b43de85f
feat(core): Pair data table columns by id on package import (#40709)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 13:40:47 +00:00
Alex Grozav
7ceb84f06c
fix(editor): Fix the last updated label in the migration report (no-changelog) (#40889)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 13:38:00 +00:00
Alex Grozav
2a51f38d55
fix(editor): Rename the "Executions fail" label to "Executions will fail" (#40888)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 13:37:54 +00:00
Alex Grozav
e0958fe094
feat(editor): Suggest won't fix for workflows that have not run in 12 months (no-changelog) (#40782)
Co-authored-by: Michael Siega <michael.siega@n8n.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 13:33:30 +00:00
MHMD
bc8b803471
test: Wait for the Overview list before the sidebar click in a flaky project test (no-changelog) (#40864) 2026-10-12 13:26:08 +00:00
Danny Martini
26d93b601a
docs(engine): Record the resume-token decision (no-changelog) (#39089)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-12 13:17:09 +00:00
Pierantonio Cangianiello
c9cb3ee3ab
fix(core): Let the editor download binary data of an Engine v2 execution (#40663) 2026-10-12 13:08:38 +00:00
Svetoslav Dekov
22ff2a0636
feat(editor): Show who created each preference on the Preferences page (#40868)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 13:04:59 +00:00
Alex Grozav
7dae1b64de
feat(editor): Add bulk state actions to the migration rule detail (no-changelog) (#40780)
Co-authored-by: Michael Siega <michael.siega@n8n.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 12:51:08 +00:00
Stephen Wright
27aa660a74
fix(Microsoft SharePoint Trigger Node): Name the real cause in the missing-target error (no-changelog) (#40400) 2026-10-12 12:44:08 +00:00
Veljko Terzic
6cd3539b1e
feat(core): Run the pending-authorization cleanup as a durable system task (no-changelog) (#40859) 2026-10-12 12:17:23 +00:00
Jaakko Husso
97ca4c49e0
feat(editor): Let the Agents chat host provide its own interactive cards (no-changelog) (#40609)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:08:23 +00:00
Robin Braumann
bd22105646
feat(core): Add Google Vertex AI support for agents (no-changelog) (#40545) 2026-10-12 12:05:53 +00:00
Lorent Lempereur
e16066c322
feat(core): Give long system tasks an explicit timeout (no-changelog) (#40521) 2026-10-12 12:00:59 +00:00
Andreas Fitzek
7cd9fcd3d1
feat(core): Map roles from trusted-source rules on resolve (no-changelog) (#40809)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 11:53:47 +00:00
Mutasem Aldmour
2176819865
fix(core): Validate execution timeout against the configured maximum in Instance AI saves (#40769)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 11:48:02 +00:00
Ali Elkhateeb
d038f89e91
feat(API): Delete expired Public API idempotency keys (no-changelog) (#40748)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Sam Wooler <swooler592@gmail.com>
2026-10-12 11:45:00 +00:00
Lorent Lempereur
3dd805d9ed
feat(core): Bound a poll by the timeout of its occurrence (no-changelog) (#40518)
Some checks are pending
Build: Benchmark Image / build (push) Waiting to run
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: Veljko Terzic <veljko.terzic@n8n.io>
2026-10-12 11:31:52 +00:00
Michael Drury
16e29da52f
fix(editor): Keep typed characters in Markdown fields (#40805) 2026-10-12 11:25:33 +00:00
n8n-cat-bot[bot]
b8b0387a31
fix: Use ID token claims when UserInfo is unavailable (#40779)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Andreas Fitzek <andreas.fitzek@n8n.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 11:14:06 +00:00
Lorent Lempereur
8b6352a6fa
feat(core): Report a timed-out system task run as timed out (no-changelog) (#40516)
Co-authored-by: Veljko Terzic <veljko.terzic@n8n.io>
2026-10-12 10:46:12 +00:00
Lorent Lempereur
839b841dda
feat(core): Stop a scheduled task run at its timeout (no-changelog) (#40515)
Co-authored-by: Veljko Terzic <veljko.terzic@n8n.io>
2026-10-12 10:46:11 +00:00
Lorent Lempereur
53a6019ad5
feat(core): Store and provision the timeout of scheduled jobs (no-changelog) (#40513) 2026-10-12 10:46:10 +00:00
Lorent Lempereur
2a1fde3634
feat(core): Add settings for the timeout of scheduled tasks (no-changelog) (#40512) 2026-10-12 10:46:09 +00:00
Mutasem Aldmour
37799ad2f2
fix(editor): Align nodes panel to the embedded canvas in the n8n Assistant split view (#40795)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:29:40 +00:00
n8n-cat-bot[bot]
a1800105ef
fix: Report resource-operation violations as errors (#40830)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-12 09:52:15 +00:00
Aryan Bahl
6b77613ee8
fix(Odoo Node): Normalize datetime fields to Odoo format before write (#34186)
Co-authored-by: Toni Devine <tonidevine24@gmail.com>
Co-authored-by: Toni Devine <118546181+devinetoni@users.noreply.github.com>
2026-10-12 09:46:07 +00:00
n8n-cat-bot[bot]
205c1c3ee5
refactor: Share appender for connection changes (#39472)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-12 09:11:50 +00:00
Andreas Fitzek
65cc931018
feat(core): Link and provision users from trusted-source identities (no-changelog) (#40755)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 09:01:17 +00:00
Andreas Fitzek
3dd3efc42f
feat(core): Resolve trusted-source subjects to users on the read path (no-changelog) (#40698)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 09:01:16 +00:00
Svetoslav Dekov
ca9b10054c
refactor(core): Route agent skill refs through one service (#40754)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:25:58 +00:00
Jaakko Husso
c9f54fab4d
feat(editor): Add composer and message extensions to the Agents chat panel (no-changelog) (#40613)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:17:36 +00:00
Jaakko Husso
82d21934d3
feat(editor): Add a host extension API to the Agents chat panel (no-changelog) (#40608)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:17:35 +00:00
Lorent Lempereur
1f27b98366
feat(core): Add a timeout column to the scheduler tables (no-changelog) (#40510) 2026-10-12 08:08:45 +00:00
Nikhil Kuriakose
30d883e866
chore: Assign the CLI skill commands to Context in OWNERS (no-changelog) (#40766)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-12 08:04:32 +00:00
Robin Braumann
3ec89b24dc
feat(core): Add first message time to agent telemetry (no-changelog) (#40857) 2026-10-12 07:29:04 +00:00
Luigi Gubello
2e049bb3de
chore: Remove xregexp from the FTP client (#40676)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: RomanDavydchuk <roman.davydchuk@n8n.io>
2026-10-12 07:21:06 +00:00
Robin Braumann
8297b2e6d2
fix(editor): Load node types before authenticated pages render (#40812) 2026-10-12 07:18:43 +00:00
Elias Meire
601304543d
fix(editor): Keep same-named parameter values apart when their types differ (#40170) 2026-10-12 07:05:03 +00:00
n8n-assistant[bot]
ceaa82cd4f
chore: Update e2e impact map (#40853)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-12 06:24:15 +00:00
n8n-assistant[bot]
444e4a9384
chore: Update node popularity data (#40849)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-12 00:49:02 +00:00
n8n-cat-bot[bot]
45938389b8
fix: Wait for MockServer HTTP status without following logs (#40846)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-11 22:09:03 +00:00
Jakub Kadlubiec
3301b2cc0f
chore: Move Assistant frontend and AI usage tests ownership to Assistant (no-changelog) (#40804)
Some checks failed
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Util: Update Node Popularity / update-popularity (push) Has been cancelled
Util: Update Credential Setupability / update-setupability (push) Has been cancelled
Util: Update Node Popularity / approve-and-automerge (push) Has been cancelled
Util: Update Credential Setupability / approve-and-automerge (push) Has been cancelled
2026-10-11 11:50:01 +00:00
n8n-assistant[bot]
529b0d1c3b
chore: Update e2e impact map (#40828)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-11 06:14:03 +00:00
Bernhard Wittmann
7424f0ca04
feat(editor): Add an upload step to the Microsoft Teams setup (no-changelog) (#40339) 2026-10-09 22:13:36 +00:00
Alex Antoniuk
a33ecf7481
refactor(editor): Move log streaming route and modal to a module descriptor (no-changelog) (#40426) 2026-10-09 20:23:49 +00:00
Alex Antoniuk
3085f45f71
feat(editor): Add placeholder page for inactive frontend modules (no-changelog) (#40553) 2026-10-09 20:23:48 +00:00
José Braulio González Valido
ed4fbf4ed3
fix(ai-builder): Make eval file fixtures match the scenario (no-changelog) (#40725)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 18:55:59 +00:00
Sam Wooler
feed158ba2
feat(API): Support binary response bodies on Public API controller routes (no-changelog) (#40704)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:46:14 +00:00
Jaakko Husso
9ff8d4fc73
fix(core): Handle each Chat Hub segment once in queue mode (#40791)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 17:07:01 +00:00
José Braulio González Valido
5bc6021477
fix(ai-builder): Run agents that a workflow calls with mocked tools in eval runs (no-changelog) (#40584)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 15:56:40 +00:00
Toni Devine
50c2ca0a27
feat(core): Handle inbound media handling for whatsapp (#39409)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:24:15 +00:00
Jaakko Husso
d6370a0002
fix(editor): Show one attach button in the agent chat composer (no-changelog) (#40810)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:09:42 +00:00
Jaakko Husso
44a6370aba
feat(core): Send client context with Agents chat messages (no-changelog) (#40612)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:09:17 +00:00
Robin Braumann
ef000908e7
docs: Link Google Vertex AI credential documentation (#40729) 2026-10-09 15:01:53 +00:00
Ali Elkhateeb
cdf3787834
feat(API): Add an idempotency key table for the Public API (no-changelog) (#40381)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-09 14:52:36 +00:00
Anne Aguirre
5e01906a1e
feat(core): Post a budget-stop message on external agent channels (#40767) 2026-10-09 14:41:51 +00:00
Mike Repeć
f05b29abeb
test(core): Skip radix toString parity under quickjs in native evaluation fuzz (no-changelog) (#40800) 2026-10-09 14:33:43 +00:00
Guillaume Jacquart
b33b0db040
feat(core): Add an extends selector to credential type policies (no-changelog) (#40305)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 14:33:37 +00:00
Guillaume Jacquart
d1422fb652
refactor(core): Split type availability policy DTOs by policy kind (no-changelog) (#40602)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 14:33:29 +00:00
Rúben Castro
87c9b1c52a
fix(core): Move the policy scope version by one on each accepted write (no-changelog) (#40802)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-09 14:31:18 +00:00
Mutasem Aldmour
be402be262
fix(editor): Keep the prompt visible while the assistant starts a new thread (#40796)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 14:29:57 +00:00
Robin Braumann
cdc23c666f
feat(editor): Add project selection to Vertex AI credentials (#40667) 2026-10-09 14:23:57 +00:00
Iván Ovejero
bbd35e281b
fix(core): Keep waiting executions unchanged when a resume request fails (#40683)
Co-authored-by: Danny Martini <danny@n8n.io>
2026-10-09 14:23:36 +00:00
Mutasem Aldmour
55eb2ce442
fix(editor): Keep the workflow setup step counter on one line (#40771)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 14:12:42 +00:00
Mutasem Aldmour
4032797790
fix(editor): Show the renamed agent name in the assistant artifact tab (#40770)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 14:12:33 +00:00
ocdejong
c7135451eb
feat(core): Emit opt-in start marker spans for workflow and node executions (#40762)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 14:09:39 +00:00
Charlie Kolb
88a2531558
refactor(core): Move data-table dependency services to shared package (#40526) 2026-10-09 13:54:56 +00:00
Theodore Nastos
75de193d90
feat(editor): Explain a restricted model provider with the restriction popover (#40705) 2026-10-09 13:52:20 +00:00
Jaakko Husso
ee8a608a22
fix(core): Let the streaming heartbeat stop on Chat Hub responses (#40792)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:43:59 +00:00
Jaakko Husso
55def8cc2b
feat(core): Add atomic thread metadata patches to Agents memory (no-changelog) (#40610)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:41:05 +00:00
Alex Grozav
85f690d3b9
feat(editor): Add filters and sorting to migration rule details (no-changelog) (#40781)
Co-authored-by: Michael Siega <michael.siega@n8n.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 13:36:01 +00:00
Yen Su
c2e625bbb0
fix(Databricks Node): Reset dependent Lakebase locators and pin the picker filters (no-changelog) (#40787) 2026-10-09 13:21:15 +00:00
Thanasis G
344dea5c77
feat(core): Report the deployment artifact in telemetry (#40319)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:14:32 +00:00
Theodore Nastos
b28dceb462
fix(editor): Place the restricted node popover per host, keep one open, and stack the admin dialog (#40678) 2026-10-09 13:12:49 +00:00
Robin Braumann
a7cba90a85
feat: Add Google Vertex AI credentials (#40665) 2026-10-09 13:06:56 +00:00
Andreas Fitzek
0d7df52a45
feat(core): Translate trusted-source claims into an external identity (no-changelog) (#40681)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 13:06:31 +00:00
Robin Braumann
55200605dd
docs: Keep routine pre-commit checks lightweight (#40790) 2026-10-09 13:05:11 +00:00
Theodore Nastos
cde7a9dbd9
feat(editor): Disable restricted model providers in the agent model selector (#40658) 2026-10-09 12:59:23 +00:00
Alex Grozav
ff7ef60841
feat(editor): Redesign the migration rule detail list (no-changelog) (#40654)
Co-authored-by: Michael Siega <michael.siega@n8n.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 12:50:38 +00:00
Benjamin Schroth
c92141d141
fix(editor): Fix header on n8n Chat page (no-changelog) (#40737) 2026-10-09 12:47:20 +00:00
Benjamin Schroth
9e1866736c
feat(editor): Always show n8n Chat in agent channels (no-changelog) (#40784) 2026-10-09 12:42:22 +00:00
ocdejong
22ba4e7bc5
feat(core): Add execution identity to every OTel span (#40649)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:41:36 +00:00
Yuliia Pominchuk
033a9b512c
fix(core): Allow reverting a workflow version that uses credentials the user cannot use (#40701) 2026-10-09 12:37:11 +00:00
Anne Aguirre
e2b4fee0ec
fix(editor): Keep Agent Builder settings inside a narrow column (#40759) 2026-10-09 12:28:20 +00:00
mfsiega
35078f1d04
feat(core): Let admins assign the owner of a workflow's migration findings (no-changelog) (#40452)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 12:15:27 +00:00
Mike Repeć
a2ac242b91
feat(core): Accept array literals and single-parameter callbacks in native expression evaluation (no-changelog) (#40292) 2026-10-09 12:08:12 +00:00
Robin Braumann
e64dc46055
feat(API): Export Agents in n8n packages (no-changelog) (#40428) 2026-10-09 12:05:33 +00:00
Yen Su
fd06e07efa
feat(Databricks Node): Add the Lakebase pickers and endpoint URL assembly (no-changelog) (#40407) 2026-10-09 11:47:21 +00:00
Robin Braumann
21af799450
feat(core): Add source breakdowns to agent telemetry (no-changelog) (#40726) 2026-10-09 11:42:26 +00:00
mfsiega
fd04f01e8e
feat(core): Show the suggested owner of each affected workflow (no-changelog) (#40394)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 11:04:19 +00:00
n8n-cat-bot[bot]
af98327a5e
fix: Keep execution details clear of preview canvas (#40176)
Some checks failed
Build: Benchmark Image / build (push) Has been cancelled
CI: Master (Build, Test, Lint) / Build & format (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Lint (push) Has been cancelled
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Has been cancelled
Util: Publish API Schema / Cloudflare Pages (push) Has been cancelled
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Has been cancelled
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-09 10:46:58 +00:00
Raúl Gómez Morales
c21ed4f13a
fix(Google Gemini Chat Model Node): Point the AI builder to the model catalog for new Gemini models (#40159)
Co-authored-by: Mutasem Aldmour <mutasem@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 10:43:38 +00:00
Anne Aguirre
8b7065a876
fix(core): Block the assistant when AI credits hit zero (#40659) 2026-10-09 10:39:03 +00:00
Garrit Franke
8f9f673b86
feat(core): Support minor levels in n8nNodesApiVersion (no-changelog) (#39308)
Co-authored-by: Benny Neugebauer <mail@bennycode.com>
2026-10-09 10:35:40 +00:00
Irénée
a3391ba93c
fix(core): Check the credential save policy before package import creates stubs (#40687)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 10:30:47 +00:00
James Martin
56e2802e67
chore(core): Moves the grant authorization method into the TrustedSourceGate (no-changelog) (#40571) 2026-10-09 10:27:04 +00:00
Robin Braumann
aee0b23b8e
feat(core): Add Agent package selection and dependency resolution (no-changelog) (#40418) 2026-10-09 10:22:12 +00:00
mfsiega
d3173d3893
fix(core): Remove the Execute Sub-workflow each-mode breaking-change rule and migration (no-changelog) (#40753)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 10:17:18 +00:00
MHMD
19337237ec
fix(editor): Show the unavailable credential at the top of the node picker (#40693) 2026-10-09 10:01:39 +00:00
Raúl Gómez Morales
d900743bce
fix(core): Improve node placement when n8n Assistant edits a workflow (#40139)
Co-authored-by: Mutasem Aldmour <mutasem@n8n.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:56:40 +00:00
Irénée
978f810bbe
fix(core): Remove credentials of deleted projects on source control pull (#40692)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 09:29:07 +00:00
Rodrigo Santos da Silva
ae15984493
feat(editor): Suggest and apply fixes for failed agent eval checks (no-changelog) (#40711)
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-09 09:27:15 +00:00
Mutasem Aldmour
9f85b8379c
fix(editor): Remove trailing newline from AI Assistant code blocks (no-changelog) (#40731)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-09 09:24:16 +00:00
Mike Repeć
24b9db41b3
feat(core): Declare what on main awaits a queued run's outcome (no-changelog) (#40567) 2026-10-09 09:05:07 +00:00
Robin Braumann
503092ccca
feat(core): Add the Agent package export adapter (no-changelog) (#40402) 2026-10-09 08:59:58 +00:00
Robin Braumann
16a8db6675
feat(core): Define and validate the Agent package format (no-changelog) (#40391) 2026-10-09 08:59:57 +00:00
Jaakko Husso
bc0a9cbc11
fix(core): Report approval tools with no tool connected in workflow validation (#40561)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 08:56:43 +00:00
mfsiega
c6f1e20d9e
feat(engine): Accept seeded steps on execution start (#40456)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 08:56:37 +00:00
Sandra Zollner
a249e20290
feat(core): Promote folder and project renames alongside selected workflows (#40502)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-09 08:51:21 +00:00
Raúl Gómez Morales
1c84af2e96
fix(core): Resume tool calls for deferred tools that a skill loaded (#40126)
Co-authored-by: Mutasem Aldmour <mutasem@n8n.io>
2026-10-09 08:49:16 +00:00
Nikhil Kuriakose
aca9033dca
chore: Split Adore OWNERS entries into Context and Knowledge (no-changelog) (#40355)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-09 08:28:57 +00:00
mfsiega
2efdd3e021
feat(core): Add the migration owner suggestion service (no-changelog) (#40224)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 08:28:25 +00:00
Theodore Nastos
01da88958d
feat(editor): Show the restriction callout and lock the agent tool configuration form (#40657) 2026-10-09 08:09:15 +00:00
James Martin
6f5f79321b
chore(core): Adds LocalAuthorizationServer implementation for token authentication (no-changelog) (#40702) 2026-10-09 07:56:57 +00:00
Pierantonio Cangianiello
690fbb525a
test(core): Prove binary data on engine v2 across the planes in the container parity suite (no-changelog) (#40417) 2026-10-09 07:53:35 +00:00
José Braulio González Valido
5480258fb5
feat(ai-builder): Report the eval harness's own model usage per case (no-changelog) (#40718)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 07:52:09 +00:00
José Braulio González Valido
bfebfd26c1
feat(ai-builder): Move eval mocks to Sonnet 5.5 (no-changelog) (#40721)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 07:50:47 +00:00
Robin Braumann
806a2aa94b
refactor(core): Use typed package requirement consumers (#40621) 2026-10-09 07:42:23 +00:00
Dmitrii
13679fcb97
feat(core): Report insights billable executions as the daily instance metric (no-changelog) (#40536) 2026-10-09 06:57:43 +00:00
Svetoslav Dekov
75a9cc7e12
feat(core): Search, create and reopen agents from the command bar (#40504)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 06:29:53 +00:00
n8n-assistant[bot]
03248c11bf
chore: Update e2e impact map (#40733)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-09 06:25:02 +00:00
n8n-cat-bot[bot]
23b6de7028
test: Assign Chat Hub E2E specs to Assistant (#40732)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-09 06:21:50 +00:00
Predrag Ristic
53d33b0f95
fix(core): Support streamed reasoning from custom AI endpoints (no-changelog) (#37875) 2026-10-09 05:45:07 +00:00
Jakub Kadlubiec
c4f84e2f78
test: Remove flaky Chat Hub conversation flow e2e test (no-changelog) (#40728)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 05:36:24 +00:00
Jan
3ba522910b
fix(editor): Use pulsing status dot for the enabled n8n Assistant setting (#40712)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-09 05:35:12 +00:00
José Braulio González Valido
6fc0aeda35
fix(ai-builder): Read eval pin data when the model explains it after the JSON (no-changelog) (#40682)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:57:15 +00:00
José Braulio González Valido
5b78e8b655
fix(ai-builder): Run each eval build in its own empty project (no-changelog) (#40715)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 21:15:11 +00:00
Alex Antoniuk
cef87cfcab
feat(editor): Add order to settings sidebar links (#40505)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 20:48:14 +00:00
Charlie Kolb
a2c8142013
ci: Remove the /test-workflows PR comment trigger (no-changelog) (#40688)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 17:54:09 +00:00
Luigi Gubello
cabf0bf3b4
feat(core): Serve the instance CSP on all non-script responses (#40205)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Danny Martini <danny@n8n.io>
2026-10-08 17:33:44 +00:00
n8n-cat-bot[bot]
84d2fd6f3a
fix: Align execution retry menu and clarify actions (#40710)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-08 17:16:52 +00:00
Sam Wooler
ce1be847c8
refactor(API): Migrate n8n-packages import-selection endpoint to PublicApiController (no-changelog) (#40554)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 16:12:09 +00:00
Anne Aguirre
fe26ba8d3a
feat(core): Experiment to show assistant composer and starter templates in agent list (#40396)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-08 15:50:16 +00:00
n8n-cat-bot[bot]
03f7c76acd
fix: Update runtime and MCP dependency APIs (no-changelog) (#40204)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Michael Drury <michael.drury@n8n.io>
2026-10-08 15:43:51 +00:00
Raúl Gómez Morales
e67e5a9ef1
test(core): Run routing cases that start with state (no-changelog) (#40378) 2026-10-08 15:33:58 +00:00
Benjamin Schroth
3a37ec1dcc
feat: Show when an n8n Chat agent is no longer available (no-changelog) (#40430) 2026-10-08 15:25:09 +00:00
Uddish Verma
77a692a4bb
fix(API): Give a union of object schemas a root type: object in the generated spec (no-changelog) (#40409)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:24:14 +00:00
mfsiega
6a41a5ed39
feat(core): Send telemetry with the migration report overview counts (no-changelog) (#40387)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 15:24:12 +00:00
mfsiega
ea8417e29a
feat(core): Block create/update of workflows with deprecated nodes (#31360)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Danny Martini <danny@n8n.io>
Co-authored-by: Emilia <100027345+sovietspaceship@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-08 14:39:14 +00:00
Benjamin Schroth
5543023723
feat: Show the attach button and sub-agent names in n8n Chat (no-changelog) (#40429) 2026-10-08 14:38:28 +00:00
Arvin A
05a4abd700
fix(core): Label instance activity entry IDs (no-changelog) (#40691) 2026-10-08 14:32:43 +00:00
Benjamin Schroth
740159411a
feat: Search and delete n8n Chat agent threads (no-changelog) (#40431) 2026-10-08 14:29:26 +00:00
Danny Martini
346cbaa316
docs: Add a README for the ADR directories (no-changelog) (#39006)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 14:28:52 +00:00
Lorent Lempereur
a83c438af4
refactor(core): Keep metrics database reads behind a guarded query service (#40221) 2026-10-08 14:20:12 +00:00
Mike Repeć
49a3812bc3
feat(core): Extend native expression evaluation to node references and more data roots (no-changelog) (#40239) 2026-10-08 14:11:19 +00:00
Jan Kalkan
9946cfe901
feat(core): Allow confirming destructive data table changes when continuing an apply (#40491)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:06:33 +00:00
Yen Su
4c0902f6bf
fix(Databricks Node): Request float encoding for embeddings from the Unity AI Gateway (#40684) 2026-10-08 13:51:27 +00:00
Arvin A
d7e9b00234
feat(core): Index activity entries by resource (no-changelog) (#38040) 2026-10-08 13:50:52 +00:00
MHMD
63dc285493
fix(editor): Show the owner and credential type for credentials the user cannot use (#40669) 2026-10-08 13:49:45 +00:00
Sam Wooler
5a46327401
refactor(API): Migrate n8n-packages import endpoint to PublicApiController (no-changelog) (#40216)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 13:38:52 +00:00
Jaakko Husso
1ad3c54803
feat(editor): Recommend Claude Opus 5.5 and GPT-6 models in AI Assistant setup (#40660)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:23:49 +00:00
Jan Kalkan
5d208c85ed
feat(core): Report destructive data table changes in the promotion apply preflight (#40454)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:10:08 +00:00
Anne Aguirre
209adf9a7e
feat(core): Show the agent chat attach button on the left of the composer (#40320)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 13:07:33 +00:00
Rob Hough
04e1e7fc47
feat(editor): Animate the n8n Assistant activity indicator (#40497) 2026-10-08 13:05:29 +00:00
Riqwan Thamir
2aa3fbf185
perf(core): Send agent builder config writes as objects with changed fields only (#40666) 2026-10-08 12:55:54 +00:00
Uddish Verma
983f5ecea8
refactor(API): Rename the log streaming destination path parameter to destinationId (no-changelog) (#40533)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:42:58 +00:00
Lorent Lempereur
4e217aeac6
feat(core): Serve Prometheus metrics while the database is unavailable (#39908) 2026-10-08 12:41:12 +00:00
Lorent Lempereur
d904e7a247
feat(core): Add Prometheus metrics for database connection recovery (#39890) 2026-10-08 12:41:11 +00:00
Michael Drury
f7e52a851e
fix(editor): Add resend to retryable Agent preview errors (#40592) 2026-10-08 12:37:33 +00:00
Michael Drury
1e6da4e615
fix: Separate agent response steps and show saved reasoning (#40590) 2026-10-08 12:33:01 +00:00
Michael Drury
40c0525bed
fix(core): Use the native Vercel gateway endpoint for agents (no-changelog) (#40524) 2026-10-08 12:32:53 +00:00
Srinjoy
0f02b509be
fix(editor): Preserve Live Data highlights on scroll in expression editor (#33842)
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: alex <alex@n8n.io>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 12:28:21 +00:00
n8n-cat-bot[bot]
8ad9b5415e
docs: Document workflow activation with activeVersionId (#40500)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Sam Wooler <swooler592@gmail.com>
2026-10-08 12:16:52 +00:00
Alex Grozav
e11fe8afb0
feat(editor): Show open and won't fix counts on migration report cards (no-changelog) (#40652)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 12:15:04 +00:00
Dimitri Lavrenük
fdc394819c
fix: Prevent node types corruption when multiple loaders run at the same time (#40633) 2026-10-08 11:45:37 +00:00
Robin Braumann
74a8a37134
refactor(core): Consolidate agent delegation tools (no-changelog) (#40650) 2026-10-08 11:45:34 +00:00
Rodrigo Santos da Silva
86877c557a
feat(editor): Preview agent output and review agent checks in the Evals tab (no-changelog) (#39886)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-08 11:28:45 +00:00
Declan Carroll
b464dea23c
build: Move the n8n-pc image onto n8nio/node-pc:26.10.0 (#40642) 2026-10-08 11:18:06 +00:00
Robin Braumann
d008d6cee9
fix(core): Validate canonical package manifest paths (no-changelog) (#40619) 2026-10-08 11:03:31 +00:00
Anne Aguirre
96a8ff4d15
fix(core): Wire Entra OAuth2 into agent Azure OpenAI runtime (#40218)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-08 10:59:14 +00:00
Toni Devine
a195096733
fix(editor): Show why a node on another branch can't be referenced (#40565) 2026-10-08 10:53:30 +00:00
Irénée
aef216cb25
feat(editor): Apply a selection of workflows at the destination (no-changelog) (#39788)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-08 10:41:39 +00:00
Rúben Castro
370b30f919
feat(core): Hide credential-only nodes under a credential type policy (#39652)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 10:13:44 +00:00
Alexander Gekov
97c54f618c
feat(Google Gemini Node): Show Nano Banana 2.1 and 2 Lite in the image model list (#40596)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-08 09:59:08 +00:00
MHMD
c133d6c893
fix(editor): Shorten the share button label on the credential sharing tab (#40637) 2026-10-08 09:47:49 +00:00
Csaba Tuncsik
a584136b6f
test(Databricks Node): Cover the Vector Store Retrieve search mode and filter (no-changelog) (#39528) 2026-10-08 09:34:26 +00:00
Ilfat Mindubaev
986f6b49e3
feat(Databricks Node): Add the hidden Databricks AI Agent with MLflow trace export (no-changelog) (#39465)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 09:29:23 +00:00
Jaakko Husso
1d63a8f3a1
chore(core): Log npm output when the sandbox workspace link install fails (#40591)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 09:26:41 +00:00
n8n-cat-bot[bot]
f212d5b900
fix: Validate and default public API pagination limits (#40483)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Sam Wooler <swooler592@gmail.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 09:02:02 +00:00
mfsiega
f4f7dcf13b
feat(core): Show every user the migration report, scoped to the workflows they can edit (#40449)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-08 08:46:46 +00:00
Eugene
6cdd1ffbbb
test(editor): Stabilize chat session reset assertions (#40587) 2026-10-08 08:43:10 +00:00
Stephen Wright
81f34c2374
fix(S3 Node): Percent-encode the copy source path (#40282) 2026-10-08 08:41:37 +00:00
Predrag Ristic
168a0ac860
feat(core): Persist self-healing results and handoffs (#40115) 2026-10-08 08:11:11 +00:00
Benjamin Schroth
727f2a74ea
refactor(core): Keep each chat channel's setup in one place in agent execution (no-changelog) (#40605) 2026-10-08 08:02:48 +00:00
Robert Squires
3b0337d894
feat(editor): Refine the Assistant preferences card and plus menu (no-changelog) (#40558) 2026-10-08 07:59:07 +00:00
Toni Devine
3c33856997
feat(core): Handle message quality rate limits for whatsapp integration (#39205)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:57:33 +00:00
Raúl Gómez Morales
6b8ac76d96
test(core): Add a routing mode to the Assistant discovery evals (no-changelog) (#40288) 2026-10-08 07:55:26 +00:00
Declan Carroll
d138b8ff4b
build: Move the n8n and runners images onto n8nio/base:26.10.0 (#40632) 2026-10-08 07:22:42 +00:00
n8n-assistant[bot]
566a1dafc6
chore: NODE-6095 - Generate Schema Files (no-changelog) (#40026)
Co-authored-by: Jon <jonathan.bennetts@gmail.com>
2026-10-08 07:20:26 +00:00
Bernhard Wittmann
6bc5c93513
fix(Microsoft Outlook Trigger Node): Report failed polls and limit the backlog per poll (#40147) 2026-10-08 07:20:03 +00:00
Nour Alhadi Mahmoud
b28adf9789
fix: Load only the publish history that the editor shows (#40183)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 07:14:26 +00:00
Rodrigo Santos da Silva
7397073d86
refactor: Use AsyncLocalStorageContextManager in OTel test harness (no-changelog) (#40173)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-08 07:11:25 +00:00
Benjamin Schroth
1ea8397d69
feat: Steer and reorder queued messages in n8n Chat (no-changelog) (#40433) 2026-10-08 07:07:46 +00:00
n8n-assistant[bot]
86847d7f16
chore: Update e2e impact map (#40627)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-08 06:31:04 +00:00
Veljko Terzic
33cd4d5cc1
feat(core): Run the license renewal as a durable system task (no-changelog) (#39055) 2026-10-08 06:06:38 +00:00
Bernhard Wittmann
60ad29f242
feat(core): Let the Microsoft Teams agent read every message in channels and group chats (no-changelog) (#40334) 2026-10-07 20:50:50 +00:00
Guillaume Jacquart
9191cf3fb0
fix(core): Apply the credential type policy to credential tests and probes (#39927)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 19:16:11 +00:00
Guillaume Jacquart
668e71d54a
feat(core): Report crashed executions from startup log recovery (#40390)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 18:02:40 +00:00
Luigi Gubello
1e919c7c56
chore: Update npm dependencies (#40597)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 17:12:50 +00:00
Uddish Verma
9d3eb0c637
refactor(API): Migrate the log streaming destination test and delete to the decorator pattern (no-changelog) (#40532)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 16:43:03 +00:00
Nour Alhadi Mahmoud
bef5f3f97a
fix(core): Expose 1Password fields for secret preview and autocomplete (#40538) 2026-10-07 16:23:24 +00:00
Eugene
c194d22b5c
test(editor): Isolate the legacy workflow builder smoke test (#40576) 2026-10-07 16:04:19 +00:00
n8n-cat-bot[bot]
df8605e875
feat: Make health check timeout configurable (#39548)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: James Gee <1285296+geemanjs@users.noreply.github.com>
2026-10-07 15:37:57 +00:00
Anne Aguirre
fbbbadc292
fix(core): Report MCP server connection failures in verify_mcp_server (#40559) 2026-10-07 15:36:06 +00:00
n8n-cat-bot[bot]
4b5b68ee29
fix: Update matching nodes when selecting imported credentials (#39650)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-07 15:33:28 +00:00
Jan
09436b78f9
feat(editor): Rebuild the personal settings page with the design-system settings components (#40298)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-07 15:25:38 +00:00
Anne Aguirre
19e1f70ea6
feat(core): Email the agent owner when a monthly budget alert is crossed (#40544) 2026-10-07 14:53:27 +00:00
Anne Aguirre
0722ddbbd5
docs(core): Document the built-in tools the agent runtime attaches (#40062) 2026-10-07 14:49:18 +00:00
ocdejong
88b22abf15
feat(core): Add a restricted node filter to the workflow list (no-changelog) (#39876)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:48:35 +00:00
Svetoslav Dekov
e36093b11d
feat(editor): Rebuild the command bar with type tabs and paged search (#40461)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:40:54 +00:00
n8n-cat-bot[bot]
2f4a3868fb
fix: Remove loading flashes from data tables (#39874)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Arvin Ansari <arvin.ansari@n8n.io>
2026-10-07 14:31:09 +00:00
Riqwan Thamir
3816c4afec
feat(ai-builder): Honor reasoning effort override for Anthropic models in evals (no-changelog) (#39809)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-07 14:18:33 +00:00
Yuliia Pominchuk
48f05f92fe
feat(editor): Explain instance-role access in the credential sharing "Used in" row (#40555)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 14:16:00 +00:00
Benjamin Schroth
e2733c5b86
feat: Run background tasks in n8n Chat (no-changelog) (#40434) 2026-10-07 14:09:59 +00:00
Alex Grozav
682d0920ba
feat(editor): Add a state column to the migration rule detail (#40388)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 14:05:20 +00:00
Nour Alhadi Mahmoud
2863939580
feat(core): Allow GitLab promotion providers in the database (no-changelog) (#40412) 2026-10-07 14:00:36 +00:00
Jaakko Husso
35972133b0
fix(core): Require execution delete scope for assistant execution cleanup (no-changelog) (#40548)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:00:31 +00:00
Theodore Nastos
0bbea438e1
fix(core): Show the policy block before node issues on execute and publish (#40290) 2026-10-07 13:30:44 +00:00
Riqwan Thamir
6b4da93dcb
perf(core): Let the agent builder chain config writes without re-reading the config (#40462) 2026-10-07 13:30:10 +00:00
Riqwan Thamir
e16566b313
fix(core): Start a new test session when call_agent gets an unknown session ID (#40458) 2026-10-07 13:29:45 +00:00
Riqwan Thamir
1de45a1a9b
perf(core): Let the agent builder edit skill instructions in place (#40447) 2026-10-07 13:29:29 +00:00
Jaakko Husso
1ef77564e3
fix(core): Fix Kimi K3 and other OpenAI-compatible Assistant model providers on self-hosted Assistant (#40439)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:26:58 +00:00
n8n-cat-bot[bot]
4c8b76b6f0
fix: Use configured OAuth server rate limits (#40542)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-07 13:24:56 +00:00
CodeShakingSheep
00849c4dea
fix(Invoice Ninja Node): Fix retrieving currencies for bank transactions (#12997)
Co-authored-by: Jonathan <jon@n8n.io>
Co-authored-by: Jon <jonathan.bennetts@gmail.com>
2026-10-07 13:23:37 +00:00
Pierantonio Cangianiello
a440967b12
feat(core): Set up binary data storage on the engine v2 data plane host (no-changelog) (#40341) 2026-10-07 13:19:22 +00:00
Anne Aguirre
1c63119756
fix(editor): Show the Agent artifact while the builder creates it (#40535) 2026-10-07 13:12:15 +00:00
Michael Drury
022b23945b
fix(editor): Respect the default voice for chat read aloud (#40440) 2026-10-07 12:55:44 +00:00
MHMD
f31e1a83eb
fix(editor): Add a Remove access tooltip to the remove button in the sharing modal (no-changelog) (#40494) 2026-10-07 12:45:51 +00:00
MHMD
eac5d1d7b3
fix(editor): Rename the Only you label to Available to you in the credential sharing modal (no-changelog) (#40411) 2026-10-07 12:39:05 +00:00
Predrag Ristic
eaf8c25d1c
feat(core): Add workflow suggestion review actions (#39858) 2026-10-07 12:36:55 +00:00
Uddish Verma
741ba22943
refactor(API): Migrate the log streaming destination writes to the decorator pattern (no-changelog) (#40397)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:34:47 +00:00
Guillaume Jacquart
e7d8533880
fix(core): Apply node type policies to agent node tools (#39711)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 12:32:29 +00:00
mfsiega
bb76c3c99b
feat(core): Add the migration workflow owner table (no-changelog) (#40223)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 11:36:58 +00:00
Emilia
f26747bfc6
fix(core): Return jobs fetched before a worker paused to the queue (#40162)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:32:56 +00:00
Emilia
084f3b376a
fix(core): Return jobs dequeued during worker shutdown to the queue (#40086)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:32:55 +00:00
Emilia
dc5c80f26a
fix(core): Count jobs as running from the start of job processing (#40050)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:32:55 +00:00
Iván Ovejero
5ab2187416
fix(core): Pass storedAt into WorkflowExecute at every callsite (#40495) 2026-10-07 11:21:18 +00:00
Jan Kalkan
dc33c29d1a
feat(core): Add overwrite-non-destructive schema conflict policy to data table package import (#40446)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:04:50 +00:00
MHMD
4975ab82fe
fix(editor): Remove the disabled cursor on the group headings in the credential dropdown (no-changelog) (#40499) 2026-10-07 10:51:01 +00:00
n8n-cat-bot[bot]
447e939b73
fix: Allow per-process crash journal paths (#37670)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-07 10:47:34 +00:00
Andreas Fitzek
4e802ff11d
feat(core): Verify bearer tokens against trusted sources (no-changelog) (#40279)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-07 10:30:24 +00:00
n8n-cat-bot[bot]
0885af531a
chore: Promotions DB snapshot comparison is order-sensitive on role scopes, failing CLI integration + Postgres unit-coverage job (#40475)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-07 10:21:15 +00:00
n8n-cat-bot[bot]
b1ec8e525c
fix: Ignore generated Vitest reports (no-changelog) (#40498)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-07 10:20:54 +00:00
Sandra Zollner
a694531a03
fix(editor): Align tooltip trigger box with its content (#40422)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-07 09:53:07 +00:00
MHMD
684b181101
fix(editor): Show credentials the user cannot use in the workflow dependency list (no-changelog) (#40481)
Some checks failed
Build: Benchmark Image / build (push) Has been cancelled
CI: Master (Build, Test, Lint) / Build & format (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Has been cancelled
CI: Master (Build, Test, Lint) / Lint (push) Has been cancelled
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Has been cancelled
Util: Publish API Schema / Cloudflare Pages (push) Has been cancelled
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Has been cancelled
2026-10-07 09:31:24 +00:00
Ilfat Mindubaev
df4af50125
feat(Databricks Node): Add the LangChain to MLflow span converter (no-changelog) (#39396)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 09:29:36 +00:00
Yuliia Pominchuk
02af7e7659
feat(editor): Sort the credential sharing list by people, then projects (#40478)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 09:22:00 +00:00
Veljko Terzic
b7d7a18980
feat(core): Run the agent interrupted-execution sweep as a durable system task (no-changelog) (#40424) 2026-10-07 08:57:44 +00:00
Charlie Kolb
16dc6fe403
refactor(core): Enforce TypeORM persistence boundary with Code Health (#40384) 2026-10-07 08:39:18 +00:00
mfsiega
779ddcc517
test(engine): Run the EmailSend SMTP spec on engine v2 (#40455)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-07 08:33:56 +00:00
Daria Staferova
e1025f35e4
fix(editor): Keep folder creation available globally after Community registration (#40082) 2026-10-07 08:26:02 +00:00
Veljko Terzic
3014c64264
feat(core): Run the external secrets refresh as an instance system task (no-changelog) (#40404) 2026-10-07 08:25:02 +00:00
Charlie Kolb
4ae863717e
feat(core): Add backend module scaffolding (no-changelog) (#40482) 2026-10-07 08:22:23 +00:00
Yuliia Pominchuk
3cff8dc4ec
feat(core): Apply the member credential rule to Owners and admins in team projects (#40448) 2026-10-07 07:52:27 +00:00
Emilia
81cdc278c2
ci: Add jscpd duplication-scan workflow (no-changelog) (#40331)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-07 07:37:01 +00:00
Emilia
e500386a54
build: Add jscpd config for local and CI duplication scans (#39198)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-07 07:37:00 +00:00
James Martin
e882e8a483
chore(core): Begins minting asymmetric signed access tokens (#40286)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 07:23:43 +00:00
Yuliia Pominchuk
66b382de52
fix(editor): Keep the selected credential visible when the picker does not list it (#40450) 2026-10-07 07:20:15 +00:00
Jan Kalkan
3af98170d0
feat(core): Add overwrite schema conflict policy to data table package import (#40345)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 07:19:44 +00:00
Charlie Kolb
6ef70a7303
refactor(core): Move Insights tests into module (#40423) 2026-10-07 07:17:47 +00:00
MHMD
32d472fe76
fix(editor): Name the target on the project Share button in the credential sharing modal (no-changelog) (#40413) 2026-10-07 07:14:49 +00:00
n8n-assistant[bot]
6d1144cea3
chore: Update e2e impact map (#40473)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-07 06:24:33 +00:00
José Braulio González Valido
0e1c754999
fix(ai-builder): Take feature flags from env only on Instance AI eval instances (no-changelog) (#40117)
Some checks are pending
CI: Master (Build, Test, Lint) / Build & format (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (26.5.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Verify single-instance deps (npm-install) (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 20:46:09 +00:00
Riqwan Thamir
90b8205b62
feat(core): Load Instance AI skill references progressively (no-changelog) (#40296)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-06 20:03:33 +00:00
Rob Hough
883ade4b9d
feat(editor): Add agent setup checklist (#39919) 2026-10-06 16:30:46 +00:00
Riqwan Thamir
b36c2b506b
perf(core): Prepare Instance AI skill sandbox in the background (#40363) 2026-10-06 16:17:43 +00:00
Riqwan Thamir
ee1db0bc12
perf(core): Keep agent builder system prompt static for prompt caching (#40364) 2026-10-06 16:17:22 +00:00
Toni Devine
b0d1a30f50
feat(core): Add the WhatsApp agent channel (#38750)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-06 16:02:03 +00:00
Theodore Nastos
77370af99a
fix(core): Check the target project's policy on folder, credential and delete-transfer moves (#40301) 2026-10-06 15:52:14 +00:00
Michael Drury
a13705aa8f
fix(editor): Keep Assistant fixes in Agent builder and correct tooltips (#40348) 2026-10-06 15:44:49 +00:00
Tuukka Kantola
76cab1e26d
fix(editor): Polish n8n Assistant and Execute button UI details (#40414)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:31:43 +00:00
n8n-cat-bot[bot]
8059b63da8
fix: Return UTC ISO dates without changing other cells (#40156)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-06 14:56:43 +00:00
n8n-cat-bot[bot]
e7f8fa38cf
chore: Add .build/ to root .gitignore (#39647)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-06 14:41:57 +00:00
MHMD
f72461d56d
fix(editor): Block workflow runs and disable run buttons for credentials the user cannot use (no-changelog) (#40410) 2026-10-06 14:34:13 +00:00
Alex Grozav
24dda0d435
refactor(editor): Extract MCP access into a frontend module package (no-changelog) (#39493)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-10-06 14:31:21 +00:00
Nikhil Kuriakose
8924a11fda
chore: Reassign shared OWNERS paths away from Adore (no-changelog) (#40353)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-06 14:28:52 +00:00
Stephen Wright
0932e9572e
feat(Databricks Node): Guard the Vector Store insert against the wrong index (#39699) 2026-10-06 13:46:39 +00:00
Stephen Wright
830486a851
feat(Embeddings Azure OpenAI Node): Add Entra ID authentication (#39390)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-06 13:46:28 +00:00
Uddish Verma
15d597c4ca
refactor(API): Migrate the log streaming destination reads to the decorator pattern (no-changelog) (#40235)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 13:35:35 +00:00
Declan Carroll
2fe28ea0ee
test: Fix credential visibility timeouts on the coverage run (#40386) 2026-10-06 13:20:30 +00:00
Theodore Nastos
c2f6920794
feat(core): Add credentialTransfer policy enforcement point (#40300) 2026-10-06 13:15:19 +00:00
Mike Repeć
2b685c33b6
test(core): Run only expression tests once per engine in the workflow package (no-changelog) (#40242) 2026-10-06 13:07:49 +00:00
Charlie Kolb
62fd028089
refactor(core): Extract Insights backend module package (#40261) 2026-10-06 13:05:38 +00:00
Rob Hough
fb11cc482e
refactor(editor): Improve agent message queue and steering controls (#40287)
Co-authored-by: Bob Hough <15361982+heymynameisrob@users.noreply.github.com>
Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
2026-10-06 12:58:33 +00:00
Ricardo Espinoza
d5132d21e0
feat(core): Add SCIM provisioning module foundation (no-changelog) (#40226) 2026-10-06 12:57:50 +00:00
n8n-cat-bot[bot]
0bd0cd2873
chore: Move the deprecated GET /workflows/{id}/{versionId} alias to the decorator controller and delete the legacy workflows handler (#40340)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Sam Wooler <swooler592@gmail.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 12:26:56 +00:00
Thanasis G
e5c8a8bb62
feat(Azure OpenAI Chat Model Node): Rename the Foundry nodes and credentials to Microsoft Foundry (#40398)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-06 12:23:59 +00:00
Jakub Kadlubiec
611a6cccda
chore: Move AI Workflow Builder and AI service ownership to Assistant (no-changelog) (#40312)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 12:23:36 +00:00
Veljko Terzic
38eea65993
feat(core): Run the workflow history trim as a durable daily system task (#40085) 2026-10-06 12:17:32 +00:00
MHMD
c4112794fa
fix(core): Offer credentials shared directly with a user in that user's project workflows (no-changelog) (#40395) 2026-10-06 12:04:28 +00:00
Yen Su
3d8b3b99c6
feat(Databricks Node): Add the Lakebase Data API request helper (no-changelog) (#40401) 2026-10-06 12:03:46 +00:00
Veljko Terzic
fc1c26d867
feat(core): Run the data table upload cleanup as a per-instance system task (no-changelog) (#40389) 2026-10-06 11:55:51 +00:00
Jaakko Husso
5d3866775c
feat(ai-builder): Reopen closed AI Assistant tabs when the agent changes the artifact (#40081)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:34:27 +00:00
Yen Su
c134edf473
feat(Azure AI Foundry Chat Model Node): Speak the Anthropic Messages schema for Claude deployments (#40306) 2026-10-06 11:26:07 +00:00
Jaakko Husso
3366d2b9b3
fix(core): Stop marking the Message an Agent node as hidden (#40358)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:24:39 +00:00
Stephen Wright
7c126fab32
feat(Microsoft SharePoint Trigger Node): Add the Microsoft SharePoint Trigger node (#40055) 2026-10-06 11:14:11 +00:00
Lorent Lempereur
72960562ce
refactor(core): Run the workflow statistics rollup as a durable system task (#40083) 2026-10-06 11:10:53 +00:00
Lorent Lempereur
de72b7fa0d
refactor(core): Run the workflow statistics rollup as a system task (#40019) 2026-10-06 11:10:52 +00:00
Benjamin Schroth
d767edfacd
feat(editor): Add a Chat with agent picker for n8n Chat variant B (no-changelog) (#40337) 2026-10-06 11:02:56 +00:00
Stephen Wright
cf54433c14
feat(Microsoft SharePoint Node): Add list item events to the trigger (no-changelog) (#40052) 2026-10-06 10:51:32 +00:00
mfsiega
a6f1e79394
fix(core): Serve the migration report on every main in a multi-main setup (#40180)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 10:30:44 +00:00
Stephen Wright
3287ef1409
feat(Microsoft SharePoint Node): Implement the trigger poll (no-changelog) (#40051) 2026-10-06 10:25:43 +00:00
Benjamin Schroth
73dad06bea
feat(editor): Show the user's agents on the n8n Assistant page for n8n Chat variant A (no-changelog) (#40336) 2026-10-06 10:02:44 +00:00
Robin Braumann
4cb98aea9f
feat: Configure workflow tool inputs for Agents (#40285) 2026-10-06 10:02:23 +00:00
n8n-cat-bot[bot]
05a992d4f2
chore: Move the shared runtime-credential getter into BaseExecuteContext (#39470)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
2026-10-06 09:37:56 +00:00
Benjamin Schroth
8b68b0fcdd
feat: Add the n8n Chat agent chat page, agents library and merged chat lists (no-changelog) (#40335) 2026-10-06 09:35:35 +00:00
Anne Aguirre
958d23d81c
fix(editor): Small UI fixes to budget guardrail setup components (#40383) 2026-10-06 09:25:08 +00:00
Matsu
3c18f58ee0
ci: Reduce sync noise for sec-sync-bundle-branches (#40385) 2026-10-06 09:16:03 +00:00
Andreas Fitzek
2d09135c6e
feat(core): Persist trusted source metadata and discover it asynchronously (no-changelog) (#40048)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 09:14:52 +00:00
Stephen Wright
b7960d7a85
feat(Microsoft SharePoint Node): Add the event selector to the trigger (no-changelog) (#39987) 2026-10-06 09:11:25 +00:00
Stephen Wright
71ffb4c8cc
feat(Microsoft SharePoint Node): Add site and document library selection to the trigger (no-changelog) (#39943) 2026-10-06 09:11:25 +00:00
Stephen Wright
6ea9b1f3bb
refactor(Microsoft SharePoint Node): Move the pickers and helpers out of the v2 action node (no-changelog) (#39902) 2026-10-06 09:11:16 +00:00
Lorent Lempereur
264a931de2
feat(core): Stop a poll trigger task when its claim is lost (no-changelog) (#40273) 2026-10-06 09:07:38 +00:00
Lorent Lempereur
e82087bcc8
feat(core): Stop a durable system task run when its claim is lost (no-changelog) (#40272) 2026-10-06 09:07:37 +00:00
Lorent Lempereur
fe91413e5d
feat(core): Renew the lease of a durable task while its handler runs (no-changelog) (#40271)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: Emilia <100027345+sovietspaceship@users.noreply.github.com>
2026-10-06 09:07:36 +00:00
Lorent Lempereur
f4c734e7e0
feat(core): Add a lease heartbeat to the durable scheduler (no-changelog) (#40270) 2026-10-06 09:07:35 +00:00
Lorent Lempereur
4b69c32541
feat(core): Add a lease renewal query to the scheduled task repository (no-changelog) (#40269) 2026-10-06 09:07:34 +00:00
Lorent Lempereur
39abed9124
refactor(core): Simplify the poll trigger task handler (no-changelog) (#40268) 2026-10-06 09:07:33 +00:00
Lorent Lempereur
f26d7fbfe7
refactor(core): Extract the handler failure path of the scheduler executor (no-changelog) (#40267) 2026-10-06 09:07:32 +00:00
Robin Braumann
9ee5d1ad7a
fix(core): Save agent draft definitions atomically (#40374) 2026-10-06 09:01:38 +00:00
Matsu
7db7fa42c8
fix: Multiple migration timestamps withing a single PR (#40379) 2026-10-06 07:28:47 +00:00
Matsu
3f2144f062
chore: Update turbo.json oxlint configurations (#40377) 2026-10-06 07:27:56 +00:00
7053 changed files with 456610 additions and 92449 deletions

View File

@ -14,7 +14,8 @@ Flag a diff that **adds** entries to any of these, and ask for the fix instead:
| `.code-health-baseline.json` | `@n8n/code-health` violations |
| `.boundaries-baseline.json` | `turbo boundaries` issue count |
| `packages/quality/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
| `packages/cli/eslint.config.mjs` | the `misplaced-n8n-typeorm-import` and public-API allowlists, each captioned "NEVER add to this list" |
| `.code-health-baseline.json` | the `typeorm-persistence-boundary` baseline, which only shrinks as business logic moves queries into repositories |
| `packages/cli/eslint.config.mjs` | the public-API allowlists captioned "NEVER add to this list" |
| `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
Removals are the healthy direction and need no comment.

View File

@ -659,6 +659,12 @@ nodes get LLM-generated pin data). So:
otherwise
accept the static-data red as a harness limit and carry the logic in
`outcomeExpectations`. Note the agent may *choose* static-data dedup on its own.
- **Message an Agent runs the called Agent with its real model.** The harness
mocks the Agent's tools, MCP servers and web search, and records those calls
under the calling node. It turns off the Agent's memory, vector stores and chat
integrations. Only OpenAI models are supported today, and only when
`EVAL_OPENAI_API_KEY` is set (CI sets it): put the Agent on an `openai/*` model
and declare an `openAiApi` credential.
- Don't assert exact counts that depend on mock generation ("exactly 7 posts").
Say "fewer than the original 10".

View File

@ -78,11 +78,24 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
body structure, then populate each section with actual content before
creating the PR:
- **Summary**: describe what the PR does
- **Why**: explain why the change is needed (see Why below)
- **How to test**: describe how to test the changes. Include an example workflow if appropriate
- **Related tickets**: add the Linear ticket URL (`https://linear.app/n8n/issue/[TICKET-ID]`) and any GitHub issue links
- **Checklist**: keep as-is from the template
- Add a "🤖 PR Summary generated by AI" at the end of the body
Before you create the PR, proofread the body with these checks. Rewrite
each sentence that fails a check:
1. The Summary and the Why together are under the word budget. If not,
first remove the details that the reviewer reads in the diff, then
shorten the Why.
2. Each sentence names who does what: a component, a process, or a person
as the subject, with a plain technical verb.
3. Each domain term has a plain definition where it first appears.
4. Each sentence is true for the diff.
When a later commit changes a fact in the body, update the body.
```bash
gh pr create --draft --title "<type>(<scope>): <summary>" --body "$(cat <<'EOF'
<populated body based on pull_request_template.md>
@ -94,12 +107,43 @@ Creates GitHub PRs with titles that pass n8n's `check-pr-title` CI validation.
Based on `.github/pull_request_template.md`:
Write the PR body in ASD-STE100 Simplified Technical English: use short
sentences, the active voice, and one instruction for each sentence.
Write for a reviewer who does not know this area of the code. The reviewer
must understand the change before they open the diff. Use ASD-STE100
Simplified Technical English: short sentences, active voice.
1. Keep the Summary and the Why together under about 200 words. A small PR
needs much less.
2. Define each domain term in plain words where it first appears. Do not use
a ticket ID or an internal name as an explanation.
3. Leave out details that the reviewer reads in the diff: error cases, status
codes, renamed fields, the order of calls, file lists, test names. Mention
one only when it changes behaviour that a reviewer could miss. Also leave
out what reviewers already know (framework behaviour).
4. Use plain technical verbs, not metaphors ("passes the value", not
"hands the value" or "lets the value through").
### Summary Section
- Describe what the PR does
- Include screenshots/videos for UI changes
### Summary: what the PR does
1. Start with one to three sentences of background for a reader who is new to
the area. Then state what the PR does.
2. Write **prose** about behaviour, not a list of code edits. Name a class or
a function when it represents an essential/core concept and it helps
the reader to navigate the change.
3. If the PR changes no product code, say so in the first sentence
(for example: "This PR only adds tests.").
4. Do not justify the change here. The justification goes in the Why.
5. Include screenshots or videos for UI changes.
### Why: the reasoning
Always fill the `## Why` section. The Summary and the core of the
Why are what the reviewer needs most. Keep the core short:
1. Do not repeat the Summary.
2. State the problem or the constraint that makes the pre-existing code unfit.
3. If there are accepted gaps, or changes for existing code paths, state them
in a separate short paragraph.
Add rejected alternatives only when a reviewer would otherwise ask about them,
if any. Use one or two sentences for each alternative: state the concrete
reason it fails. Write an alternative that was never built in the conditional.
Together, the alternatives must be shorter than the core Why section.
### How to test Section
- Explain how to test the changes

View File

@ -110,6 +110,7 @@ Run through this before requesting review. Each item is a real, recurring review
- [ ] Identifiers go through **`escape.tableName(...)` / `escape.columnName(...)`**. Never hand-write `n8n_table` prefixes. — [Always escape identifiers](#always-escape-identifiers)
- [ ] **Match column type to value semantics.** Native `uuid` for UUIDs, `timestampTimezone()` for timestamps, a numeric type for numbers, `bool` for booleans, `json` for structured data. Never `varchar` as a catch-all. — [Column types](#column-types)
- [ ] **Pick the narrowest sane type within that category:** `int`/`smallint` not `bigint` when range allows; `text` not `varchar(255)` for unbounded strings; never `double` for version numbers. — [Column types](#column-types)
- [ ] **A generated `id` key is an integer** (`int.primary.autoGenerate2`). A UUID key is a UUIDv7. For a UUID or string key, write the reason in the PR description. A join table uses its foreign-key columns as the key, not an `id` column. — [Primary Keys](#primary-keys)
- [ ] **Default `notNull`**, relax only when justified. PK is implicitly NOT NULL. Migration's `notNull` matches the entity's nullability. — [NOT NULL and entity parity](#not-null-and-entity-parity)
- [ ] **Enum-like columns** carry `.withEnumCheck([...])` AND `.comment('explains values')`. Opaque IDs / unix timestamps / JSON shapes also get `.comment()`. — [Constrain enum-like strings](#constrain-enum-like-strings), [Add comments on columns](#add-comments-on-columns)
- [ ] **Every reference column has an explicit FK** with deliberate `onDelete`. Name FKs explicitly when SQLite recreate cycles risk duplicating them. Avoid polymorphic `(typeCol, idCol)` patterns. — [Foreign Key Constraints](#foreign-key-constraints), [General Design Guidance](#general-design-guidance)
@ -439,11 +440,32 @@ Don't set the default of a `status` column to a terminal value — `"running"` m
### Primary Keys
Every table needs a primary key. Choose the type in this order:
Every table needs a primary key. **Use an integer key by default.** Read the options below in order. Use the first option that fits. Do not skip an option only because you like a later one better.
1. **Integer** — `column('id').int.primary.autoGenerate2`. Preferred for new tables: compact, fast joins, no ordering surprises. `autoGenerate2` uses Postgres `IDENTITY` (preferred over the deprecated `serial`-based `autoGenerate`).
2. **UUID** — `column('id').uuid.primary`. Use when IDs are generated client-side, exposed in URLs, or need to be unguessable. Generate UUIDs in application code via `randomUUID()` from `node:crypto`; do **not** chain `.autoGenerate2` on `.uuid` (the DSL throws — `DEFAULT uuid_generate_v4()` fails on managed Postgres like Supabase because it needs the `uuid-ossp` extension in `public`). Use `.uuid` instead of `.varchar(36)`.
3. **String** — `column('id').varchar(36).primary` for IDs whose format isn't a UUID (e.g. nanoid-style IDs). Convention: nanoid length 16 for entity IDs.
1. **Integer (default)** — `column('id').int.primary.autoGenerate2`. Use this for every new table. Use a different option only when one of its conditions is true. Integer keys are small and make joins fast. `autoGenerate2` uses Postgres `IDENTITY`. Do not use `autoGenerate`. It uses `serial`, which is deprecated.
2. **UUID** — `column('id').uuid.primary`. Use this only when one of these conditions is true:
- The application must know the ID before the insert. For example, the client makes the ID.
- The ID is in a URL or a public API. Users must not see how many rows exist or guess other IDs.
**You must use UUIDv7.** Make the ID in application code with the `uuid` package: `import { v7 as uuidv7 } from 'uuid'`.
Why: a UUIDv7 starts with a timestamp. Thus, new rows go near the end of the index. A UUIDv4 is random, for example `randomUUID()` from `node:crypto`. New rows go to random positions in the index. This splits index pages and makes the index larger.
A UUIDv7 does not give an exact creation order. Two IDs made in the same millisecond can sort in the wrong order. Clock differences between hosts can also cause this. If you need the exact order, sort by a timestamp column.
Do **not** chain `.autoGenerate2` on `.uuid`. The DSL throws an error. The database default makes a UUIDv4. Also, `uuid_generate_v4()` needs the `uuid-ossp` extension. Some managed Postgres services, such as Supabase, do not have it. Use `.uuid`, not `.varchar(36)`.
3. **String** — `column('id').varchar(36).primary`. Use this only when one of these conditions is true:
- The entity already uses IDs in a format that is not a UUID. For example, the public API or the UI already shows nanoid IDs for this entity.
- The UI shows the ID to users. A UUID is too long to read, copy, or type.
Entity IDs are usually nanoids with 16 characters.
**Give a reason for every key that is not an integer.** Write the reason in the PR description. Reviewers will ask for it. These reasons are not sufficient:
- "Other tables use string IDs."
- "This module already does it this way."
- "We might need it later."
**These options apply to a generated `id` column only.** A join table that only links other tables does not need an `id` column. Use its foreign-key columns as a composite key (see below). A foreign-key column always has the same type as the key that it refers to.
**Keep ID-column types consistent across related tables.** Mixing `uuid` and `varchar(36)` for what is "the same kind of ID" creates JOIN footguns.
@ -696,7 +718,7 @@ If only Postgres needs the change, just put the file in `postgresdb/`; don't wri
### Postgres-version-aware UUID generation
`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, generate at the application level with `randomUUID()` — see [Primary Keys](#primary-keys).
`gen_random_uuid()` requires Postgres ≥ 13. n8n dropped Postgres 12 — prefer it over `uuid_generate_v4()` (which needs the `uuid-ossp` extension and breaks on managed services like Supabase). For UUID PKs, make a UUIDv7 in application code — see [Primary Keys](#primary-keys).
### SQLite doesn't enforce `varchar(N)` length

View File

@ -0,0 +1,158 @@
---
name: n8n:env-var-docs
description: >-
Writes and reviews the JSDoc on `@Env(...)` config properties. The docs team
extracts these comments from source into the public environment variable
reference, so they are user documentation, not code comments. Use when adding
or changing an `@Env` property, a config class, or reviewing a PR that touches
one.
---
# Environment variable docs
The JSDoc above every `@Env('...')` property is exported almost verbatim into
the [environment variables reference](https://docs.n8n.io/deploy/host-n8n/configure-n8n/basic-configuration/use-environment-variables)
on docs.n8n.io. The generated table has four columns: variable name,
description, data type, default. Your JSDoc is the description column. The
reader is a self-hosting operator, not an n8n developer.
Follow the [n8n docs style guide](https://docs.n8n.io/contribute/contribution-guide-for-n8n-docs/style-guide-for-n8n-docs)
for these comments. It overrides the ASD-STE100 rule in `AGENTS.md` for this
one case. The rules below are the parts of that guide that matter for a
one-paragraph description.
## Rules
- **Every `@Env` property has a JSDoc.** No exceptions. A missing comment
ships as an empty description cell.
- **Write for the operator.** Say what the setting does, when to change it,
and what happens when it's unset or wrong. Don't explain why the code is
written the way it is.
- **No code references.** No file names, class names, function names,
constants, license keys, or other internals the reader can't open. Values
the operator types into the variable are fine and often needed, for example
a credential type ID like `googleOAuth2Api` or a mode like `postgresdb`.
The test: would the operator paste this into a config file, or would they
need the source to know what it means? Details for developers go in a
regular `//` comment on the implementation, not in the JSDoc.
- **No attack scenarios.** Say what the setting protects, not how an
attacker would exploit it. This is a public page.
- **Don't repeat the default.** The default renders in its own column. Don't
write "Defaults to true" or "Default: 1 hour". Do say what the unset state
means when it's not obvious, for example "When unset, n8n uses the Redis
host".
- **State units and formats.** Say "in milliseconds", "in bytes", "a
comma-separated list", "a JSON object". The data type column only says
`Number` or `String`.
- **Use allowed values when the set is small.** "One of `sqlite`,
`postgresdb`." For a list, show one inline example in the sentence.
- **No JSDoc tags.** No `@example`, `@default`, `@see`, `@deprecated`. The
exporter copies the tag name into the docs. Put the example or the
deprecation notice in a sentence instead.
- **Product names:** "Gateway credits", not "n8n Connect" or "AI Gateway".
"Publish a workflow", not "activate". `n8n` is always lowercase.
- **Style:** present tense, active voice, second person. Use contractions.
No em dashes, use a comma or a new sentence. No "e.g." or "i.e.", write
"for example" and "that is". Spell out zero to nine, numerals from 10.
Values the operator types stay numerals, for example "Set to 0".
Sentence case. One idea per sentence. End every sentence with a period.
- **Plain words:** "use" not "utilize", "lets you" not "allows you to", cut
"powerful", "seamless", "simply", "it's important to note".
- **Length:** one to three sentences. If the setting needs more, the extra
belongs in a docs page the comment can't link to yet; keep the comment to
what fits a table cell.
## Examples
Bad, as found in the generated docs:
```ts
/**
* Instance-level enablement for n8n Connect (AI Gateway). Licensed instances
* are enabled by default; false opts out. Budget / free allowance still comes
* from the license (`quota:aiGatewayBudget`).
*/
```
Wrong product name, a restated default, and an internal license key. Good:
```ts
/**
* Whether Gateway credits are available on this instance. Set to false to
* turn them off on a licensed instance. The credit allowance comes from your
* license.
*/
```
Bad:
```ts
/**
* Time to live in milliseconds for Redis cache entries. Set to 0 to disable
* expiry. Default: 1 hour.
*/
```
The default is already in the table. Good:
```ts
/**
* Time to live in milliseconds for Redis cache entries. Set to 0 to keep
* entries until Redis evicts them.
*/
```
Bad:
```ts
/**
* Comma-separated list of credential types for which overwrites are skipped
* when the credential has been customized.
* @example `N8N_SKIP_CREDENTIAL_OVERWRITE=zohoOAuth2Api,salesforceOAuth2Api`
*/
```
The `@example` tag lands in the description as literal text. Good:
```ts
/**
* Comma-separated list of credential types to skip when applying credential
* overwrites, for example `zohoOAuth2Api,salesforceOAuth2Api`. n8n skips an
* overwrite only when a user has already changed one of its fields.
*/
```
Bad:
```ts
/**
* Bind OAuth credential callbacks to the browser that initiated the flow.
* When enabled, a session-scoped `n8n-oauth-binding` cookie is set on flow
* initiation and verified on callback. Defeats phishing attacks where an
* attacker injects an OAuth authorize URL into a victim's browser to land
* tokens on the attacker's credential.
*/
```
Attack narrative and an internal cookie name on a public page. Good:
```ts
/**
* Whether an OAuth credential connection must finish in the same browser
* that started it. When enabled, n8n rejects the callback if it arrives
* from a different browser.
*/
```
## Review checklist
When you review a PR that adds or changes an `@Env` property, check each
comment against this list and quote the fix in the review:
1. JSDoc present.
2. No code identifiers, file names, or internal keys.
3. No default restated.
4. Units, format, or allowed values stated.
5. No JSDoc tags.
6. No em dashes, Latin abbreviations, or filler words.
7. Reads correctly as a sentence in a table cell with no surrounding context.

View File

@ -21,8 +21,7 @@ check the code, not license to drop a team default.
## Non-negotiable rules
- New endpoints are `@PublicApiController` classes under `v1/controllers/`, one
`*.public.controller.ts` per feature. A controller is a class — never
`export =` (the legacy tuple style; `require-public-api-controller` flags it).
`*.public.controller.ts` per feature.
- Public API and internal REST are separate HTTP surfaces. A public controller
never calls an internal controller/endpoint; both reuse the same service.
- Controllers and handlers delegate to a service — never import a repository or
@ -31,7 +30,6 @@ check the code, not license to drop a team default.
`@ApiResponse(Dto)`.
- Register each controller via a side-effect import in `v1/controllers/index.ts`
(`public-api-controllers.test.ts` fails otherwise).
- Don't add business logic to legacy `express-openapi-validator` (EOV) handlers.
- Migrating a legacy endpoint must not change its public contract.
These are `n8n-local-rules` ESLint rules (see `packages/cli/eslint.config.mjs`)
@ -109,7 +107,7 @@ model; reuse only what applies. Decorators, all from `@n8n/decorators`:
| `@Get/@Post/@Put/@Patch/@Delete('/path')` | Route method. |
| `@ApiKeyScope('res:action')` | API-key grant check. |
| `@ProjectScope/@GlobalScope('res:action')` | User RBAC check. |
| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO — throws. |
| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO or a binary body — throws. For a binary response body (returned as `{ body, headers? }`), use `@ApiResponse(status, { mediaType, description?, headers? })` — see [Binary response bodies](reference.md#binary-response-bodies). |
| `@ApiErrorResponse(status)` | Declares an additional documented non-2xx status (e.g. `404`, `409`). Stack multiple for more than one. `400`/`401`/`403` are added automatically (body/query present, always, and `@ApiKeyScope` present, respectively) — don't declare those yourself. |
| `@ApiSummary(text)` / `@ApiDescription(text)` / `@ApiTags([...])` | OpenAPI summary/description/tags. `@ApiTags` sorts alphabetically regardless of the order you pass. All optional but expected on every real route. |
| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. `@Body` is JSON by default; `@Body({ mediaType: 'multipart/form-data', uploadLimits })` takes a `multipart/form-data` body instead — see [Request body media types](reference.md#request-body-media-types). |
@ -200,6 +198,5 @@ existing tests.
- [Test-before-save endpoints](reference.md#test-before-save-endpoints)
- [Errors](reference.md#errors)
- [Testing matrix](reference.md#testing-matrix)
- [Migrating legacy EOV endpoints](reference.md#migrating-legacy-eov-endpoints)
- [Verifying a migration](reference.md#verifying-a-migration)
- [CI and merging](reference.md#ci-and-merging)

View File

@ -122,11 +122,10 @@ app-wide `bodyParser` before the registry sees it. Declaring
- An unknown field (one the DTO doesn't declare, on a `{ strict: true }` DTO)
fails with `Unexpected form field "<name>"` — not the default "unrecognized
keys" wording.
- Multer's own parsing errors map to the same statuses express-openapi-validator
used: `413` for a size/count limit, `400` for any other multer error, `500`
— unmasked, since this is a `ResponseError` — for anything else (a
malformed body; a missing boundary gives `400` with `multipart file(s)
required`).
- Multer parsing errors map to these statuses: `413` for a size or count limit,
`500` for `LIMIT_UNEXPECTED_FILE`, `400` for any other multer error, and `500`
for anything else (a malformed body). The `500` is not masked, because it is a
`ResponseError`. A missing boundary gives `400` with `multipart file(s) required`.
- The body is parsed **after** every auth/scope/license/quota gate and
**before** controller/route middlewares — a caller those gates would reject
never has their (possibly huge) body read off the socket, and a middleware
@ -146,57 +145,34 @@ handler per media type (`REQUEST_BODY_HANDLERS` in
`REQUEST_BODY_HANDLERS` — the registry, resolver, generator and `/discover`
need no change, since they all read the handler, not the media type.
## Migrating legacy EOV endpoints
## Binary response bodies
Legacy `express-openapi-validator` endpoints live under
`v1/handlers/`, wired through `openapi.yml` with `x-eov-operation-*` and request
types in `packages/cli/src/public-api/types.ts`. Treat these as migration targets,
not templates.
`@ApiResponse(status, { mediaType, description?, headers? })` declares a
success body that the controller method returns as `{ body, headers? }`, for
example an `application/gzip` stream. The framework treats options whose
`mediaType` is in `BINARY_RESPONSE_MEDIA_TYPES` as a binary response.
`mediaType` accepts only those types.
- Prefer migrating to `@PublicApiController` over extending the handler.
- Preserve the public contract: path, method, scopes, status codes, response
shape, and pagination.
- Move HTTP concerns into the controller and business orchestration into the
shared service; keep the controller a thin HTTP boundary. If the legacy
handler was itself already just a thin wrapper around an internal
`@RestController` (calling its methods directly, e.g.
`Container.get(SomeController).createThing(req, res, payload)`), the new
public controller can call that same internal controller directly — no need
to duplicate its validation/business logic.
- A route must be served by either the EOV handler or a controller, not both —
the build's `mergeDecoratorDocument` (`v1/openapi-gen/generate.ts`) throws on
a path+method declared by both sides. Remove the legacy wiring (its path's
`$ref` entry in `openapi.yml`, the `x-eov-operation-*` handler, and its
`handler.ts`) only after the new controller is registered, `pnpm build`
regenerates the spec cleanly, and tests are updated.
- Fully delete the migrated legacy files: the handler's `.ts`, its
`spec/paths/*.yml` and `spec/schemas/*.yml`, and any now-dead request type
in `packages/cli/src/public-api/types.ts`. Then check
`v1/shared/spec/schemas/_index.yml` and `v1/shared/spec/parameters/_index.yml`
for entries that `$ref` one of the deleted schema/parameter files — those are
separate from the path's own `$ref` in `openapi.yml` and are easy to miss;
left dangling, the next bundle fails on a broken `$ref`.
- If the legacy handler gated on a license (`isLicensed('feat:x')` middleware),
`@Licensed('feat:x')` now replicates that for a controller route (see the
decorator table in [SKILL.md](SKILL.md#declaring-a-controller)) — but only for
a single feature. If the legacy check was an any-of/all-of over several flags
(e.g. `LicenseState.isProvisioningLicensed()`), `@Licensed` can't express
that; replicate it manually in the controller instead, don't drop it - this
is exactly what the internal `provisioning.controller.ee.ts` and
`role-mapping-rule.controller.ee.ts` already do, since neither uses
`@Licensed` for that reason.
- As a legacy file drops repository access / the `export =` tuple, remove its
entry from the `off` allowlists for `no-repository-in-public-api-handler` and
`require-public-api-controller` in `packages/cli/eslint.config.mjs` (shrink-only
— never extend them).
- A `multipart/form-data` legacy endpoint migrates onto `@Body({ mediaType:
'multipart/form-data', uploadLimits })` — see
[Request body media types](#request-body-media-types). For any other
non-standard endpoint, study the nearest existing handler first.
- Keep each field in its original position when you extract a request shape shared
by two routes, and destructure out the ones a route doesn't take. The generator
emits properties in shape order, so a moved field rewrites the `*.generated.yml`
of a route the PR wasn't changing.
- `body` is a `Buffer` or a `Readable`. The method returns the result and never
writes to `res`.
- `headers` holds the response headers. Every header declared in `headers` must
be in the result. A missing header fails the request with a `500`. Matching is
case-insensitive.
- The registry checks the headers, then sets the declared status, the headers,
and `Content-Type: <mediaType>`. A header in the result cannot replace the
`Content-Type`.
- A stream's first chunk is read before any header is set. So a stream that
fails before its first chunk gives the normal JSON error response, with no
binary `Content-Type`. A method that throws before it returns does the same.
- After the first chunk is sent, an error ends the response. A client that
disconnects is not an error, and the framework destroys the stream.
- A method that returns no binary body fails with a `500`.
- The generator documents the body as `{ type: string, format: binary }`
under the `mediaType` content key, with the description and headers.
The description defaults to `Operation successful.`.
The runtime part is `sendBinaryResponse` in
`packages/cli/src/public-api/media-types/binary-response.ts`.
## Verifying a migration

View File

@ -0,0 +1,464 @@
---
name: n8n:testmax-assistant-ui
description: >-
Tests an n8n Assistant (Instance AI) UI feature end to end. Drafts the test
paths from the feature and the diff, writes one Playwright script for each
path, runs the paths in parallel on fresh n8n instances with a real LLM, and
compares the main Assistant screens with master to find visual
regressions. Shows live screenshots during the run, and builds one HTML
report with video replays, step screenshots, LLM steps, tokens, cost and
visual diffs. Use when the user wants to test an Instance AI or Assistant
UI feature end to end, check that a UI change does not break existing
screens, or says /testmax-assistant-ui.
---
# Test an Instance AI feature
Use this skill to test a UI feature of the n8n Assistant end to end. In code,
the Assistant is "Instance AI". You draft the test paths, the user approves
them, and the skill runs them all at the same time. A local page shows each run
live. At the end, you judge each run and build one report.
The skill uses the scripts of `testmax-n8n-ai` for the n8n API, the
LangSmith usage, the live page server, and the report template. Do not copy
those scripts.
## What the skill does
```mermaid
flowchart LR
A[Feature + diff] --> B[Draft scenarios.json<br/>and scripts]
B --> T[--try and --estimate]
T --> C{User approves<br/>plan + cost}
C --> D[run.py]
D --> E[One fresh branch instance<br/>for each feature + regression]
D --> F[Fresh branch instance<br/>visual]
D --> G[Fresh master instance<br/>visual, or a kept run]
E & F & G --> H[Live page]
F & G --> I[Pixel + accessibility tree diff]
E & F & G --> J[summary.txt with triage]
J --> K[You judge: verdicts.json]
K --> L[Report rebuilds itself]
```
- Each run is one scenario: a Playwright script that drives the real
Assistant UI in Chrome, with a real LLM.
- There are three kinds of scenario:
- `feature`: tests a path of the new feature.
- `regression`: tests an existing flow that the feature touches.
- `visual`: captures screens. The skill runs it on the branch and on master,
then compares the captures.
- All instances are fresh: a new data folder, a new owner, and no workflows or
threads. Thus the screens of the branch and master are the same when the
code is the same. A fresh instance starts in about 5 seconds, because it
copies a data folder that is already migrated and seeded.
- Each feature and regression scenario gets its own fresh instance, so the
scenarios do not see each other's data. The visual scenarios share one
branch instance and one master instance.
- The feature flags are the same on all instances. The instances do not ask
PostHog for flags, so no instance gets a random experiment variant. Set the
flags that the feature needs in the plan, with `featureFlags`.
- `run.py` builds this checkout first (with turbo, so a build that is up to
date takes about 1 second). The master instance starts from a git worktree
at `~/.cache/n8n-feature-test/master`, at the merge base of `HEAD` and
`origin/master`. Thus the visual check shows only the changes of this
branch, not the changes that master got after the branch started.
- A master visual run that passed is kept. The next run with the same merge
base, the same script and the same flags uses it again, and does not start
master.
## Requirements
Run `python3 .agents/skills/testmax-assistant-ui/scripts/run.py --check`
before the first run. The commands below use `$S` for this scripts folder.
`--check` checks the tools,
Chrome, the settings, the disk space and the master build, and prints the
plan folder for this branch.
- Google Chrome. The scripts use the Playwright package of the repository.
- The Instance AI model settings in `.env.local` or `.env.ai-surfaces`, as for
your own instance. Fresh instances get the same settings, without the
license, the database, the port and the encryption key.
- `LANGSMITH_API_KEY` and LangSmith tracing, for the tokens and cost. Without
them, the report shows no tokens or cost. Do not set
`N8N_DIAGNOSTICS_ENABLED=false`: then Instance AI sends no traces.
- About 3 GB of disk space for the master worktree and its build.
Never put a key on the command line. Read settings from the env files and the
environment only.
## 1. Understand the feature
Find what changed and what the user wants to test:
```bash
git diff origin/master...HEAD --stat -- packages/frontend packages/@n8n/instance-ai packages/cli/src/modules/instance-ai
git diff origin/master...HEAD -- <the files of the feature>
```
Read the changed Vue components, the stores, and the backend events that they
use. Find the `data-test-id` values of the new elements. If the user did not
describe the feature, ask for one or two sentences about it.
If the plan has visual scenarios, start the master baseline now, in the
background. The first build takes 15 to 20 minutes. Later builds take less
than a minute when the turbo cache has the files. You can draft the plan while
it builds:
```bash
S=.agents/skills/testmax-assistant-ui/scripts
python3 $S/run.py --check # the setup, and the plan folder
python3 $S/baseline.py prepare # in the background
python3 $S/baseline.py status # later: is it ready?
```
## 2. Draft the test paths
Write the plan folder that `--check` prints:
`~/.cache/n8n-feature-test/plans/<branch>/<feature>/`. The folder stays after
a reboot. It holds `scenarios.json` and one script for each scenario. The
folder name is the name of the run folders.
Draft these paths:
- **Feature paths.** The main path, then each branch of it: other user
choices, empty and long inputs, cancel and stop, a reload in the middle, a
second thread, errors that the user can see. One path for each scenario.
- **Regression paths.** The existing flows near the change: a plain chat
reply, approvals, the plan review, the preview panel, the thread list. The
library has `smoke-chat` for the plain chat reply.
- **Visual screens.** The screens that the change can affect, on the branch
and on master. Capture each screen in a stable state. Mask all text that
comes from the LLM, and all data that changes between runs. The library has
`visual-assistant-home` and `visual-thread-reply`.
Keep each scenario short and focused. A scenario that fails must show where it
failed. Use more scenarios, not longer ones. A visual scenario must not need
the new feature, because it also runs on master.
Some UI states show only for a moment, for example a status line that folds
away when the run ends. Check them while the run is active, not after it.
If the feature is behind a feature flag or an experiment, set the flag in
`featureFlags`. Without it, the instances use the default value of each flag.
Before you show the plan to the user, check the new scripts and get the
estimate (see [Check the scripts](#check-the-scripts)). Then show the user the
list of paths: the id, the kind, the title and the expected result of each.
Also show the estimate: the number of LLM runs, the cost and the time. Change
the plan until the user approves it.
### scenarios.json
```json
{
"title": "Stop button in the Assistant",
"feature": "The user can stop a run while the Assistant works.",
"scenarios": [
{
"id": "stop-mid-run",
"title": "Stop a run in the middle",
"kind": "feature",
"script": "stop-mid-run.mjs",
"expected": "The stop button shows during the run. A click ends the run without an error message.",
"timeoutMin": 6
},
{
"id": "smoke-chat",
"title": "A prompt gets a reply",
"kind": "regression",
"script": "lib:smoke-chat.mjs",
"expected": "A short prompt gets the reply OK."
}
]
}
```
The plan has these fields: `title` (required), `feature` (the feature, for the
judge), and `featureFlags` (optional). `featureFlags` maps a flag name to
`true`, `false` or a variant name, for example
`{ "<flag name>": "variant" }`. Each instance gets these flags, on the
branch and on master. They go into `N8N_FEATURE_FLAG_OVERRIDES` and replace
the values in your env files.
Each scenario has these fields:
| Field | Required | Description |
|---|---|---|
| `id` | yes | Lowercase letters, digits and `-`. No `--`. Unique. |
| `title` | yes | A short name for the report. |
| `kind` | yes | `feature`, `regression` or `visual`. |
| `script` | yes | A path relative to `scenarios.json`, or `lib:<file>` for a library script. |
| `expected` | yes | The expected result, for the judge. |
| `autoConfirm` | no | Answer the Assistant's confirmations automatically. Default `true`. Credential and setup cards get "Later". Set `false` when the script answers a confirmation or a card. |
| `isolated` | no | Run the scenario on its own fresh instance. Default `true`. Instances start ahead of their scenario, so this costs almost no time. Set `false` to run the scenario on one branch instance that it shares with the other `false` scenarios. Visual scenarios ignore this field. |
| `timeoutMin` | no | The time limit of the scenario. Default 10. |
| `viewport` | no | The window size, for example `1600x1000` (the default). |
`examples/stop-button/` is a full example plan.
### Scenario scripts
A script exports one async function. It gets the kit `t`:
```js
export default async function (t) {
await t.step('Open the Assistant', () => t.goto());
await t.step('Send a prompt', () => t.send('Reply with exactly the word OK and nothing else.'));
await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
await t.check('The reply says OK', () => t.expect(t.ai.assistantMessages().last()).toContainText('OK'));
await t.capture('thread', { mask: [t.ai.assistantMessages()] });
}
```
| Kit member | What it does |
|---|---|
| `t.step(title, fn)` | Runs an action. Records a screenshot and the time. A failure stops the scenario. |
| `t.check(title, fn)` | Runs an assertion, with a screenshot. A failure is recorded, and the scenario continues. |
| `t.note(title)` | Adds a step with a screenshot and no action. |
| `t.goto(path = '/assistant')` | Opens a page and waits for the chat input. Closes the "Try new n8n Assistant" dialog. |
| `t.send(text)` | Types into the chat input, sends, and waits for the thread URL. Records the thread for the trace. |
| `t.waitForIdle({timeout, settle})` | Waits until the thread has no active run. Returns `'idle'`, or `'suspended'` when the run waits for the user. |
| `t.capture(name, {mask, maskText, root, fullPage})` | Saves a screen and its accessibility tree for the visual check. `mask` takes locators, and `maskText` takes texts: all elements with that text. Masked areas show as one color. In the tree, the subtree of a `mask` element and each masked text show as `[masked]`. |
| `t.api(method, path, body)` | Calls the n8n REST API as the signed-in user. Returns `data`. |
| `t.threadStatus(threadId)` | Returns `{active, suspended, background}`: the thread has a run, the run waits for the user, and the number of running background tasks. |
| `t.ai.*` | Locators for the Assistant: `chatInput`, `sendButton`, `stopButton`, `userMessages`, `assistantMessages`, `assistantText(text)`, `panelText(text)`, `sidebarToggle`, `threadList`, `newThreadButton`, `approveButton`, `denyButton`, `planApproveButton`, `previewPanel`, `previewCanvasNodes`, and more in `scripts/kit.mjs`. |
| `t.expect`, `t.page`, `t.baseUrl`, `t.sleep(ms)` | Playwright `expect`, the page, the instance URL, and a pause. |
Rules for scripts:
- Use `data-test-id` locators (`page.getByTestId`). Use text only for text that
the test controls.
- Use `t.expect(...).toBeVisible()` and other assertions that wait. Do not use
`t.sleep` to wait for the UI.
- LLM answers change from run to run. Check the UI state, not the exact words.
Ask for a fixed reply, such as "Reply with exactly the word OK", when you
need a known answer.
- To test an approval, set `"autoConfirm": false`, wait with
`t.waitForIdle()` until it returns `'suspended'`, then click the button.
### Check the scripts
Check the new scripts on fresh instances before you show the plan:
```bash
python3 $S/run.py --plan <plan>/scenarios.json --try --only id1,id2
```
`--try` does not run master, does not start the live page, and builds no
report. It prints a table with the status, the steps and the first error of
each run. The runs are in `/tmp/feature-test/try/`. Fix the scripts that fail
because of the script, then try them again. Each LLM scenario in a try costs
money too, so try only the new scripts.
Then get the estimate for the full run:
```bash
python3 $S/run.py --plan <plan>/scenarios.json --estimate
```
The estimate uses the cost and time of earlier runs of the same scenario ids.
A scenario without an earlier run shows `*` and the median of all earlier
runs. A master run that is kept shows `(kept)` and costs nothing.
## 3. Run
Open the live page for the user on each run. Do not skip this step. Do this:
1. Find out if your harness has a browser tool that opens a tab in the
editor. In Cursor, this is `browser_navigate` of the `cursor-ide-browser`
MCP server.
2. If it has one, start the run in the background with `--no-open`:
```bash
python3 $S/run.py --plan <plan>/scenarios.json --no-open
```
Wait for the `live page: <url>` line in the output, which comes in a few
seconds. Then open that URL in a new editor tab that the user can see (in
Cursor: `newTab: true` and `position: "active"`). If the tab does not open,
run `open <url>` (macOS) or `xdg-open <url>` (Linux).
3. If it has none, start the run in the background without `--no-open`.
`run.py` opens the live page in the default browser:
```bash
python3 $S/run.py --plan <plan>/scenarios.json
```
4. Give the user the URL in both cases.
`run.py` does not read `$BROWSER`, so do not set it.
`run.py` does these steps:
1. Prints the estimate and checks the setup. It stops when the setup has an
error, for example a missing Chrome.
2. Copies the plan into a new run folder,
`/tmp/feature-test/<time>-<plan folder name>/`.
3. Starts the live page and prints its URL. Without `--no-open`, it opens the
page in the default browser.
4. Builds this checkout with turbo. A build error stops the run, and the
output shows the end of `build.log`.
5. Starts the instances: one for each feature and regression scenario, one
branch instance for the visual scenarios, and one master instance. It
starts the instances ahead of their scenario. Master starts only for the
master runs that it does not keep from an earlier run. It prepares the
master build first, if it is not ready.
6. Runs the scenarios, 4 at a time. Each run records a video, the steps with
screenshots, the browser errors, the thread events, and the LangSmith ids.
7. Stops the instances, compares the visual captures, writes `summary.txt`,
and builds the report without verdicts.
8. Prints a result table, the command to run the plan again, and one JSON
object on several lines: `runDir`, `report`, `live`, `summary`, `results`
(the status of each run), `visual` (the number of captures, the changed
ones, and the error when the visual comparison failed), `rerun` and
`next`.
9. Gets the tokens and cost from LangSmith, then builds the report again.
LangSmith receives traces with a delay, so this can take up to 90 seconds.
You can start to judge while it runs.
| Option | Use |
|---|---|
| `--check` | Check the setup and print the plan folder, then stop. |
| `--estimate` | Print the expected cost and time, then stop. |
| `--try` | Check new scripts: no master, no live page, no report. See [Check the scripts](#check-the-scripts). |
| `--only id1,id2` | Run only these scenarios. |
| `--rerun last` | Run the plan of the last run again, in a new folder. |
| `--rerun <run-dir>` | Run the plan of this earlier run again, in a new folder. If the plan folder of that run still exists, it uses the current plan and scripts from there. Thus a fixed script runs in its fixed version. |
| `--concurrency N` | Runs at the same time. Default 4, or `FEATURE_TEST_CONCURRENCY`. |
| `--branch-url URL` | Use a running instance for all branch runs, not fresh ones. The threads stay on that instance. The scenarios share its data, and the visual check is not reliable. |
| `--no-build` | Do not build this checkout. The output warns when source files are newer than their build. |
| `--no-baseline` | Do not run master. There is no visual check. |
| `--no-cache` | Run master again, also when a kept master run exists. |
| `--no-fetch` | Do not fetch `origin/master`. Use the merge base with the `origin/master` that git has. |
| `--no-open` | Do not open the live page in the default browser. Use this when you open it in the editor tab. |
| `--no-live` | Do not start the live page. Use this only when the user asks for it. |
| `--keep-instances` | Keep the fresh instances running after the run, for a manual check. The output gives `instancePorts`. Each instance is at `http://localhost:<port>`. Stop them yourself. |
| `--run-dir DIR` | Use this run folder. |
| Setting | Default |
|---|---|
| `FEATURE_TEST_RUNS_DIR` | `/tmp/feature-test` |
| `FEATURE_TEST_CACHE` | `~/.cache/n8n-feature-test`: the plans, the master worktree, the kept master runs, and the instance data templates |
| `FEATURE_TEST_CONCURRENCY` | `4` |
### Watch the runs
The live page shows each run as a row, with its status and its last
screenshot. The screenshot changes every few seconds. When the run is done,
the same address shows the report. The page stops 30 minutes after its last
request.
## 4. Judge
Read `<runDir>/summary.txt`. It starts with a triage: each run is `clean`, or
`look closer` with the reasons (a failed step or check, a browser error, a
failed tool call, or a visual change). Then it has, for each run: the expected
result, the steps and their status with the path of each screenshot, the
prompts, the last reply, the browser errors, and the visual results.
For a clean run, look at the last screenshot and check it against the expected
result. For a run to look at closer, look at the screenshots of the steps in
the reasons, and the diff images in `visual/<id>/`.
A script that passed is not always a pass. Check that the run shows the
expected result, not only that the script ended. Check for these problems:
- a browser error or a failed request that comes from the feature
- a step that passed, but the screenshot shows the wrong state
- a visual change that the feature did not intend
- a script that failed because of the script, not the feature. Say so, and
fix the script
Write `<runDir>/verdicts.json`. Use the run names as keys. A master run has
the name `<id>--master`, and does not need a verdict:
```json
{
"stop-mid-run": {
"verdict": "pass",
"summary": "One sentence about the result.",
"checks": [
{ "ok": true, "text": "The stop button ends the run" },
{ "ok": false, "text": "The send button comes back after 9 seconds" },
{ "ok": null, "text": "Could not confirm the stopped label" }
]
}
}
```
Use `pass` when all checks pass, `fail` when the main goal is not met, and
`partial` for all other results. For a visual scenario, judge each change:
intended by the feature, a regression, or noise (for example, a timestamp
that is not masked). Be strict and factual.
Write each check as one short sentence with one fact. Put selectors, test
IDs and UI text in backticks. The page shows them as code. The page shows
the failed checks first.
## 5. The report
While the live page runs, it builds the report again each time
`verdicts.json` changes. The open tab loads the new report by itself, and
keeps the selected run and the scroll position. Thus you do not need to load
the page again. When the live page does not run, build the report yourself:
```bash
python3 $S/report.py <runDir>
```
The report is `<runDir>/report/index.html`. It is one folder with all its
files, so you can copy or share it. It has, from top to bottom:
- the video replays of all runs
- a table of all runs: kind, result, steps passed, time, LLM calls, tool calls,
tokens, cost and browser errors. The result is your verdict. A master run
shows `Baseline`, and a run without a verdict shows `Not judged`, or
`Failed` when its script failed
- the visual check: the branch screen, the master screen, and the diff image
of each capture, with the accessibility tree diff
- for each run: the video of the run, the expected result and the verdict,
the steps with their screenshots, the browser errors, and the LLM steps
from the trace. A click on a row of the table shows the panel and the
video of that run, and pauses the video of the other runs
Each box shows closed. It opens and closes when you click its title. On the
live page, a box that you open or close stays that way when the page
updates.
A master run that is kept from an earlier run shows "Reused" in its run box.
Give the user the path and the `live` URL. Tell the user the result of each
scenario, the visual changes, and the problems that you found.
## Single steps
| Step | Command |
|---|---|
| Prepare the master baseline | `python3 $S/baseline.py prepare` |
| Show the baseline state | `python3 $S/baseline.py status` |
| Run one script on any instance | `N8N_BASE_URL=<url> node $S/run_scenario.mjs --out <dir> --script <file>` |
| Compare the visual captures again | `node $S/visual_diff.mjs <runDir>` |
| Print the judge summary | `python3 $S/report.py <runDir> --summary` |
| Serve the live page of a run | `python3 $S/live_page.py --run-dir <runDir>` |
## Limits
- LLM answers change from run to run, and a run can fail because of the LLM.
Run a failed scenario again with `--rerun last --only <id>` before you call
it a bug.
- A stopped LLM call does not show in LangSmith. The cost of a run with a stop
includes only the calls that ended.
- Each LLM scenario costs money. The tokens and cost show in the report.
- Parallel runs share the CPU and the LLM rate limits. The times include the
shared load.
- The visual check compares pixels with a small tolerance. Fonts and
animations can cause small changes. Masks hide the parts that change between
runs. The accessibility tree diff shows changes in structure and text.
- The worktree shows in `git worktree list`. To remove it, run
`git worktree remove ~/.cache/n8n-feature-test/master`.
- Kept master runs older than 14 days are deleted. The instance data
templates in `~/.cache/n8n-feature-test/templates` change when the
migrations change. You can delete old templates at any time.
- A run folder is about 35 MB. The run folders are in `/tmp`, so the system
deletes them on reboot.

View File

@ -0,0 +1,38 @@
{
"title": "Stop button in the Assistant",
"feature": "The user can stop a run while the Assistant works. After the stop, the composer is ready and a new message gets a reply. The Assistant screens look the same as on master.",
"scenarios": [
{
"id": "stop-mid-run",
"title": "Stop a run in the middle",
"kind": "feature",
"script": "stop-mid-run.mjs",
"expected": "The stop button shows during the run. A click ends the run without an error message. The send button comes back, and a follow-up message gets a reply.",
"timeoutMin": 6
},
{
"id": "smoke-chat",
"title": "A prompt gets a reply",
"kind": "regression",
"script": "lib:smoke-chat.mjs",
"expected": "A short prompt opens a new thread and gets the reply OK. The composer is ready after the reply.",
"timeoutMin": 5
},
{
"id": "assistant-home",
"title": "Assistant home and chat history",
"kind": "visual",
"script": "lib:visual-assistant-home.mjs",
"expected": "The home screen and the chat history popover look the same as on master.",
"timeoutMin": 3
},
{
"id": "thread-reply",
"title": "Thread with one reply",
"kind": "visual",
"script": "lib:visual-thread-reply.mjs",
"expected": "A thread with one short exchange looks the same as on master.",
"timeoutMin": 5
}
]
}

View File

@ -0,0 +1,31 @@
// Feature: the user stops a run while the Assistant works, and can send a new message after it.
export default async function (t) {
await t.step('Open the Assistant', () => t.goto());
await t.step('Send a prompt that takes a while', () =>
t.send(
'Explain step by step, in at least 30 numbered points, how to design a workflow that syncs ' +
'contacts between two CRMs. Do not build anything.',
),
);
await t.step('The stop button shows while the run is active', () =>
t.expect(t.ai.stopButton()).toBeVisible({ timeout: 30_000 }),
);
await t.step('Click stop', () => t.ai.stopButton().click());
await t.check('The stop button goes away', () =>
t.expect(t.ai.stopButton()).toBeHidden({ timeout: 15_000 }),
);
await t.check('The send button comes back', () =>
t.expect(t.ai.sendButton()).toBeVisible({ timeout: 15_000 }),
);
await t.step('Wait until the thread is idle', () => t.waitForIdle({ timeout: 60_000 }));
await t.check('The thread shows no error message', async () => {
await t.expect(t.ai.panelText(/something went wrong/i)).toHaveCount(0);
});
await t.step('Send a follow-up message', () =>
t.send('Reply with exactly the word OK and nothing else.'),
);
await t.step('Wait for the follow-up reply', () => t.waitForIdle({ timeout: 120_000 }));
await t.check('The follow-up gets a reply', () =>
t.expect(t.ai.assistantMessages().last()).toContainText('OK'),
);
}

View File

@ -0,0 +1,17 @@
// Regression: a prompt gets a reply in a new thread.
export default async function (t) {
await t.step('Open the Assistant', () => t.goto());
await t.step('Send a short prompt', () =>
t.send('Reply with exactly the word OK and nothing else.'),
);
await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
await t.check('The user message shows', () =>
t.expect(t.ai.userMessages().first()).toBeVisible(),
);
await t.check('The reply says OK', () =>
t.expect(t.ai.assistantMessages().last()).toContainText('OK'),
);
await t.check('The composer is ready for the next message', () =>
t.expect(t.ai.sendButton()).toBeVisible(),
);
}

View File

@ -0,0 +1,11 @@
// Visual: the Assistant home screen, and the chat history popover.
export default async function (t) {
await t.step('Open the Assistant', () => t.goto());
await t.capture('assistant-home');
await t.step('Open the chat history', async () => {
await t.ai.sidebarToggle().click();
await t.expect(t.ai.threadList()).toBeVisible();
});
// Other visual scenarios create threads at the same time, so the list content changes between runs.
await t.capture('chat-history', { mask: [t.ai.threadList()] });
}

View File

@ -0,0 +1,11 @@
// Visual: a thread with one short exchange, and the composer after the reply.
export default async function (t) {
await t.step('Open the Assistant', () => t.goto());
await t.step('Send a short prompt', () =>
t.send('Reply with exactly the word OK and nothing else.'),
);
await t.step('Wait until the run ends', () => t.waitForIdle({ timeout: 180_000 }));
// The reply text and the thread title come from the LLM. The title also shows in the main sidebar.
const title = (await t.ai.sidebarToggle().innerText()).trim();
await t.capture('thread-reply', { mask: [t.ai.assistantMessages()], maskText: [title] });
}

View File

@ -0,0 +1,117 @@
#!/usr/bin/env python3
"""Keep a built copy of master for the visual check.
baseline.py status JSON: the worktree, the built commit, and the commit to build
baseline.py prepare [--no-fetch] fetch origin/master, update the worktree, install, and build
The commit to build is the merge base of HEAD and origin/master: the master commit that this branch
starts from. Then a visual change comes from the branch, not from newer commits on master. The
build runs only when the merge base moves, for example after a rebase.
The worktree is in <FEATURE_TEST_CACHE>/master (default ~/.cache/n8n-feature-test/master). It is a
detached git worktree of this repository, so `git worktree list` shows it. The first build takes 15
to 20 minutes. The output goes to <FEATURE_TEST_CACHE>/build.log.
"""
import argparse
import fcntl
import json
import os
import subprocess
import sys
import time
from datetime import datetime, timezone
from common import CACHE, git
WORKTREE = CACHE / 'master'
STATE = CACHE / 'built.json'
BUILD_LOG = CACHE / 'build.log'
def built():
try:
state = json.loads(STATE.read_text())
except (OSError, ValueError):
return None
cli = WORKTREE / 'packages' / 'cli' / 'dist'
editor = WORKTREE / 'packages' / 'frontend' / 'editor-ui' / 'dist' / 'index.html'
return state if cli.exists() and editor.exists() else None
def target(fetch=False):
"""The master commit to build: the merge base of HEAD and origin/master."""
if fetch:
git('fetch', '--quiet', 'origin', 'master')
try:
return git('merge-base', 'HEAD', 'origin/master')
except subprocess.CalledProcessError:
return git('rev-parse', 'origin/master')
def status():
state = built() or {}
try:
commit = target()
except subprocess.CalledProcessError:
commit = None
return {'worktree': str(WORKTREE), 'exists': WORKTREE.exists(), 'builtCommit': state.get('commit'),
'builtAt': state.get('builtAt'), 'baseCommit': commit,
'upToDate': bool(commit) and state.get('commit') == commit, 'log': str(BUILD_LOG)}
def run_logged(cmd, log, env=None):
log.write(f'\n$ {" ".join(cmd)}\n')
log.flush()
code = subprocess.run(cmd, cwd=WORKTREE, stdout=log, stderr=subprocess.STDOUT, env=env).returncode
if code != 0:
raise RuntimeError(f'{" ".join(cmd)} failed with exit code {code}. See {BUILD_LOG}')
def prepare(fetch=True, say=print):
"""Return {"root", "commit"} of a built merge base. Builds only when the merge base moved."""
CACHE.mkdir(parents=True, exist_ok=True)
with (CACHE / 'build.lock').open('w') as lock:
# Two runs at the same time must not build into the same worktree.
fcntl.flock(lock, fcntl.LOCK_EX)
if fetch:
say('fetch origin/master')
commit = target(fetch)
if not (WORKTREE / '.git').exists():
say(f'create the master worktree in {WORKTREE}')
git('worktree', 'add', '--detach', str(WORKTREE), commit)
elif git('rev-parse', 'HEAD', cwd=WORKTREE) != commit:
git('checkout', '--detach', '--force', commit, cwd=WORKTREE)
state = built()
if state and state.get('commit') == commit:
say(f'master {commit[:11]} is built')
return {'root': WORKTREE, 'commit': commit}
started = time.time()
say(f'build master {commit[:11]} (the merge base). The first build takes 15 to 20 minutes. Log: {BUILD_LOG}')
with BUILD_LOG.open('w') as log:
# DOCKER_BUILD skips the git hook install of the prepare script, so the hooks of the
# main checkout stay as they are.
run_logged(['pnpm', 'install', '--frozen-lockfile'], log, env={**os.environ, 'DOCKER_BUILD': 'true'})
run_logged(['pnpm', 'turbo', 'run', 'build', '--filter=n8n...'], log)
STATE.write_text(json.dumps({'commit': commit, 'builtAt': datetime.now(timezone.utc).isoformat()}, indent=2))
say(f'master {commit[:11]} built in {time.time() - started:.0f}s')
return {'root': WORKTREE, 'commit': commit}
def main():
ap = argparse.ArgumentParser()
ap.add_argument('action', choices=['status', 'prepare'])
ap.add_argument('--no-fetch', action='store_true')
args = ap.parse_args()
if args.action == 'status':
print(json.dumps(status(), indent=2))
return
try:
result = prepare(fetch=not args.no_fetch, say=lambda m: print(m, flush=True))
except (RuntimeError, subprocess.CalledProcessError) as e:
sys.exit(str(e))
print(json.dumps({'root': str(result['root']), 'commit': result['commit']}, indent=2))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,175 @@
"""Shared helpers: paths, settings, and fresh n8n instances for the scenario runs. Standard library only.
The skill reuses the scripts of testmax-n8n-ai (settings, REST client, report parts,
live server). This module puts them on the import path.
"""
import fcntl
import hashlib
import json
import os
import shutil
import signal
import socket
import subprocess
import sys
import time
import urllib.request
from pathlib import Path
HERE = Path(__file__).resolve().parent
SHARED = HERE.parents[1] / 'testmax-n8n-ai' / 'scripts'
sys.path.insert(0, str(SHARED))
from n8n_api import REPO_ROOT, N8nClient, config, setting # noqa: E402
CACHE = Path(setting('FEATURE_TEST_CACHE', default=str(Path.home() / '.cache' / 'n8n-feature-test'))).expanduser().resolve()
RUNS_ROOT = Path(setting('FEATURE_TEST_RUNS_DIR', default='/tmp/feature-test')).expanduser().resolve()
PLANS_ROOT = CACHE / 'plans'
TEMPLATE_VERSION = '1'
# These settings belong to the instance that the user runs. A fresh instance must not reuse them.
INSTANCE_ENV_SKIP = ('N8N_LICENSE_', 'DB_', 'N8N_PORT', 'N8N_USER_FOLDER', 'N8N_EDITOR_BASE_URL', 'WEBHOOK_URL',
'N8N_RUNNERS_BROKER_PORT', 'N8N_HOST', 'N8N_PROTOCOL', 'N8N_ENCRYPTION_KEY', 'N8N_BASE_URL')
def free_port():
with socket.socket() as s:
s.bind(('127.0.0.1', 0))
return s.getsockname()[1]
def instance_env(port, home, feature_flags=None):
"""The settings of the user's instance (.env.local and .env.ai-surfaces), with a new port and data folder.
Feature flags do not come from PostHog: PostHog gives each new instance its own experiment variants,
so the screens of two instances with the same code can differ. The flags are their defaults, plus
N8N_FEATURE_FLAG_OVERRIDES, plus feature_flags.
"""
env = {k: v for k, v in os.environ.items() if not k.startswith(INSTANCE_ENV_SKIP)}
env.update({k: v for k, v in config().items() if not k.startswith(INSTANCE_ENV_SKIP)})
# Keep N8N_DIAGNOSTICS_ENABLED as the user set it: when it is false, Instance AI sends no LangSmith traces.
env.update(
N8N_PORT=str(port), N8N_RUNNERS_BROKER_PORT=str(free_port()), N8N_USER_FOLDER=str(home),
N8N_EDITOR_BASE_URL=f'http://localhost:{port}', WEBHOOK_URL=f'http://localhost:{port}/',
N8N_PERSONALIZATION_ENABLED='false', N8N_INSTANCE_AI_EVAL_INSTANCE='true',
)
if feature_flags:
try:
overrides = json.loads(env.get('N8N_FEATURE_FLAG_OVERRIDES') or '{}')
except ValueError:
overrides = {}
env['N8N_FEATURE_FLAG_OVERRIDES'] = json.dumps({**overrides, **feature_flags})
return env
def template_dir(root):
"""The data folder template of a checkout. The key has the database migrations and the owner, so a build
with other migrations or another N8N_EVAL_EMAIL gets a new template. Change TEMPLATE_VERSION when seed()
changes."""
migrations = Path(root) / 'packages' / '@n8n' / 'db' / 'dist' / 'migrations'
names = sorted(str(p.relative_to(migrations)) for p in migrations.rglob('*.js')) if migrations.exists() else []
owner = client('http://localhost')
parts = [TEMPLATE_VERSION, str(Path(root).resolve()), owner.email, owner.password, *names]
return CACHE / 'templates' / hashlib.sha256('\n'.join(parts).encode()).hexdigest()[:16]
class Instance:
"""A fresh n8n instance with its own port and data folder, started from a checkout that is built.
The first instance of a checkout runs the database migrations and the owner setup, then saves its
data folder as a template. Later instances start from a copy of the template, which is faster.
"""
def __init__(self, name, root, home, log, feature_flags=None):
self.name, self.root, self.home, self.log_path = name, Path(root), Path(home), Path(log)
self.feature_flags = feature_flags
self.port = free_port()
self.url = f'http://localhost:{self.port}'
self.proc = None
def start(self, timeout=180):
template = template_dir(self.root)
if (template / '.ready').exists():
shutil.copytree(template / '.n8n', self.home / '.n8n', dirs_exist_ok=True)
self._boot(timeout)
return self
template.parent.mkdir(parents=True, exist_ok=True)
with (template.parent / f'{template.name}.lock').open('w') as lock:
# Only one instance makes the template. The others start without it and do not wait.
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
owner = True
except BlockingIOError:
owner = False
self._boot(timeout)
seed(self.url)
if owner and not (template / '.ready').exists():
self._save_template(template)
return self
def _save_template(self, template):
"""Stop, copy the data folder, and start again on the same port."""
self.stop()
shutil.rmtree(template, ignore_errors=True)
shutil.copytree(self.home / '.n8n', template / '.n8n', ignore=shutil.ignore_patterns('n8nEventLog*', '*.log'))
(template / '.ready').touch()
self._boot(60)
def _boot(self, timeout):
self.home.mkdir(parents=True, exist_ok=True)
log = self.log_path.open('a')
# A new session, so stop() can end n8n and its task runners together.
self.proc = subprocess.Popen(['node', str(self.root / 'packages' / 'cli' / 'bin' / 'n8n'), 'start'],
cwd=self.home, env=instance_env(self.port, self.home, self.feature_flags),
stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
deadline = time.time() + timeout
while time.time() < deadline:
if self.proc.poll() is not None:
raise RuntimeError(f'{self.name} instance stopped at start (exit {self.proc.returncode}), see {self.log_path}')
try:
with urllib.request.urlopen(f'{self.url}/healthz/readiness', timeout=2) as res:
if res.status == 200:
return
except OSError:
pass
time.sleep(0.25)
raise RuntimeError(f'{self.name} instance did not become ready in {timeout}s, see {self.log_path}')
def stop(self, clean=False):
"""Stop n8n. clean also deletes the static file cache (about 80 MB), which n8n writes again at each start."""
if self.proc and self.proc.poll() is None:
try:
os.killpg(self.proc.pid, signal.SIGTERM)
self.proc.wait(timeout=30)
except (ProcessLookupError, subprocess.TimeoutExpired):
try:
os.killpg(self.proc.pid, signal.SIGKILL)
except ProcessLookupError:
pass
if clean:
shutil.rmtree(self.home / '.cache', ignore_errors=True)
def client(base_url):
c = N8nClient()
c.base_url = base_url.rstrip('/')
return c
def seed(base_url):
"""Create the owner (N8N_EVAL_EMAIL and N8N_EVAL_PASSWORD) and finish the Assistant setup."""
c = client(base_url)
c.request('POST', '/rest/owner/setup',
{'email': c.email, 'firstName': 'Feature', 'lastName': 'Test', 'password': c.password})
c.login()
# Web search is optional. Without a choice, the Assistant shows its setup screen instead of the chat.
# A search provider from the environment makes the choice, and the API rejects searchDisabled then.
current = c.request('GET', '/rest/instance-ai/settings') or {}
update = {'enabled': True}
if not current.get('searchEnvConfigured'):
update['searchDisabled'] = True
c.request('PUT', '/rest/instance-ai/settings', update)
def git(*args, cwd=REPO_ROOT):
return subprocess.run(['git', *args], cwd=cwd, capture_output=True, text=True, check=True).stdout.strip()

View File

@ -0,0 +1,351 @@
// The helper object `t` that a scenario script receives. See SKILL.md for the full API.
//
// export default async function (t) {
// await t.step('Open the Assistant', () => t.goto());
// await t.step('Send a prompt', () => t.send('Reply with exactly: OK'));
// await t.step('Wait for the reply', () => t.waitForIdle());
// await t.check('The reply shows', () => t.expect(t.ai.assistantMessages().first()).toBeVisible());
// await t.capture('thread', { mask: [t.ai.assistantMessages()] });
// }
//
// A step that throws ends the scenario. A check that fails is recorded and the scenario goes on.
// Each step and check writes a screenshot. The runner shows them in the report.
import { mkdirSync, renameSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { expect } from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
const THREAD_URL = /\/assistant\/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/;
const RUN_GRACE_MS = 10_000;
/**
* Replace the accessibility subtree of each masked element with one line. Masked texts alone keep
* the structure, so a list with a different number of rows still shows as a change. Each subtree
* replaces only its first unmasked match, so an unmasked element with the same content stays.
*/
function maskSubtrees(tree, subtrees) {
const lines = tree.split('\n');
for (const subtree of subtrees) {
const sub = subtree.split('\n').filter((l) => l.trim());
if (!sub.length || sub[0].startsWith('#')) continue;
for (let i = 0; i + sub.length <= lines.length; i++) {
if (!lines[i].endsWith(sub[0])) continue;
const indent = lines[i].slice(0, lines[i].length - sub[0].length);
if (indent.trim() || !sub.every((l, j) => lines[i + j] === indent + l)) continue;
lines.splice(i, sub.length, `${indent}- text: "[masked]"`);
break;
}
}
return lines.join('\n');
}
/** The accessibility tree without the masked texts and the instance port, so branch and master compare. */
function maskTree(tree, texts) {
let out = tree.replace(/localhost(:|%3A)\d+/g, 'localhost$1PORT');
for (const text of [...texts].sort((a, b) => b.length - a.length))
out = out.split(text).join('[masked]');
return out;
}
export class StepFailed extends Error {}
/** Locators of the Instance AI page. They mirror pages/InstanceAiPage.ts in the Playwright tests. */
function locators(page) {
const container = () => page.getByTestId('instance-ai-container');
return {
container,
chatInput: () => container().getByRole('textbox').or(container().getByRole('combobox')),
composer: () => container().getByTestId('instance-ai-composer'),
sendButton: () => container().getByTestId('instance-ai-send-button'),
stopButton: () => container().getByTestId('instance-ai-stop-button'),
userMessages: () => container().getByTestId('instance-ai-user-message'),
assistantMessages: () => container().getByTestId('instance-ai-assistant-message'),
assistantText: (text) =>
container().getByTestId('instance-ai-assistant-message').getByText(text),
panelText: (text) => container().getByText(text),
emptyState: () => container().getByTestId('instance-ai-empty-state'),
statusBar: () => container().getByTestId('instance-ai-status-bar'),
sidebarToggle: () => container().getByTestId('instance-ai-sidebar-toggle'),
threadList: () => page.getByTestId('instance-ai-thread-list'),
newThreadButton: () =>
page
.getByTestId('project-instance-ai-menu-item')
.getByRole('menuitem', { name: 'Assistant', exact: true }),
mentionButton: () => container().getByTestId('instance-ai-mention-button'),
mentionMenu: () => page.getByTestId('instance-ai-mention-menu-content'),
fileInput: () => container().locator('input[type="file"]'),
approveButton: () => container().getByTestId('approval-card-allow-once'),
alwaysAllowButton: () => container().getByTestId('approval-card-always-allow'),
denyButton: () => container().getByTestId('approval-card-deny'),
domainAllowOnce: () => container().getByTestId('domain-access-allow-once'),
planApproveButton: () => container().getByTestId('instance-ai-plan-approve'),
credentialContinue: () => container().getByTestId('instance-ai-credential-continue-button'),
previewPanel: () => container().getByTestId('instance-ai-preview-panel'),
previewCanvasNodes: () =>
container().getByTestId('workflow-canvas-host').locator('[data-test-id="canvas-node"]'),
previewToggle: () => container().getByTestId('instance-ai-artifacts-preview-toggle'),
artifactsSidebar: () => container().getByTestId('instance-ai-artifacts-sidebar'),
outOfCredits: () => container().getByTestId('instance-ai-out-of-credits'),
};
}
export function createKit({ page, out, baseUrl, autoConfirm, emit }) {
const started = Date.now();
const steps = [];
const threads = [];
const prompts = [];
const ai = locators(page);
let stopped = false;
mkdirSync(join(out, 'captures'), { recursive: true });
const seconds = () => (Date.now() - started) / 1000;
function save() {
const partial = join(out, 'steps.tmp.json');
writeFileSync(partial, JSON.stringify(steps, null, 1));
renameSync(partial, join(out, 'steps.json'));
}
async function shot(file) {
try {
await page.screenshot({ path: join(out, file) });
return file;
} catch {
return null;
}
}
function message(error) {
return String(error?.message ?? error)
.replace(/\u001b\[[0-9;]*m/g, '')
.slice(0, 2000);
}
async function record(kind, title, fn) {
if (stopped) throw new StepFailed('The scenario has stopped.');
const entry = { i: steps.length + 1, kind, title, status: 'running', t: seconds() };
steps.push(entry);
save();
emit(`${kind} ${title}`);
let value;
let failed = false;
try {
value = typeof fn === 'function' ? await fn() : fn;
if (value === false) throw new Error('The check returned false.');
} catch (error) {
failed = true;
entry.error = message(error);
}
// After a stop, run_scenario.mjs owns the steps. A late result must not change them.
if (stopped) throw new StepFailed('The scenario has stopped.');
entry.status = failed ? 'failed' : 'passed';
if (failed) emit(`${kind} failed: ${entry.error.split('\n')[0]}`);
entry.duration = seconds() - entry.t;
entry.shot = await shot(`${kind}-${String(entry.i).padStart(2, '0')}.png`);
if (!stopped) save();
if (failed && kind === 'step') throw new StepFailed(entry.error);
return failed ? undefined : value;
}
async function api(method, path, body) {
return await page.evaluate(
async ({ method, path, body }) => {
const res = await fetch(path, {
method,
headers: {
'content-type': 'application/json',
'browser-id': localStorage.getItem('n8n-browserId') ?? '',
},
body: body === undefined ? undefined : JSON.stringify(body),
});
const text = await res.text();
if (!res.ok) throw new Error(`${method} ${path} -> ${res.status}: ${text.slice(0, 300)}`);
const json = text ? JSON.parse(text) : null;
return json && typeof json === 'object' && Object.keys(json).length === 1 && 'data' in json
? json.data
: json;
},
{ method, path, body },
);
}
function currentThread() {
return page.url().match(THREAD_URL)?.[1] ?? null;
}
function noteThread(id) {
if (id && !threads.includes(id)) {
threads.push(id);
emit(`thread ${id}`);
}
}
async function threadStatus(id) {
const s = await api('GET', `/rest/instance-ai/threads/${id}/status`);
const background = (s?.backgroundTasks ?? []).filter((x) => x.status === 'running').length;
return { active: Boolean(s?.hasActiveRun), suspended: Boolean(s?.isSuspended), background };
}
const t = {
page,
ai,
expect,
baseUrl,
threads,
prompts,
/** A step. When it throws, the scenario ends as failed. */
step: (title, fn) => record('step', title, fn),
/** A soft check. A failure is recorded and the scenario goes on. fn can return false to fail. */
check: (title, fn) => record('check', title, fn),
/** A line in the step list, with a screenshot. */
note: (title) => record('note', title, () => true),
/** Open a page of the instance. Default: the Assistant, with the composer ready. */
async goto(path = '/assistant') {
await page.goto(baseUrl + path, { waitUntil: 'domcontentloaded' });
if (path.startsWith('/assistant')) {
const enable = page.getByRole('dialog').filter({ hasText: 'Try new n8n Assistant' });
if (await enable.isVisible({ timeout: 1500 }).catch(() => false)) {
await enable
.getByRole('button', { name: /Enable n8n Assistant on this instance/ })
.click();
await enable.getByRole('button', { name: /^(Continue|Enable)$/ }).click();
}
await expect(ai.chatInput()).toBeVisible({ timeout: 30_000 });
}
noteThread(currentThread());
},
/** Type a message in the composer and send it. Returns the thread id. */
async send(text) {
const before = await ai.assistantMessages().count();
await expect(ai.chatInput()).toBeVisible({ timeout: 30_000 });
await ai.chatInput().fill(text);
await expect(ai.sendButton()).toBeEnabled({ timeout: 30_000 });
await ai.sendButton().click();
await page.waitForURL(THREAD_URL, { timeout: 30_000 });
const id = currentThread();
noteThread(id);
prompts.push({
t: seconds(),
text,
thread: id,
assistantMessagesBefore: before,
sentAt: Date.now(),
});
return id;
},
/**
* Wait until the thread has no active run, no open confirmation, and no running background task.
* Returns 'idle', or 'suspended' when a confirmation card waits for the scenario
* (autoConfirm is off, or the card stays open for 20 seconds).
*/
async waitForIdle({ timeout = 600_000, settle = 3000 } = {}) {
const id = currentThread() ?? threads.at(-1);
if (!id) throw new Error('No thread is open. Call t.send() first.');
const last = prompts.filter((p) => p.thread === id).at(-1);
const deadline = Date.now() + timeout;
let seenRun = false;
let idleSince = null;
let suspendedSince = null;
while (!stopped && Date.now() < deadline) {
let s;
try {
s = await threadStatus(id);
} catch {
await page.waitForTimeout(1000);
continue;
}
// The status can show no run for a moment after the message is sent. A run that the user
// stops at once can end before it shows, so after a grace period "no run" means "done".
if (!seenRun && Date.now() - (last?.sentAt ?? 0) > RUN_GRACE_MS) {
seenRun = true;
}
if (!seenRun) {
const replies = await ai
.assistantMessages()
.count()
.catch(() => 0);
seenRun =
s.active ||
s.suspended ||
s.background > 0 ||
replies > (last?.assistantMessagesBefore ?? 0);
}
if (s.suspended) {
suspendedSince ??= Date.now();
if (!autoConfirm || Date.now() - suspendedSince > 20_000) return 'suspended';
} else {
suspendedSince = null;
}
const idle = seenRun && !s.active && !s.suspended && s.background === 0;
if (!idle) {
idleSince = null;
} else {
idleSince ??= Date.now();
if (Date.now() - idleSince >= settle) {
await ai
.sendButton()
.waitFor({ state: 'visible', timeout: 10_000 })
.catch(() => {});
return 'idle';
}
}
await page.waitForTimeout(1000);
}
throw new Error(`The thread did not go idle in ${Math.round(timeout / 1000)}s.`);
},
/**
* Save a screen for the visual check. Visual scenarios run on this branch and on master, and the
* report compares the screens with the same name. mask hides changing parts, such as LLM text.
* root is the part of the page for the accessibility tree diff (default: the whole page).
*/
async capture(name, { mask = [], maskText = [], root, fullPage = false } = {}) {
if (!/^[a-z0-9][a-z0-9-]*$/.test(name))
throw new Error(`Capture name "${name}": use a-z, 0-9 and "-".`);
return await record('capture', `Capture ${name}`, async () => {
await page.waitForLoadState('networkidle', { timeout: 5000 }).catch(() => {});
await page.waitForTimeout(500);
const texts = new Set(maskText.filter(Boolean));
for (const locator of mask) {
for (const inner of await locator.allInnerTexts().catch(() => [])) {
for (const line of inner.split('\n'))
if (line.trim().length > 1) texts.add(line.trim());
}
}
const textMasks = maskText.filter(Boolean).map((s) => page.getByText(s, { exact: true }));
await page.screenshot({
path: join(out, 'captures', `${name}.png`),
mask: [...mask, ...textMasks],
maskColor: '#FF00FF',
animations: 'disabled',
caret: 'hide',
fullPage,
});
const tree = await (root ?? page.locator('body'))
.ariaSnapshot({ timeout: 10_000 })
.catch((e) => `# ${message(e)}`);
const subtrees = [];
for (const locator of mask) {
for (const element of await locator.all().catch(() => [])) {
subtrees.push(await element.ariaSnapshot({ timeout: 5_000 }).catch(() => ''));
}
}
writeFileSync(
join(out, 'captures', `${name}.aria.yml`),
maskTree(maskSubtrees(tree, subtrees), texts) + '\n',
);
});
},
/** A REST call as the signed-in user. Returns the `data` field of the answer when it has one. */
api,
threadStatus,
sleep: (ms) => page.waitForTimeout(ms),
};
/** End the scenario: later steps throw, and a running step does not change the steps. */
const stop = () => {
stopped = true;
};
return { t, steps, threads, prompts, stop };
}

View File

@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Serve the live page of a feature test run, then its report.
live_page.py --run-dir <run-dir> [--parent-pid <pid>] [--port <port>]
run.py starts it in the background. It uses the server of testmax-n8n-ai
(live_server.py): 127.0.0.1 only, the address in <run-dir>/live-url.txt, and it stops
30 minutes after the last request when the run is done. When verdicts.json changes after the
run, the page builds the report again, and the open report reloads.
"""
import argparse
import threading
import time
from pathlib import Path
import common # noqa: F401 (puts the shared scripts on the import path)
import report
from live_server import read_status, serve
from report import ScenarioApp
def rebuild_on_verdicts(run_dir):
verdicts, built = run_dir / 'verdicts.json', None
while True:
time.sleep(2)
if read_status(run_dir).get('phase') != 'done' or not verdicts.exists():
continue
mtime = verdicts.stat().st_mtime
if mtime == built:
continue
try:
report.build(run_dir)
built = mtime
except ValueError:
# The judge is still writing the file. The next check reads it again.
pass
except Exception as e:
print(f'cannot build the report: {e}', flush=True)
built = mtime
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--run-dir', required=True)
ap.add_argument('--parent-pid', type=int)
ap.add_argument('--port', type=int, default=0)
args = ap.parse_args()
run_dir = Path(args.run_dir).resolve()
threading.Thread(target=rebuild_on_verdicts, args=(run_dir,), daemon=True).start()
serve(ScenarioApp(run_dir), args.parent_pid, args.port)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,690 @@
#!/usr/bin/env python3
"""Build the report of a feature test run, and the live page while it runs.
report.py <run-dir> build <run-dir>/report/index.html (run it again after verdicts.json)
report.py <run-dir> --summary print the summary for the judge
Inputs in <run-dir>:
plan/scenarios.json the plan (run.py copies it)
status.json the status of each run (run.py writes it)
runs/<name>/ the files of one run (run_scenario.mjs and run.py write them)
visual.json the visual check (visual_diff.mjs and run.py write it)
verdicts.json optional: {"<name>": {"verdict": "pass|partial|fail", "summary": "...",
"checks": [{"ok": true|false|null, "text": "..."}]}}
The page uses the template and the LLM steps view of testmax-n8n-ai.
"""
import fcntl
import json
import shutil
import sys
import time
from datetime import datetime
from pathlib import Path
import common # noqa: F401 (puts the shared scripts on the import path)
import build_report as br
import live_server as ls
STATUS = {'waiting': ('st-waiting', 'Waiting'), 'starting': ('st-waiting', 'Starting'), 'running': ('st-running', 'Running'),
'passed': ('st-done', 'Passed'), 'failed': ('st-failed', 'Failed'), 'error': ('st-failed', 'Error'),
'skipped': ('st-skipped', 'Skipped')}
FINISHED = ('passed', 'failed', 'error', 'skipped')
KIND_LABELS = {'feature': 'Feature', 'regression': 'Regression', 'visual': 'Visual'}
VISUAL = {'changed': ('vis-changed', 'Changed'), 'unchanged': ('vis-same', 'Unchanged'),
'branch-only': ('vis-only', 'Only on branch'), 'master-only': ('vis-only', 'Only on master')}
THEAD = ('<tr><th>Scenario</th><th>Kind</th><th>Result</th><th>Steps</th><th>Time</th><th>LLM calls</th>'
'<th>Tool calls</th><th>Input tokens</th><th>Output tokens</th><th>Cost</th><th>Browser errors</th></tr>')
CSS = """<style>
.run-video video{display:block;width:100%;max-height:70vh;aspect-ratio:16/10;object-fit:contain;background:var(--black);border-radius:var(--r8)}
.badge.vis-changed{background:var(--pink-70);color:var(--bg)}
.badge.vis-same{background:var(--bg-3);color:var(--black)}
.badge.vis-only{box-shadow:inset 0 0 0 1px var(--line-strong);color:var(--dim)}
.facts{display:grid;grid-template-columns:max-content minmax(0,1fr);gap:var(--s4) var(--s16);margin:0}
.facts dt{font:10px/18px var(--mono);text-transform:uppercase;letter-spacing:.02em;color:var(--dim)}
.facts dd{margin:0;word-break:break-word}
.run-error{margin:var(--s12) 0 0;background:var(--pink-10);color:var(--pink-70);border-radius:var(--r8);padding:var(--s12);white-space:pre-wrap;word-break:break-word;font:12px/16px var(--mono);max-height:240px;overflow:auto}
.tsteps{list-style:none;margin:0;padding:0;display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--s16)}
@media (max-width:1000px){.tsteps{grid-template-columns:repeat(2,minmax(0,1fr))}}
@media (max-width:640px){.tsteps{grid-template-columns:minmax(0,1fr)}}
.tstep{display:flex;flex-direction:column;min-width:0;background:var(--white);border:1px solid var(--line);border-radius:var(--r8);overflow:hidden}
.tstep.failed{background:var(--pink-10);border-color:var(--pink-70);box-shadow:0 0 0 1px var(--pink-70)}
.tstep-shot{display:block;width:100%;aspect-ratio:16/10;padding:0;border:0;border-bottom:1px solid var(--line);background:var(--bg-3);cursor:zoom-in;overflow:hidden}
.tstep-shot img{display:block;width:100%;height:100%;object-fit:cover;object-position:top center;transition:transform .15s}
.tstep-shot:hover img{transform:scale(1.03)}
.tstep-shot:focus-visible{outline:2px solid var(--pink);outline-offset:-2px}
@media (prefers-reduced-motion:reduce){.tstep-shot img{transition:none}}
.tstep-noshot{display:grid;place-items:center;aspect-ratio:16/10;background:var(--bg-2);border-bottom:1px solid var(--line);color:var(--gray-50);font:10px/14px var(--mono);text-transform:uppercase;letter-spacing:.02em}
.tstep-info{display:flex;gap:var(--s12);align-items:flex-start;padding:var(--s12);min-width:0}
.tstep-info > div{flex:1;min-width:0;display:flex;flex-direction:column;gap:var(--s4)}
.tstep-info.running .num{background:var(--pink-10);color:var(--pink-70)}
.tstep-info.passed .num{background:var(--pink)}
.tstep-info.failed .num{background:var(--pink-70);color:var(--bg)}
.tstep-title{font-weight:500;padding-top:3px;word-break:break-word}
.tstep-title .mono-label{font-weight:400;margin-right:var(--s8)}
.tstep-info pre{margin:var(--s4) 0 0;white-space:pre-wrap;word-break:break-word;font:12px/16px var(--mono);color:var(--pink-70);max-height:200px;overflow:auto}
.shot-viewer .theater-bar{flex-wrap:nowrap;align-items:flex-start}
.shot-viewer-info{flex:1;min-width:0;padding-top:3px}
.shot-viewer-actions{flex:none;display:flex;align-items:center;gap:var(--s16)}
.shot-viewer-info .tstep-info{padding:0}
.shot-viewer-info .tstep-title{font-size:18px;line-height:22px}
.shot-viewer-img{flex:1;min-height:0;display:flex;align-items:center;justify-content:center;cursor:zoom-out}
.shot-viewer-img img{display:block;max-width:100%;max-height:100%;border-radius:var(--r8);box-shadow:0 0 0 1px var(--line);cursor:default}
.err-list{margin:0;padding-left:var(--s16);font:12px/16px var(--mono)}
.err-list li{margin-bottom:var(--s4);word-break:break-word}
.vis{background:var(--white);border-radius:var(--r8);padding:var(--s12);box-shadow:inset 0 0 0 1px var(--line);margin-top:var(--s12)}
.vis.changed{box-shadow:inset 4px 0 0 var(--pink-70),inset 0 0 0 1px var(--line)}
.vis-grid{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:var(--s12)}
.vis-grid figure{margin:0}
.vis-grid figcaption{margin-bottom:var(--s4)}
.vis-grid img{display:block;width:100%;height:auto;border-radius:var(--r4);box-shadow:0 0 0 1px var(--line)}
.vis details summary{cursor:pointer;color:var(--dim);margin-top:var(--s8);font:10px/14px var(--mono);text-transform:uppercase;letter-spacing:.02em}
.vis pre{background:var(--bg);border-radius:var(--r4);padding:var(--s12);margin:var(--s8) 0 0;overflow:auto;max-height:320px;font-size:12px;line-height:16px;white-space:pre}
</style>"""
def read_json(path, default):
try:
return json.loads(path.read_text())
except (OSError, ValueError):
return default
def seconds(value):
return f'{value:.0f}s'
def load_trace(sub, result):
"""LLM steps from the thread events, and per-call tokens from usage.json when LangSmith had them."""
data = ls.load_instance_ai_live(sub)
events_file = sub / 'events.jsonl'
rows = br.read_jsonl(events_file) if events_file.exists() else []
starts = [r['ts'] / 1000 for r in rows if r.get('type') == 'run-start' and r.get('ts')]
usage = read_json(sub / 'usage.json', None)
if usage and starts:
start = starts[0]
data['calls'] = sorted((
{'t': br.ts(u['start']) - start, 'start': br.ts(u['start']) - start,
'end': br.ts(u['end']) - start if u.get('end') else None,
'name': (u.get('name') or '').removeprefix('llm: ') or None,
'prompt': u['in'], 'cache_read': u['cache_read'], 'cache_write': u['cache_write'],
'out': u['out'], 'cost': u['cost']} for u in usage), key=lambda c: c['t'])
data['model'] = ', '.join(sorted({u['model'] for u in usage if u.get('model')})) or None
data['cost'] = f'${sum(c["cost"] for c in data["calls"]):.2f}'
data['costSource'] = 'langsmith'
if result.get('startedAt') and result.get('finishedAt'):
data['duration'] = seconds(br.ts(result['finishedAt']) - br.ts(result['startedAt']))
return data
def load(run_dir):
"""The plan, the status, and one dict for each run in plan order."""
plan = read_json(run_dir / 'plan' / 'scenarios.json', {'scenarios': []})
status = ls.read_status(run_dir)
entries = status.get('runs') or {}
runs = []
for s in plan['scenarios']:
for name in (s['id'], f'{s["id"]}--master'):
if name not in entries:
continue
entry = entries[name]
sub = run_dir / 'runs' / name
result = read_json(sub / 'result.json', {})
data = load_trace(sub, result)
if entry.get('status') in ('waiting', 'running'):
data['cost'] = '-'
if entry.get('startedAt'):
data['duration'] = seconds((entry.get('finishedAt') or time.time()) - entry['startedAt'])
runs.append({
'name': name, 'scenario': s, 'entry': entry, 'master': name.endswith('--master'),
'steps': read_json(sub / 'steps.json', []), 'result': result,
'console': read_json(sub / 'console.json', {}), 'meta': read_json(sub / 'run.json', {}),
'data': data,
})
return plan, status, runs
def status_badge(state):
cls, label = STATUS.get(state, ('st-waiting', state))
return f'<span class="badge {cls}">{br.esc(label)}</span>'
def run_title(run):
return run['scenario']['title'] + (' (master)' if run['master'] else '')
def check_count(steps):
checked = [s for s in steps if s['kind'] in ('step', 'check', 'capture')]
return f'{sum(s["status"] == "passed" for s in checked)}/{len(checked)}' if checked else '-'
def row_cells(run, result_html):
data, result = run['data'], run['result']
calls, tools = data['calls'], [e for e in data['events'] if e['kind'] == 'tool']
errors = sum(e['error'] for e in tools)
browser = (result.get('consoleErrors') or 0) + (result.get('requestErrors') or 0)
cells = [
f'<span class="run-name">{br.esc(run_title(run))}</span>',
KIND_LABELS.get(run['scenario']['kind'], run['scenario']['kind']),
result_html, check_count(run['steps']), data['duration'], str(len(calls)) if calls else '-',
f'{len(tools)}' + (f' <span class="err">({errors} err)</span>' if errors else ''),
br.kilo(sum(c['prompt'] for c in calls)) if calls else '-',
br.kilo(sum(c['out'] for c in calls)) if calls else '-',
f'<strong>{br.esc(data["cost"])}</strong>',
f'<span class="err">{browser}</span>' if browser else '0',
]
return ''.join(f'<td>{c}</td>' for c in cells)
def result_html(run, verdict):
"""The judge's verdict. The script status shows only while it runs or when it fails, because the Steps
column already counts the steps that passed."""
if verdict and verdict.get('verdict'):
return br.badge(verdict['verdict'])
state = run['entry'].get('status', 'waiting')
if state != 'passed':
return status_badge(state)
if run['master']:
return '<span class="badge st-done">Baseline</span>'
return br.badge(None)
def render_video(screen):
"""The video of the run, above the expected outcome. Only the built report has it."""
if not screen:
return ''
poster = f' poster="{br.esc(screen["poster"])}"' if screen.get('poster') else ''
return br.fold('Video', f'<span class="mono-label">n8n UI · {br.esc(screen["duration"])}</span>',
f'<div class="run-video"><video controls muted playsinline preload="metadata" '
f'src="{br.esc(screen["src"])}"{poster}></video></div>', cls='brief-card prompt')
def render_expected(run, verdict):
head = br.verdict_head(verdict) if verdict else ''
body = f'<div class="msg">{br.esc(run["scenario"]["expected"])}</div>'
if verdict:
body += f'<div class="verdict-body">{br.render_checks(verdict)}</div>'
elif run['entry'].get('status') in FINISHED:
body += '<p class="dim">Not judged yet.</p>'
return br.fold('Expected outcome', head, body)
def render_run_box(run):
entry, result, meta = run['entry'], run['result'], run['meta']
facts = [('Instance', f'{"master" if run["master"] else "branch"} · {entry.get("baseUrl") or "-"}')]
if entry.get('startedAt'):
facts.append(('Started', datetime.fromtimestamp(entry['startedAt']).strftime('%H:%M:%S')))
if result.get('threads'):
facts.append(('Threads', ', '.join(result['threads'])))
if meta.get('cached'):
facts.append(('Reused', 'This master run is from an earlier run of the same master commit and script.'))
if meta.get('decisions'):
facts.append(('Confirmations', f'{len(meta["decisions"])} answered by the auto-approve policy'))
if result:
facts.append(('Browser errors', f'{result.get("consoleErrors", 0)} console, {result.get("requestErrors", 0)} requests'))
dl = ''.join(f'<dt>{br.esc(k)}</dt><dd>{br.esc(v)}</dd>' for k, v in facts)
error = result.get('error') or (entry.get('reason') if entry.get('status') in ('error', 'skipped') else None)
error_html = f'<pre class="run-error">{br.esc(error)}</pre>' if error else ''
return br.fold('Run', status_badge(entry.get('status', 'waiting')), f'<dl class="facts">{dl}</dl>{error_html}')
def render_scenario_steps(steps, files):
if not steps:
return br.fold('Scenario steps', '', '<p class="dim">No steps yet.</p>')
items = []
for s in steps:
status, title = br.esc(s['status']), br.esc(s['title'])
duration = f' · took {s["duration"]:.1f}s' if s.get('duration') is not None else ''
if s.get('shot'):
src = f'{files}{br.esc(s["shot"])}'
thumb = (f'<button type="button" class="tstep-shot" data-src="{src}" aria-label="Show step {s["i"]} '
f'full screen"><img src="{src}" loading="lazy" alt="Screen after: {title}"></button>')
else:
thumb = '<div class="tstep-noshot">No screenshot</div>'
error = f'<pre>{br.esc(s["error"])}</pre>' if s.get('error') else ''
items.append(
f'<li class="tstep {status}">{thumb}<div class="tstep-info {status}"><span class="num">{s["i"]}</span><div>'
f'<div class="tstep-title"><span class="mono-label">{br.esc(s["kind"])}</span><span>{title}</span></div>'
f'<div class="mono-label">at {s["t"]:.1f}s{duration}</div>{error}</div></div></li>')
passed = sum(s['status'] == 'passed' for s in steps)
return br.fold('Scenario steps', f'<span class="mono-label">{passed} of {len(steps)} passed</span>',
f'<ol class="tsteps">{"".join(items)}</ol>')
def render_errors(console):
console_errors, request_errors = console.get('consoleErrors') or [], console.get('requestErrors') or []
if not console_errors and not request_errors:
return ''
items = [f'<li>{br.esc(e.get("text", ""))}</li>' for e in console_errors[:50]]
items += [f'<li>{e.get("status") or "failed"} {br.esc(e.get("method", ""))} {br.esc(e.get("url", ""))}'
f'{" · " + br.esc(e["error"]) if e.get("error") else ""}</li>' for e in request_errors[:50]]
return br.fold('Browser errors', f'<span class="mono-label">{len(console_errors)} console · '
f'{len(request_errors)} requests</span>', f'<ul class="err-list">{"".join(items)}</ul>')
def render_panel(run, verdict, files, scale, screen=None):
data = run['data']
prompt = (run['result'].get('prompts') or [{}])[0].get('text', '')
load_error = f'<div class="box err">Could not read the trace: {br.esc(data["loadError"])}</div>' if data.get('loadError') else ''
return (f'{load_error}{render_video(screen)}'
f'{render_expected(run, verdict)}{render_run_box(run)}'
f'{render_scenario_steps(run["steps"], files)}'
f'{br.render_steps(data["calls"], data["events"], *scale, prompt)}'
f'{render_errors(run["console"])}')
def render_case(plan, status):
branch, master = status.get('branch') or {}, status.get('master')
kinds = [s['kind'] for s in plan['scenarios']]
lines = [f'Branch {branch.get("commit") or "?"}' + (' with local changes' if branch.get('dirty') else '')
+ (f' on {branch["url"]}' if branch.get('url') else ' on a fresh instance')]
if master is not None:
lines.append(f'Master {(master.get("commit") or "(building)")[:11]} (the merge base) on a fresh instance')
lines.append(f'{len(kinds)} scenarios: ' + ', '.join(f'{kinds.count(k)} {k}' for k in KIND_LABELS if kinds.count(k)))
lines.append(f'{status.get("concurrency", 1)} at a time')
return (f'<div class="case"><div class="box"><div class="mono-label">Feature</div><div class="msg">'
f'{br.esc(plan.get("feature") or "-")}</div></div><div class="box"><div class="mono-label">Run</div>'
f'<div class="msg">{br.esc(chr(10).join(lines))}</div></div></div>')
def notes(status, runs, verdicts):
parts = []
if not verdicts and status.get('phase') == 'done':
parts.append('The runs are not judged yet. This page reloads when the verdicts and the token usage are ready.')
parts.append('Times are seconds from the start of each scenario. Several scenarios ran at the same time on one '
'machine, so their times include the shared load. Tokens and cost come from LangSmith, matched by the '
'traces of the threads of each scenario. An asterisk on a time marks text placed at the previous tool call.')
if any(r['master'] for r in runs):
parts.append('Visual scenarios ran on a fresh branch instance and on a fresh master instance, so both start '
'from the same empty state. Master is the merge base of the branch.')
if status.get('stale'):
parts.append('Warning: these packages had source changes newer than their build, so the branch instance ran '
f'an old build: {", ".join(status["stale"])}.')
return br.esc(' '.join(parts))
def render_visual(results, runs, files):
if not results:
return ''
titles = {r['scenario']['id']: r['scenario']['title'] for r in runs}
cards = []
for r in results:
cls, label = VISUAL.get(r['status'], ('vis-only', r['status']))
ratio = f'{100 * r["ratio"]:.2f}% of pixels' if r.get('ratio') is not None else ''
sizes = (f' · size {r["masterSize"][0]}×{r["masterSize"][1]} → {r["branchSize"][0]}×{r["branchSize"][1]}'
if r.get('branchSize') and r['branchSize'] != r.get('masterSize') else '')
figures = ''
for caption, key in (('Master', 'master'), ('Branch', 'branch'), ('Difference', 'diff')):
src = files(r, key)
figures += (f'<figure><figcaption class="mono-label">{caption}</figcaption>'
+ (f'<a href="{src}" target="_blank"><img src="{src}" loading="lazy" alt="{caption}: {br.esc(r["name"])}"></a>'
if src else '<p class="dim">No screen.</p>') + '</figure>')
tree = (f'<details><summary>Accessibility tree diff</summary><pre>{br.esc(r["ariaDiff"])}</pre></details>'
if r.get('ariaDiff') else '')
cards.append(
f'<div class="vis {"changed" if r["status"] != "unchanged" else ""}"><div class="head">'
f'<h3>{br.esc(titles.get(r["scenario"], r["scenario"]))} › {br.esc(r["name"])}</h3>'
f'<span class="mono-label">{ratio}{sizes}</span><span class="badge {cls}">{label}</span></div>'
f'<div class="vis-grid">{figures}</div>{tree}</div>')
changed = sum(r['status'] != 'unchanged' for r in results)
return br.fold('Visual check against master',
f'<span class="mono-label">{len(results)} captures · {changed} changed</span>',
'<p class="fineprint">Pink pixels in the difference changed by more than a small color threshold. '
'Magenta boxes are masked parts, such as LLM text. The accessibility tree diff shows changes in '
'structure, roles and labels.</p>' + ''.join(cards))
STEP_VIEWER_JS = """
// A step screenshot opens full screen. The listeners are delegated, because the live page replaces the panels.
(()=>{
const viewer = document.createElement('div');
viewer.className = 'theater shot-viewer';
viewer.hidden = true;
viewer.setAttribute('role', 'dialog');
viewer.setAttribute('aria-modal', 'true');
viewer.setAttribute('aria-label', 'Step screenshot');
viewer.innerHTML = '<div class="theater-bar"><div class="shot-viewer-info"></div><div class="shot-viewer-actions">'
+ '<span class="mono-label" data-shot-count></span>'
+ '<button type="button" class="btn" data-shot-move="-1">Previous</button>'
+ '<button type="button" class="btn" data-shot-move="1">Next</button>'
+ '<button type="button" class="btn accent" data-shot-close>Close</button></div></div>'
+ '<div class="shot-viewer-img"><img alt=""></div>'
+ '<p class="fineprint">Press the left or right arrow key to change the step. Press Esc to close.</p>';
document.body.append(viewer);
let run = null, index = 0, opener = null;
const shots = () => [...(document.getElementById(run)?.querySelectorAll('.tstep-shot') || [])];
function render(){
const items = shots();
if (!items.length) return close();
index = Math.min(index, items.length - 1);
const button = items[index], img = viewer.querySelector('.shot-viewer-img img');
img.src = button.dataset.src;
img.alt = button.querySelector('img').alt;
viewer.querySelector('.shot-viewer-info').replaceChildren(button.closest('.tstep').querySelector('.tstep-info').cloneNode(true));
viewer.querySelector('[data-shot-count]').textContent = `${index + 1} of ${items.length}`;
}
function open(button){
run = button.closest('.run').id;
index = Math.max(0, shots().indexOf(button));
opener = button;
viewer.hidden = false;
document.body.classList.add('theater-open');
render();
viewer.querySelector('[data-shot-close]').focus();
}
function close(){
viewer.hidden = true;
run = null;
document.body.classList.remove('theater-open');
if (opener?.isConnected) opener.focus();
}
function move(step){
const count = shots().length;
if (!count) return close();
index = (index + step + count) % count;
render();
}
document.addEventListener('click', e=>{
const button = e.target.closest('.tstep-shot');
if (button) open(button);
});
viewer.addEventListener('click', e=>{
const step = e.target.closest('[data-shot-move]');
if (step) move(Number(step.dataset.shotMove));
else if (e.target.closest('[data-shot-close]') || e.target.classList.contains('shot-viewer-img')) close();
});
document.addEventListener('keydown', e=>{
if (viewer.hidden) return;
if (e.key === 'Escape') close();
else if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') { e.preventDefault(); move(e.key === 'ArrowRight' ? 1 : -1); }
});
})();
"""
VIDEO_JS = """
// The video of a run plays only while its panel shows.
const showPanel = show;
show = (id)=>{
showPanel(id);
document.querySelectorAll('.run:not(.active) .run-video video').forEach(v=>v.pause());
};
"""
RELOAD_JS = """
// Served by the live page: reload when the report is built again, for example after the verdicts.
(()=>{
if (!location.protocol.startsWith('http')) return;
const saved = JSON.parse(sessionStorage.getItem('report-view') || 'null');
if (saved) {
sessionStorage.removeItem('report-view');
if (document.getElementById(saved.run)) show(saved.run);
window.scrollTo(0, saved.y);
}
setInterval(async ()=>{
try {
const version = (await (await fetch('version.txt', {cache: 'no-store'})).text()).trim();
if (version && version !== REPORT_VERSION) {
const run = document.querySelector('.run.active')?.id;
sessionStorage.setItem('report-view', JSON.stringify({run, y: window.scrollY}));
location.reload();
}
} catch {}
}, 3000);
})();
"""
def build(run_dir):
"""Write report/index.html with its images and videos. Returns its path. Two builds of one run can
happen at the same time (run.py, report.py, the live page), so they take turns."""
run_dir.mkdir(parents=True, exist_ok=True)
with (run_dir / 'report.lock').open('w') as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
return _build(run_dir)
def _build(run_dir):
plan, status, runs = load(run_dir)
verdicts = br.load_verdicts(run_dir)
report = run_dir / 'report'
for sub in ('runs', 'screens', 'visual'):
shutil.rmtree(report / sub, ignore_errors=True)
report.mkdir(parents=True, exist_ok=True)
for sub in ('fonts', 'brand'):
if (br.ASSET_CACHE / sub).exists():
shutil.copytree(br.ASSET_CACHE / sub, report / sub, dirs_exist_ok=True)
scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
rows, panels, screens = [], [], []
for i, run in enumerate(runs):
name, sub = run['name'], run_dir / 'runs' / run['name']
shots = report / 'runs' / name
for png in sub.glob('*.png'):
if png.name not in ('live.png', 'screen-poster.png'):
shots.mkdir(parents=True, exist_ok=True)
shutil.copy(png, shots / png.name)
verdict = verdicts.get(name)
screen = None
if (sub / 'screen.webm').exists():
(report / 'screens').mkdir(exist_ok=True)
shutil.copy(sub / 'screen.webm', report / 'screens' / f'{name}.webm')
screen = {'name': name, 'title': run_title(run), 'duration': run['data']['duration'], 'type': 'video',
'src': f'screens/{name}.webm', 'verdict': (verdict or {}).get('verdict'),
'label': KIND_LABELS.get(run['scenario']['kind'], '')}
if (sub / 'screen-poster.png').exists():
shutil.copy(sub / 'screen-poster.png', report / 'screens' / f'{name}-poster.png')
screen['poster'] = f'screens/{name}-poster.png'
screens.append(screen)
active = ' active' if i == 0 else ''
rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">'
+ row_cells(run, result_html(run, verdict)) + '</tr>')
panels.append(f'<section class="run{active}" id="{name}">'
+ render_panel(run, verdict, f'runs/{name}/', scale, screen) + '</section>')
results = read_json(run_dir / 'visual.json', [])
for r in results:
target = report / 'visual' / r['scenario']
target.mkdir(parents=True, exist_ok=True)
for key, source in (('branch', run_dir / 'runs' / r['scenario'] / 'captures' / f'{r["name"]}.png'),
('master', run_dir / 'runs' / f'{r["scenario"]}--master' / 'captures' / f'{r["name"]}.png'),
('diff', run_dir / (r.get('diff') or 'missing'))):
if source.is_file():
shutil.copy(source, target / f'{r["name"]}.{key}.png')
def visual_file(r, key):
return (f'visual/{r["scenario"]}/{r["name"]}.{key}.png'
if (report / 'visual' / r['scenario'] / f'{r["name"]}.{key}.png').exists() else None)
replay = br.render_replay(screens, 'Each video shows one scenario in the n8n UI. Press Esc to close.')
version = str(time.time_ns())
page = br.fill_template(
br.esc(plan.get('title') or 'Instance AI feature test'), ''.join(rows), notes(status, runs, verdicts),
''.join(panels), replay=replay, after_table=render_visual(results, runs, visual_file), head=CSS,
scripts=(f'<script>const SCREENS = {json.dumps(screens)}; const REPORT_VERSION = "{version}";</script>'
f'<script>window.addEventListener("DOMContentLoaded", ()=>{{{VIDEO_JS}{RELOAD_JS}}});</script>'
f'<script>{STEP_VIEWER_JS}</script>'),
case=render_case(plan, status), thead=THEAD, kicker='Assistant UI test')
(report / 'index.html').write_text(page)
(report / 'version.txt').write_text(version)
return report / 'index.html'
def clock(status):
started = status.get('startedAt')
elapsed = seconds((status.get('finishedAt') or time.time()) - started) if started else ''
return ' · '.join(x for x in (f'{status.get("concurrency", 1)} at a time', elapsed, status.get('step')) if x)
def live_box(status, runs):
cards = []
for run in runs:
name, title = run['name'], br.esc(run_title(run))
cards.append(
f'<div class="live-card"><div class="live-screen"><button type="button" class="live-zoom" '
f'data-zoom="{name}" aria-label="Show {title} full screen"><img data-shot="{name}" '
f'alt="{title} in the n8n UI" hidden></button>'
f'<div class="live-empty" data-shot-empty="{name}">Waiting for the first screenshot</div></div>'
f'<div class="reel-meta"><span class="run-name">{title}</span>'
f'<span data-live-badge="{name}">{status_badge(run["entry"].get("status", "waiting"))}</span></div></div>')
return (f'<div class="box"><div class="head"><h2>Live</h2><span class="mono-label" id="live-clock">'
f'{br.esc(clock(status))}</span></div><div class="live-grid">{"".join(cards)}</div></div>')
def render_live(run_dir):
plan, status, runs = load(run_dir)
verdicts = br.load_verdicts(run_dir)
scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
rows, panels = [], []
for i, run in enumerate(runs):
name, active = run['name'], ' active' if i == 0 else ''
rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">'
+ row_cells(run, result_html(run, verdicts.get(name))) + '</tr>')
panels.append(f'<section class="run{active}" id="{name}">'
+ render_panel(run, verdicts.get(name), f'/files/runs/{name}/', scale) + '</section>')
text = ('This page updates while the scenarios run, and changes to the full report when the run is done. '
'Tokens, cost, the visual check and verdicts come after the run.')
return br.fill_template(
br.esc(plan.get('title') or 'Instance AI feature test'), ''.join(rows), br.esc(text), ''.join(panels),
replay=live_box(status, runs), head=CSS,
scripts=f'<script>const SCREENS = [];</script><script>{ls.LIVE_JS}</script><script>{STEP_VIEWER_JS}</script>',
case=render_case(plan, status), thead=THEAD, kicker='Assistant UI test')
class ScenarioApp:
"""The live page for live_server.serve(): one card, table row and panel for each scenario run."""
def __init__(self, run_dir):
self.run_dir = run_dir
def entry(self, name):
return ls.read_status(self.run_dir).get('runs', {}).get(name)
def finished(self, name):
return (self.entry(name) or {}).get('status') in FINISHED
def done(self):
return ls.read_status(self.run_dir).get('phase') == 'done' and (self.run_dir / 'report' / 'index.html').exists()
def page(self):
return render_live(self.run_dir)
def state(self):
plan, status, runs = load(self.run_dir)
verdicts = br.load_verdicts(self.run_dir)
scale = br.token_scale([(r['name'], {}, r['data']) for r in runs])
state = {'phase': status.get('phase', 'running'), 'clock': clock(status), 'runs': {}}
if self.done():
state['report'] = '/report/index.html'
for run in runs:
name = run['name']
shot = self.shot(name)
state['runs'][name] = {
'row': row_cells(run, result_html(run, verdicts.get(name))),
'panel': render_panel(run, verdicts.get(name), f'/files/runs/{name}/', scale),
'badge': status_badge(run['entry'].get('status', 'waiting')),
'shot': shot.stat().st_mtime if shot.exists() else None,
}
return state
def shot(self, name):
return self.run_dir / 'runs' / name / 'live.png'
def casts(self, name):
return ()
def file(self, path):
"""The step screenshots of a run while it runs: /files/runs/<name>/<file>.png."""
if not path.startswith('/files/runs/') or not path.endswith('.png'):
return None
base = (self.run_dir / 'runs').resolve()
candidate = (self.run_dir / path.removeprefix('/files/')).resolve()
return candidate if candidate.is_relative_to(base) and candidate.is_file() else None
def triage(run, visual):
"""The reasons to look closer at a run. An empty list means that the run looks clean: every step and check
passed, with no browser error, no failed tool call, and no visual change."""
reasons = []
status = run['entry'].get('status')
if status != 'passed':
reasons.append(f'the run {status}')
failed = [s for s in run['steps'] if s['status'] == 'failed']
if failed:
reasons.append(f'{len(failed)} failed step(s) or check(s)')
console = run['console']
errors = len(console.get('consoleErrors') or []) + len(console.get('requestErrors') or [])
if errors:
reasons.append(f'{errors} browser error(s)')
tool_errors = sum(e['error'] for e in run['data']['events'] if e['kind'] == 'tool')
if tool_errors:
reasons.append(f'{tool_errors} failed tool call(s)')
changed = [r['name'] for r in visual if r['scenario'] == run['scenario']['id'] and r['status'] != 'unchanged']
if changed and not run['master']:
reasons.append(f'visual change in {", ".join(changed)}')
return reasons
def summarize(run_dir):
"""Plain text for the judge: a triage list, then each run with its steps, final reply, errors and visual
results."""
plan, status, runs = load(run_dir)
visual = read_json(run_dir / 'visual.json', [])
out = [f'# {plan.get("title", "")}', '', 'Feature:', plan.get('feature') or '-', '', '## Triage', '',
'Clean runs passed every step and check, with no browser error, failed tool call or visual change.',
'For these, check that the last reply and the final screenshot match the expected result.',
'Look closely at the other runs: their screenshots, errors and diffs.', '']
judged = [r for r in runs if not r['master']]
for run in judged:
reasons = triage(run, visual)
out.append(f'- {run["name"]}: ' + ('clean' if not reasons else 'look closer: ' + '; '.join(reasons)))
out.append('')
for run in runs:
s, result, data = run['scenario'], run['result'], run['data']
out += [f'## {run["name"]}: {run_title(run)} ({s["kind"]}, {"master" if run["master"] else "branch"})',
f'Expected: {s["expected"]}',
f'Status: {run["entry"].get("status")}' + (f' ({result["error"]})' if result.get('error') else ''),
'Steps:']
for st in run['steps']:
shot = f' (runs/{run["name"]}/{st["shot"]})' if st.get('shot') else ''
out.append(f' {st["i"]}. [{st["status"]}] {st["kind"]}: {st["title"]}{shot}')
if st.get('error'):
out.append(' ' + st['error'].replace('\n', '\n ')[:1500])
for p in result.get('prompts') or []:
out.append(f'Prompt (thread {p.get("thread")}): {p["text"]}')
tools = [e for e in data['events'] if e['kind'] == 'tool']
if tools:
out.append('Tool calls: ' + ', '.join(e['name'] + (' (error)' if e['error'] else '') for e in tools))
texts = [e['text'] for e in data['events'] if e['kind'] == 'text']
if texts:
out.append('Last reply: ' + texts[-1].strip()[:2000])
for d in run['meta'].get('decisions') or []:
out.append(f'Confirmation: {d["decision"]} [{d.get("toolName")}]')
console = run['console']
for e in (console.get('consoleErrors') or [])[:8]:
out.append(f'Console error: {e.get("text", "")[:300]}')
for e in (console.get('requestErrors') or [])[:8]:
out.append(f'Request error: {e.get("status")} {e.get("method")} {e.get("url")}')
if s['kind'] == 'visual' and not run['master']:
for r in (v for v in visual if v['scenario'] == s['id']):
ratio = f', {100 * r["ratio"]:.2f}% of pixels' if r.get('ratio') is not None else ''
out.append(f'Visual {r["name"]}: {r["status"]}{ratio}')
if r.get('ariaDiff'):
out.append(' Accessibility tree diff:\n ' + '\n '.join(r['ariaDiff'].splitlines()[:40]))
out.append('')
return '\n'.join(out)
def main():
if len(sys.argv) < 2:
sys.exit(__doc__)
run_dir = Path(sys.argv[1]).resolve()
if '--summary' in sys.argv:
print(summarize(run_dir))
else:
print(build(run_dir))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,931 @@
#!/usr/bin/env python3
"""Run a test plan of Instance AI scenarios and build one report. The judge step is separate.
run.py --plan <dir>/scenarios.json [--only id1,id2] [--concurrency 4]
run.py --rerun last|<run-dir> the plan of an earlier run, again in a new folder
run.py --plan ... --try a quick check of new scripts: no master, no live page, no report
run.py --plan ... --estimate print the expected cost and time, then stop
run.py --check check the setup, then stop
Steps:
1. Check the setup, and build this checkout with turbo (a few seconds when nothing changed).
2. Start fresh n8n instances from the build. Each feature and regression scenario gets its own
instance, unless it sets "isolated": false. Instances start ahead of their scenario, so a
scenario does not wait for one.
3. Run the scenarios, several at the same time.
4. At the same time: build the master merge base (baseline.py), start a fresh master instance and
a fresh branch instance, and run each visual scenario on both. A master run that passed is kept
and used again while the merge base and the script stay the same.
5. Compare the visual captures, write summary.txt for the judge, and build the report.
6. Fetch the token usage from LangSmith and build the report again.
A local page shows the runs live (live_page.py). It opens in the default browser, unless --no-open.
Settings come from .env.local and .env.ai-surfaces.
"""
import argparse
import difflib
import hashlib
import json
import os
import re
import shutil
import signal
import socket
import statistics
import subprocess
import sys
import threading
import time
import webbrowser
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
from pathlib import Path
import baseline
import report
from common import CACHE, HERE, PLANS_ROOT, REPO_ROOT, RUNS_ROOT, SHARED, Instance, client, git, setting
from langsmith_usage import fetch_calls, totals
from live import enabled as live_enabled
from run_instance_ai import decide
KINDS = ('feature', 'regression', 'visual')
LIB = HERE.parent / 'scenarios'
ID = re.compile(r'^[a-z0-9][a-z0-9-]{0,60}$')
MASTER_CACHE = CACHE / 'master-runs'
MASTER_CACHE_DAYS = 14
PRINT_LOCK = threading.Lock()
RUNNERS = set()
S = '.agents/skills/testmax-assistant-ui/scripts'
def say(line, prefix=None):
with PRINT_LOCK:
print(f'[{prefix}] {line}' if prefix else line, flush=True)
def step(title):
say(f'\n=== {title} ===')
def now_iso():
return datetime.now(timezone.utc).isoformat()
def slug(text, words=8):
return '-'.join(re.findall(r'[a-z0-9]+', text.lower())[:words]) or 'run'
def script_path(spec, base):
return LIB / spec[4:] if spec.startswith('lib:') else (base / spec).resolve()
def load_plan(path):
"""Read and check scenarios.json. Returns the plan with each script path resolved."""
path = Path(path).resolve()
plan = json.loads(path.read_text())
errors, seen = [], set()
if not plan.get('scenarios'):
errors.append('The plan has no scenarios.')
if not isinstance(plan.get('featureFlags', {}), dict):
errors.append('featureFlags must be an object of flag names and values.')
for s in plan.get('scenarios', []):
sid = s.get('id', '')
if not ID.match(sid) or '--' in sid:
errors.append(f'Scenario id "{sid}": use a-z, 0-9 and single "-" characters.')
if sid in seen:
errors.append(f'Scenario id "{sid}" is used two times.')
seen.add(sid)
if s.get('kind') not in KINDS:
errors.append(f'Scenario "{sid}": kind must be one of {", ".join(KINDS)}.')
if not s.get('script') or not script_path(s['script'], path.parent).is_file():
errors.append(f'Scenario "{sid}": script {s.get("script")} not found.')
if not s.get('title') or not s.get('expected'):
errors.append(f'Scenario "{sid}": give a title and an expected outcome.')
if errors:
sys.exit('The plan is not valid:\n ' + '\n '.join(errors))
for s in plan['scenarios']:
s['scriptPath'] = str(script_path(s['script'], path.parent))
plan.setdefault('source', str(path))
plan['name'] = path.parent.name if path.parent.name != 'plan' else slug(plan.get('title') or 'run')
return plan
def run_dirs():
"""Earlier run folders, newest first."""
if not RUNS_ROOT.exists():
return []
found = [d for d in RUNS_ROOT.iterdir() if (d / 'plan' / 'scenarios.json').is_file()]
return sorted(found, key=lambda d: (d / 'plan' / 'scenarios.json').stat().st_mtime, reverse=True)
def rerun_plan(arg):
"""The plan to run again. The source plan wins over the copy, so a fixed script is used."""
if arg == 'last':
found = run_dirs()
if not found:
sys.exit(f'No earlier run in {RUNS_ROOT}.')
run_dir = found[0]
else:
run_dir = Path(arg).expanduser().resolve()
copy = run_dir / 'plan' / 'scenarios.json'
if not copy.is_file():
sys.exit(f'{run_dir} has no plan/scenarios.json.')
source = json.loads(copy.read_text()).get('source')
if source and Path(source).is_file():
say(f'rerun of {run_dir.name}, with the current plan and scripts in {Path(source).parent}')
return source
say(f'rerun of {run_dir.name}, with the scripts copied into that run')
return copy
def new_run_dir(root, name):
base = root / f'{datetime.now():%Y%m%d-%H%M%S}-{name}'
run_dir, n = base, 2
while run_dir.exists():
run_dir, n = base.with_name(f'{base.name}-{n}'), n + 1
run_dir.mkdir(parents=True)
return run_dir
def copy_plan(plan, run_dir):
"""Keep the plan and its scripts in the run folder, so --rerun can repeat the run."""
plan_dir = run_dir / 'plan'
(plan_dir / 'scripts').mkdir(parents=True, exist_ok=True)
saved = json.loads(json.dumps(plan))
saved.pop('name', None)
for s in saved['scenarios']:
target = plan_dir / 'scripts' / f'{s["id"]}.mjs'
shutil.copy(s.pop('scriptPath'), target)
s['script'] = f'scripts/{target.name}'
(plan_dir / 'scenarios.json').write_text(json.dumps(saved, indent=2))
class Progress:
"""Writes status.json, which the live page reads."""
def __init__(self, run_dir, data):
self.path, self.lock, self.data = run_dir / 'status.json', threading.Lock(), data
self.save()
def save(self):
tmp = self.path.with_suffix('.tmp')
tmp.write_text(json.dumps(self.data, indent=2))
os.replace(tmp, self.path)
def update(self, **fields):
with self.lock:
self.data.update(fields)
self.save()
def run(self, name, **fields):
with self.lock:
self.data['runs'][name].update(fields)
self.save()
def finish(self):
with self.lock:
for entry in self.data['runs'].values():
if entry['status'] in ('waiting', 'starting'):
entry.update(status='skipped', reason=entry.get('reason') or 'The run did not start.')
elif entry['status'] == 'running':
entry['status'] = 'error'
if self.data.get('phase') != 'done':
self.data.update(phase='done', step='Done', finishedAt=time.time())
self.save()
def close_stream(res):
"""Close an event stream that another thread reads. close() alone waits for the next server
keep-alive (about 13 seconds), because the reader holds the buffer lock. A socket shutdown ends the
read at once."""
try:
with socket.socket(fileno=os.dup(res.fileno())) as sock:
sock.shutdown(socket.SHUT_RDWR)
except (OSError, ValueError):
pass
try:
res.close()
except Exception:
pass
class ThreadEvents:
"""Reads the event stream of each thread that a scenario opens. The stream starts at the first event."""
def __init__(self, base_url, out, auto_confirm, prefix):
self.client = client(base_url).login()
self.out, self.auto_confirm, self.prefix = out, auto_confirm, prefix
self.file, self.lock = (out / 'events.jsonl').open('a'), threading.Lock()
self.run_ids, self.decisions, self.streams, self.closed = [], [], [], False
def follow(self, thread_id):
threading.Thread(target=self._read, args=(thread_id,), daemon=True).start()
def _read(self, thread_id):
try:
res = self.client.open_stream(f'/rest/instance-ai/events/{thread_id}')
except Exception as e:
say(f'event stream of {thread_id} did not open: {e}', self.prefix)
return
self.streams.append(res)
data, name = [], None
try:
for raw in res:
line = raw.decode(errors='replace').rstrip('\r\n')
if line.startswith('data:'):
data.append(line[5:].lstrip())
elif line.startswith('event:'):
name = line[6:].strip()
elif line == '' and data:
try:
self._event(json.loads('\n'.join(data)))
except json.JSONDecodeError:
pass
data, name = [], None
except Exception:
# close() ends the stream when the scenario stops.
pass
def _event(self, event):
with self.lock:
if self.closed:
return
self.file.write(json.dumps(event) + '\n')
self.file.flush()
payload = event.get('payload') or {}
if event.get('type') == 'run-start' and payload.get('langsmithRunId'):
self.run_ids.append(payload['langsmithRunId'])
if event.get('type') == 'confirmation-request' and self.auto_confirm:
body, label = decide(payload)
try:
self.client.request('POST', f'/rest/instance-ai/confirm/{payload["requestId"]}', body)
ok = True
except RuntimeError as e:
ok, label = False, f'{label} (failed: {str(e)[:200]})'
self.decisions.append({'at': now_iso(), 'toolName': payload.get('toolName'), 'decision': label, 'ok': ok})
say(f'confirm: {label} [{payload.get("toolName")}]', self.prefix)
def close(self):
with self.lock:
self.closed = True
self.file.close()
for res in self.streams:
close_stream(res)
def run_job(name, scenario, url, instance, run_dir, progress, live):
sub = run_dir / 'runs' / name
sub.mkdir(parents=True, exist_ok=True)
progress.run(name, status='running', startedAt=time.time(), baseUrl=url)
auto_confirm = scenario.get('autoConfirm', True)
timeout_min = float(scenario.get('timeoutMin', 10))
started = now_iso()
try:
events = ThreadEvents(url, sub, auto_confirm, name)
except Exception as e:
progress.run(name, status='error', finishedAt=time.time(), reason=f'Cannot sign in to {url}: {e}')
return
cmd = ['node', str(HERE / 'run_scenario.mjs'), '--out', str(sub), '--script', scenario['scriptPath'],
'--timeout-min', str(timeout_min), '--viewport', scenario.get('viewport', '1600x1000')]
cmd += ['--live'] if live else []
cmd += ['--auto-confirm'] if auto_confirm else []
proc = subprocess.Popen(cmd, cwd=HERE, env={**os.environ, 'N8N_BASE_URL': url},
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
RUNNERS.add(proc)
# The scenario has its own time limit. This one covers a runner that hangs outside the scenario.
watchdog = threading.Timer(timeout_min * 60 + 180, proc.kill)
watchdog.start()
with (sub / 'runner.log').open('w') as log:
for line in proc.stdout:
log.write(line)
say(line.rstrip(), name)
match = re.match(r'thread (\S+)', line)
if match:
events.follow(match[1])
proc.wait()
RUNNERS.discard(proc)
watchdog.cancel()
# The last events of a run can arrive just after the scenario ends.
time.sleep(1)
events.close()
try:
result = json.loads((sub / 'result.json').read_text())
except (OSError, ValueError):
result = {'status': 'error', 'error': f'The runner stopped without a result (exit {proc.returncode}). See runner.log.'}
(sub / 'result.json').write_text(json.dumps(result, indent=2))
meta = {
'kind': 'scenario', 'scenario': scenario['id'], 'instance': instance, 'baseUrl': url,
'threadIds': result.get('threads', []), 'langsmithRunIds': events.run_ids, 'decisions': events.decisions,
'startedAt': started, 'finishedAt': now_iso(),
'screen': 'screen.webm' if (sub / 'screen.webm').exists() else None,
}
(sub / 'run.json').write_text(json.dumps(meta, indent=2))
progress.run(name, status=result['status'], finishedAt=time.time(), reason=result.get('error'))
def stale_packages():
"""Packages with a source change in this branch that is newer than their build output."""
try:
base = subprocess.run(['git', 'merge-base', 'HEAD', 'origin/master'], cwd=REPO_ROOT, capture_output=True,
text=True, check=True).stdout.strip()
changed = subprocess.run(['git', 'diff', '--name-only', base], cwd=REPO_ROOT, capture_output=True,
text=True, check=True).stdout.split()
changed += [line[3:] for line in subprocess.run(['git', 'status', '--porcelain'], cwd=REPO_ROOT,
capture_output=True, text=True, check=True).stdout.splitlines()]
except subprocess.CalledProcessError:
return []
newest_dist, stale = {}, set()
for rel in changed:
path = REPO_ROOT / rel
if not rel.startswith('packages/') or '/src/' not in rel or re.search(r'(\.test\.|__tests__|\.md$)', rel):
continue
if not path.is_file():
continue
pkg = REPO_ROOT / rel.split('/src/', 1)[0]
dist = pkg / 'dist'
if not dist.exists():
continue
if pkg not in newest_dist:
newest_dist[pkg] = max((f.stat().st_mtime for f in dist.rglob('*') if f.is_file()), default=0)
if path.stat().st_mtime > newest_dist[pkg]:
stale.add(str(pkg.relative_to(REPO_ROOT)))
return sorted(stale)
def build_branch(log_path):
"""Build n8n and its packages with turbo. Unchanged packages come from the turbo cache in seconds."""
started = time.time()
with log_path.open('w') as log:
code = subprocess.run(['pnpm', 'turbo', 'run', 'build', '--filter=n8n...', '--output-logs=errors-only'],
cwd=REPO_ROOT, stdout=log, stderr=subprocess.STDOUT).returncode
text = log_path.read_text()
if code != 0:
sys.exit(f'The build failed. The end of {log_path}:\n' + '\n'.join(text.splitlines()[-30:]))
cached = re.search(r'Cached:\s+(\d+) cached, (\d+) total', text)
if cached and cached[1] == cached[2]:
say(f'build: up to date ({time.time() - started:.0f}s)')
else:
say(f'build: done in {time.time() - started:.0f}s')
def preflight(build, need_master, local_instances=True):
"""Problems that stop a run (errors) or make the report less useful (warnings).
local_instances is False with --branch-url: the branch runs then need no local build."""
errors, warnings = [], []
for tool in ('node', 'git') + (('pnpm',) if build or need_master else ()):
if not shutil.which(tool):
errors.append(f'{tool} is not on the PATH.')
if shutil.which('node'):
probe = ("import(process.argv[1]).then(async (m) => { const b = await m.launchBrowser(); await b.close(); })"
".catch((e) => { console.error(e.message.split('\\n')[0]); process.exit(1); })")
res = subprocess.run(['node', '-e', probe, str(SHARED / 'n8n_browser.mjs')], capture_output=True, text=True,
timeout=60)
if res.returncode != 0:
errors.append('Playwright cannot start a browser: '
f'{(res.stderr or res.stdout).strip()[:300]}. Run pnpm install, and install Google Chrome.')
if not build and local_instances:
for path in ('packages/cli/dist', 'packages/frontend/editor-ui/dist/index.html'):
if not (REPO_ROOT / path).exists():
errors.append(f'{path} does not exist. Build first, or run without --no-build.')
if not setting('N8N_INSTANCE_AI_MODEL'):
warnings.append('N8N_INSTANCE_AI_MODEL is not set in .env.local, .env.ai-surfaces or the environment. '
'Scenarios that send a prompt can fail.')
if not setting('LANGSMITH_API_KEY'):
warnings.append('LANGSMITH_API_KEY is not set. The report has no tokens or cost.')
elif (setting('LANGSMITH_TRACING') or '').lower() != 'true':
warnings.append('LANGSMITH_TRACING is not true. The instances send no traces, so the report has no tokens or cost.')
if (setting('N8N_DIAGNOSTICS_ENABLED') or '').lower() == 'false':
warnings.append('N8N_DIAGNOSTICS_ENABLED is false. Instance AI sends no traces, so the report has no tokens or cost.')
RUNS_ROOT.mkdir(parents=True, exist_ok=True)
free = shutil.disk_usage(RUNS_ROOT).free / 1e9
if free < 2:
warnings.append(f'Only {free:.1f} GB free in {RUNS_ROOT}. A run needs about 0.5 GB.')
if need_master and not baseline.built():
CACHE.mkdir(parents=True, exist_ok=True)
free = shutil.disk_usage(CACHE).free / 1e9
warnings.append('The master build does not exist yet. The first build takes 15 to 20 minutes'
+ (f', and needs about 3 GB (free: {free:.1f} GB).' if free < 5 else '.'))
return errors, warnings
def history():
"""Cost and time of earlier runs, by scenario id. Master runs and runs without a result are left out."""
by_id, every = {}, []
for run_dir in run_dirs():
try:
runs = json.loads((run_dir / 'status.json').read_text()).get('runs', {})
except (OSError, ValueError):
continue
for name, entry in runs.items():
if name.endswith('--master') or entry.get('status') not in ('passed', 'failed'):
continue
if not (entry.get('startedAt') and entry.get('finishedAt')):
continue
usage = run_dir / 'runs' / name / 'usage.json'
try:
if usage.exists():
cost = sum(c['cost'] for c in json.loads(usage.read_text()))
else:
meta = json.loads((run_dir / 'runs' / name / 'run.json').read_text())
cost = None if meta.get('langsmithRunIds') else 0.0
except (OSError, ValueError):
cost = None
sample = (cost, entry['finishedAt'] - entry['startedAt'])
by_id.setdefault(entry.get('scenario', name), []).append(sample)
every.append(sample)
return by_id, every
def estimate(jobs, concurrency, compare_master, flags):
"""Expected cost and time of the jobs, from the earlier runs of the same scenario ids. A master run that
can be reused costs nothing."""
by_id, every = history()
try:
base = baseline.target() if compare_master else None
except subprocess.CalledProcessError:
base = None
def median(samples, index, default):
values = [x[index] for x in samples if x[index] is not None]
return statistics.median(values) if values else default
any_cost, any_time = median(every, 0, None), median(every, 1, 60)
rows, cost_total, known = [], 0.0, True
for name, s, role in jobs:
if role == 'master' and base and (MASTER_CACHE / master_key(s, base, flags) / 'result.json').is_file():
rows.append((name + ' (kept)', 0.0, 0, True))
continue
samples = by_id.get(s['id'], [])
cost = median(samples, 0, any_cost)
seconds = median(samples, 1, any_time)
if cost is None:
known = False
else:
cost_total += cost
rows.append((name, cost, seconds, bool(samples)))
times = sorted((r[2] for r in rows), reverse=True)
wall = max(times[0] if times else 0, sum(times) / max(1, concurrency)) + 15
return {'rows': rows, 'cost': cost_total if known else None, 'seconds': wall,
'masterBuild': compare_master and not baseline.built()}
def print_estimate(est):
say('Estimate (median of earlier runs; * means no earlier run of this scenario):')
for name, cost, seconds, seen in est['rows']:
price = f'${cost:.2f}' if cost is not None else ' ? '
say(f' {name:44} {price:>7} {seconds:5.0f}s{"" if seen else " *"}')
total = f'about ${est["cost"]:.2f}' if est['cost'] is not None else 'unknown cost'
extra = ', plus 15 to 20 minutes for the first master build' if est['masterBuild'] else ''
say(f' {len(est["rows"])} runs, {total}, about {est["seconds"] / 60:.1f} minutes{extra}')
def fetch_usage(run_dir, names, timeout=90):
"""Token usage and cost of each run from LangSmith, in one query for all runs. LangSmith receives traces
with a delay, so the query repeats until each run has calls and the count stops changing."""
if not setting('LANGSMITH_API_KEY'):
return
metas = {}
for name in names:
file = run_dir / 'runs' / name / 'run.json'
meta = json.loads(file.read_text()) if file.exists() else {}
if meta.get('langsmithRunIds') and not meta.get('cached'):
metas[name] = meta
if not metas:
return
start = min(m['startedAt'] for m in metas.values())
end = max(m['finishedAt'] for m in metas.values())
traces = {t for m in metas.values() for t in m['langsmithRunIds']}
deadline, previous, calls = time.time() + timeout, None, []
while True:
try:
calls = fetch_calls(start, end, traces)
except (SystemExit, OSError, ValueError) as e:
say(f'usage failed: {str(e)[:200]}')
return
seen = {c['trace'] for c in calls}
complete = all(set(m['langsmithRunIds']) & seen for m in metas.values())
if (complete and len(calls) == previous) or time.time() > deadline:
break
previous = len(calls)
time.sleep(10)
for name, meta in metas.items():
own = [c for c in calls if c['trace'] in set(meta['langsmithRunIds'])]
(run_dir / 'runs' / name / 'usage.json').write_text(json.dumps(own, indent=1))
usage = totals(own)
say(f'{usage["calls"]} LLM calls, ${usage["cost"]:.2f}', name)
def compare_trees(run_dir):
"""Add the accessibility tree diff of each capture to visual.json."""
path = run_dir / 'visual.json'
if not path.exists():
return []
results = json.loads(path.read_text())
for r in results:
trees = []
for name in (f'{r["scenario"]}--master', r['scenario']):
file = run_dir / 'runs' / name / 'captures' / f'{r["name"]}.aria.yml'
trees.append(file.read_text().splitlines() if file.exists() else [])
diff = '\n'.join(difflib.unified_diff(*trees, 'master', 'branch', lineterm='', n=2))
r['ariaDiff'] = diff
r['pixelStatus'] = r['status']
if diff and r['status'] == 'unchanged':
r['status'] = 'changed'
path.write_text(json.dumps(results, indent=2))
return results
def master_key(scenario, commit, flags):
"""A master run gives the same screens while the commit, the flags, the script and the runner stay the same."""
h = hashlib.sha256()
overrides = setting('N8N_FEATURE_FLAG_OVERRIDES') or ''
for part in (commit, json.dumps(flags or {}, sort_keys=True), overrides, scenario.get('viewport', '1600x1000'),
str(scenario.get('autoConfirm', True))):
h.update(part.encode() + b'\0')
for file in (Path(scenario['scriptPath']), HERE / 'kit.mjs', HERE / 'run_scenario.mjs'):
h.update(file.read_bytes())
return h.hexdigest()[:20]
def save_master_run(sub, key):
if json.loads((sub / 'result.json').read_text()).get('status') != 'passed':
return
target = MASTER_CACHE / key
shutil.rmtree(target, ignore_errors=True)
shutil.copytree(sub, target, ignore=shutil.ignore_patterns('live.png', 'usage.json'))
cutoff = time.time() - MASTER_CACHE_DAYS * 86400
for old in MASTER_CACHE.iterdir():
if old.is_dir() and old.stat().st_mtime < cutoff:
shutil.rmtree(old, ignore_errors=True)
def reuse_master_run(sub, key):
"""Copy a kept master run into this run. Returns False when there is none."""
source = MASTER_CACHE / key
if not (source / 'result.json').is_file():
return False
shutil.copytree(source, sub, dirs_exist_ok=True)
meta = json.loads((sub / 'run.json').read_text())
meta['cached'] = True
(sub / 'run.json').write_text(json.dumps(meta, indent=2))
return True
def start_live_server(run_dir):
url_file = run_dir / 'live-url.txt'
url_file.unlink(missing_ok=True)
log = (run_dir / 'live-server.log').open('w')
# A new session, so the page still serves the report after this script ends.
subprocess.Popen([sys.executable, str(HERE / 'live_page.py'), '--run-dir', str(run_dir),
'--parent-pid', str(os.getpid())],
cwd=HERE, stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
for _ in range(50):
if url_file.exists() and url_file.read_text().strip():
return url_file.read_text().strip()
time.sleep(0.1)
return None
def open_in_default_browser(url):
"""Opens url in the default browser. Returns False if no browser opened.
$BROWSER is not used: agents often set it to a command that does nothing.
"""
command = {'darwin': ['open'], 'linux': ['xdg-open']}.get(sys.platform)
if command and shutil.which(command[0]):
try:
return subprocess.run(command + [url], capture_output=True, timeout=15).returncode == 0
except (OSError, subprocess.TimeoutExpired):
return False
try:
return webbrowser.open(url)
except webbrowser.Error:
return False
def result_table(run_dir, runs):
"""A plain text table of the results, with the first problem of each run that did not pass."""
lines = [f' {"run":44} {"status":15} {"steps":>7} {"time":>6} {"errors":>6}']
for name, entry in runs.items():
sub = run_dir / 'runs' / name
steps = report.read_json(sub / 'steps.json', [])
result = report.read_json(sub / 'result.json', {})
checked = [s for s in steps if s['kind'] in ('step', 'check', 'capture')]
passed = f'{sum(s["status"] == "passed" for s in checked)}/{len(checked)}' if checked else '-'
took = (entry.get('finishedAt') or 0) - (entry.get('startedAt') or 0)
errors = (result.get('consoleErrors') or 0) + (result.get('requestErrors') or 0)
status = entry['status'] + (' (kept)' if report.read_json(sub / 'run.json', {}).get('cached') else '')
lines.append(f' {name:44} {status:15} {passed:>7} {took:5.0f}s {errors:>6}')
failed = next((s for s in steps if s['status'] == 'failed'), None)
if failed:
problem = f'{failed["kind"]} "{failed["title"]}": {failed.get("error") or ""}'
else:
problem = entry.get('reason') if entry['status'] != 'passed' else ''
if (problem or '').strip():
lines.append(f' {problem.strip().splitlines()[0][:160]}')
return '\n'.join(lines)
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--plan', help='scenarios.json')
ap.add_argument('--rerun', help='"last", or an earlier run folder. Uses the current plan and scripts when they exist.')
ap.add_argument('--only', help='Comma-separated scenario ids.')
ap.add_argument('--run-dir')
ap.add_argument('--concurrency', type=int, default=int(setting('FEATURE_TEST_CONCURRENCY', default='4')))
ap.add_argument('--branch-url', help='Use this running instance for the branch runs instead of fresh ones.')
ap.add_argument('--try', dest='try_mode', action='store_true',
help='A quick check of new scripts: no master, no live page, no usage, no report.')
ap.add_argument('--estimate', action='store_true', help='Print the expected cost and time, then stop.')
ap.add_argument('--check', action='store_true', help='Check the setup, then stop.')
ap.add_argument('--no-build', action='store_true', help='Do not build this checkout first.')
ap.add_argument('--no-baseline', action='store_true', help='Skip master. Visual scenarios run on the branch only.')
ap.add_argument('--no-cache', action='store_true', help='Run master again, also when a kept master run exists.')
ap.add_argument('--no-fetch', action='store_true', help='Use origin/master as it is, without git fetch.')
ap.add_argument('--no-live', action='store_true', help='Do not start the live page.')
ap.add_argument('--no-open', action='store_true',
help='Do not open the live page in the default browser. Use this when you open it in the editor browser.')
ap.add_argument('--keep-instances', action='store_true', help='Keep the fresh instances running after the run.')
args = ap.parse_args()
if args.try_mode:
args.no_baseline = args.no_live = True
build = not args.no_build and not args.branch_url
if args.check and not (args.plan or args.rerun):
errors, warnings = preflight(build, need_master=True)
for w in warnings:
say(f'warning: {w}')
for e in errors:
say(f'error: {e}')
branch = git('rev-parse', '--abbrev-ref', 'HEAD').replace('/', '-')
say(f'plan folder for this branch: {PLANS_ROOT / branch}/<feature>/')
say('The setup is ready.' if not errors else 'Fix the errors, then run again.')
sys.exit(1 if errors else 0)
if not (args.plan or args.rerun):
sys.exit('Give --plan <scenarios.json>, --rerun last|<run-dir>, or --check.')
# The fresh instances run in their own sessions. Exit through the cleanup, so they stop too.
for sig in (signal.SIGTERM, signal.SIGHUP):
signal.signal(sig, lambda signum, _frame: sys.exit(128 + signum))
plan = load_plan(rerun_plan(args.rerun) if args.rerun else args.plan)
if args.only:
wanted = {x.strip() for x in args.only.split(',')}
plan['scenarios'] = [s for s in plan['scenarios'] if s['id'] in wanted]
if not plan['scenarios']:
sys.exit(f'No scenario matches --only {args.only}.')
visual = [s for s in plan['scenarios'] if s['kind'] == 'visual']
compare_master = bool(visual) and not args.no_baseline
jobs = [] # (name, scenario, instance role)
for s in plan['scenarios']:
if args.branch_url:
role = 'branch'
elif s['kind'] == 'visual':
role = 'branch-visual'
else:
role = 'isolated' if s.get('isolated', True) else 'branch'
jobs.append((s['id'], s, role))
if s['kind'] == 'visual' and compare_master:
jobs.append((f'{s["id"]}--master', s, 'master'))
est = estimate(jobs, args.concurrency, compare_master, plan.get('featureFlags'))
print_estimate(est)
if args.estimate:
return
errors, warnings = preflight(build, compare_master, local_instances=not args.branch_url)
for w in warnings:
say(f'warning: {w}')
if errors:
sys.exit('The setup is not ready:\n ' + '\n '.join(errors))
if args.run_dir:
run_dir = Path(args.run_dir).expanduser().resolve()
run_dir.mkdir(parents=True, exist_ok=True)
else:
run_dir = new_run_dir(RUNS_ROOT / 'try' if args.try_mode else RUNS_ROOT, plan['name'])
copy_plan(plan, run_dir)
say(f'run directory: {run_dir}')
branch_commit = subprocess.run(['git', 'rev-parse', '--short', 'HEAD'], cwd=REPO_ROOT, capture_output=True,
text=True).stdout.strip()
dirty = bool(subprocess.run(['git', 'status', '--porcelain', '--', 'packages'], cwd=REPO_ROOT,
capture_output=True, text=True).stdout.strip())
live = live_enabled() and not args.no_live
progress = Progress(run_dir, {
'phase': 'running', 'step': 'Building the branch' if build else 'Starting the instances',
'startedAt': time.time(), 'concurrency': args.concurrency,
'branch': {'commit': branch_commit, 'dirty': dirty, 'url': args.branch_url},
'master': {'commit': None} if compare_master else None,
'runs': {name: {'status': 'waiting', 'scenario': s['id'], 'instance': role.split('-')[0]}
for name, s, role in jobs},
})
live_url = None
if live:
live_url = start_live_server(run_dir)
if live_url:
say(f'live page: {live_url}/report/live')
if not args.no_open and not open_in_default_browser(f'{live_url}/report/live'):
say(f'no browser opened. Open {live_url}/report/live to watch the runs.')
else:
say(f'the live page did not start, see {run_dir / "live-server.log"}')
instances, lock = [], threading.Lock()
keep = args.keep_instances
pool = ThreadPoolExecutor(max_workers=max(1, args.concurrency))
# Shared instances start at once. Isolated instances start ahead of their scenario, with at most one
# more alive than the number of scenarios that run at the same time.
shared_pool = ThreadPoolExecutor(max_workers=3)
boot_pool = ThreadPoolExecutor(max_workers=args.concurrency + 1)
alive = threading.BoundedSemaphore(args.concurrency + 1)
futures, shared = [], {}
def start_instance(name, root):
inst = Instance(name, root, run_dir / 'instances' / name, run_dir / 'instances' / f'{name}.log',
plan.get('featureFlags'))
with lock:
instances.append(inst)
started = time.time()
inst.start()
# The port only: Cursor opens an editor tab for each new localhost URL that a command prints.
say(f'{name} instance ready in {time.time() - started:.0f}s on port {inst.port}')
return inst
def boot_isolated(name):
alive.acquire()
try:
return start_instance(name, REPO_ROOT)
except Exception:
alive.release()
raise
def run_on(name, scenario, role, instance):
"""Run one job when its instance is ready. instance is a future of an Instance, or a URL."""
progress.run(name, status='starting')
try:
inst = instance.result() if hasattr(instance, 'result') else None
except Exception as e:
progress.run(name, status='error', finishedAt=time.time(), reason=f'The instance did not start: {e}')
return
url = inst.url if inst else instance
try:
run_job(name, scenario, url, role.split('-')[0], run_dir, progress, live)
if role == 'master' and not args.no_cache:
save_master_run(run_dir / 'runs' / name,
master_key(scenario, progress.data['master']['commit'], plan.get('featureFlags')))
finally:
if role == 'isolated':
if not keep:
inst.stop(clean=True)
alive.release()
def submit(name, scenario, role, instance):
futures.append(pool.submit(run_on, name, scenario, role, instance))
def skip(role, reason):
for name, _, r in jobs:
if r == role and progress.data['runs'][name]['status'] == 'waiting':
progress.run(name, status='skipped', reason=reason)
master_jobs = [(name, s) for name, s, r in jobs if r == 'master']
base_commit = None
if compare_master:
try:
base_commit = baseline.target(fetch=not args.no_fetch)
except subprocess.CalledProcessError as e:
say(f'cannot find the master merge base: {e}', 'master')
if base_commit:
progress.update(master={'commit': base_commit})
if not args.no_cache:
left = []
for name, s in master_jobs:
if reuse_master_run(run_dir / 'runs' / name, master_key(s, base_commit, plan.get('featureFlags'))):
now = time.time()
progress.run(name, status='passed', startedAt=now, finishedAt=now)
say(f'master run kept from an earlier run of {base_commit[:11]}', name)
else:
left.append((name, s))
master_jobs = left
def master_side():
try:
built = baseline.prepare(fetch=False, say=lambda m: say(m, 'master'))
progress.update(master={'commit': built['commit']})
inst = start_instance('master', built['root'])
for name, s in master_jobs:
submit(name, s, 'master', inst.url)
except Exception as e:
say(f'master is not available: {e}', 'master')
skip('master', f'Master is not available: {e}')
visual_error = None
try:
if build:
step('Build this checkout')
build_branch(run_dir / 'build.log')
elif not args.branch_url:
stale = stale_packages()
if stale:
say('warning: these packages have source changes that are newer than their build: '
f'{", ".join(stale)}. The fresh instances run the old build.')
progress.update(stale=stale)
step(f'Run {len(jobs)} scenario runs, {args.concurrency} at a time')
progress.update(step='Running the scenarios')
for role in ('branch', 'branch-visual'):
if any(r == role for _, _, r in jobs):
shared[role] = args.branch_url or shared_pool.submit(start_instance, role, REPO_ROOT)
boots = {name: boot_pool.submit(boot_isolated, name) for name, _, r in jobs if r == 'isolated'}
master_thread = None
if compare_master and master_jobs and base_commit:
master_thread = threading.Thread(target=master_side, daemon=True)
master_thread.start()
elif compare_master and not base_commit:
skip('master', 'The master merge base is not known.')
for name, s, role in jobs:
if role == 'isolated':
submit(name, s, role, boots[name])
elif role in shared:
submit(name, s, role, shared[role])
if master_thread:
master_thread.join()
# Jobs can be added while the pool runs, so wait until the list stops growing.
while any(not f.done() for f in list(futures)):
time.sleep(1)
for f in futures:
if f.exception():
say(f'a run failed: {f.exception()}')
pool.shutdown()
if not keep:
for inst in instances:
inst.stop(clean=True)
names = [name for name, _, _ in jobs]
if compare_master:
step('Compare the visual captures with master')
progress.update(step='Comparing the captures with master')
diff = subprocess.run(['node', 'visual_diff.mjs', str(run_dir)], cwd=HERE)
if diff.returncode != 0:
visual_error = f'visual_diff.mjs failed with exit code {diff.returncode}. The visual check is not complete.'
say(f'error: {visual_error}')
progress.update(visualError=visual_error)
trees = [r for r in compare_trees(run_dir) if r.get('ariaDiff')]
if trees:
say(f'{len(trees)} captures with a difference in the accessibility tree: '
+ ', '.join(f'{r["scenario"]}/{r["name"]}' for r in trees))
step('Summarize the runs for the judge')
summary = report.summarize(run_dir)
if visual_error:
summary = f'VISUAL CHECK FAILED: {visual_error}\n\n{summary}'
(run_dir / 'summary.txt').write_text(summary)
if not args.try_mode:
step('Build the report')
progress.update(step='Building the report')
report.build(run_dir)
progress.finish()
finally:
pool.shutdown(wait=False, cancel_futures=True)
boot_pool.shutdown(wait=False, cancel_futures=True)
shared_pool.shutdown(wait=False, cancel_futures=True)
for proc in list(RUNNERS):
proc.kill()
if not keep:
for inst in instances:
inst.stop(clean=True)
progress.finish()
runs = progress.data['runs']
visual_results = json.loads((run_dir / 'visual.json').read_text()) if (run_dir / 'visual.json').exists() else []
not_passed = [n for n, e in runs.items() if e['status'] != 'passed' and not n.endswith('--master')]
rerun = f'python3 {S}/run.py --rerun {run_dir}' + (f' --only {",".join(not_passed)}' if not_passed else '')
usage_later = not args.try_mode and bool(setting('LANGSMITH_API_KEY'))
say('\n=== Result ===')
say(result_table(run_dir, runs))
if not_passed:
say(f'Run the ones that did not pass again: {rerun}')
print(json.dumps({
'runDir': str(run_dir),
'report': None if args.try_mode else str(run_dir / 'report' / 'index.html'),
'live': f'{live_url}/report/index.html' if live_url else None,
'summary': str(run_dir / 'summary.txt'),
'results': {name: runs[name]['status'] for name in runs},
'visual': {'captures': len(visual_results),
'changed': [f'{r["scenario"]}/{r["name"]}' for r in visual_results if r['status'] != 'unchanged'],
'error': visual_error},
'instancePorts': {i.name: i.port for i in instances} if keep else None,
'rerun': rerun,
'usage': 'The token usage comes in the next lines. The report updates when it is done.' if usage_later else None,
'next': ('Fix the scripts that failed, then run --try again, or run the full plan.' if args.try_mode else
'Judge each run from summary.txt and write verdicts.json. The live page rebuilds the report.'),
}, indent=2), flush=True)
if usage_later:
step('Fetch the token usage from LangSmith')
fetch_usage(run_dir, [n for n in runs])
report.build(run_dir)
say('usage: done, the report is updated')
if __name__ == '__main__':
main()

View File

@ -0,0 +1,203 @@
#!/usr/bin/env node
// Run one scenario script in a browser that is signed in to an n8n instance.
//
// node run_scenario.mjs --out <dir> --script <file.mjs> [--live] [--auto-confirm]
// [--timeout-min 10] [--viewport 1600x1000]
//
// N8N_BASE_URL selects the instance. Writes to <dir>:
// steps.json the steps, checks and captures with their screenshots (updated while it runs)
// step-NN.png ... one screenshot for each step, check and capture
// captures/<name>.png screens for the visual check, with <name>.aria.yml
// live.png the screen every 2.5 seconds, for the live page (with --live)
// screen.webm the video, with screen-poster.png
// console.json browser errors and failed requests to the instance
// result.json {status: passed|failed|error, error, threads, prompts, startedAt, finishedAt, ...}
// Prints "thread <id>" for each thread that the scenario opens, so run.py can read its events.
import { randomUUID } from 'node:crypto';
import { copyFileSync, mkdirSync, mkdtempSync, renameSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
import { parseArgs } from 'node:util';
import {
baseUrl,
launchBrowser,
selectors,
setting,
signIn,
} from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
import { StepFailed, createKit } from './kit.mjs';
const { values: args } = parseArgs({
options: {
out: { type: 'string' },
script: { type: 'string' },
live: { type: 'boolean', default: false },
'auto-confirm': { type: 'boolean', default: false },
'timeout-min': { type: 'string', default: '10' },
viewport: { type: 'string', default: '1600x1000' },
},
});
const out = resolve(args.out);
mkdirSync(out, { recursive: true });
const [width, height] = args.viewport.split('x').map(Number);
const size = { width, height };
const LIVE_SHOT_MS = 2500;
const emit = (line) => console.log(line.replace(/\s+/g, ' ').slice(0, 300));
/**
* Sign in through the REST API, which is about 1.3 seconds faster than the sign-in page. The auth cookie
* is bound to the browser-id header of the login, so the page gets the same id in localStorage.
* Falls back to the sign-in page.
*/
async function signedInState(browser) {
const login = await browser.newContext();
try {
const browserId = randomUUID();
const res = await login.request.post(`${baseUrl}/rest/login`, {
headers: { 'browser-id': browserId },
data: {
emailOrLdapLoginId: setting('N8N_EVAL_EMAIL', 'nathan@n8n.io'),
password: setting('N8N_EVAL_PASSWORD', 'PlaywrightTest123'),
},
});
if (res.ok()) {
const state = await login.storageState();
state.origins = [
{
origin: new URL(baseUrl).origin,
localStorage: [{ name: 'n8n-browserId', value: browserId }],
},
];
return state;
}
await signIn(await login.newPage());
return await login.storageState();
} finally {
await login.close();
}
}
selectors.setTestIdAttribute('data-test-id');
const browser = await launchBrowser();
const storageState = await signedInState(browser);
// The video starts after the sign-in, at the first page of the scenario.
const videoDir = mkdtempSync(join(tmpdir(), 'feature-test-video-'));
const context = await browser.newContext({
viewport: size,
storageState,
recordVideo: { dir: videoDir, size },
});
const page = await context.newPage();
const origin = new URL(baseUrl).origin;
const consoleErrors = [];
const requestErrors = [];
page.on('console', (m) => {
if (m.type() === 'error')
consoleErrors.push({ text: m.text().slice(0, 1000), url: m.location()?.url });
});
page.on('pageerror', (e) => consoleErrors.push({ text: `Uncaught: ${e.message}`.slice(0, 1000) }));
page.on('response', (r) => {
if (r.status() >= 400 && r.url().startsWith(origin)) {
requestErrors.push({
status: r.status(),
method: r.request().method(),
url: r.url().slice(origin.length),
});
}
});
page.on('requestfailed', (r) => {
const failure = r.failure()?.errorText ?? '';
// Closed event streams and aborted navigations are normal.
if (r.url().startsWith(origin) && !/ERR_ABORTED|NS_BINDING_ABORTED/.test(failure)) {
requestErrors.push({
status: 0,
method: r.method(),
url: r.url().slice(origin.length),
error: failure,
});
}
});
let shooting = false;
async function liveShot() {
if (shooting) return;
shooting = true;
const partial = join(out, 'live.tmp.png');
try {
await page.screenshot({ path: partial });
renameSync(partial, join(out, 'live.png'));
} catch {
// The page can be in the middle of a navigation. The next interval tries again.
} finally {
shooting = false;
}
}
const liveTimer = args.live ? setInterval(liveShot, LIVE_SHOT_MS) : null;
const { t, threads, prompts, steps, stop } = createKit({
page,
out,
baseUrl,
autoConfirm: args['auto-confirm'],
emit,
});
const result = { status: 'passed', startedAt: new Date().toISOString(), baseUrl };
const timeoutMs = Number(args['timeout-min']) * 60_000;
let timer;
try {
const scenario = (await import(pathToFileURL(resolve(args.script)).href)).default;
if (typeof scenario !== 'function')
throw new Error('The script must export a default async function (t).');
await Promise.race([
scenario(t),
new Promise((_, reject) => {
timer = setTimeout(() => {
// The race does not end the scenario function. stop() makes its next step throw.
stop();
reject(new Error(`The scenario did not finish in ${args['timeout-min']} minutes.`));
}, timeoutMs);
}),
]);
if (steps.some((s) => s.status === 'failed')) result.status = 'failed';
} catch (error) {
result.status = error instanceof StepFailed || steps.length ? 'failed' : 'error';
result.error = String(error?.message ?? error)
.replace(/\u001b\[[0-9;]*m/g, '')
.slice(0, 2000);
if (!(error instanceof StepFailed)) emit(`error: ${result.error.split('\n')[0]}`);
} finally {
clearTimeout(timer);
}
stop();
clearInterval(liveTimer);
for (const s of steps.filter((x) => x.status === 'running')) {
s.status = 'failed';
s.error = 'The scenario stopped during this step.';
}
writeFileSync(join(out, 'steps.json'), JSON.stringify(steps, null, 1));
await page.waitForTimeout(1500);
if (args.live) await liveShot();
await page.screenshot({ path: join(out, 'screen-poster.png') }).catch(() => {});
const video = page.video();
await context.close();
if (video) copyFileSync(await video.path(), join(out, 'screen.webm'));
await browser.close();
rmSync(videoDir, { recursive: true, force: true });
Object.assign(result, {
finishedAt: new Date().toISOString(),
threads,
prompts,
consoleErrors: consoleErrors.length,
requestErrors: requestErrors.length,
});
writeFileSync(join(out, 'console.json'), JSON.stringify({ consoleErrors, requestErrors }, null, 1));
writeFileSync(join(out, 'result.json'), JSON.stringify(result, null, 2));
emit(`result ${result.status}`);
// A scenario that timed out can still wait on its own timers. The results are written, so end here.
process.exit(0);

View File

@ -0,0 +1,96 @@
#!/usr/bin/env node
// Compare the captures of each visual scenario: this branch against master.
//
// node visual_diff.mjs <run-dir>
//
// Reads runs/<id>/captures/*.png (branch) and runs/<id>--master/captures/*.png (master) for each
// visual scenario in plan/scenarios.json. Writes visual/<id>/<name>.diff.png and visual.json:
// [{scenario, name, status: unchanged|changed|branch-only|master-only, changedPixels, ratio,
// branchSize, masterSize, diff}]
// A pixel counts as changed when one color channel differs by more than PIXEL_THRESHOLD. A capture
// counts as changed when more than RATIO_THRESHOLD of its pixels changed, or its size changed.
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { loadPng } from '../../testmax-n8n-ai/scripts/n8n_browser.mjs';
const PIXEL_THRESHOLD = 32;
const RATIO_THRESHOLD = 0.0005;
const PNG = loadPng();
const runDir = resolve(process.argv[2] ?? '.');
const plan = JSON.parse(readFileSync(join(runDir, 'plan', 'scenarios.json'), 'utf8'));
const read = (file) => PNG.sync.read(readFileSync(file));
const names = (dir) =>
existsSync(dir)
? readdirSync(dir)
.filter((f) => f.endsWith('.png'))
.map((f) => f.slice(0, -4))
: [];
function compare(branch, master) {
const width = Math.max(branch.width, master.width);
const height = Math.max(branch.height, master.height);
const diff = new PNG({ width, height });
let changed = 0;
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
const o = (y * width + x) * 4;
const inB = x < branch.width && y < branch.height;
const inM = x < master.width && y < master.height;
const b = inB ? (y * branch.width + x) * 4 : -1;
const m = inM ? (y * master.width + x) * 4 : -1;
let differs = !inB || !inM;
if (!differs) {
for (let c = 0; c < 4 && !differs; c++)
differs = Math.abs(branch.data[b + c] - master.data[m + c]) > PIXEL_THRESHOLD;
}
if (differs) {
changed++;
diff.data.set([224, 17, 95, 255], o);
} else {
// The unchanged page shows as a faint gray copy of the branch screen.
const gray =
0.299 * branch.data[b] + 0.587 * branch.data[b + 1] + 0.114 * branch.data[b + 2];
const v = Math.round(255 - (255 - gray) * 0.25);
diff.data.set([v, v, v, 255], o);
}
}
}
return { diff, changed, ratio: changed / (width * height) };
}
const results = [];
for (const s of plan.scenarios.filter((x) => x.kind === 'visual')) {
const branchDir = join(runDir, 'runs', s.id, 'captures');
const masterDir = join(runDir, 'runs', `${s.id}--master`, 'captures');
const all = [...new Set([...names(branchDir), ...names(masterDir)])].sort();
for (const name of all) {
const entry = { scenario: s.id, name };
const hasB = existsSync(join(branchDir, `${name}.png`));
const hasM = existsSync(join(masterDir, `${name}.png`));
if (!hasB || !hasM) {
results.push({ ...entry, status: hasB ? 'branch-only' : 'master-only' });
continue;
}
const branch = read(join(branchDir, `${name}.png`));
const master = read(join(masterDir, `${name}.png`));
const { diff, changed, ratio } = compare(branch, master);
const out = join(runDir, 'visual', s.id);
mkdirSync(out, { recursive: true });
writeFileSync(join(out, `${name}.diff.png`), PNG.sync.write(diff));
const resized = branch.width !== master.width || branch.height !== master.height;
results.push({
...entry,
status: resized || ratio > RATIO_THRESHOLD ? 'changed' : 'unchanged',
changedPixels: changed,
ratio,
branchSize: [branch.width, branch.height],
masterSize: [master.width, master.height],
diff: `visual/${s.id}/${name}.diff.png`,
});
}
}
writeFileSync(join(runDir, 'visual.json'), JSON.stringify(results, null, 2));
const changed = results.filter((r) => r.status !== 'unchanged').length;
console.log(`${results.length} captures compared, ${changed} with a difference`);

View File

@ -0,0 +1,745 @@
---
name: n8n:testmax-n8n-ai
description: >-
Runs one workflow prompt through Instance AI, Claude Code with the n8n MCP
server, and Codex with the n8n MCP server on a local n8n instance. Collects
the traces, per-call tokens, cost, and canvas screenshots, judges each run
against the expected outcome, and builds one HTML comparison page. Claude
Code and Codex run in a sandbox that sees only the n8n MCP server. Use when
the user wants to compare builders, test a prompt across Instance AI and
MCP, or says /testmax-n8n-ai.
---
# Compare workflow builders
This skill sends the same prompt to up to three builders on the local n8n
instance. The builders run at the same time, and a local page shows them
live. It then builds one page that shows, for each run:
- the result against the expected outcome (pass, partial or fail, with reasons
and the evidence that a panel of blind judges cited)
- deterministic checks of each workflow: the node definitions, the node
issues in the n8n editor, the graph, the expression references, and a run
of each branch with pinned data
- a separate n8n standards score, from the rules in
[n8n-io/skills](https://github.com/n8n-io/skills) and the workflow-sdk best
practices. This score does not change the verdict
- the canvas of the workflow that the builder made
- a replay of the run: the exact terminal screen, or a video of the n8n UI
- the cost
- the LLM steps: one entry for each LLM call, with its tokens (cache read,
cache write, uncached input, output) and the agent text, reasoning and tool
calls (with input and output) that the call made
The page uses the light mode of the
[n8n brand guidelines](https://guidelines.n8n.design/).
![An example comparison page](assets/comparison-page.png)
All scripts are in `scripts/` next to this file. They use only the Python
standard library and the Playwright copy in the repository.
```bash
S="$(git rev-parse --show-toplevel)/.agents/skills/testmax-n8n-ai/scripts"
```
## Settings
All settings are in `.env.ai-surfaces` in the repository root. Git ignores
this file. The template is `.env.ai-surfaces.example`, also in the
repository root. The scripts read,
from low to high priority: `.env.local`, `.env.ai-surfaces`, the shell
environment. Empty values are ignored. Thus keys that are already in
`.env.local` work without a change.
If `.env.ai-surfaces` does not exist, copy `.env.ai-surfaces.example` to it. Tell the user
which values to fill in. Never read the values back to the user, never print
them, and never put a secret on a command line.
The most important settings:
| Setting | Use |
|---|---|
| `COMPARE_BUILDERS` | Default builders, for example `iai,claude,codex` |
| `COMPARE_PARALLEL` | `1` (default) runs the builders at the same time. `0` runs them one at a time |
| `COMPARE_LIVE` | `1` (default) opens a live page of the runs. `0` turns it off |
| `COMPARE_LIVE_WINDOWS` | `1` also opens the Instance AI thread and a terminal window for each CLI. Default `0` |
| `COMPARE_RECORD` | `1` (default) records the Instance AI video for the replay. `0` turns it off |
| `COMPARE_CLI_MODE` | `interactive` (default) records the real CLI screen. `headless` uses the JSON event stream |
| `COMPARE_RECORD_REQUESTS` | `1` (default) records the Codex request bodies, without headers, for the tool list. `0` turns it off |
| `COMPARE_SCENARIOS` | `1` (default) runs each workflow with pinned data through the n8n MCP server. `0` turns it off |
| `COMPARE_WIRE` | `1` (default) runs each workflow in process with recorded API fixtures (`run_wire.mjs`). `0` turns it off |
| `COMPARE_BINARY_CHECKS` | `1` (default) runs the deterministic binary checks of the Instance AI eval suite. `0` turns it off |
| `GOOGLE_GENERATIVE_AI_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `MISTRAL_API_KEY`, `GROQ_API_KEY` | Model keys for the real AI runs. A workflow model without a key skips those runs |
| `CLAUDE_MODEL`, `CODEX_MODEL` | Model ids. Empty uses the CLI default. A prefix such as `anthropic/` is removed |
| `COMPARE_CLAUDE_TOOLS` | `none` (default) gives Claude Code only the n8n tools. `default` also gives it its built-in tools, all allowed (see "How the builders run") |
| `CLAUDE_API_KEY` | Anthropic key for Claude Code. Falls back to `ANTHROPIC_API_KEY` |
| `CODEX_API_KEY` | OpenAI key for Codex. Falls back to `OPENAI_API_KEY` |
| `N8N_MCP_TOKEN` | MCP key of the n8n user. Optional |
| `LANGSMITH_*` | Instance AI cost and tokens |
## 1. Ask the user
Ask for these two items. If the harness has a structured question tool, use
it. If not, ask in chat. Do not guess them.
1. **Prompt.** The exact text to send to each builder.
2. **Expected outcome.** What a correct result does. Ask for concrete checks,
for example "WhatsApp trigger", "Gemini compares the question to the FAQ
sheet", "the false branch makes a Notion ticket and a Slack message".
Use the builders and models from the settings. Ask about them only if the
user wants a change for this run.
Write one expected outcome item on each line, numbered or bulleted. The
number of an item is its id: the judges cite it, and the page groups the
checks by it.
Then write the scenarios for the pinned-data runs, from the expected
outcome. You do not need to ask the user for them. A scenario sets the data
of the trigger, the service nodes and the AI nodes, and says which nodes
must or must not run. Logic nodes (IF, Switch, Set, Code, Aggregate and
others) run for real. Make one scenario for each branch of the expected
outcome:
```json
{
"pinByType": {
"googleSheets": [{ "Question": "What are your opening hours?", "Answer": "9:00 to 17:00" }]
},
"scenarios": [
{ "name": "The FAQ answers the question", "ai": { "booleans": true },
"expect": { "reach": ["whatsApp"], "skip": ["notion", "slack"] } },
{ "name": "The FAQ does not answer the question", "ai": { "booleans": false },
"expect": { "reach": ["notion", "slack"] } }
]
}
```
- `pinByType`: output items for every node of a type, in all scenarios. A
type is the full name or the part after the last dot. A scenario can have
its own `pinByType`.
- `trigger`: output items of the trigger. Without it, the trigger gets a
fixture in the shape that the trigger code emits (WhatsApp, Webhook, Chat,
Schedule, Telegram, Form and Manual triggers have one).
- `ai`: the output of each AI node. `booleans` sets every boolean in the
output, `set` sets fields, and `text` sets the whole text of a node
without a structured output parser. The output follows the schema of the
structured output parser, or the JSON example in the prompt of the node.
- `expect`: `reach` lists node types that must run without an error,
`skip` lists node types that must not run.
- `triggerNode`: the name of the trigger that the scenario starts from, for a
workflow with more than one trigger. Without it, the scenario starts from
the trigger that reaches the most `reach` types.
Without a scenarios file, `run_scenarios.py` makes one scenario for each
value of the AI booleans, with no expectations.
`run_wire.mjs` runs the same scenarios in process. There, service nodes
send real requests that recorded fixtures answer, so a scenario can also
check the API contract. Add these keys to the scenarios file:
```json
{
"scenarios": [
{ "name": "The FAQ does not answer the question", "ai": { "booleans": false },
"expect": { "reach": ["notion", "slack"], "requests": [
{ "type": "notion", "method": "POST", "contains": ["\"title\"", "@trigger.messages[0].from"] },
{ "type": "slack", "method": "POST",
"contains": ["@trigger.messages[0].text.body", ["@response.notion.id", "@response.notion.url"]] } ] } }
],
"edge": [
{ "name": "The FAQ sheet is empty", "pinByType": { "googleSheets": [] }, "ai": { "booleans": false },
"expect": { "reach": ["notion", "slack"] } }
],
"questions": [
{ "name": "Opening hours, which the FAQ answers", "message": "Hi! What time do you open?",
"expect": { "skip": ["notion"], "requests": [
{ "type": "whatsApp", "method": "POST", "contains": ["@trigger.messages[0].from", "9:00"] } ] } }
]
}
```
- `expect.requests`: each entry needs a request from a node of `type`,
optionally with `method` and an exact `count`, whose URL, query or body
contains every value in `contains`. An array in `contains` means any one
of its values. `@trigger.<path>` is a value of the trigger item.
`@response.<type>.<path>` is a value of the response to the last write
request of that node type, for example the id of the Notion page.
- `expect.noWrites`: the scenario must send no write request.
- `edge`: inputs that the prompt does not describe. The trigger kits in
`sdk.mjs` add standard edge cases, for example two WhatsApp messages in one
webhook call (each sender must get a request) and a status update (no
writes). Edge cases are a score, not part of the verdict.
- `questions`: the AI nodes call the real model with this customer
`message`. The trigger fixture carries the message. A real-AI run needs a
model key, and is a score, not part of the verdict.
## 2. Run the builders
```bash
python3 $S/compare.py --title "<short title>" \
--prompt-file <file with the prompt> --expected-file <file with the outcome> \
--scenarios-file <file with the scenarios>
```
Write the prompt, the expected outcome and the scenarios to files with the
file tool first. Do not put them on the command line, because they can
contain quotes. Use `--builders iai,claude` to change the builders for one
run. Use `--rerun <old run dir>` to run the same case again. It also copies
`scenarios.json`.
The command takes 5 to 30 minutes. Start it in the background if the harness
supports it, and read its output until it ends. It prints one line for each
tool call, with the builder name in front, so the user can follow the run.
It does these steps:
1. Checks the setup and skips a builder that cannot run. It prints the
reason.
2. Starts the live page (see below).
3. Runs the builders at the same time. `--sequential` or
`COMPARE_PARALLEL=0` runs them one at a time.
4. Fetches the Instance AI cost from LangSmith.
5. Takes a screenshot of each canvas, and saves the node issues that the
n8n editor shows (`canvas-issues.json`).
6. Runs each workflow once for each scenario, with pinned data, through the
`test_workflow` tool of the n8n MCP server (`exec.json`). It turns on
"Available in MCP" for a workflow that does not have it.
7. Runs each workflow in process with recorded API fixtures, for the
scenarios, the edge cases and the real AI questions (`wire.json`).
8. Runs the deterministic eval binary checks (`binary-checks.json`) and
checks the final-message claims (`claims.json`).
9. Derives the deterministic checks (`auto-checks.json`). See "Checks on
the page".
10. Writes `summary.txt`, and `judge/summary-blind.txt` for the judges.
11. Builds the page without verdicts.
The last output is JSON with `runDir`, `report`, `live`, `parallel`,
`summary`, `blindSummary`, `models` and `skipped`.
Parallel runs share one n8n instance and one machine. Thus the times of the
builders include the shared load. The page says "run in parallel" under
the title. Use
`--sequential` when the user wants to compare times exactly.
### Watch the runs
`compare.py` starts `live_server.py` and opens its page in the default
browser. It prints `live page: <url>`. Give the user this URL. The page
shows each builder while it runs, and updates every 2 seconds without a
reload:
- the table: status, time, LLM calls, tool calls and tokens of each builder
- **Live:** the real terminal screen of Claude Code and Codex, streamed from
the tmux recording, and a screenshot of the Instance AI thread every
2.5 seconds
- the LLM steps of each builder. Claude Code and Codex steps come from the
session files of the CLIs, with tokens. Instance AI steps come from its
event stream, without tokens until the run ends
The cost, the canvases and the Instance AI tokens come after the run. Until
then, the Expected outcome box shows "Not verified yet", and the
Correctness, Execution and n8n standards boxes show "Not run yet". When
the report is ready, the page changes to it. The server listens on
`127.0.0.1` only. It stops 30 minutes after the last request, when the run
is done.
The report shows the price and judge steps (steps 3 and 4) as queued until
you start them. The Result column shows "Judge queued" or "Judging", and
the Cost column of a CLI builder without a price shows "Price queued" or
"Finding price". Each `build_report.py --step` call builds the page again.
While a step is queued or running, the page on the live server reloads
itself when the page changes. A page that you open from the file needs a
manual reload.
Headless mode and clean isolation have no terminal screen. For them, the
page shows only the table and the LLM steps. `COMPARE_RECORD=0` removes the
Instance AI screenshot.
`COMPARE_LIVE_WINDOWS=1` also opens the old live views:
- **Instance AI:** the thread opens in the default browser. The browser must
be logged in as `N8N_EVAL_EMAIL`, because a thread is visible only to its
owner. The script answers the confirmation cards, so the user must not
click them.
- **Claude Code and Codex:** a terminal window shows the real screen of the
CLI, read-only (`docker exec … tmux attach -r`). In headless mode, the
window follows `<builder>/live.log` instead. On macOS the window opens in
Terminal. The first time, macOS asks if the agent's app can control
Terminal.
Use `--no-live` or `COMPARE_LIVE=0` for runs without a display, for example
in CI.
### Replay on the page
The Replay section shows the last screen of each builder. The play button
on a screen plays that run in place on the page. **Expand** opens a
full-screen player. In that player, the user sees all builders side by
side, or picks one builder, and plays the runs at 1× to 8×:
- **Claude Code and Codex:** `<builder>/screen.cast` is an asciinema
recording of the exact terminal screen. The page plays it with the
asciinema player.
- **Instance AI:** `iai/screen.webm` is a video of the thread in the n8n UI.
A headless browser records it while the run continues.
`COMPARE_RECORD=0` skips the Instance AI video. The terminal recordings come
with interactive mode. `COMPARE_CLI_MODE=headless` has no terminal
recording.
### Canvas on the page
`capture_canvas.mjs` expands the node groups, zooms to fit, and crops the
screenshot to the nodes, their labels and their subtitles. Empty click areas
and hidden hover toolbars do not change the crop. The page shows the canvas
next to the expected outcome, at its real size and at most 360 px high.
A click on the canvas opens a full-screen viewer. It has a tab for each run
that has a canvas, the verdict and the n8n link of the run, and a zoom
button. The left and right arrow keys switch runs. A click on the image
zooms in to that point. Esc closes the viewer, and the page then shows the
last run that the user looked at. Cmd-click or Ctrl-click still opens the
image in a new tab.
### Checks on the page
Under the expected outcome and the canvas, each run has sections from
`derive_checks.mjs`. No LLM makes these checks, except the model calls of
the real AI runs. Correctness, Execution with pinned data and Service calls
change the verdict. The other sections show a score only.
**Correctness.** A failed check changes the verdict.
| Id | Check | Source |
|---|---|---|
| C1 | Node parameters match the node definitions | `validateWorkflow` of `@n8n/workflow-sdk`, with the built node definitions |
| C2 | The n8n editor shows no node errors | The issues of each node in the editor. Missing credentials and empty required pickers are setup for the user (C2s, information only) |
| C3 | Every node can be reached from a trigger | The connections |
| C4 | Each `$('Node')` names a node that exists and runs before the node that reads it | The expressions |
| C5 | Each expression field is in the output of the node that it reads | The real output of the pinned-data runs |
| C6 | No keys or tokens in node parameters | The parameters |
C5 fails only when the output is real: a node that ran, the trigger
fixture, the case data, or the schema of the structured output parser. A
field that is missing only from a node schema sample or a prompt example
shows as "Unconfirmed".
**Execution with pinned data.** One check for each scenario that ran without
an error, and one for each `reach` or `skip` expectation. A failed check
changes the verdict.
**Service calls.** `run_wire.mjs` loads the n8n execution engine from the
built `packages/core` and runs each workflow in process. Triggers and AI
nodes get pinned data. Service nodes run their real code, and the core
eval hook answers each HTTP request with a recorded fixture before any
credential is used. The process has no sandbox, and the hook sees only the
requests that go through the n8n request helpers. Thus each node that runs
must be on an allow-list (`runsInProcess` in `run_wire.mjs`):
- logic nodes from `LOGIC_TYPES`, HTTP Request, Wait and Telegram
- declarative nodes
- nodes whose credentials all authenticate HTTP requests (an `authenticate`
block or OAuth)
- a fixed set of AI nodes, with models that have a key in `MODEL_CREDENTIALS`
Code, Function and AI Transform nodes, and nodes with database, SSH, FTP,
SMTP, file or shell access, are not on the list. Each scenario checks the
nodes that run from its trigger. Pinned nodes do not run, so a node under
a pinned AI node counts only in the real AI runs. If a node outside the list
would run, the scenario is skipped. The report shows it as skipped, not
failed, with the names of the nodes.
The pinned-data runs still run Code nodes in the n8n task runner. No request
leaves the machine, except the model calls of the real AI runs. Empty pickers get
`mock-<parameter>` values for the run. Each scenario row lists the nodes
that ran with their item counts, and each request. A failed check changes
the verdict.
**Edge cases** and **Real AI runs.** The same in-process runs for the
`edge` and `questions` entries of the scenarios file. A score.
**Final message claims.** `claims.py` compares the final message with the
trace and the workflow. A score.
| Id | Claim | Source |
|---|---|---|
| M1 | The workflow id or URL is a workflow of this run | `run.json` |
| M2 | The publish state (draft or active) | `workflow.json` `active` |
| M3 | The workflow passed validation | The last validation tool call |
| M4 | The workflow was tested, or was not run | Test and execution tool calls. Scripts that report passed tests count, with a note that they ran outside n8n |
| M5 | Values in `code spans` | The workflow, the prompt and the tool calls |
| M6 | A claim that the workflow was checked agrees with the service-call runs | `wire.json` |
**Eval binary checks.** `binary_checks.mts` runs the deterministic binary
checks of the Instance AI eval suite
(`packages/@n8n/instance-ai/evaluations/binaryChecks`), for example
`no_empty_set_nodes` and `no_unnecessary_code_nodes`. Checks that do not
apply are left out. A score.
**Eval quality checks.** The judges answer the LLM binary checks of the
same suite: `valid_data_flow`, `correct_node_operations`,
`handles_multiple_items`, `descriptive_node_names` and
`response_describes_changes_accurately`. `summarize_runs.py` reads their
rules from the check sources. A score.
**n8n standards.** A score, not part of the verdict. The builders do not get
these rules: Claude Code and Codex run without the n8n skills, and Instance
AI uses its own prompts. Only the rules that apply to the workflow count.
| Id | Rule | Source |
|---|---|---|
| S1 | An unattended workflow has an error workflow | `n8n-error-handling-official` |
| S12 | A failed node goes to a wired error output, not on as data | `n8n-error-handling-official` |
| S2 | Nodes that call a service have `retryOnFail` | `n8n-error-handling-official` |
| S3 | Expressions read data by node name, not `$json` | `n8n-expressions-official` |
| S4 | Structured AI output uses a manual schema with `autoFix` | `n8n-ai-nodes-official` STRUCTURED_OUTPUT.md |
| S5 | Messages do not carry the n8n attribution | `n8n-node-configuration-official` COMMS_NODES.md, and the node definitions |
| S6 | References do not read only the first item of a node that can output several | `n8n-expressions-official`, and the trigger code |
| S7 | A model that decides a branch has a temperature of 0.2 or lower | workflow-sdk best practices: triage |
| S8 | AI output that decides a branch has a reasoning field | workflow-sdk best practices: triage |
| S9 | Every branch after an AI decision, including the fallback, goes somewhere | workflow-sdk best practices: triage |
| S10 | A chatbot agent has memory | workflow-sdk best practices: chatbot |
| S11 | The memory session key names a node and identifies the conversation | workflow-sdk best practices: chatbot, and `n8n-ai-nodes-official` MEMORY.md |
The skill rules come from commit `27b6b9f` of n8n-io/skills. Each check
links to its source.
**The verdict** comes from the checks, not from the judges alone:
- `fail` when there is no workflow, or the judges say that the workflow does
not do what the prompt asks (`goalMet: false`)
- `pass` when every expected outcome item is met, and no correctness,
execution or service-call check failed
- `partial` for all other results
A met item must cite evidence that the workflow contains. Without evidence,
or with evidence that is not in the workflow, the item shows as not
confirmed. The Result column shows the verdict and the standards score
(`Std 5/10`).
An old `verdicts.json` without ids or evidence keeps the verdict of the
judge. A failed correctness or execution check changes its `pass` to
`partial`.
### LLM steps on the page
Each run has a list of LLM steps, like the LLM steps view in the Instance AI
debug panel. The box is collapsed by default. Click its title to open it. On
the live page, a box that you open stays open when the page updates. The list
shows the tools or text of each call, its time and a
token bar. Select a step, or use the arrow keys, to see the token counts and
what the call did. Step 1 starts with what the model gets before the first
call, then the user message. Tool output is collapsed, except when the tool
failed.
- **Claude Code:** the system prompt and the tool definitions, from the
request shape in the session file. The tools show as chips in groups:
System (the built-in tools of the CLI), one group for each MCP server, and
Other (for example tools that the model provider runs). System and MCP
show "None" when the model had no such tools. The tools that the run
called are highlighted, with a call count and a dot for failures. Select a chip to see the description and a table of
the parameters. A filter box searches names and descriptions.
- **Codex:** the system prompt (base instructions), the developer messages
and the environment context, from the session file. Codex does not save
its tool definitions, so the runner records them (see "Isolation"). The
tools use the same chips. Codex declares most system tools inside the
description of its `exec` tool, so they show as "System · in exec" with
their TypeScript declaration. Codex does not put MCP tools in the prompt:
its scripts look them up in `ALL_TOOLS` at run time. The MCP group thus
shows only the MCP tools that the run called, with a dashed border and no
schema. Runs from before the recorder show no Codex tools.
- **Instance AI:** the system prompt, the context that n8n puts before the
prompt (for example `<thread-context>`), and the tools, from the first
LangSmith LLM call of each agent. `langsmith_usage.py` saves them in
`context.json`. Tools that the agent loads later with `load_tool` show as
"System · loaded with load_tool". A sub-agent shows its own system prompt,
briefing and tools at its first step. Runs from before `context.json` show
no Instance AI context: run `langsmith_usage.py` again for them.
- **Claude Code:** the session file links each event to its LLM call.
- **Codex:** text goes to the response that wrote it. A tool call goes to the
response that asked for it. Codex calls the n8n tools from scripts that it
runs with its `exec` tool. The page shows these scripts too, but does not
count them as tool calls.
- **Instance AI:** the LangSmith call times place each event. Text times are
approximate. An asterisk on a time marks text that the page puts at the
previous tool call.
### When a check fails
- Cannot log in: n8n is not running, or `N8N_BASE_URL`, `N8N_EVAL_EMAIL` or
`N8N_EVAL_PASSWORD` is wrong.
- The MCP token does not work: n8n shows the full MCP key only once. Ask the
user to put the key in `N8N_MCP_TOKEN`, or ask before you run
`python3 $S/prepare.py --rotate`. Rotation makes the old key stop working
in every other MCP client of this user.
- Docker is not running: Codex is skipped, and Claude runs as a clean
process.
## 3. Find the prices
Codex reports no cost, and the estimate of Claude Code can be out of date
for new models. Thus, find the published price of each model that a CLI
builder used. The models are in `models` in the output of `compare.py`, and
in `<runDir>/models.json`. Instance AI cost comes from LangSmith, so do not
look up its model.
Do not skip this step. In interactive mode neither CLI reports a cost, so
without prices the page shows "not reported" for both. `build_report.py`
prints `models without a published price in case.json: …` while a model has
no price.
First, tell the page that you look for the prices:
```bash
python3 $S/build_report.py <runDir> --step prices=running
```
For each model:
1. Search the web for the official price page of the provider, for example
the Anthropic pricing page or the OpenAI pricing page. Do not use
third-party price lists if the provider page has the model.
2. Read four prices in USD per million tokens: input, cache read, cache
write, output. If the provider has no cache-write price, use the input
price. Anthropic has two cache-write prices. Use the 5-minute price, and
tell the user that the 1-hour price is higher.
3. If you cannot find the model, write no price for it. The page then shows
the estimate of Claude Code, or "not reported" for Codex. Tell the user.
Add the prices to `<runDir>/case.json`, with the URL that you used:
```json
"prices": {
"<model id as in models.json>": {
"input": 5, "cached": 0.5, "cacheWrite": 6.25, "output": 25,
"source": "https://..."
}
}
```
The page calculates the cost of each call from its tokens. The tooltip of a
Cost cell shows the source of the cost, the prices and the URL.
## 4. Judge each run
Show the costs and tell the page that you judge the runs:
```bash
python3 $S/build_report.py <runDir> --step prices=done --step judge=running
```
A panel of three judges judges the runs blind. Each judge reads only
`<runDir>/judge/summary-blind.txt`. In that file the builders are called A,
B and C in a random order, and the final messages have no builder or model
names. `judge/blind-map.json` maps the letters back. Do not give a judge
`summary.txt`, the map, the trace files or this conversation.
If the harness can start sub-agents, start three sub-agents with a fresh
context, one for each judge. Use a model that is not one of the builder
models when the harness lets you choose. If the harness cannot start
sub-agents, judge once yourself from the blind file only, and write
`verdicts-1.json`.
Give each judge these instructions:
> Read `<runDir>/judge/summary-blind.txt`. It lists the expected outcome
> criteria with their ids, and for each builder the final message, every
> node with all its parameters, the connections, and deterministic facts
> (AUTO-CHECK FACTS). Trust the facts: they come from the node definitions,
> the n8n editor and real pinned-data runs. For each builder, make one check
> for each criterion id. A met check must cite evidence: the node name, the
> parameter path, and the exact value in the workflow. Without evidence,
> set `ok` to `null`. Set `goalMet` to false when the workflow does not do
> what the prompt asks, and give the reason in `goalReason`. Then answer
> each QUALITY CHECK in `quality`, with the check name as the `id`. Write
> the result to `<runDir>/judge/verdicts-<n>.json`.
The format of each judge file, with the builder letters as keys:
```json
{
"A": {
"summary": "One or two sentences about the result.",
"goalMet": true,
"goalReason": "",
"checks": [
{ "id": "1", "ok": true, "text": "A WhatsApp trigger starts the workflow on `messages`",
"evidence": [{ "node": "WhatsApp Message Received", "path": "parameters.updates", "value": "messages" }] },
{ "id": "9", "ok": false, "text": "Send Reply reads `$json.from`, which does not exist" },
{ "id": "16", "ok": null, "text": "Could not confirm the sheet columns" },
{ "ok": false, "text": "The Slack node is not connected" }
],
"quality": [
{ "id": "valid_data_flow", "ok": true, "text": "Every expression reads a field that its source outputs" },
{ "id": "descriptive_node_names", "ok": false, "text": "Two nodes keep the default names `Code` and `IF`" }
]
}
}
```
- `id` is the id of the criterion in brackets, as a string. Leave out `id`
for a problem that is not a criterion. The page shows it under "Other
findings".
- `evidence` lists `{node, path, value}` entries. `path` is a dotted path in
the node, for example `parameters.textBody` or
`parameters.conditions.conditions[0].leftValue`. The page checks that the
node has the path and that the path contains the value. An entry with
`"source": "message"` and a `value` quotes the final message and is not
checked.
- Write each check as one short sentence with one fact. Put expressions,
field paths and parameter names in backticks. The page shows them as
code. Do not repeat the failed checks in the summary.
Then merge the panel into `verdicts.json`:
```bash
python3 $S/merge_verdicts.py <runDir>
```
For each criterion, the merged check takes the value that all judges or a
majority gave. A split shows on the page ("3 judges · 1 split", and "2 of 3
judges" on the check). Without a majority, the check is not confirmed. The
summary and the other findings come from judge 1. The page computes the
verdict (see "Checks on the page").
## 5. Build the page again
```bash
python3 $S/build_report.py <runDir> --step judge=done
```
If the command prints `models without a published price`, do step 3 for
those models and build the page again.
The page is `<runDir>/report/index.html`. It works offline. Give the user the
path, and the `live` URL from the output of `compare.py` while the server
runs. Then, in a few sentences, report:
- the verdict of each builder, with the most important reason
- failed correctness, execution or service-call checks, and judge splits
- the standards, edge-case, real-AI, claims and eval scores of each
builder, as separate results
- the cost, time and number of LLM calls of each builder
- builders that were skipped, and data that is missing
## How the builders run
**Instance AI** runs through the REST API of the instance, with its model.
The runner answers every confirmation card with one fixed policy, so runs
can be compared. It approves actions and plans, picks the first option of a
question, allows a domain once, defers workflow and credential setup cards,
and denies access to local files and MCP connections. `iai/run.json` lists
each decision.
**Claude Code and Codex** run in a Docker container (`COMPARE_ISOLATION=container`):
- The container mounts only the run folder of the builder. The builder
cannot see host files, skills, memory, `CLAUDE.md`, `AGENTS.md` or user
settings.
- The container has no Linux capabilities and a read-only root file system.
- Claude starts with a new config folder, no built-in tools, and only the
`n8n` MCP server. It does not use `--bare`, because `--bare` replaces the
normal system prompt with a 150-byte one. The container already keeps out
host settings, so Claude gets its normal system prompt, like Codex does.
`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` stops update checks and
telemetry.
- `COMPARE_CLAUDE_TOOLS=default` gives Claude the built-in tools of a normal
Claude Code session, such as Bash, Read, Write, Edit and WebFetch, with
`--permission-mode bypassPermissions`. Claude then uses every tool without
a question, as Codex does. The tools can change files in the run folder
and read the environment, which holds the model key and the MCP token.
The secret check after the run finds a key that a tool wrote to a file.
Claude Code also turns on tool search: the model gets only the names of
the n8n tools and of some built-in tools, and loads a schema with
`ToolSearch`. The page shows these deferred tools with a dashed border.
The default `none` keeps runs comparable with older runs, because the
extra tools change the tokens and the cost.
- Codex gets a new `CODEX_HOME` that holds only the n8n MCP server.
Its plugins, apps, browser use, computer use and image generation are off.
- The container can still reach the internet and the host network. It needs
them for the model API and for n8n.
- Codex sends its model requests through a local recorder
(`request_recorder.py`). The recorder sends each request to OpenAI without
changes and writes the request body, but not the headers, to
`codex/requests.jsonl`. The key is in a header, so it stays out of the file.
The file does not hold the conversation input. It holds the tools and the
instructions only when they change. The recorder listens on `127.0.0.1`,
which Docker Desktop forwards to `host.docker.internal`. On Linux, set
`COMPARE_RECORDER_BIND` to the address of the Docker bridge. Set
`COMPARE_RECORD_REQUESTS=0` to send Codex straight to OpenAI.
By default the CLIs run in interactive mode (`COMPARE_CLI_MODE=interactive`):
- The normal terminal screen of the CLI runs in tmux, at 120 × 36
characters. tmux answers the terminal queries of the CLI, and its output
goes to `screen.cast`.
- The CLIs connect MCP servers in the background. Thus the script pastes the
prompt when the input box shows, plus a short wait, not as an argument.
- Claude uses `--permission-mode dontAsk`: the n8n tools are pre-approved,
and all other tools are denied without a question.
- The model key reaches the CLI through a variable that only the CLI reads
(`apiKeyHelper` for Claude, a provider `env_key` for Codex). Thus no screen
asks to confirm the key or shows part of it, and no key is in a file.
- The script reads the session file of the CLI for the progress lines, the
end of the turn, the trace and the tokens. Claude Code then shows no cost
estimate, so the cost needs the published prices.
- After each run, the script searches the run folder for the model key and
the MCP token. It prints a warning with the file name if it finds one.
`COMPARE_CLI_MODE=headless` uses `claude -p` and `codex exec` with a JSON
event stream instead.
The first run builds the image `n8n-builder-sandbox` from
`sandbox/Dockerfile`. A change to the Dockerfile rebuilds the image on the
next run. The image has `python3`, `jq` and `ripgrep`, because Codex and
Claude Code with `COMPARE_CLAUDE_TOOLS=default` use them in their shell. To get new CLI
versions, run `docker rmi n8n-builder-sandbox`.
`COMPARE_ISOLATION=clean` runs Claude as a clean process on the host: an
empty environment, a new config folder, no built-in tools, and only the n8n
MCP server. It ignores `COMPARE_CLAUDE_TOOLS`. It uses `--bare` to keep out host settings, so it gets the
minimal system prompt. It needs an API key, because `--bare` does not read
the subscription login. Codex has no clean mode, because its shell tool can
read the host disk.
## Single steps
`compare.py` calls these scripts. Use them to repeat one step.
| Script | Step |
|---|---|
| `prepare.py` | Check the setup. `--rotate` makes a new MCP key |
| `run_instance_ai.py --out <dir>/iai --prompt-file <file>` | Run Instance AI |
| `run_cli_builder.py --builder claude\|codex --out <dir>/<name> --prompt-file <file>` | Run a CLI builder. `--mode headless` skips the screen |
| `node record_iai.mjs <dir>/iai` | Record the Instance AI video. `run_instance_ai.py` starts it |
| `live_server.py --run-dir <dir>` | Serve the live page and the report of a run folder. Prints nothing. The URL is in `<dir>/live-url.txt` |
| `langsmith_usage.py --run-dir <dir>/iai` | Fetch Instance AI usage and model context |
| `node capture_canvas.mjs <dir>` | Save `workflow.json`, `canvas.png` and `canvas-issues.json` for each run |
| `run_scenarios.py <dir>` | Run each workflow with pinned data. Writes `<name>/exec.json` |
| `node sdk.mjs pin-plan <workflow.json> [scenarios.json]` | Print the pin data of each scenario, without a run |
| `node run_wire.mjs <dir>` | Run each workflow in process with recorded API fixtures. Writes `<name>/wire.json` |
| `pnpm --filter @n8n/instance-ai exec tsx $S/binary_checks.mts <dir>` | Run the deterministic eval binary checks. Writes `<name>/binary-checks.json` |
| `claims.py <dir>` | Check the final-message claims. Writes `<name>/claims.json` |
| `node derive_checks.mjs <dir>` | Write `<name>/auto-checks.json` from all the files above |
| `summarize_runs.py <dir> [--blind]` | Print the judge summary. `--blind` writes `judge/summary-blind.txt` and `judge/blind-map.json` |
| `merge_verdicts.py <dir>` | Merge `judge/verdicts-*.json` into `verdicts.json` |
| `build_report.py <dir> [--step judge\|prices=queued\|running\|done]` | Build the page. `--step` first sets the state of a step after the run |
If a runner did not find a workflow id, read its trace for the id. Set
`workflowId` in that `run.json`, then run `capture_canvas.mjs`,
`run_scenarios.py`, `run_wire.mjs`, `binary_checks.mts`, `claims.py`,
`derive_checks.mjs` and `summarize_runs.py` again.
## Notes
- Cost sources: Instance AI cost comes from LangSmith. For Claude Code and
Codex, the cost is tokens × the published prices in `case.json`. Without a
price, Claude Code in headless mode shows its own estimate. All other runs
show "not reported".
The tooltip of each Cost cell shows the source of the cost.
- The workflows stay in the n8n instance. Archive them in n8n when they are
no longer necessary. The pinned-data runs add test executions to them.
- The n8n skills stay out of the sandboxes, so the standards score shows
what each builder does without them. Do not add the skills to the
sandboxes to raise the score.

Binary file not shown.

After

Width:  |  Height:  |  Size: 102 KiB

View File

@ -0,0 +1,21 @@
# Sandbox for CLI workflow builders. The container sees only the run directory
# that is mounted at /out. It reaches n8n through host.docker.internal.
FROM node:26-slim
ARG CLAUDE_CODE_VERSION=latest
ARG CODEX_VERSION=latest
# Codex uses the system trust store. Claude Code uses the CA list in Node.
# tmux runs the interactive CLI screens so that they can be recorded and watched.
# Claude Code (with built-in tools) and Codex use python3, jq and ripgrep in their shell to read large tool results.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates tmux python3 jq ripgrep \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" "@openai/codex@${CODEX_VERSION}" \
&& npm cache clean --force
ENV DISABLE_AUTOUPDATER=1 \
DISABLE_TELEMETRY=1 \
DISABLE_ERROR_REPORTING=1 \
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

View File

@ -0,0 +1,100 @@
// Run an interactive CLI inside tmux and record its screen as an asciinema v2 file.
//
// node screen.mjs run <dir> start tmux, record, and wait for <dir>/stop or the CLI exit
// node screen.mjs launch <dir> (inside tmux) start the command in <dir>/command.json
// node screen.mjs record <dir> (tmux pipe-pane) write the pane output to <dir>/screen.cast
//
// <dir>/command.json is {"argv": [...], "cols": 120, "rows": 36, "input": "prompt",
// "readyText": "text on the screen when the CLI accepts input", "settleMs": 0}. tmux answers the terminal
// queries of the CLI, so the CLI behaves as in a real terminal. The recording has the raw
// output of the CLI, so a player shows the exact screen.
import { spawn, spawnSync } from 'node:child_process';
import { createWriteStream, existsSync, readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
const [mode, dir] = process.argv.slice(2);
const command = JSON.parse(readFileSync(join(dir, 'command.json'), 'utf8'));
const READY = '/tmp/screen-ready';
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const tmux = (...args) => spawnSync('tmux', args, { stdio: 'inherit' }).status;
const screenText = () =>
spawnSync('tmux', ['capture-pane', '-p', '-t', 'run'], { encoding: 'utf8' }).stdout ?? '';
// The CLIs connect MCP servers in the background. A prompt on the command line can start
// before the n8n tools exist, so the prompt is pasted when the input box is ready.
async function typePrompt({ input, readyText, settleMs = 0 }) {
const deadline = Date.now() + 60_000;
while (readyText && !screenText().includes(readyText) && Date.now() < deadline) await sleep(250);
await sleep(settleMs);
tmux('set-buffer', '-b', 'prompt', '--', input);
tmux('paste-buffer', '-p', '-d', '-b', 'prompt', '-t', 'run');
await sleep(500);
tmux('send-keys', '-t', 'run', 'Enter');
}
if (mode === 'record') {
const out = createWriteStream(join(dir, 'screen.cast'));
out.write(
JSON.stringify({
version: 2,
width: command.cols,
height: command.rows,
timestamp: Math.floor(Date.now() / 1000),
env: { TERM: 'xterm-256color' },
}) + '\n',
);
const start = process.hrtime.bigint();
const decoder = new TextDecoder('utf-8');
process.stdin.on('data', (chunk) => {
const text = decoder.decode(chunk, { stream: true });
if (!text) return;
const t = Number(process.hrtime.bigint() - start) / 1e9;
out.write(JSON.stringify([Number(t.toFixed(6)), 'o', text]) + '\n');
});
process.stdin.on('end', () => out.end());
} else if (mode === 'launch') {
while (!existsSync(READY)) await sleep(50);
const [bin, ...args] = command.argv;
const child = spawn(bin, args, { stdio: 'inherit' });
child.on('exit', (code) => {
writeFileSync(join(dir, 'exited'), String(code ?? 1));
process.exit(code ?? 1);
});
// A CLI that cannot start emits error, not exit. Without the marker, the run waits for ever.
child.on('error', (error) => {
console.error(`Cannot start ${bin}: ${error.message}`);
writeFileSync(join(dir, 'exited'), '127');
process.exit(127);
});
} else if (mode === 'run') {
const self = new URL(import.meta.url).pathname;
// mouse and focus-events stop the tmux tips that Claude Code shows in the recorded screen.
writeFileSync(
'/tmp/tmux.conf',
'set -g status off\nset -g default-terminal xterm-256color\nset -g mouse on\nset -g focus-events on\n',
);
tmux(
'-f',
'/tmp/tmux.conf',
'new-session',
'-d',
'-s',
'run',
'-x',
String(command.cols),
'-y',
String(command.rows),
`node ${self} launch ${dir}`,
);
// A watcher in another terminal size must not resize the recorded screen. tmux 3.5 stops
// when this option is in the startup config, so it is set after the session starts.
tmux('set-option', '-g', 'window-size', 'manual');
tmux('pipe-pane', '-O', '-t', 'run', `node ${self} record ${dir}`);
writeFileSync(READY, '');
if (command.input) await typePrompt(command);
while (!existsSync(join(dir, 'stop')) && !existsSync(join(dir, 'exited'))) await sleep(500);
// Keep the final screen in the recording for a moment.
await sleep(2500);
tmux('kill-server');
await sleep(500);
}

View File

@ -0,0 +1,50 @@
// Runs the deterministic binary checks of the Instance AI eval suite on each workflow of a run.
// Run it with tsx from the instance-ai package, because the checks are TypeScript sources:
//
// pnpm --filter @n8n/instance-ai exec tsx <skill>/scripts/binary_checks.mts <run-dir>
//
// Writes <builder>/binary-checks.json with {source, outcomes: [{name, description, dimension, status, comment}]}.
import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const repo = resolve(dirname(fileURLToPath(import.meta.url)), '../../../..');
const SOURCE = 'packages/@n8n/instance-ai/evaluations/binaryChecks/checks';
type Check = {
name: string;
description: string;
kind: 'deterministic' | 'llm';
dimension: string;
run(workflow: unknown, ctx: Record<string, unknown>): unknown;
};
type Result = { pass: boolean; comment?: string; applicable?: boolean; errored?: boolean };
const readJson = (file: string): unknown => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
const runDir = resolve(process.argv[2] ?? '.');
const { DETERMINISTIC_CHECKS } = (await import(pathToFileURL(join(repo, SOURCE, 'index.ts')).href)) as { DETERMINISTIC_CHECKS: Check[] };
const prompt = existsSync(join(runDir, 'prompt.txt')) ? readFileSync(join(runDir, 'prompt.txt'), 'utf8') : '';
for (const entry of readdirSync(runDir, { withFileTypes: true })) {
const dir = join(runDir, entry.name);
const workflow = entry.isDirectory() ? readJson(join(dir, 'workflow.json')) : null;
if (!workflow) continue;
const outcomes = [];
for (const check of DETERMINISTIC_CHECKS) {
let status: string;
let comment: string | undefined;
try {
const r = (await check.run(workflow, { prompt })) as Result;
status = r.errored ? 'error' : r.applicable === false ? 'n_a' : r.pass ? 'pass' : 'fail';
comment = r.comment;
} catch (error) {
status = 'error';
comment = error instanceof Error ? error.message : String(error);
}
outcomes.push({ name: check.name, description: check.description, dimension: check.dimension, status, ...(comment ? { comment } : {}) });
}
writeFileSync(join(dir, 'binary-checks.json'), JSON.stringify({ source: SOURCE, outcomes }, null, 1));
const scored = outcomes.filter((o) => o.status === 'pass' || o.status === 'fail');
console.log(`${entry.name}: binary-checks.json written (${scored.filter((o) => o.status === 'pass').length}/${scored.length})`);
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,162 @@
#!/usr/bin/env node
// Take a cropped canvas screenshot of each workflow that a run created.
//
// node capture_canvas.mjs <run-dir>
//
// Reads <run-dir>/<builder>/run.json and writes <run-dir>/<builder>/canvas.png, workflow.json and
// canvas-issues.json.
import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { baseUrl, launchBrowser, signIn } from './n8n_browser.mjs';
const runDir = resolve(process.argv[2] ?? '.');
// The status icons that CanvasNodeStatusIcons.vue draws on a node. n8n computes them from the node
// type description, so they are the verdict of n8n itself on each node.
const ISSUE_MARKERS = ['node-issues', 'node-setup-required', 'node-not-installed', 'node-restricted'];
/**
* The status icon of each node, with the issues that the editor computed for it. The tooltips
* are teleported and show after a delay, so the issues come from the workflow document store.
*/
async function readIssues(page, workflowId) {
return await page.evaluate(
({ markers, workflowId }) => {
const icons = {};
for (const el of document.querySelectorAll(
markers.map((m) => `[data-node-name] [data-test-id="${m}"]`).join(','),
)) {
icons[el.closest('[data-node-name]').dataset.nodeName] = el.dataset.testId;
}
const pinia = document.querySelector('#app')?.__vue_app__?.config.globalProperties.$pinia;
const key = [...(pinia?._s.keys() ?? [])].find((k) =>
k.startsWith(`workflowDocuments/${workflowId}@`),
);
const storeNodes = key ? (pinia._s.get(key).allNodes ?? []) : [];
const nodes = storeNodes
.filter((n) => icons[n.name] || (n.issues && Object.keys(n.issues).length))
.map((n) => ({
name: n.name,
type: n.type,
marker: icons[n.name] ?? null,
issues: n.issues ?? {},
}));
for (const [name, marker] of Object.entries(icons)) {
if (!nodes.some((n) => n.name === name)) nodes.push({ name, marker, issues: {} });
}
return { markers, store: Boolean(key), nodes };
},
{ markers: ISSUE_MARKERS, workflowId },
);
}
const runs = readdirSync(runDir, { withFileTypes: true })
.filter((d) => d.isDirectory() && existsSync(join(runDir, d.name, 'run.json')))
.map((d) => ({
dir: join(runDir, d.name),
meta: JSON.parse(readFileSync(join(runDir, d.name, 'run.json'), 'utf8')),
}))
.filter((r) => r.meta.workflowId);
if (runs.length === 0) {
console.log('No run has a workflowId. Nothing to capture.');
process.exit(0);
}
const browser = await launchBrowser();
const VIEWPORT = { width: 1800, height: 1000 };
const context = await browser.newContext({ viewport: VIEWPORT, deviceScaleFactor: 2 });
const page = await context.newPage();
await signIn(page);
for (const { dir, meta } of runs) {
const id = meta.workflowId;
const res = await page.request.get(`${baseUrl}/rest/workflows/${id}`, {
headers: {
'browser-id': await page.evaluate(() => localStorage.getItem('n8n-browserId') ?? ''),
},
});
if (res.ok())
writeFileSync(join(dir, 'workflow.json'), JSON.stringify((await res.json()).data, null, 1));
else console.log(`${id}: workflow fetch failed with ${res.status()}`);
await page.goto(`${baseUrl}/workflow/${id}`);
try {
await page.locator('[data-test-id="canvas-node"]').first().waitFor({ timeout: 30_000 });
} catch {
console.log(`${id}: no canvas nodes, skipped`);
continue;
}
// Expanding one group re-renders the others, so repeat until no collapsed group is left.
for (let round = 0; round < 5; round++) {
const clicked = await page.evaluate(() => {
const buttons = [...document.querySelectorAll('button[aria-label="Expand"]')];
buttons.forEach((b) => b.click());
return buttons.length;
});
if (clicked === 0) break;
await page.waitForTimeout(400);
}
// The layout moves after groups expand. Fit twice so the second fit sees the final size.
for (let i = 0; i < 2; i++) {
await page.waitForTimeout(1000);
await page
.locator('[data-test-id="zoom-to-fit"]')
.click({ timeout: 5_000 })
.catch(() => {});
}
await page.waitForTimeout(800);
const box = await page.evaluate(() => {
const rects = [...document.querySelectorAll('.vue-flow__node')].map((n) =>
n.getBoundingClientRect(),
);
if (rects.length === 0) return null;
// Labels, subtitles and expanded groups overflow their node element, so use the descendants too.
// Only count what draws: hit areas and hover toolbars are empty boxes around the node.
const draws = (el) =>
el instanceof SVGSVGElement ||
el instanceof HTMLImageElement ||
[...el.childNodes].some((c) => c.nodeType === Node.TEXT_NODE && c.textContent.trim());
const visible = (el) => {
for (let e = el; e && !e.classList.contains('vue-flow__node'); e = e.parentElement) {
const style = getComputedStyle(e);
if (
style.visibility === 'hidden' ||
style.display === 'none' ||
Number(style.opacity) === 0
)
return false;
}
return true;
};
const all = rects.concat(
[...document.querySelectorAll('.vue-flow__node *')]
.filter((el) => draws(el) && visible(el))
.map((n) => n.getBoundingClientRect())
.filter((r) => r.width > 0 && r.height > 0),
);
const left = Math.min(...all.map((r) => r.left));
const top = Math.min(...all.map((r) => r.top));
const right = Math.max(...all.map((r) => r.right));
const bottom = Math.max(...all.map((r) => r.bottom));
return { left, top, right, bottom };
});
const pad = 40;
const clip = box
? {
x: Math.max(0, box.left - pad),
y: Math.max(0, box.top - pad),
width: Math.min(VIEWPORT.width, box.right + pad) - Math.max(0, box.left - pad),
height: Math.min(VIEWPORT.height, box.bottom + pad) - Math.max(0, box.top - pad),
}
: undefined;
await page.screenshot({ path: join(dir, 'canvas.png'), clip });
console.log(`${id}: canvas.png written`);
const issues = await readIssues(page, id);
writeFileSync(join(dir, 'canvas-issues.json'), JSON.stringify(issues, null, 1));
console.log(`${id}: canvas-issues.json written (${issues.nodes.length} nodes with a status icon)`);
}
await browser.close();

View File

@ -0,0 +1,180 @@
#!/usr/bin/env python3
"""Check what the final message of each builder says against the trace, the workflow and the runs.
claims.py <run-dir>
Writes <builder>/claims.json with {message, checks: [{id, ok, text, details}]}. No LLM is involved.
A check is about one kind of claim: the workflow id, the publish state, validation, testing, the
values in `code spans`, and if a claim that the workflow works agrees with the service-call runs.
A message that does not make a kind of claim gets no check for it.
"""
import json
import re
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from build_report import load_run # noqa: E402
from summarize_runs import final_message # noqa: E402
TEST_TOOL = re.compile(r'test|execut|verify|run[-_]?workflow|simulat', re.I)
SHELL_TOOL = re.compile(r'command|shell|bash', re.I)
SCRIPT_PASSED = re.compile(r'"passed"\s*:\s*true|\b\d+ (?:tests? )?passed\b|\ball tests passed\b', re.I)
VALIDATE_TOOL = re.compile(r'validat', re.I)
NEGATION = re.compile(r"\b(not|never|no|nothing|without)\b|n't\b", re.I)
# A clause about what can or will happen later is not a claim about the run.
MODAL = re.compile(r"\b(can|can't|cannot|will|won't|could|would|should|once|until|after that|let me know|need)\b", re.I)
TESTED = re.compile(r'\b(tested|test run|ran|run|runs|executed|simulat\w*|routing checks?|checks? passed)\b', re.I)
VALIDATED = re.compile(r'\b(validat\w*|valid)\b', re.I)
PASSED = re.compile(r'\b(pass\w*|succeed\w*|valid|ok|clean)\b', re.I)
DRAFT = re.compile(r"\b(draft|inactive|unpublished|not (?:yet )?(?:live|active|published)|isn't (?:yet )?(?:live|active)|not turned on)\b", re.I)
LIVE = re.compile(r'\b(?:is|now) (?:live|active|published)\b', re.I)
WORKFLOW_ID = re.compile(r'/workflow/([A-Za-z0-9]{8,})|\bID `?([A-Za-z0-9]{16})`?')
CODE_SPAN = re.compile(r'`([^`\n]{2,80})`')
INVALID = re.compile(r'"valid"\s*:\s*false|"success"\s*:\s*false', re.I)
# A test tool can answer without a transport error and still report a failed execution.
RUN_FAILED = re.compile(r'"status"\s*:\s*"(?:error|failed|crashed|canceled|cancelled)"|"success"\s*:\s*false', re.I)
def clauses(message):
"""The message split into clauses, with Markdown emphasis removed."""
text = re.sub(r'[*_]{1,2}([^*_]+)[*_]{1,2}', r'\1', message or '')
# Streamed messages can join sentences with no space, as in "done.Next".
return [c.strip() for c in re.split(r'(?<=[.!?:])\s+|(?<=[a-z][.!?])(?=[A-Z])|\n+|;\s+|,\s+but\s+|\s+but\s+', text) if c.strip()]
def claim(message, pattern, extra=None):
"""(positive, negative) clauses that match pattern. extra must also match a positive clause."""
pos, neg = [], []
for c in clauses(message):
if not pattern.search(c) or MODAL.search(c):
continue
if NEGATION.search(c):
neg.append(c)
elif extra is None or extra.search(c):
pos.append(c)
return pos, neg
def tool_text(e):
return ' '.join(x if isinstance(x, str) else json.dumps(x, ensure_ascii=False) for x in (e.get('args'), e.get('code'), e.get('result')) if x)
def check(id_, ok, text, details):
return {'id': id_, 'ok': ok, 'text': text, 'details': details}
def quote(clauses_):
return [f'Message: "{c}"' for c in clauses_[:3]]
def id_check(message, meta, workflow):
ids = {a or b for a, b in WORKFLOW_ID.findall(message or '')}
if not ids:
return None
known = set(meta.get('workflowIds') or []) | {meta.get('workflowId'), (workflow or {}).get('id')}
wrong = sorted(ids - known)
return check('M1', not wrong, 'The workflow id in the message is the workflow that the run created',
[f'{i} is not a workflow of this run' for i in wrong] or [f'{", ".join(sorted(ids))} matches.'])
def state_check(message, workflow):
draft, live = DRAFT.findall(message or ''), LIVE.findall(message or '')
if workflow is None or not (draft or live):
return None
active = bool(workflow.get('active'))
says = 'active' if live and not draft else 'a draft'
ok = active == (says == 'active')
return check('M2', ok, f'The message says the workflow is {says}', [f'workflow.json active is {str(active).lower()}.'])
def validation_check(message, events):
pos, _ = claim(message, VALIDATED, PASSED)
if not pos:
return None
calls = [e for e in events if (e['kind'] == 'tool' and VALIDATE_TOOL.search(e.get('name') or ''))
or (e['kind'] == 'code' and VALIDATE_TOOL.search(e.get('code') or ''))]
if not calls:
return check('M3', False, 'The message says the workflow passed validation', quote(pos) + ['No validation tool call in the trace.'])
last = calls[-1]
failed = last.get('error') or INVALID.search(str(last.get('result') or ''))
return check('M3', not failed, 'The message says the workflow passed validation', quote(pos) + [
f'{len(calls)} validation call(s). The last one {"failed" if failed else "passed"}: {last.get("name")}.'])
def test_check(message, events):
pos, neg = claim(message, TESTED)
runs = [e for e in events if e['kind'] == 'tool' and TEST_TOOL.search(e.get('name') or '')
and not VALIDATE_TOOL.search(e.get('name') or '') and not SHELL_TOOL.search(e.get('name') or '')]
scripts = [e for e in events if e['kind'] == 'code' and SCRIPT_PASSED.search(str(e.get('result') or ''))]
if pos:
ok_runs = [e for e in runs if not e.get('error') and not RUN_FAILED.search(tool_text({'result': e.get('result')}))]
if ok_runs:
names = sorted({e['name'] for e in ok_runs})
return check('M4', True, 'The message says the workflow was tested', quote(pos) + [f'{len(ok_runs)} successful test call(s): {", ".join(names)}.'])
if scripts:
return check('M4', True, 'The message says the workflow was tested', quote(pos) + [
f'{len(scripts)} script(s) in the trace report passed tests. These ran outside n8n, not as a workflow execution.'])
return check('M4', False, 'The message says the workflow was tested', quote(pos) + ['No test or execution call in the trace.'])
if neg and runs:
return check('M4', False, 'The message says the workflow was not run', quote(neg) + [f'The trace has {len(runs)} test call(s).'])
if neg:
return check('M4', True, 'The message says the workflow was not run', quote(neg) + ['No test or execution call in the trace.'])
return None
def span_check(message, events, workflow, prompt):
spans = sorted({s.strip() for s in CODE_SPAN.findall(message or '')})
if not spans:
return None
haystack = '\n'.join([json.dumps(workflow or {}, ensure_ascii=False), prompt] + [tool_text(e) for e in events])
loose = lambda s: re.sub(r'\s+', ' ', s).lower()
hay = loose(haystack.replace('\\"', '"'))
missing = [s for s in spans if loose(s) not in hay]
return check('M5', not missing, f'Values in code spans are in the workflow, the prompt or a tool result ({len(spans) - len(missing)} of {len(spans)})',
[f'`{s}` is not in the workflow, the prompt or any tool call' for s in missing])
def works_check(message, wire):
pos, _ = claim(message, re.compile(TESTED.pattern + '|' + VALIDATED.pattern, re.I), None)
if not pos or not wire or wire.get('skipped'):
return None
failed = [s for s in wire.get('scenarios', []) if s.get('kind') == 'contract'
and (s.get('status') != 'success' or any(c.get('ok') is False for c in s.get('checks') or []))]
details = quote(pos) + [f'"{s["name"]}" fails in the service-call run: {s.get("error") or "a check fails"}.' for s in failed]
return check('M6', not failed, 'A claim that the workflow was checked agrees with the service-call runs', details)
def claims_for(sub, meta, prompt):
workflow = json.loads((sub / 'workflow.json').read_text()) if (sub / 'workflow.json').exists() else None
wire = json.loads((sub / 'wire.json').read_text()) if (sub / 'wire.json').exists() else None
try:
message = final_message(sub, meta)
except Exception as e:
return {'message': None, 'error': f'The final message is not readable: {e}', 'checks': []}
data = load_run(sub, meta, {})
events = data.get('events') or []
checks = [id_check(message, meta, workflow), state_check(message, workflow), validation_check(message, events),
test_check(message, events), span_check(message, events, workflow, prompt), works_check(message, wire)]
out = {'message': message, 'checks': [c for c in checks if c]}
if data.get('loadError'):
out['error'] = f'The trace is not readable: {data["loadError"]}'
return out
def main():
run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
prompt = (run_dir / 'prompt.txt').read_text() if (run_dir / 'prompt.txt').exists() else ''
for f in sorted(run_dir.glob('*/run.json')):
sub = f.parent
if sub.name == 'report':
continue
result = claims_for(sub, json.loads(f.read_text()), prompt)
(sub / 'claims.json').write_text(json.dumps(result, indent=1, ensure_ascii=False))
ok = sum(1 for c in result['checks'] if c['ok'])
print(f'{sub.name}: claims.json written ({ok}/{len(result["checks"])})')
if __name__ == '__main__':
main()

View File

@ -0,0 +1,328 @@
#!/usr/bin/env python3
"""Run the full comparison in one command, except the judge step.
compare.py --prompt "..." --expected "..." [--title "..."] [--builders iai,claude,codex]
compare.py --prompt-file p.txt --expected-file e.txt [--scenarios-file s.json]
compare.py --rerun /tmp/builder-compare/<old-run> same prompt, expected outcome and scenarios, new run
Steps: check the setup, run the builders at the same time (--sequential: one at a time),
fetch Instance AI usage, capture the canvases and their node issues, run each workflow with
pinned data (run_scenarios.py), run each workflow in process with recorded API requests
(run_wire.mjs), run the eval binary checks (binary_checks.mts), check the final-message claims
(claims.py), derive the deterministic checks (derive_checks.mjs), write
summary.txt and judge/summary-blind.txt for the judges, and build the report. A local page
shows the runs live (see live_server.py).
Settings come from .env.ai-surfaces (template: .env.ai-surfaces.example in the repository root).
"""
import argparse
import json
import os
import re
import shutil
import subprocess
import sys
import threading
import time
import webbrowser
from datetime import datetime
from pathlib import Path
from live import enabled as live_enabled
from live import is_on, recording
from n8n_api import setting
HERE = Path(__file__).resolve().parent
RUNNERS = {'iai', 'claude', 'codex'}
PRINT_LOCK = threading.Lock()
def slug(text):
words = re.findall(r'[a-z0-9]+', text.lower())[:5]
return '-'.join(words) or 'run'
def step(title):
with PRINT_LOCK:
print(f'\n=== {title} ===', flush=True)
def run(cmd, log=None, prefix=None):
"""Run a script, show its output, and keep a copy in log. Returns the exit code."""
with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, cwd=HERE) as proc:
lines = []
for line in proc.stdout:
shown = line if not prefix or line.startswith(f'[{prefix}]') else f'[{prefix}] {line}'
with PRINT_LOCK:
print(shown, end='', flush=True)
lines.append(line)
if log:
Path(log).write_text(''.join(lines))
return proc.returncode
class Progress:
"""Writes status.json, which the live page reads."""
def __init__(self, run_dir, mode, builders, skipped, screens):
self.path, self.lock = run_dir / 'status.json', threading.Lock()
self.data = {
'mode': mode, 'phase': 'running', 'step': 'Running the builders', 'startedAt': time.time(),
'screens': screens,
'builders': {**{b: {'status': 'waiting'} for b in builders},
**{b: {'status': 'skipped', 'reason': r} for b, r in skipped.items()}},
}
self.save()
def save(self):
tmp = self.path.with_suffix('.tmp')
tmp.write_text(json.dumps(self.data, indent=2))
os.replace(tmp, self.path)
def update(self, **fields):
with self.lock:
self.data.update(fields)
self.save()
def builder(self, name, **fields):
with self.lock:
self.data['builders'][name].update(fields)
self.save()
def finish(self):
with self.lock:
for entry in self.data['builders'].values():
if entry['status'] in ('waiting', 'running'):
entry['status'] = 'failed'
self.data.update(phase='done', step='Done', finishedAt=time.time())
self.save()
def start_live_server(run_dir):
"""Start live_server.py in the background. Returns its URL, or None."""
url_file = run_dir / 'live-url.txt'
url_file.unlink(missing_ok=True)
log = (run_dir / 'live-server.log').open('w')
# A new session, so the page still serves the final report after this script ends.
subprocess.Popen([sys.executable, str(HERE / 'live_server.py'), '--run-dir', str(run_dir),
'--parent-pid', str(os.getpid())],
cwd=HERE, stdin=subprocess.DEVNULL, stdout=log, stderr=log, start_new_session=True)
for _ in range(50):
if url_file.exists() and url_file.read_text().strip():
return url_file.read_text().strip()
time.sleep(0.1)
return None
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--prompt')
ap.add_argument('--prompt-file')
ap.add_argument('--expected')
ap.add_argument('--expected-file')
ap.add_argument('--title')
ap.add_argument('--builders', default=setting('COMPARE_BUILDERS', default='iai,claude,codex'))
ap.add_argument('--run-dir')
ap.add_argument('--scenarios-file', help='Pinned-data scenarios for run_scenarios.py (see SKILL.md).')
ap.add_argument('--rerun', help='Copy prompt.txt, case.json and scenarios.json from an earlier run directory.')
ap.add_argument('--no-live', action='store_true', help='Do not start the live page.')
ap.add_argument('--sequential', action='store_true', default=not is_on('COMPARE_PARALLEL'),
help='Run the builders one at a time. Default: COMPARE_PARALLEL=1 runs them at the same time.')
args = ap.parse_args()
if args.no_live:
os.environ['COMPARE_LIVE'] = '0'
if args.rerun:
old = Path(args.rerun)
prompt = (old / 'prompt.txt').read_text().strip()
case = json.loads((old / 'case.json').read_text())
else:
prompt = args.prompt or (Path(args.prompt_file).read_text().strip() if args.prompt_file else None)
expected = args.expected or (Path(args.expected_file).read_text().strip() if args.expected_file else None)
if not prompt or not expected:
sys.exit('Give a prompt and an expected outcome (--prompt/--prompt-file and --expected/--expected-file).')
case = {'title': args.title or prompt[:60], 'expected': expected}
builders = [b.strip() for b in args.builders.split(',') if b.strip()]
unknown = set(builders) - RUNNERS
if unknown:
sys.exit(f'Unknown builders: {", ".join(sorted(unknown))}. Use iai, claude, codex.')
root = Path(setting('COMPARE_RUNS_DIR', default='/tmp/builder-compare'))
# The child scripts run from HERE, so a relative path must not reach them.
run_dir = (Path(args.run_dir) if args.run_dir else root / f'{datetime.now():%Y%m%d-%H%M}-{slug(case["title"])}').resolve()
run_dir.mkdir(parents=True, exist_ok=True)
(run_dir / 'prompt.txt').write_text(prompt + '\n')
(run_dir / 'case.json').write_text(json.dumps(case, indent=2))
scenarios = Path(args.scenarios_file) if args.scenarios_file else (Path(args.rerun) / 'scenarios.json' if args.rerun else None)
if scenarios and scenarios.exists():
json.loads(scenarios.read_text())
shutil.copy(scenarios, run_dir / 'scenarios.json')
elif args.scenarios_file:
sys.exit(f'No scenarios file at {scenarios}.')
print(f'run directory: {run_dir}', flush=True)
step('Check the setup')
check = subprocess.run([sys.executable, 'prepare.py'], cwd=HERE, capture_output=True, text=True)
print(check.stdout, end='')
try:
status = json.loads(check.stdout)
except json.JSONDecodeError:
sys.exit(check.stderr or 'prepare.py failed')
if not status.get('login'):
sys.exit('Cannot log in to n8n. Is it running? Check N8N_BASE_URL, N8N_EVAL_EMAIL and N8N_EVAL_PASSWORD.')
isolation = setting('COMPARE_ISOLATION', default='container')
skipped = {}
if 'iai' in builders and not status.get('instanceAiEnabled'):
skipped['iai'] = 'Instance AI is not enabled on this instance.'
for cli in ('claude', 'codex'):
if cli in builders and not status.get('mcpTokenWorks'):
skipped[cli] = status.get('mcpTokenHelp', 'The MCP token does not work.')
elif cli in builders and not status['keys'].get(cli):
skipped[cli] = f'No API key. Set {cli.upper()}_API_KEY in .env.ai-surfaces.'
if isolation == 'container' and not status.get('docker'):
skipped.setdefault('codex', 'Docker is not running. Codex runs only in a container.')
if 'claude' in builders and 'claude' not in skipped:
print('Docker is not running. Claude falls back to clean-process isolation.', flush=True)
isolation = 'clean'
if isolation == 'clean':
skipped.setdefault('codex', 'Codex runs only in a container. Clean isolation is for Claude only.')
if not status.get('claudeOnHost'):
skipped.setdefault('claude', 'Clean isolation runs claude on the host, and claude is not on PATH.')
skipped = {name: reason for name, reason in skipped.items() if name in builders}
for name, reason in skipped.items():
print(f'skip {name}: {reason}', flush=True)
builders = [b for b in builders if b not in skipped]
parallel = not args.sequential and len(builders) > 1
cli_mode = setting('COMPARE_CLI_MODE', default='interactive')
screens = {b: ('shot' if recording() else None) if b == 'iai'
else ('cast' if isolation == 'container' and cli_mode == 'interactive' else None) for b in builders}
progress = Progress(run_dir, 'parallel' if parallel else 'sequential', builders, skipped, screens)
live_url = None
if live_enabled() and not args.no_live:
live_url = start_live_server(run_dir)
if live_url:
print(f'live page: {live_url}/report/live', flush=True)
webbrowser.open(f'{live_url}/report/live')
else:
print(f'the live page did not start, see {run_dir / "live-server.log"}', flush=True)
results = {}
try:
if parallel and isolation == 'container' and {'claude', 'codex'} & set(builders):
# Both CLI runners need the image. Build it once before they start.
from run_cli_builder import ensure_image
ensure_image(False)
def run_builder(builder):
progress.builder(builder, status='running', startedAt=time.time())
started = time.time()
if builder == 'iai':
cmd = [sys.executable, 'run_instance_ai.py', '--out', str(run_dir / 'iai'),
'--prompt-file', str(run_dir / 'prompt.txt')]
else:
cmd = [sys.executable, 'run_cli_builder.py', '--builder', builder, '--out', str(run_dir / builder),
'--prompt-file', str(run_dir / 'prompt.txt'), '--isolation', isolation]
code = run(cmd, run_dir / f'{builder}.log', prefix=builder if parallel else None)
# run_cli_builder.py exits 0 when the CLI fails. run.json has the exit code of the CLI.
meta_file = run_dir / builder / 'run.json'
if code == 0 and meta_file.exists():
code = json.loads(meta_file.read_text()).get('exitCode') or 0
results[builder] = {'exitCode': code, 'seconds': round(time.time() - started)}
progress.builder(builder, status='done' if code == 0 else 'failed', finishedAt=time.time(), exitCode=code)
if parallel:
step(f'Run {", ".join(builders)} at the same time')
threads = [threading.Thread(target=run_builder, args=(b,)) for b in builders]
for t in threads:
t.start()
for t in threads:
t.join()
else:
for builder in builders:
step(f'Run {builder}')
run_builder(builder)
if 'iai' in builders and (run_dir / 'iai' / 'run.json').exists():
step('Fetch Instance AI usage from LangSmith')
progress.update(step='Fetching Instance AI usage')
if status['keys'].get('langsmith'):
# LangSmith receives traces with a delay.
finished = progress.data['builders']['iai'].get('finishedAt') or time.time()
time.sleep(max(0.0, 30 - (time.time() - finished)))
run([sys.executable, 'langsmith_usage.py', '--run-dir', str(run_dir / 'iai')])
else:
print('LANGSMITH_API_KEY is not set. The report has no Instance AI cost or tokens.', flush=True)
step('Capture the canvases')
progress.update(step='Capturing the canvases')
node = shutil.which('node')
if node:
run([node, 'capture_canvas.mjs', str(run_dir)])
else:
print('node is not on PATH. No canvases.', flush=True)
if not is_on('COMPARE_SCENARIOS'):
print('COMPARE_SCENARIOS=0. The workflows do not run with pinned data.', flush=True)
elif not node or not status.get('mcpTokenWorks'):
print('Pinned-data runs need node and a working MCP token. Skipped.', flush=True)
else:
step('Run each workflow with pinned data')
progress.update(step='Running the workflows with pinned data')
run([sys.executable, 'run_scenarios.py', str(run_dir)], run_dir / 'scenarios.log')
if node and is_on('COMPARE_WIRE'):
step('Run each workflow in process with recorded API requests')
progress.update(step='Running the workflows with recorded API requests')
run([node, 'run_wire.mjs', str(run_dir)], run_dir / 'wire.log')
elif node:
print('COMPARE_WIRE=0. No in-process runs with recorded API requests.', flush=True)
pnpm = shutil.which('pnpm')
if pnpm and is_on('COMPARE_BINARY_CHECKS'):
step('Run the eval binary checks')
progress.update(step='Running the eval binary checks')
run([pnpm, '--filter', '@n8n/instance-ai', 'exec', 'tsx', str(HERE / 'binary_checks.mts'), str(run_dir)],
run_dir / 'binary-checks.log')
step('Check the final-message claims')
run([sys.executable, 'claims.py', str(run_dir)])
if node:
step('Derive the deterministic checks')
progress.update(step='Deriving the deterministic checks')
run([node, 'derive_checks.mjs', str(run_dir)])
step('Summarize the runs for the judges')
summary = subprocess.run([sys.executable, 'summarize_runs.py', str(run_dir)], cwd=HERE,
capture_output=True, text=True)
(run_dir / 'summary.txt').write_text(summary.stdout)
run([sys.executable, 'summarize_runs.py', str(run_dir), '--blind'])
step('Build the report')
# The agent does the price and judge steps after this script. The page shows them as queued until then.
progress.update(step='Building the report', postRun={'prices': 'queued', 'judge': 'queued'})
run([sys.executable, 'build_report.py', str(run_dir)])
finally:
progress.finish()
models_file = run_dir / 'models.json'
print(json.dumps({
'runDir': str(run_dir),
'report': str(run_dir / 'report' / 'index.html'),
'live': f'{live_url}/report/index.html' if live_url else None,
'parallel': parallel,
'summary': str(run_dir / 'summary.txt'),
'blindSummary': str(run_dir / 'judge' / 'summary-blind.txt'),
'models': json.loads(models_file.read_text()) if models_file.exists() else [],
'results': results,
'skipped': skipped,
'next': 'Run build_report.py --step prices=running, add published prices for the CLI models to case.json, '
'run build_report.py --step prices=done --step judge=running, give judge/summary-blind.txt to '
'3 judges with a fresh context, write judge/verdicts-1.json to verdicts-3.json, run '
'merge_verdicts.py, then run build_report.py --step judge=done.',
}, indent=2))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,743 @@
#!/usr/bin/env node
// Deterministic checks for each workflow that a run created. No LLM is involved.
//
// node derive_checks.mjs <run-dir>
//
// Reads <builder>/workflow.json, canvas-issues.json and exec.json, and the optional wire.json,
// claims.json and binary-checks.json. Writes <builder>/auto-checks.json. A group with gating true
// changes the verdict. The other groups show a separate score.
// correctness node definitions, canvas issues, graph and expression references (gating)
// execution the pinned-data scenarios of run_scenarios.py (gating)
// contract service-call requests of run_wire.mjs (gating)
// edge, ai edge-case and real-AI scenarios of run_wire.mjs
// claims the final message against the trace, from claims.py
// standards rules from n8n-io/skills and the workflow-sdk best practices
// binary the deterministic eval binary checks, from binary_checks.mts
import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import {
LOGIC_TYPES,
SINGLE_ITEM_TYPES,
TRIGGER_FIXTURES,
downstream,
graph,
isManualTrigger,
isTrigger,
loadSdk,
lookup,
reachable,
repo,
sample,
shortType,
showPath,
strings,
typeMatches,
workflowReferences,
} from './sdk.mjs';
const VERSION = 1;
const SKILLS = 'https://github.com/n8n-io/skills/blob/27b6b9f429ac0be90cae764512dbc579d7107e95/';
const GUIDES = 'packages/@n8n/workflow-sdk/src/prompts/best-practices/guides/';
const LC = '@n8n/n8n-nodes-langchain.';
const src = {
validator: { label: '@n8n/workflow-sdk validateWorkflow with the node definitions' },
canvas: { label: 'Node issues in the n8n editor (canvas status icons)' },
graph: { label: 'Workflow connections' },
expressions: { label: 'Expression references in node parameters' },
execution: { label: 'Real output of the pinned-data runs (exec.json)' },
wire: { label: 'In-process run with recorded API fixtures (wire.json)' },
claims: { label: 'Final message, trace tool calls and workflow (claims.json)' },
errorHandling: { label: 'n8n-error-handling-official', url: `${SKILLS}skills/n8n-error-handling-official/SKILL.md` },
expressionsSkill: { label: 'n8n-expressions-official', url: `${SKILLS}skills/n8n-expressions-official/SKILL.md` },
structured: {
label: 'n8n-ai-nodes-official STRUCTURED_OUTPUT.md',
url: `${SKILLS}skills/n8n-ai-nodes-official/references/STRUCTURED_OUTPUT.md`,
},
memory: { label: 'n8n-ai-nodes-official MEMORY.md', url: `${SKILLS}skills/n8n-ai-nodes-official/references/MEMORY.md` },
comms: {
label: 'n8n-node-configuration-official COMMS_NODES.md',
url: `${SKILLS}skills/n8n-node-configuration-official/references/COMMS_NODES.md`,
},
triage: { label: 'workflow-sdk best practices: triage', path: `${GUIDES}triage.ts` },
chatbot: { label: 'workflow-sdk best practices: chatbot', path: `${GUIDES}chatbot.ts` },
multiItem: {
label: 'WhatsAppTrigger.node.ts emits one item for each webhook change',
path: 'packages/nodes-base/nodes/WhatsApp/WhatsAppTrigger.node.ts',
},
};
const check = (id, ok, text, source, details = [], extra = {}) => ({ id, ok, text, source, details, ...extra });
const plural = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`;
const readJson = (file) => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
// --- Node kinds ----------------------------------------------------------------
const isAiRoot = (g, node) => node.type.startsWith(LC) && !g.subnodes.has(node.name) && g.aiInputs(node.name, 'ai_languageModel').length > 0;
const isBranch = (node) => typeMatches(node.type, 'if') || typeMatches(node.type, 'switch');
/** Nodes that call a service: not logic, not a trigger, not a sub-node. AI root nodes count. */
const isNetwork = (g, node) =>
!g.subnodes.has(node.name) && !isTrigger(node.type) && !LOGIC_TYPES.has(node.type) && !typeMatches(node.type, 'stickyNote');
// Triggers that start one execution with one item. Other triggers can batch several events.
const SINGLE_ITEM_TRIGGERS = ['manualTrigger', 'scheduleTrigger', 'cron', 'interval', 'webhook', 'chatTrigger', 'formTrigger', 'telegramTrigger', 'slackTrigger'];
const MESSAGING_TRIGGERS = ['whatsAppTrigger', 'telegramTrigger', 'slackTrigger', 'chatTrigger', 'discordTrigger', 'microsoftTeamsTrigger', 'gmailTrigger'];
const ITEM_SOURCES = ['splitOut', 'code', 'itemLists'];
/** Names of the nodes that can output more than one item in one execution. */
function multiItemNodes(g) {
const multi = new Set();
const order = [...reachable(g)];
let grew = true;
while (grew) {
grew = false;
for (const name of order) {
if (multi.has(name)) continue;
const node = g.byName.get(name);
if (!node || g.subnodes.has(name) || SINGLE_ITEM_TYPES.has(node.type) || node.executeOnce) continue;
const fromTrigger = isTrigger(node.type) && !SINGLE_ITEM_TRIGGERS.some((t) => typeMatches(node.type, t));
const source = ITEM_SOURCES.some((t) => typeMatches(node.type, t)) || (isNetwork(g, node) && !isAiRoot(g, node) && /^(read|getAll|search|list|getMany)$/.test(node.parameters?.operation ?? ''));
if (fromTrigger || source || g.mainParents(name).some((p) => multi.has(p))) {
multi.add(name);
grew = true;
}
}
}
return multi;
}
/** Names of the nodes upstream of name. A sub-node uses the nodes upstream of the root that it serves. */
function upstream(g, name) {
const roots = new Set([name]);
for (let grew = true; grew; ) {
grew = false;
for (const e of g.edges) {
if (e.type.startsWith('ai_') && roots.has(e.from) && !roots.has(e.to)) {
roots.add(e.to);
grew = true;
}
}
}
const seen = new Set();
const queue = [...roots];
while (queue.length) {
for (const parent of g.mainParents(queue.shift())) {
if (!seen.has(parent)) {
seen.add(parent);
queue.push(parent);
}
}
}
return seen;
}
// --- Correctness ---------------------------------------------------------------
function checkDefinitions(workflow) {
const loaded = loadSdk();
if (!loaded || loaded.missingDirs.length) {
const missing = loaded ? loaded.missingDirs.map((d) => d.replace(repo + '/', '')) : ['packages/@n8n/workflow-sdk/dist'];
return check('C1', null, 'Node parameters match the node definitions', src.validator, [
`Not checked. Build these first: ${missing.join(', ')}.`,
]);
}
const result = loaded.sdk.validateWorkflow(workflow, { allowDisconnectedNodes: true, allowNoTrigger: true });
const problems = [...result.errors, ...result.warnings.filter((w) => w.severity !== 'informational')];
const details = problems.map((p) =>
p.nodeName && !p.message.includes(`"${p.nodeName}"`) ? `${p.nodeName}: ${p.message}` : p.message,
);
const info = result.warnings.filter((w) => w.severity === 'informational').map((w) => `Info: ${w.message}`);
return check(
'C1',
problems.length === 0,
problems.length ? `${plural(problems.length, 'node parameter problem')} against the node definitions` : 'Node parameters match the node definitions',
src.validator,
details.concat(info),
);
}
const isEmpty = (value) =>
value === undefined || value === null || value === '' || (typeof value === 'object' && value.__rl && (value.value === '' || value.value == null));
/** Sort the editor issues into setup work for the user and real problems. */
function classifyCanvas(workflow, canvas) {
const byName = new Map((workflow.nodes ?? []).map((n) => [n.name, n]));
const setup = [];
const problems = [];
for (const node of canvas.nodes) {
if (node.marker === 'node-not-installed') problems.push(`${node.name}: the node type is not installed`);
if (node.marker === 'node-restricted') problems.push(`${node.name}: the node type is restricted`);
for (const [kind, issue] of Object.entries(node.issues ?? {})) {
if (kind === 'credentials') {
for (const type of Object.keys(issue)) setup.push(`${node.name}: connect ${type} credentials`);
} else if (kind === 'parameters') {
for (const [param, messages] of Object.entries(issue)) {
const value = byName.get(node.name)?.parameters?.[param];
for (const message of messages) {
if (/is required/i.test(message) && isEmpty(value)) setup.push(`${node.name}: ${message.replace(/\.$/, '')} (left for the user)`);
else problems.push(`${node.name}: ${message}`);
}
}
} else if (issue === true || (issue && typeof issue === 'object' && Object.keys(issue).length)) {
const text = typeof issue === 'object' ? Object.values(issue).flat().join(' ') : '';
problems.push(`${node.name}: ${kind === 'typeUnknown' ? 'n8n does not know this node type' : `${kind} issue`}${text ? ` (${text})` : ''}`);
}
}
}
return { setup, problems };
}
function checkCanvas(workflow, canvas) {
if (!canvas) return [check('C2', null, 'The n8n editor shows no node errors', src.canvas, ['Not checked. Run capture_canvas.mjs first.'])];
if (!canvas.store && canvas.nodes.some((n) => n.marker === 'node-issues')) {
return [check('C2', null, 'The n8n editor shows no node errors', src.canvas, [
'The editor store was not readable, so the node issues are not known. Icons: ' +
canvas.nodes.map((n) => `${n.name} (${n.marker})`).join(', '),
])];
}
const { setup, problems } = classifyCanvas(workflow, canvas);
return [
check('C2', problems.length === 0, problems.length ? `The n8n editor shows ${plural(problems.length, 'node error')}` : 'The n8n editor shows no node errors', src.canvas, problems),
check('C2s', true, setup.length ? `Setup left for the user: ${plural(setup.length, 'item')}` : 'No setup left for the user', src.canvas, setup, { level: 'info' }),
];
}
function checkReachable(g) {
if (g.triggers.length === 0) return check('C3', false, 'The workflow has a trigger', src.graph, ['No trigger node.']);
const seen = reachable(g);
const orphans = g.nodes.filter((n) => !seen.has(n.name)).map((n) => n.name);
return check(
'C3',
orphans.length === 0,
orphans.length ? `${plural(orphans.length, 'node')} cannot be reached from a trigger` : 'Every node can be reached from a trigger',
src.graph,
orphans.map((n) => `${n} is not connected to the flow`),
);
}
function checkNamedRefs(workflow, g, refs) {
const allNames = new Set((workflow.nodes ?? []).map((n) => n.name));
const problems = [];
for (const { node, param, ref } of refs) {
if (ref.kind !== 'named') continue;
if (!allNames.has(ref.node)) problems.push(`${node} › ${param}: no node is called "${ref.node}"`);
else if (!g.byName.has(ref.node)) problems.push(`${node} › ${param}: "${ref.node}" is disabled`);
else if (!/isExecuted/.test(ref.raw) && !upstream(g, node).has(ref.node))
problems.push(`${node} › ${param}: "${ref.node}" does not run before this node, so the reference has no data`);
}
const named = refs.filter((r) => r.ref.kind === 'named').length;
return check(
'C4',
problems.length === 0,
problems.length ? `${plural(problems.length, 'node reference')} cannot resolve` : `All ${plural(named, 'node reference')} name a node that runs before them`,
src.expressions,
[...new Set(problems)],
);
}
const DEFINITIVE = new Set(['executed', 'fixture', 'case', 'output-parser']);
const INDICATIVE = new Set(['node-schema', 'prompt-example', 'ai-root']);
// Outputs with the real number of array items. The other provenances are schema samples.
const REAL_OUTPUT = new Set(['executed', 'fixture', 'case']);
/** The output items of a node across all scenarios, with their provenance. */
function outputsOf(name, g, exec) {
const items = [];
for (const scenario of exec?.scenarios ?? []) {
const out = scenario.nodes?.[name];
if (out?.status === 'success' && Array.isArray(out.items)) items.push(...out.items.filter((i) => i && typeof i === 'object'));
}
const node = g.byName.get(name);
let provenance = exec?.executed?.includes(name) ? 'executed' : exec?.pinned?.[name];
if (items.length && provenance) return { items, provenance };
if (TRIGGER_FIXTURES[node.type]) return { items: TRIGGER_FIXTURES[node.type], provenance: 'fixture' };
const schema = loadSdk()?.sdk.discoverOutputSchemaForNode(node.type, node.typeVersion, node.parameters ?? {});
if (schema) return { items: [sample(schema, 'item')], provenance: 'node-schema' };
return { items: [], provenance: 'unknown' };
}
function checkFieldPaths(g, refs, exec) {
const missing = [];
const unconfirmed = [];
let checked = 0;
let unknown = 0;
const seen = new Set();
for (const { node, param, ref, code } of refs) {
if (code || !ref.path?.length) continue;
if (ref.kind !== 'named' && g.subnodes.has(node)) continue;
const sources = ref.kind === 'named' ? [ref.node] : g.mainParents(node);
if (!sources.length || sources.some((s) => !g.byName.has(s))) continue;
const key = `${node}|${param}|${ref.raw}`;
if (seen.has(key)) continue;
seen.add(key);
let found = false;
let anyUnknown = false;
const misses = [];
for (const source of sources) {
const { items, provenance } = outputsOf(source, g, exec);
if (!items.length || provenance === 'unknown' || provenance === 'empty' || provenance === 'placeholder') {
anyUnknown = true;
continue;
}
const results = items.map((item) => lookup(item, ref.path, { sample: !REAL_OUTPUT.has(provenance) }));
if (results.includes('found')) found = true;
else if (results.includes('unknown')) anyUnknown = true;
else misses.push({ source, provenance });
}
if (found) checked++;
else if (anyUnknown || !misses.length) unknown++;
else {
checked++;
const line = `${node} › ${param}: ${ref.raw.trim()} reads "${showPath(ref.path)}", which is not in the output of ${misses.map((m) => `${m.source} (${m.provenance})`).join(' or ')}`;
if (misses.some((m) => DEFINITIVE.has(m.provenance))) missing.push(line);
else if (misses.some((m) => INDICATIVE.has(m.provenance))) unconfirmed.push(`Unconfirmed: ${line}`);
}
}
const note = exec?.scenarios?.length ? [] : ['No pinned-data run, so node outputs come from fixtures and node schemas only.'];
return check(
'C5',
missing.length === 0,
missing.length
? `${plural(missing.length, 'expression')} read a field that the source node does not output`
: `${plural(checked, 'expression field')} found in the source node output${unknown ? `, ${unknown} not checkable` : ''}`,
src.execution,
[...missing, ...unconfirmed, ...note],
);
}
const SECRET_PATTERNS = [
/sk-(?:proj-|ant-)?[A-Za-z0-9_-]{20,}/,
/xox[abpr]-[A-Za-z0-9-]{10,}/,
/AIza[0-9A-Za-z_-]{35}/,
/gh[pousr]_[A-Za-z0-9]{36}/,
/Bearer\s+(?!\{\{)[A-Za-z0-9._~+/-]{24,}/,
/-----BEGIN [A-Z ]*PRIVATE KEY-----/,
];
function checkSecrets(g) {
const hits = [];
for (const node of g.nodes) {
for (const { path, value } of strings(node.parameters ?? {})) {
if (SECRET_PATTERNS.some((re) => re.test(value))) hits.push(`${node.name} › ${path}: a value looks like a key or token`);
}
}
return check('C6', hits.length === 0, hits.length ? 'Secrets are written into node parameters' : 'No secrets in node parameters', src.expressions, hits);
}
// --- Execution -----------------------------------------------------------------
function executionChecks(exec) {
if (!exec) return [check('E0', null, 'The workflow runs with pinned data', src.execution, ['Not run. Run run_scenarios.py first.'])];
if (exec.skipped) return [check('E0', null, 'The workflow runs with pinned data', src.execution, [exec.skipped])];
const pinned = Object.entries(exec.pinned ?? {}).map(([n, p]) => `${n} (${p})`);
const intro = [`Pinned: ${pinned.join(', ') || 'nothing'}.`, `Ran for real: ${(exec.executed ?? []).join(', ') || 'nothing'}.`];
const out = [];
exec.scenarios.forEach((s, i) => {
const errors = Object.entries(s.nodes ?? {}).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
const ran = Object.keys(s.nodes ?? {});
const ok = s.status === 'success' && errors.length === 0;
out.push(check(`E${i + 1}`, ok, `"${s.name}" runs without an error`, src.execution, [
...(i === 0 ? intro : []),
`Status: ${s.status}${s.error ? ` (${s.error})` : ''}.`,
...errors,
`Nodes that ran: ${ran.join(', ')}.`,
], { executionId: s.executionId }));
(s.checks ?? []).forEach((c, j) => {
const verb = c.kind === 'reach' ? 'reaches' : 'does not reach';
out.push(check(`E${i + 1}.${j + 1}`, c.ok, `"${s.name}" ${verb} a ${c.type} node`, src.execution, []));
});
});
return out;
}
// --- In-process runs, eval checks and claims -------------------------------------
const WIRE_KINDS = {
contract: { id: 'W', title: 'Service calls', gating: true, note: 'The workflow runs in process. Service nodes run for real against recorded API fixtures, so the check sees each HTTP request that a node sends. The AI node is pinned. A failure here changes the verdict.' },
edge: { id: 'X', title: 'Edge cases', gating: false, note: 'Inputs that the prompt does not describe but the trigger can send: several messages in one call, status updates, empty data. This score does not change the verdict.' },
ai: { id: 'A', title: 'Real AI runs', gating: false, note: 'The AI node calls the real model on labeled questions. Service nodes use the recorded API fixtures. Model output can change between runs, so this score does not change the verdict.' },
};
function wireCheckText(name, c) {
if (c.kind === 'reach') return `"${name}" reaches a ${c.type} node`;
if (c.kind === 'skip') return `"${name}" does not reach a ${c.type} node`;
if (c.kind === 'noWrites') return `"${name}" sends no write request`;
if (c.kind === 'eachItem') return `"${name}" sends a request for each of the ${c.ids.length} items`;
const what = c.contains?.length ? ` that carries ${c.contains.map((m) => m.split(' or ').map((x) => (x.startsWith('@') ? `\`${x}\`` : x)).join(' or ')).join(', ')}` : '';
const count = c.count === undefined ? 'a request' : plural(c.count, 'request');
return `"${name}" sends ${count} from a ${c.type} node${what}`;
}
function requestLines(s) {
return (s.requests ?? []).map((r) => `Request: ${r.node} › ${r.method} ${r.url}${r.answeredBy && r.answeredBy !== 'fixture' ? ` (${r.answeredBy})` : ''}`);
}
function wireGroups(wire) {
if (!wire) return [];
const groups = [];
for (const [kind, meta] of Object.entries(WIRE_KINDS)) {
const scenarios = (wire.scenarios ?? []).filter((s) => s.kind === kind);
if (!scenarios.length && !(kind === 'contract' && wire.skipped)) continue;
const checks = [];
if (wire.skipped) checks.push(check(`${meta.id}0`, null, 'The workflow runs in process', src.wire, [wire.skipped]));
scenarios.forEach((s, i) => {
const id = `${meta.id}${i + 1}`;
if (s.status === 'skipped') {
checks.push(check(id, null, `"${s.name}" runs`, src.wire, [s.error]));
return;
}
const errors = Object.entries(s.nodes ?? {}).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
const ai = Object.entries(s.aiOutputs ?? {}).map(([node, items]) => `AI output of ${node}: ${JSON.stringify(items?.[0]?.output ?? items?.[0] ?? null)}`);
const ran = Object.entries(s.nodes ?? {}).map(([name, n]) => `${name} (${plural((n.outputCounts ?? []).reduce((a, b) => a + b, 0), 'item')})`);
checks.push(check(id, s.status === 'success' && !errors.length, `"${s.name}" runs without an error`, src.wire, [
...(s.why ? [s.why[0].toUpperCase() + s.why.slice(1)] : []),
`Status: ${s.status}${s.error ? ` (${s.error})` : ''}.`,
...errors,
`Nodes that ran: ${ran.join(', ') || 'none'}.`,
...ai,
...requestLines(s),
...(s.blocked?.length ? [`Blocked network calls: ${s.blocked.join(', ')}`] : []),
]));
(s.checks ?? []).forEach((c, j) => checks.push(check(`${id}.${j + 1}`, c.ok, wireCheckText(s.name, c), src.wire, c.details ?? [])));
});
const scored = checks.filter((c) => c.ok !== null);
const filled = kind === 'contract' && wire.filled?.length ? [`Setup values filled for the run: ${wire.filled.join(', ')}.`] : [];
groups.push({
key: kind === 'contract' ? 'contract' : kind,
title: meta.title,
gating: meta.gating,
note: [meta.note, ...filled].join(' '),
...(meta.gating ? {} : { score: { passed: scored.filter((c) => c.ok).length, total: scored.length } }),
checks,
});
}
return groups;
}
function binaryGroup(binary) {
if (!binary?.outcomes) return null;
const measured = binary.outcomes.filter((o) => o.status !== 'n_a');
const checks = measured.map((o, i) =>
check(`B${i + 1}`, o.status === 'error' ? null : o.status === 'pass', `${o.name}: ${o.description}`, { label: `Instance AI eval binary checks (${o.dimension})`, path: binary.source }, o.comment ? [o.comment] : []),
);
const scored = checks.filter((c) => c.ok !== null);
const na = binary.outcomes.length - measured.length;
return {
key: 'binary',
title: 'Eval binary checks',
gating: false,
note: `The deterministic binary checks of the Instance AI eval suite.${na ? ` ${na} do not apply to this workflow.` : ''} This score does not change the verdict.`,
score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
checks,
};
}
function claimsGroup(claims) {
if (!claims) return null;
const checks = claims.checks.map((c) => check(c.id, c.ok, c.text, src.claims, c.details));
if (claims.error) checks.unshift(check('M0', null, 'The final message and trace are readable', src.claims, [claims.error]));
const scored = checks.filter((c) => c.ok !== null);
return {
key: 'claims',
title: 'Final message claims',
gating: false,
note: 'What the final message says about the build, compared with the trace tool calls, the workflow and the service-call runs. This score does not change the verdict.',
score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
checks,
};
}
// --- Standards -----------------------------------------------------------------
function unattended(g) {
return g.triggers.some((t) => !isManualTrigger(t.type) && !typeMatches(t.type, 'chatTrigger') && !typeMatches(t.type, 'formTrigger'));
}
function errorWorkflowRule(workflow, g) {
if (!unattended(g)) return null;
const set = Boolean(workflow.settings?.errorWorkflow);
return check('S1', set, 'An unattended workflow has an error workflow', src.errorHandling, set ? [] : ['settings.errorWorkflow is not set, so a failure goes nowhere.']);
}
function errorOutputRule(g) {
const problems = [];
for (const node of g.nodes) {
if (node.onError === 'continueRegularOutput') problems.push(`${node.name}: onError continueRegularOutput passes the error on as normal data`);
// The error output comes after the normal outputs, so it is never output 0.
if (node.onError === 'continueErrorOutput' && !g.mainChildren(node.name).some((e) => e.output > 0))
problems.push(`${node.name}: onError continueErrorOutput has no node on its error output`);
}
if (!g.nodes.some((n) => n.onError)) return null;
return check('S12', problems.length === 0, 'Failed nodes go to a wired error output, not on as data', src.errorHandling, problems);
}
function retryRule(g) {
const nodes = g.nodes.filter((n) => isNetwork(g, n));
if (!nodes.length) return null;
const missing = nodes.filter((n) => !n.retryOnFail).map((n) => `${n.name}: no retryOnFail`);
return check('S2', missing.length === 0, `Nodes that call a service retry on failure (${nodes.length - missing.length} of ${nodes.length})`, src.errorHandling, missing);
}
function namedRefRule(refs) {
const uses = refs.filter((r) => !r.code && (r.ref.kind === 'json' || r.ref.kind === 'input'));
if (!refs.some((r) => !r.code)) return null;
const where = [...new Set(uses.map((r) => `${r.node} › ${r.param}`))];
return check('S3', uses.length === 0, uses.length ? `${plural(where.length, 'parameter')} read $json instead of a node name` : 'Expressions read data by node name', src.expressionsSkill, where);
}
function parserParams(g, root) {
const parser = g.aiInputs(root.name, 'ai_outputParser')[0];
return parser ? { parser, p: parser.parameters ?? {} } : null;
}
function structuredRule(g) {
const roots = g.nodes.filter((n) => isAiRoot(g, n) && g.mainChildren(n.name).length);
if (!roots.length) return null;
const problems = [];
for (const root of roots) {
const found = parserParams(g, root);
if (!found) {
const declared = strings(root.parameters ?? {}).some(({ value }) => /json/i.test(value));
if (declared) problems.push(`${root.name}: the JSON format is only in the prompt, with no structured output parser`);
continue;
}
if ((found.p.schemaType ?? 'fromJson') !== 'manual') problems.push(`${found.parser.name}: schemaType is not 'manual'`);
if (!found.p.autoFix) problems.push(`${found.parser.name}: autoFix is off`);
}
return check('S4', problems.length === 0, 'Structured AI output uses a manual schema with autoFix', src.structured, problems);
}
const ATTRIBUTION_FIELDS = ['appendAttribution', 'includeLinkToWorkflow'];
const definitionCache = new Map();
/** The text of the node definition file for a node, or ''. */
function definitionText(node) {
const [pkg, short] = node.type.startsWith(LC) ? ['@n8n/nodes-langchain', node.type.slice(LC.length)] : ['nodes-base', shortType(node.type)];
const pkgDir = node.type.startsWith(LC) ? '@n8n/n8n-nodes-langchain' : 'n8n-nodes-base';
const dir = join(repo, 'packages', pkg, 'dist/node-definitions/nodes', pkgDir, short, `v${String(node.typeVersion ?? 1).replace('.', '')}`);
const snake = (s) => String(s ?? '').replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase();
const p = node.parameters ?? {};
const files = [join(dir, `resource_${snake(p.resource)}`, `operation_${snake(p.operation)}.ts`), `${dir}.ts`];
if (!p.resource && p.operation && existsSync(dir)) {
for (const entry of readdirSync(dir)) if (entry.startsWith('resource_')) files.push(join(dir, entry, `operation_${snake(p.operation)}.ts`));
}
const key = files.join('|');
if (!definitionCache.has(key)) definitionCache.set(key, files.filter(existsSync).map((f) => readFileSync(f, 'utf8')).join('\n'));
return definitionCache.get(key);
}
function findKey(value, key) {
if (!value || typeof value !== 'object') return undefined;
if (key in value) return value[key];
for (const v of Object.values(value)) {
const hit = findKey(v, key);
if (hit !== undefined) return hit;
}
return undefined;
}
function attributionRule(g) {
const nodes = [];
for (const node of g.nodes) {
const text = definitionText(node);
const field = ATTRIBUTION_FIELDS.find((f) => text.includes(`${f}?:`));
if (field) nodes.push({ node, field });
}
if (!nodes.length) return null;
const on = nodes.filter(({ node, field }) => findKey(node.parameters, field) !== false).map(({ node, field }) => `${node.name}: ${field} is not false (the default adds an n8n attribution)`);
return check('S5', on.length === 0, 'Messages do not carry the n8n attribution', src.comms, on);
}
function multiItemRule(g, refs) {
const multi = multiItemNodes(g);
if (!multi.size) return null;
const hits = refs.filter((r) => r.ref.kind === 'named' && ['first', 'last', 'all-index'].includes(r.ref.accessor) && multi.has(r.ref.node));
const where = [...new Set(hits.map((r) => `${r.node} › ${r.param}: ${r.ref.raw.trim()}`))];
const triggers = g.triggers.filter((t) => multi.has(t.name));
const source = triggers.some((t) => typeMatches(t.type, 'whatsAppTrigger')) ? { ...src.multiItem, also: src.expressionsSkill } : src.expressionsSkill;
return check(
'S6',
where.length === 0,
where.length ? `${plural(where.length, 'reference')} read only the first item of a node that can output several` : 'References keep each item paired with its own data',
source,
where.length ? [...where, `Nodes that can output several items: ${[...multi].join(', ')}.`] : [],
);
}
/** AI root nodes whose output decides a branch, with the IF or Switch nodes that they feed. */
function decidingRoots(g) {
return g.nodes
.filter((n) => isAiRoot(g, n))
.map((root) => ({ root, branches: [...downstream(g, root.name)].map((n) => g.byName.get(n)).filter((n) => n && isBranch(n)) }))
.filter((r) => r.branches.length);
}
function temperatureRule(g) {
const roots = decidingRoots(g);
if (!roots.length) return null;
const problems = [];
for (const { root } of roots) {
for (const model of g.aiInputs(root.name, 'ai_languageModel')) {
const t = findKey(model.parameters?.options, 'temperature') ?? findKey(model.parameters, 'temperature');
if (t === undefined) problems.push(`${model.name}: temperature is not set, so the model default is used`);
else if (typeof t === 'number' && t > 0.2) problems.push(`${model.name}: temperature is ${t}`);
}
}
return check('S7', problems.length === 0, 'A model that decides a branch has a temperature of 0.2 or lower', src.triage, problems);
}
/** Field names of a JSON Schema, or of a JSON example when the value is not a schema. */
function schemaKeys(value, out = new Set()) {
if (!value || typeof value !== 'object') return out;
if (Array.isArray(value)) {
value.forEach((v) => schemaKeys(v, out));
return out;
}
const isSchema = value.properties || value.items || typeof value.type === 'string';
if (isSchema) {
for (const [k, v] of Object.entries(value.properties ?? {})) {
out.add(k);
schemaKeys(v, out);
}
if (value.items) schemaKeys(value.items, out);
return out;
}
for (const [k, v] of Object.entries(value)) {
out.add(k);
schemaKeys(v, out);
}
return out;
}
function reasoningRule(g) {
const roots = decidingRoots(g);
if (!roots.length) return null;
const problems = [];
let judged = 0;
for (const { root } of roots) {
const found = parserParams(g, root);
let text = '';
if (found) text = String(found.p.inputSchema ?? found.p.jsonSchemaExample ?? found.p.jsonSchema ?? '');
else text = strings(root.parameters ?? {}).map((s) => s.value).join('\n');
if (!text.includes('{')) continue;
judged++;
let keys;
try {
keys = [...schemaKeys(JSON.parse(text))];
} catch {
keys = [...text.matchAll(/"([A-Za-z_]\w*)"\s*:/g)].map((m) => m[1]);
}
if (!keys.some((k) => /reason|rationale|explanation|why/i.test(k))) problems.push(`${root.name}: the output has no reasoning field (fields: ${keys.slice(0, 8).join(', ')})`);
}
if (!judged) return null;
return check('S8', problems.length === 0, 'AI output that decides a branch includes a reasoning field', src.triage, problems);
}
function fallbackRule(g) {
const roots = decidingRoots(g);
if (!roots.length) return null;
const problems = [];
const branches = new Map(roots.flatMap((r) => r.branches.map((b) => [b.name, b])));
for (const node of branches.values()) {
const used = new Set(g.mainChildren(node.name).map((e) => e.output));
if (typeMatches(node.type, 'if')) {
for (const [i, label] of [[0, 'true'], [1, 'false']]) if (!used.has(i)) problems.push(`${node.name}: the ${label} output goes nowhere`);
} else {
const rules = node.parameters?.rules?.values ?? node.parameters?.rules?.rules ?? [];
const fallback = node.parameters?.options?.fallbackOutput;
if (fallback !== 'extra') problems.push(`${node.name}: options.fallbackOutput is not 'extra', so unmatched items are dropped`);
else if (!used.has(rules.length)) problems.push(`${node.name}: the fallback output goes nowhere`);
rules.forEach((_, i) => !used.has(i) && problems.push(`${node.name}: output ${i} goes nowhere`));
}
}
return check('S9', problems.length === 0, 'Every branch after an AI decision, including the fallback, goes somewhere', src.triage, problems);
}
function memoryRule(g) {
const chat = g.triggers.some((t) => MESSAGING_TRIGGERS.some((m) => typeMatches(t.type, m)));
const agents = g.nodes.filter((n) => typeMatches(n.type, `${LC}agent`) && !g.subnodes.has(n.name));
if (!chat || !agents.length) return null;
const missing = agents.filter((a) => !g.aiInputs(a.name, 'ai_memory').length).map((a) => `${a.name}: no memory sub-node`);
return check('S10', missing.length === 0, 'A chatbot agent has memory', src.chatbot, missing);
}
function sessionKeyRule(g) {
const memories = g.nodes.filter((n) => g.subnodes.has(n.name) && /memory/i.test(shortType(n.type)) && n.parameters?.sessionIdType === 'customKey');
if (!memories.length) return null;
const problems = [];
for (const m of memories) {
const key = String(m.parameters?.sessionKey ?? '');
if (!key.startsWith('=')) problems.push(`${m.name}: the session key is a fixed value, so every conversation shares one memory`);
else if (/\$json\b/.test(key)) problems.push(`${m.name}: the session key reads $json, which a memory sub-node does not have`);
}
return check('S11', problems.length === 0, 'The memory session key names a node and identifies the conversation', { ...src.chatbot, also: src.memory }, problems);
}
// --- Main ----------------------------------------------------------------------
export function deriveChecks(workflow, canvas, exec, { wire, binary, claims } = {}) {
const g = graph(workflow);
const refs = workflowReferences(g);
const correctness = [
checkDefinitions(workflow),
...checkCanvas(workflow, canvas),
checkReachable(g),
checkNamedRefs(workflow, g, refs),
checkFieldPaths(g, refs, exec),
checkSecrets(g),
];
const standards = [
errorWorkflowRule(workflow, g),
errorOutputRule(g),
retryRule(g),
namedRefRule(refs),
structuredRule(g),
attributionRule(g),
multiItemRule(g, refs),
temperatureRule(g),
reasoningRule(g),
fallbackRule(g),
memoryRule(g),
sessionKeyRule(g),
].filter(Boolean);
const scored = standards.filter((c) => c.ok !== null);
return {
version: VERSION,
groups: [
{ key: 'correctness', title: 'Correctness', gating: true, note: 'From the node definitions, the n8n editor and the workflow graph. A failure here changes the verdict.', checks: correctness },
{ key: 'execution', title: 'Execution with pinned data', gating: true, note: 'Triggers, service nodes and AI nodes get pinned data. Logic nodes run for real. A failure here changes the verdict.', checks: executionChecks(exec) },
...wireGroups(wire),
claimsGroup(claims),
{
key: 'standards',
title: 'n8n standards',
gating: false,
note: 'Rules from n8n-io/skills and the workflow-sdk best practices. The builders did not get these rules. This score does not change the verdict.',
score: { passed: scored.filter((c) => c.ok).length, total: scored.length },
checks: standards,
},
binaryGroup(binary),
].filter(Boolean),
};
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
const runDir = resolve(process.argv[2] ?? '.');
for (const entry of readdirSync(runDir, { withFileTypes: true })) {
const dir = join(runDir, entry.name);
if (!entry.isDirectory() || !existsSync(join(dir, 'workflow.json'))) continue;
const result = deriveChecks(readJson(join(dir, 'workflow.json')), readJson(join(dir, 'canvas-issues.json')), readJson(join(dir, 'exec.json')), {
wire: readJson(join(dir, 'wire.json')),
binary: readJson(join(dir, 'binary-checks.json')),
claims: readJson(join(dir, 'claims.json')),
});
writeFileSync(join(dir, 'auto-checks.json'), JSON.stringify(result, null, 1));
const line = result.groups.map((group) => {
const judged = group.checks.filter((c) => c.ok !== null && c.level !== 'info');
return `${group.key} ${judged.filter((c) => c.ok).length}/${judged.length}`;
});
console.log(`${entry.name}: auto-checks.json written (${line.join(', ')})`);
}
}

View File

@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""Fetch per-call token usage and cost of an Instance AI run from LangSmith.
n8n does not store token usage. LangSmith has it when the instance runs with
LANGSMITH_TRACING=true. The script reads LANGSMITH_API_KEY, LANGSMITH_ENDPOINT and
LANGSMITH_PROJECT from the config (see n8n_api.config).
langsmith_usage.py --run-dir <run-dir>/iai
It takes all LLM runs in the project between startedAt and finishedAt from run.json.
Do not start other Instance AI work on the same instance during the run, or use --by-trace.
langsmith_usage.py --run-dir <dir> --by-trace
--by-trace keeps only the calls of the traces in run.json "langsmithRunIds" (the root run id of
each run-start event is also its LangSmith trace id). Use it when other Instance AI work runs
at the same time.
The script also writes context.json: for each agent, the system prompt, the messages before
the first model response, and the tools, from the first LLM call of that agent. Tools that the
agent loads later with load_tool come from its last call.
"""
import argparse
import json
import urllib.parse
import urllib.request
from datetime import datetime, timedelta
from pathlib import Path
from n8n_api import setting
def langsmith_client():
key = setting('LANGSMITH_API_KEY')
if not key:
raise SystemExit('LANGSMITH_API_KEY is not set. Instance AI cost and tokens are not available.')
base = setting('LANGSMITH_ENDPOINT', default='https://api.smith.langchain.com').rstrip('/')
def call(method, path, body=None):
req = urllib.request.Request(
base + path, method=method,
data=json.dumps(body).encode() if body is not None else None,
headers={'x-api-key': key, 'content-type': 'application/json'},
)
with urllib.request.urlopen(req, timeout=60) as res:
return json.load(res)
return call
def fetch_calls(started_at, finished_at, traces=None):
"""LLM calls in the LangSmith project between two ISO times. traces keeps only the calls of these traces."""
call = langsmith_client()
project = setting('LANGSMITH_PROJECT', 'LANGCHAIN_PROJECT', default='instance-ai')
sessions = call('GET', '/api/v1/sessions?' + urllib.parse.urlencode({'name': project}))
if not sessions:
raise SystemExit(f'LangSmith project "{project}" not found.')
session_id = sessions[0]['id']
start = datetime.fromisoformat(started_at) - timedelta(seconds=5)
end = datetime.fromisoformat(finished_at) + timedelta(seconds=5)
calls, cursor = [], None
while True:
body = {
'session': [session_id], 'run_type': 'llm', 'limit': 100,
'start_time': start.isoformat(),
'select': ['id', 'name', 'start_time', 'end_time', 'prompt_tokens', 'completion_tokens',
'total_cost', 'prompt_token_details', 'extra', 'trace_id'],
}
if cursor:
body['cursor'] = cursor
res = call('POST', '/api/v1/runs/query', body)
for run in res.get('runs', []):
started = datetime.fromisoformat(run['start_time'].replace('Z', '+00:00'))
if started.tzinfo is None:
started = started.replace(tzinfo=start.tzinfo)
if started > end or (traces is not None and run.get('trace_id') not in traces):
continue
details = run.get('prompt_token_details') or {}
metadata = (run.get('extra') or {}).get('metadata') or {}
calls.append({
'id': run.get('id'), 'trace': run.get('trace_id'), 'name': run.get('name'),
'start': started.isoformat(), 'end': run.get('end_time'),
'in': run.get('prompt_tokens') or 0,
'cache_read': details.get('cache_read') or 0,
'cache_write': details.get('cache_creation') or 0,
'out': run.get('completion_tokens') or 0,
'cost': float(run.get('total_cost') or 0),
'model': metadata.get('ls_model_name'),
})
cursor = (res.get('cursors') or {}).get('next')
if not cursor:
break
return sorted(calls, key=lambda c: c['start'])
def message_text(content):
if isinstance(content, str):
return content
return '\n'.join(p.get('text', '') for p in content or [] if isinstance(p, dict) and p.get('type') == 'text')
def tool_spec(t):
return {'name': t.get('name') or t.get('id'), 'description': t.get('description') or '',
'schema': t.get('inputSchema') or t.get('parameters'), 'type': t.get('type')}
def fetch_contexts(calls):
"""What each agent got before its first model response, in the order that the agents started."""
call = langsmith_client()
by_agent = {}
for c in calls:
if c.get('id'):
by_agent.setdefault(c['name'], []).append(c)
contexts = []
for name, agent_calls in by_agent.items():
first = call('GET', f'/api/v1/runs/{agent_calls[0]["id"]}').get('inputs') or {}
messages = []
for m in first.get('input') or []:
if m.get('role') not in ('system', 'user'):
break
messages.append({'role': m['role'], 'text': message_text(m.get('content'))})
tools = [tool_spec(t) for t in first.get('tools') or []]
loaded = []
if len(agent_calls) > 1:
last = call('GET', f'/api/v1/runs/{agent_calls[-1]["id"]}').get('inputs') or {}
known = {t['name'] for t in tools}
loaded = [tool_spec(t) for t in last.get('tools') or [] if (t.get('name') or t.get('id')) not in known]
contexts.append({'agent': name, 'messages': messages, 'tools': tools, 'loadedTools': loaded})
return contexts
def totals(calls):
return {
'calls': len(calls), 'models': sorted({c['model'] for c in calls if c['model']}),
'input': sum(c['in'] for c in calls), 'output': sum(c['out'] for c in calls),
'cost': round(sum(c['cost'] for c in calls), 4),
}
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--run-dir', required=True)
ap.add_argument('--by-trace', action='store_true')
args = ap.parse_args()
run_dir = Path(args.run_dir)
meta = json.loads((run_dir / 'run.json').read_text())
traces = set(meta.get('langsmithRunIds') or [])
if args.by_trace and not traces:
raise SystemExit('run.json has no langsmithRunIds. Is LANGSMITH_TRACING=true on the instance?')
calls = fetch_calls(meta['startedAt'], meta['finishedAt'], traces if args.by_trace else None)
(run_dir / 'usage.json').write_text(json.dumps(calls, indent=1))
(run_dir / 'context.json').write_text(json.dumps(fetch_contexts(calls), indent=1))
print(json.dumps(totals(calls), indent=2))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,84 @@
"""Live views of a run.
COMPARE_LIVE=1 (default): compare.py serves a live page (live_server.py). COMPARE_LIVE=0 turns it off.
COMPARE_LIVE_WINDOWS=1 also opens the Instance AI thread in the browser and a terminal window for
each CLI builder. If a view cannot open, the script prints how to open it.
COMPARE_RECORD=0 skips the screen recordings that the report replays.
"""
import shlex
import shutil
import subprocess
import sys
import webbrowser
from n8n_api import setting
def is_on(name, default='1'):
return setting(name, default=default).strip().lower() not in ('0', 'false', 'no', 'off')
def enabled():
return is_on('COMPARE_LIVE')
def windows():
return enabled() and is_on('COMPARE_LIVE_WINDOWS', default='0')
def recording():
return is_on('COMPARE_RECORD')
def open_url(url):
if not windows():
return
try:
opened = webbrowser.open(url)
except webbrowser.Error:
opened = False
if not opened:
print(f'open {url} to watch the run', flush=True)
def log_window_launchers(command):
if sys.platform == 'darwin' and shutil.which('osascript'):
# The command goes in as an argument, so it needs no AppleScript quoting.
yield ['osascript', '-e', 'on run argv', '-e',
'tell application "Terminal" to do script (item 1 of argv)', '-e', 'end run', command]
for terminal, flag in (('x-terminal-emulator', '-e'), ('gnome-terminal', '--'), ('konsole', '-e'), ('xterm', '-e')):
if shutil.which(terminal):
yield [terminal, flag, 'sh', '-c', command]
return
def open_terminal(title, command, cols=None, rows=None):
"""Open a terminal window with a title that runs a shell command. Returns False if none opened."""
if not windows():
return True
prefix = f"printf '\\033]0;%s\\007' {shlex.quote(title)}; "
if cols and rows:
# Resize the window so that the full screen of the CLI fits.
prefix += f"printf '\\033[8;{rows};{cols}t'; "
for launcher in log_window_launchers(prefix + command):
# osascript returns at once, but xterm and other emulators run as long as the window is open.
# A launcher that still runs after a short wait has opened its window.
try:
proc = subprocess.Popen(launcher, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, start_new_session=True)
except OSError:
continue
try:
if proc.wait(timeout=3) == 0:
return True
except subprocess.TimeoutExpired:
return True
return False
def open_log_window(title, path):
"""Open a terminal window that follows the log file at path."""
quoted = shlex.quote(str(path))
if not open_terminal(title, f'tail -n +1 -f {quoted}'):
print(f'watch {title}: tail -f {quoted}', flush=True)

View File

@ -0,0 +1,635 @@
#!/usr/bin/env python3
"""Serve a page that shows the builder runs while they run.
live_server.py --run-dir <run-dir> [--parent-pid <pid>]
compare.py starts this server in the background. The server listens on 127.0.0.1 only and
writes its address to <run-dir>/live-url.txt. Routes:
/report/live the live page
/report/<file> the final report and its files, when they exist
/api/state the status, table row and LLM steps of each builder, as JSON
/api/cast/<name> the terminal screen of a CLI builder, as Server-Sent Events in the
live format of the asciinema player
/api/shot/<name> the last screenshot of the Instance AI thread
compare.py writes the status of each builder to <run-dir>/status.json. The server stops
30 minutes after the last request, when the run is done or compare.py has stopped.
Other skills reuse the server: they call serve() with their own app (see BuilderApp).
"""
import argparse
import json
import mimetypes
import os
import re
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import unquote, urlparse
import build_report as br
from run_cli_builder import find_session_file
IDLE_STOP_S = 1800
KINDS = {'iai': 'instance-ai', 'claude': 'claude', 'codex': 'codex'}
STATUS_LABELS = {'waiting': 'Waiting', 'running': 'Running', 'done': 'Done', 'failed': 'Failed', 'skipped': 'Skipped'}
mimetypes.add_type('font/woff2', '.woff2')
mimetypes.add_type('text/javascript', '.js')
mimetypes.add_type('video/webm', '.webm')
def read_status(run_dir):
try:
return json.loads((run_dir / 'status.json').read_text())
except (OSError, ValueError):
return {'phase': 'running', 'builders': {}}
def empty_data(model=None):
return {'events': [], 'calls': [], 'model': model, 'duration': '-', 'cost': '-', 'costSource': None}
def load_instance_ai_live(sub):
"""Trace events from the event stream that run_instance_ai.py writes while the thread runs."""
path = sub / 'events.jsonl'
rows = br.read_jsonl(path) if path.exists() else []
if not rows:
return empty_data()
first = next((r for r in rows if r.get('type') == 'run-start'), rows[0])
start, root = first['ts'] / 1000, first.get('agentId')
events, texts, tools = [], {}, {}
for r in rows:
kind, p, agent = r.get('type'), r.get('payload') or {}, r.get('agentId')
t = (r.get('ts') or first['ts']) / 1000 - start
label = '' if agent in (root, None) else f'{agent.split("-run_")[0]} › '
if kind in ('text-delta', 'reasoning-delta'):
event_kind = 'text' if kind == 'text-delta' else 'reasoning'
key = (agent, r.get('responseId'), event_kind)
if key not in texts:
texts[key] = {'kind': event_kind, 't': t, 'text': ''}
events.append(texts[key])
texts[key]['text'] += p.get('text') or ''
elif kind == 'tool-call':
tools[p.get('toolCallId')] = {'kind': 'tool', 't': t, 'name': label + (p.get('toolName') or ''),
'args': p.get('args'), 'result': 'Waiting for the result…',
'error': False, 'duration': None}
events.append(tools[p.get('toolCallId')])
elif kind in ('tool-result', 'tool-error') and p.get('toolCallId') in tools:
result = p.get('result') if kind == 'tool-result' else p.get('error')
tools[p['toolCallId']].update(
result=result, duration=t - tools[p['toolCallId']]['t'],
error=kind == 'tool-error' or (isinstance(result, dict) and (result.get('success') is False or 'error' in result)))
elif kind == 'confirmation-request' and p.get('toolCallId') in tools:
tools[p['toolCallId']]['note'] = 'Confirmation card answered by the auto-approve policy.'
data = empty_data()
data['events'] = [e for e in events if e['kind'] == 'tool' or e['text'].strip()]
return data
def live_run(run_dir, name, case):
"""Meta and trace data of one builder. A finished builder has run.json. A running one does not."""
sub = run_dir / name
meta_file = sub / 'run.json'
if meta_file.exists():
meta = json.loads(meta_file.read_text())
if meta.get('kind') in br.LOADERS:
return meta, br.load_run(sub, meta, case.get('prices'))
meta = {'kind': KINDS.get(name, name)}
if meta['kind'] == 'instance-ai':
return meta, load_instance_ai_live(sub)
session = find_session_file(meta['kind'], sub / 'home') if (sub / 'home').exists() else None
if not session:
return meta, empty_data()
meta['sessionFile'] = str(session.relative_to(sub))
return meta, br.load_run(sub, meta, case.get('prices'))
def seconds(value):
return f'{value:.0f}s'
def gather(run_dir):
case, status, verdicts = br.load_case(run_dir), read_status(run_dir), br.load_verdicts(run_dir)
builders = status.get('builders', {})
names = sorted((n for n, b in builders.items() if b.get('status') != 'skipped'),
key=lambda n: (br.RUN_ORDER.get(KINDS.get(n, n), 9), n))
now, runs = time.time(), []
for name in names:
entry = builders[name]
state = entry.get('status', 'waiting')
meta, data = live_run(run_dir, name, case)
running = state in ('waiting', 'running')
if running and data.get('loadError'):
# A running builder can leave a file in a state that the loader cannot read yet.
data = empty_data(data.get('model'))
started, finished = entry.get('startedAt'), entry.get('finishedAt')
if started and (running or data['duration'] == '-'):
data['duration'] = seconds((finished or now) - started)
if running:
data['cost'] = '-'
runs.append((name, meta, data, entry))
return case, status, verdicts, runs
def status_badge(state):
return f'<span class="badge st-{br.esc(state)}">{STATUS_LABELS.get(state, br.esc(state))}</span>'
def clock(status):
started = status.get('startedAt')
elapsed = seconds((status.get('finishedAt') or time.time()) - started) if started else ''
mode = 'Parallel' if status.get('mode') == 'parallel' else 'One at a time'
return ' · '.join(x for x in (mode, elapsed, status.get('step')) if x)
def render_run(run_dir, case, name, meta, data, entry, verdict, scale):
row = br.render_row_cells(name, meta, data, status_badge(entry.get('status', 'waiting')))
canvas = br.render_canvas_box(name, meta, run_dir / 'report', case.get('baseUrl', 'http://localhost:5678'),
empty='The canvas appears when all builders finish.')
view, auto = br.verdict_view(verdict, run_dir / name, case.get('expected') or '')
return row, br.render_panel_body(data, view, canvas, *scale, case['prompt'], case.get('expected') or '', auto=auto,
pending=True)
def render_live_box(status, runs):
cards = []
for name, meta, _, entry in runs:
title = br.esc(br.run_title(name, meta))
screen = (status.get('screens') or {}).get(name)
if screen == 'cast':
inner = (f'<div data-cast="{name}"></div><button type="button" class="live-zoom overlay" '
f'data-zoom="{name}" aria-label="Show {title} full screen"></button>')
elif screen == 'shot':
inner = (f'<button type="button" class="live-zoom" data-zoom="{name}" aria-label="Show {title} full screen">'
f'<img data-shot="{name}" alt="{title} in the n8n UI" hidden></button>'
f'<div class="live-empty" data-shot-empty="{name}">Waiting for the first screenshot</div>')
else:
inner = '<div class="live-empty">No live screen for this builder. The LLM steps below update live.</div>'
cards.append(
f'<div class="live-card"><div class="live-screen">{inner}</div>'
f'<div class="reel-meta"><span class="run-name">{title}</span>'
f'<span data-live-badge="{name}">{status_badge(entry.get("status", "waiting"))}</span></div></div>'
)
return (f'<div class="box"><div class="head"><h2>Live</h2><span class="mono-label" id="live-clock">'
f'{br.esc(clock(status))}</span></div><div class="live-grid">{"".join(cards)}</div></div>')
def render_page(run_dir):
case, status, verdicts, runs = gather(run_dir)
scale = br.token_scale([r[:3] for r in runs])
rows, panels = [], []
for i, (name, meta, data, entry) in enumerate(runs):
active = ' active' if i == 0 else ''
row, panel = render_run(run_dir, case, name, meta, data, entry, verdicts.get(name), scale)
rows.append(f'<tr class="row{active}" data-run="{name}" onclick="show(\'{name}\')">{row}</tr>')
panels.append(f'<section class="run{active}" id="{name}">{panel}</section>')
try:
br.ensure_player_cache()
player_js, player_css = (f'vendor/{n}' for n in br.PLAYER_FILES)
except OSError:
player_js, player_css = (br.PLAYER_CDN + n for n in br.PLAYER_FILES)
notes = ('This page updates while the builders run, and changes to the full report when the run is done. '
'Costs, canvases, checks and verdicts come after the run. Times are seconds from the start of each builder.')
return br.fill_template(
br.esc(case.get('title') or 'Builder comparison'), ''.join(rows), br.esc(notes), ''.join(panels),
replay=render_live_box(status, runs), head=f'<link rel="stylesheet" href="{player_css}">',
scripts=f'<script src="{player_js}"></script><script>const SCREENS = [];</script><script>{LIVE_JS}</script>',
prompt=br.esc(case['prompt']), expected=br.esc(case.get('expected') or '-'))
def render_state(run_dir):
case, status, verdicts, runs = gather(run_dir)
scale = br.token_scale([r[:3] for r in runs])
state = {'phase': status.get('phase', 'running'), 'clock': clock(status), 'runs': {}}
if state['phase'] == 'done' and (run_dir / 'report' / 'index.html').exists():
state['report'] = '/report/index.html'
for name, meta, data, entry in runs:
row, panel = render_run(run_dir, case, name, meta, data, entry, verdicts.get(name), scale)
shot = run_dir / name / 'live.png'
state['runs'][name] = {'row': row, 'panel': panel, 'badge': status_badge(entry.get('status', 'waiting')),
'shot': shot.stat().st_mtime if shot.exists() else None}
return state
LIVE_JS = """
window.addEventListener('DOMContentLoaded', ()=>{
const seen = {}, shots = {}, userFolds = {};
// A box that the user opened or closed stays that way when its panel updates. Other boxes take the new default.
document.addEventListener('click', e=>{
const summary = e.target.closest && e.target.closest('details.fold > summary');
if (!summary || e.target.closest('.run-picker')) return;
const section = summary.closest('section.run');
if (section) (userFolds[section.id] ??= {})[summary.querySelector('h2').textContent] = !summary.parentElement.open;
});
document.addEventListener('change', e=>{
if (!e.target.matches || !e.target.matches('.run-picker')) return;
const steps = document.getElementById(e.target.value)?.querySelector('.steps');
if (steps) (userFolds[e.target.value] ??= {})['LLM steps'] = steps.open;
});
document.querySelectorAll('[data-cast]').forEach(el=>{
AsciinemaPlayer.create({driver: 'eventsource', url: `/api/cast/${el.dataset.cast}`, bufferTime: 0}, el,
{fit: 'both', autoPlay: true, controls: false});
});
// A live screen opens larger in a full-screen viewer. The viewer follows the newest screenshot, and a
// terminal gets its own player, because the stream starts with the whole recording so far.
const zoomNames = [...document.querySelectorAll('[data-zoom]')].map(b=>b.dataset.zoom);
const viewer = document.createElement('div');
let zoomed = null, opener = null, castPlayer = null, castName = null;
viewer.className = 'theater live-theater';
viewer.hidden = true;
viewer.setAttribute('role', 'dialog');
viewer.setAttribute('aria-modal', 'true');
viewer.setAttribute('aria-label', 'Live screen');
viewer.innerHTML = '<div class="theater-bar"><h2>Live</h2><span data-zoom-badge></span>'
+ '<button type="button" class="btn" data-zoom-close>Close</button></div><div class="tabs"></div>'
+ '<div class="live-zoomed"><img alt=""><div class="live-empty">Waiting for the first screenshot</div>'
+ '<div class="live-zoomed-cast" hidden></div></div>'
+ '<p class="fineprint">The screen updates every few seconds. Press the left or right arrow key to change the '
+ 'scenario. Press Esc to close.</p>';
for (const name of zoomNames) {
const tab = Object.assign(document.createElement('button'), {type: 'button', className: 'tab'});
tab.dataset.zoomTab = name;
tab.textContent = document.querySelector(`[data-zoom="${name}"]`).closest('.live-card').querySelector('.run-name').textContent;
tab.addEventListener('click', ()=>showZoom(name));
viewer.querySelector('.tabs').append(tab);
}
document.body.append(viewer);
function stopCast(){
castPlayer?.dispose();
castPlayer = castName = null;
viewer.querySelector('.live-zoomed-cast').replaceChildren();
}
function showZoom(name){
zoomed = name;
viewer.querySelectorAll('[data-zoom-tab]').forEach(t=>t.classList.toggle('active', t.dataset.zoomTab === name));
const shot = document.querySelector(`img[data-shot="${name}"]`);
const isCast = !!document.querySelector(`[data-cast="${name}"]`);
const img = viewer.querySelector('.live-zoomed img'), castBox = viewer.querySelector('.live-zoomed-cast');
if (castName !== name) stopCast();
castBox.hidden = !isCast;
if (isCast && !castPlayer) {
castName = name;
castPlayer = AsciinemaPlayer.create({driver: 'eventsource', url: `/api/cast/${name}`, bufferTime: 0}, castBox,
{fit: 'both', autoPlay: true, controls: false});
}
const shotHidden = !shot || shot.hidden;
img.hidden = isCast || shotHidden;
if (shot && !shot.hidden && img.getAttribute('src') !== shot.getAttribute('src')) { img.src = shot.src; img.alt = shot.alt; }
viewer.querySelector('.live-zoomed .live-empty').hidden = isCast || !shotHidden;
viewer.querySelector('[data-zoom-badge]').innerHTML = document.querySelector(`[data-live-badge="${name}"]`)?.innerHTML || '';
}
function closeZoom(){
stopCast();
viewer.hidden = true;
zoomed = null;
document.body.classList.remove('theater-open');
opener?.focus();
}
document.querySelectorAll('[data-zoom]').forEach(b=>b.addEventListener('click', ()=>{
opener = b;
viewer.hidden = false;
document.body.classList.add('theater-open');
showZoom(b.dataset.zoom);
viewer.querySelector('[data-zoom-close]').focus();
}));
viewer.querySelector('[data-zoom-close]').addEventListener('click', closeZoom);
document.addEventListener('keydown', e=>{
if (zoomed === null) return;
if (e.key === 'Escape') closeZoom();
else if (e.key === 'ArrowLeft' || e.key === 'ArrowRight') {
const step = e.key === 'ArrowRight' ? 1 : -1;
showZoom(zoomNames[(zoomNames.indexOf(zoomed) + step + zoomNames.length) % zoomNames.length]);
}
});
function selectQuietly(root, step){
root.querySelectorAll('.step').forEach(b=>b.classList.toggle('active', b.dataset.step === step));
root.querySelectorAll('.step-detail').forEach(d=>{ d.hidden = d.dataset.step !== step; });
}
// Keeps the selected step, the scroll positions and the open sections. When the last step is
// selected, the newest step is selected after the update.
function swapPanel(section, html, first){
const old = section.querySelector('.steps');
const items = old ? [...old.querySelectorAll('.step')] : [];
const active = items.find(b=>b.classList.contains('active'));
const follow = first || !active || active === items[items.length - 1];
// A run with no trace yet has a .steps section without the list and the pane.
const listTop = old?.querySelector('.step-list')?.scrollTop ?? 0;
const paneTop = old?.querySelector('.step-pane')?.scrollTop ?? 0;
const openState = {};
section.querySelectorAll('.step-detail').forEach(d=>d.querySelectorAll('details').forEach((x, i)=>{ openState[`${d.dataset.step}:${i}`] = x.open; }));
section.innerHTML = html;
section.querySelectorAll('details.fold').forEach(d=>{
const open = (userFolds[section.id] || {})[d.querySelector('summary h2')?.textContent];
if (open !== undefined) d.open = open;
});
addRunPickers();
section.querySelectorAll('.step-detail').forEach(d=>d.querySelectorAll('details').forEach((x, i)=>{
const key = `${d.dataset.step}:${i}`;
if (key in openState) x.open = openState[key];
}));
const root = section.querySelector('.steps');
const next = root ? [...root.querySelectorAll('.step')] : [];
if (!next.length) return;
const list = root.querySelector('.step-list'), pane = root.querySelector('.step-pane');
if (!list || !pane) return;
if (follow) {
selectQuietly(root, next[next.length - 1].dataset.step);
list.scrollTop = list.scrollHeight;
} else {
selectQuietly(root, active.dataset.step);
list.scrollTop = listTop;
pane.scrollTop = paneTop;
}
}
async function poll(){
let state;
try {
state = await (await fetch('/api/state', {cache: 'no-store'})).json();
} catch {
$('live-clock').textContent = 'Lost the connection to the live server. Trying again…';
setTimeout(poll, 3000);
return;
}
if (state.report) { location.replace(state.report); return; }
$('live-clock').textContent = state.clock;
for (const [name, run] of Object.entries(state.runs)) {
const row = document.querySelector(`.row[data-run="${name}"]`);
if (row && seen[`row:${name}`] !== run.row) { row.innerHTML = run.row; seen[`row:${name}`] = run.row; }
const section = document.getElementById(name);
if (section && seen[`panel:${name}`] !== run.panel) {
swapPanel(section, run.panel, seen[`panel:${name}`] === undefined);
seen[`panel:${name}`] = run.panel;
}
const badge = document.querySelector(`[data-live-badge="${name}"]`);
if (badge) badge.innerHTML = run.badge;
const img = document.querySelector(`img[data-shot="${name}"]`);
if (img && run.shot && shots[name] !== run.shot) {
shots[name] = run.shot;
img.src = `/api/shot/${name}?v=${run.shot}`;
img.hidden = false;
document.querySelector(`[data-shot-empty="${name}"]`)?.remove();
}
}
if (zoomed !== null) showZoom(zoomed);
setTimeout(poll, 2000);
}
poll();
});
"""
class BuilderApp:
"""What the server shows for compare.py: one live card, table row and panel for each builder.
Another skill can serve its own page with the same server: give LiveServer an object with
these methods."""
def __init__(self, run_dir):
self.run_dir = run_dir
def entry(self, name):
"""The status.json entry of a live card, or None for an unknown name."""
return read_status(self.run_dir).get('builders', {}).get(name)
def finished(self, name):
state = (self.entry(name) or {}).get('status')
return state in ('done', 'failed', 'skipped') or (self.run_dir / name / 'run.json').exists()
def done(self):
return read_status(self.run_dir).get('phase') == 'done' and (self.run_dir / 'report' / 'index.html').exists()
def page(self):
return render_page(self.run_dir)
def state(self):
return render_state(self.run_dir)
def shot(self, name):
return self.run_dir / name / 'live.png'
def casts(self, name):
"""The live recording, then the place where the runner moves it when the builder stops."""
sub = self.run_dir / name
return sub / '.screen' / 'screen.cast', sub / 'screen.cast'
def file(self, path):
"""A file for a route that the server does not know, or None."""
return None
class LiveServer(ThreadingHTTPServer):
daemon_threads = True
def __init__(self, address, app):
super().__init__(address, Handler)
port = self.server_address[1]
self.app, self.run_dir = app, app.run_dir
self.allowed_hosts = {f'127.0.0.1:{port}', f'localhost:{port}'}
self.last_request = time.time()
class Handler(BaseHTTPRequestHandler):
server_version = 'cwb-live'
def log_message(self, *args):
pass
def do_GET(self):
srv = self.server
srv.last_request = time.time()
# Only same-origin requests. A page that resolves its own host name to 127.0.0.1 gets a 403.
if self.headers.get('Host', '') not in srv.allowed_hosts:
return self.send_error(403)
path = unquote(urlparse(self.path).path)
app = srv.app
try:
if path in ('/', '/report', '/report/'):
return self.redirect('/report/index.html' if app.done() else '/report/live')
if path == '/report/live':
return self.send_body(app.page().encode(), 'text/html; charset=utf-8')
if path == '/api/state':
return self.send_body(json.dumps(app.state()).encode(), 'application/json')
name = path.rsplit('/', 1)[-1]
if path.startswith(('/api/cast/', '/api/shot/')) and not app.entry(name):
return self.send_error(404)
if path.startswith('/api/cast/'):
return self.stream_cast(name)
if path.startswith('/api/shot/'):
return self.send_file(app.shot(name))
if path.startswith('/report/'):
return self.send_file(self.static_file(path.removeprefix('/report/')))
self.send_file(app.file(path))
except (BrokenPipeError, ConnectionResetError):
pass
def redirect(self, location):
self.send_response(302)
self.send_header('Location', location)
self.send_header('Content-Length', '0')
self.end_headers()
def send_body(self, body, content_type):
self.send_response(200)
self.send_header('Content-Type', content_type)
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
self.end_headers()
self.wfile.write(body)
def static_file(self, rel):
"""A file of the report. Fonts, logos and the player come from the asset cache until the report exists."""
base = (self.server.run_dir / 'report').resolve()
candidate = (base / rel).resolve()
if candidate.is_relative_to(base) and candidate.is_file():
return candidate
parts = Path(rel).parts
if parts[:1] in (('fonts',), ('brand',)):
cache = br.ASSET_CACHE.resolve()
candidate = (cache / rel).resolve()
return candidate if candidate.is_relative_to(cache) and candidate.is_file() else None
if len(parts) == 2 and parts[0] == 'vendor' and parts[1] in br.PLAYER_FILES:
return br.PLAYER_CACHE / parts[1]
return None
def send_file(self, file):
if not file or not file.is_file():
return self.send_error(404)
size = file.stat().st_size
start, end = 0, size - 1
match = re.fullmatch(r'bytes=(\d*)-(\d*)', self.headers.get('Range') or '')
if match and size and (match[1] or match[2]):
# Videos need byte ranges to seek.
if match[1]:
start, end = int(match[1]), int(match[2]) if match[2] else size - 1
else:
start = max(size - int(match[2]), 0)
end = min(end, size - 1)
if start > end:
self.send_response(416)
self.send_header('Content-Range', f'bytes */{size}')
self.end_headers()
return
self.send_response(206)
self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
else:
self.send_response(200)
self.send_header('Content-Type', mimetypes.guess_type(file.name)[0] or 'application/octet-stream')
self.send_header('Content-Length', str(end - start + 1))
self.send_header('Accept-Ranges', 'bytes')
self.send_header('Cache-Control', 'no-cache')
self.end_headers()
with file.open('rb') as f:
f.seek(start)
left = end - start + 1
while left > 0:
chunk = f.read(min(left, 1 << 16))
if not chunk:
break
self.wfile.write(chunk)
left -= len(chunk)
def sse(self, data, event=None):
self.wfile.write(((f'event: {event}\n' if event else '') + f'data: {data}\n\n').encode())
self.wfile.flush()
def stream_cast(self, name):
"""Send the screen recording so far as one reset frame, then each new frame as it arrives."""
srv = self.server
self.send_response(200)
self.send_header('Content-Type', 'text/event-stream')
self.send_header('Cache-Control', 'no-store')
self.end_headers()
path = None
while path is None:
path = next((p for p in srv.app.casts(name) if p.exists()), None)
if path is None:
if srv.app.finished(name):
return self.sse('{}', 'done')
self.wfile.write(b': waiting\n\n')
self.wfile.flush()
srv.last_request = time.time()
time.sleep(1)
# The runner moves the recording when the builder stops. The open file stays readable.
with path.open('rb') as f:
buffer = b''
def complete_lines():
nonlocal buffer
buffer += f.read()
*lines, buffer = buffer.split(b'\n')
return [line for line in lines if line.strip()]
lines = complete_lines()
while not lines:
if srv.app.finished(name):
return self.sse('{}', 'done')
time.sleep(0.2)
lines = complete_lines()
header, frames = json.loads(lines[0]), [json.loads(line) for line in lines[1:]]
output = [f for f in frames if f[1] == 'o']
self.sse(json.dumps({'cols': header.get('width'), 'rows': header.get('height'),
'time': output[-1][0] if output else 0, 'init': ''.join(f[2] for f in output)}))
last_data = last_ping = time.time()
while True:
lines = complete_lines()
for line in lines:
frame = json.loads(line)
if frame[1] == 'o':
self.sse(json.dumps([frame[0], 'o', frame[2]]))
now = time.time()
srv.last_request = now
if lines:
last_data = now
continue
if srv.app.finished(name) and now - last_data > 3:
return self.sse('{}', 'done')
if now - last_ping > 15:
self.wfile.write(b': ping\n\n')
self.wfile.flush()
last_ping = now
time.sleep(0.2)
def pid_alive(pid):
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def stop_when_idle(server, parent_pid):
while True:
time.sleep(30)
if time.time() - server.last_request < IDLE_STOP_S:
continue
done = read_status(server.run_dir).get('phase') == 'done'
if done or (parent_pid and not pid_alive(parent_pid)):
server.shutdown()
return
def serve(app, parent_pid=None, port=0):
"""Serve app until it is idle. Writes the address to <run-dir>/live-url.txt."""
server = LiveServer(('127.0.0.1', port), app)
(app.run_dir / 'live-url.txt').write_text(f'http://127.0.0.1:{server.server_address[1]}\n')
threading.Thread(target=stop_when_idle, args=(server, parent_pid), daemon=True).start()
server.serve_forever()
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--run-dir', required=True)
ap.add_argument('--parent-pid', type=int)
ap.add_argument('--port', type=int, default=0)
args = ap.parse_args()
serve(BuilderApp(Path(args.run_dir).resolve()), args.parent_pid, args.port)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,122 @@
#!/usr/bin/env python3
"""Merge the blind judge panel into verdicts.json.
merge_verdicts.py <run-dir>
Reads judge/blind-map.json and judge/verdicts-*.json. Each judge file has one entry for each
builder letter:
{"A": {"summary": "...", "goalMet": true, "goalReason": "...", "verdict": "pass",
"checks": [{"id": "4", "ok": true, "text": "...", "evidence": [{"node": "...", "path": "...", "value": "..."}]}]}}
For each criterion id, the merged check takes the value that all judges or a majority of the judges
gave. A split is listed under disagreements. Without a majority, the check is not confirmed (ok null).
The summary and the findings outside the criteria come from the first judge.
"""
import json
import sys
from collections import Counter
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from build_report import load_case, parse_criteria # noqa: E402
VERSION = 2
VOTE_TEXT = {True: 'met', False: 'not met', None: 'not confirmed'}
def majority(values):
"""The value that more than half of the votes have, or None when no value has a majority."""
value, count = Counter(values).most_common(1)[0]
return value if count * 2 > len(values) else None
def first_check(judge, cid):
return next((c for c in judge.get('checks') or [] if str(c.get('id')) == cid), None)
def merge_quality(judges):
"""The quality checks by majority, in the order of the first judge that answered each check."""
ids = list(dict.fromkeys(str(q.get('id')) for j in judges for q in j.get('quality') or [] if q.get('id')))
merged = []
for qid in ids:
found = [next((q for q in j.get('quality') or [] if str(q.get('id')) == qid), None) for j in judges]
votes = [q.get('ok') if q else None for q in found]
ok = majority(votes)
pick = next((q for q, v in zip(found, votes) if q and v == ok), None) or next(q for q in found if q)
item = {'id': qid, 'ok': ok, 'text': pick.get('text', ''), 'votes': votes}
if len(set(votes)) > 1:
item['note'] = f'{Counter(votes)[ok]} of {len(judges)} judges.' if ok is not None else 'Judges disagree.'
merged.append(item)
return merged
def merge_builder(judges, criteria_ids):
n = len(judges)
lead = judges[0]
checks, disagreements = [], []
for cid in criteria_ids:
found = [first_check(j, cid) for j in judges]
votes = [c.get('ok') if c else None for c in found]
if all(c is None for c in found):
continue
ok = majority(votes)
pick = next((c for c, v in zip(found, votes) if c and v == ok), None) or next(c for c in found if c)
check = {k: v for k, v in pick.items() if k != 'ok'}
check.update(id=cid, ok=ok, votes=votes)
if len(set(votes)) > 1:
counts = ', '.join(f'{count} {VOTE_TEXT[v]}' for v, count in Counter(votes).items())
disagreements.append({'id': cid, 'votes': votes, 'texts': [c.get('text') if c else None for c in found]})
note = f'Judges disagree ({counts}).' if ok is None else f'{Counter(votes)[ok]} of {n} judges.'
check['note'] = note
checks.append(check)
checks += [dict(c) for c in lead.get('checks') or [] if str(c.get('id')) not in criteria_ids]
goal_votes = [j.get('goalMet') for j in judges]
goal = majority(goal_votes)
if goal is None:
goal = False if goal_votes.count(False) * 2 >= n else None
if len(set(goal_votes)) > 1:
reasons = [j.get('goalReason') for j in judges if j.get('goalMet') is False and j.get('goalReason')]
disagreements.append({'id': 'goal', 'votes': goal_votes, 'texts': reasons})
verdicts = [j.get('verdict') for j in judges if j.get('verdict')]
return {
'quality': merge_quality(judges),
'version': VERSION,
'judges': n,
'verdict': majority(verdicts) if verdicts else None,
'summary': lead.get('summary', ''),
'goalMet': goal,
'goalReason': next((j.get('goalReason') for j in judges if j.get('goalMet') is goal and j.get('goalReason')), ''),
'checks': checks,
'disagreements': disagreements,
}
def main():
run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
judge_dir = run_dir / 'judge'
blind_map = json.loads((judge_dir / 'blind-map.json').read_text())
files = sorted(judge_dir.glob('verdicts-*.json'))
if not files:
sys.exit(f'No judge/verdicts-*.json in {run_dir}.')
panel = [json.loads(f.read_text()) for f in files]
groups = parse_criteria(load_case(run_dir).get('expected') or '') or []
criteria_ids = [num for _, items in groups for num, _ in items]
merged = {}
for letter, builder in blind_map.items():
judges = [p[letter] for p in panel if letter in p]
if not judges:
print(f'{builder}: no judge gave a verdict for builder {letter}')
continue
merged[builder] = merge_builder(judges, criteria_ids)
split = len(merged[builder]['disagreements'])
print(f'{builder} (builder {letter}): {len(judges)} judges, {split} disagreement{"" if split == 1 else "s"}')
(run_dir / 'verdicts.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False))
print(run_dir / 'verdicts.json')
if __name__ == '__main__':
main()

View File

@ -0,0 +1,123 @@
"""Config loader and small REST client for a local n8n instance. Standard library only."""
import http.cookiejar
import json
import os
import stat
import urllib.error
import urllib.request
import uuid
from functools import cache
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[4]
DEFAULT_BASE_URL = 'http://localhost:5678'
TOKEN_FILE = Path.home() / '.config' / 'n8n-builder-compare' / 'mcp-token'
def read_env_file(path):
"""Return KEY=VALUE pairs from a dotenv file. Values are never printed."""
values = {}
p = Path(path)
if not p.exists():
return values
for line in p.read_text().splitlines():
line = line.strip()
if line.startswith('export '):
line = line[7:]
if not line or line.startswith('#') or '=' not in line:
continue
key, value = line.split('=', 1)
value = value.strip()
if value[:1] in ('"', "'"):
value = value[1:].split(value[0], 1)[0]
else:
value = value.split(' #', 1)[0].strip()
values[key.strip()] = value
return values
def env_files():
"""Config files from lowest to highest priority."""
custom = os.environ.get('AI_SURFACES_ENV')
return [REPO_ROOT / '.env.local', Path(custom) if custom else REPO_ROOT / '.env.ai-surfaces']
@cache
def config():
"""Merged settings: .env.local, then .env.ai-surfaces, then the process environment."""
merged = {}
for path in env_files():
merged.update({k: v for k, v in read_env_file(path).items() if v != ''})
merged.update({k: v for k, v in os.environ.items() if v != ''})
return merged
def setting(*names, default=None):
"""Return the first name that has a value."""
values = config()
for name in names:
if values.get(name):
return values[name]
return default
class N8nClient:
def __init__(self):
self.base_url = setting('N8N_BASE_URL', default=DEFAULT_BASE_URL).rstrip('/')
self.email = setting('N8N_EVAL_EMAIL', default='nathan@n8n.io')
self.password = setting('N8N_EVAL_PASSWORD', default='PlaywrightTest123')
# The auth cookie is bound to the browser-id header that was sent at login.
self.browser_id = str(uuid.uuid4())
self.cookies = http.cookiejar.CookieJar()
self.opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(self.cookies))
def headers(self, extra=None):
h = {'content-type': 'application/json', 'browser-id': self.browser_id}
h.update(extra or {})
return h
def request(self, method, path, body=None, timeout=60):
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(self.base_url + path, method=method, data=data, headers=self.headers())
try:
with self.opener.open(req, timeout=timeout) as res:
raw = res.read()
except urllib.error.HTTPError as e:
detail = e.read()[:500].decode(errors='replace')
raise RuntimeError(f'{method} {path} -> {e.code}: {detail}') from None
parsed = json.loads(raw) if raw else None
if isinstance(parsed, dict) and set(parsed) == {'data'}:
return parsed['data']
return parsed
def open_stream(self, path):
req = urllib.request.Request(
self.base_url + path, headers=self.headers({'accept': 'text/event-stream'})
)
return self.opener.open(req, timeout=600)
def login(self):
self.request('POST', '/rest/login', {'emailOrLdapLoginId': self.email, 'password': self.password})
return self
def workflow(self, workflow_id):
return self.request('GET', f'/rest/workflows/{workflow_id}')
def save_token(token):
TOKEN_FILE.parent.mkdir(parents=True, exist_ok=True)
TOKEN_FILE.write_text(token)
os.chmod(TOKEN_FILE, stat.S_IRUSR | stat.S_IWUSR)
def load_token():
"""N8N_MCP_TOKEN from the config wins over the cached token."""
token = setting('N8N_MCP_TOKEN')
if token:
return token
return TOKEN_FILE.read_text().strip() if TOKEN_FILE.exists() else None
def is_full_token(value):
return isinstance(value, str) and value.count('.') == 2 and '*' not in value

View File

@ -0,0 +1,50 @@
// Shared browser helpers: settings, Playwright launch, and n8n sign-in.
// Uses the Playwright copy in the repo test package. Falls back to the local Chrome install.
import { createRequire } from 'node:module';
import { existsSync, readFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
const here = dirname(fileURLToPath(import.meta.url));
const repo = resolve(here, '../../../..');
const require = createRequire(join(repo, 'packages/quality/testing/playwright/package.json'));
const { chromium, expect, selectors } = require('@playwright/test');
export { expect, selectors };
// pngjs, as Playwright bundles it. Image diffs use it, so the skills need no extra package.
export const loadPng = () =>
createRequire(require.resolve('@playwright/test'))('playwright-core/lib/utilsBundle').PNG;
// Same order as n8n_api.config(): .env.local, then .env.ai-surfaces, then the environment.
const settings = {};
for (const file of [
join(repo, '.env.local'),
process.env.AI_SURFACES_ENV ?? join(repo, '.env.ai-surfaces'),
]) {
if (!existsSync(file)) continue;
for (const line of readFileSync(file, 'utf8').split('\n')) {
const match = line.match(/^\s*(?:export\s+)?([A-Z0-9_]+)\s*=\s*(.*)$/);
if (!match) continue;
const value = match[2].trim().replace(/^(['"])(.*)\1$/, '$2');
if (value) settings[match[1]] = value;
}
}
export const setting = (name, fallback) => process.env[name] || settings[name] || fallback;
export const baseUrl = setting('N8N_BASE_URL', 'http://localhost:5678').replace(/\/$/, '');
export async function launchBrowser() {
const launch = existsSync(chromium.executablePath()) ? {} : { channel: 'chrome' };
return await chromium.launch({ headless: true, ...launch });
}
export async function signIn(page) {
await page.goto(`${baseUrl}/signin`);
await page
.getByRole('textbox', { name: 'Email' })
.fill(setting('N8N_EVAL_EMAIL', 'nathan@n8n.io'));
await page
.getByRole('textbox', { name: 'Password' })
.fill(setting('N8N_EVAL_PASSWORD', 'PlaywrightTest123'));
await page.getByRole('button', { name: 'Sign in', exact: true }).click();
await page.waitForURL((url) => !url.pathname.startsWith('/signin'), { timeout: 30_000 });
}

View File

@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""Check the local setup and make the n8n MCP token available.
Prints a JSON status object. It never prints a token or a key.
prepare.py check everything, reuse the cached MCP token
prepare.py --rotate create a new MCP key (this breaks other clients that use the old key)
"""
import argparse
import json
import shutil
import subprocess
import sys
import urllib.request
from n8n_api import TOKEN_FILE, N8nClient, env_files, is_full_token, load_token, save_token, setting
def mcp_handshake(base_url, token):
body = json.dumps({
'jsonrpc': '2.0', 'id': 1, 'method': 'initialize',
'params': {'protocolVersion': '2025-03-26', 'capabilities': {},
'clientInfo': {'name': 'testmax-n8n-ai', 'version': '1'}},
}).encode()
req = urllib.request.Request(base_url + '/mcp-server/http', data=body, method='POST', headers={
'content-type': 'application/json',
'accept': 'application/json, text/event-stream',
'authorization': f'Bearer {token}',
})
try:
with urllib.request.urlopen(req, timeout=20) as res:
return res.status == 200
except Exception:
return False
def docker_ready():
if not shutil.which('docker'):
return False
return subprocess.run(['docker', 'info'], capture_output=True).returncode == 0
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--rotate', action='store_true')
args = ap.parse_args()
status = {
'configFiles': [str(p) for p in env_files() if p.exists()],
'tokenFile': str(TOKEN_FILE),
'builders': setting('COMPARE_BUILDERS', default='iai,claude,codex'),
'keys': {
'claude': bool(setting('CLAUDE_API_KEY', 'ANTHROPIC_API_KEY')),
'codex': bool(setting('CODEX_API_KEY', 'OPENAI_API_KEY')),
'langsmith': bool(setting('LANGSMITH_API_KEY')),
},
}
client = N8nClient()
status['baseUrl'] = client.base_url
try:
client.login()
status['login'] = True
except Exception as e:
status['login'] = False
status['error'] = str(e)
print(json.dumps(status, indent=2))
sys.exit(1)
try:
settings = client.request('GET', '/rest/instance-ai/settings')
status['instanceAiEnabled'] = bool(settings.get('enabled'))
except Exception:
status['instanceAiEnabled'] = False
client.request('PATCH', '/rest/mcp/settings', {'mcpAccessEnabled': True})
status['mcpAccessEnabled'] = True
token = None
if args.rotate:
if setting('N8N_MCP_TOKEN'):
# The clients would keep the configured token, which the rotation makes invalid.
sys.exit('Remove N8N_MCP_TOKEN from the config before you rotate the MCP key, then run prepare.py --rotate again.')
token = client.request('POST', '/rest/mcp/api-key/rotate')['apiKey']
save_token(token)
else:
current = client.request('GET', '/rest/mcp/api-key')['apiKey']
if is_full_token(current):
# A new key was created for this user, so the full value is visible once.
save_token(current)
token = load_token()
status['mcpTokenSource'] = 'N8N_MCP_TOKEN' if setting('N8N_MCP_TOKEN') else 'cache'
status['mcpTokenAvailable'] = bool(token)
status['mcpTokenWorks'] = bool(token) and mcp_handshake(client.base_url, token)
if not status['mcpTokenWorks']:
status['mcpTokenHelp'] = (
'The MCP token is missing or invalid. Set N8N_MCP_TOKEN in .env.ai-surfaces, or run '
'prepare.py --rotate after the user agrees. Rotation invalidates the old key in every other MCP client.'
)
status['docker'] = docker_ready()
status['claudeOnHost'] = bool(shutil.which('claude'))
status['codexOnHost'] = bool(shutil.which('codex'))
print(json.dumps(status, indent=2))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,82 @@
#!/usr/bin/env node
// Record a video of an Instance AI thread in the n8n UI while the run continues.
//
// node record_iai.mjs <out-dir> [--live]
//
// --live also saves the screen to <out-dir>/live.png every few seconds, for the live page.
//
// Protocol on stdin and stdout, one line each:
// prints "ready" after sign-in
// reads "open <thread-url>" and opens the thread
// reads "stop", or stdin closes: keeps the last screen for a moment, then saves <out-dir>/screen.webm
// and the last screen as <out-dir>/screen-poster.png
// The sign-in happens in a separate context, so the video starts at the thread.
import { copyFileSync, mkdtempSync, renameSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { createInterface } from 'node:readline';
import { launchBrowser, signIn } from './n8n_browser.mjs';
const outDir = resolve(process.argv[2] ?? '.');
const live = process.argv.includes('--live');
const SIZE = { width: 1600, height: 1000 };
const LIVE_SHOT_MS = 2500;
const browser = await launchBrowser();
const login = await browser.newContext();
const loginPage = await login.newPage();
await signIn(loginPage);
const storageState = await login.storageState();
await login.close();
const videoDir = mkdtempSync(join(tmpdir(), 'cwb-video-'));
const context = await browser.newContext({
viewport: SIZE,
storageState,
recordVideo: { dir: videoDir, size: SIZE },
});
const page = await context.newPage();
console.log('ready');
let liveTimer;
let shooting = false;
async function saveLiveShot() {
if (shooting) return;
shooting = true;
// The live page can read the file at any time, so it gets only complete images.
const partial = join(outDir, 'live.tmp.png');
try {
await page.screenshot({ path: partial });
renameSync(partial, join(outDir, 'live.png'));
} catch {
// The page can be in the middle of a navigation. The next interval tries again.
} finally {
shooting = false;
}
}
const lines = createInterface({ input: process.stdin });
for await (const line of lines) {
if (line.startsWith('open ')) {
await page.goto(line.slice(5).trim(), { waitUntil: 'domcontentloaded' });
console.log('opened');
if (live && !liveTimer) liveTimer = setInterval(saveLiveShot, LIVE_SHOT_MS);
} else if (line.trim() === 'stop') {
break;
}
}
clearInterval(liveTimer);
await page.waitForTimeout(3000);
if (live) await saveLiveShot();
// The video starts on a blank page, so the report shows the last screen until playback starts.
await page.screenshot({ path: join(outDir, 'screen-poster.png') }).catch(() => {});
const video = page.video();
await context.close();
if (video) {
copyFileSync(await video.path(), join(outDir, 'screen.webm'));
console.log('saved screen.webm');
}
await browser.close();
rmSync(videoDir, { recursive: true, force: true });

View File

@ -0,0 +1,192 @@
"""Record the request bodies that a CLI sends to its model API. Standard library only.
The recorder is a local HTTP proxy. The CLI sends its model requests to the recorder, and the
recorder sends them to the real API without changes. It writes the request bodies to a JSONL file.
It never writes headers, so the API key stays out of the file.
Each row has the request settings without the conversation input. The tools and the instructions
are in a row only when they differ from the previous request of the same thread, because the CLI
sends them every turn.
The tools field also holds the tools of additional_tools input items.
"""
import gzip
import http.client
import json
import threading
import time
import zlib
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import urlparse
# Headers that apply to one connection only. The recorder sets its own values.
HOP_HEADERS = {'connection', 'keep-alive', 'proxy-connection', 'proxy-authorization', 'te', 'trailer',
'transfer-encoding', 'upgrade', 'host', 'content-length', 'accept-encoding'}
CHANGING_KEYS = ('tools', 'instructions')
def decode_body(data, encoding):
encoding = (encoding or '').lower()
if encoding == 'gzip':
return gzip.decompress(data)
if encoding == 'deflate':
return zlib.decompress(data)
if encoding == 'zstd':
from compression import zstd
return zstd.decompress(data)
return data
class RequestRecorder:
"""Start with start(). Point the CLI at base_url(host). Call stop() when the CLI is done."""
def __init__(self, upstream, out_file, bind='127.0.0.1'):
self.upstream = urlparse(upstream)
self.out_file = out_file
self.lock = threading.Lock()
self.idle = threading.Condition(self.lock)
self.active = 0
self.last = {}
self.count = 0
self.started = time.time()
self.server = ThreadingHTTPServer((bind, 0), self.handler())
self.server.daemon_threads = True
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
@property
def port(self):
return self.server.server_address[1]
def base_url(self, host):
return f'http://{host}:{self.port}{self.upstream.path.rstrip("/")}'
def start(self):
self.out_file.parent.mkdir(parents=True, exist_ok=True)
self.out_file.write_text('')
self.thread.start()
return self
def stop(self, timeout=30):
"""Stop the server. Wait until the open requests are written, so count is final."""
self.server.shutdown()
with self.idle:
self.idle.wait_for(lambda: self.active == 0, timeout=timeout)
self.server.server_close()
def begin(self):
with self.lock:
self.active += 1
def end(self):
with self.idle:
self.active -= 1
self.idle.notify_all()
def record(self, method, path, body, encoding, status, error_text):
row = {'t': round(time.time() - self.started, 3), 'method': method, 'path': path, 'status': status}
try:
payload = json.loads(decode_body(body, encoding)) if body else None
except (ValueError, OSError, ImportError):
payload, row['bodyBytes'] = None, len(body)
with self.lock:
if isinstance(payload, dict):
items = payload.get('input') if isinstance(payload.get('input'), list) else []
if items:
row['inputItems'] = len(items)
# Codex sends its tools as an additional_tools input item, not in the top-level tools field.
extra = [t for i in items if isinstance(i, dict) and i.get('type') == 'additional_tools'
for t in i.get('tools') or []]
if extra:
payload = {**payload, 'tools': (payload.get('tools') or []) + extra}
# Side tasks, such as a title request, run on their own thread with other tools.
metadata = payload.get('client_metadata') if isinstance(payload.get('client_metadata'), dict) else {}
thread = metadata.get('thread_id') or payload.get('prompt_cache_key') or ''
for key, value in payload.items():
if key == 'input':
continue
if key in CHANGING_KEYS:
if self.last.get((thread, key)) == value:
continue
self.last[(thread, key)] = value
row[key] = value
if error_text:
row['error'] = error_text
self.count += 1
with self.out_file.open('a') as f:
f.write(json.dumps(row, ensure_ascii=False) + '\n')
def handler(self):
recorder = self
class Handler(BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'
def log_message(self, *args):
pass
def read_body(self):
if self.headers.get('Content-Length'):
return self.rfile.read(int(self.headers['Content-Length']))
if 'chunked' not in (self.headers.get('Transfer-Encoding') or '').lower():
return b''
parts = []
while True:
size = int(self.rfile.readline().split(b';')[0].strip() or b'0', 16)
if not size:
self.rfile.readline()
return b''.join(parts)
parts.append(self.rfile.read(size))
self.rfile.readline()
def forward(self):
recorder.begin()
try:
self.forward_and_record()
finally:
recorder.end()
def forward_and_record(self):
body = self.read_body()
up = recorder.upstream
conn_class = http.client.HTTPSConnection if up.scheme == 'https' else http.client.HTTPConnection
conn = conn_class(up.hostname, up.port, timeout=900)
headers = {k: v for k, v in self.headers.items() if k.lower() not in HOP_HEADERS}
headers['Host'] = up.netloc
headers['Content-Length'] = str(len(body))
status, error_text, replied = 502, None, False
try:
conn.request(self.command, self.path, body=body, headers=headers)
resp = conn.getresponse()
status, replied = resp.status, True
self.send_response(resp.status, resp.reason)
for key, value in resp.getheaders():
if key.lower() not in HOP_HEADERS:
self.send_header(key, value)
self.send_header('Transfer-Encoding', 'chunked')
self.end_headers()
kept = b''
while chunk := resp.read1(65536):
if status >= 400 and len(kept) < 4000:
kept += chunk
self.wfile.write(b'%x\r\n%s\r\n' % (len(chunk), chunk))
self.wfile.flush()
self.wfile.write(b'0\r\n\r\n')
self.wfile.flush()
if kept:
error_text = kept[:4000].decode('utf-8', 'replace')
except (OSError, http.client.HTTPException) as e:
error_text = f'recorder: {type(e).__name__}'
if not replied:
try:
self.send_error(502, 'Recorder could not reach the API')
except OSError:
pass
self.close_connection = True
finally:
conn.close()
recorder.record(self.command, self.path, body, self.headers.get('Content-Encoding'),
status, error_text)
do_GET = do_POST = do_PUT = do_DELETE = do_PATCH = forward
return Handler

View File

@ -0,0 +1,613 @@
#!/usr/bin/env python3
"""Run one prompt through Claude Code or Codex with only the n8n MCP server.
run_cli_builder.py --builder claude --out <run-dir>/claude --prompt-file <run-dir>/prompt.txt
run_cli_builder.py --builder codex --out <run-dir>/codex --prompt-file <run-dir>/prompt.txt
Isolation modes:
container (default) Docker container. The only mount is the output directory.
The builder cannot see host files, skills, memory or settings.
clean Claude only. A clean process with an empty config directory, no built-in
tools and only the n8n MCP server. It needs an API key, not a subscription login.
Screen modes:
interactive (default in a container) The normal terminal screen of the CLI runs in tmux.
screen.cast records the exact screen, and a terminal window can watch it live.
The trace comes from the session file of the CLI.
headless `claude -p` or `codex exec` with a JSON event stream. Clean isolation uses this mode.
Secrets: the model key and the MCP token go through environment variables only.
They are never written to a file or put on a command line.
Codex sends its model requests through a local recorder (request_recorder.py), which writes the
request bodies, but not the headers, to requests.jsonl. Set COMPARE_RECORD_REQUESTS=0 to turn it off.
"""
import argparse
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import threading
import time
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse
from live import enabled as live_enabled
from live import open_log_window, open_terminal
from n8n_api import DEFAULT_BASE_URL, load_token, setting
from request_recorder import RequestRecorder
SKILL_DIR = Path(__file__).resolve().parents[1]
IMAGE = 'n8n-builder-sandbox'
# The builder-specific name comes first, so the CLI can use a different key than
# the ANTHROPIC_API_KEY or OPENAI_API_KEY that the instance reads from .env.local.
KEY_SETTINGS = {'claude': ['CLAUDE_API_KEY', 'ANTHROPIC_API_KEY'], 'codex': ['CODEX_API_KEY', 'OPENAI_API_KEY']}
# Codex features that add tools or downloads beyond the n8n MCP server. shell_snapshot is off
# because it writes the environment, with the API key, to a file.
CODEX_FEATURES_OFF = ['apps', 'browser_use', 'browser_use_external', 'computer_use', 'image_generation',
'in_app_browser', 'plugins', 'remote_plugin', 'shell_snapshot']
SCREEN = {'cols': 120, 'rows': 36}
OPENAI_BASE_URL = 'https://api.openai.com/v1'
# Interactive mode: the variable that the CLI reads the model key from.
SANDBOX_KEY = {'claude': 'CLAUDE_SANDBOX_KEY', 'codex': 'CODEX_SANDBOX_KEY'}
IDLE_LIMIT_S = 600
def now_iso():
return datetime.now(timezone.utc).isoformat()
def ensure_image(rebuild):
# The label holds a hash of the Dockerfile, so an edit to the Dockerfile rebuilds the image.
digest = hashlib.sha256((SKILL_DIR / 'sandbox' / 'Dockerfile').read_bytes()).hexdigest()[:16]
found = subprocess.run(['docker', 'image', 'inspect', '--format',
'{{index .Config.Labels "dockerfile-sha"}}', IMAGE], capture_output=True, text=True)
if found.returncode == 0 and found.stdout.strip() == digest and not rebuild:
return
print(f'building docker image {IMAGE}', flush=True)
subprocess.run(['docker', 'build', '--label', f'dockerfile-sha={digest}', '-t', IMAGE,
str(SKILL_DIR / 'sandbox')], check=True)
def mcp_url(base_url, isolation):
if isolation != 'container':
return f'{base_url}/mcp-server/http'
parsed = urlparse(base_url)
port = parsed.port or (443 if parsed.scheme == 'https' else 80)
host = 'host.docker.internal' if parsed.hostname in ('localhost', '127.0.0.1') else parsed.hostname
return f'{parsed.scheme}://{host}:{port}/mcp-server/http'
def claude_tool_args(tools):
# 'default' gives the normal Claude Code tools, all allowed without a question, like Codex in the
# container. Only a container may use it, because Bash and Write can change the files they see.
if tools == 'default':
return ['--permission-mode', 'bypassPermissions']
# dontAsk allows the pre-approved n8n tools and denies the rest without a prompt.
return ['--tools', '', '--allowedTools', 'mcp__n8n', '--permission-mode', 'dontAsk']
def claude_args(prompt, model, config_path, bare, tools='none'):
# --bare also replaces the system prompt with a minimal one. A container already isolates the CLI,
# so only the clean mode, which runs on the host, uses it.
args = [
'claude', '-p', prompt, *(['--bare'] if bare else []),
'--strict-mcp-config', '--mcp-config', config_path, *claude_tool_args(tools),
'--disable-slash-commands', '--output-format', 'stream-json', '--verbose',
]
if model:
args += ['--model', model]
return args
def claude_interactive_args(model, tools='none'):
args = [
'claude', '--strict-mcp-config', '--mcp-config', '/out/mcp.json', *claude_tool_args(tools),
'--disable-slash-commands', '--settings', '/out/.screen/claude-settings.json',
]
if model:
args += ['--model', model]
return args
def codex_args(prompt, model):
args = ['codex', 'exec', '--json', '--skip-git-repo-check', '--sandbox', 'read-only']
if model:
args += ['--model', model]
return args + [prompt]
def codex_interactive_args(model):
# The background app server needs `ps`, which the slim image does not have.
args = ['codex', '--no-daemon']
if model:
args += ['--model', model]
return args
def write_codex_config(home, url, key_env=None, api_base=OPENAI_BASE_URL):
"""Write config.toml. With key_env, a provider reads the key from that variable, so the
interactive screen does not ask for a sign-in and no key is stored in auth.json."""
codex_home = home / '.codex'
codex_home.mkdir(parents=True, exist_ok=True)
lines = ['approval_policy = "never"', 'sandbox_mode = "read-only"']
if key_env:
lines.append('model_provider = "sandbox-openai"')
lines += ['', '[features]'] + [f'{name} = false' for name in CODEX_FEATURES_OFF]
if key_env:
lines += ['', '[model_providers.sandbox-openai]', 'name = "OpenAI"',
f'base_url = "{api_base}"', f'env_key = "{key_env}"', 'wire_api = "responses"']
lines += ['', '[projects."/out/work"]', 'trust_level = "trusted"',
'', '[mcp_servers.n8n]', f'url = "{url}"', 'bearer_token_env_var = "N8N_MCP_TOKEN"', '']
(codex_home / 'config.toml').write_text('\n'.join(lines))
def write_claude_interactive_config(out, home, tools):
# Skip the first-run screens: theme, onboarding and the folder trust question.
(home / '.claude' / '.claude.json').write_text(json.dumps({
'hasCompletedOnboarding': True, 'theme': 'dark',
'projects': {'/out/work': {'hasTrustDialogAccepted': True, 'hasCompletedProjectOnboarding': True}},
}))
# With ANTHROPIC_API_KEY the screen asks to confirm the key and shows part of it.
# apiKeyHelper reads the key from the environment without that question.
settings = {'apiKeyHelper': f'printenv {SANDBOX_KEY["claude"]}'}
if tools == 'default':
# The bypass-mode warning screen also has a ❯ marker, so the prompt would answer "No, exit".
settings['skipDangerousModePermissionPrompt'] = True
(out / '.screen' / 'claude-settings.json').write_text(json.dumps(settings))
def live_line(builder, event):
"""One short terminal line for an interesting stream event, or None."""
if builder == 'claude':
if event.get('subtype') == 'permission_denied':
return f'denied {event.get("tool_name")}'
if event.get('type') == 'assistant':
for block in (event.get('message') or {}).get('content') or []:
if block.get('type') == 'tool_use':
return f'tool {block.get("name")}'
if block.get('type') == 'text' and block.get('text', '').strip():
return 'text ' + block['text'].strip().splitlines()[0][:120]
if event.get('type') == 'result':
return f'result {event.get("subtype")} turns={event.get("num_turns")}'
else:
item = event.get('item') or {}
if event.get('type') == 'item.started' and item.get('type') == 'mcp_tool_call':
return f'tool {item.get("server")}.{item.get("tool")}'
if event.get('type') == 'item.completed' and item.get('type') == 'agent_message':
return 'text ' + (item.get('text') or '').strip()[:120]
if event.get('type') in ('turn.completed', 'turn.failed', 'error'):
return event['type']
return None
def session_lines(builder, row):
"""Short terminal lines for one session-file row, and the end state: None, 'done' or 'error'."""
lines = []
if builder == 'claude':
if row.get('type') != 'assistant':
return lines, None
msg = row.get('message') or {}
for block in msg.get('content') or []:
if block.get('type') == 'tool_use':
lines.append(f'tool {block.get("name")}')
elif block.get('type') == 'text' and block.get('text', '').strip():
lines.append('text ' + block['text'].strip().splitlines()[0][:120])
if row.get('isApiErrorMessage'):
return lines, 'error'
return lines, 'done' if msg.get('stop_reason') == 'end_turn' else None
payload = row.get('payload') or {}
if row.get('type') != 'event_msg':
return lines, None
kind, item = payload.get('type'), payload.get('item') or {}
if kind == 'item_completed' and item.get('type') == 'McpToolCall':
lines.append(f'tool {item.get("server")}.{item.get("tool")}')
elif kind == 'item_completed' and item.get('type') == 'AgentMessage':
text = ' '.join(c.get('text', '') for c in item.get('content') or [] if isinstance(c, dict))
lines.append('text ' + text.strip()[:120])
elif kind == 'error':
lines.append(f'error {str(payload.get("message"))[:200]}')
return lines, {'task_complete': 'done', 'turn_aborted': 'error'}.get(kind)
BOLD, DIM, CYAN, GREEN, RED, RESET = '\033[1m', '\033[2m', '\033[36m', '\033[32m', '\033[31m', '\033[0m'
LABELS = {'claude': 'Claude Code + n8n MCP', 'codex': 'Codex + n8n MCP'}
def short(value, limit=400):
"""Readable one-line text for a tool input or result."""
if isinstance(value, dict) and isinstance(value.get('content'), list):
value = value['content']
if isinstance(value, list) and all(isinstance(v, dict) and 'text' in v for v in value):
value = ' '.join(v['text'] for v in value)
text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
text = ' '.join(text.split())
return text if len(text) <= limit else text[:limit] + ' …'
def log_entries(builder, event):
"""Detailed, colored lines for live.log, the file that the headless live window follows."""
lines = []
if builder == 'claude':
# A permission_denied event has a string message, not an API message.
if event.get('subtype') == 'permission_denied':
lines.append(f'{RED}denied {event.get("tool_name")}{RESET}')
message = event.get('message')
content = message.get('content') if isinstance(message, dict) else None
for block in content if isinstance(content, list) else []:
kind = block.get('type')
if kind == 'thinking' and block.get('thinking', '').strip():
lines.append(f'{DIM}thinking: {short(block["thinking"], 600)}{RESET}')
elif kind == 'text' and block.get('text', '').strip():
lines.append(block['text'].strip())
elif kind == 'tool_use':
name = block.get('name', '').removeprefix('mcp__n8n__')
lines.append(f'{BOLD}{CYAN}→ {name}{RESET} {short(block.get("input"))}')
elif kind == 'tool_result':
color = RED if block.get('is_error') else DIM
lines.append(f'{color}← {short(block.get("content"))}{RESET}')
if event.get('type') == 'result':
cost = event.get('total_cost_usd')
lines.append(f'{GREEN}done: {event.get("subtype")}, {event.get("num_turns")} turns'
+ (f', ${cost:.2f} (Claude Code estimate)' if cost is not None else '') + RESET)
else:
item, etype = event.get('item') or {}, event.get('type')
kind = item.get('type')
if etype == 'item.started' and kind == 'mcp_tool_call':
lines.append(f'{BOLD}{CYAN}→ {item.get("tool")}{RESET} {short(item.get("arguments"))}')
elif etype == 'item.completed' and kind == 'mcp_tool_call':
if item.get('error'):
lines.append(f'{RED}← {short(item["error"])}{RESET}')
else:
lines.append(f'{DIM}← {short(item.get("result"))}{RESET}')
elif etype == 'item.completed' and kind == 'reasoning' and item.get('text'):
lines.append(f'{DIM}thinking: {short(item["text"], 600)}{RESET}')
elif etype == 'item.completed' and kind == 'agent_message' and item.get('text'):
lines.append(item['text'].strip())
elif etype == 'turn.completed':
lines.append(f'{GREEN}done: turn completed{RESET}')
elif etype in ('turn.failed', 'error'):
lines.append(f'{RED}{etype}: {short(event.get("error") or event.get("message"))}{RESET}')
return lines
WORKFLOW_ID = re.compile(r'(?:workflowId|WORKFLOW_ID)\\*"?\s*[:=]\s*\\*"?([A-Za-z0-9]{10,})')
def session_candidates(builder, home):
if builder == 'claude':
# Subagent transcripts are in deeper folders. The main session is at projects/<cwd>/<id>.jsonl.
root = home / '.claude' / 'projects'
return list(root.glob('*/*.jsonl')) if root.exists() else []
root = home / '.codex' / 'sessions'
return list(root.rglob('rollout-*.jsonl')) if root.exists() else []
def find_session_file(builder, home, session_id=None):
matches = session_candidates(builder, home)
if session_id:
matches = [p for p in matches if session_id in p.name] or matches
matches.sort(key=lambda p: p.stat().st_mtime)
return matches[-1] if matches else None
class SessionReader:
"""Reads the rows that the CLI appends to its session file."""
def __init__(self, builder, home):
self.builder, self.home, self.path, self.offset = builder, home, None, 0
def rows(self):
self.path = self.path or find_session_file(self.builder, self.home)
if not self.path:
return []
with self.path.open('rb') as f:
f.seek(self.offset)
data = f.read()
end = data.rfind(b'\n') + 1
self.offset += end
rows = []
for line in data[:end].splitlines():
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def docker_command(container, out, env_names):
cmd = [
'docker', 'run', '--rm', '-i', '--name', container,
'--user', f'{os.getuid()}:{os.getgid()}',
'--cap-drop', 'ALL', '--security-opt', 'no-new-privileges',
'--read-only', '--tmpfs', '/tmp',
'--add-host', 'host.docker.internal:host-gateway',
'-v', f'{out}:/out', '-w', '/out/work',
'-e', 'HOME=/out/home',
'-e', 'CLAUDE_CONFIG_DIR=/out/home/.claude',
'-e', 'CODEX_HOME=/out/home/.codex',
'-e', 'CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1',
'-e', 'COLORTERM=truecolor',
]
# `-e NAME` without a value copies the value from this process, so secrets stay off argv.
for name in env_names:
cmd += ['-e', name]
return cmd + [IMAGE]
def check_no_secrets(out, secrets):
"""Warn if a secret value ended up in a file of the run directory. Never prints the value."""
needles = [s.encode() for s in secrets if s and len(s) >= 12]
for path in out.rglob('*'):
if not path.is_file() or path.stat().st_size > 50_000_000:
continue
data = path.read_bytes()
if any(n in data for n in needles):
print(f'WARNING: a secret value is in {path}. Delete the file before you share the run.', flush=True)
def run_headless(args, out, prompt, cmd, run_env, cwd, container):
stream_path = out / 'stream.jsonl'
session_id, workflow_ids, result, errors_in_row = None, [], None, 0
live_log = (out / 'live.log').open('w', buffering=1)
live_log.write(f'{BOLD}{LABELS[args.builder]}{RESET} model: {args.model or "CLI default"}, '
f'isolation: {args.isolation}\n{DIM}prompt: {short(prompt, 600)}{RESET}\n\n')
open_log_window(LABELS[args.builder], out / 'live.log')
# The Codex stream has no timestamps, so stream.times keeps the arrival time of each line.
with stream_path.open('w') as stream, (out / 'stream.times').open('w') as times, \
(out / 'stderr.log').open('w') as err, live_log:
proc = subprocess.Popen(cmd, env=run_env, cwd=cwd, stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE, stderr=err, text=True)
watchdog = threading.Timer(args.timeout_min * 60, proc.kill)
watchdog.start()
for line in proc.stdout:
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
stream.write(line)
stream.flush()
times.write(f'{time.time():.3f}\n')
if event.get('type') == 'system' and event.get('subtype') == 'init':
session_id = event.get('session_id')
if event.get('type') == 'thread.started':
session_id = event.get('thread_id')
if event.get('type') == 'result':
result = event
for match in WORKFLOW_ID.findall(line):
if match not in workflow_ids:
workflow_ids.append(match)
text = live_line(args.builder, event)
if text:
print(f'[{args.builder}] {text}', flush=True)
for entry in log_entries(args.builder, event):
live_log.write(entry + '\n\n')
errors_in_row = errors_in_row + 1 if event.get('type') == 'error' else 0
if errors_in_row >= 10:
print(f'[{args.builder}] stopped after 10 errors in a row: {event.get("message", "")[:200]}', flush=True)
proc.kill()
break
proc.wait()
if not watchdog.is_alive():
print('timeout: killed the builder', flush=True)
watchdog.cancel()
if args.isolation == 'container':
subprocess.run(['docker', 'kill', container], capture_output=True)
live_log.write(f'{BOLD}=== finished, exit code {proc.returncode}. You can close this window. ==={RESET}\n')
return {'exitCode': proc.returncode, 'sessionId': session_id, 'workflowIds': workflow_ids,
'reportedCostUsd': (result or {}).get('total_cost_usd')}
def run_interactive(args, out, home, prompt, run_env, container):
screen_dir = out / '.screen'
argv = (claude_interactive_args(args.model, args.claude_tools) if args.builder == 'claude'
else codex_interactive_args(args.model))
# Claude Code shows its input box before the MCP servers connect, so it waits a few seconds more.
ready = {'claude': ('❯', 5000), 'codex': ('›', 2000)}[args.builder]
(screen_dir / 'command.json').write_text(json.dumps(
{'argv': argv, **SCREEN, 'input': prompt, 'readyText': ready[0], 'settleMs': ready[1]}))
cmd = docker_command(container, out, [n for n in run_env if n in (SANDBOX_KEY[args.builder], 'N8N_MCP_TOKEN')])
cmd += ['node', '/out/.screen/screen.mjs', 'run', '/out/.screen']
reader, workflow_ids, state, timed_out = SessionReader(args.builder, home), [], None, False
with (out / 'stderr.log').open('w') as err:
proc = subprocess.Popen(cmd, env=run_env, stdin=subprocess.DEVNULL, stdout=err, stderr=err)
for _ in range(120):
if (screen_dir / 'screen.cast').exists() or proc.poll() is not None:
break
time.sleep(0.25)
watch = f'docker exec -it {container} tmux attach -r -t run'
if live_enabled() and not open_terminal(LABELS[args.builder], watch, SCREEN['cols'], SCREEN['rows']):
print(f'watch {LABELS[args.builder]}: {watch}', flush=True)
deadline, last_growth = time.time() + args.timeout_min * 60, time.time()
while proc.poll() is None and not state:
rows = reader.rows()
if rows:
last_growth = time.time()
for row in rows:
lines, row_state = session_lines(args.builder, row)
for line in lines:
print(f'[{args.builder}] {line}', flush=True)
for match in WORKFLOW_ID.findall(json.dumps(row)):
if match not in workflow_ids:
workflow_ids.append(match)
state = state or row_state
if time.time() > deadline:
print('timeout: stopped the builder', flush=True)
timed_out = True
break
if reader.path and time.time() - last_growth > IDLE_LIMIT_S:
print(f'no progress for {IDLE_LIMIT_S // 60} minutes: stopped the builder', flush=True)
timed_out = True
break
time.sleep(1)
(screen_dir / 'stop').write_text('')
try:
proc.wait(timeout=30)
except subprocess.TimeoutExpired:
subprocess.run(['docker', 'kill', container], capture_output=True)
proc.wait()
for row in reader.rows():
for match in WORKFLOW_ID.findall(json.dumps(row)):
if match not in workflow_ids:
workflow_ids.append(match)
if (screen_dir / 'screen.cast').exists():
shutil.move(screen_dir / 'screen.cast', out / 'screen.cast')
# After a finished turn the driver stops tmux, so the exit code of the CLI means nothing.
exited = screen_dir / 'exited'
if state == 'done' and not timed_out:
code = 0
else:
code = int(exited.read_text() or 1) if exited.exists() else 1
code = code or 1
session = reader.path
return {'exitCode': code, 'sessionId': session.stem.split('rollout-')[-1] if session else None,
'workflowIds': workflow_ids, 'reportedCostUsd': None}
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--builder', choices=['claude', 'codex'], required=True)
ap.add_argument('--out', required=True)
ap.add_argument('--prompt-file', required=True)
ap.add_argument('--isolation', choices=['container', 'clean'],
default=setting('COMPARE_ISOLATION', default='container'))
ap.add_argument('--mode', choices=['interactive', 'headless'], default=setting('COMPARE_CLI_MODE'),
help='Default: interactive in a container, headless in clean isolation.')
ap.add_argument('--model', help='Model id. Default: CLAUDE_MODEL or CODEX_MODEL, else the CLI default.')
ap.add_argument('--key-env', help='Name of the setting with the model API key.')
ap.add_argument('--claude-tools', choices=['none', 'default'],
default=setting('COMPARE_CLAUDE_TOOLS', default='none'),
help='none: only the n8n tools. default: also the built-in Claude Code tools, all allowed.')
ap.add_argument('--rebuild-image', action='store_true')
ap.add_argument('--timeout-min', type=float, default=30)
args = ap.parse_args()
if args.claude_tools not in ('none', 'default'):
sys.exit(f'COMPARE_CLAUDE_TOOLS must be none or default, not {args.claude_tools!r}.')
if args.builder == 'codex' and args.isolation != 'container':
sys.exit('Codex runs only in a container. Its shell tool can read the host disk.')
if args.isolation != 'container':
args.mode = 'headless'
if args.builder == 'claude' and args.claude_tools != 'none':
print('[claude] clean isolation runs on the host, so COMPARE_CLAUDE_TOOLS=default is ignored.', flush=True)
args.claude_tools = 'none'
args.mode = args.mode or 'interactive'
prefix = args.builder.upper()
args.model = args.model or setting(f'{prefix}_MODEL')
# Gateway-style ids such as anthropic/claude-opus-5-5 are not valid for the CLIs.
provider = {'claude': 'anthropic/', 'codex': 'openai/'}[args.builder]
if args.model and args.model.startswith(provider):
args.model = args.model.removeprefix(provider)
key_names = [args.key_env] if args.key_env else KEY_SETTINGS[args.builder]
api_key = setting(*key_names)
if not api_key:
sys.exit(f'Set {" or ".join(key_names)} in .env.ai-surfaces, .env.local or the environment.')
token = load_token()
if not token:
sys.exit('No MCP token. Run prepare.py first.')
out = Path(args.out).resolve()
if out.exists():
shutil.rmtree(out)
work, home = out / 'work', out / 'home'
work.mkdir(parents=True)
(home / '.claude').mkdir(parents=True)
prompt = Path(args.prompt_file).read_text().strip()
base_url = setting('N8N_BASE_URL', default=DEFAULT_BASE_URL).rstrip('/')
url = mcp_url(base_url, args.isolation)
container = f'cwb-{args.builder}-{int(time.time())}'
interactive = args.mode == 'interactive'
# Codex does not save the tools it sends to the model, so a local proxy records the request bodies.
recorder = None
if args.builder == 'codex' and setting('COMPARE_RECORD_REQUESTS', default='1') != '0':
recorder = RequestRecorder(OPENAI_BASE_URL, out / 'requests.jsonl',
setting('COMPARE_RECORDER_BIND', default='127.0.0.1')).start()
api_base = recorder.base_url('host.docker.internal') if recorder else OPENAI_BASE_URL
child_env = {'N8N_MCP_TOKEN': token}
if args.builder == 'claude':
(out / 'mcp.json').write_text(json.dumps({'mcpServers': {'n8n': {
'type': 'http', 'url': url, 'headers': {'Authorization': 'Bearer ${N8N_MCP_TOKEN}'},
}}}))
if interactive:
(out / '.screen').mkdir()
shutil.copy(SKILL_DIR / 'sandbox' / 'screen.mjs', out / '.screen' / 'screen.mjs')
child_env[SANDBOX_KEY[args.builder]] = api_key
if args.builder == 'claude':
write_claude_interactive_config(out, home, args.claude_tools)
else:
write_codex_config(home, url, SANDBOX_KEY['codex'], api_base)
elif args.builder == 'claude':
child_env['ANTHROPIC_API_KEY'] = api_key
else:
child_env['OPENAI_API_KEY'] = api_key
child_env['CODEX_API_KEY'] = api_key
write_codex_config(home, url, 'CODEX_API_KEY' if recorder else None, api_base)
started_at = now_iso()
if args.isolation == 'container':
ensure_image(args.rebuild_image)
run_env = {**os.environ, **child_env}
try:
if interactive:
result = run_interactive(args, out, home, prompt, run_env, container)
else:
cmd = docker_command(container, out, child_env)
cmd += (claude_args(prompt, args.model, '/out/mcp.json', False, args.claude_tools)
if args.builder == 'claude' else codex_args(prompt, args.model))
result = run_headless(args, out, prompt, cmd, run_env, None, container)
finally:
if recorder:
recorder.stop()
if recorder and not recorder.count:
print(f'[{args.builder}] the request recorder saw no requests, so the report has no Codex tools.', flush=True)
else:
if not shutil.which('claude'):
sys.exit('claude is not on PATH.')
run_env = {
'PATH': os.environ.get('PATH', '/usr/bin:/bin'),
'HOME': str(home),
'CLAUDE_CONFIG_DIR': str(home / '.claude'),
'DISABLE_AUTOUPDATER': '1',
'DISABLE_TELEMETRY': '1',
**child_env,
}
cmd = claude_args(prompt, args.model, str(out / 'mcp.json'), True)
result = run_headless(args, out, prompt, cmd, run_env, str(work), container)
session_file = find_session_file(args.builder, home, result['sessionId'])
meta = {
'kind': args.builder,
'isolation': args.isolation,
'mode': args.mode,
'model': args.model,
**({'claudeTools': args.claude_tools} if args.builder == 'claude' else {}),
'startedAt': started_at,
'finishedAt': now_iso(),
'exitCode': result['exitCode'],
'sessionId': result['sessionId'],
'sessionFile': str(session_file.relative_to(out)) if session_file else None,
'workflowIds': result['workflowIds'],
'workflowId': result['workflowIds'][-1] if result['workflowIds'] else None,
'reportedCostUsd': result['reportedCostUsd'],
'screen': 'screen.cast' if (out / 'screen.cast').exists() else None,
'requests': 'requests.jsonl' if recorder and recorder.count else None,
}
(out / 'run.json').write_text(json.dumps(meta, indent=2))
shutil.rmtree(home / '.codex' / 'shell_snapshots', ignore_errors=True)
check_no_secrets(out, [api_key, token])
print(json.dumps({k: meta[k] for k in ('exitCode', 'sessionId', 'workflowId', 'sessionFile', 'screen')}, indent=2))
if meta['exitCode']:
print(f'see {out / "stderr.log"}', flush=True)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,294 @@
#!/usr/bin/env python3
"""Run one prompt through Instance AI on the local n8n instance.
The script sends the prompt to a new thread, answers every confirmation card with
a fixed policy, waits for the thread to go idle, and exports the trace.
run_instance_ai.py --out <run-dir>/iai --prompt-file <run-dir>/prompt.txt
Approval policy (the same choices for every run, so runs are comparable):
- approval, plan review, web search, continue: approve
- text input: approve with "Use sensible defaults and continue."
- questions: pick the first option, or skip a free-text question
- domain access: allow once
- workflow or credential setup cards: defer (the builder must not wait for real credentials)
- local gateway resources and MCP connect cards: deny
"""
import argparse
import json
import os
import queue
import re
import shutil
import subprocess
import sys
import threading
import time
import uuid
from datetime import datetime, timezone
from pathlib import Path
from live import enabled as live_enabled
from live import open_url, recording
from n8n_api import N8nClient
DEFAULT_TEXT_ANSWER = 'Use sensible defaults and continue.'
HERE = Path(__file__).resolve().parent
class Recorder:
"""Video of the thread in the n8n UI, made by record_iai.mjs."""
def __init__(self, out):
self.proc, self.lines = None, queue.Queue()
node = shutil.which('node')
if not recording() or not node:
return
live_flag = ['--live'] if live_enabled() else []
self.proc = subprocess.Popen([node, str(HERE / 'record_iai.mjs'), str(out.resolve()), *live_flag], cwd=HERE,
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=(out / 'record.log').open('w'), text=True)
threading.Thread(target=self._read, daemon=True).start()
def _read(self):
for line in self.proc.stdout:
self.lines.put(line.strip())
def wait_for(self, word, timeout):
deadline = time.time() + timeout
while self.proc and time.time() < deadline:
try:
if self.lines.get(timeout=min(max(deadline - time.time(), 0.1), 1)) == word:
return True
except queue.Empty:
if self.proc.poll() is not None:
return False
return False
def send(self, line):
if self.proc and self.proc.poll() is None:
self.proc.stdin.write(line + '\n')
self.proc.stdin.flush()
def stop(self):
if not self.proc:
return
self.send('stop')
self.proc.stdin.close()
try:
self.proc.wait(timeout=120)
except subprocess.TimeoutExpired:
self.proc.kill()
def now_iso():
return datetime.now(timezone.utc).isoformat()
def local_time_zone():
zone = os.environ.get('TZ')
if zone:
return zone
link = os.path.realpath('/etc/localtime')
return link.split('zoneinfo/', 1)[1] if 'zoneinfo/' in link else 'UTC'
def answer_questions(questions):
answers = []
for q in questions or []:
options = q.get('options') or []
if q.get('type') in ('single', 'multi') and options:
answers.append({'questionId': q['id'], 'selectedOptions': [options[0]]})
elif q.get('required'):
answers.append({'questionId': q['id'], 'selectedOptions': [], 'customText': DEFAULT_TEXT_ANSWER})
else:
answers.append({'questionId': q['id'], 'selectedOptions': [], 'skipped': True})
return {'kind': 'questions', 'answers': answers}
def decide(payload):
"""Return (confirm body, short label) for one confirmation-request payload."""
input_type = payload.get('inputType') or 'approval'
if payload.get('credentialDestination'):
origin = payload['credentialDestination'].get('origin')
return {'kind': 'credentialDestination', 'origin': origin, 'approved': False}, 'defer credential destination'
if payload.get('setupRequests') or payload.get('credentialRequests') or payload.get('credentialFlow'):
return {'kind': 'approval', 'approved': False}, 'defer setup'
if payload.get('domainAccess'):
return {'kind': 'domainAccessApprove', 'domainAccessAction': 'allow_once'}, 'allow domain once'
if payload.get('mcpConnectRequest'):
return {'kind': 'mcpConnect', 'approved': False}, 'deny MCP connect'
if input_type == 'resource-decision':
return {'kind': 'resourceDecision', 'resourceDecision': 'denyOnce'}, 'deny local resource'
if input_type == 'questions':
return answer_questions(payload.get('questions')), 'answer questions'
if input_type == 'text':
return {'kind': 'approval', 'approved': True, 'userInput': DEFAULT_TEXT_ANSWER}, 'answer text'
return {'kind': 'approval', 'approved': True}, f'approve {input_type}'
def read_stream(client, thread_id, out_queue, stop):
try:
res = client.open_stream(f'/rest/instance-ai/events/{thread_id}')
except Exception as e:
out_queue.put(('error', str(e)))
return
out_queue.put(('open', None))
data_lines, event_name = [], None
while not stop.is_set():
raw = res.readline()
if not raw:
out_queue.put(('closed', None))
return
line = raw.decode(errors='replace').rstrip('\r\n')
if line == '':
if data_lines:
try:
out_queue.put(('event', (event_name, json.loads('\n'.join(data_lines)))))
except json.JSONDecodeError:
pass
data_lines, event_name = [], None
elif line.startswith('data:'):
data_lines.append(line[5:].lstrip())
elif line.startswith('event:'):
event_name = line[6:].strip()
def collect_workflow_ids(value, found):
text = json.dumps(value)
for match in re.finditer(r'"workflowId"\s*:\s*"([A-Za-z0-9]{10,})"', text):
if match.group(1) not in found:
found.append(match.group(1))
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--out', required=True)
ap.add_argument('--prompt-file', required=True)
ap.add_argument('--timeout-min', type=float, default=30)
args = ap.parse_args()
out = Path(args.out)
out.mkdir(parents=True, exist_ok=True)
prompt = Path(args.prompt_file).read_text().strip()
# The recorder signs in while the event stream opens, so the video starts soon after the prompt.
recorder = Recorder(out)
client = N8nClient().login()
thread_id = str(uuid.uuid4())
events_file = (out / 'events.jsonl').open('w')
q, stop = queue.Queue(), threading.Event()
threading.Thread(target=read_stream, args=(client, thread_id, q, stop), daemon=True).start()
kind, detail = q.get(timeout=30)
if kind != 'open':
sys.exit(f'Event stream did not open: {detail}')
project = client.request('GET', '/rest/projects/personal')
client.request('POST', '/rest/instance-ai/threads',
{'threadId': thread_id, 'projectId': project['id'], 'source': 'assistant_page'})
if recorder.proc and not recorder.wait_for('ready', 60):
print(f'recording failed to start, see {out / "record.log"}', flush=True)
recorder.proc.kill()
recorder.proc = None
started_at = now_iso()
chat = client.request('POST', f'/rest/instance-ai/chat/{thread_id}',
{'message': prompt, 'timeZone': local_time_zone()})
print(f'thread {thread_id} run {chat.get("runId") if isinstance(chat, dict) else chat}', flush=True)
thread_url = f'{client.base_url}/instance-ai/{thread_id}'
# The UI shows the user message only after the chat request, so the recorder opens the thread now.
recorder.send(f'open {thread_url}')
print(f'watch: {thread_url}', flush=True)
open_url(thread_url)
workflow_ids, decisions, langsmith_run_ids = [], [], []
saw_finish, idle_since = False, None
deadline = time.time() + args.timeout_min * 60
while time.time() < deadline:
try:
kind, detail = q.get(timeout=5)
except queue.Empty:
kind, detail = 'tick', None
if kind == 'event':
_, event = detail
events_file.write(json.dumps(event) + '\n')
events_file.flush()
etype, payload = event.get('type'), event.get('payload') or {}
if etype == 'run-start':
if payload.get('langsmithRunId'):
langsmith_run_ids.append(payload['langsmithRunId'])
idle_since = None
elif etype == 'run-finish':
saw_finish = True
print(f'run-finish {payload.get("status", "")}', flush=True)
elif etype == 'tool-call':
print(f'tool {payload.get("toolName")}', flush=True)
elif etype in ('tool-result', 'confirmation-request'):
collect_workflow_ids(payload, workflow_ids)
if etype == 'confirmation-request':
body, label = decide(payload)
try:
client.request('POST', f'/rest/instance-ai/confirm/{payload["requestId"]}', body)
ok = True
except RuntimeError as e:
ok = False
label += f' (failed: {e})'
decisions.append({'at': now_iso(), 'toolName': payload.get('toolName'),
'message': payload.get('message'), 'decision': label, 'ok': ok})
print(f'confirm: {label} [{payload.get("toolName")}]', flush=True)
elif kind in ('closed', 'error'):
print(f'event stream {kind}: {detail or ""}', flush=True)
if saw_finish:
status = client.request('GET', f'/rest/instance-ai/threads/{thread_id}/status')
running = [t for t in status.get('backgroundTasks', []) if t.get('status') == 'running']
if status.get('hasActiveRun') or status.get('isSuspended') or running:
idle_since = None
elif idle_since is None:
idle_since = time.time()
elif time.time() - idle_since > 10:
break
else:
print('timeout: the thread did not go idle', flush=True)
finished_at = now_iso()
stop.set()
events_file.close()
recorder.stop()
trace = {}
for key, path in (
('rich', f'/threads/{thread_id}/messages?limit=200'),
('raw', f'/threads/{thread_id}/messages?limit=200&raw=true'),
('thread', f'/threads/{thread_id}'),
('debug', f'/debug/threads/{thread_id}/runs'),
):
try:
trace[key] = client.request('GET', '/rest/instance-ai' + path)
except RuntimeError as e:
trace[key] = None
print(f'export {key} failed: {str(e)[:200]}', flush=True)
(out / 'trace.json').write_text(json.dumps(trace, indent=1))
meta = {
'kind': 'instance-ai',
'threadId': thread_id,
'startedAt': started_at,
'finishedAt': finished_at,
'workflowIds': workflow_ids,
'workflowId': workflow_ids[-1] if workflow_ids else None,
'langsmithRunIds': langsmith_run_ids,
'decisions': decisions,
'threadUrl': thread_url,
'screen': 'screen.webm' if (out / 'screen.webm').exists() else None,
}
(out / 'run.json').write_text(json.dumps(meta, indent=2))
print(json.dumps({k: meta[k] for k in ('threadId', 'workflowId', 'startedAt', 'finishedAt')}, indent=2))
if __name__ == '__main__':
main()

View File

@ -0,0 +1,172 @@
#!/usr/bin/env python3
"""Run each workflow of a run directory once for each scenario, with pinned data.
run_scenarios.py <run-dir>
Reads <run-dir>/scenarios.json (optional, see SKILL.md) and <builder>/workflow.json.
sdk.mjs makes the pin data: triggers, service nodes and AI root nodes are pinned, and logic
nodes (Set, IF, Code, Aggregate, ...) run for real. The n8n MCP tool test_workflow runs the
workflow, and get_workflow_execution reads the result of each node.
Writes <builder>/exec.json. MCP runs a workflow only when its MCP access is on, so the script
turns it on through the REST API when necessary, records that in exec.json, and turns it off
again after the scenarios. With several triggers, each scenario starts from the trigger that
sdk.mjs picks (scenarios.json "triggerNode", or the trigger that reaches the "reach" types).
"""
import json
import shutil
import subprocess
import sys
import urllib.request
from pathlib import Path
from n8n_api import N8nClient, load_token
HERE = Path(__file__).resolve().parent
TIMEOUT_SECONDS = 90
ITEMS_PER_NODE = 3
class Mcp:
def __init__(self, base_url, token):
self.url, self.token = base_url + '/mcp-server/http', token
def call(self, tool, arguments):
body = json.dumps({'jsonrpc': '2.0', 'id': 1, 'method': 'tools/call',
'params': {'name': tool, 'arguments': arguments}}).encode()
req = urllib.request.Request(self.url, data=body, method='POST', headers={
'content-type': 'application/json',
'accept': 'application/json, text/event-stream',
'authorization': f'Bearer {self.token}',
})
with urllib.request.urlopen(req, timeout=TIMEOUT_SECONDS + 60) as res:
raw = res.read().decode()
lines = [line[5:].strip() for line in raw.splitlines() if line.startswith('data:')]
reply = json.loads(lines[-1] if lines else raw)
if 'error' in reply:
raise RuntimeError(f'{tool}: {reply["error"].get("message")}')
result = reply['result']
return result.get('structuredContent') or json.loads(result['content'][0]['text'])
def matches(node_type, pattern):
short = node_type.rsplit('.', 1)[-1]
return node_type == pattern or short.lower() == pattern.lower()
def read_run(run_data, types):
"""Per node: did it run, its error, and its first output items (all outputs)."""
nodes = {}
for name, runs in (run_data or {}).items():
run = runs[-1] if runs else {}
items = []
for output in ((run.get('data') or {}).get('main') or []):
items.extend(item.get('json') for item in (output or [])[:ITEMS_PER_NODE])
error = run.get('error')
nodes[name] = {
'type': types.get(name),
'status': run.get('executionStatus'),
'error': (error.get('message') or error.get('description')) if isinstance(error, dict) else error,
'items': items[:ITEMS_PER_NODE * 2],
}
return nodes
def expectations(expect, nodes):
"""Check reach and skip: node types that must run, and node types that must not run."""
results = []
ran = [n for n in nodes.values() if n['status'] == 'success' and not n['error']]
for pattern in expect.get('reach') or []:
results.append({'kind': 'reach', 'type': pattern, 'ok': any(matches(n['type'] or '', pattern) for n in ran)})
for pattern in expect.get('skip') or []:
results.append({'kind': 'skip', 'type': pattern,
'ok': not any(matches(n['type'] or '', pattern) for n in nodes.values())})
return results
def set_mcp_access(client, workflow_id, on):
"""Set MCP access for the workflow. Returns True when the script changed the setting."""
wf = client.workflow(workflow_id)
settings = wf.get('settings') or {}
if bool(settings.get('availableInMCP')) == on:
return False
client.request('PATCH', f'/rest/workflows/{workflow_id}',
{'versionId': wf['versionId'], 'settings': {**settings, 'availableInMCP': on}})
return True
def run_builder(sub, scenarios_file, client, mcp, node):
meta = json.loads((sub / 'run.json').read_text())
workflow_id = meta.get('workflowId')
wf_file = sub / 'workflow.json'
out = {'workflowId': workflow_id, 'scenarios': []}
if not workflow_id or not wf_file.exists():
out['skipped'] = 'No workflow.'
return out
workflow = json.loads(wf_file.read_text())
types = {n['name']: n['type'] for n in workflow.get('nodes', [])}
cmd = [node, str(HERE / 'sdk.mjs'), 'pin-plan', str(wf_file)] + ([str(scenarios_file)] if scenarios_file else [])
plan = json.loads(subprocess.run(cmd, cwd=HERE, capture_output=True, text=True, check=True).stdout)
out.update(pinned=plan['pinned'], executed=plan['executed'])
try:
out['mcpAccessTurnedOn'] = set_mcp_access(client, workflow_id, True)
except Exception as e:
out['skipped'] = f'Could not turn on MCP access: {e}'
return out
try:
run_plan(plan, workflow_id, types, mcp, sub, out)
finally:
if out['mcpAccessTurnedOn']:
try:
set_mcp_access(client, workflow_id, False)
except Exception as e:
out['mcpAccessRestoreError'] = str(e)
print(f'[scenarios] {sub.name}: could not turn MCP access off again: {e}', flush=True)
return out
def run_plan(plan, workflow_id, types, mcp, sub, out):
for scenario in plan['scenarios']:
entry = {'name': scenario['name'], 'expect': scenario['expect'], 'provenance': scenario['provenance']}
args = {'workflowId': workflow_id, 'pinData': scenario['pinData'], 'timeout': TIMEOUT_SECONDS}
if scenario.get('triggerNode'):
args['triggerNodeName'] = entry['triggerNode'] = scenario['triggerNode']
try:
result = mcp.call('test_workflow', args)
entry.update(status=result.get('status'), error=result.get('error'), executionId=result.get('executionId'))
if result.get('executionId'):
execution = mcp.call('get_workflow_execution', {'workflowId': workflow_id,
'executionId': result['executionId'], 'includeData': True})
run_data = ((execution.get('data') or {}).get('resultData') or {}).get('runData')
entry['nodes'] = read_run(run_data, types)
entry['checks'] = expectations(scenario['expect'], entry['nodes'])
except Exception as e:
entry.update(status='error', error=f'{type(e).__name__}: {e}')
print(f'[scenarios] {sub.name}: {scenario["name"]} -> {entry.get("status")}'
+ (f' ({entry["error"]})' if entry.get('error') else ''), flush=True)
out['scenarios'].append(entry)
def main():
run_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.').resolve()
node = shutil.which('node')
token = load_token()
if not node or not token:
sys.exit('Scenarios need node on PATH and a working MCP token.')
client = N8nClient().login()
mcp = Mcp(client.base_url, token)
scenarios_file = run_dir / 'scenarios.json'
scenarios_file = scenarios_file if scenarios_file.exists() else None
for meta_file in sorted(run_dir.glob('*/run.json')):
sub = meta_file.parent
try:
out = run_builder(sub, scenarios_file, client, mcp, node)
except Exception as e:
out = {'skipped': f'{type(e).__name__}: {e}', 'scenarios': []}
(sub / 'exec.json').write_text(json.dumps(out, indent=1))
if out.get('skipped'):
print(f'[scenarios] {sub.name}: skipped, {out["skipped"]}', flush=True)
if __name__ == '__main__':
main()

View File

@ -0,0 +1,313 @@
#!/usr/bin/env node
// Run each workflow of a run directory with the real node code, and record its service requests.
//
// node run_wire.mjs <run-dir>
//
// wire.mjs runs the workflow in this process. Only the trigger is pinned, and the AI root
// nodes too, except in the real-AI scenarios. Every other node runs, and the HTTP requests of
// the service nodes get answers in the documented shape of each API. A node that runs must be on
// the allow-list (runsInProcess), else the scenarios that run it are skipped. Three kinds of scenario:
// contract the scenarios of scenarios.json, with the AI output pinned as in run_scenarios.py
// edge the edge cases of the trigger (TRIGGER_KITS in sdk.mjs) and scenarios.json "edge"
// ai scenarios.json "questions": the AI nodes call their real model with each question
// Writes <builder>/wire.json.
import { existsSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
import { LOGIC_TYPES, TRIGGER_KITS, graph, isAiNode, isTrigger, pinPlan, typeMatches } from './sdk.mjs';
import { MODEL_CREDENTIALS, fillSetup, isWrite, loadEngine, modelKey, nodeCredentialTypes, runWorkflow, startNode } from './wire.mjs';
const ITEMS_PER_NODE = 3;
const BODY_CHARS = 1500;
// This process holds the settings and has no sandbox. The eval hook answers only the requests that
// go through the n8n request helpers. Database, SSH, FTP and SMTP clients, files and a shell bypass
// it, so only the node types below run here.
// These nodes run code from the builder. The pinned-data runs run them in the n8n task runner.
const CODE_NODE_TYPES = new Set([
'n8n-nodes-base.code',
'n8n-nodes-base.function',
'n8n-nodes-base.functionItem',
'n8n-nodes-base.aiTransform',
'@n8n/n8n-nodes-langchain.code',
'@n8n/n8n-nodes-langchain.toolCode',
]);
// Telegram is here because its credential has no authenticate block.
const IN_PROCESS_TYPES = new Set([
...[...LOGIC_TYPES].filter((type) => !CODE_NODE_TYPES.has(type)),
'n8n-nodes-base.httpRequest',
'n8n-nodes-base.wait',
'n8n-nodes-base.telegram',
]);
// Each model node has a key in MODEL_CREDENTIALS. The other nodes call only the model or the helpers.
const IN_PROCESS_AI_TYPES = new Set(
[
'agent', 'chainLlm', 'chainSummarization', 'informationExtractor', 'textClassifier', 'sentimentAnalysis',
'openAi', 'anthropic', 'googleGemini',
'lmChatOpenAi', 'lmChatAnthropic', 'lmChatGoogleGemini', 'lmChatMistralCloud', 'lmChatGroq',
'outputParserStructured', 'outputParserAutofixing', 'outputParserItemList',
'memoryBufferWindow', 'toolCalculator', 'toolThink', 'toolHttpRequest',
].map((type) => `@n8n/n8n-nodes-langchain.${type}`),
);
const readJson = (file) => (existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null);
/** Read a path such as `messages[0].from` from a value. */
export function readPath(value, path) {
const keys = Array.isArray(path) ? path : String(path).match(/[^.[\]]+/g) ?? [];
return keys.reduce((v, k) => (v == null ? undefined : v[/^\d+$/.test(k) ? Number(k) : k]), value);
}
const requestText = (r) => JSON.stringify({ url: r.url, qs: r.qs ?? undefined, body: r.body ?? undefined });
function nodeSummary(runData, types) {
const nodes = {};
for (const [name, runs] of Object.entries(runData ?? {})) {
const run = runs.at(-1) ?? {};
const outputs = (run.data?.main ?? []).map((o) => o ?? []);
const error = run.error;
nodes[name] = {
type: types[name],
status: run.executionStatus,
error: error ? error.message ?? error.description ?? String(error) : null,
runs: runs.length,
outputCounts: outputs.map((o) => o.length),
items: outputs.flat().slice(0, ITEMS_PER_NODE).map((i) => i.json),
};
}
return nodes;
}
const trim = (value) => {
const text = value === undefined ? undefined : JSON.stringify(value);
return text && text.length > BODY_CHARS ? `${text.slice(0, BODY_CHARS)}…` : value;
};
const trimRequest = (r) => ({ ...r, body: trim(r.body), response: trim(r.response) });
/**
* Resolve a marker of an expectation:
* @trigger.<path> a value of the first trigger item, such as @trigger.messages[0].from
* @response.<type>.<path> a value of the response to the last write request of a node type,
* such as @response.notion.id (the id of the page that Notion created)
* Other strings stay as they are.
*/
function resolveMarker(value, triggerItem, requests) {
if (typeof value !== 'string') return value;
if (value.startsWith('@trigger.')) return readPath(triggerItem, value.slice('@trigger.'.length));
const response = value.match(/^@response\.([^.]+)\.(.+)$/);
if (response) {
const last = requests.filter((r) => isWrite(r) && typeMatches(r.nodeType, response[1])).at(-1);
return last ? readPath(last.response, response[2]) : undefined;
}
return value;
}
/** Check the expectations of a scenario against the nodes that ran and the requests that they sent. */
export function evaluate(scenario, nodes, requests, trigger, kit) {
const checks = [];
const expect = scenario.expect ?? {};
const ran = Object.values(nodes).filter((n) => n.status === 'success' && !n.error);
for (const pattern of expect.reach ?? []) {
checks.push({ kind: 'reach', type: pattern, ok: ran.some((n) => typeMatches(n.type ?? '', pattern)) });
}
for (const pattern of expect.skip ?? []) {
checks.push({ kind: 'skip', type: pattern, ok: !Object.values(nodes).some((n) => typeMatches(n.type ?? '', pattern)) });
}
for (const want of expect.requests ?? []) {
// An array in contains means any one of its markers.
const contains = (want.contains ?? []).map((c) => (Array.isArray(c) ? c : [c]).map((m) => ({ marker: m, value: resolveMarker(m, trigger[0], requests) })));
const sent = requests.filter((r) => typeMatches(r.nodeType, want.type) && (!want.method || r.method === want.method.toUpperCase()));
const has = (r, any) => any.some((c) => c.value !== undefined && c.value !== null && requestText(r).includes(String(c.value)));
const matching = sent.filter((r) => contains.every((any) => has(r, any)));
const ok = want.count === undefined ? matching.length > 0 : matching.length === want.count;
const show = (c) => (c.marker === c.value ? `"${c.value}"` : `${c.marker} ("${c.value ?? 'not found'}")`);
const details = sent.length
? sent.map((r) => {
const missing = contains.filter((any) => !has(r, any)).map((any) => any.map(show).join(' or '));
if (missing.length) return `${r.node}: ${r.method} ${r.url} does not carry ${missing.join(', ')}`;
const body = r.body === undefined ? '' : JSON.stringify(r.body);
return `${r.node}: ${r.method} ${r.url}${body ? ` with \`${body.length > 300 ? `${body.slice(0, 300)}…` : body}\`` : ''}`;
})
: [`No ${want.method ?? ''} request from a ${want.type} node.`.replace(' ', ' ')];
checks.push({ kind: 'request', type: want.type, contains: contains.map((any) => any.map((c) => c.marker).join(' or ')), count: want.count, ok, details });
}
if (expect.noWrites) {
const writes = requests.filter(isWrite);
checks.push({ kind: 'noWrites', ok: writes.length === 0, details: writes.map((r) => `${r.node}: ${r.method} ${r.url}`) });
}
if (scenario.derive === 'each-item' && kit) {
const ids = trigger.map((item) => String(readPath(item, kit.identity)));
const carriers = requests.filter((r) => isWrite(r) && ids.some((id) => requestText(r).includes(id)));
const byNode = new Map();
for (const r of carriers) byNode.set(r.node, [...(byNode.get(r.node) ?? []), r]);
const details = [];
let ok = byNode.size > 0;
for (const [node, list] of byNode) {
const reached = ids.filter((id) => list.some((r) => requestText(r).includes(id)));
const missed = ids.filter((id) => !reached.includes(id));
if (missed.length) ok = false;
details.push(`${node}: ${list.length} request${list.length === 1 ? '' : 's'}, for ${reached.join(', ') || 'nobody'}${missed.length ? `, nothing for ${missed.join(', ')}` : ''}`);
}
if (!byNode.size) details.push(`No outgoing request names a sender (${ids.join(', ')}).`);
checks.push({ kind: 'eachItem', ids, ok, details });
}
return checks;
}
/** The scenarios of one workflow: contract, edge and AI, with their pin data and case rows. */
function buildScenarios(workflow, cfg) {
const g = graph(workflow);
const triggerNode = g.triggers.find((t) => isTrigger(t.type));
const kit = triggerNode ? TRIGGER_KITS[triggerNode.type] : undefined;
const contract = cfg.scenarios?.length ? cfg.scenarios : null;
const edges = [...(kit?.edges ?? []), ...(cfg.edge ?? [])];
// The triggerNode of pinPlan stays: a name from scenarios.json can be missing in this workflow.
const merge = (s, entry, kind) => ({ ...s, ...entry, triggerNode: s.triggerNode, kind });
const list = [
...(contract ? pinPlan(workflow, cfg, contract).scenarios.map((s, i) => merge(s, contract[i], 'contract')) : pinPlan(workflow, cfg).scenarios.map((s) => merge(s, {}, 'contract'))),
...pinPlan(workflow, cfg, edges).scenarios.map((s, i) => merge(s, edges[i], 'edge')),
...pinPlan(workflow, cfg, cfg.questions ?? []).scenarios.map((s, i) => merge(s, cfg.questions[i], 'ai')),
];
const keep = (name) => {
const node = g.byName.get(name);
return node && (isTrigger(node.type) || isAiNode(g, node));
};
return {
g,
scenarios: list.map((s) => {
const pinData = Object.fromEntries(
Object.entries(s.pinData).filter(([name]) => keep(name) && !(s.kind === 'ai' && isAiNode(g, g.byName.get(name)))),
);
return { ...s, pinData };
}),
};
}
/** Model sub-nodes and vendor model nodes, and whether a key exists for each, for the real-AI scenarios. */
function modelReadiness(g) {
const subnodes = g.edges.filter((e) => e.type === 'ai_languageModel').map((e) => g.byName.get(e.from));
const vendors = g.nodes.filter((n) => isAiNode(g, n) && !g.aiInputs(n.name, 'ai_languageModel').length);
return [...new Set([...subnodes, ...vendors])].map((node) => {
const types = nodeCredentialTypes(node).filter((t) => MODEL_CREDENTIALS[t]);
const type = types.find((t) => modelKey(t)) ?? types[0];
const p = node.parameters ?? {};
return { node: node.name, credentialType: type ?? null, model: p.modelName ?? p.model?.value ?? p.model ?? p.modelId?.value ?? null, ready: Boolean(type && modelKey(type)) };
});
}
const isHttpCredential = (name, { credentialType, parentTypes }) =>
credentialType(name).authenticate !== undefined || [name, ...parentTypes(name)].some((p) => p === 'oAuth1Api' || p === 'oAuth2Api');
/**
* True for a node on the allow-list: the types above, declarative nodes, and nodes whose credentials
* all authenticate HTTP requests. A tool variant (slackTool) counts as its base node.
*/
export function runsInProcess(node) {
const base = node.type.startsWith('n8n-nodes-base.') && node.type.endsWith('Tool') ? node.type.slice(0, -'Tool'.length) : node.type;
if (IN_PROCESS_TYPES.has(base) || IN_PROCESS_AI_TYPES.has(base)) return true;
if (!base.startsWith('n8n-nodes-base.')) return false;
const engine = loadEngine();
try {
const { description } = engine.nodeTypes.getByNameAndVersion(base, node.typeVersion);
if (description.requestDefaults !== undefined) return true;
const credentials = (description.credentials ?? []).map((c) => c.name);
return credentials.length > 0 && credentials.every((name) => isHttpCredential(name, engine));
} catch {
return false;
}
}
/**
* The names and types of the nodes that run from the start trigger and are not on the allow-list.
* Pinned nodes do not run. A disabled node passes its input on, so the walk goes through it.
*/
function outsideAllowList(workflow, g, start, pinData) {
const seen = new Set([start]);
const running = new Set();
// A Set visits the entries that the loop adds.
for (const name of seen) {
for (const targets of workflow.connections?.[name]?.main ?? []) {
for (const { node } of targets ?? []) {
if (seen.has(node)) continue;
seen.add(node);
if (g.byName.has(node) && !Object.hasOwn(pinData, node)) running.add(node);
}
}
}
for (const name of running) {
for (const e of g.edges) if (e.to === name && e.type.startsWith('ai_')) running.add(e.from);
}
return [...running].map((name) => g.byName.get(name)).filter((n) => !runsInProcess(n)).map((n) => `${n.name} (${n.type})`);
}
async function runBuilder(dir, cfg) {
const workflow = readJson(join(dir, 'workflow.json'));
const { g, scenarios } = buildScenarios(workflow, cfg);
const canvas = readJson(join(dir, 'canvas-issues.json'));
const { workflow: filledWorkflow, filled } = fillSetup(workflow, canvas);
const types = Object.fromEntries((workflow.nodes ?? []).map((n) => [n.name, n.type]));
const models = modelReadiness(g);
const out = { filled, models, scenarios: [] };
for (const scenario of scenarios) {
const entry = { name: scenario.name, kind: scenario.kind, why: scenario.why, expect: scenario.expect ?? {}, derive: scenario.derive, pinned: Object.keys(scenario.pinData) };
if (scenario.kind === 'ai' && !models.length) {
out.scenarios.push({ ...entry, status: 'skipped', error: 'The workflow has no model sub-node.' });
continue;
}
const start = startNode(workflow, scenario.pinData, scenario.triggerNode);
const outside = start ? outsideAllowList(workflow, g, start.name, scenario.pinData) : [];
if (outside.length) {
out.scenarios.push({ ...entry, status: 'skipped', error: `Nodes outside the in-process allow-list: ${outside.join(', ')}.` });
continue;
}
if (scenario.kind === 'ai' && models.some((m) => !m.ready)) {
const missing = models.filter((m) => !m.ready).map((m) => `${m.node} (${m.credentialType ?? 'unknown credential'}: set ${MODEL_CREDENTIALS[m.credentialType]?.keys[0] ?? 'a supported model'})`);
out.scenarios.push({ ...entry, status: 'skipped', error: `No model key for ${missing.join(', ')}.` });
continue;
}
const rows = (node) => Object.entries(scenario.byType ?? {}).find(([pattern]) => typeMatches(node.type, pattern))?.[1];
const trigger = start ? scenario.pinData[start.name].map((i) => i.json) : [];
const kit = start ? TRIGGER_KITS[start.type] : undefined;
const result = await runWorkflow(filledWorkflow, { pinData: scenario.pinData, triggerNode: scenario.triggerNode, rows, realModels: scenario.kind === 'ai' });
const nodes = nodeSummary(result.runData, types);
const errors = Object.entries(nodes).filter(([, n]) => n.error).map(([name, n]) => `${name}: ${n.error}`);
Object.assign(entry, {
status: result.status,
error: result.error,
ms: result.ms,
nodes,
requests: result.requests.map(trimRequest),
blocked: result.blocked,
checks: evaluate(scenario, nodes, result.requests, trigger, kit),
});
if (scenario.kind === 'ai') {
entry.aiOutputs = Object.fromEntries(Object.entries(nodes).filter(([name]) => isAiNode(g, g.byName.get(name))).map(([name, n]) => [name, n.items]));
}
console.log(`[wire] ${dir.split('/').pop()}: ${scenario.kind} "${scenario.name}" -> ${result.status}${errors.length ? ` (${errors[0]})` : result.error ? ` (${result.error})` : ''}`);
out.scenarios.push(entry);
}
return out;
}
if (process.argv[1] && resolve(process.argv[1]) === new URL(import.meta.url).pathname) {
const runDir = resolve(process.argv[2] ?? '.');
const loaded = loadEngine();
const cfg = readJson(join(runDir, 'scenarios.json')) ?? {};
for (const entry of readdirSync(runDir, { withFileTypes: true })) {
const dir = join(runDir, entry.name);
if (!entry.isDirectory() || !existsSync(join(dir, 'workflow.json'))) continue;
let out;
if (loaded.error) out = { skipped: loaded.error, scenarios: [] };
else {
try {
out = await runBuilder(dir, cfg);
} catch (error) {
out = { skipped: `${error?.name ?? 'Error'}: ${error?.message ?? error}`, scenarios: [] };
}
}
// A failed real model request can echo its URL, with the key, in the error message.
const keys = Object.keys(MODEL_CREDENTIALS).map(modelKey).filter(Boolean);
writeFileSync(join(dir, 'wire.json'), keys.reduce((text, key) => text.replaceAll(key, '[redacted]'), JSON.stringify(out, null, 1)));
if (out.skipped) console.log(`[wire] ${entry.name}: skipped, ${out.skipped}`);
}
// Model SDKs can keep sockets open.
process.exit(0);
}

View File

@ -0,0 +1,600 @@
#!/usr/bin/env node
// Shared workflow knowledge for derive_checks.mjs and run_scenarios.py: the workflow SDK of
// the repository, graph helpers, expression references, JSON Schema samples and pin data plans.
//
// node sdk.mjs pin-plan <workflow.json> [scenarios.json] print the pin data of each scenario
import { createRequire } from 'node:module';
import { existsSync, readFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
const here = dirname(fileURLToPath(import.meta.url));
export const repo = resolve(here, '../../../..');
const require = createRequire(import.meta.url);
const SDK_DIST = join(repo, 'packages/@n8n/workflow-sdk/dist/index.js');
// The same directories that the n8n node catalog gives to setSchemaBaseDirs().
const NODE_DEFINITION_DIRS = [
join(repo, 'packages/nodes-base/dist/node-definitions/'),
join(repo, 'packages/@n8n/nodes-langchain/dist/node-definitions/'),
];
let loaded;
/** The workflow SDK with the node parameter schemas set, or null when the package is not built. */
export function loadSdk() {
if (loaded !== undefined) return loaded;
if (!existsSync(SDK_DIST)) return (loaded = null);
const sdk = require(SDK_DIST);
const schemaDirs = NODE_DEFINITION_DIRS.filter((dir) => existsSync(dir));
// Without schema directories validateNodeConfig reports every node as valid.
sdk.setSchemaBaseDirs(schemaDirs);
return (loaded = { sdk, schemaDirs, missingDirs: NODE_DEFINITION_DIRS.filter((d) => !existsSync(d)) });
}
// --- Node kinds ----------------------------------------------------------------
const BASE = 'n8n-nodes-base.';
const LC = '@n8n/n8n-nodes-langchain.';
/** Nodes that run for real in a scenario. They need no credentials and call no service. */
export const LOGIC_TYPES = new Set(
[
'set', 'if', 'switch', 'filter', 'merge', 'aggregate', 'splitOut', 'code', 'function', 'functionItem',
'noOp', 'splitInBatches', 'itemLists', 'sort', 'limit', 'removeDuplicates', 'summarize', 'dateTime',
'renameKeys', 'compareDatasets', 'markdown', 'html', 'xml', 'crypto', 'executionData', 'stopAndError',
'respondToWebhook', 'convertToFile', 'extractFromFile', 'debugHelper',
].map((t) => BASE + t),
);
/** Nodes that end up with one item, so `.first()` on them reads all of their output. */
export const SINGLE_ITEM_TYPES = new Set(['aggregate', 'summarize'].map((t) => BASE + t));
const TRIGGER_TYPES = new Set(['webhook', 'cron', 'interval', 'emailReadImap', 'start'].map((t) => BASE + t));
export const isSticky = (type) => type === `${BASE}stickyNote`;
export const isTrigger = (type) => TRIGGER_TYPES.has(type) || type.toLowerCase().includes('trigger');
export const isManualTrigger = (type) => type === `${BASE}manualTrigger`;
export const shortType = (type) => type.slice(type.lastIndexOf('.') + 1);
/** True when type matches a full type name, or its part after the last dot (case-insensitive). */
export function typeMatches(type, pattern) {
return type === pattern || shortType(type).toLowerCase() === pattern.toLowerCase();
}
const AI_ROOT_PLACEHOLDER = {
[`${LC}agent`]: (v) => ({ output: v }),
[`${LC}openAiAssistant`]: (v) => ({ output: v }),
[`${LC}informationExtractor`]: (v) => ({ output: v }),
[`${LC}chainLlm`]: (v) => ({ text: v }),
[`${LC}chainRetrievalQa`]: (v) => ({ response: v }),
[`${LC}chainSummarization`]: (v) => ({ output: { output_text: v } }),
[`${LC}sentimentAnalysis`]: (v) => ({ sentimentAnalysis: { category: v } }),
[`${LC}googleGemini`]: (v) => ({ content: { parts: [{ text: v }], role: 'model' }, finishReason: 'STOP', index: 0 }),
};
/** Vendor nodes that call a model themselves, without a model sub-node. */
const AI_VENDOR_TYPES = new Set(['googleGemini', 'openAi', 'anthropic'].map((t) => LC + t));
// --- Graph ---------------------------------------------------------------------
/** Nodes, edges and lookups of a workflow. Sticky notes and disabled nodes are left out. */
export function graph(workflow) {
const nodes = (workflow.nodes ?? []).filter((n) => !isSticky(n.type) && !n.disabled);
const byName = new Map(nodes.map((n) => [n.name, n]));
const edges = [];
for (const [from, outputs] of Object.entries(workflow.connections ?? {})) {
for (const [type, branches] of Object.entries(outputs ?? {})) {
(branches ?? []).forEach((targets, output) =>
(targets ?? []).forEach((t) => {
if (byName.has(from) && byName.has(t.node))
edges.push({ from, to: t.node, type, output, input: t.index ?? 0 });
}),
);
}
}
const subnodes = new Set(edges.filter((e) => e.type.startsWith('ai_')).map((e) => e.from));
const mainParents = (name) => [...new Set(edges.filter((e) => e.type === 'main' && e.to === name).map((e) => e.from))];
const mainChildren = (name) => edges.filter((e) => e.type === 'main' && e.from === name);
const aiInputs = (name, type) => edges.filter((e) => e.to === name && e.type === type).map((e) => byName.get(e.from));
const triggers = nodes.filter((n) => isTrigger(n.type) && !subnodes.has(n.name));
return { nodes, byName, edges, subnodes, mainParents, mainChildren, aiInputs, triggers };
}
/** Names of the nodes that a trigger reaches. AI sub-nodes count when they serve a reached node. */
export function reachable(g) {
const seen = new Set(g.triggers.map((t) => t.name));
const queue = [...seen];
while (queue.length) {
const name = queue.shift();
for (const e of g.edges) {
if (e.type === 'main' && e.from === name && !seen.has(e.to)) {
seen.add(e.to);
queue.push(e.to);
}
}
}
// A sub-node can serve another sub-node, for example a model inside an auto-fixing parser.
let grew = true;
while (grew) {
grew = false;
for (const e of g.edges) {
if (e.type.startsWith('ai_') && seen.has(e.to) && !seen.has(e.from)) {
seen.add(e.from);
grew = true;
}
}
}
return seen;
}
/** Names of the nodes downstream of name, through main connections. */
export function downstream(g, name) {
const seen = new Set();
const queue = [name];
while (queue.length) {
for (const e of g.mainChildren(queue.shift())) {
if (!seen.has(e.to)) {
seen.add(e.to);
queue.push(e.to);
}
}
}
return seen;
}
// --- Parameters and expressions --------------------------------------------------
/** Every string in a parameter tree, with its dotted path. */
export function strings(value, path = '') {
if (typeof value === 'string') return [{ path, value }];
if (Array.isArray(value)) return value.flatMap((v, i) => strings(v, `${path}[${i}]`));
if (value && typeof value === 'object')
return Object.entries(value).flatMap(([k, v]) => strings(v, path ? `${path}.${k}` : k));
return [];
}
const CODE_PARAMS = new Set(['jsCode', 'pythonCode', 'functionCode', 'code']);
/** The expression text of a parameter value, or null. Code fields count, because they use the same references. */
export function expressionText(path, value) {
if (value.startsWith('=')) return value.slice(1);
const key = path.split('.').pop().replace(/\[\d+\]$/, '');
return CODE_PARAMS.has(key) ? value : null;
}
const IDENT = /^[A-Za-z_$][\w$]*/;
/** Read a property path such as `.a?.b['c'][0]` from text at index i. Returns { path, end, call }. */
function readPath(text, i) {
const path = [];
let dynamic = false;
for (;;) {
if (text.startsWith('?.', i)) i += 2;
else if (text[i] === '.') i += 1;
else if (text[i] !== '[') break;
if (text[i] === '[') {
const close = text.indexOf(']', i);
if (close < 0) break;
const inner = text.slice(i + 1, close).trim();
if (/^\d+$/.test(inner)) path.push(Number(inner));
else if (/^(['"`])(.*)\1$/.test(inner)) path.push(inner.slice(1, -1));
else {
dynamic = true;
i = close + 1;
break;
}
i = close + 1;
continue;
}
const match = text.slice(i).match(IDENT);
if (!match) break;
path.push(match[0]);
i += match[0].length;
}
const call = /^\s*\(/.test(text.slice(i));
// A trailing method call, such as `.toUpperCase()`, is not a field.
if (call && path.length) path.pop();
return { path, end: i, call, dynamic };
}
const ACCESSORS = [
{ re: /^\.item\b/, kind: 'item' },
{ re: /^\.first\(\s*\)/, kind: 'first' },
{ re: /^\.last\(\s*\)/, kind: 'last' },
{ re: /^\.all\(\s*\)\[\s*\d+\s*\]/, kind: 'all-index' },
{ re: /^\.all\(\s*\)/, kind: 'all' },
{ re: /^\.itemMatching\([^)]*\)/, kind: 'item' },
{ re: /^\.pairedItem\([^)]*\)/, kind: 'item' },
];
/**
* References in expression text: { node, kind, accessor, path, raw }.
* node is null for `$json` and `$input`, which read the main input of the node.
* path is null when the reference does not read item JSON, for example `$('A').params`.
*/
export function references(text) {
const refs = [];
const named = /\$\(\s*(['"`])((?:\\.|(?!\1).)+)\1\s*\)|\$node\[\s*(['"`])((?:\\.|(?!\3).)+)\3\s*\]|\$items\(\s*(['"`])((?:\\.|(?!\5).)+)\5/g;
for (const m of text.matchAll(named)) {
const node = (m[2] ?? m[4] ?? m[6]).replace(/\\(.)/g, '$1');
let i = m.index + m[0].length;
let accessor = m[4] !== undefined ? 'node' : m[6] !== undefined ? 'items' : null;
if (accessor === null) {
const rest = text.slice(i);
const hit = ACCESSORS.find((a) => a.re.test(rest));
if (hit) {
accessor = hit.kind;
i += rest.match(hit.re)[0].length;
}
}
let path = null;
if (text.startsWith('.json', i) && accessor !== 'all' && accessor !== 'items') {
const read = readPath(text, i + '.json'.length);
path = read.dynamic ? null : read.path;
}
refs.push({ node, kind: 'named', accessor, path, raw: text.slice(m.index, i + 40).split(/[}\n]/)[0] });
}
for (const m of text.matchAll(/\$json\b/g)) {
const read = readPath(text, m.index + '$json'.length);
refs.push({ node: null, kind: 'json', accessor: 'item', path: read.dynamic ? null : read.path,
raw: text.slice(m.index, read.end) });
}
for (const m of text.matchAll(/\$input\.(item|first\(\s*\)|last\(\s*\)|all\(\s*\)\[\s*\d+\s*\])\.json\b/g)) {
const read = readPath(text, m.index + m[0].length);
const accessor = m[1].startsWith('first') ? 'first' : m[1].startsWith('last') ? 'last' : m[1] === 'item' ? 'item' : 'all-index';
refs.push({ node: null, kind: 'input', accessor, path: read.dynamic ? null : read.path,
raw: text.slice(m.index, read.end) });
}
return refs;
}
/** Every reference in a workflow: { node (the node that holds it), param, ref, code }. */
export function workflowReferences(g) {
const out = [];
for (const node of g.nodes) {
for (const { path, value } of strings(node.parameters ?? {})) {
const text = expressionText(path, value);
if (text === null) continue;
const code = !value.startsWith('=');
for (const ref of references(text)) out.push({ node: node.name, param: path, ref, code });
}
}
return out;
}
/**
* Look up a path in a value. Returns 'found', 'missing' or 'unknown'.
* 'unknown' means that a null or an opaque value hides the rest of the path. With sample, the
* value comes from a schema, which shows the shape of one array item and not the count, so an
* index past the end of an array is 'unknown'. In real output, that index is 'missing'.
*/
export function lookup(value, path, { sample = false } = {}) {
let current = value;
for (const key of path) {
if (current === null || current === undefined) return current === null ? 'unknown' : 'missing';
if (Array.isArray(current)) {
if (typeof key === 'number') {
if (key >= current.length) return sample ? 'unknown' : 'missing';
current = current[key];
} else if (key === 'length') return 'found';
else return 'missing';
continue;
}
if (typeof current === 'string') return key === 'length' ? 'found' : 'missing';
if (typeof current !== 'object') return 'missing';
if (!(key in current)) return 'missing';
current = current[key];
}
return current === undefined ? 'missing' : 'found';
}
export const showPath = (path) =>
path.map((k, i) => (typeof k === 'number' ? `[${k}]` : /^[A-Za-z_$][\w$]*$/.test(k) ? (i ? `.${k}` : k) : `['${k}']`)).join('');
// --- JSON Schema samples ---------------------------------------------------------
/** A deterministic sample value for a JSON Schema. key names the field, for readable strings. */
export function sample(schema, key = 'value', depth = 0) {
if (!schema || typeof schema !== 'object') return null;
if (schema.example !== undefined) return schema.example;
if (schema.const !== undefined) return schema.const;
if (Array.isArray(schema.enum) && schema.enum.length) return schema.enum[0];
if (schema.default !== undefined) return schema.default;
const variant = schema.anyOf ?? schema.oneOf;
if (Array.isArray(variant) && variant.length) return sample(variant.find((v) => v.type !== 'null') ?? variant[0], key, depth);
let type = Array.isArray(schema.type) ? schema.type.find((t) => t !== 'null') : schema.type;
if (!type) type = schema.properties ? 'object' : schema.items ? 'array' : 'string';
switch (type) {
case 'object': {
if (depth > 6) return {};
return Object.fromEntries(
Object.entries(schema.properties ?? {}).map(([k, v]) => [k, sample(v, k, depth + 1)]),
);
}
case 'array':
return schema.items && depth <= 6 ? [sample(schema.items, key, depth + 1)] : [];
case 'integer':
case 'number':
return 1;
case 'boolean':
return true;
case 'null':
return null;
default: {
const k = String(key);
if (schema.format === 'date-time' || /(^|_)(date|time|at)$|At$/.test(k)) return '2026-01-01T09:00:00.000Z';
if (schema.format === 'email' || /email/i.test(k)) return 'ada@example.com';
if (schema.format === 'uri' || /url|link|href/i.test(k)) return `https://example.com/${k}`;
if (/(^|_)id$|Id$/.test(k)) return `${k}-1`;
return `sample ${k}`;
}
}
}
/** Set every boolean in a value to flag. */
function setBooleans(value, flag) {
if (Array.isArray(value)) return value.map((v) => setBooleans(v, flag));
if (value && typeof value === 'object')
return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, setBooleans(v, flag)]));
return typeof value === 'boolean' ? flag : value;
}
function deepMerge(base, extra) {
if (!base || typeof base !== 'object' || Array.isArray(base) || !extra || typeof extra !== 'object') return extra;
const out = { ...base };
for (const [k, v] of Object.entries(extra)) out[k] = deepMerge(base[k], v);
return out;
}
// --- Pin data ------------------------------------------------------------------
/**
* Trigger output for a scenario, in the shape that the trigger code emits.
* WhatsApp: WhatsAppTrigger.node.ts emits `{ ...change.value, field }` for each change,
* and change.value is the Cloud API webhook payload of a text message.
*/
export const TRIGGER_FIXTURES = {
[`${BASE}whatsAppTrigger`]: [{
messaging_product: 'whatsapp',
metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
contacts: [{ profile: { name: 'Ada Lovelace' }, wa_id: '4915112345678' }],
messages: [{ from: '4915112345678', id: 'wamid.HBgLNDkxNTExMjM0NTY3OBUCABIYFjNFQjA', timestamp: '1767258000',
text: { body: 'What are your opening hours?' }, type: 'text' }],
field: 'messages',
}],
[`${BASE}webhook`]: [{ headers: { 'content-type': 'application/json' }, params: {}, query: {},
body: { message: 'Hello' }, webhookUrl: 'http://localhost:5678/webhook/test', executionMode: 'test' }],
[`${LC}chatTrigger`]: [{ sessionId: 'session-1', action: 'sendMessage', chatInput: 'Hello' }],
[`${BASE}manualTrigger`]: [{}],
[`${BASE}scheduleTrigger`]: [{ timestamp: '2026-01-01T09:00:00.000+00:00', 'Readable date': 'January 1st 2026, 9:00:00 am',
'Day of week': 'Thursday', Year: '2026', Month: 'January', 'Day of month': '01', Hour: '09', Minute: '00',
Second: '00', Timezone: 'UTC' }],
[`${BASE}telegramTrigger`]: [{ update_id: 1, message: { message_id: 1, date: 1767258000, text: 'Hello',
chat: { id: 1001, type: 'private', first_name: 'Ada' }, from: { id: 1001, is_bot: false, first_name: 'Ada' } } }],
[`${BASE}formTrigger`]: [{ submittedAt: '2026-01-01T09:00:00.000Z', formMode: 'test' }],
};
const whatsAppMessage = (from, name, body, id) => ({
messaging_product: 'whatsapp',
metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
contacts: [{ profile: { name }, wa_id: from }],
messages: [{ from, id, timestamp: '1767258000', text: { body }, type: 'text' }],
field: 'messages',
});
/**
* Per trigger type: identity is the path of the value that names the sender in one item,
* withMessage puts a scenario text into the fixture, and edges are the edge cases that every
* workflow with this trigger must survive. derive 'each-item' means that each trigger item
* must get its own outgoing request that names its sender.
*/
export const TRIGGER_KITS = {
[`${BASE}whatsAppTrigger`]: {
identity: ['messages', 0, 'from'],
withMessage: (items, text) => items.map((i) => ({ ...i, messages: (i.messages ?? []).map((m) => ({ ...m, text: { body: text } })) })),
edges: [
{
name: 'Two messages arrive in one webhook call',
why: 'WhatsAppTrigger.node.ts emits one item for each change, and Meta can batch several changes in one call',
derive: 'each-item',
ai: { booleans: true },
trigger: [
whatsAppMessage('4915112345678', 'Ada Lovelace', 'What are your opening hours?', 'wamid.A1'),
whatsAppMessage('4917699887766', 'Alan Turing', 'Do you ship abroad?', 'wamid.B2'),
],
},
{
name: 'A delivery status update arrives, not a customer message',
why: 'the messages update also delivers sent, delivered and read statuses, with no messages array',
expect: { noWrites: true },
trigger: [{
messaging_product: 'whatsapp',
metadata: { display_phone_number: '15550001111', phone_number_id: '106540352242922' },
statuses: [{ id: 'wamid.HBgLNDkxNTExMjM0NTY3OBUCABEYEjQ', status: 'delivered', timestamp: '1767258060', recipient_id: '4915112345678' }],
field: 'messages',
}],
},
],
},
};
/** The trigger items of a scenario: its own trigger, the case trigger, or the fixture with the scenario message. */
export function triggerItems(node, scenario = {}, scenarios = {}) {
const explicit = scenario.trigger ?? scenarios.trigger;
if (explicit) return { items: Array.isArray(explicit) ? explicit : [explicit], source: 'case' };
const fixture = TRIGGER_FIXTURES[node.type];
if (!fixture) return null;
const kit = TRIGGER_KITS[node.type];
if (scenario.message !== undefined && kit) return { items: kit.withMessage(fixture, scenario.message), source: 'case' };
return { items: fixture, source: 'fixture' };
}
export const isAiRootNode = (g, node) => node.type.startsWith(LC) && !g.subnodes.has(node.name) && g.aiInputs(node.name, 'ai_languageModel').length > 0;
/** A node whose output comes from a model: an AI root node or a vendor node such as Google Gemini. */
export const isAiNode = (g, node) => Boolean(node) && (isAiRootNode(g, node) || (AI_VENDOR_TYPES.has(node.type) && !g.subnodes.has(node.name)));
function parserSchema(parser) {
const p = parser?.parameters ?? {};
const parse = (text) => {
try {
return typeof text === 'string' ? JSON.parse(text) : text && typeof text === 'object' ? text : null;
} catch {
return null;
}
};
const schemaType = p.schemaType ?? 'fromJson';
if (schemaType === 'manual' && parse(p.inputSchema)) return { kind: 'schema', value: parse(p.inputSchema) };
if (schemaType !== 'manual' && parse(p.jsonSchemaExample)) return { kind: 'example', value: parse(p.jsonSchemaExample) };
if (parse(p.jsonSchema)) return { kind: 'schema', value: parse(p.jsonSchema) };
return null;
}
/** JSON objects that a prompt shows as examples, in order. Objects with a boolean come first. */
function promptExamples(node) {
const found = [];
for (const { value } of strings(node.parameters ?? {})) {
for (let start = value.indexOf('{'); start >= 0; start = value.indexOf('{', start + 1)) {
let depth = 0;
for (let i = start; i < value.length; i++) {
if (value[i] === '{') depth++;
else if (value[i] === '}' && --depth === 0) {
try {
const parsed = JSON.parse(value.slice(start, i + 1));
if (parsed && typeof parsed === 'object' && !Array.isArray(parsed) && Object.keys(parsed).length)
found.push(parsed);
} catch {}
break;
}
}
}
}
return found.sort((a, b) => Number(hasBoolean(b)) - Number(hasBoolean(a)));
}
const hasBoolean = (value) => Object.values(value).some((v) => typeof v === 'boolean');
/** Apply the ai settings of a scenario to a model result: every boolean, then explicit fields. */
function applyAi(value, ai) {
let out = typeof ai.booleans === 'boolean' ? setBooleans(value, ai.booleans) : value;
if (ai.set && out && typeof out === 'object') out = deepMerge(out, ai.set);
return out;
}
/**
* The output of an AI root node for a scenario. Returns { json, provenance }.
* With a structured output parser, the result follows the parser schema. Without one, the
* model returns text: ai.text when the scenario sets it, else the first JSON example in the
* prompt of the node (the contract that the builder declared), else a plain sentence.
*/
function aiRootOutput(g, node, ai = {}) {
const parserNode = g.aiInputs(node.name, 'ai_outputParser')[0];
const parser = parserNode ? parserSchema(parserNode) : null;
if (parser) {
const parsed = parser.kind === 'schema' ? sample(parser.value, 'output') : parser.value;
return { json: { output: applyAi(parsed, ai) }, provenance: 'output-parser' };
}
const wrap = AI_ROOT_PLACEHOLDER[node.type] ?? ((v) => ({ output: v }));
if (ai.text !== undefined) return { json: wrap(ai.text), provenance: 'case' };
const example = promptExamples(node)[0];
if (example) return { json: wrap(JSON.stringify(applyAi(example, ai))), provenance: 'prompt-example' };
return { json: wrap('Sample answer from the model.'), provenance: AI_ROOT_PLACEHOLDER[node.type] ? 'ai-root' : 'placeholder' };
}
/**
* The pin data of each scenario. Logic nodes run for real. Triggers, service nodes and AI root
* nodes get pin data, with the provenance of each node, so a check knows how far to trust it:
* fixture, case the shape of real data (trigger code, or the case file)
* node-schema the __schema__ file of the node
* output-parser the schema of the attached structured output parser
* prompt-example a JSON example in the prompt of an AI root node without a parser
* ai-root the known output envelope of the AI root node
* placeholder, empty nothing describes the output, so the shape is a guess or one empty item
*/
export function pinPlan(workflow, scenarios = {}, only = null) {
const g = graph(workflow);
const sdk = loadSdk()?.sdk;
const list = only ?? (scenarios.scenarios?.length ? scenarios.scenarios : defaultScenarios(g));
const plan = { pinned: {}, executed: [], subnodes: [...g.subnodes], scenarios: [] };
for (const node of g.nodes) {
if (!g.subnodes.has(node.name) && LOGIC_TYPES.has(node.type)) plan.executed.push(node.name);
}
for (const scenario of list) {
const pinData = {};
const provenance = {};
const byType = mergeByType(scenarios.pinByType, scenario.pinByType);
for (const node of g.nodes) {
if (g.subnodes.has(node.name) || LOGIC_TYPES.has(node.type)) continue;
let items;
let source;
const caseItems = Object.entries(byType).find(([pattern]) => typeMatches(node.type, pattern))?.[1];
const explicit = scenario.trigger ?? scenarios.trigger;
const trigger = isTrigger(node.type) && (explicit || !caseItems) ? triggerItems(node, scenario, scenarios) : null;
if (trigger) {
items = trigger.items;
source = trigger.source;
} else if (caseItems) {
items = caseItems;
source = 'case';
} else if (isAiRootNode(g, node) || (isAiNode(g, node) && AI_ROOT_PLACEHOLDER[node.type])) {
const out = aiRootOutput(g, node, scenario.ai);
items = [out.json];
source = out.provenance;
} else {
const schema = sdk?.discoverOutputSchemaForNode(node.type, node.typeVersion, node.parameters ?? {});
items = [schema ? sample(schema, 'item') : {}];
source = schema ? 'node-schema' : 'empty';
}
pinData[node.name] = (Array.isArray(items) ? items : [items]).map((json) => ({ json }));
provenance[node.name] = source;
}
plan.scenarios.push({ name: scenario.name, expect: scenario.expect ?? {}, pinData, provenance, byType, triggerNode: scenarioTrigger(g, scenario) });
Object.assign(plan.pinned, provenance);
}
return plan;
}
/**
* The trigger that a scenario starts from, or null when the workflow has one trigger. Without
* scenario.triggerNode, it is the trigger whose downstream nodes match the most expect.reach types.
*/
function scenarioTrigger(g, scenario) {
if (g.triggers.length < 2) return null;
if (scenario.triggerNode && g.triggers.some((t) => t.name === scenario.triggerNode)) return scenario.triggerNode;
const reach = scenario.expect?.reach ?? [];
const score = (trigger) => {
const types = [...downstream(g, trigger.name)].map((name) => g.byName.get(name)?.type ?? '');
return reach.filter((pattern) => types.some((type) => typeMatches(type, pattern))).length;
};
return g.triggers.reduce((best, t) => (score(t) > score(best) ? t : best)).name;
}
/** Case pin data by node type. A scenario entry replaces a case entry for the same short type. */
function mergeByType(base = {}, extra = {}) {
const out = { ...base };
for (const [pattern, items] of Object.entries(extra ?? {})) {
for (const key of Object.keys(out)) if (shortType(key).toLowerCase() === shortType(pattern).toLowerCase()) delete out[key];
out[pattern] = items;
}
return out;
}
/** Without a case file: one scenario for each value of the booleans that an AI node returns. */
function defaultScenarios(g) {
const hasAi = g.nodes.some((n) => g.aiInputs(n.name, 'ai_languageModel').length && !g.subnodes.has(n.name));
return hasAi
? [{ name: 'AI flags true', ai: { booleans: true } }, { name: 'AI flags false', ai: { booleans: false } }]
: [{ name: 'Default run' }];
}
// --- CLI -----------------------------------------------------------------------
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
const [command, workflowFile, scenariosFile] = process.argv.slice(2);
if (command !== 'pin-plan' || !workflowFile) {
console.error('usage: node sdk.mjs pin-plan <workflow.json> [scenarios.json]');
process.exit(2);
}
const workflow = JSON.parse(readFileSync(workflowFile, 'utf8'));
const scenarios = scenariosFile && existsSync(scenariosFile) ? JSON.parse(readFileSync(scenariosFile, 'utf8')) : {};
process.stdout.write(JSON.stringify(pinPlan(workflow, scenarios)));
}

View File

@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""Print the evidence of each run for the judge step.
summarize_runs.py <run-dir> print the summary with the builder names
summarize_runs.py <run-dir> --blind write judge/summary-blind.txt and judge/blind-map.json
The summary has the expected outcome criteria with their ids, and for each builder: the final
agent message, every node with all its parameters and settings, the connections, and the facts
from auto-checks.json. Nothing is truncated, so the judge can cite exact values.
In blind mode the builders are called A, B and C in a random order, and builder and model names
are removed from the final messages. blind-map.json maps each letter back to its builder.
"""
import argparse
import json
import random
import re
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from build_report import load_case, parse_criteria # noqa: E402
NODE_SETTINGS = ('disabled', 'onError', 'retryOnFail', 'maxTries', 'waitBetweenTries', 'executeOnce',
'alwaysOutputData', 'notesInFlow', 'notes')
BUILDER_NAMES = re.compile(
r'\b(Instance AI|n8n Assistant|Claude Code|Claude|Anthropic|Codex|OpenAI|ChatGPT|GPT-[\w.-]+|Sonnet|Opus|Haiku)\b',
re.IGNORECASE)
MARK = {True: '✓', False: '✗', None: '?'}
REPO = Path(__file__).resolve().parents[4]
LLM_CHECKS_DIR = REPO / 'packages/@n8n/instance-ai/evaluations/binaryChecks/checks'
# The LLM binary checks of the eval suite that the judges answer as quality checks.
QUALITY_CHECKS = ('valid-data-flow', 'correct-node-operations', 'handles-multiple-items', 'descriptive-node-names',
'response-describes-changes-accurately')
def quality_checks():
"""[(name, description, rules)] read from the check sources, so the rules stay those of the eval suite."""
out = []
for file in QUALITY_CHECKS:
path = LLM_CHECKS_DIR / f'{file}.ts'
if not path.exists():
continue
text = path.read_text()
name = re.search(r"name:\s*'([^']+)'", text)
desc = re.search(r"description:\s*(['\"])(.*?)\1,", text, re.S)
rules = re.search(r'systemPrompt:\s*`((?:\\`|[^`])*)`', text)
if name and rules:
out.append((name.group(1), desc.group(2) if desc else '', rules.group(1).replace('\\`', '`').strip()))
return out
def quality_text():
checks = quality_checks()
if not checks:
return ''
lines = ['QUALITY CHECKS (from the Instance AI eval suite). For each builder, answer each check once in "quality",',
'with the check name as the id. The final message is the narration of the agent. Each build starts',
'from an empty workflow, so the START workflow is empty.']
for name, desc, rules in checks:
lines += [f'--- [{name}] {desc}', rules]
return '\n'.join(lines) + '\n'
def session_final_message(sub, meta):
"""Final agent message from the session file of an interactive CLI run."""
if not meta.get('sessionFile'):
return None
text = None
for line in (sub / meta['sessionFile']).read_text().splitlines():
row = json.loads(line)
payload = row.get('payload') or {}
if row.get('type') == 'event_msg' and payload.get('type') == 'task_complete':
text = payload.get('last_agent_message') or text
if row.get('type') == 'assistant':
blocks = (row.get('message') or {}).get('content') or []
texts = [b['text'] for b in blocks if isinstance(b, dict) and b.get('type') == 'text' and b.get('text', '').strip()]
text = '\n'.join(texts) or text
return text
def final_message(sub, meta):
kind = meta.get('kind')
if kind == 'instance-ai':
trace = json.loads((sub / 'trace.json').read_text())
rich = trace['rich'].get('data', trace['rich'])
assistant = [m for m in rich['messages'] if m.get('role') == 'assistant']
return (assistant[-1].get('agentTree') or {}).get('textContent') or assistant[-1].get('content') if assistant else None
if not (sub / 'stream.jsonl').exists():
return session_final_message(sub, meta)
text = None
for line in (sub / 'stream.jsonl').read_text().splitlines():
event = json.loads(line)
if event.get('type') == 'result':
text = event.get('result')
item = event.get('item') or {}
if event.get('type') == 'item.completed' and item.get('type') == 'agent_message':
text = item.get('text')
return text
def connections(workflow):
lines = []
for source, outputs in (workflow.get('connections') or {}).items():
for kind, branches in outputs.items():
for index, targets in enumerate(branches or []):
for target in targets or []:
branch = f'[{index}]' if len(branches) > 1 else ''
# Merge and other multi-input nodes need the input index to show which side an edge feeds.
target_index = target.get('index') or 0
inlet = f'[in {target_index}]' if target_index else ''
via = '' if kind == 'main' else f' ({kind})'
lines.append(f' {source}{branch} -> {target["node"]}{inlet}{via}')
return lines
def criteria_text(expected):
"""The expected outcome with the id of each criterion, as the judge must cite it."""
groups = parse_criteria(expected or '')
if not groups:
return (expected or '(none)').strip()
lines = []
for heading, items in groups:
if heading:
lines.append(f'{heading}:')
lines.extend(f' [{num}] {text}' for num, text in items)
return '\n'.join(lines)
def auto_check_lines(sub):
path = sub / 'auto-checks.json'
if not path.exists():
return [' (none: derive_checks.mjs did not run)']
lines = []
for group in json.loads(path.read_text()).get('groups', []):
score = group.get('score')
suffix = f' · score {score["passed"]}/{score["total"]}' if score else ''
lines.append(f' {group["title"]}{suffix} — {group.get("note", "")}')
for c in group.get('checks', []):
mark = 'i' if c.get('level') == 'info' else MARK.get(c.get('ok'), '?')
lines.append(f' [{mark}] {c["id"]} {c["text"]}')
lines.extend(f' · {d}' for d in c.get('details') or [])
return lines
def run_section(sub, meta, label, blind=False):
out = [f'===== {label} workflow={meta.get("workflowId")} =====']
try:
message = (final_message(sub, meta) or '(none)').strip()
out.append('FINAL MESSAGE:\n' + (scrub(message, meta) if blind else message))
except Exception as e:
out.append(f'FINAL MESSAGE: unreadable ({e})')
wf_file = sub / 'workflow.json'
if not wf_file.exists():
out.append('WORKFLOW: none\n')
return '\n'.join(out)
workflow = json.loads(wf_file.read_text())
out.append(f'WORKFLOW: {workflow.get("name")} nodes={len(workflow.get("nodes", []))}')
out.append(f'SETTINGS: {json.dumps(workflow.get("settings") or {}, ensure_ascii=False)}')
for node in workflow.get('nodes', []):
creds = ', '.join((node.get('credentials') or {}).keys())
out.append(f'- {node["name"]} | {node["type"]} v{node.get("typeVersion")}' + (f' | creds: {creds}' if creds else ''))
settings = {k: node[k] for k in NODE_SETTINGS if k in node}
if settings:
out.append(f' (settings) {json.dumps(settings, ensure_ascii=False)}')
for key, value in (node.get('parameters') or {}).items():
text = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False)
out.append(f' {key}: {text}')
out.append('CONNECTIONS:')
out.append('\n'.join(connections(workflow)) or ' (none)')
out.append('AUTO-CHECK FACTS (deterministic, from the node definitions, the n8n editor, pinned-data runs, '
'in-process runs with recorded API requests, the eval binary checks and the final-message claims):')
out.extend(auto_check_lines(sub))
return '\n'.join(out) + '\n'
def scrub(text, meta):
"""Remove builder and model names from a final message. The workflow itself stays as built."""
for name in filter(None, [meta.get('model'), meta.get('kind')]):
text = re.sub(re.escape(name), 'the builder', text, flags=re.IGNORECASE)
return BUILDER_NAMES.sub('the builder', text)
def main():
ap = argparse.ArgumentParser(description='Print the evidence of each run for the judge step.')
ap.add_argument('run_dir', nargs='?', default='.')
ap.add_argument('--blind', action='store_true', help='Write a summary with the builders called A, B and C.')
ap.add_argument('--seed', type=int, help='Seed of the blind order, for a reproducible test.')
args = ap.parse_args()
run_dir = Path(args.run_dir)
case = load_case(run_dir)
runs = [(f.parent, json.loads(f.read_text())) for f in sorted(run_dir.glob('*/run.json')) if f.parent.name != 'report']
header = ('EXPECTED OUTCOME CRITERIA (cite each criterion by its id in brackets):\n'
f'{criteria_text(case.get("expected"))}\n\n{quality_text()}')
if not args.blind:
print(header)
for sub, meta in runs:
print(run_section(sub, meta, f'{sub.name} ({meta.get("kind")})'))
return
order = runs[:]
random.Random(args.seed).shuffle(order)
letters = [chr(ord('A') + i) for i in range(len(order))]
sections = [run_section(sub, meta, f'Builder {letter}', blind=True) for letter, (sub, meta) in zip(letters, order)]
judge_dir = run_dir / 'judge'
judge_dir.mkdir(exist_ok=True)
(judge_dir / 'summary-blind.txt').write_text(header + '\n' + '\n'.join(sections))
(judge_dir / 'blind-map.json').write_text(json.dumps({letter: sub.name for letter, (sub, _) in zip(letters, order)}, indent=2))
print(judge_dir / 'summary-blind.txt')
if __name__ == '__main__':
main()

View File

@ -0,0 +1,459 @@
// Run a workflow in this process with the real node code, and answer its HTTP requests here.
//
// The n8n execution engine and the nodes load from the compiled dist folders of the repository,
// as in the execution evaluator of ai-workflow-builder.ee. n8n-core calls additionalData
// .evalLlmMockHandler before it authenticates or sends a request (the Instance AI eval hook),
// so every service request reaches respond() below and no request leaves the machine.
// Credentials come from buildEvalMockCredentials, the same placeholders that the eval uses.
// AI root nodes either get pinned data, or call their real model with a key from the settings.
import { createRequire } from 'node:module';
import { existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { repo } from './sdk.mjs';
const coreRequire = createRequire(join(repo, 'packages/core/package.json'));
const PACKAGES = [
['n8n-nodes-base', join(repo, 'packages/nodes-base')],
['@n8n/n8n-nodes-langchain', join(repo, 'packages/@n8n/nodes-langchain')],
];
const REQUIRED_DIST = [
'packages/core/dist/execution-engine/index.js',
'packages/nodes-base/dist/known/nodes.json',
'packages/@n8n/nodes-langchain/dist/known/nodes.json',
];
// --- Settings ------------------------------------------------------------------
// Same order as n8n_api.config(): .env.local, then .env.ai-surfaces, then the environment.
const fileSettings = {};
for (const file of [join(repo, '.env.local'), process.env.AI_SURFACES_ENV ?? join(repo, '.env.ai-surfaces')]) {
if (!existsSync(file)) continue;
for (const line of readFileSync(file, 'utf8').split('\n')) {
const match = line.match(/^\s*(?:export\s+)?([A-Z0-9_]+)\s*=\s*(.*)$/);
const value = match?.[2].trim().replace(/^(['"])(.*)\1$/, '$2');
if (value) fileSettings[match[1]] = value;
}
}
export const setting = (name) => process.env[name] || fileSettings[name] || '';
/** Model credential types, the settings that can hold their key, and the hosts of their API. */
export const MODEL_CREDENTIALS = {
googlePalmApi: { keys: ['GOOGLE_GENERATIVE_AI_API_KEY', 'GEMINI_API_KEY', 'GOOGLE_API_KEY'], field: 'apiKey', hosts: ['generativelanguage.googleapis.com'] },
openAiApi: { keys: ['OPENAI_API_KEY'], field: 'apiKey', hosts: ['api.openai.com'] },
anthropicApi: { keys: ['ANTHROPIC_API_KEY'], field: 'apiKey', hosts: ['api.anthropic.com'] },
mistralCloudApi: { keys: ['MISTRAL_API_KEY'], field: 'apiKey', hosts: ['api.mistral.ai'] },
groqApi: { keys: ['GROQ_API_KEY'], field: 'apiKey', hosts: ['api.groq.com'] },
};
export const modelKey = (type) => MODEL_CREDENTIALS[type]?.keys.map(setting).find(Boolean) ?? '';
// --- Engine --------------------------------------------------------------------
let engine;
/** The execution engine, node types and credential types. Returns { error } when a dist folder is missing. */
export function loadEngine() {
if (engine) return engine;
const missing = REQUIRED_DIST.filter((p) => !existsSync(join(repo, p)));
if (missing.length) return { error: `Build these first: ${missing.join(', ')}.` };
coreRequire('reflect-metadata');
const workflowLib = coreRequire('n8n-workflow');
const core = coreRequire(join(repo, 'packages/core/dist/execution-engine/index.js'));
const nodes = new Map();
const credentials = new Map();
for (const [prefix, dir] of PACKAGES) {
const read = (file) => JSON.parse(readFileSync(join(dir, 'dist/known', file), 'utf8'));
for (const [name, info] of Object.entries(read('nodes.json'))) nodes.set(`${prefix}.${name}`, { ...info, dir });
for (const [name, info] of Object.entries(read('credentials.json'))) credentials.set(name, { ...info, dir });
}
const instances = new Map();
const instance = (index, name) => {
const key = `${index === nodes ? 'n' : 'c'}:${name}`;
if (!instances.has(key)) {
const info = index.get(name);
if (!info) throw new Error(`Unknown type ${name}`);
const mod = createRequire(join(info.dir, 'package.json'))(join(info.dir, info.sourcePath));
instances.set(key, new mod[info.className]());
}
return instances.get(key);
};
const nodeTypes = {
getByName: (type) => instance(nodes, type),
getByNameAndVersion: (type, version) => {
const loaded = instance(nodes, type);
const versioned = 'nodeVersions' in loaded ? workflowLib.NodeHelpers.getVersionedNodeType(loaded, version) : loaded;
// The same rule as shouldAssignExecuteMethod in packages/cli/src/utils.ts. Without it a
// declarative node with a webhook (WhatsApp send and wait) gets no input data for $json.
const declarative = versioned.description?.requestDefaults !== undefined;
if (!versioned.execute && !versioned.supplyData && !versioned.poll && !versioned.trigger && (!versioned.webhook || declarative) && (!versioned.methods || declarative)) {
versioned.execute = async function () {
return (await new core.RoutingNode(this, versioned).runNode()) ?? [];
};
}
return versioned;
},
getKnownTypes: () => ({}),
};
const credentialType = (name) => instance(credentials, name);
const parentTypes = (name) => {
try {
return (credentialType(name).extends ?? []).flatMap((p) => [p, ...parentTypes(p)]);
} catch {
return [];
}
};
const credentialProperties = (name) => [...(credentialType(name).extends ?? []).flatMap(credentialProperties), ...(credentialType(name).properties ?? [])];
engine = { workflowLib, core, nodeTypes, nodes, credentialType, parentTypes, credentialProperties };
return engine;
}
/** Credential types that a node can use, from its node type description. */
export function nodeCredentialTypes(node) {
try {
return (loadEngine().nodeTypes.getByNameAndVersion(node.type, node.typeVersion).description.credentials ?? []).map((c) => c.name);
} catch {
return [];
}
}
/** Apply the generic authenticate rule of a credential type, such as `qs.key` of Google Gemini. */
function applyGenericAuth(credentialType, credentials, request) {
const auth = credentialType.authenticate;
if (!auth || typeof auth !== 'object' || auth.type !== 'generic') return request;
const fill = (v) => (typeof v === 'string' ? v.replace(/^=/, '').replace(/\{\{\s*\$credentials\.(\w+)\s*\}\}/g, (_m, k) => credentials[k] ?? '') : v);
for (const part of ['headers', 'qs', 'body']) {
const props = auth.properties?.[part];
if (props) request[part] = { ...(request[part] ?? {}), ...Object.fromEntries(Object.entries(props).map(([k, v]) => [k, fill(v)])) };
}
return request;
}
function credentialsHelper(realModels) {
const { core, parentTypes, credentialProperties, credentialType } = loadEngine();
return {
getParentTypes: parentTypes,
getDecrypted: async (_data, _details, type) => {
const mocked = core.buildEvalMockCredentials(credentialProperties(type));
const model = MODEL_CREDENTIALS[type];
if (realModels && model) mocked[model.field] = modelKey(type);
return mocked;
},
// Vendor model nodes send their request through the n8n helpers, so the real key must be applied here.
authenticate: async (credentials, type, request) => (realModels && MODEL_CREDENTIALS[type] ? applyGenericAuth(credentialType(type), credentials, request) : request),
preAuthentication: async () => undefined,
runPreAuthentication: async () => undefined,
getCredentials: async () => ({}),
getCredentialsProperties: (type) => credentialProperties(type),
updateCredentials: async () => {},
updateCredentialsOauthTokenData: async () => {},
isCredentialUsableByNode: () => true,
};
}
// --- Setup values ----------------------------------------------------------------
/** Values that a node accepts for a parameter that the builder left for setup. */
const SETUP_VALUES = {
'googleSheets.sheetName': 'gid=0',
};
const isEmptyValue = (v) => v === undefined || v === null || v === '';
/**
* A copy of the workflow with placeholder values in the parameters that the builder left for
* the user: empty resource locators, and empty parameters that the editor reports as required.
* Returns { workflow, filled }.
*/
export function fillSetup(workflow, canvas) {
const copy = structuredClone(workflow);
const filled = [];
const required = new Map();
for (const node of canvas?.nodes ?? []) {
for (const [param, messages] of Object.entries(node.issues?.parameters ?? {})) {
if (messages.some((m) => /is required/i.test(m))) required.set(`${node.name}|${param}`, true);
}
}
for (const node of copy.nodes ?? []) {
const short = node.type.slice(node.type.lastIndexOf('.') + 1);
for (const [param, value] of Object.entries(node.parameters ?? {})) {
const placeholder = SETUP_VALUES[`${short}.${param}`] ?? `mock-${param}`;
if (value && typeof value === 'object' && value.__rl && isEmptyValue(value.value)) {
node.parameters[param] = { ...value, value: placeholder };
filled.push(`${node.name} › ${param}`);
} else if (isEmptyValue(value) && required.has(`${node.name}|${param}`)) {
node.parameters[param] = placeholder;
filled.push(`${node.name} › ${param}`);
}
}
}
return { workflow: copy, filled };
}
// --- Wire responses --------------------------------------------------------------
const WRITE_METHODS = new Set(['POST', 'PUT', 'PATCH', 'DELETE']);
export const isWrite = (request) => WRITE_METHODS.has(request.method);
/** Rows for a read node: [{ Question, Answer }, ...] become the header row and the value rows. */
function sheetValues(rows) {
const headers = [...new Set(rows.flatMap((r) => Object.keys(r ?? {})))];
if (!headers.length) return [['Question', 'Answer']];
return [headers, ...rows.map((r) => headers.map((h) => (r?.[h] === undefined ? '' : String(r[h]))))];
}
/** Page properties as Notion returns them: each with an id, a type, and plain_text in rich text. */
function notionProperties(sent = {}) {
const out = {};
for (const [name, value] of Object.entries(sent ?? {})) {
const type = value?.type ?? Object.keys(value ?? {}).find((k) => k !== 'id' && k !== 'type') ?? 'rich_text';
let content = value?.[type];
if (Array.isArray(content)) content = content.map((t) => ({ type: 'text', ...t, plain_text: t?.plain_text ?? t?.text?.content ?? '' }));
out[name] = { id: `prop-${Object.keys(out).length}`, type, [type]: content ?? null };
}
if (!Object.values(out).some((p) => p.type === 'title')) out.Name = { id: 'title', type: 'title', title: [] };
return out;
}
const bodyOf = (request) => {
if (typeof request.body !== 'string') return request.body ?? {};
try {
return JSON.parse(request.body);
} catch {
return {};
}
};
/**
* Responses in the shape that each API documents. A fixture matches on host, method and path.
* rows(node) gives the case rows for a read node (scenarios.json pinByType).
*/
const FIXTURES = [
{
service: 'Google Sheets',
host: /(^|\.)sheets\.googleapis\.com$/,
method: 'GET',
path: /^\/v4\/spreadsheets\/[^/]+$/,
respond: () => ({ spreadsheetId: 'mock-sheet', properties: { title: 'FAQ' }, sheets: [{ properties: { sheetId: 0, title: 'Sheet1', index: 0, gridProperties: { rowCount: 100, columnCount: 26 } } }] }),
},
{
service: 'Google Sheets',
host: /(^|\.)sheets\.googleapis\.com$/,
method: 'GET',
path: /^\/v4\/spreadsheets\/[^/]+\/values\//,
respond: (_r, ctx, node) => ({ range: 'Sheet1!A1:Z100', majorDimension: 'ROWS', values: sheetValues(ctx.rows(node) ?? []) }),
},
{
service: 'Google Sheets',
host: /(^|\.)sheets\.googleapis\.com$/,
method: 'POST',
path: /:append$|values:batchUpdate$|:batchUpdate$/,
respond: () => ({ spreadsheetId: 'mock-sheet', updates: { updatedRange: 'Sheet1!A2:B2', updatedRows: 1 }, replies: [] }),
},
{
service: 'Notion',
host: /(^|\.)api\.notion\.com$/,
method: 'GET',
path: /^\/v1\/(databases|data_sources)\/[^/]+$/,
respond: (r) => ({
object: r.url.includes('data_sources') ? 'data_source' : 'database',
id: 'mock-database',
title: [{ type: 'text', plain_text: 'Support tickets', text: { content: 'Support tickets' } }],
properties: { Name: { id: 'title', name: 'Name', type: 'title', title: {} } },
data_sources: [{ id: 'mock-data-source', name: 'Support tickets' }],
}),
},
{
service: 'Notion',
host: /(^|\.)api\.notion\.com$/,
method: 'POST',
path: /^\/v1\/pages$/,
respond: (r) => {
const body = bodyOf(r);
return { object: 'page', id: '59833787-2cf9-4fdf-8782-e53db20768a5', created_time: '2026-01-01T09:00:00.000Z', last_edited_time: '2026-01-01T09:00:00.000Z', archived: false, in_trash: false, parent: body.parent ?? {}, properties: notionProperties(body.properties), url: 'https://www.notion.so/Ticket-598337872cf94fdf8782e53db20768a5', public_url: null };
},
},
{
service: 'Notion',
host: /(^|\.)api\.notion\.com$/,
method: 'PATCH',
path: /^\/v1\/blocks\/[^/]+\/children$/,
respond: (r) => ({ object: 'list', results: (bodyOf(r).children ?? []).map((c, i) => ({ object: 'block', id: `mock-block-${i}`, ...c })), next_cursor: null, has_more: false }),
},
{
service: 'Slack',
host: /(^|\.)slack\.com$/,
method: 'POST',
path: /^\/api\/chat\.postMessage$/,
respond: (r) => {
const body = bodyOf(r);
return { ok: true, channel: body.channel ?? 'C0000000000', ts: '1767258000.000100', message: { type: 'message', text: body.text ?? '', ts: '1767258000.000100', bot_id: 'B0000000000' } };
},
},
{
service: 'WhatsApp Cloud API',
host: /(^|\.)graph\.facebook\.com$/,
method: 'POST',
path: /^\/v\d+(\.\d+)?\/[^/]+\/messages$/,
respond: (r) => {
const to = bodyOf(r).to ?? '';
return { messaging_product: 'whatsapp', contacts: [{ input: to, wa_id: to }], messages: [{ id: 'wamid.HBgLMTU1NTAwMDExMTEVAgARGBI5QTNDQTVCM0Q0Q0Q2RTY3RTcA' }] };
},
},
{
service: 'Telegram Bot API',
host: /(^|\.)api\.telegram\.org$/,
method: 'POST',
path: /\/sendMessage$/,
respond: (r) => ({ ok: true, result: { message_id: 2, date: 1767258000, chat: { id: bodyOf(r).chat_id ?? 1001, type: 'private' }, text: bodyOf(r).text ?? '' } }),
},
{
service: 'Gmail',
host: /(^|\.)gmail\.googleapis\.com$|(^|\.)www\.googleapis\.com$/,
method: 'POST',
path: /\/messages\/send$/,
respond: () => ({ id: 'mock-message', threadId: 'mock-thread', labelIds: ['SENT'] }),
},
];
/**
* A request handler for evalLlmMockHandler. Every request is recorded in ctx.requests with the
* fixture that answered it. A request with no fixture gets a generic answer: a read gets an
* empty object, a write gets its own body back with an id. A request to a host in passHosts
* goes to the network: these are the model APIs of the real-AI scenarios.
*/
export function wireHandler(ctx, passHosts = new Set()) {
return async (request, node) => {
const url = new URL(request.url ?? '', 'https://unknown.invalid');
if (request.baseURL && !/^https?:/.test(request.url ?? '')) {
const base = new URL(request.baseURL);
url.host = base.host;
url.pathname = `${base.pathname.replace(/\/$/, '')}/${(request.url ?? '').replace(/^\//, '')}`;
}
const method = String(request.method ?? 'GET').toUpperCase();
if (passHosts.has(url.hostname)) {
ctx.requests.push({ node: node.name, nodeType: node.type, method, url: url.toString(), body: request.body, answeredBy: 'real model' });
return undefined;
}
const fixture = FIXTURES.find((f) => f.host.test(url.hostname) && f.method === method && f.path.test(url.pathname));
const record = { node: node.name, nodeType: node.type, method, url: url.toString(), qs: request.qs, body: request.body, answeredBy: fixture?.service ?? 'generic' };
let body;
try {
body = fixture ? fixture.respond({ ...request, url: url.toString() }, ctx, node) : method === 'GET' ? {} : { id: 'mock-1', ...(bodyOf(request) ?? {}) };
} catch (error) {
body = { error: String(error) };
}
ctx.requests.push({ ...record, response: body });
return { statusCode: 200, headers: { 'content-type': 'application/json' }, body };
};
}
// --- Network guard ---------------------------------------------------------------
/**
* Replace global fetch while a run is active. Model SDKs use fetch directly, so this is the
* only path that could reach the network. Only the hosts in allow pass, the rest is blocked.
*/
function guardFetch(allow, blocked) {
const original = globalThis.fetch;
globalThis.fetch = async (input, init) => {
const url = new URL(typeof input === 'string' ? input : input.url ?? String(input));
if (allow.has(url.hostname)) return await original(input, init);
blocked.push(`${init?.method ?? 'GET'} ${url.origin}${url.pathname}`);
throw new Error(`Blocked network request to ${url.hostname} during a test run.`);
};
return () => {
globalThis.fetch = original;
};
}
// --- Code nodes ------------------------------------------------------------------
/**
* additionalData.startRunnerTask. The n8n server sends Code node jobs to a sandboxed task runner.
* This process has no sandbox, so it never runs builder code. run_wire.mjs skips workflows with a
* Code node. This refusal catches a node type that it does not know.
*/
async function startRunnerTask(_data, jobType) {
return { ok: false, error: { message: `${jobType} code does not run in the in-process run.` } };
}
// --- Run -------------------------------------------------------------------------
const CANCEL_GRACE_MS = 15_000;
/** The value of promise, or null after ms. */
async function within(promise, ms) {
let timer;
try {
return await Promise.race([promise, new Promise((resolve) => (timer = setTimeout(() => resolve(null), ms)))]);
} finally {
clearTimeout(timer);
}
}
/** The pinned trigger that a run starts from: triggerNode, or else the first pinned trigger. */
export const startNode = (workflow, pinData, triggerNode) =>
workflow.nodes.find((n) => (!triggerNode || n.name === triggerNode) && Object.hasOwn(pinData, n.name) && !n.disabled && Object.keys(workflow.connections ?? {}).includes(n.name) && /trigger|webhook/i.test(n.type));
/**
* Run a workflow once. pinData holds the trigger and any pinned nodes. triggerNode names the
* trigger to start from. With realModels, model sub-nodes get the real key and fetch can reach
* their API hosts. Returns { status, error, runData, requests, blocked, ms }.
*/
export async function runWorkflow(workflow, { pinData, triggerNode, rows = () => undefined, realModels = false, timeoutMs = 90_000 }) {
const { workflowLib, core, nodeTypes } = loadEngine();
const started = Date.now();
const ctx = { requests: [], rows };
const blocked = [];
const allow = new Set(realModels ? Object.values(MODEL_CREDENTIALS).flatMap((m) => m.hosts) : []);
const restore = guardFetch(allow, blocked);
let stillRunning = false;
try {
const instance = new workflowLib.Workflow({ id: 'wire-run', name: workflow.name, nodes: workflow.nodes, connections: workflow.connections, nodeTypes, active: false, settings: workflow.settings ?? {} });
const start = startNode(workflow, pinData, triggerNode);
if (!start) return { status: 'error', error: 'No pinned trigger to start from.', runData: {}, requests: [], blocked, ms: 0 };
const hooks = new core.ExecutionLifecycleHooks('manual', 'wire-run', workflow);
const finished = new Promise((resolve) => hooks.addHandler('workflowExecuteAfter', (run) => resolve(run)));
const values = {
executionId: 'wire-run',
hooks,
credentialsHelper: credentialsHelper(realModels),
evalLlmMockHandler: wireHandler(ctx, allow),
startRunnerTask,
getRunnerStatus: () => ({ available: false }),
variables: {},
restApiUrl: 'http://localhost:5678/rest',
instanceBaseUrl: 'http://localhost:5678',
webhookBaseUrl: 'http://localhost:5678/webhook',
webhookTestBaseUrl: 'http://localhost:5678/webhook-test',
webhookWaitingBaseUrl: 'http://localhost:5678/webhook-waiting',
formWaitingBaseUrl: 'http://localhost:5678/form-waiting',
userId: 'wire-run-user',
};
// Unset fields stay undefined. Methods that n8n calls for logs and UI updates do nothing.
const additionalData = new Proxy({}, {
get: (_t, prop) => (prop in values ? values[prop] : typeof prop === 'string' && /^(log|send|set|on|emit|track)/.test(prop) ? () => undefined : undefined),
});
const runExecutionData = workflowLib.createRunExecutionData({
executionData: { nodeExecutionStack: [{ node: start, data: { main: [pinData[start.name]] }, source: null }] },
resultData: { pinData },
});
const execute = new core.WorkflowExecute(additionalData, 'manual', runExecutionData);
const execution = execute.processRunExecutionData(instance);
const run = await within(execution.then(() => finished), timeoutMs);
if (!run) {
// Cancel aborts the running node. Wait for the engine to stop, so no request runs after the guard is gone.
execution.cancel();
stillRunning = !(await within(execution.then(() => true, () => true), CANCEL_GRACE_MS));
const error = `No result after ${timeoutMs / 1000} s.${stillRunning ? ' The run did not stop after the cancel.' : ''}`;
return { status: 'timeout', error, runData: {}, requests: ctx.requests, blocked, ms: Date.now() - started };
}
const error = run.data?.resultData?.error;
return { status: run.status, error: error ? error.message ?? String(error) : null, runData: run.data?.resultData?.runData ?? {}, requests: ctx.requests, blocked, ms: Date.now() - started };
} catch (error) {
return { status: 'error', error: `${error?.name ?? 'Error'}: ${error?.message ?? error}`, runData: {}, requests: ctx.requests, blocked, ms: Date.now() - started };
} finally {
// A run that did not stop keeps the guard. The process exits when all runs are done.
if (!stillRunning) restore();
}
}

View File

@ -0,0 +1 @@
../../../../.agents/skills/env-var-docs

View File

@ -0,0 +1 @@
../../../../.agents/skills/testmax-assistant-ui

View File

@ -0,0 +1 @@
../../../../.agents/skills/testmax-n8n-ai

View File

@ -989,14 +989,6 @@
"hash": "26b2999a3856"
}
],
"packages/extensions/insights/oxlint.config.mts": [
{
"rule": "lint-config-layering",
"line": 9,
"message": "packages/extensions/insights turns 'typescript/no-deprecated' down to 'off' for the whole package.",
"hash": "f936bed8fa8d"
}
],
"packages/frontend/editor-ui/eslint.config.mjs": [
{
"rule": "lint-config-layering",
@ -1412,6 +1404,650 @@
"message": "Override \"langsmith\" duplicates a catalog entry for langsmith — set the override to \"catalog:\" or remove it",
"hash": "9ac3dbae3d22"
}
],
"packages/cli/src/binary-data/database.manager.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 2,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "a4aeaccef8b6"
}
],
"packages/cli/src/credentials/credential-connection-status-provider.interface.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 1,
"message": "Business logic imports TypeORM directly.",
"hash": "4d287b55d52a"
}
],
"packages/cli/src/credentials/credential-connection-status-proxy.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 2,
"message": "Business logic imports TypeORM directly.",
"hash": "f425e2b10e7f"
}
],
"packages/cli/src/credentials/credential-dependency.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 8,
"message": "Business logic imports TypeORM directly.",
"hash": "abdb9e128ab8"
}
],
"packages/cli/src/credentials/credentials.controller.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 32,
"message": "Business logic imports TypeORM directly.",
"hash": "510cfb224f30"
}
],
"packages/cli/src/credentials/credentials.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 12,
"message": "Business logic imports TypeORM directly.",
"hash": "f759a907bfa6"
}
],
"packages/cli/src/credentials/credentials.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 32,
"message": "Business logic imports TypeORM directly.",
"hash": "7f1b3146a6a5"
}
],
"packages/cli/src/executions/execution-persistence.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 7,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "9c7a73941bb9"
},
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "packages/cli imports the TypeORM re-export `FindManyOptions` from @n8n/db.",
"hash": "2beb2d36dbee"
},
{
"rule": "typeorm-persistence-boundary",
"line": 11,
"message": "packages/cli imports the TypeORM re-export `FindOptionsWhere` from @n8n/db.",
"hash": "ef8617dcfa5a"
},
{
"rule": "typeorm-persistence-boundary",
"line": 18,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "be7b5858aa7c"
},
{
"rule": "typeorm-persistence-boundary",
"line": 18,
"message": "packages/cli imports the TypeORM re-export `Not` from @n8n/db.",
"hash": "5faea85ebf23"
}
],
"packages/cli/src/permissions.ee/check-access.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 2,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "ffa998a4f0e7"
}
],
"packages/cli/src/scheduling/durable-job-provisioner.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 5,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "7341e70aa4d7"
},
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "packages/cli imports the TypeORM re-export `DataSource` from @n8n/db.",
"hash": "64f5ed0abb25"
}
],
"packages/cli/src/scheduling/durable-scheduler.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "packages/cli imports the TypeORM re-export `DataSource` from @n8n/db.",
"hash": "aac4ac3f74d8"
}
],
"packages/cli/src/services/export.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "Business logic imports TypeORM directly.",
"hash": "079882e3843e"
}
],
"packages/cli/src/services/folder.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "Business logic imports TypeORM directly.",
"hash": "b71b84bbde01"
}
],
"packages/cli/src/services/hooks.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 15,
"message": "Business logic imports TypeORM directly.",
"hash": "112edc87061f"
}
],
"packages/cli/src/services/import.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 17,
"message": "Business logic imports TypeORM directly.",
"hash": "5f09782b0c0a"
}
],
"packages/cli/src/services/project.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 33,
"message": "Business logic imports TypeORM directly.",
"hash": "5f5086c28a45"
}
],
"packages/cli/src/services/public-api-key.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 15,
"message": "Business logic imports TypeORM directly.",
"hash": "848bd2e47604"
}
],
"packages/cli/src/services/user.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 15,
"message": "packages/cli imports the TypeORM re-export `Not` from @n8n/db.",
"hash": "afb93b8300ff"
},
{
"rule": "typeorm-persistence-boundary",
"line": 16,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "2fd9c6af03c1"
}
],
"packages/cli/src/workflows/workflow-creation.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 5,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "db8f5c7d285b"
}
],
"packages/cli/src/workflows/workflow-finder.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 11,
"message": "Business logic imports TypeORM directly.",
"hash": "81b365efba5a"
}
],
"packages/cli/src/workflows/workflow-validation.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "Business logic imports TypeORM directly.",
"hash": "fce2fb07a423"
}
],
"packages/cli/src/workflows/workflow.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 22,
"message": "Business logic imports TypeORM directly.",
"hash": "1e39cc245dd5"
}
],
"packages/cli/src/workflows/workflow.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 23,
"message": "Business logic imports TypeORM directly.",
"hash": "ba3b76242d76"
}
],
"packages/cli/src/workflows/workflows.controller.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 39,
"message": "Business logic imports TypeORM directly.",
"hash": "63742f55819c"
}
],
"packages/cli/src/commands/import/credentials.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 19,
"message": "Business logic imports TypeORM directly.",
"hash": "a5cc02a1d765"
}
],
"packages/cli/src/evaluation.ee/test-runner/test-runner.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 17,
"message": "Business logic imports TypeORM directly.",
"hash": "d34dbe0bf9e7"
}
],
"packages/cli/src/events/relays/telemetry.event-relay.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 8,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "d8a744f2e2b8"
}
],
"packages/cli/src/executions/execution-data/db-store.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 1,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "c19615f89980"
},
{
"rule": "typeorm-persistence-boundary",
"line": 2,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "47398e9fd892"
}
],
"packages/cli/src/instance-settings-loader/loaders/log-streaming.instance-settings-loader.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 3,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "b6baaab2dc92"
}
],
"packages/cli/src/modules/agents/agent-publish.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 15,
"message": "Business logic imports TypeORM directly.",
"hash": "9857b27c5bb1"
}
],
"packages/cli/src/modules/agents/agent-task.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 8,
"message": "Business logic imports TypeORM directly.",
"hash": "1799d75327d9"
}
],
"packages/cli/src/modules/agents/agents.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 18,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "e893390cf6b7"
}
],
"packages/cli/src/modules/agents/instance-ai-builder-delegate.adapter.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 13,
"message": "Business logic imports TypeORM directly.",
"hash": "a2f27310e801"
}
],
"packages/cli/src/modules/breaking-changes/breaking-changes.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 12,
"message": "Business logic imports TypeORM directly.",
"hash": "680ac8daac26"
}
],
"packages/cli/src/modules/chat-hub/chat-hub-agent.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "0aef6c89993c"
}
],
"packages/cli/src/modules/chat-hub/chat-hub-credentials.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 8,
"message": "Business logic imports TypeORM directly.",
"hash": "6125ed3d7f9b"
}
],
"packages/cli/src/modules/chat-hub/chat-hub-title.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 11,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "57eab6d29eed"
}
],
"packages/cli/src/modules/chat-hub/chat-hub-tool.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 7,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "7069ba20f7cd"
}
],
"packages/cli/src/modules/chat-hub/chat-hub-workflow.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 25,
"message": "Business logic imports TypeORM directly.",
"hash": "7305b701b234"
}
],
"packages/cli/src/modules/chat-hub/chat-hub.attachment.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 3,
"message": "Business logic imports TypeORM directly.",
"hash": "07380799ff46"
}
],
"packages/cli/src/modules/chat-hub/chat-hub.models.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "830f87d4b9eb"
}
],
"packages/cli/src/modules/chat-hub/chat-hub.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 17,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "55126ad50e59"
}
],
"packages/cli/src/modules/chat-hub/chat-hub.settings.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 11,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "ee4b7ca996c4"
}
],
"packages/cli/src/modules/data-table/data-table-ddl.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 3,
"message": "Business logic imports TypeORM directly.",
"hash": "9d6b2c45c02a"
}
],
"packages/cli/src/modules/data-table/data-table.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 18,
"message": "Business logic imports TypeORM directly.",
"hash": "b1aa220b40ab"
}
],
"packages/cli/src/modules/external-secrets.ee/secrets-providers-connections.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 20,
"message": "Business logic imports TypeORM directly.",
"hash": "45167590d2e2"
}
],
"packages/cli/src/modules/favorites/favorites.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 12,
"message": "Business logic imports TypeORM directly.",
"hash": "03cdaa860f2a"
}
],
"packages/cli/src/modules/instance-ai/instance-ai.adapter.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 117,
"message": "Business logic imports TypeORM directly.",
"hash": "afbc7f6b5ec7"
}
],
"packages/cli/src/modules/instance-ai/suspended-thread-persistence.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 3,
"message": "Business logic imports TypeORM directly.",
"hash": "5f8f9db2966f"
}
],
"packages/cli/src/modules/log-streaming.ee/log-streaming-destination.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 5,
"message": "Business logic imports TypeORM directly.",
"hash": "f31366dabfeb"
}
],
"packages/cli/src/modules/mcp/mcp-api-key.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 3,
"message": "Business logic imports TypeORM directly.",
"hash": "c2cbaae08659"
}
],
"packages/cli/src/modules/mcp/mcp.settings.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 8,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "6ff9d5453466"
},
{
"rule": "typeorm-persistence-boundary",
"line": 11,
"message": "Business logic imports TypeORM directly.",
"hash": "4c5e31384fc0"
}
],
"packages/cli/src/modules/provisioning.ee/provisioning.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 23,
"message": "Business logic imports TypeORM directly.",
"hash": "fd2fa9036a75"
}
],
"packages/cli/src/modules/provisioning.ee/role-mapping-rule.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 16,
"message": "Business logic imports TypeORM directly.",
"hash": "d060301c09ad"
}
],
"packages/cli/src/modules/provisioning.ee/role-resolver.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "Business logic imports TypeORM directly.",
"hash": "15464f0f7a2a"
}
],
"packages/cli/src/modules/source-control.ee/source-control-context.factory.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 5,
"message": "Business logic imports TypeORM directly.",
"hash": "cbc873aadc22"
}
],
"packages/cli/src/modules/source-control.ee/source-control-export.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 15,
"message": "Business logic imports TypeORM directly.",
"hash": "8e15c3127f0d"
}
],
"packages/cli/src/modules/source-control.ee/source-control-import.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "packages/cli imports the TypeORM re-export `FindOptionsWhere` from @n8n/db.",
"hash": "2ca6a59e5a14"
},
{
"rule": "typeorm-persistence-boundary",
"line": 29,
"message": "Business logic imports TypeORM directly.",
"hash": "e472973ab769"
}
],
"packages/cli/src/modules/source-control.ee/source-control-scoped.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 12,
"message": "Business logic imports TypeORM directly.",
"hash": "12e1f73d1d57"
}
],
"packages/cli/src/modules/source-control.ee/source-control-status.service.ee.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 7,
"message": "Business logic imports TypeORM directly.",
"hash": "627dc538505e"
}
],
"packages/cli/src/modules/workflow-index/workflow-dependency-query.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 17,
"message": "Business logic imports TypeORM directly.",
"hash": "27681e00a358"
}
],
"packages/cli/src/scheduling/poll-trigger-node/poll-trigger-job-registrar.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "faaf4ec6e197"
}
],
"packages/cli/src/scheduling/schedule-trigger-node/schedule-trigger-job-registrar.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "packages/cli imports the TypeORM re-export `EntityManager` from @n8n/db.",
"hash": "795cfafd58f5"
}
],
"packages/cli/src/workflows/workflow-history/workflow-history.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 12,
"message": "Business logic imports TypeORM directly.",
"hash": "4405da700deb"
}
],
"packages/cli/src/modules/agents/integrations/n8n-memory.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 45,
"message": "Business logic imports TypeORM directly.",
"hash": "3f27e7921159"
}
],
"packages/cli/src/modules/data-table/utils/sql-utils.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "Business logic imports TypeORM directly.",
"hash": "9780aefcf874"
}
],
"packages/cli/src/modules/dynamic-credentials.ee/services/credential-connection-status.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 4,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "00c000824ff4"
},
{
"rule": "typeorm-persistence-boundary",
"line": 13,
"message": "Business logic imports TypeORM directly.",
"hash": "15940453447b"
}
],
"packages/cli/src/modules/dynamic-credentials.ee/services/credential-resolver-workflow.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 1,
"message": "packages/cli imports the TypeORM re-export `In` from @n8n/db.",
"hash": "e867f51e2b7c"
}
],
"packages/cli/src/modules/dynamic-credentials.ee/services/credential-resolver.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 10,
"message": "Business logic imports TypeORM directly.",
"hash": "9b6ffe4b90ce"
}
],
"packages/cli/src/modules/instance-ai/storage/typeorm-agent-checkpoint-store.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 3,
"message": "Business logic imports TypeORM directly.",
"hash": "5cac6e5f7c07"
}
],
"packages/cli/src/modules/instance-ai/storage/typeorm-agent-memory.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 30,
"message": "Business logic imports TypeORM directly.",
"hash": "eb011718641f"
}
],
"packages/cli/src/modules/instance-ai/storage/typeorm-observation-log-store.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 20,
"message": "Business logic imports TypeORM directly.",
"hash": "e45963a37c9c"
}
],
"packages/cli/src/modules/token-exchange/services/trusted-key.service.ts": [
{
"rule": "typeorm-persistence-boundary",
"line": 6,
"message": "Business logic imports TypeORM directly.",
"hash": "b44bfdcf5ae3"
}
]
}
}

75
.env.ai-surfaces.example Normal file
View File

@ -0,0 +1,75 @@
# Example env for comparing AI workflow builders (Instance AI, Claude Code + n8n MCP,
# Codex + n8n MCP) on a local instance.
#
# Copy to `.env.ai-surfaces` (gitignored) and fill in the values that you need.
# Never commit keys. The testmax-n8n-ai skill reads the files in this
# order, from low to high priority: .env.local, .env.ai-surfaces, the shell
# environment. Empty values are ignored, so keys that are already in .env.local
# work without a change. Set AI_SURFACES_ENV to use a file in a different place.
#
# See .agents/skills/testmax-n8n-ai/SKILL.md for how the values are used.
# --- n8n ---------------------------------------------------------------------
# Empty values use .env.local, then the defaults: http://localhost:5678,
# nathan@n8n.io and PlaywrightTest123.
N8N_BASE_URL=
N8N_EVAL_EMAIL=
N8N_EVAL_PASSWORD=
# Optional. The MCP key of the user above (Settings > MCP). When set, the skill
# does not need the cached token and never asks to rotate the key.
N8N_MCP_TOKEN=
# --- Run defaults --------------------------------------------------------------
COMPARE_BUILDERS=iai,claude,codex
# container (Docker) or clean (Claude only, no Docker)
COMPARE_ISOLATION=container
COMPARE_RUNS_DIR=/tmp/builder-compare
# 1 runs the builders at the same time. 0 runs them one at a time.
COMPARE_PARALLEL=1
# 1 opens a local page that shows the runs live and then the report. 0 turns it off.
COMPARE_LIVE=1
# 1 also opens the Instance AI thread in the browser and a terminal window for
# each CLI builder.
COMPARE_LIVE_WINDOWS=0
# 1 records a video of the Instance AI thread for the replay on the page.
COMPARE_RECORD=1
# interactive records the real terminal screen of Claude Code and Codex.
# headless uses `claude -p` and `codex exec` and has no screen recording.
COMPARE_CLI_MODE=interactive
# --- Claude Code + n8n MCP -----------------------------------------------------
# Empty model uses the Claude Code default.
CLAUDE_MODEL=
# none: Claude Code has only the n8n tools. default: it also has its built-in tools,
# such as Bash, Read, Write and WebFetch, and uses all of them without a question,
# like Codex in the same container. Clean isolation always uses none.
COMPARE_CLAUDE_TOOLS=none
# 1 runs each created workflow with pinned data through the n8n MCP server, once
# for each scenario in scenarios.json. 0 turns it off.
COMPARE_SCENARIOS=1
# 1 runs each created workflow in process with recorded API fixtures, for the
# scenarios, edge cases and questions in scenarios.json. 0 turns it off.
COMPARE_WIRE=1
# 1 runs the deterministic binary checks of the Instance AI eval suite.
COMPARE_BINARY_CHECKS=1
# Model keys for the real AI runs of the questions in scenarios.json. A workflow
# model without a key skips those runs. Set the key of each model provider that
# the builders can pick: GOOGLE_GENERATIVE_AI_API_KEY, OPENAI_API_KEY,
# ANTHROPIC_API_KEY, MISTRAL_API_KEY or GROQ_API_KEY.
GOOGLE_GENERATIVE_AI_API_KEY=
# Falls back to ANTHROPIC_API_KEY.
CLAUDE_API_KEY=
# --- Codex + n8n MCP -----------------------------------------------------------
# Empty model uses the Codex default.
CODEX_MODEL=
# Falls back to OPENAI_API_KEY.
CODEX_API_KEY=
# --- Instance AI cost (LangSmith) ----------------------------------------------
# The n8n instance must trace to the same project (LANGSMITH_TRACING=true).
LANGSMITH_API_KEY=
# Use the same region as the instance. Empty uses https://api.smith.langchain.com.
# For the EU region, set https://eu.api.smith.langchain.com.
LANGSMITH_ENDPOINT=
LANGSMITH_PROJECT=instance-ai

View File

@ -85,6 +85,7 @@ return 'blacksmith';
| Accessibility buckets | `packages/quality/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
| Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
| Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
| Dive image efficiency | `packages/quality/efficiency/images/image-efficiency.spec.ts` | Layer content size, estimated waste, efficiency score, and layer count through the shared Playwright reporter |
| Container stack | `packages/quality/environments/containers/telemetry.ts` | E2E stack startup times per service |
## Secrets

32
.github/WORKFLOWS.md vendored
View File

@ -236,14 +236,6 @@ parallelism). See the `--build-via-mcp` section in
| PR merged to `release/*` | `release-publish.yml` |
| PR closed with `codespace-preview` | `util-codespace-preview.yml` |
### Manual Triggers (PR Comments)
| Command | Workflow | Permissions |
|--------------------|------------------------------|---------------------|
| `/test-workflows` | `test-workflows-callable.yml`| admin/write/maintain|
**Why:** Re-run tests without pushing commits. Useful for flaky test investigation.
### Label Triggers
| Label | Workflow | Effect |
@ -419,7 +411,9 @@ Run-workflow dropdown, which only a user with write access can pick.
|-----------------------------|---------------------------------------------------------|
| `util-codespace-preview.yml`| Wake, re-serve or delete a PR preview instance by hand |
| `util-data-tooling.yml` | SQLite/PostgreSQL export/import validation (manual) |
| `test-image-efficiency.yml` | Measure n8n and runners image size and layer waste with Dive |
| `util-probe-registry.yml` | Diagnose slow npm metadata fetches (temporary) |
| `sec-sync-bundle-branches.yml` | Sync both bundle branches on demand, including before a bundle cut |
---
@ -453,10 +447,6 @@ test-sbom-nightly.yml
test-workflows-nightly.yml (manual dispatch only — nightly schedule disabled, DEVP-544)
└──────────────────────────▶ test-workflows-callable.yml
PR Comment Dispatchers (triggered by /command in PR comments):
test-workflows-pr-comment.yml
└──────────────────────────▶ test-workflows-callable.yml
```
---
@ -597,7 +587,7 @@ Push to master
| Daily 03:00 | `sec-sync-bundle-branches.yml` | Merge the base into `bundle/*` |
| Daily 00:00 | `docker-build-push.yml` | Nightly Docker images |
| Daily 00:00 | `test-db.yml` | Database compatibility |
| Daily 00:00 | `test-e2e-performance-reusable.yml`| Performance E2E |
| Daily 00:00 | `test-e2e-performance-reusable.yml`| Browser performance and idle memory baselines |
| Daily 00:00 | `release-storybook.yml` | Storybook deploy |
| Daily 00:00 | `release-chromatic.yml` | Visual regression |
| Daily 00:00 | `util-check-docs-urls.yml` | Doc link validation |
@ -610,6 +600,7 @@ Push to master
| Daily 08:00 | `build-v3-nightly.yml` | Nightly v3 Docker images |
| Monday 00:00 | `util-update-node-popularity.yml` | Node usage stats |
| Monday 02:00 | `test-e2e-coverage-weekly.yml` | Weekly E2E coverage |
| Wednesday 04:00 | `util-update-get-n8n-versions.yml` | get-n8n stable versions |
| Saturday 22:00 | `test-evals-ai.yml` | AI workflow evals |
| 1st of month 04:00 | `util-refresh-cubic-schema.yml` | Refresh vendored cubic schema |
@ -1150,9 +1141,9 @@ open the PR there. That PR **must stay a single-parent squash** — the publish
posts to `#alerts-build` when that gate fails on a PR opened *from* `bundle/2.x` or
`bundle/1.x` (link only, no PR title, since the branch is embargoed).
`sec-sync-bundle-branches.yml` keeps those branches current after every public-to-private sync,
daily, whenever a PR is merged into one, and on `workflow_dispatch`. It **merges the public
base into** the bundle branch via
`sec-sync-bundle-branches.yml` keeps those branches current nightly at 03:00 UTC and on
`workflow_dispatch`, including before a bundle cut. It **merges the public base into** the
bundle branch via
[`scripts/sync-bundle-branch.mjs`](scripts/sync-bundle-branch.mjs) and pushes without forcing.
Every push is verified to carry exactly the tree a merge of the two sides would produce (`git
merge-tree`); a mismatch, or a conflict marker, fails the run instead of pushing.
@ -1165,7 +1156,8 @@ private-only until publication. The mirror can replace that commit with the publ
the bundle branch is deleted. The next bundle branch must not inherit the replaced commit.
Deleting a bundle branch takes its open PRs with it: GitHub moves each one onto the deleted
branch's own base, and re-creating the branch does not move them back. So the sync then calls
branch's own base, and re-creating the branch does not move them back. The next nightly or manual
sync re-creates the branch and then calls
[`sec-sync-retarget-prs.yml`](workflows/sec-sync-retarget-prs.yml), which moves every open PR
on `master` onto `bundle/2.x` and every one on `1.x` onto `bundle/1.x`. It skips a bundle
branch that does not exist, and skips PRs whose *head* is `bundle/*` — those are the
@ -1265,12 +1257,6 @@ Adding a new channel requires inviting the bot first; the first run otherwise fa
## Future Vision
### Redundancy Review
Comment trigger (`/test-workflows`) is a workaround.
Long-term: Main CI should be reliable enough to not need these.
### Workflow Testability
- Tools like `act` for local testing

View File

@ -58,6 +58,15 @@ describe('matchGlob', () => {
);
});
it('backend module pattern matches packaged module files', () => {
assert.ok(
matchGlob(
'packages/modules/insights/backend/src/insights.service.ts',
'packages/modules/*/backend/**',
),
);
});
it('* matches single-level only', () => {
assert.ok(matchGlob('README.md', '*.md'));
assert.ok(!matchGlob('docs/README.md', '*.md'));

View File

@ -1,8 +1,15 @@
## Summary
<!--
Describe what the PR does.
Photos and videos are recommended.
Start with 1 to 3 sentences of background for a new reader. Then describe what the PR does, in prose, not as a list of edits.
Add photos or videos for UI changes.
-->
## Why
<!--
State the problem or the constraint that makes the current code unfit. Do not repeat the Summary.
Add a rejected alternative only when a reviewer would ask about it, in one or two sentences.
-->
## How to test

View File

@ -747,8 +747,8 @@ packages:
fast-deep-equal@3.1.3:
resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==}
fast-uri@3.1.5:
resolution: {integrity: sha512-gHwA1O9LDIcKunMKhObS/HimwtehO1nPUECKAu5TpKgaO19fcWEl4bliWe1jWxVFvIXztJjjQ4L8XQ1EU9f7Jw==}
fast-uri@3.1.8:
resolution: {integrity: sha512-GZMtZUTNRpOVIECoXwLNZS5xUGE+mVNbTB8h/7Rwh2TFWcBQiPzTgyZi05BF9UMZKkLJv8XBRJTlU7zg8+ZfMg==}
fd-package-json@2.0.0:
resolution: {integrity: sha512-jKmm9YtsNXN789RS/0mSzOC1NUq9mkVd65vbSSVsKdjGvYXBuE4oWe2QOEoFeRmJg+lPuZxpmrfFclNhoRMneQ==}
@ -1694,7 +1694,7 @@ snapshots:
ajv@8.20.0:
dependencies:
fast-deep-equal: 3.1.3
fast-uri: 3.1.5
fast-uri: 3.1.8
json-schema-traverse: 1.0.0
require-from-string: 2.0.2
@ -1994,7 +1994,7 @@ snapshots:
fast-deep-equal@3.1.3: {}
fast-uri@3.1.5: {}
fast-uri@3.1.8: {}
fd-package-json@2.0.0:
dependencies:

68
.github/scripts/split-jscpd-report.mjs vendored Normal file
View File

@ -0,0 +1,68 @@
#!/usr/bin/env node
// Splits a jscpd JSON report's `duplicates` array into N bucket files so a
// caller can download and sample one small file instead of the whole report
// (which can run into tens of MB on a large monorepo). Round-robin, not
// contiguous chunks, so a run of similarly-sized clusters in scan order
// doesn't pile up in one bucket.
import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
import { join } from 'node:path';
export function splitDuplicates(duplicates, bucketCount) {
const buckets = Array.from({ length: bucketCount }, () => []);
duplicates.forEach((dup, i) => buckets[i % bucketCount].push(dup));
return buckets;
}
export function writeBuckets(buckets, outputDir, deps = {}) {
const mkdirImpl = deps.mkdir ?? mkdirSync;
const writeImpl = deps.write ?? writeFileSync;
mkdirImpl(outputDir, { recursive: true });
return buckets.map((duplicates, i) => {
const path = join(outputDir, `bucket-${i}.json`);
writeImpl(path, JSON.stringify({ duplicates }));
return path;
});
}
export function parseBucketCount(raw) {
const count = Number(raw ?? 20);
if (!Number.isInteger(count) || count <= 0) {
throw new Error(`BUCKET_COUNT must be a positive integer, got '${raw}'`);
}
return count;
}
export function loadDuplicates(reportPath, deps = {}) {
const readImpl = deps.read ?? readFileSync;
const report = JSON.parse(readImpl(reportPath, 'utf8'));
if (!Array.isArray(report.duplicates)) {
throw new Error('report.duplicates must be an array');
}
return report.duplicates;
}
// Exits non-zero on any failure. The workflow step already runs with
// continue-on-error, so this can't fail the job — it only stops a broken
// split (zero buckets written) from reporting success.
function main() {
const reportPath = process.env.REPORT_PATH;
const outputDir = process.env.OUTPUT_DIR;
if (!reportPath || !outputDir) {
process.stderr.write('split-jscpd-report: REPORT_PATH and OUTPUT_DIR are required\n');
process.exit(1);
}
try {
const bucketCount = parseBucketCount(process.env.BUCKET_COUNT);
const duplicates = loadDuplicates(reportPath);
const buckets = splitDuplicates(duplicates, bucketCount);
writeBuckets(buckets, outputDir);
} catch (err) {
process.stderr.write(`split-jscpd-report: ${err.message}\n`);
process.exit(1);
}
}
if (import.meta.url === `file://${process.argv[1]}`) {
main();
}

View File

@ -0,0 +1,111 @@
/**
* Run these tests by running
*
* node --test .github/scripts/split-jscpd-report.test.mjs
* */
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import { splitDuplicates, writeBuckets, parseBucketCount, loadDuplicates } from './split-jscpd-report.mjs';
describe('splitDuplicates', () => {
it('distributes entries round-robin across buckets', () => {
const duplicates = [1, 2, 3, 4, 5, 6];
const buckets = splitDuplicates(duplicates, 3);
assert.deepEqual(buckets, [
[1, 4],
[2, 5],
[3, 6],
]);
});
it('leaves later buckets empty when there are fewer entries than buckets', () => {
const buckets = splitDuplicates([1, 2], 5);
assert.equal(buckets.length, 5);
assert.deepEqual(buckets[0], [1]);
assert.deepEqual(buckets[1], [2]);
assert.deepEqual(buckets[2], []);
});
it('returns bucketCount empty arrays for an empty report', () => {
const buckets = splitDuplicates([], 4);
assert.deepEqual(buckets, [[], [], [], []]);
});
});
describe('writeBuckets', () => {
it('writes one JSON file per bucket, named by index', () => {
const written = [];
const paths = writeBuckets([[{ a: 1 }], [{ b: 2 }]], '/tmp/buckets', {
mkdir: () => {},
write: (path, contents) => written.push({ path, contents }),
});
assert.deepEqual(paths, ['/tmp/buckets/bucket-0.json', '/tmp/buckets/bucket-1.json']);
assert.deepEqual(JSON.parse(written[0].contents), { duplicates: [{ a: 1 }] });
assert.deepEqual(JSON.parse(written[1].contents), { duplicates: [{ b: 2 }] });
});
it('creates the output directory before writing', () => {
let mkdirCalledWith = null;
writeBuckets([[]], '/tmp/buckets', {
mkdir: (dir, opts) => {
mkdirCalledWith = { dir, opts };
},
write: () => {},
});
assert.deepEqual(mkdirCalledWith, { dir: '/tmp/buckets', opts: { recursive: true } });
});
});
describe('parseBucketCount', () => {
it('parses a numeric string', () => {
assert.equal(parseBucketCount('20'), 20);
});
it('defaults to 20 when unset', () => {
assert.equal(parseBucketCount(undefined), 20);
});
it('rejects zero', () => {
assert.throws(() => parseBucketCount('0'), /positive integer/);
});
it('rejects a negative count', () => {
assert.throws(() => parseBucketCount('-1'), /positive integer/);
});
it('rejects a non-integer', () => {
assert.throws(() => parseBucketCount('1.5'), /positive integer/);
});
it('rejects a non-numeric value', () => {
assert.throws(() => parseBucketCount('abc'), /positive integer/);
});
});
describe('loadDuplicates', () => {
it('returns the duplicates array from a valid report', () => {
const duplicates = loadDuplicates('/tmp/report.json', {
read: () => JSON.stringify({ duplicates: [{ a: 1 }] }),
});
assert.deepEqual(duplicates, [{ a: 1 }]);
});
it('rejects a report missing the duplicates array', () => {
assert.throws(
() => loadDuplicates('/tmp/report.json', { read: () => JSON.stringify({}) }),
/duplicates must be an array/,
);
});
it('rejects a report whose duplicates field is not an array', () => {
assert.throws(
() => loadDuplicates('/tmp/report.json', { read: () => JSON.stringify({ duplicates: 'x' }) }),
/duplicates must be an array/,
);
});
it('propagates a JSON parse failure', () => {
assert.throws(() => loadDuplicates('/tmp/report.json', { read: () => 'not json' }), SyntaxError);
});
});

File diff suppressed because one or more lines are too long

View File

@ -32,11 +32,11 @@ jobs:
# when updating.
include:
- node_version: '22'
dhi_ref: dhi.io/node:22.23.2-alpine3.24-dev@sha256:155a95d59244a71e9479e191ff718f03a7197d6829612e0707608be92988cb87
- node_version: '24.19.0'
dhi_ref: dhi.io/node:24.19.0-alpine3.24-dev@sha256:dc2989ad23938772abaf549ed7bde5d61d70377005229bb55bef94bff40f8bbd
- node_version: '26.7.0'
dhi_ref: dhi.io/node:26.7.0-alpine3.24-dev@sha256:4b494d89fb26c950ce97865acf45b480dc7a6868fdc2b81c2d66599702eeac3f
dhi_ref: dhi.io/node:22.23.3-alpine3.24-dev@sha256:23ee07ace431ea7d14e22c0e52fb52b0bd1853dabdacd89ba75f937f6dff8a8d
- node_version: '24.21.0'
dhi_ref: dhi.io/node:24.21.0-alpine3.24-dev@sha256:d22ba0e9ab2a54609ae533df4ebfd6f36c1f233b662a4c547f5a3195e8c21937
- node_version: '26.10.0'
dhi_ref: dhi.io/node:26.10.0-alpine3.24-dev@sha256:8da859df5dc853c923ace53c4cfe994a909a28917f7fcfc8508869beab8cf132
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

View File

@ -25,7 +25,7 @@ on:
type: boolean
env:
NODE_VERSION: '26.7.0'
NODE_VERSION: '26.10.0'
jobs:
build:

View File

@ -147,6 +147,8 @@ jobs:
DOCKER_BASE: ${{ secrets.DOCKER_USERNAME }}
N8N_IMAGE: ${{ needs.build.outputs.n8n_image }}
N8N_DIGEST: ${{ needs.build.outputs.n8n_digest }}
N8N_PC_IMAGE: ${{ needs.build.outputs.n8n_pc_image }}
N8N_PC_DIGEST: ${{ needs.build.outputs.n8n_pc_digest }}
RUNNERS_IMAGE: ${{ needs.build.outputs.runners_image }}
RUNNERS_DIGEST: ${{ needs.build.outputs.runners_digest }}
DISTROLESS_IMAGE: ${{ needs.build.outputs.runners_distroless_image }}
@ -195,5 +197,6 @@ jobs:
} >> "$GITHUB_STEP_SUMMARY"
promote "$N8N_IMAGE" "$N8N_DIGEST" "${DOCKER_BASE}/n8n" ""
promote "$N8N_PC_IMAGE" "$N8N_PC_DIGEST" "${DOCKER_BASE}/n8n" "-pc"
promote "$RUNNERS_IMAGE" "$RUNNERS_DIGEST" "${DOCKER_BASE}/runners" ""
promote "$DISTROLESS_IMAGE" "$DISTROLESS_DIGEST" "${DOCKER_BASE}/runners" "-distroless"

View File

@ -80,6 +80,7 @@ jobs:
!**/*.test.ts
!**/*.spec.ts
!packages/quality/testing/playwright/**
!packages/quality/efficiency/**
!packages/frontend/@n8n/storybook/**
!.devcontainer/**
!scripts/agent-setup.mjs
@ -146,10 +147,17 @@ jobs:
cubic.yaml
.agents/review-rules/**
e2e-performance:
packages/quality/efficiency/memory/idle/**
packages/quality/efficiency/playwright.config.ts
packages/quality/efficiency/package.json
packages/quality/testing/playwright/package.json
packages/quality/testing/playwright/quality-*.ts
packages/quality/testing/playwright/fixtures/**
packages/quality/testing/playwright/tests/performance/**
packages/quality/testing/playwright/utils/performance-helper.ts
packages/quality/environments/containers/**
.github/workflows/test-e2e-performance-reusable.yml
.github/workflows/test-e2e-reusable.yml
# Whole directories, not the `engine-*` files alone. Engine v2
# routing lives in ordinary control plane code: `workflow-runner.ts`
# and the webhook and trigger paths decide whether a run reaches the
@ -178,6 +186,7 @@ jobs:
packages/core/src/execution-engine/eval-mock-helpers.ts
.github/workflows/test-evals-discovery.yml
db:
packages/modules/*/backend/**
packages/cli/src/databases/**
packages/cli/src/modules/*/database/**
packages/cli/src/modules/**/*.entity.ts
@ -185,16 +194,17 @@ jobs:
packages/cli/test/integration/**
packages/cli/test/migration/**
packages/cli/test/shared/db/**
packages/cli/test/setup-testcontainers.js
packages/cli/test/teardown-testcontainers.js
packages/cli/test/setup-testcontainers.ts
packages/@n8n/db/**
packages/@n8n/backend-test-utils/**
packages/@n8n/scheduler/src/**
packages/cli/src/scheduling/**
packages/cli/src/**/*.integration.test.ts
packages/cli/src/modules/instance-registry/**
packages/cli/vitest.*
packages/quality/environments/containers/services/postgres.ts
packages/quality/environments/containers/services/redis.ts
packages/quality/environments/containers/test-db-global-setup.ts
packages/quality/environments/containers/postgres-versions.json
packages/quality/environments/containers/test-containers.ts
.github/workflows/test-db-reusable.yml
@ -209,8 +219,7 @@ jobs:
packages/@n8n/backend-test-utils/**
packages/cli/test/migration/**
packages/cli/test/shared/db/**
packages/cli/test/setup-testcontainers.js
packages/cli/test/teardown-testcontainers.js
packages/cli/test/setup-testcontainers.ts
packages/cli/vitest.config.base.ts
packages/cli/vitest.config.migration.ts
packages/cli/vitest.config.migration.testcontainers.ts

View File

@ -8,7 +8,7 @@ run-name: "${{ github.event_name == 'workflow_dispatch' && format('Docker: Build
env:
NODE_OPTIONS: '--max-old-space-size=7168'
NODE_VERSION: '26.7.0'
NODE_VERSION: '26.10.0'
on:
schedule:
@ -55,6 +55,12 @@ on:
n8n_digest:
description: 'Multi-arch manifest digest of the n8n image'
value: ${{ jobs.create_multi_arch_manifest.outputs.n8n_digest }}
n8n_pc_image:
description: 'GHCR repo for the pointer-compressed n8n image (no tag; same repo as n8n)'
value: ${{ jobs.create_multi_arch_manifest.outputs.n8n_pc_image }}
n8n_pc_digest:
description: 'Multi-arch manifest digest of the pointer-compressed n8n image'
value: ${{ jobs.create_multi_arch_manifest.outputs.n8n_pc_digest }}
runners_image:
description: 'GHCR repo for the runners image (no tag)'
value: ${{ jobs.create_multi_arch_manifest.outputs.runners_image }}

View File

@ -12,7 +12,7 @@ env:
# The images copy native bindings from the host build, so both have to use the same
# Node. dockerize-n8n.mjs reads this env for the image builds. Keep in sync with
# NODE_VERSION in docker-build-push.yml, which is what the published images use.
NODE_VERSION: '26.7.0'
NODE_VERSION: '26.10.0'
# Also build the distroless runners image and exec-check
# both runners images' interpreters, so a broken runtime assembly (e.g. a
# missing shared library after a Node bump) fails at PR time. The smoke

View File

@ -19,7 +19,7 @@ on:
default: false
env:
NODE_VERSION: '26.7.0'
NODE_VERSION: '26.10.0'
SCRATCH_TAG: sbom-probe-${{ github.run_id }}
# A separate package. These are throwaway fixtures (one is a two-layer
# alpine), and ghcr.io/<repo> is where the real release tags live - they

309
.github/workflows/jscpd.yml vendored Normal file
View File

@ -0,0 +1,309 @@
# 🍝 jscpd — deterministic copy-paste/duplication scan.
#
# Runs jscpd against this repo and uploads the full JSON+HTML report, plus
# the report split into BUCKET_COUNT small JSON files, so a caller can
# download one small bucket at random per invocation instead of the full
# report (which can run into tens of MB on a monorepo this size).
#
# Reporters/ignores/threshold come from this repo's own .jscpd.json; this
# workflow passes none of those flags itself.
name: 🍝 jscpd
on:
workflow_dispatch:
permissions:
contents: read
env:
# Number of small sample files the report is split into (split-jscpd-report.mjs).
# Fixed, not an input: changing it means adding/removing the matching
# per-bucket upload steps below, so it's a code change, not a dispatch option.
BUCKET_COUNT: '20'
jobs:
jscpd:
runs-on: ${{ vars.RUNNER_PROVIDER == 'github' && 'ubuntu-latest' || 'blacksmith-4vcpu-ubuntu-2204' }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 1
persist-credentials: false
- name: Setup Node.js and install dependencies
uses: ./.github/actions/setup-nodejs
with:
build-command: ''
- name: Run jscpd
id: jscpd
continue-on-error: true
run: |
mkdir -p "$RUNNER_TEMP/jscpd-report"
pnpm jscpd -- . --output "$RUNNER_TEMP/jscpd-report" --silent
- name: Upload jscpd report
id: report-artifact
if: always()
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-report-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/jscpd-report
retention-days: 31
if-no-files-found: warn
# jscpd exits non-zero once duplication crosses .jscpd.json's threshold,
# but still writes the report first. Gating the split on the report
# file existing (not on steps.jscpd.outcome) means an over-threshold
# run still gets its buckets instead of silently dropping all of them.
- name: Check jscpd report exists
id: report-exists
if: always()
run: |
if [ -f "$RUNNER_TEMP/jscpd-report/jscpd-report.json" ]; then
echo "exists=true" >> "$GITHUB_OUTPUT"
else
echo "exists=false" >> "$GITHUB_OUTPUT"
fi
# Splits `duplicates` into BUCKET_COUNT files so a caller can download
# one small sample instead of the full report (tens of MB on a large
# monorepo). Logic + tests live in split-jscpd-report.mjs.
- name: Split report into buckets
id: split
if: always() && steps.report-exists.outputs.exists == 'true'
continue-on-error: true
env:
REPORT_PATH: ${{ runner.temp }}/jscpd-report/jscpd-report.json
OUTPUT_DIR: ${{ runner.temp }}/jscpd-buckets
run: node .github/scripts/split-jscpd-report.mjs
# BUCKET_COUNT static upload steps, one per bucket file: GitHub Actions
# has no per-job step loop, and each bucket needs its own small
# artifact (not one artifact holding all of them) so a caller can
# fetch a single sample without pulling the rest. Keep this block's
# step count in sync with BUCKET_COUNT above.
- name: Upload jscpd bucket 0
id: bucket-artifact-0
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-0
path: ${{ runner.temp }}/jscpd-buckets/bucket-0.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 1
id: bucket-artifact-1
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-1
path: ${{ runner.temp }}/jscpd-buckets/bucket-1.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 2
id: bucket-artifact-2
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-2
path: ${{ runner.temp }}/jscpd-buckets/bucket-2.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 3
id: bucket-artifact-3
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-3
path: ${{ runner.temp }}/jscpd-buckets/bucket-3.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 4
id: bucket-artifact-4
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-4
path: ${{ runner.temp }}/jscpd-buckets/bucket-4.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 5
id: bucket-artifact-5
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-5
path: ${{ runner.temp }}/jscpd-buckets/bucket-5.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 6
id: bucket-artifact-6
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-6
path: ${{ runner.temp }}/jscpd-buckets/bucket-6.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 7
id: bucket-artifact-7
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-7
path: ${{ runner.temp }}/jscpd-buckets/bucket-7.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 8
id: bucket-artifact-8
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-8
path: ${{ runner.temp }}/jscpd-buckets/bucket-8.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 9
id: bucket-artifact-9
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-9
path: ${{ runner.temp }}/jscpd-buckets/bucket-9.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 10
id: bucket-artifact-10
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-10
path: ${{ runner.temp }}/jscpd-buckets/bucket-10.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 11
id: bucket-artifact-11
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-11
path: ${{ runner.temp }}/jscpd-buckets/bucket-11.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 12
id: bucket-artifact-12
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-12
path: ${{ runner.temp }}/jscpd-buckets/bucket-12.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 13
id: bucket-artifact-13
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-13
path: ${{ runner.temp }}/jscpd-buckets/bucket-13.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 14
id: bucket-artifact-14
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-14
path: ${{ runner.temp }}/jscpd-buckets/bucket-14.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 15
id: bucket-artifact-15
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-15
path: ${{ runner.temp }}/jscpd-buckets/bucket-15.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 16
id: bucket-artifact-16
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-16
path: ${{ runner.temp }}/jscpd-buckets/bucket-16.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 17
id: bucket-artifact-17
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-17
path: ${{ runner.temp }}/jscpd-buckets/bucket-17.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 18
id: bucket-artifact-18
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-18
path: ${{ runner.temp }}/jscpd-buckets/bucket-18.json
retention-days: 31
if-no-files-found: error
- name: Upload jscpd bucket 19
id: bucket-artifact-19
if: always() && steps.split.outcome == 'success'
continue-on-error: true
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: jscpd-bucket-${{ github.run_id }}-${{ github.run_attempt }}-19
path: ${{ runner.temp }}/jscpd-buckets/bucket-19.json
retention-days: 31
if-no-files-found: error

View File

@ -9,6 +9,16 @@ name: 'Prepare n8n Docker (reusable)'
on:
workflow_call:
secrets:
QA_METRICS_WEBHOOK_URL:
description: 'QA metrics webhook URL.'
required: false
QA_METRICS_WEBHOOK_USER:
description: 'QA metrics webhook username.'
required: false
QA_METRICS_WEBHOOK_PASSWORD:
description: 'QA metrics webhook password.'
required: false
inputs:
build-variant:
description: 'standard, coverage, or pc'

View File

@ -12,8 +12,8 @@
# Every push is verified to carry exactly the tree a merge would produce. See
# sync-bundle-branch.mjs.
#
# The public-to-private mirror dispatches this workflow after each run. The daily schedule is a
# fallback, and merged fixes trigger it so the cut PR sees the latest bundle state immediately.
# Runs nightly at 03:00 UTC. Maintainers can also dispatch it before a bundle cut or when a
# bundle branch needs to be synced sooner.
#
# One job per bundle branch: a conflict on one FAILS THAT JOB and leaves its branch untouched,
# while the other still syncs. Recovery is manual and deliberate — merge the public base locally,
@ -25,11 +25,6 @@ name: 'Security: Sync Bundle Branches'
on:
schedule:
- cron: '0 3 * * *'
pull_request: # a fix landing on a bundle branch is when freshness matters most
types: [closed]
branches:
- 'bundle/2.x'
- 'bundle/1.x'
workflow_dispatch:
# Least privilege by default; the sync job opts into exactly what it needs.
@ -38,9 +33,7 @@ permissions: {}
jobs:
sync:
name: Sync ${{ matrix.bundle }}
if: |
github.repository == 'n8n-io/n8n-private' &&
(github.event_name != 'pull_request' || github.event.pull_request.merged == true)
if: github.repository == 'n8n-io/n8n-private'
runs-on: ubuntu-latest
permissions:
contents: write
@ -68,17 +61,12 @@ jobs:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
# Always run the script from the default branch, never from a pull_request merge
# ref — the sync holds a contents:write token and fetches what it needs itself,
# so the checked-out branch is irrelevant beyond supplying trusted code.
# Always run the script from the default branch. The sync fetches both branches itself.
ref: master
fetch-depth: 0
persist-credentials: false # we push with an explicit token URL instead
- name: Merge the base into the bundle branch
# On a merged bundle PR, only the branch that received it needs syncing.
# (`matrix` is not available in a job-level `if`, so the filter lives here.)
if: github.event_name != 'pull_request' || github.event.pull_request.base.ref == matrix.bundle
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
BUNDLE_BRANCH: ${{ matrix.bundle }}

View File

@ -7,9 +7,6 @@
# Manual runs always sync (for conflict recovery after failed cherry-pick).
#
# A skipped branch or a hard failure is reported to #alerts-build.
#
# This workflow dispatches the bundle sync after each mirror run. Bundle branches merge the
# public branch directly, so they never inherit a private-only bundle cut.
name: 'Security: Sync from Public'
run-name: "${{ github.event_name == 'workflow_dispatch' && format('Security: Sync from Public (force={0})', inputs.force) || '' }}"
@ -30,7 +27,6 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
actions: write
steps:
- name: Generate App Token
@ -103,13 +99,6 @@ jobs:
git reset --hard public-1.x
git push origin 1.x --force-with-lease
- name: Dispatch bundle branch sync
if: ${{ !cancelled() }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh workflow run sec-sync-bundle-branches.yml --ref master
- name: Notify Slack on failure
if: failure()
env:

View File

@ -55,6 +55,13 @@ jobs:
working-directory: packages/cli
run: ${{ matrix.test-cmd }}
# A packaged backend module opts in by defining `test:postgres`.
- name: Run packaged backend module PostgreSQL tests
if: matrix.TEST_IMAGE_POSTGRES != ''
env:
COVERAGE_ENABLED: 'false'
run: pnpm --filter './packages/modules/*/backend' --if-present run test:postgres --maxWorkers=100%
# Regenerates the schema docs for this leg's database (sqlite temp file /
# postgres testcontainer) and fails if they differ from what's committed.
# tbls runs as a Docker image in CI.

View File

@ -1,5 +1,8 @@
name: 'Test: E2E Performance'
permissions:
contents: read
on:
workflow_call:
workflow_dispatch:
@ -24,6 +27,17 @@ jobs:
artifact-prefix: performance
secrets: inherit
idle-memory-baselines:
needs: [prepare-docker]
uses: ./.github/workflows/test-e2e-reusable.yml
with:
test-mode: docker-artifact
test-command: pnpm --filter=@n8n/efficiency test:memory
workers: '1'
currents-project-id: 'O9BJaN'
artifact-prefix: memory-performance
secrets: inherit
canvas-perf-sentinels:
name: Canvas Perf Sentinels
# Sentinels are nightly only — wall-clock browser perf has 15-30% variance,

View File

@ -195,9 +195,10 @@ jobs:
with:
name: ${{ inputs.artifact-prefix }}-shard-${{ matrix.shard }}
path: |
packages/quality/testing/playwright/test-results/
packages/quality/testing/playwright/test-results.json
packages/quality/testing/playwright/playwright-report/
packages/quality/*/test-results*
packages/quality/*/playwright-report/
packages/quality/*/*/test-results*
packages/quality/*/*/playwright-report/
packages/quality/testing/playwright/coverage/lcov.info
packages/quality/testing/playwright/coverage/by-spec/
packages/quality/testing/playwright/coverage/index.html

View File

@ -346,7 +346,7 @@ jobs:
# ANTHROPIC_API_KEY is the provider-native key resolveEvalModelConfig prefers
# when N8N_INSTANCE_AI_EVAL_MODEL differs from N8N_INSTANCE_AI_MODEL.
MODEL_ARGS+=(
-e N8N_INSTANCE_AI_EVAL_MODEL=anthropic/claude-sonnet-4-6
-e N8N_INSTANCE_AI_EVAL_MODEL=anthropic/claude-sonnet-5-5
-e ANTHROPIC_API_KEY="$EVALS_ANTHROPIC_KEY"
)

View File

@ -0,0 +1,71 @@
name: 'Test: Image Efficiency'
on:
workflow_dispatch:
permissions:
contents: read
jobs:
prepare-docker:
uses: ./.github/workflows/prepare-docker-reusable.yml
secrets:
QA_METRICS_WEBHOOK_URL: ${{ secrets.QA_METRICS_WEBHOOK_URL }}
QA_METRICS_WEBHOOK_USER: ${{ secrets.QA_METRICS_WEBHOOK_USER }}
QA_METRICS_WEBHOOK_PASSWORD: ${{ secrets.QA_METRICS_WEBHOOK_PASSWORD }}
image-efficiency:
name: Image Efficiency
needs: prepare-docker
runs-on: ${{ vars.RUNNER_PROVIDER == 'github' && 'ubuntu-latest' || 'blacksmith-2vcpu-ubuntu-2204' }}
timeout-minutes: 20
env:
CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }}
CURRENTS_PROJECT_ID: 'nHHLA5'
CURRENTS_CI_BUILD_ID: ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }}-images
QA_METRICS_WEBHOOK_URL: ${{ secrets.QA_METRICS_WEBHOOK_URL }}
QA_METRICS_WEBHOOK_USER: ${{ secrets.QA_METRICS_WEBHOOK_USER }}
QA_METRICS_WEBHOOK_PASSWORD: ${{ secrets.QA_METRICS_WEBHOOK_PASSWORD }}
steps:
- name: Checkout
uses: useblacksmith/checkout@bcec731f1eb1367240608d1c889a75b96db6ec53 # v1.5.0
- name: Setup Environment
uses: ./.github/actions/setup-nodejs
with:
build-command: pnpm turbo run build --filter=@n8n/utils...
- name: Load images from cache
uses: ./.github/actions/load-n8n-docker
- name: Install Dive
shell: bash
run: |
set -euo pipefail
# Keep this version in sync with efficiency/images/dive-results.ts.
VERSION=0.13.1
DIR=$(mktemp -d "$RUNNER_TEMP/dive.XXXXXX")
ARCHIVE="$DIR/dive.tar.gz"
curl --fail --location --retry 3 \
"https://github.com/wagoodman/dive/releases/download/v${VERSION}/dive_${VERSION}_linux_amd64.tar.gz" \
--output "$ARCHIVE"
echo "0970549eb4a306f8825a84145a2534153badb4d7dcf3febd1967c706367c3d0e $ARCHIVE" | sha256sum --check
tar -xzf "$ARCHIVE" -C "$DIR" dive
echo "$DIR" >> "$GITHUB_PATH"
- name: Measure image layer efficiency
run: >-
pnpm --filter=@n8n/efficiency exec playwright test
--project=image-efficiency:n8n --project=image-efficiency:runners
- name: Upload image efficiency reports
if: always()
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: image-efficiency-${{ github.run_attempt }}
path: |
packages/quality/efficiency/test-results/
packages/quality/efficiency/test-results.json
packages/quality/efficiency/playwright-report/
retention-days: 7
if-no-files-found: ignore

View File

@ -1,110 +0,0 @@
name: 'Test: Workflows PR Comment'
on:
issue_comment:
types: [created]
permissions:
pull-requests: read
contents: read
jobs:
handle_comment_command:
name: Handle /test-workflows Command
if: github.event.issue.pull_request && startsWith(github.event.comment.body, '/test-workflows')
runs-on: ubuntu-latest
outputs:
permission_granted: ${{ steps.pr_check_and_details.outputs.permission_granted }}
git_ref: ${{ steps.pr_check_and_details.outputs.head_sha }}
pr_number: ${{ steps.pr_check_and_details.outputs.pr_number_string }}
steps:
- name: Validate User, Get PR Details, and React
id: pr_check_and_details
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const commenter = context.actor;
const issueOwner = context.repo.owner;
const issueRepo = context.repo.repo;
const commentId = context.payload.comment.id;
const prNumber = context.issue.number; // In issue_comment on a PR, issue.number is the PR number
// Function to add a reaction to the comment
async function addReaction(content) {
try {
await github.rest.reactions.createForIssueComment({
owner: issueOwner,
repo: issueRepo,
comment_id: commentId,
content: content
});
} catch (reactionError) {
// Log if reaction fails but don't fail the script for this
console.log(`Failed to add reaction '${content}': ${reactionError.message}`);
}
}
// Initialize outputs to a non-triggering state
core.setOutput('permission_granted', 'false');
core.setOutput('head_sha', '');
core.setOutput('pr_number_string', '');
// 1. Check user permissions
try {
const { data: permissions } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: issueOwner,
repo: issueRepo,
username: commenter
});
const allowedPermissions = ['admin', 'write', 'maintain'];
if (!allowedPermissions.includes(permissions.permission)) {
console.log(`User @${commenter} has '${permissions.permission}' permission. Needs 'admin', 'write', or 'maintain'.`);
await addReaction('-1'); // User does not have permission
return; // Exit script, tests will not be triggered
}
console.log(`User @${commenter} has '${permissions.permission}' permission.`);
} catch (error) {
console.log(`Could not verify permissions for @${commenter}: ${error.message}`);
await addReaction('confused'); // Error checking permissions
return; // Exit script
}
// 2. Fetch PR details (if permission check passed)
let headSha;
try {
const { data: pr } = await github.rest.pulls.get({
owner: issueOwner,
repo: issueRepo,
pull_number: prNumber,
});
headSha = pr.head.sha;
console.log(`Workspaced PR details: SHA - ${headSha}, PR Number - ${prNumber}`);
// Set outputs for the next job
core.setOutput('permission_granted', 'true');
core.setOutput('head_sha', headSha);
core.setOutput('pr_number_string', prNumber.toString());
await addReaction('+1'); // Command accepted, tests will be triggered
} catch (error) {
console.log(`Failed to fetch PR details for PR #${prNumber}: ${error.message}`);
core.setOutput('permission_granted', 'false'); // Ensure this is false if PR fetch fails
await addReaction('confused'); // Error fetching PR details
}
trigger_reusable_tests:
name: Trigger Reusable Test Workflow
needs: handle_comment_command
if: >
always() &&
needs.handle_comment_command.result != 'skipped' &&
needs.handle_comment_command.outputs.permission_granted == 'true' &&
needs.handle_comment_command.outputs.git_ref != ''
uses: ./.github/workflows/test-workflows-callable.yml
with:
git_ref: ${{ needs.handle_comment_command.outputs.git_ref }}
secrets: inherit

View File

@ -0,0 +1,118 @@
name: 'Util: Update get-n8n Versions'
on:
schedule:
# Run every Wednesday at 04:00 UTC.
- cron: '0 4 * * 3'
workflow_dispatch:
permissions: {}
concurrency:
group: update-get-n8n-versions
cancel-in-progress: false
jobs:
update-versions:
name: Update get-n8n versions
if: >-
github.ref_name == github.event.repository.default_branch &&
(github.event_name == 'workflow_dispatch' ||
(github.event_name == 'schedule' && github.repository == 'n8n-io/n8n'))
runs-on: ubuntu-latest
timeout-minutes: 25
permissions:
contents: read
steps:
- name: Generate GitHub App Token
id: generate-token
uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1
with:
app-id: ${{ secrets.N8N_ASSISTANT_APP_ID }}
private-key: ${{ secrets.N8N_ASSISTANT_PRIVATE_KEY }}
permission-contents: write
permission-pull-requests: write
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Resolve stable versions
id: versions
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
run: |
n8n_version="$(gh api repos/n8n-io/n8n/releases/latest --jq '.tag_name | sub("^n8n@"; "")')"
registry_token="$(curl -fsSL --retry 3 'https://ghcr.io/token?scope=repository:n8n-io/n8n-sandbox-service-api:pull' | jq -er '.token')"
stable_digest="$(docker buildx imagetools inspect \
'ghcr.io/n8n-io/n8n-sandbox-service-api:stable' \
--format '{{json .Manifest}}' | jq -er '.digest')"
sandbox_version=''
while IFS= read -r version; do
version_digest="$(docker buildx imagetools inspect \
"ghcr.io/n8n-io/n8n-sandbox-service-api:${version}" \
--format '{{json .Manifest}}' | jq -er '.digest')"
if [ "$version_digest" = "$stable_digest" ]; then
sandbox_version="$version"
break
fi
done < <(curl -fsSL --retry 3 \
-H "Authorization: Bearer $registry_token" \
'https://ghcr.io/v2/n8n-io/n8n-sandbox-service-api/tags/list?n=1000' |
jq -r '.tags[]' |
grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' |
sort -Vr)
printf '%s\n' "$n8n_version" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$'
printf '%s\n' "$sandbox_version" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+$'
for image in api runner-dind sandbox; do
docker manifest inspect "ghcr.io/n8n-io/n8n-sandbox-service-${image}:${sandbox_version}" >/dev/null
done
sed -i "s/^FALLBACK_N8N_VERSION=.*/FALLBACK_N8N_VERSION=\"${n8n_version}\"/" docker/get-n8n.sh
sed -i "s/^SANDBOX_VERSION=.*/SANDBOX_VERSION=\"${sandbox_version}\"/" docker/get-n8n.sh
echo "n8n_version=$n8n_version" >> "$GITHUB_OUTPUT"
echo "sandbox_version=$sandbox_version" >> "$GITHUB_OUTPUT"
- name: Check for changes
id: changes
run: |
if git diff --quiet -- docker/get-n8n.sh; then
echo 'has-changes=false' >> "$GITHUB_OUTPUT"
else
echo 'has-changes=true' >> "$GITHUB_OUTPUT"
fi
- name: Test installer upgrade
if: steps.changes.outputs.has-changes == 'true'
run: |
shellcheck docker/get-n8n.sh docker/test-get-n8n.sh
dash -n docker/get-n8n.sh
dash -n docker/test-get-n8n.sh
sh docker/test-get-n8n.sh --e2e
- name: Create pull request
if: steps.changes.outputs.has-changes == 'true'
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0 # v8.1.0
with:
token: ${{ steps.generate-token.outputs.token }}
add-paths: docker/get-n8n.sh
commit-message: 'chore: Update get-n8n fallback versions'
labels: 'automation:scheduled-update'
title: 'chore: Update get-n8n fallback versions'
body: |
This automated PR updates the get-n8n fallback release to ${{ steps.versions.outputs.n8n_version }}.
It also updates the sandbox service release to ${{ steps.versions.outputs.sandbox_version }}.
The workflow verified that all sandbox images use the same release. It also upgraded a legacy installation to these versions and checked the running stack.
_Generated by the weekly get-n8n version update workflow._
branch: update-get-n8n-versions
base: master
delete-branch: true
author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
committer: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

4
.gitignore vendored
View File

@ -1,4 +1,5 @@
node_modules
.build/
.pnpm-store
.DS_Store
.tmp
@ -8,6 +9,7 @@ dist
coverage
coverage-by-spec
coverage-by-spec-backend
/report
npm-debug.log*
yarn.lock
google-generated-credentials.json
@ -15,6 +17,7 @@ _START_PACKAGE
.env
.env.local
.env.eval
.env.ai-surfaces
.vscode/*
!.claude
!.vscode/extensions.json
@ -39,6 +42,7 @@ build.log
tcr-dry.log
.agent-setup/
.agent-recordings/
.vitest/
.vitest-profile/
.pi-subagents/
sbom-source.cdx.json

26
.jscpd.json Normal file
View File

@ -0,0 +1,26 @@
{
"threshold": 10.5,
"reporters": ["json", "html"],
"gitignore": true,
"ignore": [
"**/__snapshots__/**",
"**/*.snap",
"**/fixtures/**",
"**/locales/**",
"**/i18n/**",
"**/*.generated.*",
"**/__schema__/**",
"**/cassettes/**",
"packages/@n8n/typeorm/test/**",
"docs/generated/**",
"**/test/**/*.workflow.json",
"**/tests/**/*.workflow.json",
"**/__tests__/**/*.workflow.json",
"**/test/**/workflow.*.json",
"**/tests/**/workflow.*.json",
"**/__tests__/**/workflow.*.json",
"**/test/**/workflows/**/*.json",
"**/tests/**/workflows/**/*.json",
"**/__tests__/**/workflows/**/*.json"
]
}

View File

@ -52,8 +52,8 @@ skip:
# Uses merge commit SHA from GitHub - the code has already been reviewed
# and merged, not arbitrary PR code.
- .github/workflows/test-unit-reusable.yml
# Permission-gated: only maintainers (admin/write/maintain) can trigger
# via /test-workflows comment. Verified in test-workflows-pr-comment.yml.
# Reachable only from test-workflows-nightly.yml, which is
# workflow_dispatch-only, so the ref comes from a maintainer, not a PR.
- .github/workflows/test-workflows-callable.yml
# Reusable workflow reachable only via workflow_dispatch (manual) or from
# ci-mcp-evals.yml, which is itself workflow_dispatch-only — never from a

View File

@ -14,6 +14,11 @@ frontend, and extensible node-based workflow engine.
- Write all technical text (code comments, PR descriptions, issue and ticket
descriptions, docs) in ASD-STE100 Simplified Technical English: use short
sentences, the active voice, and one instruction for each sentence
- **JSDoc on `@Env(...)` config properties is public documentation.** The
docs team exports these comments into the environment variable reference on
docs.n8n.io. Write them for a self-hosting operator and follow the
`n8n:env-var-docs` skill, which applies the n8n docs style guide and
overrides the ASD-STE100 rule above for these comments
- **Secrets on the command line:** if a developer opted into anonymous dev
metrics (`scripts/dev-metrics`), pnpm command arguments are recorded. Arguments
of secret-carrying words (`config`, `login`, `publish`, `token`) — whether a
@ -35,6 +40,10 @@ frontend, and extensible node-based workflow engine.
suggested by Linear, **unless it is a security fix** (see Security Fix
Hygiene below)
- Use mermaid diagrams in MD files when you need to visualise something
- Write an ADR only when asked. When a change makes a decision that an
engineer could challenge in 12 months, say so in one line and let the
human decide. Read [docs/adr/README.md](docs/adr/README.md) before you
write one. CI enforces its conventions
- **Developing v3 features:** land normal feature work on `master` behind an
opt-in flag; introduce breaking changes only on the `3.x` branch. See
[.github/DEVELOPING_V3.md](.github/DEVELOPING_V3.md).
@ -65,10 +74,15 @@ are namespaced under `n8n:`. Use `n8n:` prefix when invoking them (e.g.
## Essential Commands
For full-repo lint and typecheck, use `pnpm agent:lint` and
`pnpm agent:typecheck`. They save full logs and return compact results.
For routine pre-commit checks, follow [Verify changes](#verify-changes).
For focused tests, use `pnpm agent:test` or `pnpm agent:playwright`.
Run each command with `--help` for its package, script, and file options.
Pass test files and worker limits after `--`. Run each command with `--help`
for its options.
`pnpm agent:lint` and `pnpm agent:typecheck` run full-repo checks by default.
With `--filter <package>`, they still check the whole package. These wrappers
save logs and reduce console output. They do not reduce CPU or memory use.
Do not use them as routine pre-commit checks.
### Fresh checkout / agent setup
@ -215,12 +229,16 @@ const children = getChildNodes(workflow.connections, 'NodeName', 'main', 1);
### Persistence layer & the TypeORM boundary
TypeORM (`@n8n/typeorm`) must stay in the **persistence layer** — the `@n8n/db`
package or a backend module's own `database/` folder (entity/repository files).
TypeORM (`@n8n/typeorm`) must stay in the **persistence layer**. The shared
`@n8n/db` package and backend persistence adapters can import it. A module can
colocate or group its entities and repositories according to its domain.
Business logic — services, controllers, handlers, commands, factories — must not
import from `@n8n/typeorm` (including `@n8n/typeorm/...` subpaths). In
`packages/cli` this is enforced by the `misplaced-n8n-typeorm-import` lint rule;
a new import (or an inline `eslint-disable` of the rule) fails CI.
import from `@n8n/typeorm` (including `@n8n/typeorm/...` subpaths). The
`typeorm-persistence-boundary` Code Health rule recognizes entity and repository
declarations. It scans backend packages that depend on `@n8n/db` or
`@n8n/typeorm`. Tests, migrations, and explicit helper-only adapters are
exceptions. Existing business-logic imports stay in the shrink-only Code Health
baseline. A new business-logic import fails CI.
- **Pattern:** when a query needs operators (`In`, `IsNull`, `LessThan`,
`FindOptionsWhere`, …), put it behind a **use-case-named repository method**
@ -306,9 +324,20 @@ extends one of those layers):
### Verify changes
- Run focused tests from the owning package: `pnpm test <test-file>`.
- Run that package's `pnpm lint` and `pnpm typecheck` before committing code.
Build first when shared types or cross-package dependencies change.
- Before committing, run formatting and lint checks on changed files. Use
the package's configured tools through `pnpm exec` with explicit file paths.
Do not append file paths to `pnpm lint`; package scripts can still check
the whole package.
- Run tests that cover the changed behavior from the owning package:
`pnpm test <test-file> --maxWorkers=1`. Or use the root wrapper:
`pnpm agent:test --filter <package> -- <test-file> --maxWorkers=1`.
For Playwright, use `pnpm agent:playwright -- <spec-file> --workers=1`.
- For routine commits, run checks one at a time. Use one test worker by
default. Do not run full frontend typechecks locally. Use CI for
comprehensive lint, typechecks, and test suites.
- For documentation-only changes, review the diff and run `git diff --check`.
Skip code tests when no behavior changed.
- Report which checks ran and which checks were left to CI.
- Use Vitest for unit tests. Use
[Playwright](packages/quality/testing/playwright/AGENTS.md) when a test needs its
browser, fixtures, or managed containers.

View File

@ -1,24 +1,306 @@
## [2.43.2](https://github.com/n8n-io/n8n/compare/n8n@2.43.1...n8n@2.43.2) (2026-10-08)
# [2.45.0](https://github.com/n8n-io/n8n/compare/n8n@2.44.0...n8n@2.45.0) (2026-10-20)
### Bug Fixes
* **core:** Fix Kimi K3 and other OpenAI-compatible Assistant model providers on self-hosted Assistant ([#40552](https://github.com/n8n-io/n8n/issues/40552)) ([509a3f3](https://github.com/n8n-io/n8n/commit/509a3f36a8443e544714e3874dc55ba14f2c9463))
* **editor:** Show the Agent artifact while the builder creates it ([#40550](https://github.com/n8n-io/n8n/issues/40550)) ([204e2dd](https://github.com/n8n-io/n8n/commit/204e2ddf1745095f940659d827b6f8ca000ead49))
* **AI Agent Node:** Replay Anthropic thinking turns with their tool arguments ([#41241](https://github.com/n8n-io/n8n/issues/41241)) ([ea98bd0](https://github.com/n8n-io/n8n/commit/ea98bd06003b5687882c94245df837331cfbb544))
* Align data table and variable editing on protected instances ([#41258](https://github.com/n8n-io/n8n/issues/41258)) ([d4bde14](https://github.com/n8n-io/n8n/commit/d4bde146d6fa8f591a91f0d0ee44b56b96cb7019))
* Allow skipping table initialization ([#40437](https://github.com/n8n-io/n8n/issues/40437)) ([d245b17](https://github.com/n8n-io/n8n/commit/d245b17b898651a50a524a2411d703509afd8339))
* **API:** Apply projectId filter to executions for member API keys ([#40887](https://github.com/n8n-io/n8n/issues/40887)) ([3075b60](https://github.com/n8n-io/n8n/commit/3075b605766a73bee35d07b3afab02050c4e4a49))
* **API:** Log errors before returning generic 500 in Public API ([#41186](https://github.com/n8n-io/n8n/issues/41186)) ([fe36279](https://github.com/n8n-io/n8n/commit/fe3627914a64e5d34b39fcd5e21ba48c8268b9b4))
* Check provisioning license on Users settings page ([#41048](https://github.com/n8n-io/n8n/issues/41048)) ([066480d](https://github.com/n8n-io/n8n/commit/066480d8dbe5e3127a2cf099344a383492c7769a))
* **Convert to File Node:** Decode data URIs and validate Base64 input ([#38554](https://github.com/n8n-io/n8n/issues/38554)) ([315e0a5](https://github.com/n8n-io/n8n/commit/315e0a5377b98f9d2e53f77160150a6655f43341))
* **core:** Allow gateway model IDs and forward credential headers in agents ([#40470](https://github.com/n8n-io/n8n/issues/40470)) ([791c7bc](https://github.com/n8n-io/n8n/commit/791c7bc418bf789a9686859f10592b3980571b50))
* **core:** Allow project promotion without an instance export ([#40939](https://github.com/n8n-io/n8n/issues/40939)) ([e0fe040](https://github.com/n8n-io/n8n/commit/e0fe04084e77d296e2daf1767423a56d66455cee))
* **core:** Allow switching a node back to a credential the user cannot use after a save ([#41198](https://github.com/n8n-io/n8n/issues/41198)) ([e7f0607](https://github.com/n8n-io/n8n/commit/e7f0607bf13e5bc4a2f929caf7a2aaadf5701d26))
* **core:** Clear agent runtime cache when credential is updated ([#40961](https://github.com/n8n-io/n8n/issues/40961)) ([4cb7a6e](https://github.com/n8n-io/n8n/commit/4cb7a6ee3da84c5258ee129285cafeaba5863728))
* **core:** Close agent after run when invoked from workflow ([#41128](https://github.com/n8n-io/n8n/issues/41128)) ([f596162](https://github.com/n8n-io/n8n/commit/f596162dc7b7accdcd71bc544622b1cc13546c88))
* **core:** Confirm project deletions before instance apply ([#40941](https://github.com/n8n-io/n8n/issues/40941)) ([f631112](https://github.com/n8n-io/n8n/commit/f63111202f5cfadb989051c02a87710aafbe33e8))
* **core:** Create an XML parser for each webhook request body ([#41098](https://github.com/n8n-io/n8n/issues/41098)) ([1950434](https://github.com/n8n-io/n8n/commit/1950434a277d863c04c7f640b9dba5d69d6c4ac9))
* **core:** Describe MCP servers only from user or registry text in Agent Builder ([#40974](https://github.com/n8n-io/n8n/issues/40974)) ([248c863](https://github.com/n8n-io/n8n/commit/248c86355b0eb30f0874f14f680f1986da5c9878))
* **core:** Display an error message when failing to connect to oauth credentials ([#41058](https://github.com/n8n-io/n8n/issues/41058)) ([952b1e7](https://github.com/n8n-io/n8n/commit/952b1e73b281539b81a7d5461770b0a515a7add0))
* **core:** Evict idle mcp-browser sessions to prevent connection leak ([#36850](https://github.com/n8n-io/n8n/issues/36850)) ([efd4e21](https://github.com/n8n-io/n8n/commit/efd4e21aadbe3dd826e8d1a3df6e7921559b204b))
* **core:** Handle every escaped closing bracket in an expression ([#40487](https://github.com/n8n-io/n8n/issues/40487)) ([b09efb7](https://github.com/n8n-io/n8n/commit/b09efb706cc9f8b7f4d4328e5bfe13fee9e145eb))
* **core:** Improve base URL check for declarative nodes ([#40778](https://github.com/n8n-io/n8n/issues/40778)) ([e45a65a](https://github.com/n8n-io/n8n/commit/e45a65a9bf1c1eaf1c9282c922753ec265641846))
* **core:** Keep cancelled Assistant activity settled ([#40786](https://github.com/n8n-io/n8n/issues/40786)) ([d74f3f2](https://github.com/n8n-io/n8n/commit/d74f3f2cddabf0b732f4a22ae8a784ad66ed7f29))
* **core:** Keep Observer detail lines instead of discarding the response ([#41134](https://github.com/n8n-io/n8n/issues/41134)) ([48edbef](https://github.com/n8n-io/n8n/commit/48edbef643cc2dcd9c24c18be56fa88fda06ee68))
* **core:** Keep offline_access when a protected resource omits it ([#41384](https://github.com/n8n-io/n8n/issues/41384)) ([9675910](https://github.com/n8n-io/n8n/commit/967591041e508802f6cf77a1b1651ec48175f680))
* **core:** Keep pending setup for nodes that n8n Assistant added in an earlier build ([#41070](https://github.com/n8n-io/n8n/issues/41070)) ([50c4ced](https://github.com/n8n-io/n8n/commit/50c4ced2241430e13fa7042e976ed33f07bf5009))
* **core:** Keep the workflow source file when n8n Assistant sends blank source ([#41201](https://github.com/n8n-io/n8n/issues/41201)) ([b78913f](https://github.com/n8n-io/n8n/commit/b78913f75d2e7d93c22b380160e0b26d198097d9))
* **core:** Key each OAuth authorization request to its own session ([#39930](https://github.com/n8n-io/n8n/issues/39930)) ([3c447b4](https://github.com/n8n-io/n8n/commit/3c447b42a7f019f8fa24cf4b1c599929fc9abb85))
* **core:** Key queue job outcomes by job on shutdown resume ([#40569](https://github.com/n8n-io/n8n/issues/40569)) ([6e04287](https://github.com/n8n-io/n8n/commit/6e04287582ebb66d550faf166a865d0a27f936fa))
* **core:** Let LLM failed-attempt handling retry and honour Retry-After ([#41002](https://github.com/n8n-io/n8n/issues/41002)) ([67c1981](https://github.com/n8n-io/n8n/commit/67c1981f2f2018e817706273d3a91a1d8bdf79ee))
* **core:** Let preview chat agents make changes through their tools ([#41403](https://github.com/n8n-io/n8n/issues/41403)) ([8323dfd](https://github.com/n8n-io/n8n/commit/8323dfd1e1136e33a036ed7a2621cc1e05ca1e0e))
* **core:** Log custom data truncation only above the 512-character limit ([#40899](https://github.com/n8n-io/n8n/issues/40899)) ([e74f1c1](https://github.com/n8n-io/n8n/commit/e74f1c19daa8b7784767f9332541f616aabd1d5f))
* **core:** Pass reasoning effort on open router model node ([#41179](https://github.com/n8n-io/n8n/issues/41179)) ([683fd11](https://github.com/n8n-io/n8n/commit/683fd112594f91b5d90bba6720f0e1081977158a))
* **core:** Redact a parent execution if any sub-workflows have redaction enabled ([#41129](https://github.com/n8n-io/n8n/issues/41129)) ([c70b5d4](https://github.com/n8n-io/n8n/commit/c70b5d49b322c4fbd7bebe940f255ec6e8510adb))
* **core:** Redact the parent execution when a sub-workflow uses a credential the viewer cannot use ([#41085](https://github.com/n8n-io/n8n/issues/41085)) ([a57c8f3](https://github.com/n8n-io/n8n/commit/a57c8f3abaf92aadddd8d2798418bc7e0a53ab18))
* **core:** Reflect failed publications in the active workflows endpoints ([#40641](https://github.com/n8n-io/n8n/issues/40641)) ([96eff1e](https://github.com/n8n-io/n8n/commit/96eff1efb92c85434c1324a68afa4a67e05c9276))
* **core:** Remove unused N8N_SCHEDULER_MIN_INTERVAL setting ([#41194](https://github.com/n8n-io/n8n/issues/41194)) ([c839d6c](https://github.com/n8n-io/n8n/commit/c839d6cd490e355de1efa356327002b74d86ed75))
* **core:** Reuse connections in the log streaming webhook destination ([#41261](https://github.com/n8n-io/n8n/issues/41261)) ([1581524](https://github.com/n8n-io/n8n/commit/158152496aca4f509a54c6af9710e0224f980be8))
* **core:** Skip resource mapper validation when the mapped value is null ([#41069](https://github.com/n8n-io/n8n/issues/41069)) ([898701b](https://github.com/n8n-io/n8n/commit/898701b05b57c56cc2e88f5324b17adcc317d140))
* **core:** Stop asking to set up a node when n8n Assistant only adds a step next to it ([#41067](https://github.com/n8n-io/n8n/issues/41067)) ([578eea4](https://github.com/n8n-io/n8n/commit/578eea403d406bd5e92db18067577e8145599c50))
* **core:** Trace verification helpers and provider attempts ([#40936](https://github.com/n8n-io/n8n/issues/40936)) ([f73dc33](https://github.com/n8n-io/n8n/commit/f73dc337f8dc61a4fd44aebe41c55faafe4f53a4))
* **core:** Turn off expression spans by default ([#41211](https://github.com/n8n-io/n8n/issues/41211)) ([842ec5b](https://github.com/n8n-io/n8n/commit/842ec5b62d9274bff50901d290ff02f837a2e904))
* **core:** Use the module registry to decide if the Assistant has the Agents features ([#41026](https://github.com/n8n-io/n8n/issues/41026)) ([726a582](https://github.com/n8n-io/n8n/commit/726a5829c826f8419cf9ec2d9993c1cb19b5246a))
* **core:** Web search misconfigs in agents as UserError ([#41257](https://github.com/n8n-io/n8n/issues/41257)) ([f844135](https://github.com/n8n-io/n8n/commit/f84413516a80c87f2dca194306f5c1ed18ac2d11))
* **Databricks Node:** Bring the AI Agent panel to parity with AI Agent 2.2 ([#40989](https://github.com/n8n-io/n8n/issues/40989)) ([e859aba](https://github.com/n8n-io/n8n/commit/e859abaf189041f751fcee7333c79acde212f994))
* **Databricks Node:** Rename Genie resource labels to Genie Agent and the MCP tile to Databricks Genie One ([#40760](https://github.com/n8n-io/n8n/issues/40760)) ([14c7790](https://github.com/n8n-io/n8n/commit/14c77900df54948c8ca5f566055e31ec417eae09))
* **editor:** Apply the standard settings layout to the Context preferences page ([#40716](https://github.com/n8n-io/n8n/issues/40716)) ([b3e8634](https://github.com/n8n-io/n8n/commit/b3e8634c83cf660f71ea08319c8c19f975ad795f))
* **editor:** Cancel the Assistant thread when the hand-off navigation fails ([#41056](https://github.com/n8n-io/n8n/issues/41056)) ([d26f914](https://github.com/n8n-io/n8n/commit/d26f914e3d4fb3fd09d3004a435d8dd92198a34f))
* **editor:** Cleanup legacy button variants ([#41175](https://github.com/n8n-io/n8n/issues/41175)) ([ba97e46](https://github.com/n8n-io/n8n/commit/ba97e46afc579c48eacacb42832df524b49f790e))
* **editor:** Display custom evaluation metrics correctly ([#40852](https://github.com/n8n-io/n8n/issues/40852)) ([f6b24d0](https://github.com/n8n-io/n8n/commit/f6b24d0ae93b55cceefa50045a0efb3cc9279a86))
* **editor:** Improve destination validation and surface save errors ([#40897](https://github.com/n8n-io/n8n/issues/40897)) ([eaf47cf](https://github.com/n8n-io/n8n/commit/eaf47cfe39778fb8a85cae29358eeba6356d1e23))
* **editor:** Improve NDV credential options performance on large instances ([#40198](https://github.com/n8n-io/n8n/issues/40198)) ([b98640f](https://github.com/n8n-io/n8n/commit/b98640f0abb2746983230412d85963030b0aaecb))
* **editor:** Keep pending navigation when workflows list query is unchanged ([#41125](https://github.com/n8n-io/n8n/issues/41125)) ([116a81e](https://github.com/n8n-io/n8n/commit/116a81e06ec9f3b3478759003f0cb2aef380aec6))
* **editor:** Keep the used-in rows on the credential sharing tab when personal-space sharing is disabled ([#41207](https://github.com/n8n-io/n8n/issues/41207)) ([ca9661c](https://github.com/n8n-io/n8n/commit/ca9661c091acd43b3dcd2408df22321d6662b07b))
* **editor:** Keep workflow name in edit mode after Rename menu action ([#41120](https://github.com/n8n-io/n8n/issues/41120)) ([fc9f957](https://github.com/n8n-io/n8n/commit/fc9f9575e2cea830da1f2f1efccd73a49b43a26a))
* **editor:** Load credential types on startup ([#41400](https://github.com/n8n-io/n8n/issues/41400)) ([9a025a1](https://github.com/n8n-io/n8n/commit/9a025a188f6164723f1e8cd928f1f21d30765587))
* **editor:** Match the credential dropdown width to its field and remove horizontal scroll ([#41260](https://github.com/n8n-io/n8n/issues/41260)) ([51e53ad](https://github.com/n8n-io/n8n/commit/51e53ad9b77cc95c0fd8e683be4204ea9c7c85f7))
* **editor:** Offer the managed OAuth choice when the app is supplied ([#40284](https://github.com/n8n-io/n8n/issues/40284)) ([14d2d30](https://github.com/n8n-io/n8n/commit/14d2d305630217271244c7fe962e36aab8f826f0))
* **editor:** Place sticky notes and nodes at the right-click and clear the selection ([#41365](https://github.com/n8n-io/n8n/issues/41365)) ([28602ed](https://github.com/n8n-io/n8n/commit/28602ed56ea703c6f3d05652be00f9dc471f2828))
* **editor:** Prevent doubled base path when opening folders with N8N_PATH set ([#33065](https://github.com/n8n-io/n8n/issues/33065)) ([a32c652](https://github.com/n8n-io/n8n/commit/a32c652c0957011bdea308439357b537d78ad1cd))
* **editor:** Rename Admin Panel to Cloud console ([#41187](https://github.com/n8n-io/n8n/issues/41187)) ([8120328](https://github.com/n8n-io/n8n/commit/81203286613a2732301006cdd376f9a6ff3a8b1e))
* **editor:** Restore From AI button corner radii ([#41176](https://github.com/n8n-io/n8n/issues/41176)) ([20ea671](https://github.com/n8n-io/n8n/commit/20ea6717a05c2bcef4d5541002cfe4b409618875))
* **editor:** Restore the new session shortcut in agent chat ([#40907](https://github.com/n8n-io/n8n/issues/40907)) ([ebf078b](https://github.com/n8n-io/n8n/commit/ebf078b64c6f7578e0186300b5404804dc277feb))
* **editor:** Return to the requested page after SSO sign-in ([#41151](https://github.com/n8n-io/n8n/issues/41151)) ([4a5fdd5](https://github.com/n8n-io/n8n/commit/4a5fdd5b68f676945e1e012963cafca64521058f))
* **editor:** Save only the OpenTelemetry status when enabling or disabling ([#41142](https://github.com/n8n-io/n8n/issues/41142)) ([1903d53](https://github.com/n8n-io/n8n/commit/1903d53bc07f43729aa003fef4412a49c343e26f))
* **editor:** Show follow-up run events in the Assistant after a reconnect ([#40883](https://github.com/n8n-io/n8n/issues/40883)) ([ffda639](https://github.com/n8n-io/n8n/commit/ffda639b3f5eac1ce1c58b0decff45b92682be04))
* **editor:** Start the empty-state icon cards cycling on load and use them on the MCP and external secrets pages ([#40232](https://github.com/n8n-io/n8n/issues/40232)) ([1f223c8](https://github.com/n8n-io/n8n/commit/1f223c85f8a8fef43e785f86c27c9d4a75695b72))
* **editor:** Stop auto-selecting the Evaluation Trigger over other triggers ([#38883](https://github.com/n8n-io/n8n/issues/38883)) ([8b5ea6d](https://github.com/n8n-io/n8n/commit/8b5ea6d9e0af5800c0a631e0a25665e0b1d3f3b9))
* **editor:** Stop credential icon overlapping the credential name in the agent credential picker ([#41262](https://github.com/n8n-io/n8n/issues/41262)) ([70f0131](https://github.com/n8n-io/n8n/commit/70f0131be5a95e9d33bf4b542137b8d20ef59f32))
* **editor:** Unify documentation links across settings pages ([#40289](https://github.com/n8n-io/n8n/issues/40289)) ([76cce99](https://github.com/n8n-io/n8n/commit/76cce9962685e585977fbcc3d400ae5654942bd3))
* **editor:** Use the deployment name on the Az Foundry path ([#41103](https://github.com/n8n-io/n8n/issues/41103)) ([b3e8b7d](https://github.com/n8n-io/n8n/commit/b3e8b7d9ce0a00e97f5d03634a8928112019f060))
* **Email Trigger (IMAP) Node:** Check the mailbox when the trigger activates ([#41013](https://github.com/n8n-io/n8n/issues/41013)) ([a6719b8](https://github.com/n8n-io/n8n/commit/a6719b8db71802ee6818f87266558417e302d11f))
* **Email Trigger (IMAP) Node:** Find new mail on servers with a broken IMAP4rev2 mode ([#41018](https://github.com/n8n-io/n8n/issues/41018)) ([236220b](https://github.com/n8n-io/n8n/commit/236220b248bc6c553fd9a2f6f1734677b81a6a82))
* **ERPNext Node:** Return each row once and tolerate empty Get Many options ([#36860](https://github.com/n8n-io/n8n/issues/36860)) ([3968a60](https://github.com/n8n-io/n8n/commit/3968a60bf74dbe7b72996fda53a5b5eaaed272a4))
* **Execute Command Node:** Don't spawn console on Windows ([#40909](https://github.com/n8n-io/n8n/issues/40909)) ([52aed7a](https://github.com/n8n-io/n8n/commit/52aed7ac97c641f125e74bb68964bc31a29ef701))
* **GitlabTrigger:** use NodeApiError.httpCode and safe description access in webhook checkExists ([#35486](https://github.com/n8n-io/n8n/issues/35486)) ([054288e](https://github.com/n8n-io/n8n/commit/054288e48855b5aa6372be2a4bf1864c9401f6c1))
* **Gmail Node:** Keep the References header on a draft created in a thread ([#41392](https://github.com/n8n-io/n8n/issues/41392)) ([06ba85c](https://github.com/n8n-io/n8n/commit/06ba85cbb9bfeb7db6899c411e19bfb0269430af))
* **HomeAssistant Node:** Only sent Home Assistant attributes on upsert if set ([#36298](https://github.com/n8n-io/n8n/issues/36298)) ([b8780d8](https://github.com/n8n-io/n8n/commit/b8780d8fee5e445a12d3583041a1e09a36213254))
* Keep free-floating sticky notes clear of workflow nodes ([#41195](https://github.com/n8n-io/n8n/issues/41195)) ([131e9c8](https://github.com/n8n-io/n8n/commit/131e9c8679f96254873d401fbb7e9e57c5e976f2))
* **MCP Trigger Node:** Pass through MCP content arrays instead of double-wrapping tool results ([#39131](https://github.com/n8n-io/n8n/issues/39131)) ([f483ebd](https://github.com/n8n-io/n8n/commit/f483ebd9d1b8bb032d05aab335413256aa590305))
* **Microsoft Teams Node:** Add the TeamMember.Read.All scope and correct the Trigger scope notice ([#39819](https://github.com/n8n-io/n8n/issues/39819)) ([ddc9769](https://github.com/n8n-io/n8n/commit/ddc97694b5ad3d7888923940c2e9a20fe2fb042e))
* **OpenAI Node:** Report token usage for image operations ([#38463](https://github.com/n8n-io/n8n/issues/38463)) ([a3f4711](https://github.com/n8n-io/n8n/commit/a3f47117cf6484184b4b4604319cf3d3191b0e16))
* **OpenAI Node:** Send max_completion_tokens when it analyzes an image ([#38656](https://github.com/n8n-io/n8n/issues/38656)) ([e972309](https://github.com/n8n-io/n8n/commit/e9723095d6d0360d8d8afc1ea28d34dbce16feb6))
* Pass structured inputs to OpenAI Responses tools ([#40638](https://github.com/n8n-io/n8n/issues/40638)) ([1d2c879](https://github.com/n8n-io/n8n/commit/1d2c879fabcfe100afc2cc66ceb1c66551a17b66))
* **Postgres Node:** Keep comma-containing values as one query parameter ([#40240](https://github.com/n8n-io/n8n/issues/40240)) ([f5dc06f](https://github.com/n8n-io/n8n/commit/f5dc06f753e54de31b089c095f980057e95aa33b))
* Restore command bar focus and standardize creation icons ([#41185](https://github.com/n8n-io/n8n/issues/41185)) ([9e3b52c](https://github.com/n8n-io/n8n/commit/9e3b52cbbb9336118e07e68a5bdec7e56047ec76))
* **Send Email Node:** Send to every recipient when a recipient field is an array ([#36467](https://github.com/n8n-io/n8n/issues/36467)) ([660849d](https://github.com/n8n-io/n8n/commit/660849d93e3fa02fde0e12f33ce93a80f18643eb))
* Skip dependency files and continue after custom node load ([#41117](https://github.com/n8n-io/n8n/issues/41117)) ([5a3fc39](https://github.com/n8n-io/n8n/commit/5a3fc3997284b3fdd7a9285b52f4a16230bd3b44))
* **Splunk Node:** Preserve errors when continuing after failed items ([#40368](https://github.com/n8n-io/n8n/issues/40368)) ([e29c3a4](https://github.com/n8n-io/n8n/commit/e29c3a40856d17914649394036905ea9138fdda3))
* Validate attendee values ([#40953](https://github.com/n8n-io/n8n/issues/40953)) ([21221ad](https://github.com/n8n-io/n8n/commit/21221adc78ff3bb5695c96b487418329e75484be))
## [2.43.1](https://github.com/n8n-io/n8n/compare/n8n@2.43.0...n8n@2.43.1) (2026-10-07)
### Features
### Bug Fixes
* **editor:** Polish n8n Assistant and Execute button UI details ([#40438](https://github.com/n8n-io/n8n/issues/40438)) ([345ae01](https://github.com/n8n-io/n8n/commit/345ae0100c073de418cab4eaf3f48926b414e6e1))
* **API:** Honor Idempotency-Key on Public API POST requests ([#40978](https://github.com/n8n-io/n8n/issues/40978)) ([b7b3d72](https://github.com/n8n-io/n8n/commit/b7b3d7273756ae137e7ba45007f09735d76f6362))
* **API:** Return the parent folder on workflow read endpoints ([#40981](https://github.com/n8n-io/n8n/issues/40981)) ([a906d2f](https://github.com/n8n-io/n8n/commit/a906d2fdfcc5481799a571b4b666eea864fa07a1))
* **Azure OpenAI Chat Model Node:** Add an optional Project field to the Foundry credentials ([#41181](https://github.com/n8n-io/n8n/issues/41181)) ([0c1a787](https://github.com/n8n-io/n8n/commit/0c1a78793a5546851d0585607c06767790d25ccb))
* **core:** Add an endpoint that notifies workflow owners about migration findings ([#40952](https://github.com/n8n-io/n8n/issues/40952)) ([d8a0666](https://github.com/n8n-io/n8n/commit/d8a06666daf8b9ddda768a5ed6d419bd48b420fc))
* **core:** Add an optional model for n8n Assistant memory tasks ([#41137](https://github.com/n8n-io/n8n/issues/41137)) ([f34c093](https://github.com/n8n-io/n8n/commit/f34c093f082978f29e44fdb71620f0047ef75ca5))
* **core:** Add create-with-values credential missing mode to package import ([#36767](https://github.com/n8n-io/n8n/issues/36767)) ([d3bc275](https://github.com/n8n-io/n8n/commit/d3bc2754b44b7a350927bf3b0d9dcfe6e377a155))
* **core:** Add self-healing results to Inbox ([#40741](https://github.com/n8n-io/n8n/issues/40741)) ([1405d4a](https://github.com/n8n-io/n8n/commit/1405d4a351db542b363f5f2c8675e948b353c3ee))
* **core:** Add the blocking project and opt-in state to policy violations ([#40985](https://github.com/n8n-io/n8n/issues/40985)) ([4419fc4](https://github.com/n8n-io/n8n/commit/4419fc4ba237fa825dbeb90d9e697994c96b58aa))
* **core:** Add typing indicator, media download and media retention … ([#40920](https://github.com/n8n-io/n8n/issues/40920)) ([adedbbf](https://github.com/n8n-io/n8n/commit/adedbbf5608e70acca75e6481ae5b98c518a9f6d))
* **core:** Add WhatsApp credential check and webhook handshake status ([#40921](https://github.com/n8n-io/n8n/issues/40921)) ([9a524e7](https://github.com/n8n-io/n8n/commit/9a524e7c21ab5ae09c74d9fbae9f2ab5a8f95fe6))
* **core:** Append node tool attributions to agent replies and add Genie attribution and deep links to the Databricks node ([#40765](https://github.com/n8n-io/n8n/issues/40765)) ([44c8a31](https://github.com/n8n-io/n8n/commit/44c8a312c37e15ba226c05fd675b7702da25639a))
* **core:** Apply budget alerts and stops to sub-agent run ([#40884](https://github.com/n8n-io/n8n/issues/40884)) ([9498088](https://github.com/n8n-io/n8n/commit/949808825637633d6114afb06bdaa28b023107d7))
* **core:** Enable node and credential type policies by default ([#41022](https://github.com/n8n-io/n8n/issues/41022)) ([dd79a61](https://github.com/n8n-io/n8n/commit/dd79a610c5afa1daa07aa548c795d8cad98e6187))
* **core:** Enable the durable scheduler by default ([#40891](https://github.com/n8n-io/n8n/issues/40891)) ([f6861fe](https://github.com/n8n-io/n8n/commit/f6861fe4948567ff792c71aedaba52624ab48fdc))
* **core:** Implement automated migrations for removed nodes ([#40877](https://github.com/n8n-io/n8n/issues/40877)) ([739c59e](https://github.com/n8n-io/n8n/commit/739c59e9913a27328256a10b6570d48402ef5779))
* **core:** Only show execution input and output for agent subworkflows in timeline ([#41189](https://github.com/n8n-io/n8n/issues/41189)) ([4d292a6](https://github.com/n8n-io/n8n/commit/4d292a6009d14a8c634d802ef09c54c4cab5acf0))
* **core:** Refuse revoked access tokens of the local server on the inbound pipeline ([#41266](https://github.com/n8n-io/n8n/issues/41266)) ([0936dfd](https://github.com/n8n-io/n8n/commit/0936dfddc32280a45e270ef1ec12c33f31c54795))
* **core:** Remove the startup pass from workflow history compaction ([#40886](https://github.com/n8n-io/n8n/issues/40886)) ([45eea1b](https://github.com/n8n-io/n8n/commit/45eea1bc35355305409988bc3b1ccb0ac74bf649))
* **core:** Report the CPU and memory limits n8n was given in telemetry ([#40421](https://github.com/n8n-io/n8n/issues/40421)) ([d6d24f2](https://github.com/n8n-io/n8n/commit/d6d24f267c9658aae7a565d739cdda11f8d580e7))
* **core:** Report the runtime n8n runs on in telemetry ([#40420](https://github.com/n8n-io/n8n/issues/40420)) ([b2edcb5](https://github.com/n8n-io/n8n/commit/b2edcb5727ea07c08428056ec9a36f84b486c244))
* **core:** Run the Form and Chat trigger redirect login against a trusted source ([#41123](https://github.com/n8n-io/n8n/issues/41123)) ([862c7ec](https://github.com/n8n-io/n8n/commit/862c7eceab3bfe6b3e757a37c4806124c3d1be99))
* **core:** Store promotion exports in n8n directory ([#40863](https://github.com/n8n-io/n8n/issues/40863)) ([333a010](https://github.com/n8n-io/n8n/commit/333a0101121352f032c93e6b10c268eb59bb4dfc))
* **core:** Surface the responding model name in LLM sub-node run data ([#41161](https://github.com/n8n-io/n8n/issues/41161)) ([b10c97e](https://github.com/n8n-io/n8n/commit/b10c97ed453b8d6fff043677e67b0a7e82866eba))
* **core:** Suspend in-flight production executions at worker shutdown ([#40568](https://github.com/n8n-io/n8n/issues/40568)) ([f3e2bd8](https://github.com/n8n-io/n8n/commit/f3e2bd83b3a18308324971e5275b39f7a88e4395))
* **core:** Treat notified migration findings as unresolved ([#40951](https://github.com/n8n-io/n8n/issues/40951)) ([347385b](https://github.com/n8n-io/n8n/commit/347385b07b449a26ac214f69d0d198385a108581))
* **Databricks Node:** Show the Vector Store, Embeddings and AI Agent nodes in the node picker ([#41206](https://github.com/n8n-io/n8n/issues/41206)) ([191ea98](https://github.com/n8n-io/n8n/commit/191ea984824ba92ab33e9d2019a125af23b4712c))
* Don't save MCP connections without user approval ([#39996](https://github.com/n8n-io/n8n/issues/39996)) ([d27de60](https://github.com/n8n-io/n8n/commit/d27de60b955fcc4afcda4319873431f2ce1b2bd7))
* **editor:** Align the WhatsApp channel setup and settings with the … ([#41283](https://github.com/n8n-io/n8n/issues/41283)) ([275f7eb](https://github.com/n8n-io/n8n/commit/275f7ebf685b67cfcfe0c006f0116b04a8249d4b))
* **editor:** Extend N8nCopyInput with accessibility, redaction, loading and actions ([#34942](https://github.com/n8n-io/n8n/issues/34942)) ([c084d99](https://github.com/n8n-io/n8n/commit/c084d996afd41406ca5a21b4fcdf2f9aa0f43a0c))
* **editor:** Let admins notify workflow owners from the migration rule detail ([#40949](https://github.com/n8n-io/n8n/issues/40949)) ([a972eeb](https://github.com/n8n-io/n8n/commit/a972eebe513b0e43e2fa32eb3c726626c1f9cdc0))
* **editor:** Let admins notify workflow owners in bulk from the migration rule detail ([#41060](https://github.com/n8n-io/n8n/issues/41060)) ([de42c95](https://github.com/n8n-io/n8n/commit/de42c9515d43a7357bb196f224c65e22bf173a9a))
* **editor:** Link to the credential sharing tab when execution data is redacted ([#40976](https://github.com/n8n-io/n8n/issues/40976)) ([da3491f](https://github.com/n8n-io/n8n/commit/da3491fc8786525670e24d35a2c136a0482b32cc))
* **editor:** Publish agent modal and modal for warning about unpublished workflows ([#40987](https://github.com/n8n-io/n8n/issues/40987)) ([e47940e](https://github.com/n8n-io/n8n/commit/e47940eecd3523105c7c097e949c4da91770328e))
* **editor:** Rework the WhatsApp channel setup to match the new design ([#40923](https://github.com/n8n-io/n8n/issues/40923)) ([af0202d](https://github.com/n8n-io/n8n/commit/af0202d840b4f48a31011e4c28fd9759df17a7a1))
* **editor:** Show agent personalisation icon in subagent modal ([#41094](https://github.com/n8n-io/n8n/issues/41094)) ([6160624](https://github.com/n8n-io/n8n/commit/616062487ae56e85a31002d7116917b9c7815069))
* **editor:** Show multiple workflows in the review detail view ([#40695](https://github.com/n8n-io/n8n/issues/40695)) ([f6d70c1](https://github.com/n8n-io/n8n/commit/f6d70c140b9bdd9cb4f777311610b8a8909bbbfe))
* **editor:** Show policy-blocked items in the source control pull modal ([#40604](https://github.com/n8n-io/n8n/issues/40604)) ([1ea18e9](https://github.com/n8n-io/n8n/commit/1ea18e9ef9e95cead7fe91ad4a7cb3fa46b3dfd7))
* **editor:** Show the model that answered in the AI log header ([#40988](https://github.com/n8n-io/n8n/issues/40988)) ([caeaff7](https://github.com/n8n-io/n8n/commit/caeaff72300b4af739a6256046faa637e72627ca))
* **editor:** Show the policy reason when an agent tool call is refused ([#40673](https://github.com/n8n-io/n8n/issues/40673)) ([bb45bca](https://github.com/n8n-io/n8n/commit/bb45bca5ff7ef7d2cd2b2f2e98812d96a5971b01))
* **editor:** Update budget panel when agent executes ([#40971](https://github.com/n8n-io/n8n/issues/40971)) ([632d8b9](https://github.com/n8n-io/n8n/commit/632d8b9eb0bb1f088856520b7770e75e115ffd60))
* **editor:** Update UI with n8n rebrand ([#40342](https://github.com/n8n-io/n8n/issues/40342)) ([bffe7d8](https://github.com/n8n-io/n8n/commit/bffe7d8392d614b542beef65867a88aa671baa89))
* Implements workflow previews above Assistant empty state experiment ([#40913](https://github.com/n8n-io/n8n/issues/40913)) ([eac2f46](https://github.com/n8n-io/n8n/commit/eac2f46b0c7a0e82c60d4391986c80865367124b))
* **Kafka Node:** Make version 2 the default for new Kafka nodes and triggers ([#40744](https://github.com/n8n-io/n8n/issues/40744)) ([e5866ac](https://github.com/n8n-io/n8n/commit/e5866ac6c17c61cd7bfdfc9b49294c936af3d566))
* **Monday.com Node:** Add V2 with full V1 parity as the default ([#35515](https://github.com/n8n-io/n8n/issues/35515)) ([8977833](https://github.com/n8n-io/n8n/commit/8977833ee2f76df7f92a7764589d5b3164706f46))
* Overhaul MCP tool permissions for Agents ([#39878](https://github.com/n8n-io/n8n/issues/39878)) ([bda9118](https://github.com/n8n-io/n8n/commit/bda9118168ce47b709355aca12bfb0e0046ba479))
* Show Gemini models by name and list prioritized models first in model pickers ([#40965](https://github.com/n8n-io/n8n/issues/40965)) ([acf5291](https://github.com/n8n-io/n8n/commit/acf5291d800aeffec50911f9d63215e5c12dc134))
* **Text Classifier Node:** Add opt-in confidence scores to the output ([#40743](https://github.com/n8n-io/n8n/issues/40743)) ([3800bce](https://github.com/n8n-io/n8n/commit/3800bce8126cebbff80536e45851e7f21d9ad8ec))
### Performance Improvements
* **core:** Keep agent builder system prompt static for prompt caching ([#40441](https://github.com/n8n-io/n8n/issues/40441)) ([9ea7835](https://github.com/n8n-io/n8n/commit/9ea78350fa4a3972cbef036ccfd8355b0e127262))
* **core:** Prepare Instance AI skill sandbox in the background ([#40442](https://github.com/n8n-io/n8n/issues/40442)) ([fda8dac](https://github.com/n8n-io/n8n/commit/fda8dac7d8bd7a3fe427285de281911d68f8f63b))
* **editor:** Load prettier only when the user formats code ([#41014](https://github.com/n8n-io/n8n/issues/41014)) ([e2529a8](https://github.com/n8n-io/n8n/commit/e2529a849432bdb3082615c04b1702df61cb91d5))
# [2.44.0](https://github.com/n8n-io/n8n/compare/n8n@2.43.0...n8n@2.44.0) (2026-10-13)
### Bug Fixes
* Align execution retry menu and clarify actions ([#40710](https://github.com/n8n-io/n8n/issues/40710)) ([84d2fd6](https://github.com/n8n-io/n8n/commit/84d2fd6f3ae68e973aa49f7e2b1dd5dbf06ad18c))
* Allow per-process crash journal paths ([#37670](https://github.com/n8n-io/n8n/issues/37670)) ([447e939](https://github.com/n8n-io/n8n/commit/447e939b732b778662572098d81f014527e3ffe2))
* **core:** Allow reverting a workflow version that uses credentials the user cannot use ([#40701](https://github.com/n8n-io/n8n/issues/40701)) ([033a9b5](https://github.com/n8n-io/n8n/commit/033a9b512c2d52d05b43d4dd2d0e10634dc36e2a))
* **core:** Apply node type policies to agent node tools ([#39711](https://github.com/n8n-io/n8n/issues/39711)) ([e7d8533](https://github.com/n8n-io/n8n/commit/e7d8533880df0d36438e39a07f0594fcaaaaab55))
* **core:** Apply the credential type policy to credential tests and probes ([#39927](https://github.com/n8n-io/n8n/issues/39927)) ([9191cf3](https://github.com/n8n-io/n8n/commit/9191cf3fb07ce6bbb7b2217592ce56372b52f6c2))
* **core:** Block the assistant when AI credits hit zero ([#40659](https://github.com/n8n-io/n8n/issues/40659)) ([8b7065a](https://github.com/n8n-io/n8n/commit/8b7065a87684bb3a055d8bcfc5d34c9b70094080))
* **core:** Check the credential save policy before package import creates stubs ([#40687](https://github.com/n8n-io/n8n/issues/40687)) ([a3391ba](https://github.com/n8n-io/n8n/commit/a3391ba93c4574405d06e79e2efc791b6ba8a21f))
* **core:** Check the target project's policy on folder, credential and delete-transfer moves ([#40301](https://github.com/n8n-io/n8n/issues/40301)) ([77370af](https://github.com/n8n-io/n8n/commit/77370af99a273073faf7df57a1fe8f71d3833bbe))
* **core:** Count jobs as running from the start of job processing ([#40050](https://github.com/n8n-io/n8n/issues/40050)) ([dc5c80f](https://github.com/n8n-io/n8n/commit/dc5c80f26af8d32e5887ccbf7634f3da1be5f771))
* **core:** Expose 1Password fields for secret preview and autocomplete ([#40538](https://github.com/n8n-io/n8n/issues/40538)) ([bef5f3f](https://github.com/n8n-io/n8n/commit/bef5f3f97aca163ac289af61e10869c0fa7fff60))
* **core:** Fix Kimi K3 and other OpenAI-compatible Assistant model providers on self-hosted Assistant ([#40439](https://github.com/n8n-io/n8n/issues/40439)) ([1ef7756](https://github.com/n8n-io/n8n/commit/1ef77564e3bef2e34c57d8a67df892ace3858a5d))
* **core:** Handle each Chat Hub segment once in queue mode ([#40791](https://github.com/n8n-io/n8n/issues/40791)) ([9ff8d4f](https://github.com/n8n-io/n8n/commit/9ff8d4fc738ca0e54db863c574e5a6d93df9ce2e))
* **core:** Improve node placement when n8n Assistant edits a workflow ([#40139](https://github.com/n8n-io/n8n/issues/40139)) ([d900743](https://github.com/n8n-io/n8n/commit/d900743bce0cf8274c42e1f4a579fc8b5cf74997))
* **core:** Keep waiting executions unchanged when a resume request fails ([#40683](https://github.com/n8n-io/n8n/issues/40683)) ([bbd35e2](https://github.com/n8n-io/n8n/commit/bbd35e281b100a36aa209518c5ab027eabf85a1f))
* **core:** Let the editor download binary data of an Engine v2 execution ([#40663](https://github.com/n8n-io/n8n/issues/40663)) ([c9cb3ee](https://github.com/n8n-io/n8n/commit/c9cb3ee3abd23fe146cd4a2daed49c4bddb80d97))
* **core:** Let the streaming heartbeat stop on Chat Hub responses ([#40792](https://github.com/n8n-io/n8n/issues/40792)) ([ee8a608](https://github.com/n8n-io/n8n/commit/ee8a608a22ec7a23dc795ee0554edd34186e87d2))
* **core:** Match regex case-insensitively when ignore case is enabled ([#33589](https://github.com/n8n-io/n8n/issues/33589)) ([2596f3a](https://github.com/n8n-io/n8n/commit/2596f3ac1c657aa4d2ddf40430b9e5940fb9c1b0))
* **core:** Match the Brazilian real symbol in the in-isolate currency regex ([#40893](https://github.com/n8n-io/n8n/issues/40893)) ([d3deffb](https://github.com/n8n-io/n8n/commit/d3deffbfa59d358614458f8b1ae2d56d118a9ca5))
* **core:** Pass `storedAt` into `WorkflowExecute` at every callsite ([#40495](https://github.com/n8n-io/n8n/issues/40495)) ([5ab2187](https://github.com/n8n-io/n8n/commit/5ab21874169d44eb01a19d6f0ba19bb7a254d952))
* **core:** Remove credentials of deleted projects on source control pull ([#40692](https://github.com/n8n-io/n8n/issues/40692)) ([978f810](https://github.com/n8n-io/n8n/commit/978f810bbef9e284b564db4897be7085257064b1))
* **core:** Report approval tools with no tool connected in workflow validation ([#40561](https://github.com/n8n-io/n8n/issues/40561)) ([bc0a9cb](https://github.com/n8n-io/n8n/commit/bc0a9cbc112aaed2853f14c32157082f41ba42b4))
* **core:** Report MCP server connection failures in verify_mcp_server ([#40559](https://github.com/n8n-io/n8n/issues/40559)) ([fbbbadc](https://github.com/n8n-io/n8n/commit/fbbbadc292791112e30101edee229e9d5f4dbc02))
* **core:** Require workflow create scope for single node runs in n8n Assistant ([#40908](https://github.com/n8n-io/n8n/issues/40908)) ([27fcb96](https://github.com/n8n-io/n8n/commit/27fcb968fb711a9957eed48fa78c5adfa78fc607))
* **core:** Resume tool calls for deferred tools that a skill loaded ([#40126](https://github.com/n8n-io/n8n/issues/40126)) ([1c84af2](https://github.com/n8n-io/n8n/commit/1c84af2e96bda41cd84852af82e5cae5b823f0f8))
* **core:** Return jobs dequeued during worker shutdown to the queue ([#40086](https://github.com/n8n-io/n8n/issues/40086)) ([084f3b3](https://github.com/n8n-io/n8n/commit/084f3b376a0d64594320853bae90f13537d3b8c9))
* **core:** Return jobs fetched before a worker paused to the queue ([#40162](https://github.com/n8n-io/n8n/issues/40162)) ([f26747b](https://github.com/n8n-io/n8n/commit/f26747bfc665ce0c137b0b7645245b1a97d1d01b))
* **core:** Save agent draft definitions atomically ([#40374](https://github.com/n8n-io/n8n/issues/40374)) ([9ee5d1a](https://github.com/n8n-io/n8n/commit/9ee5d1ad7a54cde445ac945fc56a7ba5a4ee4707))
* **core:** Serve the migration report on every main in a multi-main setup ([#40180](https://github.com/n8n-io/n8n/issues/40180)) ([a6f1e79](https://github.com/n8n-io/n8n/commit/a6f1e79394adced5958219f812e8ca14a10b9962))
* **core:** Show the policy block before node issues on execute and publish ([#40290](https://github.com/n8n-io/n8n/issues/40290)) ([0bbea43](https://github.com/n8n-io/n8n/commit/0bbea438e1906a0773cdbb46e273b4c01fea359b))
* **core:** Stabilise background child approval wait ([#40834](https://github.com/n8n-io/n8n/issues/40834)) ([2ca4e55](https://github.com/n8n-io/n8n/commit/2ca4e55502e03985d2c9b1b0033b6f5151ccb8f4))
* **core:** Start a new test session when call_agent gets an unknown session ID ([#40458](https://github.com/n8n-io/n8n/issues/40458)) ([e16566b](https://github.com/n8n-io/n8n/commit/e16566b31324991a369708b7eb1f46027f80445b))
* **core:** Stop marking the Message an Agent node as hidden ([#40358](https://github.com/n8n-io/n8n/issues/40358)) ([3366d2b](https://github.com/n8n-io/n8n/commit/3366d2b9b3151a769116a66499668c3a36cf65a4))
* **core:** Validate execution timeout against the configured maximum in Instance AI saves ([#40769](https://github.com/n8n-io/n8n/issues/40769)) ([2176819](https://github.com/n8n-io/n8n/commit/2176819865a03e9247dac0a61177572ac4f7711c))
* **core:** Validate the size argument of the chunk() expression extension ([#40880](https://github.com/n8n-io/n8n/issues/40880)) ([3ff9c83](https://github.com/n8n-io/n8n/commit/3ff9c83bdaf873509e4470f97e26919e0653add8))
* **core:** Wire Entra OAuth2 into agent Azure OpenAI runtime ([#40218](https://github.com/n8n-io/n8n/issues/40218)) ([96a8ff4](https://github.com/n8n-io/n8n/commit/96a8ff4d15baa2ab15debc8ea02da81847f5a504))
* **Databricks Node:** Request float encoding for embeddings from the Unity AI Gateway ([#40684](https://github.com/n8n-io/n8n/issues/40684)) ([4c0902f](https://github.com/n8n-io/n8n/commit/4c0902f6bf4a0aa6634d5598e75fc677aa55f429))
* **editor:** Add resend to retryable Agent preview errors ([#40592](https://github.com/n8n-io/n8n/issues/40592)) ([f7e52a8](https://github.com/n8n-io/n8n/commit/f7e52a851ed9416615167826daf706d90a19d53b))
* **editor:** Align nodes panel to the embedded canvas in the n8n Assistant split view ([#40795](https://github.com/n8n-io/n8n/issues/40795)) ([37799ad](https://github.com/n8n-io/n8n/commit/37799ad2f21067ec7951c2656c9e938dc87443e1))
* **editor:** Align tooltip trigger box with its content ([#40422](https://github.com/n8n-io/n8n/issues/40422)) ([a694531](https://github.com/n8n-io/n8n/commit/a694531a0367bc6b7991b194ec762e7ad51aaa0a))
* **editor:** Keep Agent Builder settings inside a narrow column ([#40759](https://github.com/n8n-io/n8n/issues/40759)) ([e2b4fee](https://github.com/n8n-io/n8n/commit/e2b4fee0ec0f0f34245664c23869153089362236))
* **editor:** Keep Assistant fixes in Agent builder and correct tooltips ([#40348](https://github.com/n8n-io/n8n/issues/40348)) ([a13705a](https://github.com/n8n-io/n8n/commit/a13705aa8fff5a170ba5b069946e205e9acc1985))
* **editor:** Keep folder creation available globally after Community registration ([#40082](https://github.com/n8n-io/n8n/issues/40082)) ([e1025f3](https://github.com/n8n-io/n8n/commit/e1025f35e4203b49436aedb89dcb679e96169528))
* **editor:** Keep same-named parameter values apart when their types differ ([#40170](https://github.com/n8n-io/n8n/issues/40170)) ([6013045](https://github.com/n8n-io/n8n/commit/601304543d2b87e6aa42211c3a8c6c2766c2c61d))
* **editor:** Keep the prompt visible while the assistant starts a new thread ([#40796](https://github.com/n8n-io/n8n/issues/40796)) ([be402be](https://github.com/n8n-io/n8n/commit/be402be262974d46f79ba3464e20efc0ee734ac4))
* **editor:** Keep the selected credential visible when the picker does not list it ([#40450](https://github.com/n8n-io/n8n/issues/40450)) ([66b382d](https://github.com/n8n-io/n8n/commit/66b382de52fb56db87987910929e932292837305))
* **editor:** Keep the workflow setup step counter on one line ([#40771](https://github.com/n8n-io/n8n/issues/40771)) ([55eb2ce](https://github.com/n8n-io/n8n/commit/55eb2ce44291648adf6ca05b5b1605efea8d77f7))
* **editor:** Keep typed characters in Markdown fields ([#40805](https://github.com/n8n-io/n8n/issues/40805)) ([16e29da](https://github.com/n8n-io/n8n/commit/16e29da52f3173ee5edc7b96697a4c227b4983d0))
* **editor:** Load node types before authenticated pages render ([#40812](https://github.com/n8n-io/n8n/issues/40812)) ([8297b2e](https://github.com/n8n-io/n8n/commit/8297b2e6d263d63571926545bf8b5c490f5125bf))
* **editor:** Place the restricted node popover per host, keep one open, and stack the admin dialog ([#40678](https://github.com/n8n-io/n8n/issues/40678)) ([b28dceb](https://github.com/n8n-io/n8n/commit/b28dceb462d8e51855d4e13a87e9c975638ef977))
* **editor:** Polish n8n Assistant and Execute button UI details ([#40414](https://github.com/n8n-io/n8n/issues/40414)) ([76cab1e](https://github.com/n8n-io/n8n/commit/76cab1e26d97cbbf50790926f1bbb14cfb12d078))
* **editor:** Preserve Live Data highlights on scroll in expression editor ([#33842](https://github.com/n8n-io/n8n/issues/33842)) ([0f02b50](https://github.com/n8n-io/n8n/commit/0f02b509be304523955e7700c7c9eeefe58d0aa9))
* **editor:** Remove redundant description for create workflow option in agent config ([#40882](https://github.com/n8n-io/n8n/issues/40882)) ([b542f66](https://github.com/n8n-io/n8n/commit/b542f6611998c23b40b54d25d1e923841ee24160))
* **editor:** Rename the "Executions fail" label to "Executions will fail" ([#40888](https://github.com/n8n-io/n8n/issues/40888)) ([2a51f38](https://github.com/n8n-io/n8n/commit/2a51f38d55f37f65382ebf3900a340fc62b15314))
* **editor:** Respect the default voice for chat read aloud ([#40440](https://github.com/n8n-io/n8n/issues/40440)) ([022b239](https://github.com/n8n-io/n8n/commit/022b23945ba576024685bfe1702489f427a3c9fe))
* **editor:** Shorten the share button label on the credential sharing tab ([#40637](https://github.com/n8n-io/n8n/issues/40637)) ([c133d6c](https://github.com/n8n-io/n8n/commit/c133d6c893a879c907d20bb71a10817d358db553))
* **editor:** Show the Agent artifact while the builder creates it ([#40535](https://github.com/n8n-io/n8n/issues/40535)) ([1c63119](https://github.com/n8n-io/n8n/commit/1c631197563b8f65d343d060355050c4551f381d))
* **editor:** Show the owner and credential type for credentials the user cannot use ([#40669](https://github.com/n8n-io/n8n/issues/40669)) ([63dc285](https://github.com/n8n-io/n8n/commit/63dc2854936f3e282ca422cd9b9babeab65a2375))
* **editor:** Show the renamed agent name in the assistant artifact tab ([#40770](https://github.com/n8n-io/n8n/issues/40770)) ([4032797](https://github.com/n8n-io/n8n/commit/40327977903613fee5d033efd31c6e01416a2bf0))
* **editor:** Show the unavailable credential at the top of the node picker ([#40693](https://github.com/n8n-io/n8n/issues/40693)) ([1933723](https://github.com/n8n-io/n8n/commit/19337237ec59bd03c89e300a23289d2bdfbf269e))
* **editor:** Show why a node on another branch can't be referenced ([#40565](https://github.com/n8n-io/n8n/issues/40565)) ([a195096](https://github.com/n8n-io/n8n/commit/a19509673336fbbf6687b107ce0af5bcfec68393))
* **editor:** Small UI fixes to budget guardrail setup components ([#40383](https://github.com/n8n-io/n8n/issues/40383)) ([958d23d](https://github.com/n8n-io/n8n/commit/958d23d81cd99d7b9fd63d2484b79382c2d20c23))
* **editor:** Use pulsing status dot for the enabled n8n Assistant setting ([#40712](https://github.com/n8n-io/n8n/issues/40712)) ([3ba5229](https://github.com/n8n-io/n8n/commit/3ba522910b1cc3009c108a202634388fd838717f))
* **Google Gemini Chat Model Node:** Point the AI builder to the model catalog for new Gemini models ([#40159](https://github.com/n8n-io/n8n/issues/40159)) ([c21ed4f](https://github.com/n8n-io/n8n/commit/c21ed4f13aee6a1551207a92122bea2859e1b8d3))
* **Invoice Ninja Node:** Fix retrieving currencies for bank transactions ([#12997](https://github.com/n8n-io/n8n/issues/12997)) ([00849c4](https://github.com/n8n-io/n8n/commit/00849c4deabada87641c146773c0981e873a6328))
* Keep execution details clear of preview canvas ([#40176](https://github.com/n8n-io/n8n/issues/40176)) ([af98327](https://github.com/n8n-io/n8n/commit/af98327a5e4f2332b78b4d460e6eb208f0782d28))
* Load only the publish history that the editor shows ([#40183](https://github.com/n8n-io/n8n/issues/40183)) ([b28adf9](https://github.com/n8n-io/n8n/commit/b28adf97895bbdba2e0c777bdfb5cd11d04c0e56))
* **Microsoft Outlook Trigger Node:** Report failed polls and limit the backlog per poll ([#40147](https://github.com/n8n-io/n8n/issues/40147)) ([6bc5c93](https://github.com/n8n-io/n8n/commit/6bc5c93513efbe4d236c5794cfd07e1ac4d916f6))
* Multiple migration timestamps withing a single PR ([#40379](https://github.com/n8n-io/n8n/issues/40379)) ([7db7fa4](https://github.com/n8n-io/n8n/commit/7db7fa42c80a8a24fb5ca5ec7448ffb12431fad9))
* **Odoo Node:** Normalize datetime fields to Odoo format before write ([#34186](https://github.com/n8n-io/n8n/issues/34186)) ([6b77613](https://github.com/n8n-io/n8n/commit/6b77613ee827560d9f34f551478a74daf4848dec))
* Prevent node types corruption when multiple loaders run at the same time ([#40633](https://github.com/n8n-io/n8n/issues/40633)) ([fdc3948](https://github.com/n8n-io/n8n/commit/fdc394819cecf96379ed7ba475364912099a6296))
* Remove loading flashes from data tables ([#39874](https://github.com/n8n-io/n8n/issues/39874)) ([2f4a386](https://github.com/n8n-io/n8n/commit/2f4a3868fbd74c07e91888eb35b1f3d4bcb2acb1))
* Report resource-operation violations as errors ([#40830](https://github.com/n8n-io/n8n/issues/40830)) ([a180010](https://github.com/n8n-io/n8n/commit/a1800105ef8f7c229ffd044e1233d273e95b6cd2))
* Return UTC ISO dates without changing other cells ([#40156](https://github.com/n8n-io/n8n/issues/40156)) ([8059b63](https://github.com/n8n-io/n8n/commit/8059b63da89ba89309cbc6eacce715110c9c5748))
* **S3 Node:** Percent-encode the copy source path ([#40282](https://github.com/n8n-io/n8n/issues/40282)) ([81f34c2](https://github.com/n8n-io/n8n/commit/81f34c23744424d17ca99eaf03beb9861c21a1ca))
* Separate agent response steps and show saved reasoning ([#40590](https://github.com/n8n-io/n8n/issues/40590)) ([1e6da4e](https://github.com/n8n-io/n8n/commit/1e6da4e615900d7665e6eeafe9ce6002d1510c31))
* Update matching nodes when selecting imported credentials ([#39650](https://github.com/n8n-io/n8n/issues/39650)) ([4b5b68e](https://github.com/n8n-io/n8n/commit/4b5b68ee2929344288b93469efe732379e3904a1))
* Use configured OAuth server rate limits ([#40542](https://github.com/n8n-io/n8n/issues/40542)) ([4c8b76b](https://github.com/n8n-io/n8n/commit/4c8b76b6f08d090bf6e7cf22f66d60ec3d0efdc6))
* Use ID token claims when UserInfo is unavailable ([#40779](https://github.com/n8n-io/n8n/issues/40779)) ([b8b0387](https://github.com/n8n-io/n8n/commit/b8b0387a31b556e296529235f9c4a75b307bb8f1))
* Validate and default public API pagination limits ([#40483](https://github.com/n8n-io/n8n/issues/40483)) ([f212d5b](https://github.com/n8n-io/n8n/commit/f212d5b900fe92c3012001b32f4cf17bd831921e))
* Wait for MockServer HTTP status without following logs ([#40846](https://github.com/n8n-io/n8n/issues/40846)) ([4593838](https://github.com/n8n-io/n8n/commit/45938389b86b6bfe25a8a8f3d132b600ee807f58))
### Features
* Add Antigravity to the MCP client connection dialog ([#40879](https://github.com/n8n-io/n8n/issues/40879)) ([f0a38cf](https://github.com/n8n-io/n8n/commit/f0a38cfe5801ba8d62b97f6d4ffeb031f23aa231))
* Add Google Vertex AI credentials ([#40665](https://github.com/n8n-io/n8n/issues/40665)) ([a7cba90](https://github.com/n8n-io/n8n/commit/a7cba90a854ea34198dd9c1fcb4f7eeae08c364d))
* **ai-builder:** Reopen closed AI Assistant tabs when the agent changes the artifact ([#40081](https://github.com/n8n-io/n8n/issues/40081)) ([5d38667](https://github.com/n8n-io/n8n/commit/5d3866775c828e766ca16642d5ae2a340d4133ca))
* **Azure AI Foundry Chat Model Node:** Speak the Anthropic Messages schema for Claude deployments ([#40306](https://github.com/n8n-io/n8n/issues/40306)) ([c134edf](https://github.com/n8n-io/n8n/commit/c134edf473276ff63387d28aea0d08c9ff676f7c))
* **Azure OpenAI Chat Model Node:** Rename the Foundry nodes and credentials to Microsoft Foundry ([#40398](https://github.com/n8n-io/n8n/issues/40398)) ([e5c8a8b](https://github.com/n8n-io/n8n/commit/e5c8a8bb62d2ce468d1cda3965562eeaa69996f8))
* Configure workflow tool inputs for Agents ([#40285](https://github.com/n8n-io/n8n/issues/40285)) ([4cb98ae](https://github.com/n8n-io/n8n/commit/4cb98aea9fbcede8ae414c3f817c7b0490d401f8))
* **core:** Add credentialTransfer policy enforcement point ([#40300](https://github.com/n8n-io/n8n/issues/40300)) ([c2f6920](https://github.com/n8n-io/n8n/commit/c2f6920794e66e7108b0005233506f7694a11a7e))
* **core:** Add execution identity to every OTel span ([#40649](https://github.com/n8n-io/n8n/issues/40649)) ([22ba4e7](https://github.com/n8n-io/n8n/commit/22ba4e7bc5c6fbe5243ff5c227c718ab7197c5e1))
* **core:** Add overwrite schema conflict policy to data table package import ([#40345](https://github.com/n8n-io/n8n/issues/40345)) ([3af9817](https://github.com/n8n-io/n8n/commit/3af98170d0d98a201df1a45f2ecee17f5f00f298))
* **core:** Add overwrite-non-destructive schema conflict policy to data table package import ([#40446](https://github.com/n8n-io/n8n/issues/40446)) ([dc33c29](https://github.com/n8n-io/n8n/commit/dc33c29d1af3f47a6179a76ff21ccc7bd1f75357))
* **core:** Add Prometheus metrics for database connection recovery ([#39890](https://github.com/n8n-io/n8n/issues/39890)) ([d904e7a](https://github.com/n8n-io/n8n/commit/d904e7a2476794acdf3928faacd86ae9b3e45220))
* **core:** Add the WhatsApp agent channel ([#38750](https://github.com/n8n-io/n8n/issues/38750)) ([b0d1a30](https://github.com/n8n-io/n8n/commit/b0d1a30f50fdf6b3720c2a5773e28efff5c5b0b5))
* **core:** Add workflow suggestion review actions ([#39858](https://github.com/n8n-io/n8n/issues/39858)) ([eaf8c25](https://github.com/n8n-io/n8n/commit/eaf8c25d1c2b680fe55f58fc9f6d10647e6ae640))
* **core:** Allow confirming destructive data table changes when continuing an apply ([#40491](https://github.com/n8n-io/n8n/issues/40491)) ([9946cfe](https://github.com/n8n-io/n8n/commit/9946cfe901b538ec5ca29e85cde3ec2044d33d00))
* **core:** Apply the member credential rule to Owners and admins in team projects ([#40448](https://github.com/n8n-io/n8n/issues/40448)) ([3cff8dc](https://github.com/n8n-io/n8n/commit/3cff8dc4ecac5fd173d198f3e319ab473cdd7b22))
* **core:** Block create/update of workflows with deprecated nodes ([#31360](https://github.com/n8n-io/n8n/issues/31360)) ([ea8417e](https://github.com/n8n-io/n8n/commit/ea8417e29ad6f8b981d9a04b17283d5d77f2b5bb))
* **core:** Email the agent owner when a monthly budget alert is crossed ([#40544](https://github.com/n8n-io/n8n/issues/40544)) ([19e1f70](https://github.com/n8n-io/n8n/commit/19e1f70ea62ad151abe18b5a25721caa35d7bedd))
* **core:** Emit opt-in start marker spans for workflow and node executions ([#40762](https://github.com/n8n-io/n8n/issues/40762)) ([c713545](https://github.com/n8n-io/n8n/commit/c7135451ebd552267aedf4898d845c53a97aab54))
* **core:** Experiment to show assistant composer and starter templates in agent list ([#40396](https://github.com/n8n-io/n8n/issues/40396)) ([fe26ba8](https://github.com/n8n-io/n8n/commit/fe26ba8d3a63963d6c7868338e73fb77378e94a9))
* **core:** Handle inbound media handling for whatsapp ([#39409](https://github.com/n8n-io/n8n/issues/39409)) ([50c2ca0](https://github.com/n8n-io/n8n/commit/50c2ca0a27189136d82339bdafd5d8c7513a9916))
* **core:** Handle message quality rate limits for whatsapp integration ([#39205](https://github.com/n8n-io/n8n/issues/39205)) ([3c33856](https://github.com/n8n-io/n8n/commit/3c33856997b084a3a87038c0d72f80d9b91af1b7))
* **core:** Hide credential-only nodes under a credential type policy ([#39652](https://github.com/n8n-io/n8n/issues/39652)) ([370b30f](https://github.com/n8n-io/n8n/commit/370b30f9195ba8d5ceaf75fb4cdfcf127105a3c6))
* **core:** Pair data table columns by id on package import ([#40709](https://github.com/n8n-io/n8n/issues/40709)) ([27b43de](https://github.com/n8n-io/n8n/commit/27b43de85f987ddd783e7c7ccbde7977d09a6d03))
* **core:** Persist self-healing results and handoffs ([#40115](https://github.com/n8n-io/n8n/issues/40115)) ([168a0ac](https://github.com/n8n-io/n8n/commit/168a0ac86056afd743bfde63863abbab4e8515f4))
* **core:** Post a budget-stop message on external agent channels ([#40767](https://github.com/n8n-io/n8n/issues/40767)) ([5e01906](https://github.com/n8n-io/n8n/commit/5e01906a1e6c177c0ce02d18317ce2d7b6ee7e97))
* **core:** Promote folder and project renames alongside selected workflows ([#40502](https://github.com/n8n-io/n8n/issues/40502)) ([a249e20](https://github.com/n8n-io/n8n/commit/a249e202905f30829fe3a7eacd2eba8dff6573ef))
* **core:** Report crashed executions from startup log recovery ([#40390](https://github.com/n8n-io/n8n/issues/40390)) ([668e71d](https://github.com/n8n-io/n8n/commit/668e71d54a26da7c12d092a7375a7cb3349a8471))
* **core:** Report destructive data table changes in the promotion apply preflight ([#40454](https://github.com/n8n-io/n8n/issues/40454)) ([5d208c8](https://github.com/n8n-io/n8n/commit/5d208c85ed275c22cc5597124b7a9cf590d18f11))
* **core:** Report the deployment artifact in telemetry ([#40319](https://github.com/n8n-io/n8n/issues/40319)) ([344dea5](https://github.com/n8n-io/n8n/commit/344dea5c77fc5f61ce10171a2553faa9bef4e609))
* **core:** Run the workflow history trim as a durable daily system task ([#40085](https://github.com/n8n-io/n8n/issues/40085)) ([38eea65](https://github.com/n8n-io/n8n/commit/38eea65993339ea3e8156cdbacef247749331351))
* **core:** Search, create and reopen agents from the command bar ([#40504](https://github.com/n8n-io/n8n/issues/40504)) ([75a9cc7](https://github.com/n8n-io/n8n/commit/75a9cc7e1219d35b441b02c20446dd89db186ff7))
* **core:** Serve Prometheus metrics while the database is unavailable ([#39908](https://github.com/n8n-io/n8n/issues/39908)) ([4e217ae](https://github.com/n8n-io/n8n/commit/4e217aeac6b029692fb8a692a3849ae8e5310b5a))
* **core:** Serve the instance CSP on all non-script responses ([#40205](https://github.com/n8n-io/n8n/issues/40205)) ([cabf0bf](https://github.com/n8n-io/n8n/commit/cabf0bf3b466a725deada3b532347b7db530dcac))
* **core:** Show every user the migration report, scoped to the workflows they can edit ([#40449](https://github.com/n8n-io/n8n/issues/40449)) ([f4f7dcf](https://github.com/n8n-io/n8n/commit/f4f7dcf13b6d93d46a2860e2c2577f9f6dda4c52))
* **core:** Show the agent chat attach button on the left of the composer ([#40320](https://github.com/n8n-io/n8n/issues/40320)) ([209adf9](https://github.com/n8n-io/n8n/commit/209adf9a7ee9e71cfe3d5119316ca9fce4b3ac68))
* **Databricks Node:** Guard the Vector Store insert against the wrong index ([#39699](https://github.com/n8n-io/n8n/issues/39699)) ([0932e95](https://github.com/n8n-io/n8n/commit/0932e9572e2650165b260fc591d1f02204c17aae))
* **editor:** Add a state column to the migration rule detail ([#40388](https://github.com/n8n-io/n8n/issues/40388)) ([682d092](https://github.com/n8n-io/n8n/commit/682d0920ba5cb2ab5e3ac0671f993c8e529e8389))
* **editor:** Add agent setup checklist ([#39919](https://github.com/n8n-io/n8n/issues/39919)) ([883ade4](https://github.com/n8n-io/n8n/commit/883ade4b9d2505f359057eb10c8d71ab153033a7))
* **editor:** Add order to settings sidebar links ([#40505](https://github.com/n8n-io/n8n/issues/40505)) ([cef87cf](https://github.com/n8n-io/n8n/commit/cef87cfcab3f59a657c8c27309f4807bd2d4a93a))
* **editor:** Add project selection to Vertex AI credentials ([#40667](https://github.com/n8n-io/n8n/issues/40667)) ([cdc23c6](https://github.com/n8n-io/n8n/commit/cdc23c666ffd6ed55f019aa24c99efdaf10fad14))
* **editor:** Animate the n8n Assistant activity indicator ([#40497](https://github.com/n8n-io/n8n/issues/40497)) ([04e1e7f](https://github.com/n8n-io/n8n/commit/04e1e7fc4771ff1371ac41f1d74aef7ced26b175))
* **editor:** Disable restricted model providers in the agent model selector ([#40658](https://github.com/n8n-io/n8n/issues/40658)) ([cde7a9d](https://github.com/n8n-io/n8n/commit/cde7a9dbd985ae9e83de2d39dccc508a41d9a676))
* **editor:** Explain a restricted model provider with the restriction popover ([#40705](https://github.com/n8n-io/n8n/issues/40705)) ([75de193](https://github.com/n8n-io/n8n/commit/75de193d9090d160dd191667adb85dc423951045))
* **editor:** Explain instance-role access in the credential sharing "Used in" row ([#40555](https://github.com/n8n-io/n8n/issues/40555)) ([48f05f9](https://github.com/n8n-io/n8n/commit/48f05f92fe9e7bd1c1d220a53a8c17055a17e70c))
* **editor:** Rebuild the command bar with type tabs and paged search ([#40461](https://github.com/n8n-io/n8n/issues/40461)) ([e36093b](https://github.com/n8n-io/n8n/commit/e36093b11db0580add10306d27d090a27790f4fb))
* **editor:** Rebuild the personal settings page with the design-system settings components ([#40298](https://github.com/n8n-io/n8n/issues/40298)) ([09436b7](https://github.com/n8n-io/n8n/commit/09436b78f9f6f320073f911ca11160ad9a66d94f))
* **editor:** Recommend Claude Opus 5.5 and GPT-6 models in AI Assistant setup ([#40660](https://github.com/n8n-io/n8n/issues/40660)) ([1ad3c54](https://github.com/n8n-io/n8n/commit/1ad3c54803569dd83976c055785b51b62555bf44))
* **editor:** Replace Windsurf with Devin in the MCP client dialog ([#40912](https://github.com/n8n-io/n8n/issues/40912)) ([e2c4054](https://github.com/n8n-io/n8n/commit/e2c4054cba14a2f1dc29215fc623f6ae42b7a238))
* **editor:** Show the restriction callout and lock the agent tool configuration form ([#40657](https://github.com/n8n-io/n8n/issues/40657)) ([01da889](https://github.com/n8n-io/n8n/commit/01da88958d490e5006874cce2b05554347301b9b))
* **editor:** Show who created each preference on the Preferences page ([#40868](https://github.com/n8n-io/n8n/issues/40868)) ([22ff2a0](https://github.com/n8n-io/n8n/commit/22ff2a06368b49fa25b0ad1c2cf74640c56a0284))
* **editor:** Sort the credential sharing list by people, then projects ([#40478](https://github.com/n8n-io/n8n/issues/40478)) ([02af7e7](https://github.com/n8n-io/n8n/commit/02af7e7659cce85749c719bb6a1b910043357baf))
* **Embeddings Azure OpenAI Node:** Add Entra ID authentication ([#39390](https://github.com/n8n-io/n8n/issues/39390)) ([830486a](https://github.com/n8n-io/n8n/commit/830486a851ceb3eeada1eb945e7cd11c05262901))
* **Google Gemini Node:** Show Nano Banana 2.1 and 2 Lite in the image model list ([#40596](https://github.com/n8n-io/n8n/issues/40596)) ([97c54f6](https://github.com/n8n-io/n8n/commit/97c54f618c6210bbe1dcfadaac702d40e89a820b))
* Make health check timeout configurable ([#39548](https://github.com/n8n-io/n8n/issues/39548)) ([df8605e](https://github.com/n8n-io/n8n/commit/df8605e8750443d66388060212306cd205a4f5d3))
* **Microsoft SharePoint Trigger Node:** Add the Microsoft SharePoint Trigger node ([#40055](https://github.com/n8n-io/n8n/issues/40055)) ([7c126fa](https://github.com/n8n-io/n8n/commit/7c126fab32772241473e1f180c6806af4ce189f8))
### Performance Improvements
* **core:** Keep agent builder system prompt static for prompt caching ([#40364](https://github.com/n8n-io/n8n/issues/40364)) ([ee1db0b](https://github.com/n8n-io/n8n/commit/ee1db0bc12bebb572afd04e6e53efc30ffa01209))
* **core:** Let the agent builder chain config writes without re-reading the config ([#40462](https://github.com/n8n-io/n8n/issues/40462)) ([6b4da93](https://github.com/n8n-io/n8n/commit/6b4da93dcbb909bcf86c03251e3c74033d0dd70f))
* **core:** Let the agent builder edit skill instructions in place ([#40447](https://github.com/n8n-io/n8n/issues/40447)) ([1de45a1](https://github.com/n8n-io/n8n/commit/1de45a1a9b0b0db7036cb25e27bffc95a7df41e2))
* **core:** Prepare Instance AI skill sandbox in the background ([#40363](https://github.com/n8n-io/n8n/issues/40363)) ([b36c2b5](https://github.com/n8n-io/n8n/commit/b36c2b506b92f391f859ec10067e57d96c8b09c8))
* **core:** Send agent builder config writes as objects with changed fields only ([#40666](https://github.com/n8n-io/n8n/issues/40666)) ([2aa3fbf](https://github.com/n8n-io/n8n/commit/2aa3fbf18547cfb6719baabdcd8719fa6d0f53af))
# [2.43.0](https://github.com/n8n-io/n8n/compare/n8n@2.42.0...n8n@2.43.0) (2026-10-06)

202
OWNERS
View File

@ -30,6 +30,7 @@ packages/@n8n/backend-network/ @n8n-io/catalysts
packages/@n8n/backend-services/ @n8n-io/catalysts
packages/@n8n/backend-services/src/services/role.service.ts @n8n-io/iam
packages/@n8n/backend-services/src/services/role-cache.service.ts @n8n-io/iam
packages/@n8n/backend-services/src/services/jwt.service.ts @n8n-io/iam
packages/@n8n/backend-test-utils/ @n8n-io/catalysts
packages/@n8n/di/ @n8n-io/catalysts
packages/@n8n/errors/ @n8n-io/catalysts
@ -52,21 +53,20 @@ packages/@n8n/db/src/migrations/ @n8n-io/migrations-revie
# Top-level paths
scripts/ @n8n-io/qa-dx
patches/ @n8n-io/qa-dx
assets/ @n8n-io/adore
security/ @n8n-io/qa-dx
.code-health-baseline.json @n8n-io/qa-dx required
OWNERS @n8n-io/qa-dx required
# @n8n/cli
packages/@n8n/cli/ @n8n-io/adore
packages/@n8n/cli/ @n8n-io/context
packages/@n8n/cli/src/commands/credential/ @n8n-io/iam
packages/@n8n/cli/src/commands/user/ @n8n-io/iam
packages/@n8n/cli/src/commands/data-table/ @n8n-io/adore
packages/@n8n/cli/src/commands/tag/ @n8n-io/adore
packages/@n8n/cli/src/commands/data-table/ @n8n-io/knowledge
packages/@n8n/cli/src/commands/tag/ @n8n-io/context
packages/@n8n/cli/src/commands/project/ @n8n-io/ligo
packages/@n8n/cli/src/commands/source-control/ @n8n-io/ligo
packages/@n8n/cli/src/commands/variable/ @n8n-io/ligo
packages/@n8n/cli/src/commands/skill/ @n8n-io/ai
packages/@n8n/cli/src/commands/skill/ @n8n-io/context
# packages/cli
packages/cli/ @n8n-io/catalysts
@ -85,7 +85,6 @@ packages/cli/src/security-audit/ @n8n-io/catalysts
packages/cli/src/modules/workflow-index/ @n8n-io/catalysts
packages/cli/src/modules/breaking-changes/ @n8n-io/catalysts
packages/cli/src/workflows/publication/ @n8n-io/catalysts required
packages/cli/src/modules/otel/ @n8n-io/ligo
packages/cli/src/auth/ @n8n-io/iam
packages/cli/src/credentials/ @n8n-io/iam
@ -109,23 +108,25 @@ packages/cli/src/environments.ee/ @n8n-io/ligo
packages/cli/src/public-api/ @n8n-io/ligo
packages/cli/src/modules/source-control.ee/ @n8n-io/ligo
packages/cli/src/modules/external-secrets.ee/ @n8n-io/ligo
packages/cli/src/modules/insights/ @n8n-io/ligo
packages/modules/insights/ @n8n-io/ligo
packages/cli/src/collaboration/ @n8n-io/catalysts
packages/cli/src/binary-data/ @n8n-io/catalysts
packages/cli/src/posthog/ @n8n-io/adore
packages/cli/src/modules/data-table/ @n8n-io/adore
packages/cli/src/posthog/ @n8n-io/growth
packages/cli/src/modules/data-table/ @n8n-io/knowledge
packages/cli/src/modules/knowledge/ @n8n-io/knowledge
packages/cli/src/evaluation.ee/ @n8n-io/ai
packages/cli/src/chat/ @n8n-io/ai
packages/cli/src/evaluation.ee/ @n8n-io/ai-trust
packages/cli/src/chat/ @n8n-io/ai-agents
packages/cli/src/modules/agents/ @n8n-io/ai-agents
packages/cli/src/tool-generation/ @n8n-io/ai
packages/cli/src/modules/workflow-builder/ @n8n-io/ai
packages/cli/src/modules/mcp/ @n8n-io/adore
packages/cli/src/modules/quick-connect/ @n8n-io/ai
packages/modules/workflow-builder/ @n8n-io/ai-assistant
packages/cli/src/modules/mcp/ @n8n-io/context
packages/cli/src/modules/skills/ @n8n-io/context
packages/cli/test/integration/skills/ @n8n-io/context
packages/cli/src/modules/chat-hub/ @n8n-io/ai-assistant
packages/cli/src/modules/instance-ai/ @n8n-io/ai-assistant
packages/cli/src/modules/community-packages/ @n8n-io/nodes
packages/modules/inbox/ @n8n-io/ligo
# CLI controllers
packages/cli/src/controllers/auth.controller.ts @n8n-io/iam
@ -140,14 +141,14 @@ packages/cli/src/controllers/user-settings.controller.ts @n8n-io/iam
packages/cli/src/controllers/api-keys.controller.ts @n8n-io/iam
packages/cli/src/controllers/security-settings.controller.ts @n8n-io/iam
packages/cli/src/controllers/oauth/ @n8n-io/iam
packages/cli/src/controllers/ai.controller.ts @n8n-io/ai
packages/cli/src/controllers/ai.controller.ts @n8n-io/ai-assistant
packages/cli/src/controllers/annotation-tags.controller.ee.ts @n8n-io/ai
packages/cli/src/controllers/cta.controller.ts @n8n-io/adore
packages/cli/src/controllers/folder.controller.ts @n8n-io/adore
packages/cli/src/controllers/tags.controller.ts @n8n-io/adore
packages/cli/src/controllers/binary-data.controller.ts @n8n-io/adore
packages/cli/src/controllers/posthog.controller.ts @n8n-io/adore
packages/cli/src/controllers/translation.controller.ts @n8n-io/adore
packages/cli/src/controllers/cta.controller.ts @n8n-io/knowledge
packages/cli/src/controllers/folder.controller.ts @n8n-io/context
packages/cli/src/controllers/tags.controller.ts @n8n-io/context
packages/cli/src/controllers/binary-data.controller.ts @n8n-io/knowledge
packages/cli/src/controllers/posthog.controller.ts @n8n-io/context
packages/cli/src/controllers/translation.controller.ts @n8n-io/catalysts
packages/cli/src/controllers/project.controller.ts @n8n-io/ligo
packages/cli/src/controllers/workflow-statistics.controller.ts @n8n-io/ligo
packages/cli/src/controllers/node-types.controller.ts @n8n-io/nodes
@ -155,22 +156,23 @@ packages/cli/src/controllers/dynamic-node-parameters.controller.ts @n8n-io/node
packages/cli/src/controllers/e2e.controller.ts @n8n-io/qa-dx
# CLI services
packages/cli/src/services/jwt.service.ts @n8n-io/iam
packages/cli/src/services/user.service.ts @n8n-io/iam
packages/cli/src/services/password.utility.ts @n8n-io/iam
packages/cli/src/services/public-api-key.service.ts @n8n-io/iam
packages/cli/src/services/security-settings.service.ts @n8n-io/iam
packages/cli/src/services/static-auth-service.ts @n8n-io/iam
packages/cli/src/services/access.service.ts @n8n-io/iam
packages/cli/src/services/ai.service.ts @n8n-io/ai
packages/cli/src/services/ai-usage.service.ts @n8n-io/ai
packages/cli/src/services/ai-workflow-builder.service.ts @n8n-io/ai
packages/cli/src/services/annotation-tag.service.ee.ts @n8n-io/ai
packages/cli/src/services/folder.service.ts @n8n-io/adore
packages/cli/src/services/tag.service.ts @n8n-io/adore
packages/cli/src/services/cta.service.ts @n8n-io/adore
packages/cli/src/services/frontend.service.ts @n8n-io/adore
packages/cli/src/services/banner.service.ts @n8n-io/adore
packages/cli/src/services/ai.service.ts @n8n-io/ai-assistant
packages/cli/src/services/ai-usage.service.ts @n8n-io/ai-assistant
packages/cli/src/services/__tests__/ai-usage.service.test.ts @n8n-io/ai-assistant
packages/cli/src/controllers/__tests__/ai.controller.test.ts @n8n-io/ai-assistant
packages/cli/src/services/ai-workflow-builder.service.ts @n8n-io/ai-assistant
packages/cli/src/services/annotation-tag.service.ee.ts @n8n-io/context
packages/cli/src/services/folder.service.ts @n8n-io/context
packages/cli/src/services/tag.service.ts @n8n-io/context
packages/cli/src/services/cta.service.ts @n8n-io/knowledge
packages/cli/src/services/frontend.service.ts @n8n-io/catalysts
packages/cli/src/services/banner.service.ts @n8n-io/context
packages/cli/src/services/project.service.ee.ts @n8n-io/ligo
packages/cli/src/services/workflow-statistics.service.ts @n8n-io/ligo
packages/cli/src/services/export.service.ts @n8n-io/ligo
@ -178,11 +180,11 @@ packages/cli/src/services/import.service.ts @n8n-io/ligo
packages/cli/src/services/ownership.service.ts @n8n-io/ligo
packages/cli/src/services/dynamic-node-parameters.service.ts @n8n-io/nodes
# Adore
packages/frontend/editor-ui/ @n8n-io/frontend
packages/frontend/editor-ui/src/features/ai/ @n8n-io/ai
packages/frontend/editor-ui/src/features/ai/assistant/ @n8n-io/ai-assistant
packages/frontend/editor-ui/src/features/ai/chatHub/ @n8n-io/ai-assistant
packages/frontend/editor-ui/src/features/ai/instanceAi/ @n8n-io/ai-assistant
packages/frontend/editor-ui/src/features/credentials/ @n8n-io/iam
packages/frontend/editor-ui/src/features/execution/ @n8n-io/ligo
packages/frontend/editor-ui/src/features/integrations/ @n8n-io/ligo
@ -195,15 +197,42 @@ packages/frontend/@n8n/storybook/ @n8n-io/design-engineeri
packages/frontend/@n8n/i18n/ @n8n-io/frontend
packages/@n8n/stylelint-config/ @n8n-io/qa-dx
packages/frontend/editor-ui/src/features/ai/gateway/ @n8n-io/relay
packages/frontend/editor-ui/src/app/composables/useAiGatewayTopUp.ts @n8n-io/relay
packages/frontend/editor-ui/src/app/composables/useFreeAiCredits.ts @n8n-io/relay
packages/frontend/editor-ui/src/features/settings/communityNodes/ @n8n-io/nodes
packages/frontend/editor-ui/src/features/credentials/quickConnect/ @n8n-io/nodes required
packages/frontend/editor-ui/src/features/ndv/runData/schemaPreview.api.ts @n8n-io/nodes
packages/frontend/editor-ui/src/features/ndv/runData/schemaPreview.store.ts @n8n-io/nodes
packages/frontend/editor-ui/src/features/agents/channels/ @n8n-io/nodes
# AI
packages/@n8n/instance-ai/ @n8n-io/ai-assistant
packages/@n8n/nodes-langchain/ @n8n-io/ai
packages/@n8n/ai-utilities/ @n8n-io/ai
packages/@n8n/ai-node-sdk/ @n8n-io/ai
packages/@n8n/ai-workflow-builder.ee/ @n8n-io/ai
packages/@n8n/agents/ @n8n-io/ai
packages/frontend/@n8n/chat/ @n8n-io/ai
packages/@n8n/nodes-langchain/nodes/agents/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/chains/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/code/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/output_parser/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/llms/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/vendors/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/ModelSelector/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/embeddings/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/vector_store/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/document_loaders/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/text_splitters/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/retrievers/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/rerankers/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/memory/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/tools/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/ToolExecutor/ @n8n-io/ai-agents
packages/@n8n/nodes-langchain/nodes/trigger/ @n8n-io/ai-agents
packages/@n8n/ai-utilities/ @n8n-io/ai-agents
packages/@n8n/ai-workflow-builder.ee/ @n8n-io/ai-assistant
packages/@n8n/agents/ @n8n-io/ai-agents
packages/frontend/@n8n/chat/ @n8n-io/ai-agents
packages/frontend/editor-ui/src/features/agents/ @n8n-io/ai-agents
# Chat Hub
@ -218,6 +247,7 @@ packages/@n8n/nodes-langchain/nodes/vector_store/shared/chatHub.ts @n8n-io/ai-as
packages/@n8n/nodes-langchain/credentials/ChatHubVectorStorePGVectorApi.credentials.ts @n8n-io/ai-assistant
packages/@n8n/nodes-langchain/credentials/ChatHubVectorStorePineconeApi.credentials.ts @n8n-io/ai-assistant
packages/@n8n/nodes-langchain/credentials/ChatHubVectorStoreQdrantApi.credentials.ts @n8n-io/ai-assistant
packages/@n8n/nodes-langchain/nodes/mcp/ @n8n-io/nodes required
packages/frontend/editor-ui/src/features/shared/commandBar/composables/useChatHubCommands.ts @n8n-io/ai-assistant
packages/cli/src/modules/breaking-changes/rules/v3/chat-hub-deprecated.rule.ts @n8n-io/ai-assistant
@ -227,37 +257,89 @@ packages/@n8n/codemirror-lang/ @n8n-io/nodes
packages/@n8n/codemirror-lang-html/ @n8n-io/nodes
packages/@n8n/codemirror-lang-sql/ @n8n-io/nodes
packages/nodes-base/ @n8n-io/nodes
packages/nodes-base/nodes/Code/ @n8n-io/nodes required
packages/nodes-base/nodes/ExecuteCommand/ @n8n-io/nodes required
packages/nodes-base/nodes/Ssh/ @n8n-io/nodes required
packages/nodes-base/nodes/HttpRequest/ @n8n-io/nodes required
packages/nodes-base/nodes/Webhook/ @n8n-io/nodes required
packages/nodes-base/nodes/RespondToWebhook/ @n8n-io/nodes required
packages/nodes-base/nodes/Form/ @n8n-io/nodes required
packages/nodes-base/nodes/Wait/ @n8n-io/nodes required
packages/nodes-base/utils/sendAndWait/ @n8n-io/nodes required
packages/nodes-base/nodes/Git/ @n8n-io/nodes required
packages/@n8n/decorators/ @n8n-io/catalysts
packages/node-dev/ @n8n-io/nodes
packages/@n8n/create-node/ @n8n-io/nodes
packages/@n8n/node-cli/ @n8n-io/community-engineering
packages/@n8n/imap/ @n8n-io/iam
packages/@n8n/syslog-client/ @n8n-io/iam
packages/@n8n/scan-community-package/ @n8n-io/community-engineering
packages/@n8n/eslint-plugin-community-nodes/ @n8n-io/community-engineering
packages/@n8n/computer-use/ @n8n-io/nodes
packages/@n8n/local-gateway/ @n8n-io/nodes
packages/@n8n/mcp-browser/ @n8n-io/nodes
packages/@n8n/mcp-browser-extension/ @n8n-io/nodes
packages/@n8n/imap/ @n8n-io/nodes
packages/@n8n/computer-use/ @n8n-io/nodes required
packages/@n8n/local-gateway/ @n8n-io/nodes required
packages/@n8n/mcp-browser/ @n8n-io/nodes required
packages/@n8n/mcp-browser-extension/ @n8n-io/nodes required
packages/modules/quick-connect/ @n8n-io/nodes required
packages/@n8n/api-types/src/dto/quick-connect/ @n8n-io/nodes required
packages/@n8n/api-types/src/quick-connect.ts @n8n-io/nodes required
packages/cli/src/modules/community-packages/ @n8n-io/nodes
packages/cli/src/modules/mcp-registry/ @n8n-io/nodes required
packages/cli/src/modules/agents/integrations/ @n8n-io/nodes
packages/@n8n/api-types/src/schemas/mcp-registry.schema.ts @n8n-io/nodes required
packages/cli/templates/send-and-wait-no-action-required.handlebars @n8n-io/nodes required
packages/@n8n/ai-node-sdk/ @n8n-io/nodes required
# Relay (AI gateway & credits)
packages/cli/src/services/ai-gateway.service.ts @n8n-io/relay required
packages/cli/src/services/ai-gateway-eligibility.ts @n8n-io/relay required
packages/cli/src/services/free-ai-credits.service.ts @n8n-io/relay required
packages/@n8n/config/src/configs/ai-gateway.config.ts @n8n-io/relay
packages/@n8n/api-types/src/dto/ai/ai-gateway-config-response.dto.ts @n8n-io/relay
packages/@n8n/api-types/src/dto/ai/ai-gateway-usage-response.dto.ts @n8n-io/relay
packages/@n8n/api-types/src/dto/ai/ai-gateway-wallet-response.dto.ts @n8n-io/relay
packages/cli/src/modules/instance-ai/instance-ai-gateway.service.ts @n8n-io/relay
packages/cli/src/modules/mcp/mcp-ai-gateway.helper.ts @n8n-io/relay
packages/cli/src/modules/agents/json-config/node-tool-ai-gateway.service.ts @n8n-io/relay
packages/frontend/editor-ui/src/app/stores/aiGateway.store.ts @n8n-io/relay
packages/frontend/editor-ui/src/app/composables/useAiGateway.ts @n8n-io/relay
packages/frontend/editor-ui/src/features/credentials/gatewayCreditsPromotion/ @n8n-io/relay
packages/frontend/editor-ui/src/features/shared/nodeCreator/components/SectionHeaderCreditsTag.vue @n8n-io/relay
# Community engineering
.agents/skills/create-community-node-lint-rule/ @n8n-io/community-engineering
.agents/skills/community-pr-readiness-check/ @n8n-io/community-engineering
packages/@n8n/scan-community-package/ @n8n-io/community-engineering
packages/@n8n/eslint-plugin-community-nodes/ @n8n-io/community-engineering
packages/@n8n/node-cli/ @n8n-io/community-engineering
packages/@n8n/create-node/ @n8n-io/community-engineering
# IAM
packages/@n8n/permissions/ @n8n-io/iam
packages/@n8n/client-oauth2/ @n8n-io/iam
packages/@n8n/inbound-auth/ @n8n-io/iam
# LiGo
packages/extensions/insights/ @n8n-io/ligo
packages/@n8n/syslog-client/ @n8n-io/iam
# Governance & Observability
packages/modules/type-availability-policies/ @n8n-io/governance-and-observability
packages/cli/src/modules/type-availability-policies/ @n8n-io/governance-and-observability
packages/modules/otel/ @n8n-io/governance-and-observability
packages/cli/src/modules/otel/ @n8n-io/governance-and-observability
packages/cli/src/public-api/v1/controllers/otel.public.controller.ts @n8n-io/governance-and-observability
packages/cli/src/public-api/v1/shared/otel.mapper.ts @n8n-io/governance-and-observability
packages/cli/src/public-api/v1/controllers/node-type-policies.public.controller.ts @n8n-io/governance-and-observability
packages/cli/src/public-api/v1/controllers/credential-type-policies.public.controller.ts @n8n-io/governance-and-observability
packages/cli/src/public-api/v1/handlers/node-type-policies/ @n8n-io/governance-and-observability
packages/cli/src/public-api/v1/handlers/credential-type-policies/ @n8n-io/governance-and-observability
packages/cli/src/modules/policy-infrastructure/ @n8n-io/governance-and-observability
packages/cli/src/policy/ @n8n-io/governance-and-observability
packages/@n8n/decorators/src/policy-check/ @n8n-io/governance-and-observability
packages/@n8n/api-types/src/dto/otel/ @n8n-io/governance-and-observability
packages/@n8n/api-types/src/dto/type-availability-policies/ @n8n-io/governance-and-observability
packages/@n8n/api-types/src/node-type-availability.ts @n8n-io/governance-and-observability
packages/@n8n/api-types/src/credential-type-availability.ts @n8n-io/governance-and-observability
packages/@n8n/api-types/src/schemas/policy-check-failure.schema.ts @n8n-io/governance-and-observability
packages/@n8n/api-types/src/schemas/policy-violation.schema.ts @n8n-io/governance-and-observability
packages/@n8n/telemetry/src/events/policies.ts @n8n-io/governance-and-observability
packages/@n8n/telemetry/src/events/type-availability-policies.ts @n8n-io/governance-and-observability
packages/workflow/src/errors/policy-refusal.ts @n8n-io/governance-and-observability
packages/frontend/editor-ui/src/app/composables/usePolicyViolationToast.ts @n8n-io/governance-and-observability
# CI/CD
@ -285,3 +367,15 @@ packages/@n8n/db/src/entities/deployment-key.ts @n8n-io/iam required
packages/@n8n/db/src/repositories/deployment-key.repository.ts @n8n-io/iam required
packages/@n8n/eslint-config/src/configs/encryption-boundary.ts @n8n-io/iam required
packages/@n8n/eslint-config/src/rules/no-encryption-guardrail-disable.ts @n8n-io/iam required
# AI Trust (evaluations)
# Placed last so these rules override the broader package rules above.
packages/@n8n/instance-ai/evaluations/ @n8n-io/ai-trust
packages/cli/src/modules/instance-ai/eval/ @n8n-io/ai-trust
packages/quality/testing/playwright/tests/evals/instance-ai/ @n8n-io/ai-trust
.agents/skills/create-instance-ai-eval/ @n8n-io/ai-trust
.agents/skills/create-agent-builder-eval/ @n8n-io/ai-trust
packages/cli/src/modules/agent-evals/ @n8n-io/ai-trust
packages/@n8n/agents/src/evals/ @n8n-io/ai-trust
packages/frontend/editor-ui/src/features/ai/evaluation.ee/ @n8n-io/ai-trust

View File

@ -1,4 +1,4 @@
![Banner image](https://user-images.githubusercontent.com/10284570/173569848-c624317f-42b1-45a6-ab09-f0ea3c247648.png)
![Banner image](https://raw.githubusercontent.com/n8n-io/n8n/master/assets/readme-banner.jpg)
# n8n – The Platform for AI Agents and Workflow Automation

View File

@ -1,6 +1,6 @@
{
"repository": "n8n-io/cat-bot",
"releaseTag": "harness-v0.3.0",
"version": "0.3.0",
"assetName": "n8n-opencode-harness-0.3.0.tgz"
"releaseTag": "harness-v0.6.0",
"version": "0.6.0",
"assetName": "n8n-opencode-harness-0.6.0.tgz"
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 98 KiB

After

Width:  |  Height:  |  Size: 355 KiB

BIN
assets/readme-banner.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 289 KiB

View File

@ -1,7 +1,7 @@
# Every n8n image build uses this file. `pnpm build:docker` and CI both drive
# these targets, so a pin changed here changes both.
variable "NODE_VERSION" { default = "26.7.0" }
variable "NODE_VERSION" { default = "26.10.0" }
variable "N8N_VERSION" { default = "snapshot" }
variable "N8N_RELEASE_TYPE" { default = "dev" }
@ -15,14 +15,14 @@ variable "BUILDER_IMAGE" { default = "" }
variable "RUNTIME_IMAGE" { default = "" }
variable "PC_BUILDER_IMAGE" {
default = "n8nio/node-pc:26.7.0-dev@sha256:c64642dcb9464e50bd08aef8504affafeb795d01aed2f7ab158976c334a36ef6"
default = "n8nio/node-pc:26.10.0-dev@sha256:61e4b1f3b3824ffb908812e01bcbddabb64a01ef23d408d561696c344c6679e3"
}
variable "PC_RUNTIME_IMAGE" {
default = "n8nio/node-pc:26.7.0@sha256:6577d0742ad043baaa39db7dda2ce8aeb73f32c84eb241c49bf716f170c53297"
default = "n8nio/node-pc:26.10.0@sha256:386bae24d41c0091d21b4eee124a86464710fff24d5cb7f37386c4c5649b0c89"
}
variable "DHI_REF" {
default = "dhi.io/node:26.7.0-alpine3.24-dev@sha256:4b494d89fb26c950ce97865acf45b480dc7a6868fdc2b81c2d66599702eeac3f"
default = "dhi.io/node:26.10.0-alpine3.24-dev@sha256:8da859df5dc853c923ace53c4cfe994a909a28917f7fcfc8508869beab8cf132"
}
variable "N8N_TAGS" { default = "" }

View File

@ -15,11 +15,11 @@ set -eu
SCRIPT_VERSION="1.4.0"
# The version to install is derived from the latest stable GitHub release in
# resolve_n8n_version(); this fallback only applies when that lookup fails.
FALLBACK_N8N_VERSION="2.32.0"
FALLBACK_N8N_VERSION="2.43.4"
# Sandbox service release (API, runner and sandbox images ship together).
# Pinned here, not resolved at run time: the stack definition must match it.
# --upgrade moves existing installs to this pin.
SANDBOX_VERSION="1.3.4"
SANDBOX_VERSION="1.8.1"
N8N_DIR="${N8N_DIR:-./n8n}"
N8N_PORT=5678
SOURCE_URL="https://github.com/n8n-io/n8n/blob/master/docker/get-n8n.sh"

View File

@ -1,6 +1,6 @@
# CI passes one DHI reference per published Node version (see the matrix in
# build-base-image.yml). The default keeps plain `docker build` working.
ARG DHI_REF=dhi.io/node:24.19.0-alpine3.24-dev@sha256:dc2989ad23938772abaf549ed7bde5d61d70377005229bb55bef94bff40f8bbd
ARG DHI_REF=dhi.io/node:24.21.0-alpine3.24-dev@sha256:d22ba0e9ab2a54609ae533df4ebfd6f36c1f233b662a4c547f5a3195e8c21937
FROM ${DHI_REF}
# Install all dependencies in a single layer to minimize image size

View File

@ -1,9 +1,9 @@
ARG NODE_VERSION=26.7.0
ARG NODE_VERSION=26.10.0
ARG N8N_VERSION=snapshot
# The digest pins node:26.7.0-alpine3.24 for amd64 and arm64. Change the tag and the digest together.
ARG BUILDER_IMAGE=node:26.7.0-alpine3.24@sha256:aadf416b2cdce311a8811ba3f0608a61b77dbf997500e2eafe781b51f6a0b019
ARG RUNTIME_IMAGE=n8nio/base:26.7.0@sha256:33687300c4e94dc00f42ec79ae15082ae07330ecd82ae1167125905b65908ff8
# The digest pins node:26.10.0-alpine3.24 for amd64 and arm64. Change the tag and the digest together.
ARG BUILDER_IMAGE=node:26.10.0-alpine3.24@sha256:0b36e8c136b94cd4fcf02188228e76c31ad5872eef3fec8cbd2eee500cfd9e80
ARG RUNTIME_IMAGE=n8nio/base:26.10.0@sha256:9205c76a7745bfe71de35c2a0b7420238ccb0bdf69fc5273ab2e617fe918cf5f
# The runtime base has no compiler, so this stage supplies one.
FROM ${BUILDER_IMAGE} AS toolchain

View File

@ -10,14 +10,14 @@
# dev - plain Alpine with apk usable, for the n8n builder
# stage, which must compile native addons against this
# Node's headers
ARG NODE_VERSION=26.7.0
ARG NODE_VERSION=26.10.0
# Compiling V8 needs 1.5-2 GB of memory per parallel job. Lower this on
# builders whose memory-to-core ratio cannot cover all cores.
ARG MAKE_JOBS=
# Pinned to a multi-arch index digest (linux/amd64 + linux/arm64) for reproducible builds.
# Bump the tag and digest together when updating.
FROM alpine:3.24@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b AS builder
FROM alpine:3.24@sha256:294b683cb724975bec92580e1e685676bd4b50bda910ddb8c51d4cabeaec77e6 AS builder
ARG NODE_VERSION
ARG MAKE_JOBS
@ -70,7 +70,7 @@ RUN apk add --no-cache --virtual .build-deps \
# Remove unused OpenSSL headers to save ~34MB. See https://github.com/nodejs/node/issues/46451
&& find /node-install/usr/local/include/node/openssl/archs -mindepth 1 -maxdepth 1 ! -name "$OPENSSL_ARCH" -exec rm -rf {} \;
FROM alpine:3.24@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b AS base
FROM alpine:3.24@sha256:294b683cb724975bec92580e1e685676bd4b50bda910ddb8c51d4cabeaec77e6 AS base
RUN addgroup -g 1000 node \
&& adduser -u 1000 -G node -s /bin/sh -D node \
@ -90,7 +90,7 @@ ENV npm_config_nodedir=/usr/local
# npx, corepack) are swapped for the pointer-compressed build's files.
# Pinned to a multi-arch index digest (linux/amd64 + linux/arm64) for reproducible builds.
# Bump the tag and digest together when updating.
FROM n8nio/base:26.7.0@sha256:33687300c4e94dc00f42ec79ae15082ae07330ecd82ae1167125905b65908ff8 AS runtime
FROM n8nio/base:26.10.0@sha256:9205c76a7745bfe71de35c2a0b7420238ccb0bdf69fc5273ab2e617fe918cf5f AS runtime
RUN rm -f /usr/bin/node /usr/bin/nodejs /usr/bin/npm /usr/bin/npx /usr/bin/corepack /usr/local/bin/node
COPY --from=builder /node-install/usr/local /usr/local

View File

@ -1,4 +1,4 @@
ARG NODE_VERSION=26.7.0
ARG NODE_VERSION=26.10.0
ARG PNPM_VERSION=12.4.2
ARG PYTHON_VERSION=3.13

View File

@ -12,7 +12,7 @@
# - Uses distroless nonroot user (UID 65532)
# ==============================================================================
ARG NODE_VERSION=26.7.0
ARG NODE_VERSION=26.10.0
ARG PNPM_VERSION=12.4.2
ARG PYTHON_VERSION=3.13

183
docs/adr/README.md Normal file
View File

@ -0,0 +1,183 @@
# Architecture Decision Records
An Architecture Decision Record (ADR) is a short, dated record of one
important technical decision: the context, the decision, the alternatives,
and the consequences.
The company process is defined in the Notion page
[Engineering Technical Decisions][process] and its sub-page
[Architecture Decision Records][adr-page].
This README applies that process to this repository and records the
conventions that the reviews of the first records settled.
## When to write an ADR
Write an ADR when the reason for a decision can matter in the future. The
test from the company process: if an engineer challenges this choice in 12
months, will the original context and alternatives help them?
Write one record for each decision. The test: can this decision be reversed on
its own? If yes, it gets its own file. Do not put two independent decisions
into one record.
Do not write an ADR for an implementation detail, a refactor, or a choice that
follows an existing decision.
## Location
Store a record in the package where the implementation lives, in its
`docs/adr/` directory. For example, a record about engine v2 goes in
`packages/@n8n/engine/docs/adr/`. A record that concerns the whole repository
goes in the root `docs/adr/`, next to this README. CI rejects any other
location.
## File name
The file name follows from the title and the date:
```text
ADR-YYYYMMDD-<title in kebab-case>.md
```
The title is the H1 of the record. It states the decision, not the topic:
"Use PostgreSQL advisory locks", not "Database locking". The date is the date
of the decision, and it matches the `Date` field. CI checks both.
Do not rename a record after it is merged. Links depend on the file name.
Refer to a record by its full file name, for example
`ADR-20260828-obtain-trigger-output-before-creating-the-execution`, not by a
number. CI checks that every record ID a record mentions names a file.
## Template
Copy [`docs/ADR_TEMPLATE.md`](../ADR_TEMPLATE.md) from the repository root.
It is the one template, and CI checks every record against its structure.
The header has three required fields, `Date`, `Status`, and `Decision Owner`,
in that order, and three optional fields, `Source`, `Supersedes`, and
`Superseded by`. Omit an optional field that has no value. `Source` names
where the decision came from: a ticket, an RFC, or an incident. The body
names none of these. The `RFC` field under `Links` holds the link to the RFC.
## Status
Use one of three values:
- **Active.** The decision stands.
- **Superseded.** A newer record replaced the decision. Link to it in
`Superseded by`.
- **Deprecated.** The decision no longer applies and no record replaces it.
There is no Draft or Proposed status. The pull request holds that state while
the record is written and checked.
The status says whether the decision stands. It does not say whether the code
exists. Implementation progress belongs nowhere in the record.
## What goes where
**Context.** The problem and the constraints. State the world as it is, in
the present tense: "In engine v1, `putExecutionToWait` sleeps in the process
for a time wait under 65 seconds." Do not date a fact ("at the date of this
record"), do not narrate history ("core has since decided"), and do not phase
work ("the shim's part lands later"). The record is dated once, in its header.
Aim for fewer than 150 words.
**Decision.** What we decided, and the main reason. State the behaviour the
decision guarantees, in the present tense and the active voice: "A wait fires
at its deadline and not before it. A wait fires once, however many replicas
run." Describe the mechanism only when the mechanism is the decision. A sweep
that sleeps until the next deadline is how the engine fires waits today; the
guarantee is what the record decides. Do not write implementation state here.
A sentence such as "the engine does not do this yet" goes stale exactly where
a reader takes the text as the current contract.
**Alternatives Considered.** Each option we did not select, and why. Lead with
the reason: "We rejected it for two reasons." An option that reads as an equal
choice leaves the reader to guess why it lost. Use the conditional mood for
what an alternative would do: "This option would need a column." The
indicative mood ("this option needs a column") reads as a description of
shipped code.
**Consequences.** What follows from the decision: benefits, trade-offs,
constraints, and operational effects. A constraint on the mechanism belongs
here: "One statement must read the step rows and write the status." How the
code meets the constraint today, and the order in which the work lands, do
not. This is the one section that may grow after the record is merged. A
consequence that the team learned later belongs here. A changed decision does
not.
**Links.** Exactly three fields, each on one line: `RFC`, `Documentation`, and
`Related ADRs`. Write `-` when a field has no value. Name related records by
full file name. There is no `Tickets` field.
## Changing a decision
A merged record is history. Do not edit its Context, Decision, or Alternatives
to reflect a later change. Write a new record instead.
Therefore a change to the Context, the Decision, or the Alternatives rides the
pull request that merges the record. In a stack, the change belongs in the
first pull request that carries the record, not in the follow-up that
implements it. A decision can merge before its code. That is normal, and not a
reason to move or edit the record.
When a new decision replaces an old one:
1. Write a new record. Explain the new context.
2. Set `Supersedes` in the new record and `Superseded by` in the old one.
3. Set the old record's status to `Superseded`.
When a decision no longer applies and nothing replaces it, set its status to
`Deprecated`.
## Style
- Write in ASD-STE100 Simplified Technical English, as `AGENTS.md` requires:
short sentences, the active voice, one idea for each sentence.
- Wrap paragraph lines at 100 columns. CI enforces it. The `Links` fields
are exempt and stay on one line each.
- Do not reference a ticket in the body. Every sentence must stand without
the ID: "Send-and-wait nodes need credentials to send their message." The
`Source` field is the one place for a ticket.
- Use plain words. "Nothing moves a step from `waiting` to `completed` in one
step" beats "No component completes a waiting step directly".
- Keep the record short. A reader should understand the decision in a few
minutes without the pull request or the design document.
- Use a Mermaid diagram when a picture says it better than prose.
## Checked by CI
The `adr-conventions` rule of `@n8n/code-health` runs in the Static Analysis
check of every pull request. For every `ADR-*.md` file it checks:
- a `docs/adr` directory at the repository root or at a package root;
- a file name that matches the H1 and the `Date` field;
- the header fields, their order, and one blank line between them;
- a `Decision Owner` from the allowed list in
`packages/quality/policy/code-health/src/index.ts`;
- a full file name in `Supersedes` and `Superseded by`;
- exactly the five sections, in order, each with content;
- the three `Links` fields, one line each;
- paragraph lines of at most 100 characters outside `Links`;
- that every record ID the record mentions is a full ID that names a file in
this repository. A date alone, `ADR-20260902`, is rejected.
Run it locally:
```bash
pnpm --filter=@n8n/code-health check --rule=adr-conventions --ignore-baseline
```
## Process
1. The decision is made: in an RFC, a design review, a team discussion, or a
pull request.
2. The Decision Owner, or a delegate, writes the record.
3. Open a pull request. The review checks that the record is accurate and
clear. It does not reopen the decision.
4. Merge. The record is `Active`.
5. Implementation pull requests link to the record they implement.
[process]: https://app.notion.com/p/n8n/3bd5b6e0c94f804bac20ecc31500ce37
[adr-page]: https://app.notion.com/p/n8n/3bf5b6e0c94f8046b204c4b50533ec8b

View File

@ -9,7 +9,7 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| Name | Columns | Comment | Type |
| ---- | ------- | ------- | ---- |
| [public.activity_event](public.activity_event.md) | 11 | | BASE TABLE |
| [public.agent_background_job](public.agent_background_job.md) | 20 | | BASE TABLE |
| [public.agent_background_job](public.agent_background_job.md) | 23 | | BASE TABLE |
| [public.agent_budget_applied_call](public.agent_budget_applied_call.md) | 2 | | BASE TABLE |
| [public.agent_budget_spend](public.agent_budget_spend.md) | 4 | | BASE TABLE |
| [public.agent_channel_status](public.agent_channel_status.md) | 11 | | BASE TABLE |
@ -19,22 +19,24 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.agent_credential_dependency](public.agent_credential_dependency.md) | 3 | | BASE TABLE |
| [public.agent_eval_dataset](public.agent_eval_dataset.md) | 10 | | BASE TABLE |
| [public.agent_eval_rating](public.agent_eval_rating.md) | 8 | | BASE TABLE |
| [public.agent_eval_result](public.agent_eval_result.md) | 15 | | BASE TABLE |
| [public.agent_eval_result](public.agent_eval_result.md) | 16 | | BASE TABLE |
| [public.agent_eval_run](public.agent_eval_run.md) | 14 | | BASE TABLE |
| [public.agent_execution](public.agent_execution.md) | 23 | | BASE TABLE |
| [public.agent_execution_message_links](public.agent_execution_message_links.md) | 5 | | BASE TABLE |
| [public.agent_execution_threads](public.agent_execution_threads.md) | 19 | | BASE TABLE |
| [public.agent_execution_threads](public.agent_execution_threads.md) | 20 | | BASE TABLE |
| [public.agent_files](public.agent_files.md) | 10 | | BASE TABLE |
| [public.agent_history](public.agent_history.md) | 9 | | BASE TABLE |
| [public.agent_message_queue](public.agent_message_queue.md) | 10 | | BASE TABLE |
| [public.agent_plan](public.agent_plan.md) | 8 | | BASE TABLE |
| [public.agent_plan_history](public.agent_plan_history.md) | 6 | | BASE TABLE |
| [public.agent_skill_dependency](public.agent_skill_dependency.md) | 4 | | BASE TABLE |
| [public.agent_task_definition](public.agent_task_definition.md) | 8 | | BASE TABLE |
| [public.agent_task_run_lock](public.agent_task_run_lock.md) | 6 | | BASE TABLE |
| [public.agent_task_snapshot](public.agent_task_snapshot.md) | 9 | | BASE TABLE |
| [public.agent_thread_grants](public.agent_thread_grants.md) | 4 | | BASE TABLE |
| [public.agent_version_skill_dependency](public.agent_version_skill_dependency.md) | 4 | | BASE TABLE |
| [public.agent_workflow_dependency](public.agent_workflow_dependency.md) | 3 | | BASE TABLE |
| [public.agents](public.agents.md) | 14 | | BASE TABLE |
| [public.agents](public.agents.md) | 15 | | BASE TABLE |
| [public.agents_memory_entries](public.agents_memory_entries.md) | 13 | | BASE TABLE |
| [public.agents_memory_entry_candidates](public.agents_memory_entry_candidates.md) | 14 | | BASE TABLE |
| [public.agents_memory_entry_locks](public.agents_memory_entry_locks.md) | 6 | | BASE TABLE |
@ -48,6 +50,9 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.ai_builder_temporary_workflow](public.ai_builder_temporary_workflow.md) | 4 | | BASE TABLE |
| [public.ai_preference](public.ai_preference.md) | 8 | | BASE TABLE |
| [public.annotation_tag_entity](public.annotation_tag_entity.md) | 4 | | BASE TABLE |
| [public.app](public.app.md) | 9 | | BASE TABLE |
| [public.app_artifact](public.app_artifact.md) | 8 | | BASE TABLE |
| [public.app_version](public.app_version.md) | 10 | | BASE TABLE |
| [public.auth_identity](public.auth_identity.md) | 5 | | BASE TABLE |
| [public.auth_provider_sync_history](public.auth_provider_sync_history.md) | 11 | | BASE TABLE |
| [public.binary_data](public.binary_data.md) | 9 | | BASE TABLE |
@ -75,6 +80,7 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.execution_metadata](public.execution_metadata.md) | 4 | | BASE TABLE |
| [public.folder](public.folder.md) | 6 | | BASE TABLE |
| [public.folder_tag](public.folder_tag.md) | 2 | | BASE TABLE |
| [public.idempotency_key](public.idempotency_key.md) | 9 | | BASE TABLE |
| [public.insights_by_period](public.insights_by_period.md) | 6 | | BASE TABLE |
| [public.insights_metadata](public.insights_metadata.md) | 5 | | BASE TABLE |
| [public.insights_raw](public.insights_raw.md) | 5 | | BASE TABLE |
@ -102,6 +108,7 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.mcp_registry_server](public.mcp_registry_server.md) | 7 | | BASE TABLE |
| [public.migration_finding](public.migration_finding.md) | 10 | | BASE TABLE |
| [public.migration_finding_sync](public.migration_finding_sync.md) | 3 | | BASE TABLE |
| [public.migration_workflow_owner](public.migration_workflow_owner.md) | 7 | | BASE TABLE |
| [public.oauth_access_tokens](public.oauth_access_tokens.md) | 3 | | BASE TABLE |
| [public.oauth_authorization_codes](public.oauth_authorization_codes.md) | 13 | | BASE TABLE |
| [public.oauth_clients](public.oauth_clients.md) | 10 | | BASE TABLE |
@ -113,28 +120,34 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.project_pool_settings](public.project_pool_settings.md) | 4 | | BASE TABLE |
| [public.project_relation](public.project_relation.md) | 5 | | BASE TABLE |
| [public.project_secrets_provider_access](public.project_secrets_provider_access.md) | 5 | | BASE TABLE |
| [public.promotion_config](public.promotion_config.md) | 7 | | BASE TABLE |
| [public.promotion_connection](public.promotion_connection.md) | 7 | | BASE TABLE |
| [public.promotion_connection_project](public.promotion_connection_project.md) | 4 | | BASE TABLE |
| [public.promotion_environment](public.promotion_environment.md) | 9 | | BASE TABLE |
| [public.promotion_environment_project](public.promotion_environment_project.md) | 5 | | BASE TABLE |
| [public.promotion_provider](public.promotion_provider.md) | 8 | | BASE TABLE |
| [public.promotion_review](public.promotion_review.md) | 10 | | BASE TABLE |
| [public.promotion_run](public.promotion_run.md) | 8 | | BASE TABLE |
| [public.role](public.role.md) | 7 | | BASE TABLE |
| [public.role_mapping_rule](public.role_mapping_rule.md) | 7 | | BASE TABLE |
| [public.role_mapping_rule_project](public.role_mapping_rule_project.md) | 2 | | BASE TABLE |
| [public.role_scope](public.role_scope.md) | 2 | | BASE TABLE |
| [public.scheduled_job](public.scheduled_job.md) | 24 | | BASE TABLE |
| [public.scheduled_task](public.scheduled_task.md) | 18 | | BASE TABLE |
| [public.scheduled_job](public.scheduled_job.md) | 25 | | BASE TABLE |
| [public.scheduled_task](public.scheduled_task.md) | 19 | | BASE TABLE |
| [public.scope](public.scope.md) | 3 | | BASE TABLE |
| [public.secrets_provider_connection](public.secrets_provider_connection.md) | 7 | | BASE TABLE |
| [public.self_healing_investigation](public.self_healing_investigation.md) | 12 | | BASE TABLE |
| [public.self_healing_result](public.self_healing_result.md) | 15 | | BASE TABLE |
| [public.settings](public.settings.md) | 3 | | BASE TABLE |
| [public.shared_credentials](public.shared_credentials.md) | 5 | | BASE TABLE |
| [public.shared_workflow](public.shared_workflow.md) | 5 | | BASE TABLE |
| [public.skill](public.skill.md) | 8 | | BASE TABLE |
| [public.skill_file](public.skill_file.md) | 5 | | BASE TABLE |
| [public.skill_version](public.skill_version.md) | 11 | | BASE TABLE |
| [public.tag_entity](public.tag_entity.md) | 4 | | BASE TABLE |
| [public.test_case_execution](public.test_case_execution.md) | 14 | | BASE TABLE |
| [public.test_run](public.test_run.md) | 16 | | BASE TABLE |
| [public.token_exchange_jti](public.token_exchange_jti.md) | 3 | | BASE TABLE |
| [public.trusted_key](public.trusted_key.md) | 4 | | BASE TABLE |
| [public.trusted_key_source](public.trusted_key_source.md) | 8 | | BASE TABLE |
| [public.trusted_source](public.trusted_source.md) | 12 | | BASE TABLE |
| [public.trusted_source](public.trusted_source.md) | 15 | | BASE TABLE |
| [public.trusted_source_identity](public.trusted_source_identity.md) | 8 | | BASE TABLE |
| [public.type_availability_policy](public.type_availability_policy.md) | 7 | | BASE TABLE |
| [public.type_availability_policy_attachment](public.type_availability_policy_attachment.md) | 6 | | BASE TABLE |
@ -161,8 +174,8 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.workflow_review_request_workflow](public.workflow_review_request_workflow.md) | 5 | | BASE TABLE |
| [public.workflow_statistics](public.workflow_statistics.md) | 7 | | BASE TABLE |
| [public.workflow_statistics_delta](public.workflow_statistics_delta.md) | 6 | | BASE TABLE |
| [public.workflow_suggestion](public.workflow_suggestion.md) | 11 | | BASE TABLE |
| [public.workflow_suggestion_activity](public.workflow_suggestion_activity.md) | 6 | | BASE TABLE |
| [public.workflow_suggestion](public.workflow_suggestion.md) | 16 | | BASE TABLE |
| [public.workflow_suggestion_activity](public.workflow_suggestion_activity.md) | 7 | | BASE TABLE |
| [public.workflows_tags](public.workflows_tags.md) | 2 | | BASE TABLE |
## Stored procedures and functions
@ -190,6 +203,8 @@ erDiagram
"public.activity_event" }o--o| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.activity_event" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.agent_background_job" }o--|| "public.agents" : "FOREIGN KEY (#quot;parentAgentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agent_background_job" }o--o| "public.agent_execution" : "FOREIGN KEY (#quot;sourceExecutionId#quot;) REFERENCES agent_execution(id) ON DELETE SET NULL"
"public.agent_background_job" }o--o| "public.agent_plan" : "FOREIGN KEY (#quot;planId#quot;) REFERENCES agent_plan(id) ON DELETE SET NULL"
"public.agent_channel_status" }o--|| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agent_chat_attachments" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.agent_chat_attachments" }o--o| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
@ -220,13 +235,19 @@ erDiagram
"public.agent_message_queue" }o--|| "public.agent_execution_threads" : "FOREIGN KEY (#quot;threadId#quot;) REFERENCES agent_execution_threads(id) ON DELETE CASCADE"
"public.agent_plan" }o--|| "public.agent_execution_threads" : "FOREIGN KEY (#quot;threadId#quot;) REFERENCES agent_execution_threads(id) ON DELETE CASCADE"
"public.agent_plan_history" }o--|| "public.agent_plan" : "FOREIGN KEY (#quot;planId#quot;) REFERENCES agent_plan(id) ON DELETE CASCADE"
"public.agent_skill_dependency" }o--|| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agent_skill_dependency" }o--|| "public.skill" : "FOREIGN KEY (#quot;skillId#quot;) REFERENCES skill(id) ON DELETE CASCADE"
"public.agent_skill_dependency" }o--o| "public.skill_version" : "FOREIGN KEY (#quot;skillVersionId#quot;) REFERENCES skill_version(id)"
"public.agent_task_definition" }o--|| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agent_task_run_lock" }o--|| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agent_task_snapshot" }o--|| "public.agent_history" : "FOREIGN KEY (#quot;versionId#quot;) REFERENCES agent_history(#quot;versionId#quot;) ON DELETE CASCADE"
"public.agent_thread_grants" }o--|| "public.agent_execution_threads" : "FOREIGN KEY (#quot;threadId#quot;) REFERENCES agent_execution_threads(id) ON DELETE CASCADE"
"public.agent_version_skill_dependency" }o--|| "public.agent_history" : "FOREIGN KEY (#quot;agentVersionId#quot;) REFERENCES agent_history(#quot;versionId#quot;) ON DELETE CASCADE"
"public.agent_version_skill_dependency" }o--|| "public.skill" : "FOREIGN KEY (#quot;skillId#quot;) REFERENCES skill(id) ON DELETE CASCADE"
"public.agent_version_skill_dependency" }o--|| "public.skill_version" : "FOREIGN KEY (#quot;skillVersionId#quot;) REFERENCES skill_version(id)"
"public.agent_workflow_dependency" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.agent_workflow_dependency" }o--|| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agents" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.agents" }o--o| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.agents" }o--o| "public.agent_history" : "FOREIGN KEY (#quot;activeVersionId#quot;) REFERENCES agent_history(#quot;versionId#quot;) ON DELETE SET NULL"
"public.agents_memory_entries" }o--|| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agents_memory_entries" }o--|| "public.agents_resources" : "FOREIGN KEY (#quot;resourceId#quot;) REFERENCES agents_resources(id) ON DELETE CASCADE"
@ -256,6 +277,12 @@ erDiagram
"public.ai_preference" }o--o| "public.user" : "FOREIGN KEY (#quot;createdById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.ai_preference" }o--o| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.ai_preference" }o--o| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.app" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.app" }o--o| "public.app_version" : "FOREIGN KEY (#quot;draftVersionId#quot;) REFERENCES app_version(id) ON DELETE SET NULL"
"public.app" }o--o| "public.app_version" : "FOREIGN KEY (#quot;publishedVersionId#quot;) REFERENCES app_version(id) ON DELETE SET NULL"
"public.app_version" }o--|| "public.app" : "FOREIGN KEY (#quot;appId#quot;) REFERENCES app(id) ON DELETE CASCADE"
"public.app_version" }o--o| "public.app_artifact" : "FOREIGN KEY (#quot;bundleArtifactId#quot;) REFERENCES app_artifact(id) ON DELETE SET NULL"
"public.app_version" }o--|| "public.app_artifact" : "FOREIGN KEY (#quot;sourceArtifactId#quot;) REFERENCES app_artifact(id) ON DELETE RESTRICT"
"public.auth_identity" }o--o| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id)"
"public.chat_hub_agent_tools" }o--|| "public.chat_hub_agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES chat_hub_agents(id) ON DELETE CASCADE"
"public.chat_hub_agent_tools" }o--|| "public.chat_hub_tools" : "FOREIGN KEY (#quot;toolId#quot;) REFERENCES chat_hub_tools(id) ON DELETE CASCADE"
@ -298,6 +325,7 @@ erDiagram
"public.folder" }o--o| "public.folder" : "FOREIGN KEY (#quot;parentFolderId#quot;) REFERENCES folder(id) ON DELETE CASCADE"
"public.folder_tag" }o--|| "public.tag_entity" : "FOREIGN KEY (#quot;tagId#quot;) REFERENCES tag_entity(id) ON DELETE CASCADE"
"public.folder_tag" }o--|| "public.folder" : "FOREIGN KEY (#quot;folderId#quot;) REFERENCES folder(id) ON DELETE CASCADE"
"public.idempotency_key" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.insights_by_period" }o--|| "public.insights_metadata" : "FOREIGN KEY (#quot;metaId#quot;) REFERENCES insights_metadata(#quot;metaId#quot;) ON DELETE CASCADE"
"public.insights_metadata" }o--o| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE SET NULL"
"public.insights_metadata" }o--o| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE SET NULL"
@ -326,6 +354,9 @@ erDiagram
"public.instance_ai_threads" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.instance_credential_assignment" }o--|| "public.credentials_entity" : "FOREIGN KEY (#quot;credentialId#quot;) REFERENCES credentials_entity(id) ON DELETE RESTRICT"
"public.migration_finding" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.migration_workflow_owner" }o--o| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.migration_workflow_owner" }o--o| "public.user" : "FOREIGN KEY (#quot;assignedById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.migration_workflow_owner" |o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.oauth_access_tokens" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.oauth_access_tokens" }o--|| "public.oauth_clients" : "FOREIGN KEY (#quot;clientId#quot;) REFERENCES oauth_clients(id) ON DELETE CASCADE"
"public.oauth_authorization_codes" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
@ -343,20 +374,40 @@ erDiagram
"public.project_relation" }o--|| "public.role" : "FOREIGN KEY (role) REFERENCES role(slug)"
"public.project_secrets_provider_access" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.project_secrets_provider_access" }o--|| "public.secrets_provider_connection" : "FOREIGN KEY (#quot;secretsProviderConnectionId#quot;) REFERENCES secrets_provider_connection(id) ON DELETE CASCADE"
"public.promotion_config" }o--|| "public.promotion_connection" : "FOREIGN KEY (#quot;connectionId#quot;) REFERENCES promotion_connection(id) ON DELETE CASCADE"
"public.promotion_connection" }o--|| "public.promotion_provider" : "FOREIGN KEY (#quot;providerId#quot;) REFERENCES promotion_provider(id) ON DELETE RESTRICT"
"public.promotion_connection_project" |o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.promotion_connection_project" }o--|| "public.promotion_connection" : "FOREIGN KEY (#quot;connectionId#quot;) REFERENCES promotion_connection(id) ON DELETE CASCADE"
"public.promotion_environment" }o--|| "public.promotion_provider" : "FOREIGN KEY (#quot;providerId#quot;) REFERENCES promotion_provider(id) ON DELETE RESTRICT"
"public.promotion_environment_project" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.promotion_environment_project" }o--|| "public.promotion_environment" : "FOREIGN KEY (#quot;environmentId#quot;) REFERENCES promotion_environment(id) ON DELETE CASCADE"
"public.promotion_review" }o--o| "public.user" : "FOREIGN KEY (#quot;approvedById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.promotion_review" }o--o| "public.user" : "FOREIGN KEY (#quot;createdById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.promotion_review" }o--o| "public.promotion_environment" : "FOREIGN KEY (#quot;environmentId#quot;) REFERENCES promotion_environment(id) ON DELETE SET NULL"
"public.promotion_run" }o--o| "public.user" : "FOREIGN KEY (#quot;createdById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.promotion_run" }o--o| "public.promotion_environment" : "FOREIGN KEY (#quot;environmentId#quot;) REFERENCES promotion_environment(id) ON DELETE SET NULL"
"public.promotion_run" }o--o| "public.promotion_review" : "FOREIGN KEY (#quot;reviewId#quot;) REFERENCES promotion_review(id) ON DELETE SET NULL"
"public.role_mapping_rule" }o--|| "public.role" : "FOREIGN KEY (role) REFERENCES role(slug) ON UPDATE CASCADE ON DELETE CASCADE"
"public.role_mapping_rule_project" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.role_mapping_rule_project" }o--|| "public.role_mapping_rule" : "FOREIGN KEY (#quot;roleMappingRuleId#quot;) REFERENCES role_mapping_rule(id) ON DELETE CASCADE"
"public.role_scope" }o--|| "public.scope" : "FOREIGN KEY (#quot;scopeSlug#quot;) REFERENCES scope(slug) ON UPDATE CASCADE ON DELETE CASCADE"
"public.role_scope" }o--|| "public.role" : "FOREIGN KEY (#quot;roleSlug#quot;) REFERENCES role(slug) ON UPDATE CASCADE ON DELETE CASCADE"
"public.scheduled_task" }o--|| "public.scheduled_job" : "FOREIGN KEY (#quot;jobId#quot;) REFERENCES scheduled_job(id) ON DELETE CASCADE"
"public.self_healing_investigation" }o--|| "public.user" : "FOREIGN KEY (#quot;backgroundUserId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.self_healing_investigation" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.self_healing_investigation" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.self_healing_investigation" }o--o| "public.self_healing_result" : "FOREIGN KEY (#quot;resultId#quot;) REFERENCES self_healing_result(id) ON DELETE CASCADE"
"public.self_healing_result" }o--|| "public.user" : "FOREIGN KEY (#quot;backgroundUserId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.self_healing_result" }o--o| "public.user" : "FOREIGN KEY (#quot;dismissedById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.self_healing_result" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.self_healing_result" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.self_healing_result" }o--o| "public.workflow_suggestion" : "FOREIGN KEY (#quot;suggestionId#quot;) REFERENCES workflow_suggestion(id) ON DELETE CASCADE"
"public.shared_credentials" }o--|| "public.credentials_entity" : "FOREIGN KEY (#quot;credentialsId#quot;) REFERENCES credentials_entity(id) ON DELETE CASCADE"
"public.shared_credentials" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.shared_workflow" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.shared_workflow" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.skill" }o--o| "public.user" : "FOREIGN KEY (#quot;createdById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.skill" }o--o| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.skill" }o--o| "public.agents" : "FOREIGN KEY (#quot;agentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.skill_file" }o--|| "public.skill_version" : "FOREIGN KEY (#quot;skillVersionId#quot;) REFERENCES skill_version(id) ON DELETE CASCADE"
"public.skill_version" }o--o| "public.user" : "FOREIGN KEY (#quot;createdById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.skill_version" }o--|| "public.skill" : "FOREIGN KEY (#quot;skillId#quot;) REFERENCES skill(id) ON DELETE CASCADE"
"public.test_case_execution" }o--o| "public.execution_entity" : "FOREIGN KEY (#quot;executionId#quot;) REFERENCES execution_entity(id) ON DELETE SET NULL"
"public.test_case_execution" }o--|| "public.test_run" : "FOREIGN KEY (#quot;testRunId#quot;) REFERENCES test_run(id) ON DELETE CASCADE"
"public.test_run" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
@ -406,6 +457,7 @@ erDiagram
"public.workflow_suggestion" }o--|| "public.user" : "FOREIGN KEY (#quot;backgroundUserId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.workflow_suggestion" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.workflow_suggestion" }o--|| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE CASCADE"
"public.workflow_suggestion_activity" }o--o| "public.user" : "FOREIGN KEY (#quot;actorId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.workflow_suggestion_activity" }o--|| "public.workflow_suggestion" : "FOREIGN KEY (#quot;suggestionId#quot;) REFERENCES workflow_suggestion(id) ON DELETE CASCADE"
"public.workflows_tags" }o--|| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE CASCADE"
"public.workflows_tags" }o--|| "public.tag_entity" : "FOREIGN KEY (#quot;tagId#quot;) REFERENCES tag_entity(id) ON DELETE CASCADE"
@ -436,8 +488,11 @@ erDiagram
varchar_255_ parentResourceId
varchar_128_ parentThreadId
uuid pauseRequestId
uuid planId FK
uuid planTaskId
text result
timestamp_3__with_time_zone settledAt
varchar_36_ sourceExecutionId FK
varchar_16_ status
varchar_36_ subAgentId
timestamp_3__with_time_zone timeoutAt
@ -542,6 +597,7 @@ erDiagram
varchar status
json toolCalls
timestamp_3__with_time_zone updatedAt
json verdict
}
"public.agent_eval_run" {
varchar_36_ agentVersionId
@ -604,6 +660,7 @@ erDiagram
varchar_255_ projectId FK
integer sessionNumber
varchar_32_ taskId
json taskStop
varchar_36_ taskVersionId FK
varchar_255_ title
integer totalCompletionTokens
@ -665,6 +722,12 @@ erDiagram
uuid planId FK
integer revision
}
"public.agent_skill_dependency" {
varchar_36_ agentId FK
timestamp_3__with_time_zone createdAt
varchar_36_ skillId FK
uuid skillVersionId FK
}
"public.agent_task_definition" {
varchar_36_ agentId FK
timestamp_3__with_time_zone createdAt
@ -700,6 +763,12 @@ erDiagram
varchar_128_ threadId FK
timestamp_3__with_time_zone updatedAt
}
"public.agent_version_skill_dependency" {
varchar_36_ agentVersionId FK
timestamp_3__with_time_zone createdAt
varchar_36_ skillId FK
uuid skillVersionId FK
}
"public.agent_workflow_dependency" {
varchar_36_ agentId FK
timestamp_3__with_time_zone createdAt
@ -715,6 +784,7 @@ erDiagram
varchar_255_ projectId FK
integer revision
json schema
varchar_16_ scope
timestamp_3__with_time_zone setupCompletedAt
json skills
json tools
@ -852,6 +922,39 @@ erDiagram
varchar_24_ name
timestamp_3__with_time_zone updatedAt
}
"public.app" {
timestamp_3__with_time_zone createdAt
uuid draftVersionId FK
uuid id
varchar_128_ name
varchar_128_ namespace
varchar_36_ projectId FK
uuid publishedVersionId FK
timestamp_3__with_time_zone updatedAt
varchar_16_ visibility
}
"public.app_artifact" {
uuid appId
timestamp_3__with_time_zone createdAt
integer id
integer sizeBytes
text storageKey
varchar_2_ storedAt
timestamp_3__with_time_zone updatedAt
varchar_16_ uploadStatus
}
"public.app_version" {
uuid appId FK
integer buildAttempts
varchar_16_ buildStatus
timestamp_3__with_time_zone buildStatusUpdatedAt
integer bundleArtifactId FK
timestamp_3__with_time_zone createdAt
uuid id
varchar_255_ label
integer sourceArtifactId FK
timestamp_3__with_time_zone updatedAt
}
"public.auth_identity" {
timestamp_3__with_time_zone createdAt
varchar_255_ providerId
@ -1111,6 +1214,17 @@ erDiagram
varchar_36_ folderId FK
varchar_36_ tagId FK
}
"public.idempotency_key" {
timestamp_3__with_time_zone createdAt
text fingerprint
varchar_16_ id
varchar_128_ idempotencyKey
json responseBody
smallint responseStatus
varchar_16_ status
timestamp_3__with_time_zone updatedAt
uuid userId FK
}
"public.insights_by_period" {
integer id
integer metaId FK
@ -1363,6 +1477,15 @@ erDiagram
timestamp_3__with_time_zone syncedAt
varchar_16_ targetVersion
}
"public.migration_workflow_owner" {
timestamp_3__with_time_zone assignedAt
uuid assignedById FK
timestamp_3__with_time_zone createdAt
varchar_16_ source
timestamp_3__with_time_zone updatedAt
uuid userId FK
varchar_36_ workflowId FK
}
"public.oauth_access_tokens" {
varchar clientId FK
varchar token
@ -1459,27 +1582,21 @@ erDiagram
integer secretsProviderConnectionId FK
timestamp_3__with_time_zone updatedAt
}
"public.promotion_config" {
varchar_36_ connectionId FK
"public.promotion_environment" {
timestamp_3__with_time_zone createdAt
varchar_16_ direction
varchar_36_ id
uuid id
varchar_128_ name
varchar_36_ providerId FK
json remote
varchar_16_ scope
json settings
timestamp_3__with_time_zone updatedAt
}
"public.promotion_connection" {
timestamp_3__with_time_zone createdAt
varchar_36_ id
varchar_128_ name
varchar_36_ providerId FK
varchar_16_ scope
json target
timestamp_3__with_time_zone updatedAt
}
"public.promotion_connection_project" {
varchar_36_ connectionId FK
"public.promotion_environment_project" {
timestamp_3__with_time_zone createdAt
varchar_16_ direction
uuid environmentId FK
varchar_36_ projectId FK
timestamp_3__with_time_zone updatedAt
}
@ -1493,6 +1610,28 @@ erDiagram
varchar_32_ type
timestamp_3__with_time_zone updatedAt
}
"public.promotion_review" {
timestamp_3__with_time_zone approvedAt
uuid approvedById FK
timestamp_3__with_time_zone createdAt
uuid createdById FK
uuid environmentId FK
uuid id
timestamp_3__with_time_zone lastStateChangeAt
varchar_255_ remoteReviewId
varchar_32_ state
varchar_16_ type
}
"public.promotion_run" {
timestamp_3__with_time_zone createdAt
uuid createdById FK
json details
varchar_16_ direction
uuid environmentId FK
uuid id
uuid reviewId FK
varchar_16_ type
}
"public.role" {
timestamp_3__with_time_zone createdAt
text description
@ -1542,6 +1681,7 @@ erDiagram
integer recurrenceSize
varchar_16_ recurrenceUnit
varchar_128_ taskType
integer timeoutSeconds
varchar_64_ timezone
timestamp_3__with_time_zone updatedAt
}
@ -1564,6 +1704,7 @@ erDiagram
timestamp_3__with_time_zone startedAt
varchar_16_ status
varchar_128_ taskType
integer timeoutSeconds
}
"public.scope" {
text description
@ -1579,6 +1720,37 @@ erDiagram
varchar_36_ type
timestamp_3__with_time_zone updatedAt
}
"public.self_healing_investigation" {
uuid backgroundUserId FK
json baseline
timestamp_3__with_time_zone createdAt
varchar_36_ executionId
timestamp_3__with_time_zone finishedAt
varchar_36_ id
varchar_36_ projectId FK
varchar_36_ resultId FK
uuid runId
uuid threadId
timestamp_3__with_time_zone updatedAt
varchar_36_ workflowId FK
}
"public.self_healing_result" {
uuid backgroundUserId FK
timestamp_3__with_time_zone completedAt
timestamp_3__with_time_zone createdAt
timestamp_3__with_time_zone dismissedAt
uuid dismissedById FK
varchar_36_ executionId
varchar_36_ id
varchar_16_ outcome
varchar_36_ projectId FK
text report
varchar_36_ suggestionId FK
text summary
timestamp_3__with_time_zone updatedAt
json usage
varchar_36_ workflowId FK
}
"public.settings" {
varchar_255_ key
boolean loadOnStartup
@ -1598,6 +1770,36 @@ erDiagram
timestamp_3__with_time_zone updatedAt
varchar_36_ workflowId FK
}
"public.skill" {
varchar_36_ agentId FK
boolean availableToAssistant
timestamp_3__with_time_zone createdAt
uuid createdById FK
varchar_36_ id
varchar_36_ projectId FK
varchar_16_ source
timestamp_3__with_time_zone updatedAt
}
"public.skill_file" {
text content
timestamp_3__with_time_zone createdAt
varchar_512_ path
uuid skillVersionId FK
timestamp_3__with_time_zone updatedAt
}
"public.skill_version" {
varchar_64_ contentHash
timestamp_3__with_time_zone createdAt
uuid createdById FK
varchar_1024_ description
json frontmatter
uuid id
text instructions
varchar_128_ name
varchar_36_ skillId FK
timestamp_3__with_time_zone updatedAt
integer version
}
"public.tag_entity" {
timestamp_3__with_time_zone createdAt
varchar_36_ id
@ -1663,11 +1865,14 @@ erDiagram
text config
integer configVersion
timestamp_3__with_time_zone createdAt
varchar_36_ discoveryClaimToken
timestamp_3__with_time_zone discoveryClaimedAt
varchar_36_ id
varchar issuer
timestamp_3__with_time_zone lastCheckedAt
text lastError
varchar_16_ managedBy
text metadata
varchar_128_ name
varchar_16_ status
varchar_32_ type
@ -1922,6 +2127,10 @@ erDiagram
varchar_128_ workflowName
}
"public.workflow_suggestion" {
varchar_32_ appliedAction
uuid appliedActorId
varchar_64_ appliedChecksum
varchar_36_ appliedVersionId
uuid backgroundUserId FK
timestamp_3__with_time_zone closedAt
varchar_16_ closedReason
@ -1930,12 +2139,14 @@ erDiagram
varchar_36_ id
json payload
varchar_36_ projectId FK
varchar_16_ resultKind
varchar_16_ state
timestamp_3__with_time_zone updatedAt
varchar_36_ workflowId FK
}
"public.workflow_suggestion_activity" {
varchar_16_ action
uuid actorId FK
varchar_16_ author
timestamp_3__with_time_zone createdAt
varchar_36_ id

View File

@ -35,6 +35,7 @@
| Name | Definition |
| ---- | ---------- |
| IDX_activity_event_project | CREATE INDEX "IDX_activity_event_project" ON public.activity_event USING btree ("projectId", id) |
| IDX_activity_event_resource | CREATE INDEX "IDX_activity_event_resource" ON public.activity_event USING btree ("resourceId", id) |
| IDX_activity_event_user | CREATE INDEX "IDX_activity_event_user" ON public.activity_event USING btree ("userId", id) |
| PK_c2c1e9fdda754a6bf7f664d7e04 | CREATE UNIQUE INDEX "PK_c2c1e9fdda754a6bf7f664d7e04" ON public.activity_event USING btree (id) |

View File

@ -16,8 +16,11 @@
| parentResourceId | varchar(255) | | false | | | Memory resource of the parent agent run |
| parentThreadId | varchar(128) | | false | | | |
| pauseRequestId | uuid | | true | | | Groups jobs stopped by one user request |
| planId | uuid | | true | | [public.agent_plan](public.agent_plan.md) | Plan that owns this work |
| planTaskId | uuid | | true | | | Task ID in the saved plan document |
| result | text | | true | | | Final answer of a settled sub-agent job |
| settledAt | timestamp(3) with time zone | | true | | | |
| sourceExecutionId | varchar(36) | | true | | [public.agent_execution](public.agent_execution.md) | Execution that dispatched this job |
| status | varchar(16) | | false | | | running: child works; suspended: child awaits approval; paused: user stopped the child; completed, failed, cancelled: terminal |
| subAgentId | varchar(36) | | true | | | Sub-agent jobs only |
| timeoutAt | timestamp(3) with time zone | | true | | | When reconciliation fails the job as timed out; NULL means no timeout |
@ -31,6 +34,8 @@
| ---- | ---- | ---------- |
| CHK_agent_background_job_kind | CHECK | CHECK (((kind)::text = ANY ((ARRAY['subagent'::character varying, 'workflow'::character varying])::text[]))) |
| CHK_agent_background_job_status | CHECK | CHECK (((status)::text = ANY ((ARRAY['running'::character varying, 'suspended'::character varying, 'paused'::character varying, 'completed'::character varying, 'failed'::character varying, 'cancelled'::character varying])::text[]))) |
| FK_4db3b7ebea4ed4f69649982018c | FOREIGN KEY | FOREIGN KEY ("sourceExecutionId") REFERENCES agent_execution(id) ON DELETE SET NULL |
| FK_ce131bff629d89015893e350e9c | FOREIGN KEY | FOREIGN KEY ("planId") REFERENCES agent_plan(id) ON DELETE SET NULL |
| FK_d46c6f00730c2ef8bcb6ee24b67 | FOREIGN KEY | FOREIGN KEY ("parentAgentId") REFERENCES agents(id) ON DELETE CASCADE |
| PK_6e0db58281aa2b4c956dc0d58e9 | PRIMARY KEY | PRIMARY KEY (id) |
| agent_background_job_createdAt_not_null | n | NOT NULL "createdAt" |
@ -51,6 +56,8 @@
| IDX_93d62baabe9858816b5adafb44 | CREATE INDEX "IDX_93d62baabe9858816b5adafb44" ON public.agent_background_job USING btree ("parentThreadId", status) |
| IDX_agent_background_job_childExecutionId | CREATE UNIQUE INDEX "IDX_agent_background_job_childExecutionId" ON public.agent_background_job USING btree ("childExecutionId") WHERE ("childExecutionId" IS NOT NULL) |
| IDX_agent_background_job_parentThreadId | CREATE INDEX "IDX_agent_background_job_parentThreadId" ON public.agent_background_job USING btree ("parentThreadId") WHERE (((status)::text <> 'running'::text) AND ("notifiedAt" IS NULL)) |
| IDX_agent_background_job_planId_planTaskId | CREATE INDEX "IDX_agent_background_job_planId_planTaskId" ON public.agent_background_job USING btree ("planId", "planTaskId") |
| IDX_agent_background_job_sourceExecutionId | CREATE INDEX "IDX_agent_background_job_sourceExecutionId" ON public.agent_background_job USING btree ("sourceExecutionId") |
| IDX_agent_background_job_timeoutAt | CREATE INDEX "IDX_agent_background_job_timeoutAt" ON public.agent_background_job USING btree ("timeoutAt") WHERE ((status)::text = ANY ((ARRAY['running'::character varying, 'suspended'::character varying])::text[])) |
| IDX_d46c6f00730c2ef8bcb6ee24b6 | CREATE INDEX "IDX_d46c6f00730c2ef8bcb6ee24b6" ON public.agent_background_job USING btree ("parentAgentId") |
| IDX_e43e630272995a93dfeb94ab3e | CREATE INDEX "IDX_e43e630272995a93dfeb94ab3e" ON public.agent_background_job USING btree ("settledAt") |
@ -62,6 +69,8 @@
erDiagram
"public.agent_background_job" }o--|| "public.agents" : "FOREIGN KEY (#quot;parentAgentId#quot;) REFERENCES agents(id) ON DELETE CASCADE"
"public.agent_background_job" }o--o| "public.agent_plan" : "FOREIGN KEY (#quot;planId#quot;) REFERENCES agent_plan(id) ON DELETE SET NULL"
"public.agent_background_job" }o--o| "public.agent_execution" : "FOREIGN KEY (#quot;sourceExecutionId#quot;) REFERENCES agent_execution(id) ON DELETE SET NULL"
"public.agent_background_job" {
varchar_36_ childExecutionId
@ -76,8 +85,11 @@ erDiagram
varchar_255_ parentResourceId
varchar_128_ parentThreadId
uuid pauseRequestId
uuid planId FK
uuid planTaskId
text result
timestamp_3__with_time_zone settledAt
varchar_36_ sourceExecutionId FK
varchar_16_ status
varchar_36_ subAgentId
timestamp_3__with_time_zone timeoutAt
@ -95,12 +107,48 @@ erDiagram
varchar_255_ projectId FK
integer revision
json schema
varchar_16_ scope
timestamp_3__with_time_zone setupCompletedAt
json skills
json tools
timestamp_3__with_time_zone updatedAt
varchar_36_ versionId
}
"public.agent_plan" {
timestamp_3__with_time_zone closedAt
timestamp_3__with_time_zone createdAt
json data
integer formatVersion
uuid id
integer revision
varchar_128_ threadId FK
timestamp_3__with_time_zone updatedAt
}
"public.agent_execution" {
boolean acceptsSteering
json attachments
json author
integer completionTokens
double_precision cost
timestamp_3__with_time_zone createdAt
integer duration
text error
json failureSummary
varchar_16_ hitlStatus
varchar_36_ id
varchar_255_ model
integer promptTokens
varchar_32_ source
timestamp_3__with_time_zone startedAt
varchar_16_ status
timestamp_3__with_time_zone stoppedAt
varchar_2_ storedAt
varchar_128_ threadId FK
json timeline
integer totalTokens
timestamp_3__with_time_zone updatedAt
text userMessage
}
```
---

View File

@ -70,6 +70,7 @@ erDiagram
varchar_255_ projectId FK
integer revision
json schema
varchar_16_ scope
timestamp_3__with_time_zone setupCompletedAt
json skills
json tools

Some files were not shown because too many files have changed in this diff Show More