Commit Graph

135 Commits

Author SHA1 Message Date
José Braulio González Valido
5b149495c5
feat(ai-builder): Grade token consumption and memory compaction in Instance AI evals (no-changelog) (#38974)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 11:12:10 +00:00
José Braulio González Valido
ecda0d4680
fix(ai-builder): Report un-intercepted vendor sub-nodes and stop blaming the mock layer for a root's own crash (no-changelog) (#38155)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 12:18:30 +00:00
Arvin A
d9dc457d40
feat(core): Track credential description usage (no-changelog) (#38986) 2026-09-21 09:09:43 +00:00
Michael Drury
d605e8ca80
test(core): Support Agent attachments in Instance AI evaluations (#38874) 2026-09-21 08:57:39 +00:00
José Braulio González Valido
326030418d
fix(core): Attribute a draft-Agent eval scenario to the harness, not the builder (no-changelog) (#39007)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 11:44:52 +00:00
Riqwan Thamir
0a4976e62c
feat(editor): Open Edit with AI without a greeting turn (#38745)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 16:45:07 +00:00
Roman Makowiak
5b199f5cdf
feat(core): Add folder seeding to the Instance AI eval harness (no-changelog) (#38550)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-16 07:47:46 +00:00
Mutasem Aldmour
2f352bde2b
fix(core): Offer Computer Use only where the client shows its entry points (#38108)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 15:35:01 +00:00
Michael Drury
684b930fa6
feat(core): Improve Agent Builder evaluation and intent routing (#38329) 2026-09-14 13:13:51 +00:00
Luka Zivkovic
70e5a2bf16
feat: Export structured agent artifacts in eval results (#38394) 2026-09-11 15:11:02 +00:00
oleg
1bee3bcac7
feat(core): Add progressive workflow building (no-changelog) (#37996) 2026-09-10 13:42:43 +00:00
José Braulio González Valido
44fae2aac9
fix(ai-builder): Give agent narration a larger judge budget than tool payloads (no-changelog) (#38192)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 08:37:07 +00:00
Predrag Ristic
418f85e9eb
refactor(core): Remove legacy Instance AI eval agent (no-changelog) (#36740) 2026-09-08 10:14:32 +00:00
Roman Makowiak
9769dacbe3
feat(core): Add folder exploration to Instance AI (#37865)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 20:52:53 +00:00
Predrag Ristic
be678fcb69
feat(core): Guide Instance AI credential selection (#37447) 2026-09-04 13:04:42 +00:00
Arvin A
95f7e6a8a8
feat(core): Run seeded workflows before the graded turn in Instance AI evals (no-changelog) (#37516) 2026-09-03 14:09:18 +00:00
José Braulio González Valido
614dfa62f8
feat(ai-builder): Surface thread provenance on Instance AI traces (no-changelog) (#36909)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 07:50:01 +00:00
Jaakko Husso
a7661cb4f4
test(core): Support tool-result cleanup in the eval stub workspace (no-changelog) (#37202)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 12:47:15 +00:00
Jaakko Husso
7fe438e530
fix(core): Keep AI Assistant workflow builds honest about their project (#37177)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 09:40:04 +00:00
Mutasem Aldmour
ef13ad0abc
fix(core): Report what credential setup actually selected (#37019)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 09:31:43 +00:00
José Braulio González Valido
5ec90c5331
test(ai-builder): Pin threadIds in the eval-results dispatcher contract (no-changelog) (#36884)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:19:28 +00:00
Bernhard Wittmann
9c4b439122
fix(core): Stop code editors mangling values typed by browser_type (no-changelog) (#36804) 2026-08-21 13:08:05 +00:00
Bernhard Wittmann
44041d5d3c
test(core): Add openai, gemini and slack credential-setup eval fixtures (no-changelog) (#36461)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 06:27:16 +00:00
Rodrigo Santos da Silva
69ba5a8618
feat(ai-builder): Let the eval harness mint Simplified Custom Auth credentials from setup-card recipes (no-changelog) (#36192)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-08-19 08:44:23 +00:00
Milorad FIlipović
a37b9a0c49
feat(core): Extend evaluations to support node groups (no-changelog) (#35629)
Co-authored-by: Miguel Ángel Moreno <1411774+miguelsaddress@users.noreply.github.com>
2026-08-19 08:42:53 +00:00
Elias Meire
fdbf88b469
test(core): Add MCP registry discovery evals (no-changelog) (#36063) 2026-08-18 12:28:05 +00:00
Jaakko Husso
6ace2a5b4f
fix(core): Remember skipped Instance AI credential setup for the rest of a thread (#36308)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:53:22 +00:00
Robin Braumann
6b656c97af
refactor(core): Consolidate agent workspace string replacement tools (no-changelog) (#36178)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Riqwan Thamir <rmthamir@gmail.com>
2026-08-17 11:11:51 +00:00
Iván Ovejero
c0acde43a1
ci: Run PR pipeline on Node 26 (#34032)
Co-authored-by: Danny Martini <danny@n8n.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 07:54:25 +00:00
Bernhard Wittmann
62d5de3ec7
feat: Add the credential-setup browser eval lane (no-changelog) (#35982)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 15:56:24 +00:00
Rodrigo Santos da Silva
2d37afc13d
test(ai-builder): Let a case declare a credential already broken before the conversation started (no-changelog) (#36029)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-13 13:33:11 +00:00
Riqwan Thamir
91ebeecb4e
feat(core): Improve Instance AI model experiments (no-changelog) (#35034)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-13 10:15:03 +00:00
Robin Braumann
01ce4490d0
feat(core): Add writable workspaces to agents (#36033)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-12 12:24:33 +00:00
Arvin A
45f243419d
test(core): Drive external workflow edits from the user proxy (#35736) 2026-08-12 08:42:20 +00:00
Svetoslav Dekov
9205eb3f19
test(core): Support credential seeding in MCP workflow evals (no-changelog) (#35782)
Some checks failed
CI: Master (Build, Test, Lint) / Build for Github Cache (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (22.23.2) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Performance (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Build: v3 Nightly Docker Images / Compute date tag (push) Has been cancelled
Build: v3 Nightly Docker Images / Build and push v3-nightly images (push) Has been cancelled
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ricardo Espinoza <ricardo@n8n.io>
2026-08-10 19:56:03 +00:00
Raúl Gómez Morales
53841f1b0e
feat: Add Simplified Custom Auth credential with recipe-guided setup in the AI Assistant (#35056) 2026-08-07 09:22:36 +00:00
Svetoslav Dekov
ff37ef5b3f
ci(ai-builder): Gate the mcp-gate eval tier absolutely (no-changelog) (#35762)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 14:37:04 +00:00
José Braulio González Valido
9b8f964a55
feat(ai-builder): Support seeding agents alongside workflows in eval threads (no-changelog) (#35548)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 12:40:14 +00:00
José Braulio González Valido
9ac12d212f
test(ai-builder): Let seeded eval credentials pass their connection test (no-changelog) (#35675)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:40:27 +00:00
José Braulio González Valido
1575fce711
fix(ai-builder): Bound eval lane requests and stop abandoned mocked executions (no-changelog) (#35228)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:23:37 +00:00
José Braulio González Valido
166771c77d
feat(ai-builder): Push a seeded case, and hand the agent its workflow (no-changelog) (#35304)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 16:09:01 +00:00
José Braulio González Valido
7c61f2826a
fix(ai-builder): Give a case's scenario data tables per-run names (no-changelog) (#35543)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 12:48:00 +00:00
José Braulio González Valido
d57a81ac60
fix(ai-builder): Keep model-provider outages out of eval builder scores (no-changelog) (#35293)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 10:52:11 +00:00
José Braulio González Valido
5d8f2353bf
fix(ai-builder): Give a restored seed workflow a unique name and evict leftovers (no-changelog) (#35299)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 17:26:21 +00:00
José Braulio González Valido
e62417356f
feat(ai-builder): Collapse the eval case's three seed keys into one seed union (no-changelog) (#35292)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 14:37:06 +00:00
José Braulio González Valido
a2b8383bf9
feat(ai-builder): Let eval cases set up a working credential (no-changelog) (#35410)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 11:21:11 +00:00
Rodrigo Santos da Silva
a25a1ec608
feat: Add manual credential-setup path to instance-ai eval harness (no-changelog) (#35197)
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-03 09:11:25 +00:00
José Braulio González Valido
be12a4ccdc
feat(ai-builder): Let eval runs exercise the builder's web search (no-changelog) (#35274)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:31:10 +00:00
Mutasem Aldmour
22c8117321
fix(ai-builder): Route eval report artifacts through --output-dir (no-changelog) (#35083)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 11:04:42 +00:00
José Braulio González Valido
2ae18ad9f6
feat(ai-builder): Validate the eval conversation-seed message envelope (no-changelog) (#35138)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:58:55 +00:00