José Braulio González Valido
1575fce711
fix(ai-builder): Bound eval lane requests and stop abandoned mocked executions (no-changelog) ( #35228 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:23:37 +00:00
José Braulio González Valido
166771c77d
feat(ai-builder): Push a seeded case, and hand the agent its workflow (no-changelog) ( #35304 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 16:09:01 +00:00
oleg
f9bddc7ff3
fix: Include node behavior flags in instance-ai eval judge context (no-changelog) ( #35644 )
2026-08-05 14:59:37 +00:00
José Braulio González Valido
7c61f2826a
fix(ai-builder): Give a case's scenario data tables per-run names (no-changelog) ( #35543 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 12:48:00 +00:00
José Braulio González Valido
a2b8383bf9
feat(ai-builder): Let eval cases set up a working credential (no-changelog) ( #35410 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 11:21:11 +00:00
José Braulio González Valido
ba31860da7
fix(core): Require an authenticated session for the MCP registry test seed endpoint (no-changelog) ( #34880 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 13:20:38 +00:00
Riqwan Thamir
c0fe8ba4db
feat(editor): Track Instance AI entry point as action_source on builder messages ( #34671 )
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-22 13:24:29 +00:00
José Braulio González Valido
ee46e64d50
chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) ( #34673 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 08:18:41 +00:00
José Braulio González Valido
5fd800de20
feat(ai-builder): Support execution scenarios for first-class agents in evals ( #34384 )
...
Build: Benchmark Image / build (push) Waiting to run
CI: Master (Build, Test, Lint) / Build for Github Cache (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (22.22.3) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.0) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Performance (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Util: Sync API Docs / sync-public-api (push) Waiting to run
Test: E2E Performance / prepare-docker (push) Has been cancelled
Release: Storybook / Cloudflare Pages (push) Has been cancelled
Test: E2E Performance / build-and-test-performance (push) Has been cancelled
Test: E2E Performance / Canvas Perf Sentinels (push) Has been cancelled
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:54:06 +00:00
Mutasem Aldmour
02af123574
feat: Add typed seed data tables with rows on execution scenarios ( #34420 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 17:30:19 +00:00
Benjamin Schroth
d6dcf3b749
test(ai-builder): Support evals for agent building and for config evals (no-changelog) ( #33888 )
...
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-07-15 12:34:21 +00:00
José Braulio González Valido
2233d720ae
ci: Bound and instrument Instance AI eval lane containers (no-changelog) ( #33903 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:28:46 +00:00
Milorad FIlipović
579656db2e
test: Parallelize MCP workflow evals across build lanes (no-changelog) ( #33468 )
2026-07-03 08:33:42 +00:00
José Braulio González Valido
2ad33a1303
fix(ai-builder): Fix AI-node eval mock-execution timeouts (no-changelog) ( #33323 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 08:33:51 +00:00
José Braulio González Valido
fe11536df1
feat(ai-builder): Add conversation pre-seeding to workflow evals (no-changelog) ( #32196 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-23 17:30:30 +00:00
José Braulio González Valido
13239d1cfe
feat(ai-builder): Add per-test-case eval credentials via thread credential-view pinning (no-changelog) ( #32161 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-22 14:02:57 +00:00
Riqwan Thamir
c7143e0e17
feat(core): Await memory tasks when running agent through evals (no-changelog) ( #32693 )
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-22 11:11:56 +00:00
Riqwan Thamir
b60bc923d4
feat(core): Add ability to view agent traces locally under a flag (no-changelog) ( #32397 )
2026-06-17 08:36:38 +00:00
Albert Alises
4271d42398
fix(core): Teach the builder the SDK language subset and stop repeated invalid repairs ( #31911 )
2026-06-09 07:42:28 +00:00
José Braulio González Valido
fd9e070071
fix(ai-builder): Send projectId when creating eval threads (no-changelog) ( #31921 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 15:32:16 +00:00
Bernhard Wittmann
f4e998f245
fix(core): Prefer MCP registry nodes when wiring AI Agent tools ( #30774 )
...
Co-authored-by: Roman Davydchuk <roman.davydchuk@n8n.io>
2026-06-04 12:24:13 +00:00
Albert Alises
6cd07e5b42
fix: Add item-flow builder guidance (no-changelog) ( #31660 )
2026-06-04 09:23:20 +00:00
Albert Alises
332d2df44e
fix(core): Move instance AI builds to a skill ( #31412 )
2026-06-02 16:06:00 +00:00
Arvin A
55d8b59a48
feat(core): Stream tool calls and ship M3 fixtures from LLM eval wire server (no-changelog) ( #30983 )
2026-05-27 14:53:43 +01:00
Albert Alises
17b64da0f0
fix: Add Switch fallback output guidance for workflow builder ( #30449 )
2026-05-15 07:33:25 +00:00
Bernhard Wittmann
b445221c6a
feat: Computer-use evaluation harness (no-changelog) ( #29797 )
...
Co-authored-by: Elias Meire <elias@meire.dev>
2026-05-12 08:36:12 +00:00
Bernhard Wittmann
68560fbb9a
refactor: Extract shared eval helpers (no-changelog) ( #29800 )
2026-05-07 08:05:01 +00:00
Charlie Kolb
f775604c25
refactor: Split up instance-ai confirmation endpoint DTO by action ( #29179 )
2026-05-04 10:47:38 +00:00
Benjamin Schroth
c961849226
feat(ai-builder): Add sub-agent evaluation harness with binary checks (no-changelog) ( #28289 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 07:50:46 +00:00
José Braulio González Valido
16e5f9572f
feat(ai-builder): Add LangSmith integration for workflow eval tracking (no-changelog) ( #28835 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 08:47:02 +00:00
José Braulio González Valido
560f300716
test: Add Instance AI workflow evals CI pipeline (no-changelog) ( #28366 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 14:15:41 +00:00
José Braulio González Valido
fef91c97dd
feat(ai-builder): Add --keep-workflows flag and fix eval execution errors (no-changelog) ( #28129 )
...
Build: Benchmark Image / build (push) Waiting to run
CI: Master (Build, Test, Lint) / Build for Github Cache (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (22.x) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.14.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (25.x) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Performance (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Util: Sync API Docs / sync-public-api (push) Waiting to run
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 17:35:04 +00:00
José Braulio González Valido
2383749980
feat(ai-builder): Workflow evaluation framework with LLM mock execution ( #27818 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Arvin A <51036481+DeveloperTheExplorer@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-07 13:31:16 +00:00
