José Braulio González Valido
5fd800de20
feat(ai-builder): Support execution scenarios for first-class agents in evals ( #34384 )
...
Build: Benchmark Image / build (push) Waiting to run
CI: Master (Build, Test, Lint) / Build for Github Cache (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (22.22.3) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.18.0) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Performance (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Util: Sync API Docs / sync-public-api (push) Waiting to run
Test: E2E Performance / prepare-docker (push) Has been cancelled
Release: Storybook / Cloudflare Pages (push) Has been cancelled
Test: E2E Performance / build-and-test-performance (push) Has been cancelled
Test: E2E Performance / Canvas Perf Sentinels (push) Has been cancelled
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:54:06 +00:00
Benjamin Schroth
d6dcf3b749
test(ai-builder): Support evals for agent building and for config evals (no-changelog) ( #33888 )
...
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-07-15 12:34:21 +00:00
Anne Aguirre
5c6544af4a
feat(core): Port agent builder skills and tools to Instance AI (no-changelog) ( #33384 )
...
Co-authored-by: Robin Braumann <robin.braumann@n8n.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: bjorger <50590409+bjorger@users.noreply.github.com>
Co-authored-by: Michael Drury <michael.drury@n8n.io>
Co-authored-by: Michael Drury <me@michaeldrury.co.uk>
2026-07-08 12:41:12 +00:00
Robin Braumann
f25699fe2c
test(core): Extend Instance AI intent eval fixtures (no-changelog) ( #33224 )
...
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 12:49:32 +00:00
José Braulio González Valido
a89f85b052
fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) ( #33562 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:29:53 +00:00
Milorad FIlipović
c5d65998da
test: Reduce non-builder noise in MCP workflow evals (no-changelog) ( #33565 )
2026-07-06 13:48:11 +00:00
oleg
6f0d30fbb1
fix: Improve AI Assistant post-build follow-ups ( #33419 )
2026-07-02 11:34:21 +00:00
José Braulio González Valido
3380ca3790
fix(ai-builder): Reconstruct file-based seeded workflows and judge the whole conversation (no-changelog) ( #33048 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:06:20 +00:00
José Braulio González Valido
5fc00bf17d
fix(ai-builder): Guard optional conversation in run-debug report label (no-changelog) ( #32883 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 12:24:26 +00:00
José Braulio González Valido
fe11536df1
feat(ai-builder): Add conversation pre-seeding to workflow evals (no-changelog) ( #32196 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-23 17:30:30 +00:00
Riqwan Thamir
b60bc923d4
feat(core): Add ability to view agent traces locally under a flag (no-changelog) ( #32397 )
2026-06-17 08:36:38 +00:00
José Braulio González Valido
4573969629
fix(ai-builder): Harden eval mock layer for text responses, trigger binary and lookup attribution (no-changelog) ( #32179 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 11:04:59 +00:00
José Braulio González Valido
5fa988104b
test(ai-builder): Richer multi-turn eval framework (no-changelog) ( #32054 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 12:57:08 +00:00
oleg
77b95c611d
feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) ( #31951 )
2026-06-09 12:10:10 +00:00
José Braulio González Valido
7791a940dd
feat(ai-builder): Author-level conversation expectations on eval test cases (no-changelog) ( #31787 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 10:06:07 +00:00
José Braulio González Valido
64f25b2ad8
feat(core): Show eval-mock executions in canvas list (no-changelog) ( #31244 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 13:14:51 +00:00
José Braulio González Valido
a1369e7736
fix(core): Preserve execution output fidelity in eval verifier artifact (no-changelog) ( #30989 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 08:55:01 +00:00
José Braulio González Valido
700b32237f
feat(ai-builder): Surface WHAT-dimension binary checks per built workflow (no-changelog) ( #30932 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 12:18:52 +01:00
José Braulio González Valido
81ea56fa6b
test(ai-builder): Add multi-turn capability for IAI evals (no-changelog) ( #30586 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:03:35 +00:00
José Braulio González Valido
16e5f9572f
feat(ai-builder): Add LangSmith integration for workflow eval tracking (no-changelog) ( #28835 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 08:47:02 +00:00
José Braulio González Valido
560f300716
test: Add Instance AI workflow evals CI pipeline (no-changelog) ( #28366 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 14:15:41 +00:00
José Braulio González Valido
2383749980
feat(ai-builder): Workflow evaluation framework with LLM mock execution ( #27818 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Arvin A <51036481+DeveloperTheExplorer@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-07 13:31:16 +00:00
