n8n/packages/@n8n/instance-ai/evaluations/__tests__
Anne Aguirre 5c6544af4a
feat(core): Port agent builder skills and tools to Instance AI (no-changelog) (#33384)
Co-authored-by: Robin Braumann <robin.braumann@n8n.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: bjorger <50590409+bjorger@users.noreply.github.com>
Co-authored-by: Michael Drury <michael.drury@n8n.io>
Co-authored-by: Michael Drury <me@michaeldrury.co.uk>
2026-07-08 12:41:12 +00:00
..
aggregator.test.ts fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) (#33562) 2026-07-07 08:29:53 +00:00
args.test.ts test: Parallelize MCP workflow evals across build lanes (no-changelog) (#33468) 2026-07-03 08:33:42 +00:00
build-expectations-verifier.test.ts feat(ai-builder): Author-level conversation expectations on eval test cases (no-changelog) (#31787) 2026-06-09 10:06:07 +00:00
capture-run-debug.test.ts feat(core): Add ability to view agent traces locally under a flag (no-changelog) (#32397) 2026-06-17 08:36:38 +00:00
case-files-schema.test.ts test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224) 2026-07-07 12:49:32 +00:00
checklist-verifier.test.ts fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) (#33562) 2026-07-07 08:29:53 +00:00
ci-metadata.test.ts ci: Record git provenance metadata on eval experiments (no-changelog) (#33024) 2026-06-29 09:12:31 +00:00
collect-expectations.test.ts feat(core): Make instanceAI evals more mcp-friendly (no-changelog) (#32916) 2026-06-25 06:33:50 +00:00
comparison-compare.test.ts chore: Migrate instance-ai from Jest to Vitest (#31463) 2026-06-03 06:48:27 +00:00
comparison-format.test.ts feat(core): Port agent builder skills and tools to Instance AI (no-changelog) (#33384) 2026-07-08 12:41:12 +00:00
comparison-gate.test.ts fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) (#33562) 2026-07-07 08:29:53 +00:00
comparison-statistics.test.ts feat(ai-builder): Add per-PR eval regression detection vs LangSmith baseline (#29456) 2026-05-06 08:15:08 +00:00
conversation-seed.test.ts fix(ai-builder): Reconstruct file-based seeded workflows and judge the whole conversation (no-changelog) (#33048) 2026-06-26 10:06:20 +00:00
conversation-text.test.ts fix(ai-builder): Reconstruct file-based seeded workflows and judge the whole conversation (no-changelog) (#33048) 2026-06-26 10:06:20 +00:00
data-discovery.test.ts fix(core): Move instance AI builds to a skill (#31412) 2026-06-02 16:06:00 +00:00
data-workflows-schema.test.ts test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224) 2026-07-07 12:49:32 +00:00
data-workflows.test.ts test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224) 2026-07-07 12:49:32 +00:00
dataset-sync.test.ts test: Reduce non-builder noise in MCP workflow evals (no-changelog) (#33565) 2026-07-06 13:48:11 +00:00
event-parser.test.ts fix(ai-builder): Track consolidated-tool data table creations for eval cleanup (no-changelog) (#33580) 2026-07-06 09:51:36 +00:00
export-latest-verifier-request.test.ts feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951) 2026-06-09 12:10:10 +00:00
fetch-baseline.test.ts feat(core): Enable LangSmith for mcp evaluations (no-changelog) (#32995) 2026-06-25 09:33:22 +00:00
lane-allocator.test.ts test(ai-builder): Add multi-turn capability for IAI evals (no-changelog) (#30586) 2026-05-21 13:03:35 +00:00
lanes.test.ts ci(ai-builder): Parallelize Instance AI eval CI across multiple n8n containers (#29545) 2026-04-30 07:22:28 +00:00
langsmith-seed.test.ts feat(ai-builder): Capture compiled workflow JSON as a trace event for eval seed reconstruction (no-changelog) (#33302) 2026-07-06 15:03:38 +00:00
langtracer-client.test.ts feat(core): Add eval:langtracer-push to push eval cases to a suite (no-changelog) (#33781) 2026-07-08 07:52:23 +00:00
langtracer-normalize.test.ts feat(ai-builder): Dual-tenant LangSmith reads for Instance AI evals (no-changelog) (#33230) 2026-06-29 17:17:14 +00:00
langtracer-provider.test.ts feat(ai-builder): Source eval test cases from LangTracer (no-changelog) (#33067) 2026-06-29 10:17:49 +00:00
langtracer-push.test.ts feat(core): Add eval:langtracer-push to push eval cases to a suite (no-changelog) (#33781) 2026-07-08 07:52:23 +00:00
langtracer-to-exported.test.ts feat(core): Add eval:langtracer-push to push eval cases to a suite (no-changelog) (#33781) 2026-07-08 07:52:23 +00:00
mcp-builder-timeout.test.ts test: Parallelize MCP workflow evals across build lanes (no-changelog) (#33468) 2026-07-03 08:33:42 +00:00
mcp-builder.test.ts test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224) 2026-07-07 12:49:32 +00:00
normalize-workflow.test.ts chore: Migrate instance-ai from Jest to Vitest (#31463) 2026-06-03 06:48:27 +00:00
pair-records.test.ts feat: Add pairwise workflow eval pipeline (#29123) 2026-05-04 13:26:27 +00:00
prebuilt-coverage.test.ts feat(ai-builder): Source eval test cases from LangTracer (no-changelog) (#33067) 2026-06-29 10:17:49 +00:00
prebuilt-workflows.test.ts feat(core): Allow deleting pre-built workflows after evaluating (no-changelog) (#31960) 2026-06-09 08:50:31 +00:00
redact.test.ts feat(ai-builder): Author-level conversation expectations on eval test cases (no-changelog) (#31787) 2026-06-09 10:06:07 +00:00
reshape.test.ts feat(ai-builder): Make eval executionScenarios optional for build-only cases (no-changelog) (#33252) 2026-06-30 09:41:14 +00:00
run-debug-report.test.ts fix(ai-builder): Guard optional conversation in run-debug report label (no-changelog) (#32883) 2026-06-25 12:24:26 +00:00
runner-prebuilt.test.ts feat(ai-builder): Add per-test-case eval credentials via thread credential-view pinning (no-changelog) (#32161) 2026-06-22 14:02:57 +00:00
runner-thread-id.test.ts fix(core): Teach the builder the SDK language subset and stop repeated invalid repairs (#31911) 2026-06-09 07:42:28 +00:00
runner-workflow-checks.test.ts chore: Migrate instance-ai from Jest to Vitest (#31463) 2026-06-03 06:48:27 +00:00
sandbox-config.test.ts feat(core): Use n8n default sandbox for Instance AI (no-changelog) (#31335) 2026-06-04 08:31:27 +00:00
select-expectations.test.ts feat(core): Make instanceAI evals more mcp-friendly (no-changelog) (#32916) 2026-06-25 06:33:50 +00:00
stub-services.test.ts feat: Add pairwise workflow eval pipeline (#29123) 2026-05-04 13:26:27 +00:00
subagent-runner.test.ts feat(ai-builder): Author-level conversation expectations on eval test cases (no-changelog) (#31787) 2026-06-09 10:06:07 +00:00
transcript-from-events.test.ts test(ai-builder): Richer multi-turn eval framework (no-changelog) (#32054) 2026-06-11 12:57:08 +00:00
transient-error.test.ts fix(core): Harden MCP eval execution against transient network errors (no-changelog) (#33398) 2026-07-02 08:51:49 +00:00
user-proxy.test.ts fix(core): Make AI Assistant workflow verification and setup repeatable (#33085) 2026-06-26 13:39:28 +00:00
verification-artifact.test.ts fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) (#33562) 2026-07-07 08:29:53 +00:00
verifier.test.ts feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951) 2026-06-09 12:10:10 +00:00
workflow-checks-applicability.test.ts feat(ai-builder): Surface WHAT-dimension binary checks per built workflow (no-changelog) (#30932) 2026-05-26 12:18:52 +01:00
workflow-checks-errors.test.ts fix: Improve AI Assistant post-build follow-ups (#33419) 2026-07-02 11:34:21 +00:00
workflow-checks-feedback.test.ts fix: Improve AI Assistant post-build follow-ups (#33419) 2026-07-02 11:34:21 +00:00
workflow-discovery.test.ts feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951) 2026-06-09 12:10:10 +00:00
workflow-report.test.ts test: Reduce non-builder noise in MCP workflow evals (no-changelog) (#33565) 2026-07-06 13:48:11 +00:00
workspace-tool-contract.test.ts fix(ai-builder): Reconstruct file-based seeded workflows and judge the whole conversation (no-changelog) (#33048) 2026-06-26 10:06:20 +00:00