José Braulio González Valido
b0a10229a0
refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) ( #34834 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 09:01:09 +00:00
José Braulio González Valido
b93a99945b
feat(ai-builder): Harden the eval harness — crash-recovery journal, parity fixes, extension points (no-changelog) ( #34747 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 12:25:55 +00:00
Milorad FIlipović
c5d65998da
test: Reduce non-builder noise in MCP workflow evals (no-changelog) ( #33565 )
2026-07-06 13:48:11 +00:00
Milorad FIlipović
579656db2e
test: Parallelize MCP workflow evals across build lanes (no-changelog) ( #33468 )
2026-07-03 08:33:42 +00:00
José Braulio González Valido
23d70d1bb9
feat(ai-builder): Source eval test cases from LangTracer (no-changelog) ( #33067 )
...
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 10:17:49 +00:00
Milorad FIlipović
e12ce0ead0
feat(core): Enable LangSmith for mcp evaluations (no-changelog) ( #32995 )
2026-06-25 09:33:22 +00:00
Milorad FIlipović
9bb77cc55e
feat(core): Allow deleting pre-built workflows after evaluating (no-changelog) ( #31960 )
2026-06-09 08:50:31 +00:00
José Braulio González Valido
2a1f3d6b2d
feat(ai-builder): Add datasets field for PR-tier suite (no-changelog) ( #31429 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 13:30:31 +00:00
Arvin A
55d8b59a48
feat(core): Stream tool calls and ship M3 fixtures from LLM eval wire server (no-changelog) ( #30983 )
2026-05-27 14:53:43 +01:00
José Braulio González Valido
30d9a168bc
feat(ai-builder): Add --prebuilt-workflows flag for eval CLI (no-changelog) ( #29830 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 11:47:29 +00:00
José Braulio González Valido
bbe3e2d148
feat(ai-builder): Add per-PR eval regression detection vs LangSmith baseline ( #29456 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 08:15:08 +00:00
José Braulio González Valido
4fd68bfc99
ci(ai-builder): Parallelize Instance AI eval CI across multiple n8n containers ( #29545 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 07:22:28 +00:00
José Braulio González Valido
16e5f9572f
feat(ai-builder): Add LangSmith integration for workflow eval tracking (no-changelog) ( #28835 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 08:47:02 +00:00
Luca Mattiazzi
714981eea3
feat: Add multiple runs to instanceAI eval (no-changelog) ( #28493 )
...
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-22 07:11:10 +00:00
José Braulio González Valido
560f300716
test: Add Instance AI workflow evals CI pipeline (no-changelog) ( #28366 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 14:15:41 +00:00
José Braulio González Valido
fef91c97dd
feat(ai-builder): Add --keep-workflows flag and fix eval execution errors (no-changelog) ( #28129 )
...
Build: Benchmark Image / build (push) Waiting to run
CI: Master (Build, Test, Lint) / Build for Github Cache (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (22.x) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (24.14.1) (push) Waiting to run
CI: Master (Build, Test, Lint) / Unit tests (25.x) (push) Waiting to run
CI: Master (Build, Test, Lint) / Lint (push) Waiting to run
CI: Master (Build, Test, Lint) / Performance (push) Waiting to run
CI: Master (Build, Test, Lint) / Notify Slack on failure (push) Blocked by required conditions
Util: Sync API Docs / sync-public-api (push) Waiting to run
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 17:35:04 +00:00
José Braulio González Valido
2383749980
feat(ai-builder): Workflow evaluation framework with LLM mock execution ( #27818 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Arvin A <51036481+DeveloperTheExplorer@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-07 13:31:16 +00:00
