Commit Graph

31 Commits

Author SHA1 Message Date
Riqwan Thamir
d13b63054b
ci: Create ephemeral sandboxes in Instance AI eval runs (no-changelog) (#39459)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-24 11:53:58 +00:00
Michael Drury
684b930fa6
feat(core): Improve Agent Builder evaluation and intent routing (#38329) 2026-09-14 13:13:51 +00:00
Jaakko Husso
02464a1ca1
test: Use the hosted sandbox service when it is configured (#37297)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 13:12:23 +00:00
Riqwan Thamir
91ebeecb4e
feat(core): Improve Instance AI model experiments (no-changelog) (#35034)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-13 10:15:03 +00:00
José Braulio González Valido
1575fce711
fix(ai-builder): Bound eval lane requests and stop abandoned mocked executions (no-changelog) (#35228)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:23:37 +00:00
Charlie Kolb
e41aae0fb6
ci: Show workflow_dispatch inputs in workflow run names (#35496)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 09:46:14 +00:00
José Braulio González Valido
be12a4ccdc
feat(ai-builder): Let eval runs exercise the builder's web search (no-changelog) (#35274)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:31:10 +00:00
José Braulio González Valido
32d4c1a123
ci(ai-builder): Point eval workflows at the consolidated LangTracer baseline suite (no-changelog) (#34402)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 13:55:41 +00:00
José Braulio González Valido
a774a18b95
ci(ai-builder): Fetch eval test cases from LangTracer (no-changelog) (#33982)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:52:27 +00:00
José Braulio González Valido
2233d720ae
ci: Bound and instrument Instance AI eval lane containers (no-changelog) (#33903)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:28:46 +00:00
José Braulio González Valido
a8f1b2c1b8
ci: Scope Instance AI evals dispatcher to PR gating (no-changelog) (#33499)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 07:29:09 +00:00
José Braulio González Valido
af009e64a1
fix(ai-builder): Trace eval backend threads and add model input to exec evals CI (no-changelog) (#33458)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:17:47 +00:00
José Braulio González Valido
0b992c8ac1
fix(ai-builder): Send eval build traces to a separate LangSmith project (no-changelog) (#33178)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 09:13:57 +00:00
José Braulio González Valido
c3741ce12a
feat(ai-builder): Re-run Instance AI evals against the PR head on demand (no-changelog) (#33148)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:00:43 +00:00
José Braulio González Valido
05c554dad5
ci: Align instance AI eval workflow with main CI docker image build (no-changelog) (#32094)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 07:35:00 +00:00
José Braulio González Valido
46a222a15b
ci: Re-enable Instance AI evals on PRs with cost-bounded trigger (no-changelog) (#31430)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-05 08:40:23 +00:00
José Braulio González Valido
f25e12de87
ci(ai-builder): Add iterations + experiment-name inputs to instance-ai eval dispatch (no-changelog) (#31631)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 09:57:08 +00:00
Tomi Turtiainen
3c61c305e0
ci: Replace start-sandbox.ts with pnpm services --network (#31379) 2026-06-01 18:48:45 +00:00
Tomi Turtiainen
93a176b997
test: Add sandbox service test container (#31192) 2026-05-28 13:49:32 +00:00
Tomi Turtiainen
c5c8983f34
ci: Integrate n8n sandbox service into Instance AI evals (no-changelog) (#31051) 2026-05-27 08:55:54 +00:00
José Braulio González Valido
96a9521394
ci: Use PR head ref in eval experiment names (no-changelog) (#30898)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 11:06:50 +00:00
José Braulio González Valido
81ea56fa6b
test(ai-builder): Add multi-turn capability for IAI evals (no-changelog) (#30586)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:03:35 +00:00
José Braulio González Valido
e9b1c7c48f
chore(ai-builder): Tag Daytona sandboxes with name prefix and labels (no-changelog) (#30697)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 12:43:36 +00:00
José Braulio González Valido
be220ab32a
ci: Split Instance AI evals to own workflow, fire on approval only (no-changelog) (#30815)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 13:38:53 +00:00
n8n-cat-bot[bot]
90b215939f
ci: Pass workflow inputs through env vars in run steps (#30606)
Co-authored-by: n8n-cat-bot[bot] <n8n-cat-bot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 08:20:49 +00:00
José Braulio González Valido
95cf41c37c
chore(core): Enable Daytona sandbox in Instance AI evals (no-changelog) (#29931)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 07:43:04 +00:00
José Braulio González Valido
bbe3e2d148
feat(ai-builder): Add per-PR eval regression detection vs LangSmith baseline (#29456)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 08:15:08 +00:00
José Braulio González Valido
4fd68bfc99
ci(ai-builder): Parallelize Instance AI eval CI across multiple n8n containers (#29545)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 07:22:28 +00:00
José Braulio González Valido
16e5f9572f
feat(ai-builder): Add LangSmith integration for workflow eval tracking (no-changelog) (#28835)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 08:47:02 +00:00
Luca Mattiazzi
714981eea3
feat: Add multiple runs to instanceAI eval (no-changelog) (#28493)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-22 07:11:10 +00:00
José Braulio González Valido
560f300716
test: Add Instance AI workflow evals CI pipeline (no-changelog) (#28366)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 14:15:41 +00:00