This commit is contained in:
Mutasem Aldmour 2026-05-07 10:26:45 +02:00
commit cdbf6ce89b
No known key found for this signature in database
GPG Key ID: 3DFA8122BB7FD6B8
1391 changed files with 103083 additions and 18581 deletions

View File

@ -0,0 +1,150 @@
---
description: >-
Checks if a community pull request is ready for human review. Verifies CLA
signature, PR title format, description completeness, test coverage, and
cubic-dev-ai issues. Use when given a PR number or branch name to review,
or when the user says /community-pr-review, /pr-review, or asks to check if
a PR is ready for review.
allowed-tools: Bash(gh:*), Bash(git:*), Read, Glob, Grep
---
# Community PR Review
Given a PR number or branch name, determine whether it is ready for human review.
## Steps
### 1. Resolve the PR
If given a branch name, find the PR number first:
```bash
gh pr view <branch> --repo n8n-io/n8n --json number --jq .number
```
### 2. Fetch PR data
```bash
gh pr view <number> --repo n8n-io/n8n \
--json number,title,body,author,headRefName,headRefOid,files,isDraft,state
```
Fetch in parallel:
```bash
# CLA commit status (primary signal) — statuses are newest-first; use the first returned entry
gh api --paginate "repos/n8n-io/n8n/commits/<headRefOid>/statuses" \
--jq '[.[] | select(.context == "license/cla") | {state, description}] | first'
# CLAassistant issue comment (fallback when no commit status) — use the last returned entry
gh api --paginate "repos/n8n-io/n8n/issues/<number>/comments" \
--jq '[.[] | select(.user.login == "CLAassistant") | .body] | last'
# cubic-dev-ai PR review comments (streamed so results concatenate cleanly across pages)
gh api --paginate "repos/n8n-io/n8n/pulls/<number>/comments" \
--jq '.[] | select(.user.login == "cubic-dev-ai[bot]") | {body: .body, path: .path}'
```
### 3. Run the five checks
#### A. CLA signed
Check the `license/cla` commit status first; fall back to the CLAassistant comment if no status exists.
**Commit status** (`context == "license/cla"`):
- `state: "success"` → ✅ signed
- `state: "failure"` or `state: "error"` → ❌ not signed
- `state: "pending"` → ⏳ pending
- Not present → fall back to comment
**CLAassistant issue comment** (fallback):
- Body contains `"All committers have signed the CLA."` → ✅ signed
- Body contains `"not signed"` or a link to sign → ❌ not signed
- No comment → ❌ treat as not signed
#### B. PR title format
For all types except `revert`, the title must match:
```
^(feat|fix|perf|test|docs|refactor|build|ci|chore)(\([a-zA-Z0-9 ]+( Node)?\))?!?: [A-Z].+[^.]$
```
For `revert` titles, the summary is the original commit header (which starts with a lowercase type), so capitalization is not enforced:
```
^revert(\([a-zA-Z0-9 ]+( Node)?\))?!?: .+[^.]$
```
- Type must be one of: `feat fix perf test docs refactor build ci chore revert`
- Scope is optional, in parentheses e.g. `(editor)` or `(Slack Node)`
- Breaking changes: `!` before the colon
- Summary: starts with capital letter (lowercase allowed for `revert:`), no trailing period
- No Linear ticket IDs in the title (e.g. `N8N-1234`)
#### C. PR description completeness
1. **Summary** (`## Summary`) — must have non-empty content below the heading (not just the HTML comment).
2. **Related tickets** (`## Related Linear tickets, Github issues, and Community forum posts`) — acceptable content: a URL (`http`), a GitHub closing keyword (`closes #N`, `fixes #N`, `resolves #N`, etc.), or empty. Only flag if the section heading is missing entirely.
3. **Checklist** (`## Review / Merge checklist`) — all four items must be present. Unchecked checkboxes are expected for community PRs; do **not** flag them as missing.
#### D. Tests
Skip this check if the PR type (from the title) is `docs`, `ci`, `chore`, or `build`.
Otherwise:
1. Identify source files changed: non-test files under `packages/` from the `files` list.
2. If there are source file changes, check out the PR in a temporary worktree:
```bash
git fetch origin pull/<number>/head:pr/<number>
git worktree add /tmp/pr-<number>-review pr/<number>
```
3. Read the changed source files from the worktree to understand whether the changes introduce logic that warrants tests (new functions, bug fixes, behaviour changes, data transformations). Pure config changes, type-only changes, and trivial renames do not require tests.
4. Look for matching test files (`*.test.ts`, `*.spec.ts`, files inside `__tests__/`) among the changed files.
5. **Always clean up the worktree**, even if a previous check failed:
```bash
git worktree remove /tmp/pr-<number>-review --force
git branch -D pr/<number>
```
Report:
- ✅ Tests present, or change does not require tests
- ❌ Source logic changed but no test files found
#### E. cubic-dev-ai issues
Review the PR review comments fetched in step 2. `cubic-dev-ai[bot]` leaves comments for every issue it finds.
- No comments from `cubic-dev-ai[bot]`, or every comment explicitly states no issues were found → ✅
- Any other comment → ❌ report the total count and priority breakdown (e.g. "3 issues: 1× P1, 1× P2, 1× P3")
### 4. Output
Always output valid JSON in this exact shape:
```json
{
"readyForReview": <true if all passing checks allow merge, false otherwise>,
"messageForUser": "<Human-readable summary of what needs to change, written as if posted directly to the PR contributor. 'N/A' if nothing is needed.>",
"checks": {
"CLA": <true if signed, false if not signed or pending>,
"Title": <true if title matches convention, false otherwise>,
"Description": <true if all three template sections are complete, false otherwise>,
"TestsNeeded": <true if the code changes require tests, false if not applicable>,
"TestsIncluded": <true if test files are present in the PR, false otherwise>,
"CubicIssues": <true if cubic-dev-ai raised issues, false if no issues>
}
}
```
`readyForReview` is `true` only when: `CLA`, `Title`, and `Description` are all `true`; `CubicIssues` is `false`; and either `TestsNeeded` is `false` or `TestsIncluded` is `true`.
`messageForUser` should be a short, friendly message directed at the contributor listing exactly what they need to address. If `readyForReview` is `true`, set it to `"N/A"`.
Output nothing other than the JSON block.
## Notes
- Draft PRs — report all findings but note the PR is a draft.
- If the PR is already merged or closed, say so and skip the checks.
- Always remove the worktree even if earlier checks failed.

View File

@ -19,12 +19,15 @@ Reference these guidelines when:
- Follow guidelines in `packages/frontend/@n8n/design-system/src/styleguide/*.mdx`
- ALWAYS use CSS variables for styles from `packages/frontend/@n8n/design-system/src/css/_tokens.scss` or `packages/frontend/@n8n/design-system/src/css/_primtivies.scss`. Use hard-coded values only when no suitable tokens.
- ALWAYS prefer using existing components from `packages/frontend/@n8n/design-system/src/components`. Prefer components that aren't marked `@deprecated`.
- When reviewing UI changes or adding new components, follow `rules/web-interface-guidelines.md`
- Use `light-dark()` when alternating colors for ligh/dark mode
- When working with animations or transitions, ALWAYS prefer using mixins from `packages/frontend/@n8n/design-system/src/css/mixins/motion.scss`
- When reviewing animations, follow the guides in `rules/web-animation-guidelines.md`
- When reviewing UI changes or adding new components, follow `rules/web-interface-guidelines.md`
## Examples
- "Add a modal dialog for confirming workflow deletion" → Use `N8nDialog`
- "Add a dropdown to select workflow status" → Use `N8nDropdown` or `N8nSelect`
- "Add button with + icon to add new tiem" → Wrap `N8nButton` with `iconOnly` prop with `N8nTooltip` and wrap in `N8nTooltip`. Use `N8nIcon` and proper aria-label.
- "Add a destructive action button" → use `N8nButton` with `variant="destructive"`
- Make background color white/black → Use `var(--background--surface)` for white on light mode and "black" on dark mode
- "Make background color white/black" → Use `var(--background--surface)` for white on light mode and "black" on dark mode
- "Animate the title in gracefully" -> Use `fade-in-up` mixin from `motion.scss` with `var(--duration--base)`

View File

@ -0,0 +1,93 @@
# Web Motion Guidelines
Design and implement web animations that feel natural and purposeful
## Timing and Duration
## Duration Guidelines
| Element Type | Duration |
| --------------------------------- | --------- |
| Micro-interactions | 100-150ms |
| Standard UI (tooltips, dropdowns) | 150-250ms |
| Modals, drawers | 200-300ms |
**Rules:**
- UI animations should stay under 300ms
- Larger elements animate slower than smaller ones
- Exit animations can be ~20% faster than entrance
- Match duration to distance - longer travel = longer duration
### The Frequency
Determine how often users will see the animation:
- **100+ times/day** → No animation (or drastically reduced)
- **Occasional use** → Standard animation
- **Rare/first-time** → Can be more special
**Example:** Raycast never animates because users open it hundreds of times a day.
## When to Animate
**Do animate:**
- Enter/exit transitions for spatial consistency
- State changes that benefit from visual continuity
- Responses to user actions (feedback)
- Rarely-used interactions where delight adds value
**Don't animate:**
- Keyboard-initiated actions
- Hover effects on frequently-used elements
- Anything users interact with 100+ times daily
- When speed matters more than smoothness
## Performance
Prefer animating `transform` and `opacity`. These skip layout and paint stages, running entirely on the GPU.
**Avoid animating:**
- `padding`, `margin`, `height`, `width` (trigger layout)
- `blur` filters above 20px (expensive, especially Safari)
- CSS variables in deep component trees
### Optimization Techniques
```css
/* Force GPU acceleration */
.animated-element {
will-change: transform;
}
```
## Practical Tips
Quick reference for common scenarios. See [PRACTICAL-TIPS.md](PRACTICAL-TIPS.md) for detailed implementations.
| Scenario | Solution |
| ------------------------------- | ----------------------------------------------- |
| Make buttons feel responsive | Add `transform: scale(0.97)` on `:active` |
| Element appears from nowhere | Start from `scale(0.95)`, not `scale(0)` |
| Shaky/jittery animations | Add `will-change: transform` |
| Hover causes flicker | Animate child element, not parent |
| Popover scales from wrong point | Set `transform-origin` to trigger location |
| Sequential tooltips feel slow | Skip delay/animation after first tooltip |
| Small buttons hard to tap | Use 44px minimum hit area (pseudo-element) |
| Something still feels off | Add subtle blur (under 20px) to mask it |
| Hover triggers on mobile | Use `@media (hover: hover) and (pointer: fine)` |
## Easing Decision Flowchart
Is the element entering or exiting the viewport?
├── Yes → ease-out
└── No
├── Is it moving/morphing on screen?
│ └── Yes → ease-in-out
└── Is it a hover change?
├── Yes → ease
└── Is it constant motion?
├── Yes → linear
└── Default → ease-out

8
.github/CODEOWNERS vendored
View File

@ -1,6 +1,6 @@
packages/@n8n/db/src/migrations/ @n8n-io/migrations-review
.github/workflows @n8n-io/ci-admins
.github/scripts @n8n-io/ci-admins
.github/actions @n8n-io/ci-admins
.github/poutine-rules @n8n-io/ci-admins
.github/workflows @n8n-io/qa-dx
.github/scripts @n8n-io/qa-dx
.github/actions @n8n-io/qa-dx
.github/poutine-rules @n8n-io/qa-dx

View File

@ -54,6 +54,10 @@ runs:
echo "${EXPECTED_SHA256} install-safe-chain.sh" | sha256sum -c -
sh install-safe-chain.sh --ci
rm install-safe-chain.sh
# Exclude first-party @n8n/* packages from SafeChain's minimum-package-age
# filter so freshly-published versions stay visible to every subsequent
# step in the job (install, build, and publish).
echo "SAFE_CHAIN_MINIMUM_PACKAGE_AGE_EXCLUSIONS=@n8n/*,n8n,n8n-containers,n8n-core,n8n-editor-ui,n8n-node-dev,n8n-nodes-base,n8n-playwright,n8n-workflow" >> "$GITHUB_ENV"
shell: bash
- name: Install Dependencies

View File

@ -11,7 +11,7 @@ const exec = promisify(child_process.exec);
/**
* @param {string | semver.SemVer} currentVersion
*/
function generateExperimentalVersion(currentVersion) {
export function generateExperimentalVersion(currentVersion) {
const parsed = semver.parse(currentVersion);
if (!parsed) throw new Error(`Invalid version: ${currentVersion}`);
@ -28,84 +28,31 @@ function generateExperimentalVersion(currentVersion) {
return `${parsed.major}.${parsed.minor}.${parsed.patch}-exp.0`;
}
const rootDir = process.cwd();
const releaseType = /** @type { import('semver').ReleaseType | "experimental" } */ (
process.env.RELEASE_TYPE
);
assert.match(releaseType, /^(patch|minor|major|experimental|premajor)$/, 'Invalid RELEASE_TYPE');
// TODO: if releaseType is `auto` determine release type based on the changelog
const lastTag = (await exec('git describe --tags --match "n8n@*" --abbrev=0')).stdout.trim();
const packages = JSON.parse(
(
await exec(
`pnpm ls -r --only-projects --json | jq -r '[.[] | { name: .name, version: .version, path: .path, private: .private}]'`,
)
).stdout,
);
const packageMap = {};
for (let { name, path, version, private: isPrivate } of packages) {
if (isPrivate && path !== rootDir) {
continue;
}
if (path === rootDir) {
name = 'monorepo-root';
}
const isDirty = await exec(`git diff --quiet HEAD ${lastTag} -- ${path}`)
.then(() => false)
.catch((error) => true);
packageMap[name] = { path, isDirty, version };
/**
* @param {{ pnpm?: { overrides?: Record<string, string> }, overrides?: Record<string, string> }} pkg
* @returns {Record<string, string>}
*/
export function getOverrides(pkg) {
return { ...pkg.pnpm?.overrides, ...pkg.overrides };
}
assert.ok(
Object.values(packageMap).some(({ isDirty }) => isDirty),
'No changes found since the last release',
);
// Propagate isDirty transitively: if a package's dependency will be bumped,
// that package also needs a bump (e.g. design-system → editor-ui → cli).
// Detect root-level changes that affect resolved dep versions without touching individual
// package.json files: pnpm.overrides (applies to all specifiers)
// and pnpm-workspace.yaml catalog entries (applies only to deps using a "catalog:…" specifier).
const rootPkgJson = JSON.parse(await readFile(resolve(rootDir, 'package.json'), 'utf-8'));
const rootPkgJsonAtTag = await exec(`git show ${lastTag}:package.json`)
.then(({ stdout }) => JSON.parse(stdout))
.catch(() => ({}));
const getOverrides = (pkg) => ({ ...pkg.pnpm?.overrides, ...pkg.overrides });
const currentOverrides = getOverrides(rootPkgJson);
const previousOverrides = getOverrides(rootPkgJsonAtTag);
const changedOverrides = new Set(
Object.keys({ ...currentOverrides, ...previousOverrides }).filter(
(k) => currentOverrides[k] !== previousOverrides[k],
),
);
const parseWorkspaceYaml = (content) => {
/**
* @param {string} content
* @returns {Record<string, unknown>}
*/
export function parseWorkspaceYaml(content) {
try {
return /** @type {Record<string, unknown>} */ (parse(content) ?? {});
} catch {
return {};
}
};
const workspaceYaml = parseWorkspaceYaml(
await readFile(resolve(rootDir, 'pnpm-workspace.yaml'), 'utf-8').catch(() => ''),
);
const workspaceYamlAtTag = parseWorkspaceYaml(
await exec(`git show ${lastTag}:pnpm-workspace.yaml`)
.then(({ stdout }) => stdout)
.catch(() => ''),
);
const getCatalogs = (ws) => {
}
/**
* @param {Record<string, unknown>} ws
* @returns {Map<string, Record<string, string>>}
*/
export function getCatalogs(ws) {
const result = new Map();
if (ws.catalog) {
result.set('default', /** @type {Record<string,string>} */ (ws.catalog));
@ -116,98 +63,232 @@ const getCatalogs = (ws) => {
}
return result;
};
// changedCatalogEntries: Map<catalogName, Set<depName>>
const currentCatalogs = getCatalogs(workspaceYaml);
const previousCatalogs = getCatalogs(workspaceYamlAtTag);
const changedCatalogEntries = new Map();
for (const catalogName of new Set([...currentCatalogs.keys(), ...previousCatalogs.keys()])) {
const current = currentCatalogs.get(catalogName) ?? {};
const previous = previousCatalogs.get(catalogName) ?? {};
const changedDeps = new Set(
Object.keys({ ...current, ...previous }).filter((dep) => current[dep] !== previous[dep]),
);
if (changedDeps.size > 0) {
changedCatalogEntries.set(catalogName, changedDeps);
}
}
// Store full dep objects (with specifiers) so we can inspect "catalog:…" values below.
const depsByPackage = {};
for (const packageName in packageMap) {
const packageFile = resolve(packageMap[packageName].path, 'package.json');
const packageJson = JSON.parse(await readFile(packageFile, 'utf-8'));
depsByPackage[packageName] = /** @type {Record<string,string>} */ (
packageJson.dependencies ?? {}
/**
* @param {Record<string, string>} currentOverrides
* @param {Record<string, string>} previousOverrides
* @returns {Set<string>}
*/
export function computeChangedOverrides(currentOverrides, previousOverrides) {
return new Set(
Object.keys({ ...currentOverrides, ...previousOverrides }).filter(
(k) => currentOverrides[k] !== previousOverrides[k],
),
);
}
// Mark packages dirty if any dep had a root-level override or catalog version change.
for (const [packageName, deps] of Object.entries(depsByPackage)) {
if (packageMap[packageName].isDirty) continue;
for (const [dep, specifier] of Object.entries(deps)) {
if (changedOverrides.has(dep)) {
packageMap[packageName].isDirty = true;
break;
/**
* @param {Map<string, Record<string, string>>} currentCatalogs
* @param {Map<string, Record<string, string>>} previousCatalogs
* @returns {Map<string, Set<string>>}
*/
export function computeChangedCatalogEntries(currentCatalogs, previousCatalogs) {
const changedCatalogEntries = new Map();
for (const catalogName of new Set([...currentCatalogs.keys(), ...previousCatalogs.keys()])) {
const current = currentCatalogs.get(catalogName) ?? {};
const previous = previousCatalogs.get(catalogName) ?? {};
const changedDeps = new Set(
Object.keys({ ...current, ...previous }).filter((dep) => current[dep] !== previous[dep]),
);
if (changedDeps.size > 0) {
changedCatalogEntries.set(catalogName, changedDeps);
}
if (typeof specifier === 'string' && specifier.startsWith('catalog:')) {
const catalogName = specifier === 'catalog:' ? 'default' : specifier.slice(8);
if (changedCatalogEntries.get(catalogName)?.has(dep)) {
}
return changedCatalogEntries;
}
/**
* Mark packages as dirty if any dep had a root-level override or catalog version change.
* Mutates packageMap in place.
*
* @param {Record<string, { isDirty: boolean }>} packageMap
* @param {Record<string, Record<string, string>>} depsByPackage
* @param {Set<string>} changedOverrides
* @param {Map<string, Set<string>>} changedCatalogEntries
*/
export function markDirtyByRootChanges(
packageMap,
depsByPackage,
changedOverrides,
changedCatalogEntries,
) {
for (const [packageName, deps] of Object.entries(depsByPackage)) {
if (packageMap[packageName].isDirty) continue;
for (const [dep, specifier] of Object.entries(deps)) {
if (changedOverrides.has(dep)) {
packageMap[packageName].isDirty = true;
break;
}
if (typeof specifier === 'string' && specifier.startsWith('catalog:')) {
const catalogName = specifier === 'catalog:' ? 'default' : specifier.slice(8);
if (changedCatalogEntries.get(catalogName)?.has(dep)) {
packageMap[packageName].isDirty = true;
break;
}
}
}
}
}
let changed = true;
while (changed) {
changed = false;
for (const packageName in packageMap) {
if (packageMap[packageName].isDirty) continue;
if (Object.keys(depsByPackage[packageName]).some((dep) => packageMap[dep]?.isDirty)) {
packageMap[packageName].isDirty = true;
changed = true;
/**
* Propagate isDirty transitively: if a package's dependency will be bumped,
* that package also needs a bump. Mutates packageMap in place.
*
* @param {Record<string, { isDirty: boolean }>} packageMap
* @param {Record<string, Record<string, string>>} depsByPackage
*/
export function propagateDirtyTransitively(packageMap, depsByPackage) {
let changed = true;
while (changed) {
changed = false;
for (const packageName in packageMap) {
if (packageMap[packageName].isDirty) continue;
if (Object.keys(depsByPackage[packageName]).some((dep) => packageMap[dep]?.isDirty)) {
packageMap[packageName].isDirty = true;
changed = true;
}
}
}
}
// Keep the monorepo version up to date with the released version
packageMap['monorepo-root'].version = packageMap['n8n'].version;
for (const packageName in packageMap) {
const { path, version, isDirty } = packageMap[packageName];
const packageFile = resolve(path, 'package.json');
const packageJson = JSON.parse(await readFile(packageFile, 'utf-8'));
const dependencyIsDirty = Object.keys(packageJson.dependencies || {}).some(
(dependencyName) => packageMap[dependencyName]?.isDirty,
);
let newVersion = version;
if (isDirty || dependencyIsDirty) {
switch (releaseType) {
case 'experimental':
newVersion = generateExperimentalVersion(version);
break;
case 'premajor':
newVersion = semver.inc(
/**
* @param {string} version
* @param {import('semver').ReleaseType | 'experimental'} releaseType
* @returns {string}
*/
export function computeNewVersion(version, releaseType) {
switch (releaseType) {
case 'experimental':
return generateExperimentalVersion(version);
case 'premajor':
return /** @type {string} */ (
semver.inc(
version,
version.includes('-rc.') ? 'prerelease' : 'premajor',
undefined,
'rc',
);
break;
default:
newVersion = semver.inc(version, releaseType);
break;
}
)
);
default:
return /** @type {string} */ (semver.inc(version, releaseType));
}
packageJson.version = packageMap[packageName].nextVersion = newVersion;
await writeFile(packageFile, JSON.stringify(packageJson, null, 2) + '\n');
}
console.log(packageMap['n8n'].nextVersion);
async function bumpVersions() {
const rootDir = process.cwd();
const releaseType = /** @type { import('semver').ReleaseType | "experimental" } */ (
process.env.RELEASE_TYPE
);
assert.match(releaseType, /^(patch|minor|major|experimental|premajor)$/, 'Invalid RELEASE_TYPE');
// TODO: if releaseType is `auto` determine release type based on the changelog
const lastTag = (await exec('git describe --tags --match "n8n@*" --abbrev=0')).stdout.trim();
const packages = JSON.parse(
(
await exec(
`pnpm ls -r --only-projects --json | jq -r '[.[] | { name: .name, version: .version, path: .path, private: .private}]'`,
)
).stdout,
);
/** @type {Record<string, { path: string, isDirty: boolean, version: string, nextVersion?: string }>} */
const packageMap = {};
for (let { name, path, version, private: isPrivate } of packages) {
if (isPrivate && path !== rootDir) {
continue;
}
if (path === rootDir) {
name = 'monorepo-root';
}
const isDirty = await exec(`git diff --quiet HEAD ${lastTag} -- ${path}`)
.then(() => false)
.catch(() => true);
packageMap[name] = { path, isDirty, version };
}
assert.ok(
Object.values(packageMap).some(({ isDirty }) => isDirty),
'No changes found since the last release',
);
// Propagate isDirty transitively: if a package's dependency will be bumped,
// that package also needs a bump (e.g. design-system → editor-ui → cli).
// Detect root-level changes that affect resolved dep versions without touching individual
// package.json files: pnpm.overrides (applies to all specifiers)
// and pnpm-workspace.yaml catalog entries (applies only to deps using a "catalog:…" specifier).
const rootPkgJson = JSON.parse(await readFile(resolve(rootDir, 'package.json'), 'utf-8'));
const rootPkgJsonAtTag = await exec(`git show ${lastTag}:package.json`)
.then(({ stdout }) => JSON.parse(stdout))
.catch(() => ({}));
const changedOverrides = computeChangedOverrides(
getOverrides(rootPkgJson),
getOverrides(rootPkgJsonAtTag),
);
const workspaceYaml = parseWorkspaceYaml(
await readFile(resolve(rootDir, 'pnpm-workspace.yaml'), 'utf-8').catch(() => ''),
);
const workspaceYamlAtTag = parseWorkspaceYaml(
await exec(`git show ${lastTag}:pnpm-workspace.yaml`)
.then(({ stdout }) => stdout)
.catch(() => ''),
);
const changedCatalogEntries = computeChangedCatalogEntries(
getCatalogs(workspaceYaml),
getCatalogs(workspaceYamlAtTag),
);
// Store full dep objects (with specifiers) so we can inspect "catalog:…" values below.
/** @type {Record<string, Record<string, string>>} */
const depsByPackage = {};
for (const packageName in packageMap) {
const packageFile = resolve(packageMap[packageName].path, 'package.json');
const packageJson = JSON.parse(await readFile(packageFile, 'utf-8'));
depsByPackage[packageName] = /** @type {Record<string,string>} */ (
packageJson.dependencies ?? {}
);
}
// Mark packages dirty if any dep had a root-level override or catalog version change.
markDirtyByRootChanges(packageMap, depsByPackage, changedOverrides, changedCatalogEntries);
propagateDirtyTransitively(packageMap, depsByPackage);
// Keep the monorepo version up to date with the released version
packageMap['monorepo-root'].version = packageMap['n8n'].version;
for (const packageName in packageMap) {
const { path, version, isDirty } = packageMap[packageName];
const packageFile = resolve(path, 'package.json');
const packageJson = JSON.parse(await readFile(packageFile, 'utf-8'));
const dependencyIsDirty = Object.keys(packageJson.dependencies || {}).some(
(dependencyName) => packageMap[dependencyName]?.isDirty,
);
let newVersion = version;
if (isDirty || dependencyIsDirty) {
newVersion = computeNewVersion(version, releaseType);
}
packageJson.version = packageMap[packageName].nextVersion = newVersion;
await writeFile(packageFile, JSON.stringify(packageJson, null, 2) + '\n');
}
console.log(packageMap['n8n'].nextVersion);
}
// only run when executed directly, not when imported by tests
if (import.meta.url === `file://${process.argv[1]}`) {
bumpVersions();
}

380
.github/scripts/bump-versions.test.mjs vendored Normal file
View File

@ -0,0 +1,380 @@
/**
* Run these tests with:
*
* node --test ./.github/scripts/bump-versions.test.mjs
*/
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import {
generateExperimentalVersion,
getOverrides,
parseWorkspaceYaml,
getCatalogs,
computeChangedOverrides,
computeChangedCatalogEntries,
markDirtyByRootChanges,
propagateDirtyTransitively,
computeNewVersion,
} from './bump-versions.mjs';
describe('generateExperimentalVersion', () => {
it('creates -exp.0 from a stable version', () => {
assert.equal(generateExperimentalVersion('1.2.3'), '1.2.3-exp.0');
});
it('increments exp minor when already at exp.0', () => {
assert.equal(generateExperimentalVersion('1.2.3-exp.0'), '1.2.3-exp.1');
});
it('increments exp minor when already at exp.5', () => {
assert.equal(generateExperimentalVersion('1.2.3-exp.5'), '1.2.3-exp.6');
});
it('creates -exp.0 from a version with a different pre-release tag', () => {
assert.equal(generateExperimentalVersion('1.2.3-beta.1'), '1.2.3-exp.0');
});
it('handles multi-digit version numbers', () => {
assert.equal(generateExperimentalVersion('10.20.30'), '10.20.30-exp.0');
});
it('throws on an invalid version string', () => {
assert.throws(() => generateExperimentalVersion('not-a-version'), /Invalid version/);
});
});
describe('getOverrides', () => {
it('returns empty object when no overrides exist', () => {
assert.deepEqual(getOverrides({}), {});
});
it('returns pnpm.overrides when only pnpm.overrides is set', () => {
assert.deepEqual(getOverrides({ pnpm: { overrides: { lodash: '^4.0.0' } } }), {
lodash: '^4.0.0',
});
});
it('returns overrides when only top-level overrides is set', () => {
assert.deepEqual(getOverrides({ overrides: { lodash: '^4.0.0' } }), { lodash: '^4.0.0' });
});
it('merges both fields with top-level overrides taking precedence for the same key', () => {
assert.deepEqual(
getOverrides({
pnpm: { overrides: { lodash: '^3.0.0', underscore: '^1.0.0' } },
overrides: { lodash: '^4.0.0' },
}),
{ lodash: '^4.0.0', underscore: '^1.0.0' },
);
});
});
describe('parseWorkspaceYaml', () => {
it('parses valid YAML into an object', () => {
assert.deepEqual(parseWorkspaceYaml('catalog:\n lodash: "^4.0.0"'), {
catalog: { lodash: '^4.0.0' },
});
});
it('returns empty object for an empty string', () => {
assert.deepEqual(parseWorkspaceYaml(''), {});
});
it('returns empty object for invalid YAML', () => {
assert.deepEqual(parseWorkspaceYaml(': - invalid: [yaml}'), {});
});
});
describe('getCatalogs', () => {
it('returns empty map when no catalog or catalogs field exists', () => {
assert.equal(getCatalogs({}).size, 0);
});
it('returns a "default" entry for the top-level catalog field', () => {
const result = getCatalogs({ catalog: { lodash: '^4.0.0' } });
assert.equal(result.size, 1);
assert.deepEqual(result.get('default'), { lodash: '^4.0.0' });
});
it('returns named entries from the catalogs field', () => {
const result = getCatalogs({ catalogs: { react18: { react: '^18.0.0' } } });
assert.equal(result.size, 1);
assert.deepEqual(result.get('react18'), { react: '^18.0.0' });
});
it('returns both default and named catalog entries when both fields are present', () => {
const result = getCatalogs({
catalog: { lodash: '^4.0.0' },
catalogs: { react18: { react: '^18.0.0' } },
});
assert.equal(result.size, 2);
assert.deepEqual(result.get('default'), { lodash: '^4.0.0' });
assert.deepEqual(result.get('react18'), { react: '^18.0.0' });
});
});
describe('computeChangedOverrides', () => {
it('returns empty set when nothing changed', () => {
assert.equal(computeChangedOverrides({ lodash: '^4' }, { lodash: '^4' }).size, 0);
});
it('detects an added override', () => {
const result = computeChangedOverrides({ lodash: '^4' }, {});
assert.ok(result.has('lodash'));
});
it('detects a removed override', () => {
const result = computeChangedOverrides({}, { lodash: '^4' });
assert.ok(result.has('lodash'));
});
it('detects a changed override value', () => {
const result = computeChangedOverrides({ lodash: '^4' }, { lodash: '^3' });
assert.ok(result.has('lodash'));
});
it('does not include unchanged overrides', () => {
const result = computeChangedOverrides(
{ lodash: '^4', underscore: '^1' },
{ lodash: '^4', underscore: '^1' },
);
assert.equal(result.size, 0);
});
it('handles mixed changed and unchanged overrides', () => {
const result = computeChangedOverrides(
{ lodash: '^4', underscore: '^2' },
{ lodash: '^4', underscore: '^1' },
);
assert.equal(result.size, 1);
assert.ok(result.has('underscore'));
assert.ok(!result.has('lodash'));
});
});
describe('computeChangedCatalogEntries', () => {
it('returns empty map when nothing changed', () => {
const current = new Map([['default', { lodash: '^4' }]]);
const previous = new Map([['default', { lodash: '^4' }]]);
assert.equal(computeChangedCatalogEntries(current, previous).size, 0);
});
it('detects an added dep in a catalog', () => {
const current = new Map([['default', { lodash: '^4' }]]);
const previous = new Map([['default', {}]]);
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.get('default')?.has('lodash'));
});
it('detects a removed dep from a catalog', () => {
const current = new Map([['default', {}]]);
const previous = new Map([['default', { lodash: '^4' }]]);
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.get('default')?.has('lodash'));
});
it('detects a changed dep version in a catalog', () => {
const current = new Map([['default', { lodash: '^4' }]]);
const previous = new Map([['default', { lodash: '^3' }]]);
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.get('default')?.has('lodash'));
});
it('detects changes in a named catalog', () => {
const current = new Map([['react18', { react: '^18' }]]);
const previous = new Map([['react18', { react: '^17' }]]);
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.get('react18')?.has('react'));
});
it('detects a newly added catalog', () => {
const current = new Map([['newCatalog', { lodash: '^4' }]]);
const previous = new Map();
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.get('newCatalog')?.has('lodash'));
});
it('detects a removed catalog', () => {
const current = new Map();
const previous = new Map([['oldCatalog', { lodash: '^4' }]]);
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.get('oldCatalog')?.has('lodash'));
});
it('does not include a catalog that has no changed entries', () => {
const current = new Map([
['default', { lodash: '^4' }],
['react18', { react: '^18' }],
]);
const previous = new Map([
['default', { lodash: '^3' }],
['react18', { react: '^18' }],
]);
const result = computeChangedCatalogEntries(current, previous);
assert.ok(result.has('default'));
assert.ok(!result.has('react18'));
});
});
describe('markDirtyByRootChanges', () => {
it('marks a package dirty when its dep appears in changedOverrides', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { lodash: '^4' } };
markDirtyByRootChanges(packageMap, depsByPackage, new Set(['lodash']), new Map());
assert.ok(packageMap['pkg-a'].isDirty);
});
it('skips already-dirty packages', () => {
const packageMap = { 'pkg-a': { isDirty: true } };
// No deps, but package is already dirty — should not throw or change state
const depsByPackage = { 'pkg-a': {} };
markDirtyByRootChanges(packageMap, depsByPackage, new Set(['lodash']), new Map());
assert.ok(packageMap['pkg-a'].isDirty);
});
it('marks a package dirty when its dep uses "catalog:" (default catalog) and that entry changed', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { lodash: 'catalog:' } };
const changedCatalogEntries = new Map([['default', new Set(['lodash'])]]);
markDirtyByRootChanges(packageMap, depsByPackage, new Set(), changedCatalogEntries);
assert.ok(packageMap['pkg-a'].isDirty);
});
it('marks a package dirty when its dep uses "catalog:<name>" and that named catalog entry changed', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { react: 'catalog:react18' } };
const changedCatalogEntries = new Map([['react18', new Set(['react'])]]);
markDirtyByRootChanges(packageMap, depsByPackage, new Set(), changedCatalogEntries);
assert.ok(packageMap['pkg-a'].isDirty);
});
it('does not mark a package dirty when none of its deps changed', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { lodash: '^4' } };
markDirtyByRootChanges(packageMap, depsByPackage, new Set(['underscore']), new Map());
assert.ok(!packageMap['pkg-a'].isDirty);
});
it('does not mark a package dirty when a catalog: dep is in a catalog with no changes', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { lodash: 'catalog:' } };
const changedCatalogEntries = new Map([['default', new Set(['underscore'])]]);
markDirtyByRootChanges(packageMap, depsByPackage, new Set(), changedCatalogEntries);
assert.ok(!packageMap['pkg-a'].isDirty);
});
it('does not mark a package dirty when a catalog: dep is in a different catalog than the one that changed', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { react: 'catalog:react18' } };
const changedCatalogEntries = new Map([['default', new Set(['react'])]]);
markDirtyByRootChanges(packageMap, depsByPackage, new Set(), changedCatalogEntries);
assert.ok(!packageMap['pkg-a'].isDirty);
});
});
describe('propagateDirtyTransitively', () => {
it('does nothing when no packages are dirty', () => {
const packageMap = {
'pkg-a': { isDirty: false },
'pkg-b': { isDirty: false },
};
const depsByPackage = {
'pkg-a': { 'pkg-b': 'workspace:*' },
'pkg-b': {},
};
propagateDirtyTransitively(packageMap, depsByPackage);
assert.ok(!packageMap['pkg-a'].isDirty);
assert.ok(!packageMap['pkg-b'].isDirty);
});
it('propagates dirty state one level up the dependency chain', () => {
const packageMap = {
'pkg-a': { isDirty: false },
'pkg-b': { isDirty: true },
};
const depsByPackage = {
'pkg-a': { 'pkg-b': 'workspace:*' },
'pkg-b': {},
};
propagateDirtyTransitively(packageMap, depsByPackage);
assert.ok(packageMap['pkg-a'].isDirty);
});
it('propagates dirty state through multiple levels', () => {
const packageMap = {
'pkg-a': { isDirty: false },
'pkg-b': { isDirty: false },
'pkg-c': { isDirty: true },
};
const depsByPackage = {
'pkg-a': { 'pkg-b': 'workspace:*' },
'pkg-b': { 'pkg-c': 'workspace:*' },
'pkg-c': {},
};
propagateDirtyTransitively(packageMap, depsByPackage);
assert.ok(packageMap['pkg-b'].isDirty, 'pkg-b should be dirty (depends on dirty pkg-c)');
assert.ok(packageMap['pkg-a'].isDirty, 'pkg-a should be dirty (depends on dirty pkg-b)');
});
it('does not mark packages dirty when their deps are external (not in packageMap)', () => {
const packageMap = { 'pkg-a': { isDirty: false } };
const depsByPackage = { 'pkg-a': { lodash: '^4' } };
propagateDirtyTransitively(packageMap, depsByPackage);
assert.ok(!packageMap['pkg-a'].isDirty);
});
it('handles diamond dependency graphs without infinite loops', () => {
// pkg-a depends on pkg-b and pkg-c; both depend on pkg-d (dirty)
const packageMap = {
'pkg-a': { isDirty: false },
'pkg-b': { isDirty: false },
'pkg-c': { isDirty: false },
'pkg-d': { isDirty: true },
};
const depsByPackage = {
'pkg-a': { 'pkg-b': 'workspace:*', 'pkg-c': 'workspace:*' },
'pkg-b': { 'pkg-d': 'workspace:*' },
'pkg-c': { 'pkg-d': 'workspace:*' },
'pkg-d': {},
};
propagateDirtyTransitively(packageMap, depsByPackage);
assert.ok(packageMap['pkg-b'].isDirty);
assert.ok(packageMap['pkg-c'].isDirty);
assert.ok(packageMap['pkg-a'].isDirty);
});
});
describe('computeNewVersion', () => {
it('increments patch version', () => {
assert.equal(computeNewVersion('1.2.3', 'patch'), '1.2.4');
});
it('increments minor version (resets patch)', () => {
assert.equal(computeNewVersion('1.2.3', 'minor'), '1.3.0');
});
it('increments major version (resets minor and patch)', () => {
assert.equal(computeNewVersion('1.2.3', 'major'), '2.0.0');
});
it('creates -exp.0 from a stable version for experimental', () => {
assert.equal(computeNewVersion('1.2.3', 'experimental'), '1.2.3-exp.0');
});
it('increments exp minor for experimental when already an exp version', () => {
assert.equal(computeNewVersion('1.2.3-exp.0', 'experimental'), '1.2.3-exp.1');
});
it('creates a premajor rc version from a stable version', () => {
assert.equal(computeNewVersion('1.2.3', 'premajor'), '2.0.0-rc.0');
});
it('increments the rc prerelease number for premajor when already an rc version', () => {
assert.equal(computeNewVersion('2.0.0-rc.0', 'premajor'), '2.0.0-rc.1');
});
it('increments rc correctly across multiple premajor calls', () => {
assert.equal(computeNewVersion('2.0.0-rc.4', 'premajor'), '2.0.0-rc.5');
});
});

View File

@ -40,6 +40,8 @@ export const EXCLUDE_PATTERNS = [
'packages/testing/**',
// Lock file (can produce massive diffs on dependency changes)
'pnpm-lock.yaml',
'**/*.md',
'**/*.mdx'
];
const BOT_MARKER = '<!-- pr-size-check -->';

View File

@ -203,4 +203,13 @@ describe('countFilteredAdditions', () => {
];
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
});
it('applies EXCLUDE_PATTERNS to markdown files', () => {
const files = [
{ filename: 'packages/cli/src/service.ts', additions: 50 },
{ filename: 'packages/cli/AGENTS.md', additions: 100 },
{ filename: 'packages/frontend/STORIES.mdx', additions: 100 },
];
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
});
});

View File

@ -1,6 +1,7 @@
name: 'CI: PR Quality Checks'
on:
merge_group:
pull_request:
types:
- opened
@ -99,3 +100,21 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: node .github/scripts/quality/check-pr-size.mjs
required-pr-quality-checks:
name: Required PR Quality Checks
needs: [check-ownership-checkbox, check-pr-size]
if: always()
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
sparse-checkout: .github/actions/ci-filter
sparse-checkout-cone-mode: false
- name: Validate required checks
uses: ./.github/actions/ci-filter
with:
mode: validate
job-results: ${{ toJSON(needs) }}

View File

@ -22,6 +22,7 @@ jobs:
ci: ${{ fromJSON(steps.ci-filter.outputs.results).ci == true }}
unit: ${{ fromJSON(steps.ci-filter.outputs.results).unit == true }}
e2e: ${{ fromJSON(steps.ci-filter.outputs.results).e2e == true }}
dev_server_smoke: ${{ fromJSON(steps.ci-filter.outputs.results)['dev-server-smoke'] == true }}
workflows: ${{ fromJSON(steps.ci-filter.outputs.results).workflows == true }}
workflow_scripts: ${{ fromJSON(steps.ci-filter.outputs.results)['workflow-scripts'] == true }}
db: ${{ fromJSON(steps.ci-filter.outputs.results).db == true }}
@ -63,6 +64,16 @@ jobs:
.github/actions/load-n8n-docker/**
packages/testing/playwright/**
packages/testing/containers/**
dev-server-smoke:
packages/frontend/editor-ui/vite.config.mts
pnpm-workspace.yaml
packages/@n8n/*/package.json
packages/testing/playwright/tests/dev-server-smoke/**
packages/testing/playwright/playwright.config.ts
packages/testing/playwright/playwright-projects.ts
packages/testing/playwright/package.json
.github/workflows/test-dev-server-smoke-reusable.yml
.github/workflows/ci-pull-requests.yml
workflows: .github/**
workflow-scripts: .github/scripts/**
performance:
@ -221,6 +232,19 @@ jobs:
upload-failure-artifacts: ${{ github.event.pull_request.head.repo.fork == true }}
secrets: inherit
# Boots the editor-ui against the Vite dev server and fails on any console
# or page error during load. Catches regressions in dev-mode module
# resolution (missing Vite alias, broken workspace package interop) that
# the production-bundle e2e job bundles around.
dev-server-smoke:
name: Dev-server boot smoke
needs: install-and-build
if: needs.install-and-build.outputs.dev_server_smoke == 'true' && github.event_name != 'merge_group'
uses: ./.github/workflows/test-dev-server-smoke-reusable.yml
with:
ref: ${{ needs.install-and-build.outputs.commit_sha }}
secrets: inherit
db-tests:
name: DB Tests
needs: install-and-build
@ -296,6 +320,7 @@ jobs:
check-packaging,
sqlite-sanity,
e2e,
dev-server-smoke,
db-tests,
performance,
security-checks,

View File

@ -76,11 +76,9 @@ jobs:
cp README.md packages/cli/README.md
sed -i "s/default: 'dev'/default: '${{ needs.determine-version-info.outputs.release_type }}'/g" packages/cli/dist/config/schema.js
- name: Publish n8n to NPM with rc tag
env:
PUBLISH_BRANCH: ${{ github.event.pull_request.base.ref }}
run: pnpm --filter n8n publish --publish-branch "$PUBLISH_BRANCH" --access public --tag rc --no-git-checks
# Publishing via `pnpm publish -r` is idempotent, as it checks if the package exists
# and only publishes if it doesn't. This is why we do the sub-packages before the main n8n package.
# So if anything goes wrong, we can easily re-try the run instead of abandoning the release.
- name: Publish other packages to NPM
env:
PUBLISH_BRANCH: ${{ github.event.pull_request.base.ref }}
@ -92,6 +90,12 @@ jobs:
fi
pnpm publish -r --filter '!n8n' --publish-branch "$PUBLISH_BRANCH" --access public --tag "$PUBLISH_TAG" --no-git-checks
# If we don't use the --tag rc, all releases will default to "latest".
- name: Publish n8n to NPM with rc tag
env:
PUBLISH_BRANCH: ${{ github.event.pull_request.base.ref }}
run: pnpm --filter n8n publish --publish-branch "$PUBLISH_BRANCH" --access public --tag rc --no-git-checks
- name: Cleanup rc tag
run: npm dist-tag rm n8n rc
continue-on-error: true
@ -107,7 +111,7 @@ jobs:
build-daytona-snapshot:
name: Build Daytona snapshot
needs: [determine-version-info]
needs: [determine-version-info, publish-to-npm]
if: github.event.pull_request.merged == true
uses: ./.github/workflows/release-build-daytona-snapshot.yml
with:

View File

@ -0,0 +1,49 @@
name: 'Test: Dev-server boot smoke'
on:
workflow_call:
inputs:
ref:
description: 'Git ref to test'
required: true
type: string
env:
NODE_OPTIONS: '--max-old-space-size=6144'
PLAYWRIGHT_BROWSERS_PATH: packages/testing/playwright/.playwright-browsers
jobs:
smoke:
name: Dev-server smoke
runs-on: ${{ vars.RUNNER_PROVIDER == 'github' && 'ubuntu-latest' || 'blacksmith-4vcpu-ubuntu-2204' }}
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 1
ref: ${{ inputs.ref }}
- name: Setup and Build
uses: ./.github/actions/setup-nodejs
- name: Install Browsers
run: pnpm turbo run install-browsers --filter=n8n-playwright
- name: Run dev-server smoke spec
# Run from repo root so PLAYWRIGHT_BROWSERS_PATH (relative) resolves
# correctly. cd-ing into the playwright package double-nests it.
run: pnpm --filter=n8n-playwright test:dev-server-smoke --reporter=list
- name: Upload Failure Artifacts
if: ${{ failure() }}
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: dev-server-smoke-report
path: |
packages/testing/playwright/test-results/
packages/testing/playwright/playwright-report/
retention-days: 7

View File

@ -37,7 +37,7 @@ jobs:
uses: ./.github/workflows/test-e2e-reusable.yml
with:
test-mode: docker-artifact
test-command: pnpm --filter=n8n-playwright test:all --project='${{ matrix.profile }}:infrastructure' --workers=1
test-command: pnpm --filter=n8n-playwright test:all --project=${{ matrix.profile }}:infrastructure --workers=1
runner: ${{ matrix.runner }}
timeout-minutes: 60
secrets: inherit

View File

@ -143,7 +143,7 @@ jobs:
--base-url "$BASE_URLS" \
--concurrency 32 \
--verbose \
--iterations 3 \
--iterations 5 \
${{ inputs.filter && format('--filter "{0}"', inputs.filter) || '' }}
- name: Stop n8n containers
@ -160,22 +160,16 @@ jobs:
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
RESULTS_FILE="packages/@n8n/instance-ai/eval-results.json"
if [ ! -f "$RESULTS_FILE" ]; then
echo "No eval results file found"
# The eval CLI writes the full PR comment as eval-pr-comment.md
# (see comparison/format.ts:formatComparisonMarkdown). It includes
# the alert, aggregate, comparison sections, per-test-case results
# collapsed, and failure details collapsed. CI just relays it.
COMMENT_FILE="packages/@n8n/instance-ai/eval-pr-comment.md"
if [ ! -f "$COMMENT_FILE" ]; then
echo "No PR comment file found (eval likely cancelled before writing results)"
exit 0
fi
# Build the full comment body with jq
jq -r '
"### Instance AI Workflow Eval Results\n\n" +
"**\(.summary.built)/\(.summary.testCases) built | \(.totalRuns) run(s) | pass@\(.totalRuns): \(.summary.passAtK * 100 | floor)% | pass^\(.totalRuns): \(.summary.passHatK * 100 | floor)% | iterations: \(.summary.passRatePerIter)**\n\n" +
"| Workflow | Build | pass@\(.totalRuns) | pass^\(.totalRuns) |\n|---|---|---|---|\n" +
([.testCases[] as $tc | "| \($tc.name) | \($tc.buildSuccessCount)/\($tc.totalRuns) | \(([$tc.scenarios[] | .passAtK] | add) / ($tc.scenarios | length) * 100 | floor)% | \(([$tc.scenarios[] | .passHatK] | add) / ($tc.scenarios | length) * 100 | floor)% |"] | join("\n")) +
"\n\n<details><summary>Failure details</summary>\n\n" +
([.testCases[] as $tc | $tc.scenarios[] | select(.passHatK < 1) | "**\($tc.name) / \(.name)** — \(.passCount)/\(.totalRuns) passed" + "\n" + ([.runs[] | select(.passed == false) | "> Run\(if .failureCategory then " [\(.failureCategory)]" else "" end): \(.reasoning | .[0:200])"] | join("\n"))] | join("\n\n")) +
"\n</details>"
' "$RESULTS_FILE" > /tmp/eval-comment.md
cp "$COMMENT_FILE" /tmp/eval-comment.md
# Find and update existing eval comment, or create new one
COMMENT_ID=$(gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments" \

2
.gitignore vendored
View File

@ -36,6 +36,7 @@ packages/testing/playwright/playwright-report
packages/testing/playwright/test-results
packages/testing/playwright/eval-results.json
packages/@n8n/instance-ai/eval-results.json
packages/@n8n/instance-ai/eval-pr-comment.md
packages/testing/playwright/.playwright-browsers
packages/testing/playwright/.playwright-cli
test-results/
@ -61,6 +62,7 @@ packages/cli/src/commands/export/outputs
.claude/settings.local.json
.claude/plans/
.claude/worktrees/
.claude/specs/
.cursor/plans/
.superset
.conductor

View File

@ -1,3 +1,119 @@
# [2.20.0](https://github.com/n8n-io/n8n/compare/n8n@2.19.0...n8n@2.20.0) (2026-05-05)
### Bug Fixes
* **ai-builder:** Add boundaries on the workflow builder remediation loops ([#29430](https://github.com/n8n-io/n8n/issues/29430)) ([2259f32](https://github.com/n8n-io/n8n/commit/2259f32de88c103b088b450bf46990ad2e939942))
* **ai-builder:** Allow skipping final ask-user question ([#29563](https://github.com/n8n-io/n8n/issues/29563)) ([661f990](https://github.com/n8n-io/n8n/commit/661f9908bce51076811c76c854f165f4c5acaccf))
* **ai-builder:** Filter LangSmith eval dataset by local file slugs ([#29507](https://github.com/n8n-io/n8n/issues/29507)) ([54d9286](https://github.com/n8n-io/n8n/commit/54d9286d922e0cad17d5c5de10a052d653c1591b))
* **ai-builder:** Handle properties with contradicting displayOptions as OR alternatives instead of AND ([#29500](https://github.com/n8n-io/n8n/issues/29500)) ([84ac811](https://github.com/n8n-io/n8n/commit/84ac8110f8d70dd653b4d40cb63259522731b0d0))
* **ai-builder:** Stop builder from adding auth to inbound trigger nodes by default ([#29648](https://github.com/n8n-io/n8n/issues/29648)) ([c28d501](https://github.com/n8n-io/n8n/commit/c28d501ba1630861fa0993d0d85f08efb635a5a4))
* Allow 5-field cron expressions with step values in polling nodes ([#29447](https://github.com/n8n-io/n8n/issues/29447)) ([d18f183](https://github.com/n8n-io/n8n/commit/d18f183b211416d5b74cfdc2e740b9c663ede134))
* **Anthropic Chat Model Node:** Add adaptive thinking mode for Claude Opus 4.7+ ([#29467](https://github.com/n8n-io/n8n/issues/29467)) ([90d875c](https://github.com/n8n-io/n8n/commit/90d875ce3e5a2a004a5a3d8f28ac4e9820b109f4))
* **Compare Datasets Node:** Preserve falsy values in mix mode except fields ([#29666](https://github.com/n8n-io/n8n/issues/29666)) ([62ddc5c](https://github.com/n8n-io/n8n/commit/62ddc5c443273559c286a1d2eb19efdca345ac9a))
* **core:** Accept placeholder() inside node credentials slot ([#29691](https://github.com/n8n-io/n8n/issues/29691)) ([dc6bd68](https://github.com/n8n-io/n8n/commit/dc6bd68de3b419fb1e23806781bbc125b621ed8a))
* **core:** Acquire expression isolate for dynamic node parameter requests ([#29671](https://github.com/n8n-io/n8n/issues/29671)) ([418f1f2](https://github.com/n8n-io/n8n/commit/418f1f2edb6abfebe1085b8c3b5c1b22530f1a5c))
* **core:** Add file path validation to localFile source ([#29464](https://github.com/n8n-io/n8n/issues/29464)) ([7277566](https://github.com/n8n-io/n8n/commit/7277566c64c36f5e43c17a2e620da2408ab1dcb7))
* **core:** Add GET handler to MCP endpoint for Streamable HTTP spec compliance ([#28787](https://github.com/n8n-io/n8n/issues/28787)) ([4ae0322](https://github.com/n8n-io/n8n/commit/4ae0322ef246348892000d0539904e56c122d204))
* **core:** Add timeout to external secrets provider refresh ([#29679](https://github.com/n8n-io/n8n/issues/29679)) ([e350429](https://github.com/n8n-io/n8n/commit/e35042999f7d477ed1da59f43ef03605763ac2bf))
* **core:** Apply credential allowed domains in declarative node requests ([#29082](https://github.com/n8n-io/n8n/issues/29082)) ([8551b1b](https://github.com/n8n-io/n8n/commit/8551b1b90ce16b31a017bd07177694ef39ad226d))
* **core:** Correct LDAP search filter construction ([#29388](https://github.com/n8n-io/n8n/issues/29388)) ([32dd743](https://github.com/n8n-io/n8n/commit/32dd7433b7ef168161e32c20939859060da9827c))
* **core:** Fix code node executions hanging when idle timer overlaps with task acceptance ([#29239](https://github.com/n8n-io/n8n/issues/29239)) ([7bd3532](https://github.com/n8n-io/n8n/commit/7bd3532f07c151568634e84f3ae24f38ab8e60e4))
* **core:** Fix MCP OAuth discovery URL construction and grant type selection ([#27283](https://github.com/n8n-io/n8n/issues/27283)) ([d92ec16](https://github.com/n8n-io/n8n/commit/d92ec168aa5f984513874e2978f73d8f2cbdc80e))
* **core:** Force saving executions when instance AI executes WFs ([#29515](https://github.com/n8n-io/n8n/issues/29515)) ([ef56501](https://github.com/n8n-io/n8n/commit/ef56501d4729b5b508a4c5e60263d10a8fc9db76))
* **core:** Gate Instance AI edits to pre-existing workflows ([#29501](https://github.com/n8n-io/n8n/issues/29501)) ([6175fd6](https://github.com/n8n-io/n8n/commit/6175fd6f7b56ead0176938657085b763c1204681))
* **core:** Generate array types for properties with multipleValues ([#29410](https://github.com/n8n-io/n8n/issues/29410)) ([fb65c61](https://github.com/n8n-io/n8n/commit/fb65c6155ee9ae5b11a2c409f35e98c206aaf164))
* **core:** Handle missing runData during execution recovery ([#29513](https://github.com/n8n-io/n8n/issues/29513)) ([8b7b4f5](https://github.com/n8n-io/n8n/commit/8b7b4f575d9d9b5b02a8ddf67aaff6b3d5279d78))
* **core:** Harden Set node workflow SDK contract ([#29568](https://github.com/n8n-io/n8n/issues/29568)) ([625ed5e](https://github.com/n8n-io/n8n/commit/625ed5e95a90f30e07e88253515713056e406f5b))
* **core:** Include stack trace in error logs for non-ApplicationError errors ([#29496](https://github.com/n8n-io/n8n/issues/29496)) ([16d1461](https://github.com/n8n-io/n8n/commit/16d1461858107697eac399039c834c7296fe8868))
* **core:** Increase default task runner grant token TTL to 30s ([#29443](https://github.com/n8n-io/n8n/issues/29443)) ([328f4b8](https://github.com/n8n-io/n8n/commit/328f4b8b964d587763bf14b1980916046878f0f0))
* **core:** Isolate expressions on chat resumption and test webhook deactivation ([#29703](https://github.com/n8n-io/n8n/issues/29703)) ([568e5a2](https://github.com/n8n-io/n8n/commit/568e5a24bf8f4e73d0b134dbac1631535bba10a7))
* **core:** Make MCP client registration cap tunable and surface a proper limit error ([#29429](https://github.com/n8n-io/n8n/issues/29429)) ([dad4231](https://github.com/n8n-io/n8n/commit/dad423155f1ee105e3ed1eab0b65a8d8bc2ee3a3))
* **core:** Make task runner grant token TTL configurable ([#29357](https://github.com/n8n-io/n8n/issues/29357)) ([3f350a8](https://github.com/n8n-io/n8n/commit/3f350a85770680895be5723803ef51453476fed2))
* **core:** Pass nodeTypesProvider to validate workflows fully at instance AI ([#29333](https://github.com/n8n-io/n8n/issues/29333)) ([388cd79](https://github.com/n8n-io/n8n/commit/388cd79908418d558fff36f938969cdc79fc60c2))
* **core:** Persist execution context before writing to db ([#28973](https://github.com/n8n-io/n8n/issues/28973)) ([c4bb5ae](https://github.com/n8n-io/n8n/commit/c4bb5ae8df8e7de4c7b919a82d3cf2f492edcc5b))
* **core:** Recreate data table backing tables on entity import ([#29454](https://github.com/n8n-io/n8n/issues/29454)) ([6bca1fa](https://github.com/n8n-io/n8n/commit/6bca1fa26f0d1a23c8c7e175dc6ae590eeb2036e))
* **core:** Reject empty webhookMethods in community lint rule ([#29474](https://github.com/n8n-io/n8n/issues/29474)) ([34d7a02](https://github.com/n8n-io/n8n/commit/34d7a02df73f233ef55fc78e3ea8167bc2b32a1f))
* **core:** Reset Redis retry counter on successful reconnect ([#29377](https://github.com/n8n-io/n8n/issues/29377)) ([7722023](https://github.com/n8n-io/n8n/commit/7722023abd8ffb2f96a7dbec0ba51e4d7454ea05))
* **core:** Respect global admin scope when listing favorites ([#29472](https://github.com/n8n-io/n8n/issues/29472)) ([d9d1e7c](https://github.com/n8n-io/n8n/commit/d9d1e7c44a1bcf074cdbec234b0d8d4ddb8d7d5e))
* **core:** Restore peer project discovery in share dropdowns ([#29537](https://github.com/n8n-io/n8n/issues/29537)) ([2a0e2fb](https://github.com/n8n-io/n8n/commit/2a0e2fb47ae1d82cd2354db8c2013ea46f24f21e))
* **core:** Round fractional time saved values before inserting into insights BIGINT column ([#29553](https://github.com/n8n-io/n8n/issues/29553)) ([74d55b9](https://github.com/n8n-io/n8n/commit/74d55b9c681273ae79fbaf39693bd3b37d83b66a))
* **core:** Show AI Builder draft workflows in workflow list ([#29670](https://github.com/n8n-io/n8n/issues/29670)) ([dc52bbd](https://github.com/n8n-io/n8n/commit/dc52bbd5329a27245a5fe2a1da45d9e8efe6a549))
* **core:** Use editor base URL for workflow and execution links ([#23630](https://github.com/n8n-io/n8n/issues/23630)) ([896461b](https://github.com/n8n-io/n8n/commit/896461bee3c356e66b282763cd31427a137ebd62))
* **core:** Validate workflow import URL requests ([#29178](https://github.com/n8n-io/n8n/issues/29178)) ([ecd0ba8](https://github.com/n8n-io/n8n/commit/ecd0ba8ebabc99055441290d543f0bd87a33df31))
* **core:** Wire EncryptionKeyProxy provider on bootstrap ([#29581](https://github.com/n8n-io/n8n/issues/29581)) ([ee7260c](https://github.com/n8n-io/n8n/commit/ee7260c4959b0dff8636606aebdac10eddd76e36))
* **DeepL Node:** Update credentials to use header-based authentication ([#24614](https://github.com/n8n-io/n8n/issues/24614)) ([b72bd19](https://github.com/n8n-io/n8n/commit/b72bd1987c33b15cd658d2a038b9763c6fb83b55))
* Drop template search tools from builder ([#29573](https://github.com/n8n-io/n8n/issues/29573)) ([9b00ccb](https://github.com/n8n-io/n8n/commit/9b00ccbfd1cfb123533397126123f5d2ad34071f))
* **editor:** Add proper bg color for hover state with color-mix() ([#29590](https://github.com/n8n-io/n8n/issues/29590)) ([6698c42](https://github.com/n8n-io/n8n/commit/6698c42e4ed4706825f5d2e3bac39641e261f153))
* **editor:** Align message box button radius with N8nButton ([#29397](https://github.com/n8n-io/n8n/issues/29397)) ([bc315d0](https://github.com/n8n-io/n8n/commit/bc315d087fd772218b2f3caa047c86493c048f27))
* **editor:** Fix OAuth2 credential showing "Needs first setup" after connecting ([#29617](https://github.com/n8n-io/n8n/issues/29617)) ([243f665](https://github.com/n8n-io/n8n/commit/243f665e60bff1c2531977c3f860aa7589a321e9))
* **editor:** Fix sub-workflow folder placement and connection loss ([#28770](https://github.com/n8n-io/n8n/issues/28770)) ([44579d6](https://github.com/n8n-io/n8n/commit/44579d6d3ae59a1f4eedf9a0b49cecb006053072))
* **editor:** Ignore paste events on read-only canvas ([#29673](https://github.com/n8n-io/n8n/issues/29673)) ([34c49b9](https://github.com/n8n-io/n8n/commit/34c49b9c238de5d5ee0b9421918435c4582eb13a))
* **editor:** Keep publish actions menu enabled for published workflows ([#29396](https://github.com/n8n-io/n8n/issues/29396)) ([c65fa28](https://github.com/n8n-io/n8n/commit/c65fa28e1caac5a49e6a5e82d3354ed631be0df4))
* **editor:** Load more executions on tall screens ([#29407](https://github.com/n8n-io/n8n/issues/29407)) ([a273a9d](https://github.com/n8n-io/n8n/commit/a273a9d3f498d8112605f1277ce7848d8bd357c3))
* **editor:** Make instance ai resource link chips open resources ([#29577](https://github.com/n8n-io/n8n/issues/29577)) ([b97ca36](https://github.com/n8n-io/n8n/commit/b97ca36a99d099288cfc127df98038b2b64c03d5))
* **editor:** Make textarea resize handle accessible in NDV ([#29676](https://github.com/n8n-io/n8n/issues/29676)) ([9fda733](https://github.com/n8n-io/n8n/commit/9fda7332c4c0a8851a7482365a967ea18db2a816))
* **editor:** Mark workflow dirty after debug pinData changes ([#28886](https://github.com/n8n-io/n8n/issues/28886)) ([2beb006](https://github.com/n8n-io/n8n/commit/2beb0062a5f92c883f18abaf9ea33590a41aca49))
* **editor:** Never block publishing on node execution issues ([#29479](https://github.com/n8n-io/n8n/issues/29479)) ([5a56459](https://github.com/n8n-io/n8n/commit/5a564591291989f13ac667eed575332f7f4d2a6a))
* **editor:** Polish encryption keys date range filter ([#29569](https://github.com/n8n-io/n8n/issues/29569)) ([56412bc](https://github.com/n8n-io/n8n/commit/56412bcce2ef1d364acdbe422f5c88762319bb22))
* **editor:** Remove clipping for focus panel textarea ([#28677](https://github.com/n8n-io/n8n/issues/28677)) ([5361257](https://github.com/n8n-io/n8n/commit/5361257a80e515e1cc26cdf10e8ceb78c9ec70be))
* **editor:** Restore read-only mode for archived workflows on canvas ([#29559](https://github.com/n8n-io/n8n/issues/29559)) ([a7ef741](https://github.com/n8n-io/n8n/commit/a7ef7416b111384d250f975e718c691b2674fef6))
* **editor:** Show permission-aware message on redacted input/output panels ([#29521](https://github.com/n8n-io/n8n/issues/29521)) ([83c400e](https://github.com/n8n-io/n8n/commit/83c400e8d47c875f57dce26680358595822ce012))
* **editor:** Surface unofficial verified community node tools in AI Tools picker ([#28985](https://github.com/n8n-io/n8n/issues/28985)) ([f77dfd1](https://github.com/n8n-io/n8n/commit/f77dfd1a11591124e6db61c72ed207067bae6214))
* Fix ollama node url path and thinking tokens ([#23963](https://github.com/n8n-io/n8n/issues/23963)) ([4ea1153](https://github.com/n8n-io/n8n/commit/4ea1153dfb903346bead9e6d328ec8f543c80559))
* **Google Drive Node:** Resolve original file name when copying with empty name ([#28896](https://github.com/n8n-io/n8n/issues/28896)) ([c274976](https://github.com/n8n-io/n8n/commit/c2749768aa5d173c3354e8d31a18c438ebd5fdfb))
* **Merge Node:** Improve SQL Query mode memory efficiency and error reporting ([#28993](https://github.com/n8n-io/n8n/issues/28993)) ([12275c8](https://github.com/n8n-io/n8n/commit/12275c86d992115fef2ded4e5f172730222c5669))
* **Microsoft Outlook Trigger Node:** Use per-folder endpoints for folder-scoped message polling ([#29663](https://github.com/n8n-io/n8n/issues/29663)) ([f401f91](https://github.com/n8n-io/n8n/commit/f401f9101d08fc62eef7e051f3baa23638c80c1b))
* No Credits state for n8n Connect badge ([#29375](https://github.com/n8n-io/n8n/issues/29375)) ([47ad397](https://github.com/n8n-io/n8n/commit/47ad39777f9525324524f2595fc4506065f33a9c))
* **Notion Node:** Support app.notion.com URL format for page and block ID extraction ([#29554](https://github.com/n8n-io/n8n/issues/29554)) ([221c7f7](https://github.com/n8n-io/n8n/commit/221c7f7410d25b89b052e89d745184675b69dc53))
* **Postgres Node:** Output Large-Format Numbers As option ignored after pool is cached ([#29477](https://github.com/n8n-io/n8n/issues/29477)) ([a65e181](https://github.com/n8n-io/n8n/commit/a65e181a2213f1b984c225539302a1a12a30cc9b))
* **Salesforce Node:** Allow overriding JWT audience with My Domain URL ([#29016](https://github.com/n8n-io/n8n/issues/29016)) ([9decb1e](https://github.com/n8n-io/n8n/commit/9decb1e2a9f6d6612014354d7ca6f8b62600ce9d))
* **Schedule Node:** Cap day-of-month jitter at 28 ([#29614](https://github.com/n8n-io/n8n/issues/29614)) ([86f47ee](https://github.com/n8n-io/n8n/commit/86f47ee6dc88397b05bfb784b0092674ba3b4289))
* Skip AI tool generation for community trigger nodes ([#29453](https://github.com/n8n-io/n8n/issues/29453)) ([c724dac](https://github.com/n8n-io/n8n/commit/c724dace38ec1e3aa69de40d48e068cf36c962b0))
* **Snowflake Node:** Avoid call stack overflow on large result sets ([#29200](https://github.com/n8n-io/n8n/issues/29200)) ([b2ac67f](https://github.com/n8n-io/n8n/commit/b2ac67f15452c625d4dee146a040b6324cdfefbb))
* **Telegram Trigger Node:** Drop pending updates when creating a new webhook ([#29103](https://github.com/n8n-io/n8n/issues/29103)) ([4358f1d](https://github.com/n8n-io/n8n/commit/4358f1d51c588e76d03aa677f9b7deabbbc1af9d))
* **Todoist Node:** Migrate to Todoist unified API v1 endpoints ([#29532](https://github.com/n8n-io/n8n/issues/29532)) ([5799481](https://github.com/n8n-io/n8n/commit/5799481d1c3bf14806d11ba2928af4f7f88db29f))
* Use explicit node references for AI memory session keys ([#29473](https://github.com/n8n-io/n8n/issues/29473)) ([139b803](https://github.com/n8n-io/n8n/commit/139b803daefca44fd66a92156867d77ccdffcc66))
* Validate sql ([#24706](https://github.com/n8n-io/n8n/issues/24706)) ([47a6658](https://github.com/n8n-io/n8n/commit/47a6658b2d4cd2d4be5e59b0d61f9bd25b553007))
* **Zammad Node:** Add To and CC fields for email articles ([#28860](https://github.com/n8n-io/n8n/issues/28860)) ([e04f027](https://github.com/n8n-io/n8n/commit/e04f027b5dd008eb0c9354d166c716a93cdc48b7))
### Features
* Add instance-level JWKS URI endpoint for JWE public key distribution ([#29498](https://github.com/n8n-io/n8n/issues/29498)) ([794334c](https://github.com/n8n-io/n8n/commit/794334cd79f1ee5a05cd0d818fc801920e0fe6d9))
* Add no-runtime-dependencies ESLint rule ([#29366](https://github.com/n8n-io/n8n/issues/29366)) ([8aace75](https://github.com/n8n-io/n8n/commit/8aace75535f53ebf37c2a547849e044948c99cb8))
* Add pairwise workflow eval pipeline ([#29123](https://github.com/n8n-io/n8n/issues/29123)) ([fdceec2](https://github.com/n8n-io/n8n/commit/fdceec21b996a1456ceb44389e760a80d75d49a1))
* Add valid-credential-references ESLint rule ([#29452](https://github.com/n8n-io/n8n/issues/29452)) ([c6c6f8f](https://github.com/n8n-io/n8n/commit/c6c6f8ff3889a48ac73d5e5bb242e88818707fc0))
* **core:** Add --include and --exclude flags to import:credentials command ([#29364](https://github.com/n8n-io/n8n/issues/29364)) ([f5132b9](https://github.com/n8n-io/n8n/commit/f5132b9e9abe23eb1a2b1225d889f1dd83d83f94))
* **core:** Add configurable event log path per process ([#29403](https://github.com/n8n-io/n8n/issues/29403)) ([45effb8](https://github.com/n8n-io/n8n/commit/45effb8959e4013d46a022a5a3f901e9d0284d35))
* **core:** Add endpoint to toggle mcp access for multiple workflows ([#29007](https://github.com/n8n-io/n8n/issues/29007)) ([0d907d6](https://github.com/n8n-io/n8n/commit/0d907d67945dfd9624eda6f3fb634cee4bd2d195))
* **core:** Add JWE decryption to OAuth2 credential flow ([#29497](https://github.com/n8n-io/n8n/issues/29497)) ([ad7cdcc](https://github.com/n8n-io/n8n/commit/ad7cdcc04f47e1c34754636098ff698b7b153d05))
* **core:** Add MCP tool search executions ([#29161](https://github.com/n8n-io/n8n/issues/29161)) ([1d9548c](https://github.com/n8n-io/n8n/commit/1d9548c81f6a984882aadd7091cd649967aa7201))
* **core:** Add migration for postgres variable values ([#29489](https://github.com/n8n-io/n8n/issues/29489)) ([898ba5a](https://github.com/n8n-io/n8n/commit/898ba5ae2562542af11031b5dfdf0400afb91fbd))
* **core:** Add preAuthentication support to requestOAuth2 pipeline ([#29418](https://github.com/n8n-io/n8n/issues/29418)) ([473d49c](https://github.com/n8n-io/n8n/commit/473d49c9b18ff4d8226f54fe0c5c8a2a1c6fdca5))
* **core:** Bootstrap legacy CBC and initial GCM encryption keys on startup ([#29400](https://github.com/n8n-io/n8n/issues/29400)) ([9576ab9](https://github.com/n8n-io/n8n/commit/9576ab907cc3bdb560d1b40a1582ecf67c253d3a))
* **core:** Broadcast workflow settings updates ([#29459](https://github.com/n8n-io/n8n/issues/29459)) ([9cb1605](https://github.com/n8n-io/n8n/commit/9cb160585c05ccb1770554cd0998ea4d9b0ab3cc))
* **core:** Decouple insights pruning max age from license ([#29527](https://github.com/n8n-io/n8n/issues/29527)) ([45c18fb](https://github.com/n8n-io/n8n/commit/45c18fb09c04749063edc3545c38ad37006c0c49))
* **core:** Fix user access control logic ([#29481](https://github.com/n8n-io/n8n/issues/29481)) ([484cb2e](https://github.com/n8n-io/n8n/commit/484cb2efba8b33555c4d34bb95680d16a3328c1e))
* **core:** Manage MCP settings via environment variables ([#29368](https://github.com/n8n-io/n8n/issues/29368)) ([05e10e2](https://github.com/n8n-io/n8n/commit/05e10e268083fd7f9f1176634f0c1cab88297b94))
* **core:** Run evaluation test cases in parallel behind PostHog rollout flag ([#29412](https://github.com/n8n-io/n8n/issues/29412)) ([4c76aa1](https://github.com/n8n-io/n8n/commit/4c76aa1467d08d5f188cf8b7716b52b410f2bd65))
* **core:** Use versioned prebuilt Daytona snapshots for Instance AI sandboxes ([#29359](https://github.com/n8n-io/n8n/issues/29359)) ([308d0b4](https://github.com/n8n-io/n8n/commit/308d0b42b32a3372bac3a759b15ee410c9d095eb))
* **core:** Warn and skip on duplicate scheduled executions ([#28649](https://github.com/n8n-io/n8n/issues/28649)) ([b8b7571](https://github.com/n8n-io/n8n/commit/b8b75719ba373a27f60c6f471b170216fe7c41a9))
* **editor:** Add data encryption keys settings page ([#29068](https://github.com/n8n-io/n8n/issues/29068)) ([656f9c2](https://github.com/n8n-io/n8n/commit/656f9c2d7fc635c117efaeb40bb0fb98256f5ba3))
* **editor:** Add environment variable to disable workflow autosave ([#25144](https://github.com/n8n-io/n8n/issues/25144)) ([a2afc47](https://github.com/n8n-io/n8n/commit/a2afc47c226a716b7ae059306e684748c9d65947))
* **editor:** Add reveal redacted data permission to custom roles execution section ([#29526](https://github.com/n8n-io/n8n/issues/29526)) ([be22095](https://github.com/n8n-io/n8n/commit/be22095646c0daf2bbdc2afb7ebc4c1e4a50e349))
* **editor:** Add transition on Sidebar collapsed ([#29650](https://github.com/n8n-io/n8n/issues/29650)) ([07b5343](https://github.com/n8n-io/n8n/commit/07b53430f9e9efefaa78d90d3a613d5518ede4e5))
* **editor:** Hide model selector for unsupported AI Gateway actions ([#29588](https://github.com/n8n-io/n8n/issues/29588)) ([0f7776e](https://github.com/n8n-io/n8n/commit/0f7776e972c1d94d0f61d6d8855865802ef2a273))
* **editor:** Move Switch component to core design system ([#27322](https://github.com/n8n-io/n8n/issues/27322)) ([758f89c](https://github.com/n8n-io/n8n/commit/758f89c9ef4b936e1904c244698ccb4d92f6dd51))
* **editor:** Track IdP role mapping in provisioning telemetry ([#29416](https://github.com/n8n-io/n8n/issues/29416)) ([40da23f](https://github.com/n8n-io/n8n/commit/40da23f68899bc11240b252d417aa01dec8485a9))
* **editor:** Update copy for mcp settings ([#29399](https://github.com/n8n-io/n8n/issues/29399)) ([5f93b48](https://github.com/n8n-io/n8n/commit/5f93b48e79067251e782940489848f81f897d3a4))
* Include updatedAt in encryption key response DTO ([#29424](https://github.com/n8n-io/n8n/issues/29424)) ([569f94b](https://github.com/n8n-io/n8n/commit/569f94bb828bdd662bb291bd1d566e4e2a8ebdae))
* **instance-ai:** Orchestrator-executed checkpoint tasks for planned workflow verification ([#29049](https://github.com/n8n-io/n8n/issues/29049)) ([ad359b5](https://github.com/n8n-io/n8n/commit/ad359b5e2ceaaf2ba04559e43117d81bc5f2df25))
* **Netlify Trigger Node:** Add webhook request verification ([#29256](https://github.com/n8n-io/n8n/issues/29256)) ([1516ec7](https://github.com/n8n-io/n8n/commit/1516ec7c06ab797dbf94fd1b8a0322209e6ee0bc))
* **Slack Node:** Allow users to configure OAuth2 scopes ([#28728](https://github.com/n8n-io/n8n/issues/28728)) ([aa0daf9](https://github.com/n8n-io/n8n/commit/aa0daf9fb630661d35e8bd006ed3b749051f7a7d))
* Validate workflow-sdk output topology against mode ([#29363](https://github.com/n8n-io/n8n/issues/29363)) ([0a80722](https://github.com/n8n-io/n8n/commit/0a80722dcb3fcdbc23d9e768413b3141ec329adc))
# [2.19.0](https://github.com/n8n-io/n8n/compare/n8n@2.18.0...n8n@2.19.0) (2026-04-28)

View File

@ -1,6 +1,6 @@
{
"name": "n8n-monorepo",
"version": "2.19.0",
"version": "2.20.0",
"private": true,
"engines": {
"node": ">=22.16",

View File

@ -367,7 +367,7 @@ At end of turn, `saveToMemory()` uses `list.turnDelta()` and
`saveMessagesToThread`. If **semantic recall** is configured with an embedder
and `memory.saveEmbeddings`, new messages are embedded and stored.
**Working memory:** when configured, the runtime injects an `updateWorkingMemory`
**Working memory:** when configured, the runtime injects an `update_working_memory`
tool into the agent's tool set. The current state is included in the system prompt
so the model can read it; when new information should be persisted the model calls
the tool, which validates the input and asynchronously persists via
@ -415,7 +415,7 @@ src/
tool-adapter.ts — buildToolMap, executeTool, toAiSdkTools, suspend / agent-result guards
stream.ts — convertChunk, toTokenUsage
runtime-helpers.ts — normalizeInput, usage merge, stream error helpers, …
working-memory.ts — instruction text, updateWorkingMemory tool builder
working-memory.ts — instruction text, update_working_memory tool builder
strip-orphaned-tool-messages.ts
title-generation.ts
logger.ts

View File

@ -24,23 +24,31 @@
"test:integration": "vitest run --config vitest.integration.config.mjs"
},
"dependencies": {
"@ai-sdk/amazon-bedrock": "catalog:",
"@ai-sdk/anthropic": "^3.0.58",
"@ai-sdk/azure": "catalog:",
"@ai-sdk/cohere": "catalog:",
"@ai-sdk/deepseek": "catalog:",
"@ai-sdk/gateway": "catalog:",
"@ai-sdk/google": "^3.0.43",
"@ai-sdk/groq": "catalog:",
"@ai-sdk/mistral": "catalog:",
"@ai-sdk/openai": "^3.0.41",
"@ai-sdk/xai": "^3.0.67",
"@ai-sdk/provider-utils": "^4.0.21",
"@modelcontextprotocol/sdk": "1.26.0",
"ajv": "^8.18.0",
"@ai-sdk/xai": "^3.0.67",
"@libsql/client": "^0.17.0",
"@modelcontextprotocol/sdk": "1.26.0",
"@openrouter/ai-sdk-provider": "catalog:",
"ai": "^6.0.116",
"ajv": "^8.18.0",
"pg": "catalog:",
"zod": "catalog:"
},
"peerDependencies": {
"langsmith": ">=0.3.0",
"@opentelemetry/sdk-trace-node": ">=1.0.0",
"@opentelemetry/exporter-trace-otlp-http": ">=0.50.0",
"@opentelemetry/sdk-trace-base": ">=1.0.0",
"@opentelemetry/exporter-trace-otlp-http": ">=0.50.0"
"@opentelemetry/sdk-trace-node": ">=1.0.0",
"langsmith": ">=0.3.0"
},
"peerDependenciesMeta": {
"langsmith": {

View File

@ -6,7 +6,8 @@ import { isLlmMessage } from '../sdk/message';
import { Tool, Tool as ToolBuilder } from '../sdk/tool';
import { AgentEvent } from '../types/runtime/event';
import type { StreamChunk } from '../types/sdk/agent';
import type { ContentToolResult, Message } from '../types/sdk/message';
import type { BuiltMemory } from '../types/sdk/memory';
import type { ContentToolCall, Message } from '../types/sdk/message';
import type { BuiltTool, InterruptibleToolContext } from '../types/sdk/tool';
import type { BuiltTelemetry } from '../types/telemetry';
@ -236,9 +237,9 @@ describe('AgentRuntime.generate() — graceful error contract', () => {
generateText.mockRejectedValue(new Error('API failure'));
const { runtime } = createRuntime();
const result = await runtime.generate('hello');
await runtime.generate('hello');
expect(result.getState().status).toBe('failed');
expect(runtime.getState().status).toBe('failed');
});
it('emits AgentEvent.Error (not AgentEnd) when the LLM call throws', async () => {
@ -266,10 +267,10 @@ describe('AgentRuntime.generate() — graceful error contract', () => {
// Abort during AgentStart so the loop's first abort-check fires before generateText is called
bus.on(AgentEvent.AgentStart, () => bus.abort());
const result = await runtime.generate('hello');
await runtime.generate('hello');
expect(errorEvents.length).toBe(0);
expect(result.getState().status).toBe('cancelled');
expect(runtime.getState().status).toBe('cancelled');
});
it('returns finishReason "error" and sets cancelled status on abort', async () => {
@ -282,7 +283,7 @@ describe('AgentRuntime.generate() — graceful error contract', () => {
const result = await runtime.generate('hello');
expect(result.finishReason).toBe('error');
expect(result.getState().status).toBe('cancelled');
expect(runtime.getState().status).toBe('cancelled');
});
it('is reusable after an error — subsequent call with a good LLM response succeeds', async () => {
@ -400,10 +401,10 @@ describe('AgentRuntime.stream() — graceful error contract', () => {
});
const { runtime } = createRuntime();
const { stream: readableStream, getState } = await runtime.stream('hello');
const { stream: readableStream } = await runtime.stream('hello');
await collectChunks(readableStream);
expect(getState().status).toBe('failed');
expect(runtime.getState().status).toBe('failed');
});
it('yields error chunk and finishes cleanly on abort', async () => {
@ -412,13 +413,13 @@ describe('AgentRuntime.stream() — graceful error contract', () => {
const { runtime, bus } = createRuntime();
bus.on(AgentEvent.TurnStart, () => bus.abort());
const { stream: readableStream, getState } = await runtime.stream('hello');
const { stream: readableStream } = await runtime.stream('hello');
const chunks = await collectChunks(readableStream);
const errorChunks = chunks.filter((c) => c.type === 'error');
expect(errorChunks.length).toBeGreaterThan(0);
expect(getState().status).toBe('cancelled');
expect(runtime.getState().status).toBe('cancelled');
});
it('stream is reusable after an error', async () => {
@ -466,6 +467,120 @@ describe('AgentRuntime.stream() — graceful error contract', () => {
});
});
// ---------------------------------------------------------------------------
// stream() — working memory
// ---------------------------------------------------------------------------
describe('AgentRuntime.stream() — working memory', () => {
beforeEach(() => {
jest.clearAllMocks();
});
function makeMemory(savedWorkingMemory: string[]): BuiltMemory {
return {
getThread: jest.fn().mockResolvedValue(null),
saveThread: jest.fn(async (thread) => {
await Promise.resolve();
return {
...thread,
createdAt: new Date(),
updatedAt: new Date(),
};
}),
deleteThread: jest.fn(),
getMessages: jest.fn().mockResolvedValue([]),
saveMessages: jest.fn(),
deleteMessages: jest.fn(),
getWorkingMemory: jest.fn().mockResolvedValue(null),
saveWorkingMemory: jest.fn(async (_params, content: string) => {
await Promise.resolve();
savedWorkingMemory.push(content);
}),
describe: jest
.fn()
.mockReturnValue({ name: 'test', constructorName: 'TestMemory', connectionParams: {} }),
};
}
it('persists working memory and streams the tool chunks unfiltered', async () => {
const savedWorkingMemory: string[] = [];
const memoryContent = '# Thread memory\n- User facts: Alice likes concise answers';
const memory = makeMemory(savedWorkingMemory);
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'You are a test assistant.',
memory,
lastMessages: 5,
workingMemory: {
template: '# Thread memory\n- User facts:',
structured: false,
scope: 'thread',
},
});
streamText
.mockReturnValueOnce({
fullStream: makeChunkStream([
{ type: 'tool-input-start', id: 'wm-1', toolName: 'update_working_memory' },
{ type: 'tool-input-delta', id: 'wm-1', delta: memoryContent },
{
type: 'tool-call',
toolCallId: 'wm-1',
toolName: 'update_working_memory',
input: { memory: memoryContent },
},
]),
finishReason: Promise.resolve('tool-calls'),
usage: Promise.resolve({ inputTokens: 10, outputTokens: 5, totalTokens: 15 }),
response: Promise.resolve({
messages: [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'wm-1',
toolName: 'update_working_memory',
args: { memory: memoryContent },
},
],
},
],
}),
toolCalls: Promise.resolve([
{
toolCallId: 'wm-1',
toolName: 'update_working_memory',
input: { memory: memoryContent },
},
]),
})
.mockReturnValueOnce(makeStreamSuccess('Done'));
const { stream } = await runtime.stream('remember this', {
persistence: { threadId: 'thread-1', resourceId: 'user-1' },
});
const chunks = await collectChunks(stream);
expect(savedWorkingMemory).toEqual([memoryContent]);
expect(chunks).toContainEqual(
expect.objectContaining({
type: 'tool-call',
toolCallId: 'wm-1',
toolName: 'update_working_memory',
}),
);
expect(chunks).toContainEqual(
expect.objectContaining({
type: 'tool-result',
toolCallId: 'wm-1',
toolName: 'update_working_memory',
}),
);
});
});
// ---------------------------------------------------------------------------
// resume() — graceful error contract
// ---------------------------------------------------------------------------
@ -497,37 +612,35 @@ describe('AgentRuntime — state transitions on error', () => {
jest.clearAllMocks();
});
it('starts idle before first run', () => {
it('starts idle, then reflects running→failed after a generate error', async () => {
const { runtime } = createRuntime();
expect(runtime.getState().status).toBe('idle');
});
it('result.getState() reflects failed after a generate error', async () => {
generateText.mockRejectedValue(new Error('oops'));
const runDone = runtime.generate('hi');
const { runtime } = createRuntime();
const result = await runtime.generate('hi');
expect(result.getState().status).toBe('failed');
await runDone;
expect(runtime.getState().status).toBe('failed');
});
it('result.getState() reflects cancelled on abort', async () => {
it('starts idle, then reflects running→cancelled on abort', async () => {
generateText.mockResolvedValue(makeGenerateSuccess());
const { runtime, bus } = createRuntime();
bus.on(AgentEvent.AgentStart, () => bus.abort());
const result = await runtime.generate('hi');
expect(result.getState().status).toBe('cancelled');
await runtime.generate('hi');
expect(runtime.getState().status).toBe('cancelled');
});
it('result.getState() transitions to success on a clean run', async () => {
it('transitions to success on a clean run', async () => {
generateText.mockResolvedValue(makeGenerateSuccess());
const { runtime } = createRuntime();
const result = await runtime.generate('hi');
await runtime.generate('hi');
expect(result.getState().status).toBe('success');
expect(runtime.getState().status).toBe('success');
});
});
@ -675,7 +788,7 @@ describe('AgentRuntime — concurrent tool execution', () => {
expect(result.pendingSuspend![0].toolCallId).toBe('tc-1');
// Verify tc-3 is in the persisted state as a pending tool call (without suspendPayload)
const state = result.getState();
const state = runtime.getState();
expect(state.pendingToolCalls['tc-3']).toBeDefined();
expect(state.pendingToolCalls['tc-3'].suspended).toBe(false);
});
@ -905,7 +1018,7 @@ describe('AgentRuntime — concurrent tool execution', () => {
it('tool error produces an error tool-result in the message list and loop continues', async () => {
type ToolOutputContent = {
type: string;
output?: { type: string; value?: { error?: string } };
output?: { type: string; value?: unknown };
};
type ToolMessage = { role: string; content: ToolOutputContent[] };
const receivedMessages: unknown[] = [];
@ -932,13 +1045,15 @@ describe('AgentRuntime — concurrent tool execution', () => {
expect(result.finishReason).toBe('stop');
// LLM was called a second time — it saw the error tool result and continued
expect(generateText).toHaveBeenCalledTimes(2);
// The second LLM call received a tool message whose output carries the error description
// The second LLM call received a tool message whose output carries the error description.
const toolMsg = receivedMessages.find(
(m): m is ToolMessage =>
typeof m === 'object' && m !== null && (m as ToolMessage).role === 'tool',
);
expect(toolMsg).toBeDefined();
const hasErrorOutput = toolMsg!.content.some((c) => !!c.output?.value?.error);
const hasErrorOutput = toolMsg!.content.some(
(c) => c.output?.type === 'error-text' || c.output?.type === 'error-json',
);
expect(hasErrorOutput).toBe(true);
});
@ -982,9 +1097,9 @@ describe('AgentRuntime — concurrent tool execution', () => {
]),
);
const result = await runtime.generate('run tools');
await runtime.generate('run tools');
const state = result.getState();
const state = runtime.getState();
expect(state.pendingToolCalls['tc-1']).toBeDefined();
expect(state.pendingToolCalls['tc-1'].toolName).toBe('suspend_tool');
});
@ -1007,9 +1122,9 @@ describe('AgentRuntime — concurrent tool execution', () => {
]),
);
const result = await runtime.generate('run tools');
await runtime.generate('run tools');
const state = result.getState();
const state = runtime.getState();
expect(state.pendingToolCalls['tc-2']).toBeDefined();
expect(state.pendingToolCalls['tc-2'].toolName).toBe('normal_tool');
expect(state.pendingToolCalls['tc-2'].suspended).toBe(false);
@ -1554,17 +1669,14 @@ describe('AgentRuntime — runtime input schema validation', () => {
// the LLM responds with 'done' on the next turn.
expect(result.finishReason).toBe('stop');
const toolErrorMessage = result.messages.find(
(m) => isLlmMessage(m) && m.role === 'tool' && m.content[0].type === 'tool-result',
const assistantMsg = result.messages.find(
(m) =>
isLlmMessage(m) && m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
) as Message;
expect(toolErrorMessage).toBeDefined();
const content = toolErrorMessage.content[0] as ContentToolResult;
expect(content.result).toEqual(
expect.objectContaining({
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
error: expect.stringContaining('Expected string, received number'),
}),
);
expect(assistantMsg).toBeDefined();
const call = assistantMsg.content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(call.state).toBe('rejected');
expect(call.state === 'rejected' && call.error).toContain('Expected string, received number');
});
});
@ -1603,13 +1715,14 @@ describe('AgentRuntime — runtime JSON Schema input validation', () => {
const result = await runtime.generate('go');
expect(result.finishReason).toBe('stop');
// No tool-result error — the tool ran successfully
const toolResultMsg = result.messages.find(
(m) => isLlmMessage(m) && m.role === 'tool',
// No error — the tool ran successfully
const assistantMsg = result.messages.find(
(m) =>
isLlmMessage(m) && m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
) as Message;
expect(toolResultMsg).toBeDefined();
const content = toolResultMsg.content[0] as ContentToolResult;
expect(content.isError).toBeFalsy();
expect(assistantMsg).toBeDefined();
const call = assistantMsg.content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(call.state).toBe('resolved');
});
it('surfaces a validation error as a tool error outcome when LLM provides the wrong type', async () => {
@ -1639,14 +1752,14 @@ describe('AgentRuntime — runtime JSON Schema input validation', () => {
const result = await runtime.generate('go');
expect(result.finishReason).toBe('stop');
const toolResultMsg = result.messages.find(
(m) => isLlmMessage(m) && m.role === 'tool',
const assistantMsg = result.messages.find(
(m) =>
isLlmMessage(m) && m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
) as Message;
expect(toolResultMsg).toBeDefined();
console.log('ToolResultMsg', toolResultMsg);
const content = toolResultMsg.content[0] as ContentToolResult;
expect(content.isError).toBe(true);
expect(JSON.stringify(content.result)).toContain('Invalid tool input');
expect(assistantMsg).toBeDefined();
const call = assistantMsg.content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(call.state).toBe('rejected');
expect(call.state === 'rejected' && call.error).toContain('Invalid tool input');
});
it('surfaces a validation error when a required property is missing', async () => {
@ -1677,15 +1790,15 @@ describe('AgentRuntime — runtime JSON Schema input validation', () => {
});
const result = await runtime.generate('go');
console.log('Result', result.error);
expect(result.finishReason).toBe('stop');
const toolResultMsg = result.messages.find(
(m) => isLlmMessage(m) && m.role === 'tool',
const assistantMsg = result.messages.find(
(m) =>
isLlmMessage(m) && m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
) as Message;
const content = toolResultMsg.content[0] as ContentToolResult;
expect(content.isError).toBe(true);
expect(JSON.stringify(content.result)).toContain('Invalid tool input');
const call = assistantMsg.content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(call.state).toBe('rejected');
expect(call.state === 'rejected' && call.error).toContain('Invalid tool input');
});
it('does not invoke the handler when JSON Schema validation fails', async () => {
@ -1718,6 +1831,142 @@ describe('AgentRuntime — runtime JSON Schema input validation', () => {
});
});
// ---------------------------------------------------------------------------
// Tool builder — JSON Schema input integration
//
// Mirrors the resolveNodeTool() code path in node-tool-factory.ts where the
// input schema is a raw JSON Schema object (converted from Zod by ToolFromNode).
// ---------------------------------------------------------------------------
describe('AgentRuntime — Tool builder with JSON Schema input', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('passes valid input to the handler when built via Tool builder', async () => {
const handlerFn = jest.fn().mockResolvedValue({ found: true });
const tool = new Tool('lookup')
.description('Look up a record by id')
.input({
type: 'object',
properties: { id: { type: 'string' } },
required: ['id'],
})
.handler(handlerFn)
.build();
generateText
.mockResolvedValueOnce(makeGenerateWithToolCall('tc-1', 'lookup', { id: 'abc-123' }))
.mockResolvedValueOnce(makeGenerateSuccess('done'));
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'test',
tools: [tool],
});
const result = await runtime.generate('go');
expect(result.finishReason).toBe('stop');
expect(handlerFn).toHaveBeenCalledWith(
expect.objectContaining({ id: 'abc-123' }),
expect.anything(),
);
const assistantMsg = result.messages.find(
(m) =>
isLlmMessage(m) && m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
) as Message;
const call = assistantMsg.content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(call.state).toBe('resolved');
});
it('produces a tool error when the LLM sends input that fails JSON Schema validation', async () => {
const handlerFn = jest.fn().mockResolvedValue({ found: true });
const tool = new Tool('lookup')
.description('Look up a record by id')
.input({
type: 'object',
properties: {
id: { type: 'string' },
count: { type: 'integer', minimum: 1 },
},
required: ['id', 'count'],
})
.handler(handlerFn)
.build();
generateText
// LLM sends count: 0 (violates minimum: 1) and id as a number (wrong type)
.mockResolvedValueOnce(makeGenerateWithToolCall('tc-1', 'lookup', { id: 42, count: 0 }))
.mockResolvedValueOnce(makeGenerateSuccess('corrected'));
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'test',
tools: [tool],
});
const result = await runtime.generate('go');
expect(result.finishReason).toBe('stop');
// Handler must not be called — validation should block execution
expect(handlerFn).not.toHaveBeenCalled();
const assistantMsg = result.messages.find(
(m) =>
isLlmMessage(m) && m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
) as Message;
const call = assistantMsg.content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(call.state).toBe('rejected');
expect(call.state === 'rejected' && call.error).toContain('Invalid tool input');
});
it('validates enum and pattern constraints defined in JSON Schema', async () => {
const handlerFn = jest.fn().mockResolvedValue({ ok: true });
const tool = new Tool('set_status')
.description('Set the status of a record')
.input({
type: 'object',
properties: {
status: { type: 'string', enum: ['active', 'inactive', 'pending'] },
},
required: ['status'],
})
.handler(handlerFn)
.build();
// First call: invalid enum value
generateText
.mockResolvedValueOnce(makeGenerateWithToolCall('tc-1', 'set_status', { status: 'deleted' }))
// Second call: valid enum value after self-correction
.mockResolvedValueOnce(makeGenerateWithToolCall('tc-2', 'set_status', { status: 'inactive' }))
.mockResolvedValueOnce(makeGenerateSuccess('done'));
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'test',
tools: [tool],
});
const result = await runtime.generate('go');
expect(result.finishReason).toBe('stop');
// Handler called exactly once — only for the valid input
expect(handlerFn).toHaveBeenCalledTimes(1);
expect(handlerFn).toHaveBeenCalledWith(
expect.objectContaining({ status: 'inactive' }),
expect.anything(),
);
});
});
// ---------------------------------------------------------------------------
// Runtime validation — resume data schema
// ---------------------------------------------------------------------------
@ -1953,6 +2202,114 @@ describe('provider options merging', () => {
// Instruction providerOptions
// ---------------------------------------------------------------------------
describe('tool systemInstruction merging', () => {
beforeEach(() => {
jest.clearAllMocks();
});
function getSystemMessageText(): string {
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
const callArgs = generateText.mock.calls[0][0] as Record<string, unknown>;
const messages = callArgs.messages as Array<Record<string, unknown>>;
const systemMsg = messages[0];
expect(systemMsg.role).toBe('system');
return String(systemMsg.content);
}
it("wraps a tool's systemInstruction in a built_in_rules block above user instructions", async () => {
generateText.mockResolvedValue(makeGenerateSuccess());
const toolWithDirective: BuiltTool = {
name: 'show_card',
description: 'show a card',
systemInstruction: 'Prefer this tool over plain text when posting images.',
inputSchema: z.object({ value: z.string().optional() }),
handler: async () => await Promise.resolve('ok'),
};
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'You are a helpful assistant.',
tools: [toolWithDirective],
});
await runtime.generate('hello');
const text = getSystemMessageText();
expect(text).toContain('<built_in_rules>');
expect(text).toContain('- Prefer this tool over plain text when posting images.');
expect(text).toContain('</built_in_rules>');
expect(text).toContain('You are a helpful assistant.');
expect(text.indexOf('<built_in_rules>')).toBeLessThan(text.indexOf('You are a helpful'));
});
it('joins multiple tools systemInstructions into a single block', async () => {
generateText.mockResolvedValue(makeGenerateSuccess());
const toolA: BuiltTool = {
name: 'a',
description: 'a',
systemInstruction: 'Rule A.',
inputSchema: z.object({}),
handler: async () => await Promise.resolve('ok'),
};
const toolB: BuiltTool = {
name: 'b',
description: 'b',
systemInstruction: 'Rule B.',
inputSchema: z.object({}),
handler: async () => await Promise.resolve('ok'),
};
const toolC: BuiltTool = {
name: 'c',
description: 'c',
inputSchema: z.object({}),
handler: async () => await Promise.resolve('ok'),
};
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'base',
tools: [toolA, toolB, toolC],
});
await runtime.generate('hello');
const text = getSystemMessageText();
const block = text.match(/<built_in_rules>([\s\S]*?)<\/built_in_rules>/);
expect(block).not.toBeNull();
expect(block![1]).toContain('- Rule A.');
expect(block![1]).toContain('- Rule B.');
expect(block![1]).not.toContain('Rule C');
});
it('does not add a built_in_rules block when no tool sets a systemInstruction', async () => {
generateText.mockResolvedValue(makeGenerateSuccess());
const plainTool: BuiltTool = {
name: 'plain',
description: 'plain',
inputSchema: z.object({}),
handler: async () => await Promise.resolve('ok'),
};
const runtime = new AgentRuntime({
name: 'test',
model: 'openai/gpt-4o-mini',
instructions: 'You are a helpful assistant.',
tools: [plainTool],
});
await runtime.generate('hello');
const text = getSystemMessageText();
expect(text).not.toContain('<built_in_rules>');
expect(text).toContain('You are a helpful assistant.');
});
});
describe('instruction providerOptions', () => {
beforeEach(() => {
jest.clearAllMocks();

View File

@ -1,445 +0,0 @@
/**
* Tests for the Agent builder focusing on per-run isolation guarantees introduced
* by the "shared config, per-run runtime" refactor.
*/
import { Agent } from '../sdk/agent';
import { AgentEvent } from '../types/runtime/event';
// ---------------------------------------------------------------------------
// Module mocks (same as agent-runtime.test.ts)
// ---------------------------------------------------------------------------
jest.mock('@ai-sdk/openai', () => ({
createOpenAI: () => () => ({ provider: 'openai', modelId: 'mock', specificationVersion: 'v3' }),
}));
jest.mock('@ai-sdk/anthropic', () => ({
createAnthropic: () => () => ({
provider: 'anthropic',
modelId: 'mock',
specificationVersion: 'v3',
}),
}));
// eslint-disable-next-line @typescript-eslint/consistent-type-imports
type AiImport = typeof import('ai');
jest.mock('ai', () => {
const actual = jest.requireActual<AiImport>('ai');
return {
...actual,
generateText: jest.fn(),
streamText: jest.fn(),
tool: jest.fn((config: unknown) => config),
Output: {
object: jest.fn(({ schema }: { schema: unknown }) => ({ _type: 'object', schema })),
},
};
});
// Prevent real catalog HTTP calls
jest.mock('../sdk/catalog', () => ({
getModelCost: jest.fn().mockResolvedValue(undefined),
computeCost: jest.fn(),
}));
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { generateText, streamText } = require('ai') as {
generateText: jest.Mock;
streamText: jest.Mock;
};
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function makeGenerateSuccess(text = 'OK') {
return {
finishReason: 'stop',
usage: { inputTokens: 10, outputTokens: 5, totalTokens: 15 },
response: {
messages: [{ role: 'assistant', content: [{ type: 'text', text }] }],
},
toolCalls: [],
};
}
function* makeChunkStream(chunks: Array<Record<string, unknown>>) {
for (const c of chunks) yield c;
}
function makeStreamSuccess(text = 'Hello') {
return {
fullStream: makeChunkStream([{ type: 'text-delta', textDelta: text }]),
finishReason: Promise.resolve('stop'),
usage: Promise.resolve({ inputTokens: 10, outputTokens: 5, totalTokens: 15 }),
response: Promise.resolve({
messages: [{ role: 'assistant', content: [{ type: 'text', text }] }],
}),
toolCalls: Promise.resolve([]),
};
}
async function drainStream(stream: ReadableStream<unknown>): Promise<void> {
const reader = stream.getReader();
while (true) {
const { done } = await reader.read();
if (done) break;
}
}
function buildAgent() {
return new Agent('test').model('openai/gpt-4o-mini').instructions('You are a test assistant.');
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('Agent — per-run isolation', () => {
beforeEach(() => {
jest.clearAllMocks();
});
describe('concurrent generate() calls', () => {
it('returns independent results for each call', async () => {
generateText
.mockResolvedValueOnce(makeGenerateSuccess('Result A'))
.mockResolvedValueOnce(makeGenerateSuccess('Result B'));
const agent = buildAgent();
const [resultA, resultB] = await Promise.all([
agent.generate('Prompt A'),
agent.generate('Prompt B'),
]);
const textA = resultA.messages
.flatMap((m) => ('content' in m ? m.content : []))
.filter((c) => c.type === 'text')
.map((c) => ('text' in c ? c.text : ''))
.join('');
const textB = resultB.messages
.flatMap((m) => ('content' in m ? m.content : []))
.filter((c) => c.type === 'text')
.map((c) => ('text' in c ? c.text : ''))
.join('');
expect(textA).toBe('Result A');
expect(textB).toBe('Result B');
expect(resultA.runId).not.toBe(resultB.runId);
});
it('aborting one generate() does not cancel the other', async () => {
const abortControllerA = new AbortController();
// Run A resolves only after a delay; we'll abort it via its signal.
// Run B resolves immediately.
let resolveA!: (v: unknown) => void;
const pendingA = new Promise((res) => {
resolveA = res;
});
generateText.mockImplementation(async ({ abortSignal }: { abortSignal?: AbortSignal }) => {
if (abortSignal === abortControllerA.signal || abortSignal?.aborted) {
// Simulate the AI SDK throwing on abort
await new Promise((_, rej) =>
abortSignal.addEventListener('abort', () => rej(new Error('aborted')), {
once: true,
}),
);
}
// Run B path — return immediately
await pendingA;
return makeGenerateSuccess('Result B');
});
const agent = buildAgent();
// Start both runs; abort run A immediately
const runAPromise = agent.generate('Prompt A', { abortSignal: abortControllerA.signal });
abortControllerA.abort();
resolveA(undefined);
const runA = await runAPromise;
expect(runA.finishReason).toBe('error');
// Run B separately (no abort)
generateText.mockResolvedValueOnce(makeGenerateSuccess('Result B'));
const runB = await agent.generate('Prompt B');
const textB = runB.messages
.flatMap((m) => ('content' in m ? m.content : []))
.filter((c) => c.type === 'text')
.map((c) => ('text' in c ? c.text : ''))
.join('');
expect(textB).toBe('Result B');
});
});
describe('concurrent stream() calls', () => {
it('returns independent streams for each call', async () => {
streamText
.mockReturnValueOnce(makeStreamSuccess('Stream A'))
.mockReturnValueOnce(makeStreamSuccess('Stream B'));
const agent = buildAgent();
const [resultA, resultB] = await Promise.all([
agent.stream('Prompt A'),
agent.stream('Prompt B'),
]);
// Both streams should be distinct ReadableStream objects
expect(resultA.stream).not.toBe(resultB.stream);
expect(resultA.runId).not.toBe(resultB.runId);
// Drain both streams to completion
await Promise.all([drainStream(resultA.stream), drainStream(resultB.stream)]);
});
it('aborting one stream does not cancel the other', async () => {
const abortControllerA = new AbortController();
streamText.mockImplementation(({ abortSignal }: { abortSignal?: AbortSignal }) => {
if (abortSignal === abortControllerA.signal) {
return {
fullStream: (async function* () {
// Wait until aborted then throw
await new Promise<void>((_, rej) => {
abortSignal.addEventListener('abort', () => rej(new Error('aborted')), {
once: true,
});
});
yield 'something';
})(),
finishReason: Promise.resolve('error'),
usage: Promise.resolve({ inputTokens: 0, outputTokens: 0, totalTokens: 0 }),
response: Promise.resolve({ messages: [] }),
toolCalls: Promise.resolve([]),
};
}
return makeStreamSuccess('Stream B');
});
const agent = buildAgent();
const [resultA, resultB] = await Promise.all([
agent.stream('Prompt A', { abortSignal: abortControllerA.signal }),
agent.stream('Prompt B'),
]);
// Abort run A
abortControllerA.abort();
// Drain stream B — it should complete successfully regardless of A being aborted
await drainStream(resultB.stream);
// Drain stream A — it will error but shouldn't affect B
await drainStream(resultA.stream).catch(() => {});
});
});
describe('event handlers (on())', () => {
it('fires registered handlers for every concurrent run', async () => {
generateText
.mockResolvedValueOnce(makeGenerateSuccess('A'))
.mockResolvedValueOnce(makeGenerateSuccess('B'));
const agent = buildAgent();
const agentStartEvents: string[] = [];
agent.on(AgentEvent.AgentStart, () => {
agentStartEvents.push('start');
});
await Promise.all([agent.generate('Prompt A'), agent.generate('Prompt B')]);
// Handler should have fired once per run
expect(agentStartEvents).toHaveLength(2);
});
it('handlers registered before first run still fire on every subsequent run', async () => {
generateText
.mockResolvedValueOnce(makeGenerateSuccess('First'))
.mockResolvedValueOnce(makeGenerateSuccess('Second'));
const agent = buildAgent();
const events: string[] = [];
agent.on(AgentEvent.AgentEnd, () => {
events.push('end');
});
await agent.generate('First');
await agent.generate('Second');
expect(events).toHaveLength(2);
});
});
describe('abort() broadcast', () => {
it('aborts all active runs when agent.abort() is called', async () => {
let resolveA!: (v: unknown) => void;
generateText.mockImplementation(async ({ abortSignal }: { abortSignal?: AbortSignal }) => {
// Each call waits until its resolver is called or the signal fires
await new Promise((res, rej) => {
abortSignal?.addEventListener('abort', () => rej(new Error('aborted')), {
once: true,
});
resolveA ??= res;
});
return makeGenerateSuccess();
});
const agent = buildAgent();
const runAPromise = agent.generate('A');
const runBPromise = agent.generate('B');
// Give both calls time to reach the mock and register abort listeners
await new Promise((res) => setTimeout(res, 10));
// Broadcast abort — both runs should be cancelled
agent.abort();
const [runA, runB] = await Promise.all([runAPromise, runBPromise]);
expect(runA.finishReason).toBe('error');
expect(runB.finishReason).toBe('error');
});
});
describe('off() — event handler removal', () => {
it('removes a specific handler so it no longer fires', async () => {
generateText
.mockResolvedValueOnce(makeGenerateSuccess('A'))
.mockResolvedValueOnce(makeGenerateSuccess('B'));
const agent = buildAgent();
const events: string[] = [];
const handler = () => events.push('end');
agent.on(AgentEvent.AgentEnd, handler);
await agent.generate('First');
agent.off(AgentEvent.AgentEnd, handler);
await agent.generate('Second');
// Handler should have fired only for the first run
expect(events).toHaveLength(1);
});
it('removing one handler does not affect other handlers for the same event', async () => {
generateText.mockResolvedValueOnce(makeGenerateSuccess('A'));
const agent = buildAgent();
const firedA: string[] = [];
const firedB: string[] = [];
const handlerA = () => firedA.push('a');
const handlerB = () => firedB.push('b');
agent.on(AgentEvent.AgentEnd, handlerA);
agent.on(AgentEvent.AgentEnd, handlerB);
agent.off(AgentEvent.AgentEnd, handlerA);
await agent.generate('Hello');
expect(firedA).toHaveLength(0);
expect(firedB).toHaveLength(1);
});
it('off() on a handler that was never registered is a no-op', () => {
const agent = buildAgent();
expect(() => agent.off(AgentEvent.AgentEnd, () => {})).not.toThrow();
});
});
describe('trackStreamBus — cleanup on stream cancel', () => {
it('removes the bus from active runs when the consumer cancels the stream', async () => {
streamText.mockReturnValueOnce(makeStreamSuccess('Hello'));
const agent = buildAgent();
// Access the private set via casting so we can assert its size
const getActiveBuses = () =>
(agent as unknown as { activeEventBuses: Set<unknown> }).activeEventBuses;
const { stream } = await agent.stream('Hello');
// Bus is registered while the stream is live
expect(getActiveBuses().size).toBe(1);
// Consumer cancels instead of draining
await stream.cancel();
// Bus must be removed immediately after cancel
expect(getActiveBuses().size).toBe(0);
});
it('removes the bus from active runs when the consumer drains the stream normally', async () => {
streamText.mockReturnValueOnce(makeStreamSuccess('Hello'));
const agent = buildAgent();
const getActiveBuses = () =>
(agent as unknown as { activeEventBuses: Set<unknown> }).activeEventBuses;
const { stream } = await agent.stream('Hello');
expect(getActiveBuses().size).toBe(1);
await drainStream(stream);
expect(getActiveBuses().size).toBe(0);
});
it('abort() after stream cancel does not throw on a disposed bus', async () => {
streamText.mockReturnValueOnce(makeStreamSuccess('Hello'));
const agent = buildAgent();
const { stream } = await agent.stream('Hello');
await stream.cancel();
// agent.abort() should be harmless — no active buses remain
expect(() => agent.abort()).not.toThrow();
});
});
describe('result.getState()', () => {
it('generate() result.getState() reports success after a clean run', async () => {
generateText.mockResolvedValueOnce(makeGenerateSuccess());
const agent = buildAgent();
const result = await agent.generate('Hello');
expect(result.getState().status).toBe('success');
});
it('generate() result.getState() reports failed after an error', async () => {
generateText.mockRejectedValueOnce(new Error('boom'));
const agent = buildAgent();
const result = await agent.generate('Hello');
expect(result.getState().status).toBe('failed');
});
it('stream() result.getState() reports success after the stream is consumed', async () => {
streamText.mockReturnValueOnce(makeStreamSuccess());
const agent = buildAgent();
const { stream, getState } = await agent.stream('Hello');
// State is running while stream is open
expect(getState().status).toBe('running');
await drainStream(stream);
expect(getState().status).toBe('success');
});
});
});

View File

@ -1,405 +0,0 @@
import { z } from 'zod';
import { Agent } from '../sdk/agent';
import { McpClient } from '../sdk/mcp-client';
import { Telemetry } from '../sdk/telemetry';
import { Tool } from '../sdk/tool';
import type { BuiltEval, BuiltGuardrail, BuiltMemory, BuiltProviderTool } from '../types';
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function makeMockMemory(): BuiltMemory {
return {
getThread: jest.fn(),
saveThread: jest.fn(),
deleteThread: jest.fn(),
getMessages: jest.fn(),
saveMessages: jest.fn(),
deleteMessages: jest.fn(),
} as unknown as BuiltMemory;
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('Agent.describe()', () => {
it('returns null/empty fields for an unconfigured agent', () => {
const agent = new Agent('test-agent');
const schema = agent.describe();
expect(schema.model).toEqual({ provider: null, name: null });
expect(schema.credential).toBeNull();
expect(schema.instructions).toBeNull();
expect(schema.description).toBeNull();
expect(schema.tools).toEqual([]);
expect(schema.providerTools).toEqual([]);
expect(schema.memory).toBeNull();
expect(schema.evaluations).toEqual([]);
expect(schema.guardrails).toEqual([]);
expect(schema.mcp).toBeNull();
expect(schema.telemetry).toBeNull();
expect(schema.checkpoint).toBeNull();
expect(schema.config.structuredOutput).toEqual({ enabled: false, schemaSource: null });
expect(schema.config.thinking).toBeNull();
expect(schema.config.toolCallConcurrency).toBeNull();
expect(schema.config.requireToolApproval).toBe(false);
});
// --- Model parsing ---
it('parses two-arg model (provider, name)', () => {
const agent = new Agent('test-agent').model('anthropic', 'claude-sonnet-4-5');
const schema = agent.describe();
expect(schema.model).toEqual({ provider: 'anthropic', name: 'claude-sonnet-4-5' });
});
it('parses single-arg model with slash', () => {
const agent = new Agent('test-agent').model('anthropic/claude-sonnet-4-5');
const schema = agent.describe();
expect(schema.model).toEqual({ provider: 'anthropic', name: 'claude-sonnet-4-5' });
});
it('parses model without slash', () => {
const agent = new Agent('test-agent').model('gpt-4o');
const schema = agent.describe();
expect(schema.model).toEqual({ provider: null, name: 'gpt-4o' });
});
it('handles object model config', () => {
const agent = new Agent('test-agent').model({
id: 'anthropic/claude-sonnet-4-5',
apiKey: 'sk-test',
});
const schema = agent.describe();
expect(schema.model).toEqual({ provider: null, name: null, raw: 'object' });
});
// --- Credential ---
it('returns credential name', () => {
const agent = new Agent('test-agent').credential('my-anthropic-key');
const schema = agent.describe();
expect(schema.credential).toBe('my-anthropic-key');
});
// --- Instructions ---
it('returns instructions text', () => {
const agent = new Agent('test-agent').instructions('You are helpful.');
const schema = agent.describe();
expect(schema.instructions).toBe('You are helpful.');
});
// --- Custom tool ---
it('describes a custom tool with handler, input schema, and suspend/resume', () => {
const suspendSchema = z.object({ reason: z.string() });
const resumeSchema = z.object({ approved: z.boolean() });
const tool = new Tool('danger')
.description('A dangerous action')
.input(z.object({ target: z.string() }))
.output(z.object({ result: z.string() }))
.suspend(suspendSchema)
.resume(resumeSchema)
.handler(async ({ target }) => await Promise.resolve({ result: target }))
.build();
const agent = new Agent('test-agent').tool(tool);
const schema = agent.describe();
expect(schema.tools).toHaveLength(1);
const ts = schema.tools[0];
expect(ts.name).toBe('danger');
expect(ts.editable).toBe(true);
expect(ts.hasSuspend).toBe(true);
expect(ts.hasResume).toBe(true);
expect(ts.hasToMessage).toBe(false);
expect(ts.inputSchema).toBeTruthy();
expect(ts.outputSchema).toBeTruthy();
// handlerSource is a fallback (compiled JS), CLI overrides with real TypeScript
expect(ts.handlerSource).toContain('target');
// Source string fields are null — CLI patches with original TypeScript
expect(ts.inputSchemaSource).toBeNull();
expect(ts.outputSchemaSource).toBeNull();
expect(ts.suspendSchemaSource).toBeNull();
expect(ts.resumeSchemaSource).toBeNull();
expect(ts.toMessageSource).toBeNull();
expect(ts.requireApproval).toBe(false);
expect(ts.needsApprovalFnSource).toBeNull();
expect(ts.providerOptions).toBeNull();
});
// --- Provider tool ---
it('describes a provider tool in providerTools array', () => {
const providerTool: BuiltProviderTool = {
name: 'anthropic.web_search_20250305',
args: { maxResults: 5 },
};
const agent = new Agent('test-agent').providerTool(providerTool);
const schema = agent.describe();
// Provider tools are now in a separate array
expect(schema.tools).toHaveLength(0);
expect(schema.providerTools).toHaveLength(1);
expect(schema.providerTools[0].name).toBe('anthropic.web_search_20250305');
expect(schema.providerTools[0].source).toBe('');
});
// --- MCP servers ---
it('describes MCP servers in mcp field', () => {
const client = new McpClient([
{ name: 'browser', url: 'http://localhost:9222/mcp', transport: 'streamableHttp' },
{ name: 'fs', command: 'echo', args: ['test'] },
]);
const agent = new Agent('test-agent').mcp(client);
const schema = agent.describe();
// MCP servers are now in a separate mcp field
expect(schema.tools).toHaveLength(0);
expect(schema.mcp).toHaveLength(2);
expect(schema.mcp![0].name).toBe('browser');
expect(schema.mcp![0].configSource).toBe('');
expect(schema.mcp![1].name).toBe('fs');
expect(schema.mcp![1].configSource).toBe('');
});
it('returns null mcp when no clients are configured', () => {
const agent = new Agent('test-agent');
const schema = agent.describe();
expect(schema.mcp).toBeNull();
});
// --- Guardrails ---
it('describes input and output guardrails', () => {
const inputGuard: BuiltGuardrail = {
name: 'pii-filter',
guardType: 'pii',
strategy: 'redact',
_config: { types: ['email', 'phone'] },
};
const outputGuard: BuiltGuardrail = {
name: 'moderation-check',
guardType: 'moderation',
strategy: 'block',
_config: {},
};
const agent = new Agent('test-agent').inputGuardrail(inputGuard).outputGuardrail(outputGuard);
const schema = agent.describe();
expect(schema.guardrails).toHaveLength(2);
expect(schema.guardrails[0]).toEqual({
name: 'pii-filter',
guardType: 'pii',
strategy: 'redact',
position: 'input',
config: { types: ['email', 'phone'] },
source: '',
});
expect(schema.guardrails[1]).toEqual({
name: 'moderation-check',
guardType: 'moderation',
strategy: 'block',
position: 'output',
config: {},
source: '',
});
});
// --- Telemetry ---
it('returns telemetry schema when telemetry builder is set', () => {
const agent = new Agent('test-agent').telemetry(new Telemetry());
const schema = agent.describe();
expect(schema.telemetry).toEqual({ source: '' });
});
it('returns null telemetry when not configured', () => {
const agent = new Agent('test-agent');
const schema = agent.describe();
expect(schema.telemetry).toBeNull();
});
// --- Checkpoint ---
it('returns memory checkpoint when checkpoint is memory', () => {
const agent = new Agent('test-agent').checkpoint('memory');
const schema = agent.describe();
expect(schema.checkpoint).toBe('memory');
});
it('returns null checkpoint when not configured', () => {
const agent = new Agent('test-agent');
const schema = agent.describe();
expect(schema.checkpoint).toBeNull();
});
// --- Memory ---
it('describes memory configuration', () => {
const agent = new Agent('test-agent').memory({
memory: makeMockMemory(),
lastMessages: 20,
semanticRecall: {
topK: 5,
messageRange: { before: 2, after: 2 },
embedder: 'openai/text-embedding-3-small',
},
workingMemory: {
template: 'Current state: {{state}}',
structured: false,
scope: 'resource' as const,
},
});
const schema = agent.describe();
expect(schema.memory).toBeTruthy();
expect(schema.memory!.source).toBeNull();
expect(schema.memory!.lastMessages).toBe(20);
expect(schema.memory!.semanticRecall).toEqual({
topK: 5,
messageRange: { before: 2, after: 2 },
embedder: 'openai/text-embedding-3-small',
});
expect(schema.memory!.workingMemory).toEqual({
type: 'freeform',
template: 'Current state: {{state}}',
});
});
it('describes structured working memory', () => {
const agent = new Agent('test-agent').memory({
memory: makeMockMemory(),
lastMessages: 10,
workingMemory: {
template: '',
structured: true,
schema: z.object({ notes: z.string() }),
scope: 'resource' as const,
},
});
const schema = agent.describe();
expect(schema.memory!.workingMemory!.type).toBe('structured');
expect(schema.memory!.workingMemory!.schema).toBeTruthy();
});
// --- Evaluations ---
it('describes evaluations with evalType, modelId, and handlerSource', () => {
const checkEval: BuiltEval = {
name: 'has-greeting',
description: 'Checks for greeting',
evalType: 'check',
modelId: null,
credentialName: null,
_run: jest.fn(),
};
const judgeEval: BuiltEval = {
name: 'quality-judge',
description: undefined,
evalType: 'judge',
modelId: 'anthropic/claude-haiku-4-5',
credentialName: 'anthropic-key',
_run: jest.fn(),
};
const agent = new Agent('test-agent').eval(checkEval).eval(judgeEval);
const schema = agent.describe();
expect(schema.evaluations).toHaveLength(2);
expect(schema.evaluations[0]).toEqual({
name: 'has-greeting',
description: 'Checks for greeting',
type: 'check',
modelId: null,
hasCredential: false,
credentialName: null,
handlerSource: null,
});
expect(schema.evaluations[1]).toEqual({
name: 'quality-judge',
description: null,
type: 'judge',
modelId: 'anthropic/claude-haiku-4-5',
hasCredential: true,
credentialName: 'anthropic-key',
handlerSource: null,
});
});
// --- Thinking config ---
it('describes anthropic thinking config', () => {
const agent = new Agent('test-agent')
.model('anthropic', 'claude-sonnet-4-5')
.thinking('anthropic', { budgetTokens: 10000 });
const schema = agent.describe();
expect(schema.config.thinking).toEqual({
provider: 'anthropic',
budgetTokens: 10000,
});
});
it('describes openai thinking config', () => {
const agent = new Agent('test-agent')
.model('openai', 'o3-mini')
.thinking('openai', { reasoningEffort: 'high' });
const schema = agent.describe();
expect(schema.config.thinking).toEqual({
provider: 'openai',
reasoningEffort: 'high',
});
});
// --- requireToolApproval ---
it('reflects requireToolApproval flag', () => {
const agent = new Agent('test-agent').requireToolApproval();
const schema = agent.describe();
expect(schema.config.requireToolApproval).toBe(true);
});
// --- toolCallConcurrency ---
it('reflects toolCallConcurrency', () => {
const agent = new Agent('test-agent').toolCallConcurrency(5);
const schema = agent.describe();
expect(schema.config.toolCallConcurrency).toBe(5);
});
// --- Structured output ---
it('describes structured output with schemaSource null', () => {
const outputSchema = z.object({ code: z.string(), explanation: z.string() });
const agent = new Agent('test-agent').structuredOutput(outputSchema);
const schema = agent.describe();
expect(schema.config.structuredOutput.enabled).toBe(true);
expect(schema.config.structuredOutput.schemaSource).toBeNull();
});
});

View File

@ -0,0 +1,150 @@
import { z } from 'zod';
import { Tool } from '../sdk/tool';
import type { AgentMessage } from '../types/sdk/message';
import type { InterruptibleToolContext } from '../types/sdk/tool';
// ---------------------------------------------------------------------------
// Tool.describe() tests
// ---------------------------------------------------------------------------
describe('Tool.describe()', () => {
it('returns a ToolDescriptor with name, description, and inputSchema', () => {
const tool = new Tool('search')
.description('Search the web')
.input(z.object({ query: z.string() }))
.handler(async ({ query }) => await Promise.resolve({ result: query }));
const descriptor = tool.describe();
expect(descriptor.name).toBe('search');
expect(descriptor.description).toBe('Search the web');
expect(descriptor.inputSchema).toBeDefined();
expect(descriptor.outputSchema).toBeNull();
expect(descriptor.hasSuspend).toBe(false);
expect(descriptor.hasResume).toBe(false);
expect(descriptor.hasToMessage).toBe(false);
expect(descriptor.requireApproval).toBe(false);
expect(descriptor.providerOptions).toBeNull();
expect(descriptor.systemInstruction).toBeNull();
});
it('persists systemInstruction through describe() so it survives JSON-config round-trip', () => {
const directive = 'Always cite a URL when summarising web search results.';
const tool = new Tool('search')
.description('Search the web')
.systemInstruction(directive)
.input(z.object({ query: z.string() }))
.handler(async ({ query }) => await Promise.resolve({ result: query }));
const descriptor = tool.describe();
expect(descriptor.systemInstruction).toBe(directive);
});
it('sets hasSuspend/hasResume when suspend/resume are defined', () => {
const tool = new Tool('approve')
.description('Approve an action')
.input(z.object({ action: z.string() }))
.suspend(z.object({ message: z.string() }))
.resume(z.object({ approved: z.boolean() }))
.handler(async (input, ctx) => {
const interruptCtx = ctx as InterruptibleToolContext;
if (!interruptCtx.resumeData) {
return await interruptCtx.suspend({ message: `Approve: ${input.action}?` });
}
return {
result: (interruptCtx.resumeData as { approved: boolean }).approved
? 'approved'
: 'denied',
};
});
const descriptor = tool.describe();
expect(descriptor.hasSuspend).toBe(true);
expect(descriptor.hasResume).toBe(true);
});
it('sets hasToMessage when toMessage is defined', () => {
const tool = new Tool('get_data')
.description('Get data')
.input(z.object({ id: z.string() }))
.output(z.object({ value: z.string() }))
.toMessage(
(output) =>
({
type: 'custom',
data: {
components: [{ type: 'section', text: output.value }],
},
}) as unknown as AgentMessage,
)
.handler(async ({ id }) => await Promise.resolve({ value: id }));
const descriptor = tool.describe();
expect(descriptor.hasToMessage).toBe(true);
});
it('sets requireApproval when .requireApproval() is called', () => {
const tool = new Tool('delete')
.description('Delete a record')
.input(z.object({ id: z.string() }))
.requireApproval()
.handler(async ({ id }) => await Promise.resolve({ deleted: id }));
const descriptor = tool.describe();
expect(descriptor.requireApproval).toBe(true);
});
it('sets outputSchema when output schema is defined', () => {
const tool = new Tool('compute')
.description('Compute something')
.input(z.object({ value: z.number() }))
.output(z.object({ result: z.number() }))
.handler(async ({ value }) => await Promise.resolve({ result: value * 2 }));
const descriptor = tool.describe();
expect(descriptor.outputSchema).toBeDefined();
expect(descriptor.outputSchema).not.toBeNull();
});
it('throws if name is missing', () => {
const tool = new Tool('');
expect(() => tool.describe()).toThrow('Tool name is required');
});
it('throws if description is missing', () => {
const tool = new Tool('no-desc')
.input(z.object({ x: z.string() }))
.handler(async ({ x }) => await Promise.resolve({ x }));
expect(() => tool.describe()).toThrow('"no-desc" requires a description');
});
it('throws if input schema is missing', () => {
const tool = new Tool('no-input').description('No input');
expect(() => tool.describe()).toThrow('"no-input" requires an input schema');
});
it('inputSchema conforms to JSON Schema format', () => {
const tool = new Tool('typed')
.description('Typed tool')
.input(
z.object({
name: z.string().describe('The name'),
count: z.number().int().min(1),
}),
)
.handler(async ({ name, count }) => await Promise.resolve({ name, count }));
const descriptor = tool.describe();
expect(descriptor.inputSchema).toBeDefined();
const schema = descriptor.inputSchema as Record<string, unknown>;
expect(schema.type).toBe('object');
expect(schema.properties).toBeDefined();
});
});

View File

@ -1,606 +0,0 @@
import { z } from 'zod';
import { Agent } from '../sdk/agent';
import { isSuspendResult } from '../sdk/from-schema';
import type { HandlerExecutor } from '../types/sdk/handler-executor';
import type { AgentSchema, ToolSchema } from '../types/sdk/schema';
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function mockExecutor(): HandlerExecutor {
return {
executeTool: jest.fn().mockResolvedValue({ result: 'mocked' }),
executeToMessage: jest.fn().mockResolvedValue(undefined),
executeEval: jest.fn().mockResolvedValue({ score: 1 }),
evaluateSchema: jest.fn().mockResolvedValue(undefined),
evaluateExpression: jest.fn().mockResolvedValue(undefined),
};
}
function minimalSchema(overrides: Partial<AgentSchema> = {}): AgentSchema {
return {
model: { provider: 'anthropic', name: 'claude-sonnet-4-5' },
credential: 'my-credential',
instructions: 'You are helpful.',
description: null,
tools: [],
providerTools: [],
memory: null,
evaluations: [],
guardrails: [],
mcp: null,
telemetry: null,
checkpoint: null,
config: {
structuredOutput: { enabled: false, schemaSource: null },
thinking: null,
toolCallConcurrency: null,
requireToolApproval: false,
},
...overrides,
};
}
function makeToolSchema(overrides: Partial<ToolSchema> = {}): ToolSchema {
return {
name: 'test-tool',
description: 'A test tool',
type: 'custom',
editable: true,
inputSchemaSource: null,
outputSchemaSource: null,
handlerSource: null,
suspendSchemaSource: null,
resumeSchemaSource: null,
toMessageSource: null,
requireApproval: false,
needsApprovalFnSource: null,
providerOptions: null,
inputSchema: { type: 'object', properties: { query: { type: 'string' } } },
outputSchema: null,
hasSuspend: false,
hasResume: false,
hasToMessage: false,
...overrides,
};
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('Agent.fromSchema()', () => {
it('reconstructs basic agent config', async () => {
const schema = minimalSchema();
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.model).toEqual({ provider: 'anthropic', name: 'claude-sonnet-4-5' });
expect(described.credential).toBe('my-credential');
expect(described.instructions).toBe('You are helpful.');
});
it('reconstructs model with only name (no provider)', async () => {
const schema = minimalSchema({
model: { provider: null, name: 'gpt-4o' },
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.model).toEqual({ provider: null, name: 'gpt-4o' });
});
it('reconstructs thinking config with correct provider arg', async () => {
const schema = minimalSchema({
config: {
structuredOutput: { enabled: false, schemaSource: null },
thinking: { provider: 'anthropic', budgetTokens: 10000 },
toolCallConcurrency: null,
requireToolApproval: false,
},
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.config.thinking).toEqual({
provider: 'anthropic',
budgetTokens: 10000,
});
});
it('reconstructs openai thinking config', async () => {
const schema = minimalSchema({
model: { provider: 'openai', name: 'o3-mini' },
config: {
structuredOutput: { enabled: false, schemaSource: null },
thinking: { provider: 'openai', reasoningEffort: 'high' },
toolCallConcurrency: null,
requireToolApproval: false,
},
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.config.thinking).toEqual({
provider: 'openai',
reasoningEffort: 'high',
});
});
it('creates proxy handlers for custom tools', async () => {
const toolSchema = makeToolSchema({
name: 'search',
description: 'Search the web',
});
const schema = minimalSchema({ tools: [toolSchema] });
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.tools).toHaveLength(1);
expect(described.tools[0].name).toBe('search');
expect(described.tools[0].description).toBe('Search the web');
expect(described.tools[0].editable).toBe(true);
});
it('adds WorkflowTool markers for non-editable tools', async () => {
const toolSchema = makeToolSchema({ name: 'Send Email', type: 'workflow', editable: false });
const schema = minimalSchema({ tools: [toolSchema] });
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
// Non-editable tools become WorkflowTool markers in declaredTools
const markers = agent.declaredTools.filter(
(t) => '__workflowTool' in t && (t as Record<string, unknown>).__workflowTool === true,
);
expect(markers).toHaveLength(1);
expect(markers[0].name).toBe('Send Email');
});
it('reconstructs memory from schema fields', async () => {
const schema = minimalSchema({
memory: {
source: null,
storage: 'memory',
lastMessages: 20,
semanticRecall: null,
workingMemory: null,
},
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.memory).toBeTruthy();
expect(described.memory!.lastMessages).toBe(20);
expect(described.memory!.storage).toBe('memory');
});
it('sets toolCallConcurrency when specified', async () => {
const schema = minimalSchema({
config: {
structuredOutput: { enabled: false, schemaSource: null },
thinking: null,
toolCallConcurrency: 5,
requireToolApproval: false,
},
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.config.toolCallConcurrency).toBe(5);
});
it('sets requireToolApproval when true', async () => {
const schema = minimalSchema({
config: {
structuredOutput: { enabled: false, schemaSource: null },
thinking: null,
toolCallConcurrency: null,
requireToolApproval: true,
},
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.config.requireToolApproval).toBe(true);
});
it('sets checkpoint when specified', async () => {
const schema = minimalSchema({ checkpoint: 'memory' });
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.checkpoint).toBe('memory');
});
it('delegates tool execution to handlerExecutor', async () => {
const executor = mockExecutor();
const toolSchema = makeToolSchema({ name: 'my-tool' });
const schema = minimalSchema({ tools: [toolSchema] });
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
// Access the built tool's handler via declaredTools
const tools = agent.declaredTools;
expect(tools).toHaveLength(1);
const result = await tools[0].handler!({ query: 'test' }, { parentTelemetry: undefined });
expect(executor.executeTool).toHaveBeenCalledWith(
'my-tool',
{ query: 'test' },
{ parentTelemetry: undefined },
);
expect(result).toEqual({ result: 'mocked' });
});
it('reconstructs guardrails with correct position', async () => {
const schema = minimalSchema({
guardrails: [
{
name: 'pii-guard',
guardType: 'pii',
strategy: 'redact',
position: 'input',
config: { detectionTypes: ['email', 'phone'] },
source: '',
},
{
name: 'mod-guard',
guardType: 'moderation',
strategy: 'block',
position: 'output',
config: {},
source: '',
},
],
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.guardrails).toHaveLength(2);
expect(described.guardrails[0].name).toBe('pii-guard');
expect(described.guardrails[0].position).toBe('input');
expect(described.guardrails[0].guardType).toBe('pii');
expect(described.guardrails[1].name).toBe('mod-guard');
expect(described.guardrails[1].position).toBe('output');
});
it('reconstructs evals with proxy _run', async () => {
const executor = mockExecutor();
const schema = minimalSchema({
evaluations: [
{
name: 'accuracy',
description: 'Check accuracy',
type: 'check',
modelId: null,
credentialName: null,
hasCredential: false,
handlerSource: null,
},
{
name: 'quality',
description: 'Judge quality',
type: 'judge',
modelId: 'anthropic/claude-sonnet-4-5',
credentialName: 'anthropic',
hasCredential: true,
handlerSource: null,
},
],
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const described = agent.describe();
expect(described.evaluations).toHaveLength(2);
expect(described.evaluations[0].name).toBe('accuracy');
expect(described.evaluations[0].type).toBe('check');
expect(described.evaluations[1].name).toBe('quality');
expect(described.evaluations[1].type).toBe('judge');
});
it('reconstructs provider tools', async () => {
const schema = minimalSchema({
providerTools: [{ name: 'anthropic.web_search_20250305', source: '' }],
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
const described = agent.describe();
expect(described.providerTools).toHaveLength(1);
expect(described.providerTools[0].name).toBe('anthropic.web_search_20250305');
});
it('evaluates provider tool source via evaluateExpression', async () => {
const executor = mockExecutor();
(executor.evaluateExpression as jest.Mock).mockResolvedValue({
name: 'anthropic.web_search_20250305',
args: { maxUses: 5 },
});
const schema = minimalSchema({
providerTools: [
{
name: 'anthropic.web_search_20250305',
source: 'providerTools.anthropicWebSearch({ maxUses: 5 })',
},
],
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const described = agent.describe();
expect(executor.evaluateExpression).toHaveBeenCalledWith(
'providerTools.anthropicWebSearch({ maxUses: 5 })',
);
expect(described.providerTools).toHaveLength(1);
expect(described.providerTools[0].name).toBe('anthropic.web_search_20250305');
});
it('evaluates structuredOutput schema via evaluateSchema', async () => {
const zodSchema = z.object({ answer: z.string() });
const executor = mockExecutor();
(executor.evaluateSchema as jest.Mock).mockResolvedValue(zodSchema);
const schema = minimalSchema({
config: {
structuredOutput: { enabled: true, schemaSource: 'z.object({ answer: z.string() })' },
thinking: null,
toolCallConcurrency: null,
requireToolApproval: false,
},
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const described = agent.describe();
expect(executor.evaluateSchema).toHaveBeenCalledWith('z.object({ answer: z.string() })');
expect(described.config.structuredOutput.enabled).toBe(true);
});
it('handles suspend result detection via isSuspendResult', () => {
const suspendMarker = Symbol.for('n8n.agent.suspend');
const suspendResult = { [suspendMarker]: true, payload: { message: 'approve?' } };
const nonSuspend = { result: 42 };
expect(isSuspendResult(suspendResult)).toBe(true);
expect(isSuspendResult(nonSuspend)).toBe(false);
expect(isSuspendResult(null)).toBe(false);
expect(isSuspendResult(undefined)).toBe(false);
});
it('delegates interruptible tool execution with suspend detection', async () => {
const suspendMarker = Symbol.for('n8n.agent.suspend');
const executor = {
...mockExecutor(),
executeTool: jest.fn().mockResolvedValue({
[suspendMarker]: true,
payload: { message: 'Please approve' },
}),
};
const toolSchema = makeToolSchema({
name: 'suspend-tool',
hasSuspend: true,
});
const schema = minimalSchema({ tools: [toolSchema] });
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const tools = agent.declaredTools;
expect(tools).toHaveLength(1);
// Call with an interruptible context
let suspendedPayload: unknown;
const ctx = {
parentTelemetry: undefined,
resumeData: undefined,
// eslint-disable-next-line @typescript-eslint/require-await
suspend: jest.fn().mockImplementation(async (payload: unknown) => {
suspendedPayload = payload;
return { suspended: true };
}),
};
await tools[0].handler!({ query: 'test' }, ctx);
expect(ctx.suspend).toHaveBeenCalledWith({ message: 'Please approve' });
expect(suspendedPayload).toEqual({ message: 'Please approve' });
});
it('reconstructs requireApproval on individual tools', async () => {
const toolSchema = makeToolSchema({
name: 'danger-tool',
requireApproval: true,
});
const schema = minimalSchema({
tools: [toolSchema],
checkpoint: 'memory',
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: mockExecutor(),
});
// The tool should be wrapped for approval, which adds suspendSchema
const tools = agent.declaredTools;
expect(tools).toHaveLength(1);
expect(tools[0].suspendSchema).toBeDefined();
});
it('reconstructs MCP servers by evaluating configSource', async () => {
const executor = mockExecutor();
(executor.evaluateExpression as jest.Mock).mockResolvedValue({
name: 'browser',
url: 'http://localhost:9222/mcp',
transport: 'streamableHttp',
});
const schema = minimalSchema({
mcp: [
{
name: 'browser',
configSource:
'({ name: "browser", url: "http://localhost:9222/mcp", transport: "streamableHttp" })',
},
],
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
expect(executor.evaluateExpression).toHaveBeenCalledWith(
'({ name: "browser", url: "http://localhost:9222/mcp", transport: "streamableHttp" })',
);
const described = agent.describe();
expect(described.mcp).toHaveLength(1);
expect(described.mcp![0].name).toBe('browser');
});
it('reconstructs multiple MCP servers', async () => {
const executor = mockExecutor();
(executor.evaluateExpression as jest.Mock)
.mockResolvedValueOnce({
name: 'browser',
url: 'http://localhost:9222/mcp',
transport: 'streamableHttp',
})
.mockResolvedValueOnce({
name: 'fs',
command: 'npx',
args: ['@anthropic/mcp-fs', '/tmp'],
});
const schema = minimalSchema({
mcp: [
{ name: 'browser', configSource: 'browserConfig' },
{ name: 'fs', configSource: 'fsConfig' },
],
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const described = agent.describe();
expect(described.mcp).toHaveLength(2);
expect(described.mcp![0].name).toBe('browser');
expect(described.mcp![1].name).toBe('fs');
});
it('skips MCP servers with empty configSource', async () => {
const schema = minimalSchema({
mcp: [{ name: 'browser', configSource: '' }],
});
const executor = mockExecutor();
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
expect(executor.evaluateExpression).not.toHaveBeenCalled();
// No MCP configs evaluated means no client is added
const described = agent.describe();
expect(described.mcp).toBeNull();
});
it('reconstructs telemetry by evaluating source', async () => {
const executor = mockExecutor();
(executor.evaluateExpression as jest.Mock).mockResolvedValue({
enabled: true,
functionId: 'my-agent',
recordInputs: true,
recordOutputs: true,
integrations: [],
});
const schema = minimalSchema({
telemetry: { source: 'new Telemetry().functionId("my-agent").build()' },
});
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
expect(executor.evaluateExpression).toHaveBeenCalledWith(
'new Telemetry().functionId("my-agent").build()',
);
const described = agent.describe();
expect(described.telemetry).not.toBeNull();
});
it('does not set telemetry when schema has no telemetry', async () => {
const schema = minimalSchema({ telemetry: null });
const executor = mockExecutor();
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const described = agent.describe();
expect(described.telemetry).toBeNull();
expect(executor.evaluateExpression).not.toHaveBeenCalled();
});
it('evaluates suspend/resume schemas via evaluateSchema', async () => {
const suspendSchema = z.object({ reason: z.string() });
const resumeSchema = z.object({ approved: z.boolean() });
const executor = mockExecutor();
(executor.evaluateSchema as jest.Mock)
.mockResolvedValueOnce(suspendSchema)
.mockResolvedValueOnce(resumeSchema);
const toolSchema = makeToolSchema({
name: 'interruptible-tool',
hasSuspend: true,
hasResume: true,
suspendSchemaSource: 'z.object({ reason: z.string() })',
resumeSchemaSource: 'z.object({ approved: z.boolean() })',
});
const schema = minimalSchema({ tools: [toolSchema] });
const agent = await Agent.fromSchema(schema, 'test-agent', {
handlerExecutor: executor,
});
const tools = agent.declaredTools;
expect(tools).toHaveLength(1);
expect(tools[0].suspendSchema).toBe(suspendSchema);
expect(tools[0].resumeSchema).toBe(resumeSchema);
});
});

View File

@ -1,5 +1,5 @@
import { InMemoryMemory } from '../runtime/memory-store';
import type { AgentDbMessage } from '../types/sdk/message';
import type { AgentDbMessage, Message } from '../types/sdk/message';
describe('InMemoryMemory working memory', () => {
it('returns null for unknown key', async () => {
@ -117,3 +117,59 @@ describe('InMemoryMemory — message createdAt', () => {
expect(loaded[1].createdAt.getTime()).toBe(t2.getTime());
});
});
// ---------------------------------------------------------------------------
// Upsert contract
// ---------------------------------------------------------------------------
describe('InMemoryMemory — saveMessages upsert by id', () => {
it('upserts by id (no duplicate rows after a re-save)', async () => {
const mem = new InMemoryMemory();
const t1 = new Date('2020-01-01T00:00:01.000Z');
await mem.saveMessages({
threadId: 't1',
messages: [makeDbMsg('msg-1', t1, 'original')],
});
const updated = { ...makeDbMsg('msg-1', t1, 'updated content') };
await mem.saveMessages({ threadId: 't1', messages: [updated] });
const result = await mem.getMessages('t1');
expect(result).toHaveLength(1);
expect(((result[0] as Message).content[0] as { type: string; text: string }).text).toBe(
'updated content',
);
});
it('preserves insertion order on upsert', async () => {
const mem = new InMemoryMemory();
const t1 = new Date('2020-01-01T00:00:01.000Z');
const t2 = new Date('2020-01-01T00:00:02.000Z');
const t3 = new Date('2020-01-01T00:00:03.000Z');
await mem.saveMessages({
threadId: 't1',
messages: [
makeDbMsg('m1', t1, 'first'),
makeDbMsg('m2', t2, 'second'),
makeDbMsg('m3', t3, 'third'),
],
});
// Update m2 in place
await mem.saveMessages({
threadId: 't1',
messages: [makeDbMsg('m2', t2, 'second-updated')],
});
const result = await mem.getMessages('t1');
expect(result).toHaveLength(3);
// Original order preserved
expect(result[0].id).toBe('m1');
expect(result[1].id).toBe('m2');
expect(result[2].id).toBe('m3');
// Updated content
expect(((result[1] as Message).content[0] as { text: string }).text).toBe('second-updated');
});
});

View File

@ -0,0 +1,327 @@
/**
* Round-trip conversion tests: toAiMessages fromAiMessages
*
* These tests exercise the message split/merge logic without making real LLM
* calls. They lock down the structural invariants that the agent runtime relies
* on, including the key interim-message ordering guarantee described in the
* plan:
*
* input: [assistant{tool-call resolved}, user{x}, assistant{y}]
* output: [assistant{tool-call}, tool{tool-result}, user{x}, assistant{y}]
*
* The tool-result is inserted right after its tool-call, regardless of what
* messages follow it in the n8n list.
*/
import { describe, it, expect } from 'vitest';
import { toAiMessages, fromAiMessages } from '../../runtime/messages';
import type { Message } from '../../types/sdk/message';
describe('toAiMessages + fromAiMessages — round-trip', () => {
it('splits a resolved tool-call into assistant + tool ModelMessages', () => {
const input: Message[] = [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'add',
input: { a: 1, b: 2 },
state: 'resolved',
output: { result: 3 },
},
],
},
];
const aiMessages = toAiMessages(input);
expect(aiMessages).toHaveLength(2);
expect(aiMessages[0].role).toBe('assistant');
expect(aiMessages[1].role).toBe('tool');
const toolCallPart = (
aiMessages[0] as { role: string; content: Array<{ type: string; toolCallId: string }> }
).content[0];
expect(toolCallPart.type).toBe('tool-call');
expect(toolCallPart.toolCallId).toBe('tc-1');
const toolResultPart = (
aiMessages[1] as {
role: string;
content: Array<{
type: string;
toolCallId: string;
output: { type: string; value: unknown };
}>;
}
).content[0];
expect(toolResultPart.type).toBe('tool-result');
expect(toolResultPart.toolCallId).toBe('tc-1');
expect(toolResultPart.output.type).toBe('json');
expect(toolResultPart.output.value).toEqual({ result: 3 });
});
it('encodes rejected tool-call as error-text in the tool ModelMessage', () => {
const input: Message[] = [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'do_it',
input: {},
state: 'rejected',
error: 'Error: something went wrong',
},
],
},
];
const aiMessages = toAiMessages(input);
expect(aiMessages).toHaveLength(2);
const toolResultPart = (
aiMessages[1] as { role: string; content: Array<{ output: { type: string; value: string } }> }
).content[0];
expect(toolResultPart.output.type).toBe('error-text');
expect(toolResultPart.output.value).toBe('Error: something went wrong');
});
it('drops pending tool-call blocks from both assistant and tool ModelMessages', () => {
const input: Message[] = [
{
role: 'assistant',
content: [
{ type: 'text', text: 'Thinking...' },
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'do_it',
input: {},
state: 'pending',
},
],
},
];
const aiMessages = toAiMessages(input);
// Only the assistant text part remains; no tool-result emitted for pending
expect(aiMessages).toHaveLength(1);
expect(aiMessages[0].role).toBe('assistant');
const content = (aiMessages[0] as { role: string; content: Array<{ type: string }> }).content;
expect(content).toHaveLength(1);
expect(content[0].type).toBe('text');
});
it('emits nothing for an assistant message whose only blocks are all pending', () => {
const input: Message[] = [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'do_it',
input: {},
state: 'pending',
},
{
type: 'tool-call',
toolCallId: 'tc-2',
toolName: 'do_more',
input: {},
state: 'pending',
},
],
},
];
const aiMessages = toAiMessages(input);
// No empty-content assistant message — the whole message is suppressed
expect(aiMessages).toHaveLength(0);
});
it('skips legacy tool-call blocks that have no state field and emits nothing when they are the only content', () => {
const input: Message[] = [
{
role: 'assistant',
content: [
// Simulate a DB row written before the state field was introduced
{
type: 'tool-call',
toolCallId: 'tc-legacy',
toolName: 'old_tool',
input: {},
} as unknown as Message['content'][number],
],
},
];
const aiMessages = toAiMessages(input);
// No empty-content assistant message and no spurious error-json tool message
expect(aiMessages).toHaveLength(0);
});
it('emits one tool ModelMessage per settled block in the same assistant turn', () => {
const input: Message[] = [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'add',
input: { a: 1, b: 2 },
state: 'resolved',
output: { result: 3 },
},
{
type: 'tool-call',
toolCallId: 'tc-2',
toolName: 'mul',
input: { a: 4, b: 5 },
state: 'resolved',
output: { result: 20 },
},
],
},
];
const aiMessages = toAiMessages(input);
// assistant{tc-1, tc-2} + tool{tc-1} + tool{tc-2}
expect(aiMessages).toHaveLength(3);
expect(aiMessages[0].role).toBe('assistant');
const assistantContent = (
aiMessages[0] as { content: Array<{ type: string; toolCallId: string }> }
).content;
expect(assistantContent).toHaveLength(2);
expect(assistantContent[0].toolCallId).toBe('tc-1');
expect(assistantContent[1].toolCallId).toBe('tc-2');
expect(aiMessages[1].role).toBe('tool');
expect(aiMessages[2].role).toBe('tool');
});
it('merges role:tool ModelMessages into the preceding assistant tool-call block', () => {
// Simulate AI SDK output: [assistant{tool-call}, tool{tool-result}]
const aiMessages = [
{
role: 'assistant' as const,
content: [
{
type: 'tool-call' as const,
toolCallId: 'tc-1',
toolName: 'add',
input: { a: 1, b: 2 },
providerExecuted: undefined,
},
],
},
{
role: 'tool' as const,
content: [
{
type: 'tool-result' as const,
toolCallId: 'tc-1',
toolName: 'add',
output: { type: 'json' as const, value: { result: 3 } },
},
],
},
];
const n8nMessages = fromAiMessages(aiMessages);
// Should produce a single assistant message with the resolved block
expect(n8nMessages).toHaveLength(1);
expect((n8nMessages[0] as Message).role).toBe('assistant');
const block = (n8nMessages[0] as Message).content[0];
expect(block.type).toBe('tool-call');
expect((block as { state: string }).state).toBe('resolved');
expect((block as { output: unknown }).output).toEqual({ result: 3 });
});
it('round-trip is structurally equivalent for a resolved tool-call', () => {
const original: Message[] = [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'echo',
input: { text: 'hello' },
state: 'resolved',
output: { echoed: 'hello' },
},
],
},
];
const aiMessages = toAiMessages(original);
const roundTripped = fromAiMessages(aiMessages);
expect(roundTripped).toHaveLength(1);
expect((roundTripped[0] as Message).role).toBe('assistant');
const block = (roundTripped[0] as Message).content[0];
expect(block.type).toBe('tool-call');
expect((block as { state: string }).state).toBe('resolved');
expect((block as { output: unknown }).output).toEqual({ echoed: 'hello' });
expect((block as { toolCallId: string }).toolCallId).toBe('tc-1');
});
it('interim-message ordering: tool-result is inserted right after its tool-call', () => {
// This is the key regression test for the interim-message scenario.
// Input n8n list: [assistant{tool-call resolved}, user{x}, assistant{y}]
// Expected AI SDK output: [assistant{tc}, tool{tr}, user{x}, assistant{y}]
const input: Message[] = [
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-1',
toolName: 'delete_file',
input: { path: 'foo.txt' },
state: 'resolved',
output: { deleted: true },
},
],
},
{
role: 'user',
content: [{ type: 'text', text: 'Actually, what is 2+2?' }],
},
{
role: 'assistant',
content: [{ type: 'text', text: 'It is 4.' }],
},
];
const aiMessages = toAiMessages(input);
// 4 messages: assistant{tool-call}, tool{tool-result}, user, assistant
expect(aiMessages).toHaveLength(4);
expect(aiMessages[0].role).toBe('assistant');
expect(aiMessages[1].role).toBe('tool');
expect(aiMessages[2].role).toBe('user');
expect(aiMessages[3].role).toBe('assistant');
// tool-result is immediately after the assistant tool-call message
const toolResultContent = (aiMessages[1] as { content: Array<{ toolCallId: string }> })
.content[0];
expect(toolResultContent.toolCallId).toBe('tc-1');
// user interim message is after the tool-result
const userContent = (aiMessages[2] as { content: Array<{ type: string; text: string }> })
.content[0];
expect(userContent.text).toBe('Actually, what is 2+2?');
});
});

View File

@ -106,7 +106,7 @@ describe('batched tool execution integration', () => {
const resumedStream = await agent.resume(
'stream',
{ approved: true },
{ runId: next.runId!, toolCallId: next.toolCallId! },
{ runId: next.runId, toolCallId: next.toolCallId },
);
const resumedChunks = await collectStreamChunks(resumedStream.stream);

View File

@ -8,7 +8,7 @@ import {
createAgentWithConcurrentMixedTools,
collectTextDeltas,
} from './helpers';
import { isLlmMessage, type StreamChunk } from '../../index';
import type { StreamChunk } from '../../index';
const describe = describeIf('anthropic');
@ -120,7 +120,7 @@ describe('concurrent tool execution integration', () => {
const resumedStream = await agent.resume(
'stream',
{ approved: true },
{ runId: next.runId!, toolCallId: next.toolCallId! },
{ runId: next.runId, toolCallId: next.toolCallId },
);
const resumedChunks = await collectStreamChunks(resumedStream.stream);
@ -147,13 +147,8 @@ describe('concurrent tool execution integration', () => {
const chunks = await collectStreamChunks(fullStream);
// list_files should auto-execute — its result should appear as a message chunk
const toolResultChunks = chunks.filter(
(c) =>
c.type === 'message' &&
isLlmMessage(c.message) &&
c.message.content.some((p) => p.type === 'tool-result'),
);
// list_files should auto-execute — its result should appear as a discrete tool-result chunk
const toolResultChunks = chunksOfType(chunks, 'tool-result');
// delete_file should be suspended
const suspendedChunks = chunksOfType(chunks, 'tool-call-suspended');
@ -170,12 +165,7 @@ describe('concurrent tool execution integration', () => {
);
// list_files result should be present even though delete_file suspended
const listResult = toolResultChunks.find(
(c) =>
c.type === 'message' &&
isLlmMessage(c.message) &&
c.message.content.some((p) => p.type === 'tool-result' && p.toolName === 'list_files'),
);
const listResult = toolResultChunks.find((c) => c.toolName === 'list_files');
expect(listResult).toBeDefined();
}
});
@ -204,7 +194,7 @@ describe('concurrent tool execution integration', () => {
'content' in m
? m.content
.filter((c) => c.type === 'text')
.map((c) => ({ type: 'text-delta' as const, delta: c.text }))
.map((c) => ({ type: 'text-delta' as const, id: '', delta: c.text }))
: [],
),
);

View File

@ -175,42 +175,53 @@ describe('event system — stream', () => {
});
// ---------------------------------------------------------------------------
// result.getState()
// getState()
// ---------------------------------------------------------------------------
describe('result.getState()', () => {
it('generate() result reports success after a successful run', async () => {
describe('getState()', () => {
it('returns idle before first run', () => {
const agent = createSimpleAgent();
const result = await agent.generate('Say hello');
expect(result.getState().status).toBe('success');
const state = agent.getState();
expect(state.status).toBe('idle');
expect(state.messageList.messages).toHaveLength(0);
});
it('stream() result reports success after the stream is fully consumed', async () => {
it('returns success after a successful generate()', async () => {
const agent = createSimpleAgent();
const { stream, getState } = await agent.stream('Say hello');
await agent.generate('Say hello');
const state = agent.getState();
expect(state.status).toBe('success');
});
it('returns success after a completed stream()', async () => {
const agent = createSimpleAgent();
const { stream } = await agent.stream('Say hello');
await collectStreamChunks(stream);
expect(getState().status).toBe('success');
const state = agent.getState();
expect(state.status).toBe('success');
});
it('stream() getState() is running while the stream is being consumed', async () => {
it('state is running during the generate loop (observed via event)', async () => {
const agent = createSimpleAgent();
const { stream, getState } = await agent.stream('Say hello');
// State is running before the stream is consumed
expect(getState().status).toBe('running');
let stateWhileRunning: string | undefined;
agent.on(AgentEvent.TurnStart, () => {
stateWhileRunning = agent.getState().status;
});
await collectStreamChunks(stream);
await agent.generate('Say hello');
expect(getState().status).toBe('success');
expect(stateWhileRunning).toBe('running');
});
it('generate() result reflects resourceId and threadId from RunOptions', async () => {
it('reflects resourceId and threadId from RunOptions', async () => {
const agent = createSimpleAgent();
const result = await agent.generate('Say hello', {
await agent.generate('Say hello', {
persistence: { resourceId: 'user-123', threadId: 'thread-abc' },
});
expect(result.getState().persistence?.resourceId).toBe('user-123');
expect(result.getState().persistence?.threadId).toBe('thread-abc');
const state = agent.getState();
expect(state.persistence?.resourceId).toBe('user-123');
expect(state.persistence?.threadId).toBe('thread-abc');
});
});

View File

@ -7,7 +7,6 @@ import { z } from 'zod';
import {
Agent,
type ContentToolCall,
type ContentToolResult,
filterLlmMessages,
Tool,
type StreamChunk,
@ -404,10 +403,10 @@ export const findAllToolCalls = (messages: AgentMessage[]): ContentToolCall[] =>
.map((m) => m.content.filter((c) => c.type === 'tool-call'))
.flat();
};
export const findAllToolResults = (messages: AgentMessage[]): ContentToolResult[] => {
return filterLlmMessages(messages)
.filter((m) => m.content.find((c) => c.type === 'tool-result'))
.map((m) => m.content.find((c) => c.type === 'tool-result') as ContentToolResult);
export const findAllToolResults = (messages: AgentMessage[]): ContentToolCall[] => {
return filterLlmMessages(messages).flatMap((m) =>
m.content.filter((c): c is ContentToolCall => c.type === 'tool-call' && c.state !== 'pending'),
);
};
export const collectTextDeltas = (chunks: StreamChunk[]): string => {
return chunks

View File

@ -0,0 +1,214 @@
/**
* Regression test: interim user message while a tool-call is suspended.
*
* Old architecture bug: if a user sent a new message between a tool-call
* suspension and its eventual resume, the message list would contain:
*
* assistant{tool-call} user{interim} tool{tool-result}
*
* This order is invalid for AI SDK providers (tool-result must immediately
* follow its tool-call). The new architecture stores the result ON the
* tool-call block, so toAiMessages always emits:
*
* assistant{tool-call} tool{tool-result} user{interim} assistant{reply}
*
* The tool-result is always adjacent to its tool-call regardless of what n8n
* messages come after it in the list.
*
* This test drives the full scenario end-to-end and asserts that:
* 1. The final result has finishReason 'stop' (no provider error).
* 2. The tool-call block on the originating assistant message transitions to
* state 'resolved' with the expected output.
* 3. The interim user/assistant messages are still present in memory.
*/
import { afterEach, expect, it } from 'vitest';
import { z } from 'zod';
import { describeIf, createSqliteMemory, getModel } from './helpers';
import { Agent, filterLlmMessages, Memory, Tool } from '../../index';
import type { AgentDbMessage } from '../../index';
import type { ContentToolCall, Message } from '../../types/sdk/message';
const describe = describeIf('anthropic');
describe('interim user message during tool suspension', () => {
const cleanups: Array<() => void> = [];
afterEach(() => {
for (const fn of cleanups) fn();
cleanups.length = 0;
});
function buildInterruptibleAgent(mem: Memory): Agent {
const deleteTool = new Tool('delete_file')
.description('Delete a file at the given path')
.input(z.object({ path: z.string().describe('File path to delete') }))
.output(z.object({ deleted: z.boolean(), path: z.string() }))
.suspend(z.object({ message: z.string(), severity: z.string() }))
.resume(z.object({ approved: z.boolean() }))
.handler(async ({ path }, ctx) => {
if (!ctx.resumeData) {
return await ctx.suspend({ message: `Delete "${path}"?`, severity: 'destructive' });
}
if (!ctx.resumeData.approved) return { deleted: false, path };
return { deleted: true, path };
});
return new Agent('interim-test-agent')
.model(getModel('anthropic'))
.instructions(
'You are a file manager. When asked to delete a file, use the delete_file tool. Be concise.',
)
.tool(deleteTool)
.memory(mem)
.checkpoint('memory');
}
for (const method of ['generate', 'stream'] as const) {
it(`[${method}] interim message does not break provider message ordering`, async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
const threadId = `thread-interim-${method}`;
const resourceId = 'res-interim';
const persistence = { threadId, resourceId };
const mem = new Memory().storage(memory);
const agent = buildInterruptibleAgent(mem);
// ----------------------------------------------------------------
// Turn 1: trigger the tool suspension
// ----------------------------------------------------------------
const suspendResult = await agent.generate('Please delete /tmp/interim-test.txt', {
persistence,
});
expect(suspendResult.finishReason).toBe('tool-calls');
expect(suspendResult.pendingSuspend).toBeDefined();
const { runId, toolCallId } = suspendResult.pendingSuspend![0];
// ----------------------------------------------------------------
// Interim turn: send a new message while the tool is suspended.
// Build a fresh agent instance to simulate a separate request.
// ----------------------------------------------------------------
const interimAgent = new Agent('interim-agent')
.model(getModel('anthropic'))
.instructions('You are helpful. Answer questions concisely.')
.memory(mem);
const interimResult = await interimAgent.generate('What is 1 + 1?', { persistence });
expect(interimResult.finishReason).toBe('stop');
// ----------------------------------------------------------------
// Resume turn: approve the suspended tool call
// ----------------------------------------------------------------
let resumeFinishReason: string;
if (method === 'generate') {
const result = await agent.resume(
'generate',
{ approved: true },
{
runId,
toolCallId,
},
);
resumeFinishReason = result.finishReason ?? 'stop';
} else {
const { stream } = await agent.resume(
'stream',
{ approved: true },
{
runId,
toolCallId,
},
);
// Drain the stream
const reader = stream.getReader();
let finishReason = 'stop';
while (true) {
const { done, value } = await reader.read();
if (done) break;
if ((value as { type: string }).type === 'finish') {
finishReason = (value as { finishReason?: string }).finishReason ?? 'stop';
}
}
resumeFinishReason = finishReason;
}
// ----------------------------------------------------------------
// Assertions
// ----------------------------------------------------------------
// 1. No provider error — the ordering was valid
expect(resumeFinishReason).toBe('stop');
// 2. The originating assistant message's tool-call block is resolved
const allMessages = await memory.getMessages(threadId);
const llmMessages = filterLlmMessages(allMessages);
const ourBlock = llmMessages
.flatMap((m) => m.content.filter((c): c is ContentToolCall => c.type === 'tool-call'))
.find((b) => b.toolCallId === toolCallId);
expect(ourBlock).toBeDefined();
expect(ourBlock!.state).toBe('resolved');
// 3. The interim user/assistant exchange is present in memory
const userMessages = allMessages.filter(
(m): m is AgentDbMessage & Message => 'role' in m && m.role === 'user',
);
// Turn-1 user + interim user (at minimum)
expect(userMessages.length).toBeGreaterThanOrEqual(2);
});
}
it('preserves chronological ordering of messages in memory after resume', async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
const threadId = 'thread-interim-ordering';
const resourceId = 'res-ordering';
const persistence = { threadId, resourceId };
const mem = new Memory().storage(memory);
const agent = buildInterruptibleAgent(mem);
// Turn 1: suspend
const suspendResult = await agent.generate('Delete /tmp/order-test.txt', { persistence });
expect(suspendResult.finishReason).toBe('tool-calls');
const { runId, toolCallId } = suspendResult.pendingSuspend![0];
// Interim turn
const interimAgent = new Agent('interim-ordering')
.model(getModel('anthropic'))
.instructions('Answer concisely.')
.memory(mem);
await interimAgent.generate('Say hi', { persistence });
// Resume
const resumeResult = await agent.resume(
'generate',
{ approved: true },
{
runId,
toolCallId,
},
);
expect(resumeResult.finishReason).toBe('stop');
// The tool-call is resolved
const allMessages = await memory.getMessages(threadId);
const llmMessages = filterLlmMessages(allMessages);
const ourBlock = llmMessages
.flatMap((m) => m.content.filter((c): c is ContentToolCall => c.type === 'tool-call'))
.find((b) => b.toolCallId === toolCallId);
expect(ourBlock).toBeDefined();
expect(ourBlock!.state).toBe('resolved');
// Messages are in chronological order (createdAt ascending)
const timestamps = allMessages.map((m) => m.createdAt.getTime());
for (let i = 1; i < timestamps.length; i++) {
expect(timestamps[i]).toBeGreaterThanOrEqual(timestamps[i - 1]);
}
});
});

View File

@ -72,12 +72,12 @@ describe('JSON Schema validation — non-MCP tools with raw JSON Schema', () =>
// The handler should have been called with valid data
expect(handler).toHaveBeenCalledWith(expect.objectContaining({ age: 25 }), expect.anything());
// No tool-result should carry an error flag
// No tool-call block should have state 'rejected'
const allMessages = filterLlmMessages(result.messages);
const toolResults = allMessages.flatMap((m) =>
m.content.filter((c) => c.type === 'tool-result'),
const toolCallBlocks = allMessages.flatMap((m) =>
m.content.filter((c) => c.type === 'tool-call'),
);
expect(toolResults.every((r) => !r.isError)).toBe(true);
expect(toolCallBlocks.every((c) => (c as { state: string }).state !== 'rejected')).toBe(true);
});
it('allows the LLM to self-correct after receiving a JSON Schema validation error', async () => {
@ -105,12 +105,12 @@ describe('JSON Schema validation — non-MCP tools with raw JSON Schema', () =>
expect(result.finishReason).toBe('stop');
expect(result.error).toBeUndefined();
// There should be at least two tool-result messages: one error, one success
// There should be at least two tool-call messages: one rejected, one resolved
const allMessages = filterLlmMessages(result.messages);
const toolResultMessages = allMessages.filter((m) =>
m.content.some((c) => c.type === 'tool-result'),
const toolCallMessages = allMessages.filter((m) =>
m.content.some((c) => c.type === 'tool-call'),
);
expect(toolResultMessages.length).toBeGreaterThanOrEqual(2);
expect(toolCallMessages.length).toBeGreaterThanOrEqual(2);
// The successful handler call should have received a valid age
expect(callCount).toBeGreaterThanOrEqual(1);

View File

@ -17,7 +17,7 @@ import {
chunksOfType,
} from './helpers';
import { startSseServer, type TestServer } from './mcp-server-helpers';
import { Agent, McpClient, Tool, isLlmMessage } from '../../index';
import { Agent, McpClient, Tool } from '../../index';
// ---------------------------------------------------------------------------
// McpClient constructor validation — no MCP server required
@ -234,13 +234,10 @@ describe_llm('agent stream() with MCP tool', () => {
const { stream } = await agent.stream('Echo "stream works" using tools_echo.');
const chunks = await collectStreamChunks(stream);
const messageChunks = chunksOfType(chunks, 'message');
const messages = messageChunks.map((c) => c.message);
const hasToolCall = messages.some(
(m) => isLlmMessage(m) && m.content.some((c) => c.type === 'tool-call'),
);
expect(hasToolCall).toBe(true);
// Tool calls now ride their own discrete `tool-call` chunks rather than
// being wrapped in `message` envelopes.
const toolCallChunks = chunksOfType(chunks, 'tool-call');
expect(toolCallChunks.length).toBeGreaterThan(0);
await client.close();
});

View File

@ -8,7 +8,7 @@
import { expect, it, beforeEach } from 'vitest';
import { Agent, Memory, type AgentDbMessage } from '../../../index';
import type { BuiltMemory, Thread } from '../../../types/sdk/memory';
import type { BuiltMemory, MemoryDescriptor, Thread } from '../../../types/sdk/memory';
import { describeIf, findLastTextContent, getModel } from '../helpers';
const describe = describeIf('anthropic');
@ -17,6 +17,9 @@ const describe = describeIf('anthropic');
// Custom in-memory BuiltMemory implementation (simulates Redis, DynamoDB, etc.)
// ---------------------------------------------------------------------------
class CustomMapMemory implements BuiltMemory {
describe(): MemoryDescriptor {
throw new Error('Method not implemented.');
}
readonly threads = new Map<string, Thread>();
readonly messages = new Map<string, AgentDbMessage[]>();
readonly workingMemory = new Map<string, string>();

View File

@ -61,6 +61,18 @@ afterAll(async () => {
}
}, 30_000);
/**
* Create a PostgresMemory instance backed by the test container connection string.
* Uses a simple inline CredentialProvider that returns the raw URL.
*/
function makePostgresMemory(namespace: string): PostgresMemory {
return new PostgresMemory({
type: 'connection',
connection: { connectionType: 'url', connection: { url: connectionString } },
options: { namespace },
});
}
/** describe that requires Docker — tests are no-ops without it. */
function describeWithDocker(name: string, fn: () => void) {
describe(name, () => {
@ -74,7 +86,7 @@ function describeWithDocker(name: string, fn: () => void) {
describeWithDocker('PostgresMemory saveThread upsert', () => {
it('preserves existing title and metadata when not provided', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'upsert_test' });
const mem = makePostgresMemory('upsert_test');
await mem.saveThread({
id: 'upsert-t1',
@ -95,7 +107,7 @@ describeWithDocker('PostgresMemory saveThread upsert', () => {
});
it('overwrites title and metadata when explicitly provided', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'upsert_ow' });
const mem = makePostgresMemory('upsert_ow');
await mem.saveThread({
id: 'upsert-t2',
@ -121,7 +133,7 @@ describeWithDocker('PostgresMemory saveThread upsert', () => {
describeWithDocker('PostgresMemory unit tests', () => {
it('creates tables on first use and round-trips a thread', async () => {
const mem = new PostgresMemory({ connection: connectionString });
const mem = makePostgresMemory('default');
const thread = await mem.saveThread({
id: 'thread-1',
@ -141,7 +153,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('saves and retrieves messages with limit', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'msg_test' });
const mem = makePostgresMemory('msg_test');
await mem.saveThread({ id: 't1', resourceId: 'u1' });
@ -180,7 +192,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('saves and retrieves working memory keyed by resourceId', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'wm_test' });
const mem = makePostgresMemory('wm_test');
expect(
await mem.getWorkingMemory({ threadId: 'thread-1', resourceId: 'user-1', scope: 'resource' }),
@ -207,7 +219,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('saves and retrieves working memory keyed by threadId (no resourceId)', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'wm_thread_test' });
const mem = makePostgresMemory('wm_thread_test');
expect(
await mem.getWorkingMemory({ threadId: 'thread-1', resourceId: 'user-1', scope: 'thread' }),
@ -225,7 +237,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('isolates working memory by resourceId', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'wm_iso_test' });
const mem = makePostgresMemory('wm_iso_test');
await mem.saveWorkingMemory(
{ threadId: 'thread-a', resourceId: 'user-a', scope: 'resource' },
@ -247,7 +259,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('stores scope=resource when resourceId is provided', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'wm_scope_test' });
const mem = makePostgresMemory('wm_scope_test');
await mem.saveWorkingMemory(
{ threadId: 'thread-1', resourceId: 'res-1', scope: 'resource' },
@ -266,10 +278,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('stores scope=thread when only threadId is provided', async () => {
const mem = new PostgresMemory({
connection: connectionString,
namespace: 'wm_scope_thread_test',
});
const mem = makePostgresMemory('wm_scope_thread_test');
await mem.saveWorkingMemory(
{ threadId: 'thread-1', resourceId: 'user-1', scope: 'thread' },
@ -288,10 +297,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('does not mix resource-scoped and thread-scoped entries with the same key value', async () => {
const mem = new PostgresMemory({
connection: connectionString,
namespace: 'wm_scope_iso_test',
});
const mem = makePostgresMemory('wm_scope_iso_test');
const sharedKey = 'same-id';
await mem.saveWorkingMemory(
@ -318,7 +324,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('deletes thread and cascades to messages', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'del_test' });
const mem = makePostgresMemory('del_test');
await mem.saveThread({ id: 'del-t1', resourceId: 'u1' });
await mem.saveMessages({
@ -342,7 +348,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('stores and queries embeddings with pgvector', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'vec_test' });
const mem = makePostgresMemory('vec_test');
await mem.saveThread({ id: 'vec-t1', resourceId: 'u1' });
@ -375,7 +381,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('filters embeddings by resourceId with scope=resource (default)', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'vec_res' });
const mem = makePostgresMemory('vec_res');
await mem.saveEmbeddings({
threadId: 't1',
@ -410,7 +416,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('filters embeddings by threadId with scope=thread', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'vec_thr' });
const mem = makePostgresMemory('vec_thr');
await mem.saveEmbeddings({
threadId: 't1',
@ -443,7 +449,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('resource scope excludes embeddings from other resources', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'vec_iso' });
const mem = makePostgresMemory('vec_iso');
await mem.saveEmbeddings({
threadId: 't1',
@ -470,7 +476,7 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('stores resourceId in the embeddings table', async () => {
const mem = new PostgresMemory({ connection: connectionString, namespace: 'vec_col' });
const mem = makePostgresMemory('vec_col');
await mem.saveEmbeddings({
threadId: 't1',
@ -492,8 +498,8 @@ describeWithDocker('PostgresMemory unit tests', () => {
});
it('isolates namespaces', async () => {
const mem1 = new PostgresMemory({ connection: connectionString, namespace: 'ns_a' });
const mem2 = new PostgresMemory({ connection: connectionString, namespace: 'ns_b' });
const mem1 = makePostgresMemory('ns_a');
const mem2 = makePostgresMemory('ns_b');
await mem1.saveThread({ id: 'shared-id', resourceId: 'u1', title: 'From A' });
await mem2.saveThread({ id: 'shared-id', resourceId: 'u1', title: 'From B' });
@ -520,7 +526,7 @@ function describeWithDockerAndApi(name: string, fn: () => void) {
describeWithDockerAndApi('PostgresMemory agent integration', () => {
it('recalls previous messages across turns', async () => {
const store = new PostgresMemory({ connection: connectionString, namespace: 'agent_recall' });
const store = makePostgresMemory('agent_recall');
const memory = new Memory().storage(store).lastMessages(10);
const agent = new Agent('pg-recall-test')
@ -540,7 +546,7 @@ describeWithDockerAndApi('PostgresMemory agent integration', () => {
});
it('persists resource-scoped working memory via Postgres backend', async () => {
const store = new PostgresMemory({ connection: connectionString, namespace: 'agent_wm' });
const store = makePostgresMemory('agent_wm');
const memory = new Memory()
.storage(store)
.lastMessages(10)
@ -574,10 +580,7 @@ describeWithDockerAndApi('PostgresMemory agent integration', () => {
});
it('persists thread-scoped working memory via Postgres backend', async () => {
const store = new PostgresMemory({
connection: connectionString,
namespace: 'agent_thread_wm',
});
const store = makePostgresMemory('agent_thread_wm');
const memory = new Memory()
.storage(store)
.lastMessages(10)
@ -617,7 +620,7 @@ describeWithDockerAndApi('PostgresMemory agent integration', () => {
});
it('works with stream() path', async () => {
const store = new PostgresMemory({ connection: connectionString, namespace: 'agent_stream' });
const store = makePostgresMemory('agent_stream');
const memory = new Memory().storage(store).lastMessages(10);
const agent = new Agent('pg-stream-test')

View File

@ -6,7 +6,6 @@ import {
collectStreamChunks,
getModel,
chunksOfType,
findAllToolResults,
collectTextDeltas,
} from './helpers';
import { Agent, Tool } from '../../index';
@ -43,15 +42,14 @@ describe('multi-tool-calls integration', () => {
);
const chunks = await collectStreamChunks(fullStream);
const messageChunks = chunksOfType(chunks, 'message');
const toolCallResults = findAllToolResults(messageChunks.map((c) => c.message));
const toolCallResults = chunksOfType(chunks, 'tool-result');
// Should have called the tool multiple times
const priceCalls = toolCallResults.filter((tc) => tc.toolName === 'lookup_price');
expect(priceCalls.length).toBeGreaterThanOrEqual(2);
// Each call should have its own correct output (not all pointing to the first result)
const outputs = priceCalls.map((tc) => tc.result as { product: string; price: number });
const outputs = priceCalls.map((tc) => tc.output as { product: string; price: number });
// Verify that different products got different prices (index-based merging works)
const uniquePrices = new Set(outputs.map((o) => o.price));
@ -90,8 +88,7 @@ describe('multi-tool-calls integration', () => {
const { stream: fullStream } = await agent.stream('What is 3 + 4 and also what is 5 * 6?');
const chunks = await collectStreamChunks(fullStream);
const messageChunks = chunksOfType(chunks, 'message');
const toolCallResults = findAllToolResults(messageChunks.map((c) => c.message));
const toolCallResults = chunksOfType(chunks, 'tool-result');
const toolCalls = toolCallResults.filter(
(tc) => tc.toolName === 'add' || tc.toolName === 'multiply',
@ -104,8 +101,8 @@ describe('multi-tool-calls integration', () => {
expect(addCall).toBeDefined();
expect(multiplyCall).toBeDefined();
expect((addCall!.result as { result: number }).result).toBe(7);
expect((multiplyCall!.result as { result: number }).result).toBe(30);
expect((addCall!.output as { result: number }).result).toBe(7);
expect((multiplyCall!.output as { result: number }).result).toBe(30);
});
it('correctly merges results via the run() path', async () => {
@ -126,15 +123,14 @@ describe('multi-tool-calls integration', () => {
'What are the lengths of "hello" and "world"? Look up each one separately.',
);
const chunks = await collectStreamChunks(fullStream);
const messageChunks = chunksOfType(chunks, 'message');
const toolCallResults = findAllToolResults(messageChunks.map((c) => c.message));
const toolCallResults = chunksOfType(chunks, 'tool-result');
const lengthCalls = toolCallResults.filter((tc) => tc.toolName === 'get_length');
expect(lengthCalls.length).toBeGreaterThanOrEqual(2);
// Each should have correct output
for (const call of lengthCalls) {
const output = call.result as { text: string; length: number };
const output = call.output as { text: string; length: number };
expect(output.length).toBe(output.text.length);
}
});

View File

@ -28,95 +28,92 @@ describe('orphaned tool messages in memory', () => {
}
/**
* Seed memory with a conversation that has tool-call / tool-result pairs
* surrounded by plain user/assistant exchanges.
* Seed memory with a conversation that has settled tool-call blocks
* (state: 'resolved') surrounded by plain user/assistant exchanges.
*
* Message layout (indices 07):
* 0: user "How many widgets?"
* 1: assistant text + tool-call(call_1)
* 2: tool tool-result(call_1)
* 3: assistant "There are 10 widgets"
* 4: user "What about gadgets?"
* 5: assistant text + tool-call(call_2)
* 6: tool tool-result(call_2)
* 7: assistant "There are 5 gadgets"
* Message layout (indices 05):
* 0: user "How many widgets?"
* 1: assistant text + tool-call(call_1, state:'resolved', output:{count:10})
* 2: assistant "There are 10 widgets"
* 3: user "What about gadgets?"
* 4: assistant text + tool-call(call_2, state:'resolved', output:{count:5})
* 5: assistant "There are 5 gadgets"
*/
function buildSeedMessages(): AgentDbMessage[] {
const now = Date.now();
return [
{
id: 'm1',
createdAt: new Date(),
createdAt: new Date(now),
role: 'user',
content: [{ type: 'text', text: 'How many widgets do we have?' }],
},
{
id: 'm2',
createdAt: new Date(),
createdAt: new Date(now + 1),
role: 'assistant',
content: [
{ type: 'text', text: 'Let me look that up.' },
{ type: 'tool-call', toolCallId: 'call_1', toolName: 'lookup', input: { id: 'widgets' } },
{
type: 'tool-call',
toolCallId: 'call_1',
toolName: 'lookup',
input: { id: 'widgets' },
state: 'resolved',
output: { count: 10 },
},
],
},
{
id: 'm3',
createdAt: new Date(),
role: 'tool',
content: [
{ type: 'tool-result', toolCallId: 'call_1', toolName: 'lookup', result: { count: 10 } },
],
},
{
id: 'm4',
createdAt: new Date(),
createdAt: new Date(now + 2),
role: 'assistant',
content: [{ type: 'text', text: 'There are 10 widgets in stock.' }],
},
{
id: 'm5',
createdAt: new Date(),
id: 'm4',
createdAt: new Date(now + 3),
role: 'user',
content: [{ type: 'text', text: 'What about gadgets?' }],
},
{
id: 'm6',
createdAt: new Date(),
id: 'm5',
createdAt: new Date(now + 4),
role: 'assistant',
content: [
{ type: 'text', text: 'Let me check.' },
{ type: 'tool-call', toolCallId: 'call_2', toolName: 'lookup', input: { id: 'gadgets' } },
{
type: 'tool-call',
toolCallId: 'call_2',
toolName: 'lookup',
input: { id: 'gadgets' },
state: 'resolved',
output: { count: 5 },
},
],
},
{
id: 'm7',
createdAt: new Date(),
role: 'tool',
content: [
{ type: 'tool-result', toolCallId: 'call_2', toolName: 'lookup', result: { count: 5 } },
],
},
{
id: 'm8',
createdAt: new Date(),
id: 'm6',
createdAt: new Date(now + 5),
role: 'assistant',
content: [{ type: 'text', text: 'There are 5 gadgets in stock.' }],
},
];
}
it('handles orphaned tool results when tool-call message is truncated from history', async () => {
it('handles partial history window when earlier messages are truncated', async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
const threadId = 'thread-orphan-result';
// Seed 8 messages into the thread
// Seed 6 messages into the thread
await memory.saveMessages({ threadId, messages: buildSeedMessages() });
// lastMessages=6 → loads messages 27
// Message at index 2 is a tool-result for call_1, but the matching
// assistant+tool-call (index 1) is truncated. This is an orphaned tool result.
const mem = new Memory().storage(memory).lastMessages(6);
// lastMessages=4 → loads messages 25
// Each tool-call block carries its own result (state:'resolved'), so there
// are no orphan issues regardless of window boundaries.
const mem = new Memory().storage(memory).lastMessages(4);
const agent = new Agent('orphan-result-test')
.model(getModel('anthropic'))
@ -132,7 +129,7 @@ describe('orphaned tool messages in memory', () => {
expect(result.finishReason).toBe('stop');
});
it('handles orphaned tool calls when tool-result message is truncated from history', async () => {
it('handles pending tool-call blocks (interrupted turn) in history', async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
@ -140,8 +137,9 @@ describe('orphaned tool messages in memory', () => {
const now = Date.now();
// Store a conversation where the last saved message is an assistant
// with a tool-call but the tool-result was never persisted (simulating
// a partial save / interrupted turn).
// with a pending tool-call block (simulating a partial save / interrupted turn).
// stripOrphanedToolMessages will drop the pending block so the LLM receives
// only the user message.
const messages: AgentDbMessage[] = [
{
id: 'm1',
@ -160,6 +158,7 @@ describe('orphaned tool messages in memory', () => {
toolCallId: 'call_orphan',
toolName: 'lookup',
input: { id: 'widgets' },
state: 'pending',
},
],
},

View File

@ -183,7 +183,7 @@ describe('external abort signal', () => {
});
expect(result.finishReason).toBe('error');
expect(result.getState().status).toBe('cancelled');
expect(agent.getState().status).toBe('cancelled');
});
it('cancels a stream() call via external AbortSignal', async () => {

View File

@ -55,10 +55,8 @@ describe('provider tools integration', () => {
const lastFinish = finishChunks[finishChunks.length - 1];
expect(lastFinish?.type === 'finish' && lastFinish.finishReason).toBe('stop');
// Collect tool calls from message chunks
const messageChunks = chunksOfType(chunks, 'message');
const allMessages = messageChunks.map((c) => c.message);
const toolCalls = findAllToolCalls(allMessages);
// Tool calls now ride their own discrete `tool-call` chunks
const toolCalls = chunksOfType(chunks, 'tool-call');
const webSearchCall = toolCalls.find((tc) => tc.toolName.includes('web_search'));
expect(webSearchCall).toBeDefined();
@ -104,9 +102,8 @@ describe('provider tools integration', () => {
expect(suspended.runId).toBeTruthy();
expect(suspended.toolCallId).toBeTruthy();
// The web search provider tool call should appear in the message history
const messageChunks = chunksOfType(chunks, 'message');
const toolCalls = findAllToolCalls(messageChunks.map((c) => c.message));
// The web search provider tool call should appear as a discrete tool-call chunk
const toolCalls = chunksOfType(chunks, 'tool-call');
const webSearchCall = toolCalls.find((tc) => tc.toolName.includes('web_search'));
expect(webSearchCall).toBeDefined();
@ -115,8 +112,8 @@ describe('provider tools integration', () => {
'stream',
{ approved: true },
{
runId: suspended.runId!,
toolCallId: suspended.toolCallId!,
runId: suspended.runId,
toolCallId: suspended.toolCallId,
},
);
const resumeChunks = await collectStreamChunks(resumeStream.stream);

View File

@ -155,16 +155,8 @@ describe('state restore after suspension', () => {
const errorChunks = resumedChunks.filter((c) => c.type === 'error');
expect(errorChunks).toHaveLength(0);
// Stream must contain the tool result message
const toolResultChunks = resumedChunks.filter(
(c) =>
c.type === 'message' &&
'message' in c &&
'content' in (c.message as object) &&
(c.message as { content: Array<{ type: string }> }).content.some(
(part) => part.type === 'tool-result',
),
);
// Stream must contain a discrete tool-result chunk for the resumed call
const toolResultChunks = chunksOfType(resumedChunks, 'tool-result');
expect(toolResultChunks.length).toBeGreaterThan(0);
// Stream must end with a finish chunk (not error)

View File

@ -7,7 +7,7 @@ import { Agent, Tool } from '../../index';
const describe = describeIf('anthropic');
describe('stream timing', () => {
it('tool-call-delta chunks arrive incrementally (not all buffered)', async () => {
it('tool-input-delta chunks arrive incrementally (not all buffered)', async () => {
const agent = new Agent('timing-test')
.model(getModel('anthropic'))
.instructions(
@ -31,16 +31,21 @@ describe('stream timing', () => {
const reader = result.stream.getReader();
// Track timestamps of each reader.read() that returns a tool-call-delta
// Track timestamps of each reader.read() that returns a tool-input-delta
// for the set_code tool. We seed `setCodeToolCallId` from the matching
// tool-input-start so subsequent deltas can be filtered by toolCallId.
// This measures when the reader YIELDS each chunk, not when the agent enqueues it.
const deltaReadTimes: number[] = [];
const start = Date.now();
let setCodeToolCallId: string | undefined;
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = value;
if (chunk.type === 'tool-call-delta' && (chunk as { name?: string }).name === 'set_code') {
if (chunk.type === 'tool-input-start' && chunk.toolName === 'set_code') {
setCodeToolCallId = chunk.toolCallId;
} else if (chunk.type === 'tool-input-delta' && chunk.toolCallId === setCodeToolCallId) {
deltaReadTimes.push(Date.now() - start);
}
}

View File

@ -5,10 +5,8 @@ import {
collectStreamChunks,
collectTextDeltas,
describeIf,
findAllToolResults,
getModel,
} from './helpers';
import type { StreamChunk } from '../../index';
import { Agent } from '../../index';
const describe = describeIf('anthropic');
@ -33,10 +31,7 @@ describe('sub-agent (asTool) integration', () => {
const chunks = await collectStreamChunks(fullStream);
const text = collectTextDeltas(chunks);
const messageChunks = chunksOfType(chunks, 'message') as Array<
StreamChunk & { type: 'message' }
>;
const toolResults = findAllToolResults(messageChunks.map((c) => c.message));
const toolResults = chunksOfType(chunks, 'tool-result');
// The orchestrator should have called the sub-agent tool
expect(toolResults.length).toBeGreaterThan(0);
@ -44,7 +39,7 @@ describe('sub-agent (asTool) integration', () => {
expect(mathCall).toBeDefined();
// The output should contain the sub-agent's response
expect(mathCall!.result).toBeDefined();
expect(mathCall!.output).toBeDefined();
// The final text should reference 60
expect(text).toBeTruthy();
@ -80,10 +75,7 @@ describe('sub-agent (asTool) integration', () => {
'Translate "hello" to French and then make it uppercase.',
);
const chunks = await collectStreamChunks(fullStream);
const messageChunks = chunksOfType(chunks, 'message') as Array<
StreamChunk & { type: 'message' }
>;
const toolResults = findAllToolResults(messageChunks.map((c) => c.message));
const toolResults = chunksOfType(chunks, 'tool-result');
// Should have called both tools
expect(toolResults.length).toBeGreaterThanOrEqual(2);

View File

@ -63,11 +63,12 @@ describe('toModelOutput integration', () => {
expect(rawOutput.total).toBe(3);
expect(rawOutput.records[0].data).toBe('x'.repeat(200));
// ContentToolResult in messages stores the transformed output (what the LLM saw)
// Tool-call block in messages stores the transformed output (what the LLM saw)
const toolResults = findAllToolResults(result.messages);
const searchToolResult = toolResults.find((tr) => tr.toolName === 'search_db');
expect(searchToolResult).toBeDefined();
const modelOutput = searchToolResult!.result as { summary: string };
expect(searchToolResult!.state).toBe('resolved');
const modelOutput = (searchToolResult as unknown as { output: { summary: string } }).output;
expect(modelOutput.summary).toContain('Found 3 records');
expect(modelOutput.summary).toContain('Widget A');
});
@ -106,15 +107,14 @@ describe('toModelOutput integration', () => {
const { stream } = await agent.stream('Get report RPT-001');
const chunks = await collectStreamChunks(stream);
// The tool result messages in the stream contain the transformed output
const messageChunks = chunksOfType(chunks, 'message');
const toolResults = findAllToolResults(messageChunks.map((c) => c.message));
// The discrete tool-result chunks in the stream contain the transformed output
const toolResults = chunksOfType(chunks, 'tool-result');
const reportResult = toolResults.find((tr) => tr.toolName === 'fetch_report');
expect(reportResult).toBeDefined();
// The model output (transformed) should have the truncated fields
const modelOutput = reportResult!.result as { id: string; title: string; pageCount: number };
const modelOutput = reportResult!.output as { id: string; title: string; pageCount: number };
expect(modelOutput.id).toBe('RPT-001');
expect(modelOutput.title).toBe('Q4 Sales Report');
expect(modelOutput.pageCount).toBe(42);
@ -140,11 +140,14 @@ describe('toModelOutput integration', () => {
const result = await agent.generate('Echo the message "hello world"');
// Without toModelOutput, tool result in messages should have the raw output
// Without toModelOutput, tool-call block in messages has the raw output
const toolResults = findAllToolResults(result.messages);
const echoResult = toolResults.find((tr) => tr.toolName === 'echo');
expect(echoResult).toBeDefined();
expect((echoResult!.result as { echoed: string }).echoed).toBe('hello world');
expect(echoResult!.state).toBe('resolved');
expect((echoResult as unknown as { output: { echoed: string } }).output.echoed).toBe(
'hello world',
);
// And toolCalls should also have the same raw output
expect(result.toolCalls).toBeDefined();
@ -196,11 +199,14 @@ describe('toModelOutput integration', () => {
expect(multiplyEntry).toBeDefined();
expect((multiplyEntry!.output as { result: number }).result).toBe(56);
// Tool result in messages stores the transformed output for the LLM
// Tool-call block in messages stores the transformed output for the LLM
const toolResults = findAllToolResults(result.messages);
const multiplyToolResult = toolResults.find((tr) => tr.toolName === 'multiply');
expect(multiplyToolResult).toBeDefined();
const modelOutput = multiplyToolResult!.result as { answer: number; note: string };
expect(multiplyToolResult!.state).toBe('resolved');
const modelOutput = (
multiplyToolResult as unknown as { output: { answer: number; note: string } }
).output;
expect(modelOutput.answer).toBe(56);
expect(modelOutput.note).toBe('multiplication complete');

View File

@ -0,0 +1,222 @@
/**
* Upsert contract: after a HITL suspend/resume cycle backed by SqliteMemory,
* the thread must contain exactly ONE assistant message with the tool-call
* block (no duplicate rows), and that block must have state: 'resolved'.
*
* The upsert matters because on resume the runtime calls saveToMemory with
* turnDelta() which includes the now-resolved assistant message restored from
* the checkpoint. Without upsert-by-id, a second row would be inserted for
* the same message, breaking the thread ordering contract.
*
* Note: messages with state:'pending' are transient and are NOT written to
* memory during suspension they only live in the checkpoint. Memory only
* receives the final settled state after resume completes.
*/
import { afterEach, expect, it } from 'vitest';
import { z } from 'zod';
import { describeIf, createSqliteMemory, getModel } from './helpers';
import { Agent, filterLlmMessages, Memory, Tool } from '../../index';
import type { AgentDbMessage } from '../../index';
import type { ContentToolCall, Message } from '../../types/sdk/message';
const describe = describeIf('anthropic');
describe('tool-call upsert via suspend/resume (SqliteMemory)', () => {
const cleanups: Array<() => void> = [];
afterEach(() => {
for (const fn of cleanups) fn();
cleanups.length = 0;
});
function extractToolCallBlocks(messages: AgentDbMessage[]): ContentToolCall[] {
return filterLlmMessages(messages).flatMap((m) =>
m.content.filter((c): c is ContentToolCall => c.type === 'tool-call'),
);
}
function buildInterruptibleAgent(memory: ReturnType<typeof createSqliteMemory>['memory']): Agent {
const deleteTool = new Tool('delete_file')
.description('Delete a file at the given path')
.input(z.object({ path: z.string().describe('File path to delete') }))
.output(z.object({ deleted: z.boolean(), path: z.string() }))
.suspend(z.object({ message: z.string(), severity: z.string() }))
.resume(z.object({ approved: z.boolean() }))
.handler(async ({ path }, ctx) => {
if (!ctx.resumeData) {
return await ctx.suspend({ message: `Delete "${path}"?`, severity: 'destructive' });
}
if (!ctx.resumeData.approved) return { deleted: false, path };
return { deleted: true, path };
});
return new Agent('upsert-test-agent')
.model(getModel('anthropic'))
.instructions(
'You are a file manager. When asked to delete a file, use the delete_file tool. Be concise.',
)
.tool(deleteTool)
.memory(new Memory().storage(memory))
.checkpoint('memory');
}
it('after resume, thread has exactly one resolved tool-call block (no duplicate rows)', async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
const threadId = 'thread-upsert-resolved';
const resourceId = 'res-1';
const persistence = { threadId, resourceId };
const agent = buildInterruptibleAgent(memory);
// Turn 1: trigger the suspend — messages with pending tool-call are
// stored in the checkpoint only, NOT in SqliteMemory yet.
const suspendResult = await agent.generate('Please delete /tmp/foo.txt', {
persistence,
});
expect(suspendResult.finishReason).toBe('tool-calls');
expect(suspendResult.pendingSuspend).toBeDefined();
const { runId, toolCallId } = suspendResult.pendingSuspend![0];
// Before resume: no tool-call blocks in memory (pending stays in checkpoint)
const msgsBefore = await memory.getMessages(threadId);
const blocksBefore = extractToolCallBlocks(msgsBefore);
expect(blocksBefore).toHaveLength(0);
// Turn 2: resume with approval — on completion saveToMemory is called and
// the assistant message (now resolved) is written for the first time.
const resumeResult = await agent.resume(
'generate',
{ approved: true },
{
runId,
toolCallId,
},
);
expect(resumeResult.finishReason).toBe('stop');
// After resume: exactly one resolved tool-call block, no duplicate rows
const msgsAfter = await memory.getMessages(threadId);
const blocksAfter = extractToolCallBlocks(msgsAfter);
expect(blocksAfter).toHaveLength(1);
expect(blocksAfter[0].state).toBe('resolved');
expect(blocksAfter[0].toolCallId).toBe(toolCallId);
expect((blocksAfter[0] as ContentToolCall & { state: 'resolved' }).output).toMatchObject({
deleted: true,
});
// No duplicate assistant messages with tool-call blocks
const assistantMsgsWithToolCalls = filterLlmMessages(msgsAfter).filter(
(m) => m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
);
expect(assistantMsgsWithToolCalls).toHaveLength(1);
});
it('after resume with denial, thread has exactly one resolved tool-call block', async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
const threadId = 'thread-upsert-denied';
const resourceId = 'res-2';
const persistence = { threadId, resourceId };
const agent = buildInterruptibleAgent(memory);
const suspendResult = await agent.generate('Please delete /tmp/bar.txt', {
persistence,
});
expect(suspendResult.finishReason).toBe('tool-calls');
const { runId, toolCallId } = suspendResult.pendingSuspend![0];
// Before resume: no messages in memory
const msgsBefore = await memory.getMessages(threadId);
expect(extractToolCallBlocks(msgsBefore)).toHaveLength(0);
const resumeResult = await agent.resume(
'generate',
{ approved: false },
{
runId,
toolCallId,
},
);
expect(resumeResult.finishReason).toBe('stop');
const msgsAfter = await memory.getMessages(threadId);
const blocksAfter = extractToolCallBlocks(msgsAfter);
// Tool ran and returned {deleted: false} — still resolved, not rejected
expect(blocksAfter).toHaveLength(1);
expect(blocksAfter[0].state).toBe('resolved');
const output = (blocksAfter[0] as ContentToolCall & { state: 'resolved' }).output;
expect(output).toMatchObject({ deleted: false });
// No duplicate rows
const assistantMsgsWithToolCalls = filterLlmMessages(msgsAfter).filter(
(m) => m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
);
expect(assistantMsgsWithToolCalls).toHaveLength(1);
});
it('if same thread is resumed twice (re-suspend then resume again), still no duplicate rows', async () => {
const { memory, cleanup } = createSqliteMemory();
cleanups.push(cleanup);
const threadId = 'thread-upsert-double';
const resourceId = 'res-3';
const persistence = { threadId, resourceId };
// Use a tool that always re-suspends on first call and approves on second
let callCount = 0;
const confirmTool = new Tool('confirm')
.description('Confirm an action')
.input(z.object({ action: z.string() }))
.output(z.object({ done: z.boolean() }))
.suspend(z.object({ question: z.string() }))
.resume(z.object({ yes: z.boolean() }))
.handler(async ({ action }, ctx) => {
callCount++;
if (!ctx.resumeData) {
return await ctx.suspend({ question: `Confirm: ${action}?` });
}
return { done: ctx.resumeData.yes };
});
const agent = new Agent('double-upsert-agent')
.model(getModel('anthropic'))
.instructions('Use confirm tool for every action. Be concise.')
.tool(confirmTool)
.memory(new Memory().storage(memory))
.checkpoint('memory');
// Turn 1: suspend
const r1 = await agent.generate('confirm action: foo', { persistence });
expect(r1.finishReason).toBe('tool-calls');
const { runId, toolCallId } = r1.pendingSuspend![0];
// No messages in memory yet
expect(await memory.getMessages(threadId)).toHaveLength(0);
// Resume: completes
const r2 = await agent.resume('generate', { yes: true }, { runId, toolCallId });
expect(r2.finishReason).toBe('stop');
const finalMessages = await memory.getMessages(threadId);
const toolCallBlocks = extractToolCallBlocks(finalMessages);
// Exactly one tool-call block, no duplicates
expect(toolCallBlocks).toHaveLength(1);
expect(toolCallBlocks[0].state).toBe('resolved');
// And the assistant message with the tool-call appears exactly once
const assistantMsgsWithCalls = filterLlmMessages(finalMessages).filter(
(m): m is Message => m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
);
expect(assistantMsgsWithCalls).toHaveLength(1);
});
});

View File

@ -5,7 +5,6 @@ import {
collectStreamChunks,
chunksOfType,
collectTextDeltas,
findAllToolResults,
createAgentWithAlwaysErrorTool,
createAgentWithFlakyTool,
} from './helpers';
@ -55,20 +54,20 @@ describe('tool error handling integration', () => {
expect(mentionsFailure).toBe(true);
});
it('error tool-result appears in the message list', async () => {
it('error tool-result appears in the stream', async () => {
const agent = createAgentWithAlwaysErrorTool('anthropic');
const { stream } = await agent.stream('Fetch the data for id "abc123".');
const chunks = await collectStreamChunks(stream);
// There should be a tool-result message in the stream
const messageChunks = chunksOfType(chunks, 'message');
const toolResults = findAllToolResults(messageChunks.map((c) => c.message));
// There should be a discrete tool-result chunk for the failed call
const toolResults = chunksOfType(chunks, 'tool-result');
// The tool should have been called and produced a result (even if it errored)
expect(toolResults.length).toBeGreaterThan(0);
const brokenResult = toolResults.find((r) => r.toolName === 'broken_tool');
expect(brokenResult).toBeDefined();
expect(brokenResult!.isError).toBe(true);
});
it('LLM can self-correct by retrying a flaky tool', async () => {

View File

@ -8,7 +8,7 @@ import {
createAgentWithMixedTools,
createAgentWithParallelInterruptibleCalls,
} from './helpers';
import { isLlmMessage, type StreamChunk } from '../../index';
import type { StreamChunk } from '../../index';
const describe = describeIf('anthropic');
@ -36,13 +36,8 @@ describe('tool interrupt integration', () => {
);
// No tool-result should appear (tool is suspended)
const contentChunks = chunks.filter(
(c) =>
c.type === 'message' &&
'content' in c &&
(c.content as { type: string }).type === 'tool-result',
);
expect(contentChunks).toHaveLength(0);
const toolResultChunks = chunksOfType(chunks, 'tool-result');
expect(toolResultChunks).toHaveLength(0);
});
it('resumes the stream after resume with approval', async () => {
@ -58,19 +53,14 @@ describe('tool interrupt integration', () => {
const resumedStream = await agent.resume(
'stream',
{ approved: true },
{ runId: suspended.runId!, toolCallId: suspended.toolCallId! },
{ runId: suspended.runId, toolCallId: suspended.toolCallId },
);
const resumedChunks = await collectStreamChunks(resumedStream.stream);
const resumedTypes = resumedChunks.map((c) => c.type);
// After approval, tool-result should appear as content chunk
const toolResultChunks = resumedChunks.filter(
(c) =>
c.type === 'message' &&
isLlmMessage(c.message) &&
c.message.content.some((c) => c.type === 'tool-result'),
);
// After approval, a discrete tool-result chunk should appear
const toolResultChunks = chunksOfType(resumedChunks, 'tool-result');
expect(toolResultChunks.length).toBeGreaterThan(0);
expect(resumedTypes).toContain('text-delta');
@ -89,7 +79,7 @@ describe('tool interrupt integration', () => {
const resumedStream = await agent.resume(
'stream',
{ approved: false },
{ runId: suspended.runId!, toolCallId: suspended.toolCallId! },
{ runId: suspended.runId, toolCallId: suspended.toolCallId },
);
const resumedChunks = await collectStreamChunks(resumedStream.stream);
@ -119,7 +109,7 @@ describe('tool interrupt integration', () => {
const stream2 = await agent.resume(
'stream',
{ approved: true },
{ runId: suspended1.runId!, toolCallId: suspended1.toolCallId! },
{ runId: suspended1.runId, toolCallId: suspended1.toolCallId },
);
const chunks2 = await collectStreamChunks(stream2.stream);
@ -136,7 +126,7 @@ describe('tool interrupt integration', () => {
const stream3 = await agent.resume(
'stream',
{ approved: true },
{ runId: suspended2.runId!, toolCallId: suspended2.toolCallId! },
{ runId: suspended2.runId, toolCallId: suspended2.toolCallId },
);
const chunks3 = await collectStreamChunks(stream3.stream);
@ -162,13 +152,8 @@ describe('tool interrupt integration', () => {
const chunks = await collectStreamChunks(fullStream);
// list_files should auto-execute — its result should appear as content
const toolResultChunks = chunks.filter(
(c) =>
c.type === 'message' &&
isLlmMessage(c.message) &&
c.message.content.some((c) => c.type === 'tool-result'),
);
// list_files should auto-execute — its result should appear as a discrete tool-result chunk
const toolResultChunks = chunksOfType(chunks, 'tool-result');
expect(toolResultChunks.length).toBeGreaterThan(0);
// delete_file should be suspended

View File

@ -69,7 +69,10 @@ describe('workspace agent integration', () => {
const readResult = toolResults.find((tr) => tr.toolName === 'workspace_read_file');
expect(readResult).toBeDefined();
expect((readResult!.result as { content: string }).content).toContain('Hello from n8n!');
expect(readResult!.state).toBe('resolved');
expect((readResult as unknown as { output: { content: string } }).output.content).toContain(
'Hello from n8n!',
);
expect(memFs.getFileContent('/greeting.txt')).toBe('Hello from n8n!');
});
@ -103,7 +106,8 @@ describe('workspace agent integration', () => {
const toolResults = findAllToolResults(result.messages);
const execResult = toolResults.find((tr) => tr.toolName === 'workspace_execute_command');
expect(execResult).toBeDefined();
expect((execResult!.result as { success: boolean }).success).toBe(true);
expect(execResult!.state).toBe('resolved');
expect((execResult as unknown as { output: { success: boolean } }).output.success).toBe(true);
});
it('agent uses workspace_mkdir and workspace_list_files together', async () => {
@ -130,7 +134,8 @@ describe('workspace agent integration', () => {
const toolResults = findAllToolResults(result.messages);
const listResult = toolResults.find((tr) => tr.toolName === 'workspace_list_files');
expect(listResult).toBeDefined();
const entries = (listResult!.result as unknown as { entries: FileEntry[] }).entries;
expect(listResult!.state).toBe('resolved');
const entries = (listResult as unknown as { output: { entries: FileEntry[] } }).output.entries;
const names = entries.map((e) => e.name);
expect(names).toContain('index.ts');
expect(names).toContain('README.md');
@ -201,7 +206,8 @@ describe('workspace agent integration', () => {
const toolResults = findAllToolResults(result.messages);
const statResult = toolResults.find((tr) => tr.toolName === 'workspace_file_stat');
expect(statResult).toBeDefined();
const stat = statResult!.result as { type: string; size: number };
expect(statResult!.state).toBe('resolved');
const stat = (statResult as unknown as { output: { type: string; size: number } }).output;
expect(stat.type).toBe('file');
expect(stat.size).toBe(29);
});
@ -233,7 +239,10 @@ describe('workspace agent integration', () => {
const readResult = toolResults.find((tr) => tr.toolName === 'workspace_read_file');
expect(readResult).toBeDefined();
expect((readResult!.result as { content: string }).content).toContain('export default {}');
expect(readResult!.state).toBe('resolved');
expect((readResult as unknown as { output: { content: string } }).output.content).toContain(
'export default {}',
);
expect(memFs.getFileContent('/app/config.ts')).toBe('export default {}');
});

View File

@ -45,12 +45,12 @@ describe('Zod validation errors surface to LLM and allow self-correction', () =>
expect(result.finishReason).toBe('stop');
expect(result.error).toBeUndefined();
// At least two tool-result messages: one error, one success
// At least two tool-call messages: one rejected, one resolved
const allMessages = filterLlmMessages(result.messages);
const toolResultMessages = allMessages.filter((m) =>
m.content.some((c) => c.type === 'tool-result'),
const toolCallMessages = allMessages.filter((m) =>
m.content.some((c) => c.type === 'tool-call'),
);
expect(toolResultMessages.length).toBeGreaterThanOrEqual(2);
expect(toolCallMessages.length).toBeGreaterThanOrEqual(2);
// The final response should mention a user (age 25 or similar)
const text = findLastTextContent(result.messages);

View File

@ -1,6 +1,6 @@
import { AgentMessageList } from '../runtime/message-list';
import { isLlmMessage } from '../sdk/message';
import type { AgentDbMessage, AgentMessage, Message } from '../types/sdk/message';
import type { AgentDbMessage, AgentMessage, ContentToolCall, Message } from '../types/sdk/message';
function makeUserMsg(text: string): AgentMessage {
return { role: 'user', content: [{ type: 'text', text }] };
@ -174,3 +174,118 @@ describe('AgentMessageList — deserialize', () => {
expect(newMsg.createdAt.getTime()).toBeGreaterThan(futureTs.getTime());
});
});
// ---------------------------------------------------------------------------
// setToolCallResult / setToolCallError
// ---------------------------------------------------------------------------
function makePendingToolCallMsg(toolCallId: string): AgentMessage {
return {
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId,
toolName: 'my_tool',
input: { x: 1 },
state: 'pending',
},
],
};
}
describe('AgentMessageList — setToolCallResult', () => {
it('sets state and output on the matching tool-call block', () => {
const list = new AgentMessageList();
list.addResponse([makePendingToolCallMsg('id-1')]);
const host = list.setToolCallResult('id-1', { ok: true });
expect(host).toBeDefined();
const block = (host as Message).content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(block.state).toBe('resolved');
expect((block as ContentToolCall & { state: 'resolved' }).output).toEqual({ ok: true });
});
it('promotes a history-only message into responseDelta after setToolCallResult', () => {
const list = new AgentMessageList();
const histMsg: AgentDbMessage = {
id: 'hist-1',
createdAt: new Date('2024-01-01T00:00:01.000Z'),
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'tc-hist',
toolName: 'my_tool',
input: {},
state: 'pending',
},
],
};
list.addHistory([histMsg]);
// Before: not in responseDelta (history only)
expect(list.responseDelta()).toHaveLength(0);
list.setToolCallResult('tc-hist', { done: true });
// After: promoted to responseDelta
const delta = list.responseDelta();
expect(delta).toHaveLength(1);
const block = (delta[0] as Message).content.find(
(c) => c.type === 'tool-call',
) as ContentToolCall;
expect(block.state).toBe('resolved');
});
it('is a no-op when toolCallId is unknown', () => {
const list = new AgentMessageList();
list.addResponse([makePendingToolCallMsg('id-1')]);
const result = list.setToolCallResult('unknown-id', { x: 1 });
expect(result).toBeUndefined();
// List unchanged
expect(list.responseDelta()).toHaveLength(1);
});
it('Set semantics make repeated calls idempotent (no duplicate messages)', () => {
const list = new AgentMessageList();
list.addResponse([makePendingToolCallMsg('id-1')]);
list.setToolCallResult('id-1', { ok: true });
list.setToolCallResult('id-1', { ok: true });
expect(list.responseDelta()).toHaveLength(1);
});
});
describe('AgentMessageList — setToolCallError', () => {
it('stringifies errors and clears any prior output', () => {
const list = new AgentMessageList();
list.addResponse([
{
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'id-1',
toolName: 'my_tool',
input: {},
state: 'resolved',
output: { prev: true },
},
],
},
]);
const host = list.setToolCallError('id-1', new Error('boom'));
expect(host).toBeDefined();
const block = (host as Message).content.find((c) => c.type === 'tool-call') as ContentToolCall;
expect(block.state).toBe('rejected');
expect((block as ContentToolCall & { state: 'rejected' }).error).toBe('Error: boom');
// output should be gone
expect((block as unknown as { output?: unknown }).output).toBeUndefined();
});
});

View File

@ -9,6 +9,8 @@ type ProviderOpts = {
headers?: Record<string, string>;
};
// All providers are mocked via jest.mock so require() inside the registry entries
// returns these stubs instead of the real packages.
jest.mock('@ai-sdk/anthropic', () => ({
createAnthropic: (opts?: ProviderOpts) => (model: string) => ({
provider: 'anthropic',
@ -33,6 +35,119 @@ jest.mock('@ai-sdk/openai', () => ({
}),
}));
jest.mock('@ai-sdk/google', () => ({
createGoogleGenerativeAI: (opts?: ProviderOpts) => (model: string) => ({
provider: 'google',
modelId: model,
apiKey: opts?.apiKey,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/xai', () => ({
createXai: (opts?: ProviderOpts) => (model: string) => ({
provider: 'xai',
modelId: model,
apiKey: opts?.apiKey,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/groq', () => ({
createGroq: (opts?: ProviderOpts) => (model: string) => ({
provider: 'groq',
modelId: model,
apiKey: opts?.apiKey,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/deepseek', () => ({
createDeepSeek: (opts?: ProviderOpts) => (model: string) => ({
provider: 'deepseek',
modelId: model,
apiKey: opts?.apiKey,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/cohere', () => ({
createCohere: (opts?: ProviderOpts) => (model: string) => ({
provider: 'cohere',
modelId: model,
apiKey: opts?.apiKey,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/mistral', () => ({
createMistral: (opts?: ProviderOpts) => (model: string) => ({
provider: 'mistral',
modelId: model,
apiKey: opts?.apiKey,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/gateway', () => ({
createGateway: (opts?: ProviderOpts) => (model: string) => ({
provider: 'vercel',
modelId: model,
apiKey: opts?.apiKey,
baseURL: opts?.baseURL,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/azure', () => ({
createAzure:
(opts?: { apiKey?: string; resourceName?: string; apiVersion?: string; baseURL?: string }) =>
(model: string) => ({
provider: 'azure-openai',
modelId: model,
apiKey: opts?.apiKey,
resourceName: opts?.resourceName,
apiVersion: opts?.apiVersion,
specificationVersion: 'v3',
}),
}));
jest.mock('@openrouter/ai-sdk-provider', () => ({
createOpenRouter: (opts?: ProviderOpts) => (model: string) => ({
provider: 'openrouter',
modelId: model,
apiKey: opts?.apiKey,
baseURL: opts?.baseURL,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
}));
jest.mock('@ai-sdk/amazon-bedrock', () => ({
createAmazonBedrock:
(opts?: {
region?: string;
accessKeyId?: string;
secretAccessKey?: string;
sessionToken?: string;
}) =>
(model: string) => ({
provider: 'aws-bedrock',
modelId: model,
region: opts?.region,
accessKeyId: opts?.accessKeyId,
secretAccessKey: opts?.secretAccessKey,
specificationVersion: 'v3',
}),
}));
const mockProxyAgent = jest.fn();
jest.mock('undici', () => ({
ProxyAgent: mockProxyAgent,
@ -58,15 +173,13 @@ describe('createModel', () => {
expect(model.modelId).toBe('claude-sonnet-4-5');
});
it('should accept an object config with url', () => {
it('should accept an object config with baseURL', () => {
const model = createModel({
id: 'openai/gpt-4o',
apiKey: 'sk-test',
url: 'https://custom.endpoint.com/v1',
baseURL: 'https://custom.endpoint.com/v1',
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe('openai');
expect(model.modelId).toBe('gpt-4o');
expect(model.apiKey).toBe('sk-test');
expect(model.baseURL).toBe('https://custom.endpoint.com/v1');
});
@ -130,4 +243,113 @@ describe('createModel', () => {
createModel('anthropic/claude-sonnet-4-5');
expect(mockProxyAgent).toHaveBeenCalledWith('http://https-proxy:8080');
});
describe('standard providers', () => {
it.each(['groq', 'deepseek', 'cohere', 'mistral', 'google', 'xai'])(
'should create model for %s',
(provider) => {
const model = createModel({
id: `${provider}/some-model`,
apiKey: 'test-key',
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe(provider);
expect(model.modelId).toBe('some-model');
expect(model.apiKey).toBe('test-key');
},
);
it('should create model for vercel gateway', () => {
const model = createModel({
id: 'vercel/gpt-4o',
apiKey: 'vk-test',
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe('vercel');
expect(model.modelId).toBe('gpt-4o');
});
it('should create model for openrouter', () => {
const model = createModel({
id: 'openrouter/openai/gpt-4o',
apiKey: 'or-test',
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe('openrouter');
expect(model.modelId).toBe('openai/gpt-4o');
expect(model.apiKey).toBe('or-test');
});
});
describe('azure-openai', () => {
it('should create model with resourceName', () => {
const model = createModel({
id: 'azure-openai/gpt-4o',
apiKey: 'az-key',
resourceName: 'my-resource',
apiVersion: '2024-02-01',
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe('azure-openai');
expect(model.modelId).toBe('gpt-4o');
expect(model.apiKey).toBe('az-key');
expect(model.resourceName).toBe('my-resource');
expect(model.apiVersion).toBe('2024-02-01');
});
it('should throw if resourceName is missing', () => {
expect(() => createModel({ id: 'azure-openai/gpt-4o', apiKey: 'az-key' })).toThrow(
/Invalid credentials for provider "azure-openai"/,
);
});
});
describe('aws-bedrock', () => {
it('should create model with AWS credentials', () => {
const model = createModel({
id: 'aws-bedrock/amazon.titan-text-lite-v1',
region: 'us-east-1',
accessKeyId: 'AKIAIOSFODNN7EXAMPLE',
secretAccessKey: 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe('aws-bedrock');
expect(model.modelId).toBe('amazon.titan-text-lite-v1');
expect(model.region).toBe('us-east-1');
expect(model.accessKeyId).toBe('AKIAIOSFODNN7EXAMPLE');
});
it('should throw if region is missing', () => {
expect(() =>
createModel({
id: 'aws-bedrock/amazon.titan-text-lite-v1',
accessKeyId: 'AKIAIOSFODNN7EXAMPLE',
secretAccessKey: 'secret',
}),
).toThrow(/Invalid credentials for provider "aws-bedrock"/);
});
it('should throw if accessKeyId is missing', () => {
expect(() =>
createModel({
id: 'aws-bedrock/amazon.titan-text-lite-v1',
region: 'us-east-1',
secretAccessKey: 'secret',
}),
).toThrow(/Invalid credentials for provider "aws-bedrock"/);
});
});
describe('unsupported provider', () => {
it('should throw for ollama', () => {
expect(() => createModel('ollama/llama3')).toThrow(/Unsupported provider: "ollama"/);
});
it('should include supported providers in the error message', () => {
expect(() => createModel('unknown-provider/some-model')).toThrow(/Supported providers:/);
});
it('should throw when no model ID is provided', () => {
expect(() => createModel('')).toThrow(/Model ID is required/);
});
it('should throw when model has no slash', () => {
expect(() => createModel('anthropic-only')).toThrow(/expected "provider\/model-name"/);
});
});
});

View File

@ -0,0 +1,146 @@
import type { JSONSchema7 } from 'json-schema';
import { z } from 'zod';
import { parseWithSchema } from '../utils/parse';
// ---------------------------------------------------------------------------
// parseWithSchema — Zod schemas
// ---------------------------------------------------------------------------
describe('parseWithSchema — Zod schemas', () => {
it('returns success with parsed data for valid input', async () => {
const schema = z.object({ name: z.string(), age: z.number() });
const result = await parseWithSchema(schema, { name: 'Alice', age: 30 });
expect(result.success).toBe(true);
if (result.success) expect(result.data).toEqual({ name: 'Alice', age: 30 });
});
it('coerces and transforms values as defined in the schema', async () => {
const schema = z.object({ id: z.string().transform((s) => s.toUpperCase()) });
const result = await parseWithSchema(schema, { id: 'abc' });
expect(result.success).toBe(true);
if (result.success) expect((result.data as { id: string }).id).toBe('ABC');
});
it('returns failure with an error message for wrong type', async () => {
const schema = z.object({ count: z.number() });
const result = await parseWithSchema(schema, { count: 'not-a-number' });
expect(result.success).toBe(false);
if (!result.success) expect(result.error).toBeTruthy();
});
it('returns failure when a required field is missing', async () => {
const schema = z.object({ name: z.string(), age: z.number() });
const result = await parseWithSchema(schema, { name: 'Alice' });
expect(result.success).toBe(false);
if (!result.success) expect(result.error).toMatch(/required/i);
});
it('returns failure when a value violates a refinement', async () => {
const schema = z.object({ age: z.number().min(18, 'must be at least 18') });
const result = await parseWithSchema(schema, { age: 5 });
expect(result.success).toBe(false);
if (!result.success) expect(result.error).toContain('must be at least 18');
});
});
// ---------------------------------------------------------------------------
// parseWithSchema — JSON Schema (AJV)
// ---------------------------------------------------------------------------
describe('parseWithSchema — JSON Schema', () => {
it('returns success with the original data for valid input', async () => {
const schema = {
type: 'object' as const,
properties: { name: { type: 'string' }, age: { type: 'integer' } },
required: ['name', 'age'],
} as JSONSchema7;
const result = await parseWithSchema(schema, { name: 'Bob', age: 25 });
expect(result.success).toBe(true);
if (result.success) expect(result.data).toEqual({ name: 'Bob', age: 25 });
});
it('returns failure when a property has the wrong type', async () => {
const schema = {
type: 'object' as const,
properties: { id: { type: 'string' } },
required: ['id'],
} as JSONSchema7;
const result = await parseWithSchema(schema, { id: 42 });
expect(result.success).toBe(false);
if (!result.success) expect(result.error).toBeTruthy();
});
it('returns failure when a required property is missing', async () => {
const schema = {
type: 'object' as const,
properties: {
name: { type: 'string' },
age: { type: 'integer' },
},
required: ['name', 'age'],
} as JSONSchema7;
const result = await parseWithSchema(schema, { name: 'Alice' });
expect(result.success).toBe(false);
if (!result.success) expect(result.error).toBeTruthy();
});
it('returns failure when a numeric constraint is violated', async () => {
const schema = {
type: 'object' as const,
properties: { age: { type: 'integer', minimum: 18, maximum: 99 } },
required: ['age'],
} as JSONSchema7;
const tooLow = await parseWithSchema(schema, { age: 5 });
expect(tooLow.success).toBe(false);
const tooHigh = await parseWithSchema(schema, { age: 150 });
expect(tooHigh.success).toBe(false);
const valid = await parseWithSchema(schema, { age: 30 });
expect(valid.success).toBe(true);
});
it('returns failure for an enum constraint violation', async () => {
const schema = {
type: 'object' as const,
properties: { status: { type: 'string', enum: ['active', 'inactive'] } },
required: ['status'],
} as JSONSchema7;
const invalid = await parseWithSchema(schema, { status: 'pending' });
expect(invalid.success).toBe(false);
const valid = await parseWithSchema(schema, { status: 'active' });
expect(valid.success).toBe(true);
});
it('validates nested object properties', async () => {
const schema = {
type: 'object' as const,
properties: {
address: {
type: 'object',
properties: { zip: { type: 'string' } },
required: ['zip'],
},
},
required: ['address'],
} as JSONSchema7;
const valid = await parseWithSchema(schema, { address: { zip: '10001' } });
expect(valid.success).toBe(true);
const invalid = await parseWithSchema(schema, { address: { zip: 12345 } });
expect(invalid.success).toBe(false);
});
});

View File

@ -578,7 +578,7 @@ describe('SqliteMemory — queryEmbeddings', () => {
describe('SqliteMemory — namespace', () => {
it('rejects invalid namespace characters', () => {
expect(() => new SqliteMemory({ url: 'file::memory:', namespace: 'bad-ns!' })).toThrow(
/Invalid namespace/,
/invalid_string/,
);
});

View File

@ -2,52 +2,38 @@ import { stripOrphanedToolMessages } from '../runtime/strip-orphaned-tool-messag
import type { AgentMessage, Message } from '../types/sdk/message';
describe('stripOrphanedToolMessages', () => {
it('returns messages unchanged when all tool pairs are complete', () => {
it('returns messages unchanged when all tool-calls are settled', () => {
const messages: AgentMessage[] = [
{ role: 'user', content: [{ type: 'text', text: 'Hello' }] },
{
role: 'assistant',
content: [
{ type: 'text', text: 'Looking up...' },
{ type: 'tool-call', toolCallId: 'c1', toolName: 'lookup', input: {} },
{
type: 'tool-call',
toolCallId: 'c1',
toolName: 'lookup',
input: {},
state: 'resolved',
output: 42,
},
],
},
{
role: 'tool',
content: [{ type: 'tool-result', toolCallId: 'c1', toolName: 'lookup', result: 42 }],
},
{ role: 'assistant', content: [{ type: 'text', text: 'Done.' }] },
];
const result = stripOrphanedToolMessages(messages);
expect(result).toBe(messages);
expect(result).toEqual(messages);
});
it('strips orphaned tool-result when matching tool-call is missing', () => {
const messages: AgentMessage[] = [
{
role: 'tool',
content: [{ type: 'tool-result', toolCallId: 'c1', toolName: 'lookup', result: 42 }],
},
{ role: 'assistant', content: [{ type: 'text', text: 'There are 42.' }] },
{ role: 'user', content: [{ type: 'text', text: 'Thanks' }] },
];
const result = stripOrphanedToolMessages(messages) as Message[];
expect(result).toHaveLength(2);
expect(result[0].role).toBe('assistant');
expect(result[1].role).toBe('user');
});
it('strips orphaned tool-call when matching tool-result is missing', () => {
it('drops pending tool-call blocks while preserving sibling content', () => {
const messages: AgentMessage[] = [
{ role: 'user', content: [{ type: 'text', text: 'Check it' }] },
{
role: 'assistant',
content: [
{ type: 'text', text: 'Checking...' },
{ type: 'tool-call', toolCallId: 'c1', toolName: 'lookup', input: {} },
{ type: 'tool-call', toolCallId: 'c1', toolName: 'lookup', input: {}, state: 'pending' },
],
},
];
@ -61,12 +47,14 @@ describe('stripOrphanedToolMessages', () => {
expect(assistantMsg.content[0].type).toBe('text');
});
it('drops assistant message entirely if it only contained an orphaned tool-call', () => {
it('drops empty messages after pending strip', () => {
const messages: AgentMessage[] = [
{ role: 'user', content: [{ type: 'text', text: 'Do it' }] },
{
role: 'assistant',
content: [{ type: 'tool-call', toolCallId: 'c1', toolName: 'action', input: {} }],
content: [
{ type: 'tool-call', toolCallId: 'c1', toolName: 'action', input: {}, state: 'pending' },
],
},
];
@ -76,44 +64,45 @@ describe('stripOrphanedToolMessages', () => {
expect(result[0].role).toBe('user');
});
it('handles mixed scenario: one complete pair and one orphaned result', () => {
it('mixed scenario — only pending blocks are removed', () => {
const messages: AgentMessage[] = [
{
role: 'tool',
content: [
{ type: 'tool-result', toolCallId: 'orphan', toolName: 'lookup', result: 'stale' },
],
},
{ role: 'assistant', content: [{ type: 'text', text: 'Old result' }] },
{ role: 'user', content: [{ type: 'text', text: 'New question' }] },
{
role: 'assistant',
content: [
{ type: 'text', text: 'Looking up...' },
{ type: 'tool-call', toolCallId: 'c2', toolName: 'lookup', input: {} },
{
type: 'tool-call',
toolCallId: 'c1',
toolName: 'lookup',
input: {},
state: 'resolved',
output: 99,
},
{
type: 'tool-call',
toolCallId: 'c2',
toolName: 'delete',
input: {},
state: 'pending',
},
{
type: 'tool-call',
toolCallId: 'c3',
toolName: 'create',
input: {},
state: 'rejected',
error: 'boom',
},
],
},
{
role: 'tool',
content: [{ type: 'tool-result', toolCallId: 'c2', toolName: 'lookup', result: 99 }],
},
{ role: 'assistant', content: [{ type: 'text', text: '99 items' }] },
];
const result = stripOrphanedToolMessages(messages) as Message[];
expect(result).toHaveLength(5);
expect(result[0].role).toBe('assistant');
expect(result[0].content[0]).toEqual(
expect.objectContaining({ type: 'text', text: 'Old result' }),
);
const toolCallMsg = result.find(
(m) => m.role === 'assistant' && m.content.some((c) => c.type === 'tool-call'),
);
expect(toolCallMsg).toBeDefined();
const toolResultMsg = result.find((m) => m.role === 'tool');
expect(toolResultMsg).toBeDefined();
expect(result).toHaveLength(1);
const blocks = result[0].content;
// c2 (pending) should be removed; c1 (resolved) and c3 (rejected) stay
expect(blocks).toHaveLength(2);
expect(blocks.map((b) => (b as { toolCallId: string }).toolCallId)).toEqual(['c1', 'c3']);
});
it('preserves custom (non-LLM) messages', () => {
@ -127,8 +116,16 @@ describe('stripOrphanedToolMessages', () => {
const messages: AgentMessage[] = [
customMsg,
{
role: 'tool',
content: [{ type: 'tool-result', toolCallId: 'orphan', toolName: 'x', result: null }],
role: 'assistant',
content: [
{
type: 'tool-call',
toolCallId: 'c1',
toolName: 'x',
input: {},
state: 'pending',
},
],
},
];
@ -137,14 +134,4 @@ describe('stripOrphanedToolMessages', () => {
expect(result).toHaveLength(1);
expect(result[0]).toBe(customMsg);
});
it('returns same array reference when no orphans exist (no-op fast path)', () => {
const messages: AgentMessage[] = [
{ role: 'user', content: [{ type: 'text', text: 'Hi' }] },
{ role: 'assistant', content: [{ type: 'text', text: 'Hello!' }] },
];
const result = stripOrphanedToolMessages(messages);
expect(result).toBe(messages);
});
});

View File

@ -120,39 +120,4 @@ describe('generateTitleFromMessage', () => {
const call = mockGenerateText.mock.calls[0][0];
expect(call.messages[0].content).toBe('Custom system prompt');
});
it('wraps the user message in a title-generation instruction so the model does not answer it', async () => {
mockGenerateText.mockResolvedValue({ text: 'Berlin rain alert' });
await generateTitleFromMessage(fakeModel, 'Build a daily Berlin rain alert workflow');
const call = mockGenerateText.mock.calls[0][0];
expect(call.messages[1].role).toBe('user');
expect(call.messages[1].content).toContain('Generate a title');
expect(call.messages[1].content).toContain('<message>');
expect(call.messages[1].content).toContain('Build a daily Berlin rain alert workflow');
expect(call.messages[1].content).toContain('</message>');
});
it('drops a streamed code fence and everything after it', async () => {
mockGenerateText.mockResolvedValue({
text: 'Here\'s your chat workflow with the requested configuration:\n\n```json\n{\n "nodes": []\n}\n```',
});
const result = await generateTitleFromMessage(
fakeModel,
'build me a chat workflow with openai',
);
expect(result).toBe("Here's your chat workflow with the requested configuration");
expect(result).not.toContain('```');
expect(result).not.toContain('\n');
});
it('collapses embedded newlines and stray backticks into a single-line title', async () => {
mockGenerateText.mockResolvedValue({
text: 'Scryfall\nrandom `card` workflow',
});
const result = await generateTitleFromMessage(
fakeModel,
'build a workflow that queries Scryfall for a random card',
);
expect(result).toBe('Scryfall random card workflow');
});
});

View File

@ -123,6 +123,37 @@ describe('Tool builder — without approval', () => {
});
});
// ---------------------------------------------------------------------------
// Tool builder — .systemInstruction()
// ---------------------------------------------------------------------------
describe('Tool builder — .systemInstruction()', () => {
it('build() carries the systemInstruction onto the BuiltTool', () => {
const tool = new Tool('fetch')
.description('Fetch data')
.systemInstruction('Always fetch with the cache disabled.')
.input(z.object({ id: z.string() }))
.handler(async ({ id }) => {
return await Promise.resolve({ data: id });
})
.build();
expect(tool.systemInstruction).toBe('Always fetch with the cache disabled.');
});
it('build() leaves systemInstruction undefined when not set', () => {
const tool = new Tool('fetch')
.description('Fetch data')
.input(z.object({ id: z.string() }))
.handler(async ({ id }) => {
return await Promise.resolve({ data: id });
})
.build();
expect(tool.systemInstruction).toBeUndefined();
});
});
// ---------------------------------------------------------------------------
// wrapToolForApproval — requireApproval: true
// ---------------------------------------------------------------------------

View File

@ -1,217 +0,0 @@
import type prettier from 'prettier';
import type {
AgentSchema,
EvalSchema,
GuardrailSchema,
MemorySchema,
ToolSchema,
} from '../types/sdk/schema';
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function escapeTemplateLiteral(str: string): string {
return str.replace(/\\/g, '\\\\').replace(/`/g, '\\`').replace(/\$/g, '\\$');
}
function escapeSingleQuote(str: string): string {
return JSON.stringify(str).slice(1, -1).replace(/'/g, "\\'");
}
let prettierInstance: typeof prettier | undefined;
/**
* Format TypeScript source code using Prettier.
* Loaded lazily to avoid startup cost when not generating code.
*/
async function formatCode(code: string): Promise<string> {
prettierInstance ??= await import('prettier');
return await prettierInstance.format(code, {
parser: 'typescript',
singleQuote: true,
useTabs: true,
trailingComma: 'all',
printWidth: 100,
});
}
/**
* Compile-time exhaustive check. If a new property is added to AgentSchema
* but not handled in generateAgentCode(), TypeScript will report an error
* here because the destructured rest object won't be empty.
*/
function assertAllHandled(_: Record<string, never>): void {
// intentionally empty — this is a compile-time-only check
}
// ---------------------------------------------------------------------------
// Section builders — each returns `.method(...)` chain fragments
// ---------------------------------------------------------------------------
function modelParts(model: AgentSchema['model']): string[] {
if (model.provider && model.name) {
return [`.model('${escapeSingleQuote(model.provider)}', '${escapeSingleQuote(model.name)}')`];
}
if (model.name) {
return [`.model('${escapeSingleQuote(model.name)}')`];
}
return [];
}
function toolPart(tool: ToolSchema): { part: string; usesWorkflowTool: boolean } {
if (!tool.editable) {
return {
part: `.tool(new WorkflowTool('${escapeSingleQuote(tool.name)}'))`,
usesWorkflowTool: true,
};
}
const parts = [`new Tool('${escapeSingleQuote(tool.name)}')`];
parts.push(`.description('${escapeSingleQuote(tool.description)}')`);
if (tool.inputSchemaSource) parts.push(`.input(${tool.inputSchemaSource})`);
if (tool.outputSchemaSource) parts.push(`.output(${tool.outputSchemaSource})`);
if (tool.suspendSchemaSource) parts.push(`.suspend(${tool.suspendSchemaSource})`);
if (tool.resumeSchemaSource) parts.push(`.resume(${tool.resumeSchemaSource})`);
if (tool.handlerSource) parts.push(`.handler(${tool.handlerSource})`);
if (tool.toMessageSource) parts.push(`.toMessage(${tool.toMessageSource})`);
if (tool.requireApproval) parts.push('.requireApproval()');
if (tool.needsApprovalFnSource) parts.push(`.needsApprovalFn(${tool.needsApprovalFnSource})`);
return { part: `.tool(${parts.join('')})`, usesWorkflowTool: false };
}
function evalPart(ev: EvalSchema): string {
const parts = [`new Eval('${escapeSingleQuote(ev.name)}')`];
if (ev.description) parts.push(`.description('${escapeSingleQuote(ev.description)}')`);
if (ev.modelId) parts.push(`.model('${escapeSingleQuote(ev.modelId)}')`);
if (ev.credentialName) parts.push(`.credential('${escapeSingleQuote(ev.credentialName)}')`);
if (ev.handlerSource) {
parts.push(ev.type === 'check' ? `.check(${ev.handlerSource})` : `.judge(${ev.handlerSource})`);
}
return `.eval(${parts.join('')})`;
}
function guardrailPart(g: GuardrailSchema): string {
const method = g.position === 'input' ? 'inputGuardrail' : 'outputGuardrail';
return `.${method}(${g.source})`;
}
function memoryPart(memory: MemorySchema): string {
if (memory.source) {
return `.memory(${memory.source})`;
}
return `.memory(new Memory().lastMessages(${memory.lastMessages ?? 10}))`;
}
function thinkingPart(thinking: NonNullable<AgentSchema['config']['thinking']>): string {
const props: string[] = [];
if (thinking.budgetTokens !== undefined) props.push(`budgetTokens: ${thinking.budgetTokens}`);
if (thinking.reasoningEffort) props.push(`reasoningEffort: '${thinking.reasoningEffort}'`);
if (props.length > 0) {
return `.thinking('${thinking.provider}', { ${props.join(', ')} })`;
}
return `.thinking('${thinking.provider}')`;
}
function buildImports(schema: AgentSchema, needsWorkflowTool: boolean): string {
const agentImports = new Set<string>(['Agent']);
if (schema.tools.some((t) => t.editable)) agentImports.add('Tool');
if (needsWorkflowTool) agentImports.add('WorkflowTool');
if (schema.memory) agentImports.add('Memory');
if (schema.mcp && schema.mcp.length > 0) agentImports.add('McpClient');
if (schema.evaluations.length > 0) agentImports.add('Eval');
const toolsNeedZod = schema.tools.some(
(t) =>
(t.inputSchemaSource?.includes('z.') ?? false) ||
(t.outputSchemaSource?.includes('z.') ?? false),
);
const structuredOutputNeedsZod =
schema.config.structuredOutput.schemaSource?.includes('z.') ?? false;
let imports = `import { ${Array.from(agentImports).sort().join(', ')} } from '@n8n/agents';`;
if (toolsNeedZod || structuredOutputNeedsZod) imports += "\nimport { z } from 'zod';";
return imports;
}
// ---------------------------------------------------------------------------
// Public API
// ---------------------------------------------------------------------------
export async function generateAgentCode(schema: AgentSchema, agentName: string): Promise<string> {
// Destructure every top-level property. If a new property is added to
// AgentSchema, TypeScript will error on assertAllHandled below until
// you handle it here AND add it to the destructure.
const {
model,
credential,
instructions,
description: _description, // entity-level, not in code
tools,
providerTools,
memory,
evaluations,
guardrails,
mcp,
telemetry,
checkpoint,
config,
...rest
} = schema;
// If this errors, you added a property to AgentSchema but didn't
// destructure it above. Add it to the destructure and handle it below.
assertAllHandled(rest);
const { thinking, toolCallConcurrency, requireToolApproval, structuredOutput, ...configRest } =
config;
assertAllHandled(configRest);
// No manual indentation — Prettier formats at the end.
const parts: string[] = [];
let needsWorkflowTool = false;
parts.push(`export default new Agent('${escapeSingleQuote(agentName)}')`);
parts.push(...modelParts(model));
if (credential) parts.push(`.credential('${escapeSingleQuote(credential)}')`);
if (instructions) parts.push(`.instructions(\`${escapeTemplateLiteral(instructions)}\`)`);
for (const tool of tools) {
const { part, usesWorkflowTool } = toolPart(tool);
if (usesWorkflowTool) needsWorkflowTool = true;
parts.push(part);
}
for (const pt of providerTools) {
parts.push(`.providerTool(${pt.source})`);
}
if (memory) parts.push(memoryPart(memory));
for (const ev of evaluations) {
parts.push(evalPart(ev));
}
for (const g of guardrails) {
parts.push(guardrailPart(g));
}
if (mcp && mcp.length > 0) {
const configs = mcp.map((s) => s.configSource).join(', ');
parts.push(`.mcp(new McpClient([${configs}]))`);
}
if (telemetry) parts.push(`.telemetry(${telemetry.source})`);
if (checkpoint) parts.push(`.checkpoint('${escapeSingleQuote(checkpoint)}')`);
if (thinking) parts.push(thinkingPart(thinking));
if (toolCallConcurrency) parts.push(`.toolCallConcurrency(${toolCallConcurrency})`);
if (requireToolApproval) parts.push('.requireToolApproval()');
if (structuredOutput.enabled && structuredOutput.schemaSource) {
parts.push(`.structuredOutput(${structuredOutput.schemaSource})`);
}
const imports = buildImports(schema, needsWorkflowTool);
const raw = `${imports}\n\n${parts.join('')};\n`;
return await formatCode(raw);
}

View File

@ -28,6 +28,7 @@ export type {
SerializableAgentState,
AgentRunState,
MemoryConfig,
MemoryDescriptor,
TitleGenerationConfig,
Thread,
SemanticRecallConfig,
@ -44,7 +45,7 @@ export type { ProviderOptions } from '@ai-sdk/provider-utils';
export { AgentEvent } from './types';
export type { AgentEventData, AgentEventHandler } from './types';
export { Tool } from './sdk/tool';
export { Tool, wrapToolForApproval } from './sdk/tool';
export { Memory } from './sdk/memory';
export { Guardrail } from './sdk/guardrail';
export { Eval } from './sdk/eval';
@ -55,6 +56,7 @@ export { Telemetry } from './sdk/telemetry';
export { LangSmithTelemetry } from './integrations/langsmith';
export type { LangSmithTelemetryConfig } from './integrations/langsmith';
export { Agent } from './sdk/agent';
export type { AgentSnapshot } from './sdk/agent';
export type {
AgentBuilder,
CredentialProvider,
@ -73,7 +75,6 @@ export type {
ContentReasoning,
ContentText,
ContentToolCall,
ContentToolResult,
Message,
MessageContent,
MessageRole,
@ -82,19 +83,10 @@ export type {
AgentDbMessage,
} from './types/sdk/message';
export type { HandlerExecutor } from './types/sdk/handler-executor';
export type {
AgentSchema,
ToolSchema,
MemorySchema,
EvalSchema,
ThinkingSchema,
ProviderToolSchema,
GuardrailSchema,
McpServerSchema,
TelemetrySchema,
} from './types/sdk/schema';
export { generateAgentCode } from './codegen/generate-agent-code';
export { filterLlmMessages, isLlmMessage } from './sdk/message';
export {
filterLlmMessages,
isLlmMessage,
} from './sdk/message';
export { fetchProviderCatalog } from './sdk/catalog';
export { providerCapabilities } from './sdk/provider-capabilities';
export type { ProviderCapability } from './sdk/provider-capabilities';
@ -105,14 +97,19 @@ export type {
ModelCost,
ModelLimits,
} from './sdk/catalog';
export { SqliteMemory } from './storage/sqlite-memory';
export { SqliteMemory, SqliteMemoryConfigSchema } from './storage/sqlite-memory';
export {
UPDATE_WORKING_MEMORY_TOOL_NAME,
WORKING_MEMORY_DEFAULT_INSTRUCTION,
} from './runtime/working-memory';
export type { SqliteMemoryConfig } from './storage/sqlite-memory';
export { PostgresMemory } from './storage/postgres-memory';
export type { PostgresMemoryConfig } from './storage/postgres-memory';
export type {
PostgresConnectionOptions,
PostgresConstructorOptions,
} from './storage/postgres-memory';
export { BaseMemory } from './storage/base-memory';
export type { ToolDescriptor } from './types/sdk/tool-descriptor';
export { createModel } from './runtime/model-factory';
export { generateTitleFromMessage } from './runtime/title-generation';
@ -151,3 +148,7 @@ export type {
SpawnProcessOptions,
ProcessInfo,
} from './workspace';
export type { JSONObject, JSONArray, JSONValue } from './types/utils/json';
export { isZodSchema, zodToJsonSchema } from './utils/zod';

File diff suppressed because it is too large Load Diff

View File

@ -32,6 +32,10 @@ export class AgentEventBus {
set.add(handler);
}
off(event: AgentEvent, handler: AgentEventHandler): void {
this.handlers.get(event)?.delete(handler);
}
emit(data: AgentEventData): void {
const set = this.handlers.get(data.type);
if (!set) return;

View File

@ -1,4 +1,4 @@
import type { BuiltMemory, Thread } from '../types';
import type { BuiltMemory, MemoryDescriptor, Thread } from '../types';
import type { AgentDbMessage } from '../types/sdk/message';
interface StoredMessage {
@ -78,6 +78,8 @@ export class InMemoryMemory implements BuiltMemory {
/**
* Save messages to the thread established by the most recent `saveThread` call.
* Always call `saveThread` before `saveMessages` to set the thread context.
* Upserts by message id if a message with the same id already exists, it is
* replaced in place (preserving insertion order). New messages are appended.
*/
// eslint-disable-next-line @typescript-eslint/require-await
async saveMessages(args: {
@ -86,8 +88,16 @@ export class InMemoryMemory implements BuiltMemory {
messages: AgentDbMessage[];
}): Promise<void> {
const existing = this.messagesByThread.get(args.threadId) ?? [];
const byId = new Map(existing.map((s, i) => [s.message.id, i]));
for (const msg of args.messages) {
existing.push({ message: msg, createdAt: msg.createdAt });
const entry: StoredMessage = { message: msg, createdAt: msg.createdAt };
const idx = byId.get(msg.id);
if (idx !== undefined) {
existing[idx] = entry;
} else {
byId.set(msg.id, existing.length);
existing.push(entry);
}
}
this.messagesByThread.set(args.threadId, existing);
}
@ -102,6 +112,10 @@ export class InMemoryMemory implements BuiltMemory {
);
}
}
describe(): MemoryDescriptor {
return { name: 'memory', constructorName: this.constructor.name, connectionParams: {} };
}
}
/**

View File

@ -2,11 +2,13 @@ import type { ProviderOptions } from '@ai-sdk/provider-utils';
import type { ModelMessage } from 'ai';
import { toAiMessages } from './messages';
import { stringifyError } from './runtime-helpers';
import { stripOrphanedToolMessages } from './strip-orphaned-tool-messages';
import { buildWorkingMemoryInstruction } from './working-memory';
import { filterLlmMessages, getCreatedAt } from '../sdk/message';
import type { SerializedMessageList } from '../types/runtime/message-list';
import type { AgentDbMessage, AgentMessage } from '../types/sdk/message';
import type { AgentDbMessage, AgentMessage, ContentToolCall } from '../types/sdk/message';
import type { JSONValue } from '../types/utils/json';
export type { SerializedMessageList };
@ -134,6 +136,76 @@ export class AgentMessageList {
this.sortAllByCreatedAt();
}
/**
* Locate the assistant message hosting the given toolCallId and mark the
* block as resolved with the supplied output.
*
* Returns the mutated host message, or `undefined` if the toolCallId is
* not found (internal invariant violation caller should log/throw).
*/
setToolCallResult(toolCallId: string, output: JSONValue): AgentDbMessage | undefined {
const host = this.findToolCallHost(toolCallId);
if (!host) return undefined;
const block = this.findToolCallBlock(host, toolCallId);
if (!block) return undefined;
const mutableBlock = block;
mutableBlock.state = 'resolved';
(mutableBlock as Extract<ContentToolCall, { state: 'resolved' }>).output = output;
if ('error' in mutableBlock) {
delete (mutableBlock as { error: unknown }).error;
}
this.responseSet.add(host);
return host;
}
/**
* Locate the assistant message hosting the given toolCallId and mark the
* block as rejected with the supplied error.
*
* Returns the mutated host message, or `undefined` if the toolCallId is
* not found (internal invariant violation caller should log/throw).
*/
setToolCallError(toolCallId: string, error: unknown): AgentDbMessage | undefined {
const host = this.findToolCallHost(toolCallId);
if (!host) return undefined;
const block = this.findToolCallBlock(host, toolCallId)!;
const mutableBlock = block;
mutableBlock.state = 'rejected';
(mutableBlock as Extract<ContentToolCall, { state: 'rejected' }>).error = stringifyError(error);
if ('output' in mutableBlock) {
delete (mutableBlock as { output: unknown }).output;
}
this.responseSet.add(host);
return host;
}
private findToolCallHost(toolCallId: string): AgentDbMessage | undefined {
// Start from the last message and go backwards to find the host message
for (let i = this.all.length - 1; i >= 0; i--) {
const m = this.all[i];
if (
'content' in m &&
Array.isArray(m.content) &&
m.content.some((c) => c.type === 'tool-call' && c.toolCallId === toolCallId)
) {
return m;
}
}
return undefined;
}
private findToolCallBlock(host: AgentDbMessage, toolCallId: string): ContentToolCall | undefined {
if (!('content' in host) || !Array.isArray(host.content)) return undefined;
return host.content.find(
(c): c is ContentToolCall => c.type === 'tool-call' && c.toolCallId === toolCallId,
);
}
/**
* Full LLM context for a generateText / streamText call.
* Prepends the system prompt (with working memory appended if configured),

View File

@ -17,7 +17,6 @@ import type {
ContentReasoning,
ContentText,
ContentToolCall,
ContentToolResult,
Message,
MessageContent,
} from '../types/sdk/message';
@ -54,10 +53,6 @@ function isToolCall(block: MessageContent): block is ContentToolCall {
return block.type === 'tool-call';
}
function isToolResult(block: MessageContent): block is ContentToolResult {
return block.type === 'tool-result';
}
/**
* Parse a JSONValue that may be a stringified JSON object back into
* its parsed form. Non-string values pass through unchanged.
@ -92,32 +87,6 @@ function toAiContent(block: MessageContent): AiContentPart | undefined {
input: parseJsonValue(block.input),
providerExecuted: block.providerExecuted,
};
}
if (isToolResult(block)) {
if (block.isError) {
if (typeof block.result === 'string') {
base = {
type: 'tool-result',
toolCallId: block.toolCallId,
toolName: block.toolName,
output: { type: 'error-text', value: block.result },
};
} else {
base = {
type: 'tool-result',
toolCallId: block.toolCallId,
toolName: block.toolName,
output: { type: 'error-json', value: block.result },
};
}
} else {
base = {
type: 'tool-result',
toolCallId: block.toolCallId,
toolName: block.toolName,
output: { type: 'json', value: block.result },
};
}
} else if (isReasoning(block)) {
base = { type: 'reasoning', text: block.text };
}
@ -128,6 +97,36 @@ function toAiContent(block: MessageContent): AiContentPart | undefined {
return base;
}
/** Build an AI SDK ToolResultPart from a resolved/rejected ContentToolCall. */
function toolCallToResultPart(
block: ContentToolCall & { state: 'resolved' | 'rejected' },
): ToolResultPart {
if (block.state === 'resolved') {
return {
type: 'tool-result',
toolCallId: block.toolCallId,
toolName: block.toolName,
output: { type: 'json', value: block.output },
};
}
// rejected
const errorValue = block.error;
if (typeof errorValue === 'string') {
return {
type: 'tool-result',
toolCallId: block.toolCallId,
toolName: block.toolName,
output: { type: 'error-text', value: errorValue },
};
}
return {
type: 'tool-result',
toolCallId: block.toolCallId,
toolName: block.toolName,
output: { type: 'error-json', value: errorValue as JSONValue },
};
}
/** Convert a single AI SDK content part to an n8n MessageContent block. */
function fromAiContent(part: AiContentPart): MessageContent | undefined {
const providerOptions = 'providerOptions' in part ? part.providerOptions : undefined;
@ -159,35 +158,11 @@ function fromAiContent(part: AiContentPart): MessageContent | undefined {
toolName: part.toolName,
input: part.input as JSONValue,
providerExecuted: part.providerExecuted,
state: 'pending',
};
break;
case 'tool-result': {
const { output } = part;
let result: JSONValue;
let isError: boolean | undefined;
if (output.type === 'json') {
result = output.value;
} else if (output.type === 'text') {
result = output.value;
} else if (output.type === 'error-json') {
result = output.value;
isError = true;
} else if (output.type === 'error-text') {
result = output.value;
isError = true;
} else {
result = null;
isError = true;
}
base = {
type: 'tool-result',
toolCallId: part.toolCallId,
toolName: part.toolName,
result,
isError,
};
break;
}
case 'tool-result':
return undefined;
// Ignore these types, because HITL is handled by our runtime
case 'tool-approval-request':
case 'tool-approval-response':
@ -201,82 +176,172 @@ function fromAiContent(part: AiContentPart): MessageContent | undefined {
return base;
}
/** Convert a single n8n Message to an AI SDK ModelMessage. */
export function toAiMessage(msg: Message): ModelMessage {
let base: ModelMessage;
/**
* Convert a single n8n Message to one or more AI SDK ModelMessages.
*
* For assistant messages with resolved/rejected tool-call blocks, this emits:
* 1. The assistant ModelMessage (tool-call parts only, no result fields)
* 2. One tool ModelMessage per settled tool-call block (resolved or rejected)
*
* Pending tool-call blocks are silently skipped (defense-in-depth; the strip
* step should already have removed them before forLlm() calls toAiMessages).
*/
function toAiMessageList(msg: Message): ModelMessage[] {
switch (msg.role) {
case 'system': {
const text = msg.content
.filter(isText)
.map((b) => b.text)
.join('');
base = { role: 'system', content: text };
break;
const base: ModelMessage = { role: 'system', content: text };
return [msg.providerOptions ? { ...base, providerOptions: msg.providerOptions } : base];
}
case 'user': {
const parts = msg.content
.map(toAiContent)
.filter((p): p is TextPart | FilePart => p?.type === 'text' || p?.type === 'file');
base = { role: 'user', content: parts };
break;
const base: ModelMessage = { role: 'user', content: parts };
return [msg.providerOptions ? { ...base, providerOptions: msg.providerOptions } : base];
}
case 'assistant': {
const parts = msg.content
.map(toAiContent)
.filter(
(p): p is TextPart | ReasoningPart | ToolCallPart | ToolResultPart | FilePart =>
p?.type === 'text' ||
p?.type === 'reasoning' ||
p?.type === 'tool-call' ||
p?.type === 'tool-result' ||
p?.type === 'file',
);
base = { role: 'assistant', content: parts };
break;
const assistantParts: AiContentPart[] = [];
const resultMessages: ModelMessage[] = [];
for (const block of msg.content) {
if (block.type === 'tool-call') {
if (!('state' in block)) {
// Legacy DB block - skip it
continue;
}
if (block.state === 'pending') {
// Skip pending blocks — defense-in-depth (strip step removes them first)
continue;
}
// Emit tool-call part (without result fields)
assistantParts.push({
type: 'tool-call',
toolCallId: block.toolCallId,
toolName: block.toolName,
input: parseJsonValue(block.input),
providerExecuted: block.providerExecuted,
});
// Emit corresponding tool-result message immediately after
const resultPart = toolCallToResultPart(block);
resultMessages.push({ role: 'tool', content: [resultPart] });
} else {
const part = toAiContent(block);
if (part) assistantParts.push(part);
}
}
const transformedMessages: ModelMessage[] = [];
if (assistantParts.length > 0) {
const assistantBase: ModelMessage = {
role: 'assistant',
content: assistantParts as Array<
TextPart | ReasoningPart | ToolCallPart | ToolResultPart | FilePart
>,
};
const assistantMsg: ModelMessage = msg.providerOptions
? { ...assistantBase, providerOptions: msg.providerOptions }
: assistantBase;
transformedMessages.push(assistantMsg);
}
if (resultMessages.length > 0) {
transformedMessages.push(...resultMessages);
}
return transformedMessages;
}
case 'tool': {
const parts = msg.content
.map(toAiContent)
.filter((p): p is ToolResultPart => p?.type === 'tool-result');
base = { role: 'tool', content: parts };
break;
// Legacy role: 'tool' messages (from old DB rows). Don't emit them.
return [];
}
default:
throw new Error(`Unknown role: ${msg.role as string}`);
}
if (msg.providerOptions) {
return { ...base, providerOptions: msg.providerOptions };
}
return base;
}
/** Convert n8n Messages to AI SDK ModelMessages for passing to stream/generateText. */
export function toAiMessages(messages: Message[]): ModelMessage[] {
return messages.map(toAiMessage);
return messages.flatMap(toAiMessageList);
}
/** Convert a single AI SDK ModelMessage to an n8n AgentDbMessage (with a generated id). */
export function fromAiMessage(msg: ModelMessage): AgentMessage {
const rawContent = msg.content;
const content: MessageContent[] =
typeof rawContent === 'string'
? [{ type: 'text', text: rawContent }]
: rawContent.map(fromAiContent).filter((p): p is MessageContent => p !== undefined);
const message: AgentMessage = { role: msg.role, content };
if ('providerOptions' in msg && msg.providerOptions) {
message.providerOptions = msg.providerOptions;
}
return message;
}
/** Convert AI SDK ModelMessages to n8n AgentDbMessages (each with a generated id). */
/**
* Convert AI SDK ModelMessages to n8n AgentMessages.
*
* This is a stateful walk: when a role:'tool' ModelMessage is encountered,
* the matching tool-call block on the preceding assistant message is mutated
* to 'resolved' or 'rejected'. The tool message itself is not emitted as a
* separate n8n message.
*
* If a tool-result references a toolCallId not in the index (orphan), it is
* silently dropped.
*/
export function fromAiMessages(messages: ModelMessage[]): AgentMessage[] {
return messages.map(fromAiMessage);
// Map from toolCallId → ContentToolCall block (mutable ref inside the n8n message)
const toolCallIndex = new Map<string, ContentToolCall>();
const result: AgentMessage[] = [];
for (const msg of messages) {
if (msg.role === 'tool') {
// Merge tool results back into the matching tool-call blocks
const toolParts = msg.content as ToolResultPart[];
for (const part of toolParts) {
const block = toolCallIndex.get(part.toolCallId);
if (!block) continue; // orphan — drop
const { output } = part;
if (output.type === 'json' || output.type === 'text') {
const mutableBlock = block as Extract<ContentToolCall, { state: 'resolved' }>;
mutableBlock.state = 'resolved';
mutableBlock.output = output.value as JSONValue;
} else if (output.type === 'error-json') {
const mutableBlock = block as Extract<ContentToolCall, { state: 'rejected' }>;
mutableBlock.state = 'rejected';
mutableBlock.error = JSON.stringify(output.value);
} else if (output.type === 'error-text') {
const mutableBlock = block as Extract<ContentToolCall, { state: 'rejected' }>;
mutableBlock.state = 'rejected';
mutableBlock.error = output.value;
} else {
const mutableBlock = block as Extract<ContentToolCall, { state: 'rejected' }>;
mutableBlock.state = 'rejected';
mutableBlock.error = JSON.stringify(output);
}
}
// Do not emit a separate n8n message for tool results
continue;
}
const rawContent = msg.content;
const content: MessageContent[] =
typeof rawContent === 'string'
? [{ type: 'text', text: rawContent }]
: rawContent.map(fromAiContent).filter((p): p is MessageContent => p !== undefined);
const agentMsg: AgentMessage = { role: msg.role, content };
if ('providerOptions' in msg && msg.providerOptions) {
agentMsg.providerOptions = msg.providerOptions;
}
result.push(agentMsg);
// Index any tool-call blocks for later merging with tool-result messages
if (msg.role === 'assistant') {
for (const block of content) {
if (block.type === 'tool-call' && block.toolCallId) {
toolCallIndex.set(block.toolCallId, block);
}
}
}
}
return result;
}
export function fromAiFinishReason(reason: AiFinishReason): FinishReason {

View File

@ -1,16 +1,16 @@
/* eslint-disable @typescript-eslint/consistent-type-imports */
/* eslint-disable @typescript-eslint/no-require-imports */
import type { EmbeddingModel, LanguageModel } from 'ai';
import type * as Undici from 'undici';
import {
PROVIDER_CREDENTIAL_SCHEMAS,
type ProviderId,
type ProviderCredentials,
} from './provider-credentials';
import type { ModelConfig } from '../types/sdk/agent';
type FetchFn = typeof globalThis.fetch;
type CreateProviderFn = (opts?: {
apiKey?: string;
baseURL?: string;
fetch?: FetchFn;
headers?: Record<string, string>;
}) => (model: string) => LanguageModel;
type CreateEmbeddingProviderFn = (opts?: { apiKey?: string }) => {
embeddingModel(model: string): EmbeddingModel;
};
@ -39,6 +39,124 @@ function getProxyFetch(): FetchFn | undefined {
})) as FetchFn;
}
type EntryBuilder<P extends ProviderId> = (
creds: ProviderCredentials<P>,
modelName: string,
fetch: FetchFn | undefined,
) => LanguageModel;
type RegistryEntry<P extends ProviderId = ProviderId> = {
build: EntryBuilder<P>;
};
type ProviderRegistry = {
[P in ProviderId]: RegistryEntry<P>;
};
/**
* Registry of language model providers.
* Each entry maps a provider id to a builder that loads its @ai-sdk/* package
* and instantiates the model. Credentials are Zod-validated before being passed in.
*/
const LANGUAGE_PROVIDERS: ProviderRegistry = {
openai: {
build: (creds, model, fetch) => {
const { createOpenAI } = require('@ai-sdk/openai') as typeof import('@ai-sdk/openai');
return createOpenAI({ ...creds, fetch })(model);
},
},
anthropic: {
build: (creds, model, fetch) => {
const { createAnthropic } =
require('@ai-sdk/anthropic') as typeof import('@ai-sdk/anthropic');
return createAnthropic({ ...creds, fetch })(model);
},
},
google: {
build: (creds, model, fetch) => {
const { createGoogleGenerativeAI } =
require('@ai-sdk/google') as typeof import('@ai-sdk/google');
return createGoogleGenerativeAI({ ...creds, fetch })(model);
},
},
xai: {
build: (creds, model, fetch) => {
const { createXai } = require('@ai-sdk/xai') as typeof import('@ai-sdk/xai');
return createXai({ ...creds, fetch })(model);
},
},
groq: {
build: (creds, model, fetch) => {
const { createGroq } = require('@ai-sdk/groq') as typeof import('@ai-sdk/groq');
return createGroq({ ...creds, fetch })(model);
},
},
deepseek: {
build: (creds, model, fetch) => {
const { createDeepSeek } = require('@ai-sdk/deepseek') as typeof import('@ai-sdk/deepseek');
return createDeepSeek({ ...creds, fetch })(model);
},
},
cohere: {
build: (creds, model, fetch) => {
const { createCohere } = require('@ai-sdk/cohere') as typeof import('@ai-sdk/cohere');
return createCohere({ ...creds, fetch })(model);
},
},
mistral: {
build: (creds, model, fetch) => {
const { createMistral } = require('@ai-sdk/mistral') as typeof import('@ai-sdk/mistral');
return createMistral({ ...creds, fetch })(model);
},
},
vercel: {
build: (creds, model, fetch) => {
const { createGateway } = require('@ai-sdk/gateway') as typeof import('@ai-sdk/gateway');
return createGateway({ ...creds, fetch })(model);
},
},
openrouter: {
build: (creds, model, fetch) => {
const { createOpenRouter } =
require('@openrouter/ai-sdk-provider') as typeof import('@openrouter/ai-sdk-provider');
return createOpenRouter({ apiKey: creds.apiKey, baseURL: creds.baseURL, fetch })(model);
},
},
'azure-openai': {
build: (creds, model, fetch) => {
const { createAzure } = require('@ai-sdk/azure') as typeof import('@ai-sdk/azure');
const { baseURL, resourceName, apiVersion, apiKey } = creds;
let normalizedBaseURL = baseURL;
// SDK expects url like `https://resourceName.openai.azure.com/openai`
if (normalizedBaseURL) {
const url = new URL(normalizedBaseURL);
if (!url.pathname.endsWith('/openai')) {
url.pathname = url.pathname.replace(/\/?$/, '/openai');
normalizedBaseURL = url.toString();
}
}
return createAzure({ resourceName, apiKey, baseURL: normalizedBaseURL, apiVersion, fetch })(
model,
);
},
},
'aws-bedrock': {
build: (creds, model, fetch) => {
const { createAmazonBedrock } =
require('@ai-sdk/amazon-bedrock') as typeof import('@ai-sdk/amazon-bedrock');
return createAmazonBedrock({
region: creds.region,
accessKeyId: creds.accessKeyId,
secretAccessKey: creds.secretAccessKey,
sessionToken: creds.sessionToken,
fetch,
})(model);
},
},
};
const SUPPORTED_PROVIDERS = Object.keys(LANGUAGE_PROVIDERS).join(', ');
/**
* Provider packages are loaded dynamically via require() so only the
* provider needed at runtime must be installed.
@ -48,55 +166,44 @@ export function createModel(config: ModelConfig): LanguageModel {
return config;
}
const stripEmpty = <T>(value: T | undefined): T | undefined => {
if (!value) return undefined;
if (typeof value === 'string' && value.trim() === '') return undefined;
return value;
};
const modelId = stripEmpty(typeof config === 'string' ? config : config.id);
const apiKey = stripEmpty(typeof config === 'string' ? undefined : config.apiKey);
const baseURL = stripEmpty(typeof config === 'string' ? undefined : config.url);
const headers = typeof config === 'string' ? undefined : config.headers;
if (!modelId) {
const rawId = typeof config === 'string' ? config : config.id;
if (!rawId || rawId.trim() === '') {
throw new Error('Model ID is required');
}
const [provider, ...rest] = modelId.split('/');
const modelName = rest.join('/');
const fetch = getProxyFetch();
switch (provider) {
case 'anthropic': {
const { createAnthropic } = require('@ai-sdk/anthropic') as {
createAnthropic: CreateProviderFn;
};
return createAnthropic({ apiKey, baseURL, fetch, headers })(modelName);
}
case 'openai': {
const { createOpenAI } = require('@ai-sdk/openai') as {
createOpenAI: CreateProviderFn;
};
return createOpenAI({ apiKey, baseURL, fetch, headers })(modelName);
}
case 'google': {
const { createGoogleGenerativeAI } = require('@ai-sdk/google') as {
createGoogleGenerativeAI: CreateProviderFn;
};
return createGoogleGenerativeAI({ apiKey, baseURL, fetch, headers })(modelName);
}
case 'xai': {
const { createXai } = require('@ai-sdk/xai') as {
createXai: CreateProviderFn;
};
return createXai({ apiKey, baseURL, fetch, headers })(modelName);
}
default:
throw new Error(
`Unsupported provider: "${provider}". Supported: anthropic, openai, google, xai`,
);
const slashIndex = rawId.indexOf('/');
if (slashIndex <= 0) {
throw new Error(`Invalid model ID "${rawId}": expected "provider/model-name" format`);
}
const provider = rawId.slice(0, slashIndex) as ProviderId;
const modelName = rawId.slice(slashIndex + 1);
const entry = LANGUAGE_PROVIDERS[provider];
if (!entry) {
throw new Error(
`Unsupported provider: "${provider}". Supported providers: ${SUPPORTED_PROVIDERS}`,
);
}
// Collect credential fields: strip `id`, pass the rest to Zod validation.
let credFields: Record<string, unknown> = {};
if (typeof config !== 'string') {
const { id: _id, ...rest } = config as { id: string; [k: string]: unknown };
credFields = rest;
}
const schema = PROVIDER_CREDENTIAL_SCHEMAS[provider];
const parsed = schema.safeParse(credFields);
if (!parsed.success) {
const issues = parsed.error.issues
.map((i) => ` - ${i.path.join('.')}: ${i.message}`)
.join('\n');
throw new Error(`Invalid credentials for provider "${provider}":\n${issues}`);
}
const fetch = getProxyFetch();
// Type cast: the registry guarantees the schema and builder are aligned per provider.
return (entry.build as EntryBuilder<typeof provider>)(parsed.data as never, modelName, fetch);
}
/**

View File

@ -0,0 +1,42 @@
import { z } from 'zod';
const apiKeyCreds = z.object({
apiKey: z.string().optional(),
baseURL: z.string().optional(),
headers: z.record(z.string(), z.string()).optional(),
});
/**
* Per-provider Zod schemas for credential validation.
* Keys are the provider prefixes used in model IDs (e.g. 'anthropic' in 'anthropic/claude-sonnet-4-5').
*/
export const PROVIDER_CREDENTIAL_SCHEMAS = {
openai: apiKeyCreds,
anthropic: apiKeyCreds,
google: apiKeyCreds,
xai: apiKeyCreds,
groq: apiKeyCreds,
deepseek: apiKeyCreds,
cohere: apiKeyCreds,
mistral: apiKeyCreds,
vercel: apiKeyCreds,
openrouter: apiKeyCreds,
'azure-openai': z.object({
apiKey: z.string().optional(),
resourceName: z.string().min(1, 'Azure resourceName is required'),
apiVersion: z.string().optional(),
baseURL: z.string().optional(),
}),
'aws-bedrock': z.object({
region: z.string().min(1, 'AWS region is required'),
accessKeyId: z.string().min(1, 'AWS accessKeyId is required'),
secretAccessKey: z.string().min(1, 'AWS secretAccessKey is required'),
sessionToken: z.string().optional(),
}),
} as const;
export type ProviderId = keyof typeof PROVIDER_CREDENTIAL_SCHEMAS;
export type ProviderCredentials<P extends ProviderId> = z.infer<
(typeof PROVIDER_CREDENTIAL_SCHEMAS)[P]
>;

View File

@ -4,8 +4,7 @@
*/
import type { GenerateResult, StreamChunk, TokenUsage } from '../types';
import { toTokenUsage } from './stream';
import type { AgentMessage, ContentToolResult } from '../types/sdk/message';
import type { JSONValue } from '../types/utils/json';
import type { AgentMessage, ContentToolCall } from '../types/sdk/message';
/**
* Normalize caller input to `AgentMessage[]` for the runtime. String input becomes a
@ -18,55 +17,16 @@ export function normalizeInput(input: AgentMessage[] | string): AgentMessage[] {
return input;
}
/** Build an AI SDK tool ModelMessage for a tool execution result. */
export function makeToolResultMessage(
toolCallId: string,
toolName: string,
result: unknown,
): AgentMessage {
return {
role: 'tool',
content: [
{
type: 'tool-result',
toolCallId,
toolName,
result: result as JSONValue,
},
],
};
/** Stringify an error value for use in a rejected tool-call block. */
export function stringifyError(error: unknown): string {
return error instanceof Error ? `${error.name}: ${error.message}` : String(error);
}
/**
* Build an AI SDK tool ModelMessage for a tool execution error.
* The LLM receives this as a tool result so it can self-correct on the next iteration.
* The error is surfaced via the output json value so the LLM can read and reason about it.
*/
export function makeErrorToolResultMessage(
toolCallId: string,
toolName: string,
error: unknown,
): AgentMessage {
const message = error instanceof Error ? `${error.name}: ${error.message}` : String(error);
return {
role: 'tool',
content: [
{
type: 'tool-result',
toolCallId,
toolName,
result: { error: message } as JSONValue,
isError: true,
},
],
};
}
/** Extract all tool-result content parts from a flat list of agent messages. */
export function extractToolResults(messages: AgentMessage[]): ContentToolResult[] {
/** Extract all settled (resolved or rejected) tool-call blocks from a flat list of agent messages. */
export function extractSettledToolCalls(messages: AgentMessage[]): ContentToolCall[] {
return messages
.flatMap((m) => ('content' in m ? m.content : []))
.filter((c): c is ContentToolResult => c.type === 'tool-result');
.filter((c): c is ContentToolCall => c.type === 'tool-call' && c.state !== 'pending');
}
/**

View File

@ -39,72 +39,62 @@ export function toTokenUsage(
return result;
}
/** Convert a single AI SDK v6 fullStream chunk to an n8n StreamChunk (or undefined to skip). */
/**
* Convert a single AI SDK v6 fullStream chunk to an n8n StreamChunk
*/
export function convertChunk(c: TextStreamPart<ToolSet>): StreamChunk | undefined {
switch (c.type) {
case 'start-step':
return { type: 'start-step' };
case 'finish-step':
return { type: 'finish-step' };
case 'text-start':
return { type: 'text-start', id: c.id };
case 'text-delta':
return { type: 'text-delta', delta: c.text ?? '' };
return { type: 'text-delta', id: c.id, delta: c.text ?? '' };
case 'text-end':
return { type: 'text-end', id: c.id };
case 'reasoning-start':
return { type: 'reasoning-start', id: c.id };
case 'reasoning-delta':
return { type: 'reasoning-delta', delta: c.text ?? '' };
return { type: 'reasoning-delta', id: c.id, delta: c.text ?? '' };
case 'reasoning-end':
return { type: 'reasoning-end', id: c.id };
case 'tool-input-start':
// AI SDK uses `id` to carry the toolCallId on tool-input-* chunks.
return { type: 'tool-input-start', toolCallId: c.id, toolName: c.toolName };
case 'tool-input-delta':
return { type: 'tool-input-delta', toolCallId: c.id, delta: c.delta };
case 'tool-call':
return {
type: 'message',
message: {
role: 'tool',
content: [
{
type: 'tool-call',
toolCallId: c.toolCallId,
toolName: c.toolName ?? '',
input: c.input as JSONValue,
},
],
},
};
case 'tool-input-start':
return {
type: 'tool-call-delta',
name: c.toolName,
};
case 'tool-input-delta':
return {
type: 'tool-call-delta',
...(c.delta !== undefined && { argumentsDelta: c.delta }),
type: 'tool-call',
toolCallId: c.toolCallId,
toolName: c.toolName ?? '',
input: c.input as JSONValue,
};
case 'tool-result':
return {
type: 'message',
message: {
role: 'tool',
content: [
{
type: 'tool-result',
toolCallId: c.toolCallId ?? '',
toolName: c.toolName ?? '',
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
result: c.output && 'value' in c.output ? (c.output.value as JSONValue) : null,
},
],
},
type: 'tool-result',
toolCallId: c.toolCallId ?? '',
toolName: c.toolName ?? '',
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
output: c.output && 'value' in c.output ? (c.output.value as JSONValue) : null,
};
case 'error':
return { type: 'error', error: c.error };
case 'finish-step': {
const usage = toTokenUsage(c.usage);
return {
type: 'finish',
finishReason: (c.finishReason ?? 'stop') as FinishReason,
...(usage && { usage }),
};
}
case 'finish': {
const usage = toTokenUsage(c.totalUsage);
return {

View File

@ -2,43 +2,15 @@ import { isLlmMessage } from '../sdk/message';
import type { AgentMessage, MessageContent } from '../types/sdk/message';
/**
* Strip orphaned tool-call and tool-result content from a message list.
*
* When memory loads the last N messages, the window boundary can split
* tool-call / tool-result pairs, leaving one side without its counterpart.
* Sending these orphans to the LLM causes provider errors because tool
* calls and results must always be paired.
* Strip pending tool-call blocks from a message list before sending to the LLM.
*
* This function:
* 1. Collects all toolCallIds present in tool-call and tool-result blocks.
* 2. Identifies orphans calls without a matching result and vice-versa.
* 3. Strips orphaned content blocks from their messages.
* 4. Drops messages that become empty after stripping (e.g. a tool message
* whose only content was the orphaned result).
* 5. Preserves non-tool content (text, reasoning, files) in mixed messages.
* 1. Drops any tool-call block whose state is 'pending'.
* 2. If a message becomes empty after stripping, drops the message entirely.
* 3. Preserves all other content (text, reasoning, files, resolved/rejected
* tool-call blocks, and non-LLM custom messages).
*/
export function stripOrphanedToolMessages<T extends AgentMessage>(messages: T[]): T[] {
const callIds = new Set<string>();
const resultIds = new Set<string>();
for (const msg of messages) {
if (!isLlmMessage(msg)) continue;
for (const block of msg.content) {
if (block.type === 'tool-call' && block.toolCallId) {
callIds.add(block.toolCallId);
} else if (block.type === 'tool-result' && block.toolCallId) {
resultIds.add(block.toolCallId);
}
}
}
const orphanedCallIds = new Set([...callIds].filter((id) => !resultIds.has(id)));
const orphanedResultIds = new Set([...resultIds].filter((id) => !callIds.has(id)));
if (orphanedCallIds.size === 0 && orphanedResultIds.size === 0) {
return messages;
}
const result: T[] = [];
for (const msg of messages) {
@ -48,14 +20,7 @@ export function stripOrphanedToolMessages<T extends AgentMessage>(messages: T[])
}
const filtered = msg.content.filter((block: MessageContent) => {
if (block.type === 'tool-call' && block.toolCallId && orphanedCallIds.has(block.toolCallId)) {
return false;
}
if (
block.type === 'tool-result' &&
block.toolCallId &&
orphanedResultIds.has(block.toolCallId)
) {
if (block.type === 'tool-call' && block.state === 'pending') {
return false;
}
return true;

View File

@ -33,6 +33,16 @@ const DEFAULT_TITLE_INSTRUCTIONS = [
'Title: Scryfall random card workflow',
].join('\n');
const DEFAULT_TITLE_AND_EMOJI_INSTRUCTIONS = [
'Generate a short title and a single emoji for a conversation based on the user message.',
'Respond with ONLY a JSON object: {"title": "...", "emoji": "..."}',
'Rules:',
'- Title: 2 to 6 words, max 50 characters, sentence case',
'- Title must not contain emoji, quotes, colons, or markdown formatting',
'- Emoji: exactly one emoji that represents the topic',
'- Respond with the JSON object only, no other text',
].join('\n');
const TRIVIAL_MESSAGE_MAX_CHARS = 15;
const TRIVIAL_MESSAGE_MAX_WORDS = 3;
const MAX_TITLE_LENGTH = 80;
@ -117,7 +127,67 @@ export async function generateTitleFromMessage(
}
/**
* Generate a title for a thread if it doesn't already have one.
* Generate a sanitized title and a representative emoji from a user message.
*
* Asks the LLM for a `{"title": "...", "emoji": "..."}` JSON object and parses
* it; falls back to treating the whole response as a plain title if the model
* ignores the JSON format.
*
* Returns `null` on empty/trivial input or empty LLM output.
*/
export async function generateTitleAndEmojiFromMessage(
model: LanguageModel,
userMessage: string,
opts?: { instructions?: string },
): Promise<{ title: string; emoji?: string } | null> {
const trimmed = userMessage.trim();
if (!trimmed) return null;
if (isTrivialMessage(trimmed)) {
return null;
}
const result = await generateText({
model,
messages: [
{ role: 'system', content: opts?.instructions ?? DEFAULT_TITLE_AND_EMOJI_INSTRUCTIONS },
{ role: 'user', content: trimmed },
],
});
let text = result.text?.trim();
if (!text) return null;
// Strip <think>...</think> blocks (e.g. from DeepSeek R1) before JSON parsing.
text = text.replace(/<think>[\s\S]*?<\/think>/g, '').trim();
if (!text) return null;
let rawTitle = '';
let emoji: string | undefined;
const jsonMatch = /\{[\s\S]*\}/.exec(text);
if (jsonMatch) {
try {
const parsed = JSON.parse(jsonMatch[0]) as { title?: string; emoji?: string };
rawTitle = parsed.title?.trim() ?? '';
emoji = parsed.emoji?.trim() ?? undefined;
} catch {
// Model returned something that looked like JSON but wasn't parseable —
// fall back to using the whole response as the title.
rawTitle = text;
}
} else {
rawTitle = text;
}
const title = sanitizeTitle(rawTitle);
if (!title) return null;
return { title, emoji };
}
/**
* Generate a title and emoji for a thread if it doesn't already have one.
*
* Designed to run fire-and-forget after the agent response is complete.
* All errors are caught and logged title generation failures never
@ -148,16 +218,21 @@ export async function generateThreadTitle(opts: {
const titleModelId = opts.titleConfig.model ?? opts.agentModel;
const titleModel = createModel(titleModelId);
const title = await generateTitleFromMessage(titleModel, userText, {
const generated = await generateTitleAndEmojiFromMessage(titleModel, userText, {
instructions: opts.titleConfig.instructions,
});
if (!title) return;
if (!generated) return;
const { title, emoji } = generated;
// Store emoji in thread metadata
const metadata = { ...(thread?.metadata ?? {}), ...(emoji && { emoji }) };
await opts.memory.saveThread({
id: opts.threadId,
resourceId: opts.resourceId,
title,
metadata: thread?.metadata,
metadata,
});
} catch (error) {
logger.warn('Failed to generate thread title', { error });

View File

@ -4,7 +4,7 @@ import type { BuiltTool } from '../types';
type ZodObjectSchema = z.ZodObject<z.ZodRawShape>;
export const UPDATE_WORKING_MEMORY_TOOL_NAME = 'updateWorkingMemory';
export const UPDATE_WORKING_MEMORY_TOOL_NAME = 'update_working_memory';
/**
* The default instruction block injected into the system prompt when working memory
@ -19,7 +19,7 @@ export const WORKING_MEMORY_DEFAULT_INSTRUCTION = [
/**
* Generate the system prompt instruction for working memory.
* Tells the LLM to call the updateWorkingMemory tool when it has new information to persist.
* Tells the LLM to call the update_working_memory tool when it has new information to persist.
*
* @param template - The working memory template or schema.
* @param structured - Whether the working memory is structured (JSON schema).
@ -73,7 +73,7 @@ export interface WorkingMemoryToolConfig {
}
/**
* Build the updateWorkingMemory BuiltTool that the agent calls to persist working memory.
* Build the update_working_memory BuiltTool that the agent calls to persist working memory.
*
* For freeform working memory the input schema is `{ memory: string }`.
* For structured working memory the input schema is the configured Zod object schema,

View File

@ -1,12 +1,13 @@
import type { ProviderOptions } from '@ai-sdk/provider-utils';
import { z } from 'zod';
import type { ModelCost } from './catalog';
import type { AgentRuntimeConfig } from '../runtime/agent-runtime';
import type { Eval } from './eval';
import type { McpClient } from './mcp-client';
import { Memory } from './memory';
import { Telemetry } from './telemetry';
import { Tool, wrapToolForApproval } from './tool';
import { AgentRuntime } from '../runtime/agent-runtime';
import { AgentEventBus } from '../runtime/event-bus';
import { InMemoryMemory } from '../runtime/memory-store';
import { RunStateManager } from '../runtime/run-state';
import { createAgentToolResult } from '../runtime/tool-adapter';
import type {
AgentEvent,
@ -17,57 +18,60 @@ import type {
BuiltGuardrail,
BuiltMemory,
BuiltProviderTool,
BuiltTelemetry,
BuiltTool,
BuiltTelemetry,
CheckpointStore,
ExecutionOptions,
GenerateResult,
MemoryConfig,
ModelConfig,
Provider,
ResumeOptions,
RunOptions,
SerializableAgentState,
StreamResult,
SubAgentUsage,
ThinkingConfig,
ThinkingConfigFor,
ResumeOptions,
} from '../types';
import { getModelCost } from './catalog';
import type { Eval } from './eval';
import { fromSchema, type FromSchemaOptions } from './from-schema';
import type { McpClient } from './mcp-client';
import { Memory } from './memory';
import { Telemetry } from './telemetry';
import { Tool, wrapToolForApproval } from './tool';
import type { StreamChunk } from '../types/sdk/agent';
import type { AgentBuilder } from '../types/sdk/agent-builder';
import type { CredentialProvider } from '../types/sdk/credential-provider';
import type { AgentMessage } from '../types/sdk/message';
import type {
AgentSchema,
EvalSchema,
GuardrailSchema,
McpServerSchema,
MemorySchema,
ProviderToolSchema,
ThinkingSchema,
ToolSchema,
} from '../types/sdk/schema';
import { zodToJsonSchema } from '../utils/zod';
import type { Workspace } from '../workspace/workspace';
const DEFAULT_LAST_MESSAGES = 10;
type ToolParameter = BuiltTool | { build(): BuiltTool };
/**
* Lightweight read-only view of an agent's configured state.
* Returned by `Agent.snapshot` for testing and debugging purposes.
*/
export interface AgentSnapshot {
/** Agent name. */
name: string;
/** Parsed model identifier. Both fields are null if no model has been set. */
model: { provider: string | null; name: string | null };
/** Instruction text passed to `.instructions()`, or null if not set. */
instructions: string | null;
/** Minimal description of each registered tool. */
tools: ReadonlyArray<{ name: string; description: string | undefined }>;
/** True when `.memory()` has been configured. */
hasMemory: boolean;
/** The thinking config if set, otherwise null. */
thinking: ThinkingConfig | null;
/** Tool-call concurrency limit if set, otherwise null. */
toolCallConcurrency: number | null;
/** Whether `.requireToolApproval()` was called. */
requireToolApproval: boolean;
}
/**
* Builder for creating AI agents with a fluent API.
*
* Usage:
* ```typescript
* const agent = new Agent('assistant')
* .model('anthropic', 'claude-sonnet-4') // typed: Agent<'anthropic'>
* .credential('anthropic')
* .model('anthropic', 'claude-sonnet-4')
* .instructions('You are a helpful assistant.')
* .tool(searchTool);
*
@ -78,9 +82,7 @@ type ToolParameter = BuiltTool | { build(): BuiltTool };
export class Agent implements BuiltAgent, AgentBuilder {
readonly name: string;
private modelId?: string;
private modelConfigObj?: ModelConfig;
private modelConfig?: ModelConfig;
private instructionProviderOpts?: ProviderOptions;
@ -106,11 +108,7 @@ export class Agent implements BuiltAgent, AgentBuilder {
private thinkingConfig?: ThinkingConfig;
private credentialName?: string;
private credProvider?: CredentialProvider;
private resolvedKey?: string;
private runtime?: AgentRuntime;
private concurrencyValue?: number;
@ -124,13 +122,9 @@ export class Agent implements BuiltAgent, AgentBuilder {
private mcpClients: McpClient[] = [];
private buildPromise: Promise<AgentRuntimeConfig> | undefined;
private buildPromise: Promise<AgentRuntime> | undefined;
/** Handlers registered via on() — copied into each per-run event bus at creation time. */
private agentHandlers = new Map<AgentEvent, Set<AgentEventHandler>>();
/** Event buses for all currently active runs, used to broadcast abort(). */
private activeEventBuses = new Set<AgentEventBus>();
private eventBus = new AgentEventBus();
private workspaceInstance?: Workspace;
@ -138,22 +132,6 @@ export class Agent implements BuiltAgent, AgentBuilder {
this.name = name;
}
/**
* Reconstruct a live Agent from an AgentSchema JSON.
* Custom tool handlers are proxied through the injected HandlerExecutor.
*
* This is the inverse of `Agent.describe()`.
*/
static async fromSchema(
schema: AgentSchema,
name: string,
options: FromSchemaOptions,
): Promise<Agent> {
const agent = new Agent(name);
await fromSchema(agent, schema, options);
return agent;
}
hasCheckpointStorage(): boolean {
return this.checkpointStore !== undefined;
}
@ -176,11 +154,9 @@ export class Agent implements BuiltAgent, AgentBuilder {
*/
model(providerOrIdOrConfig: string | ModelConfig, modelName?: string): this {
if (typeof providerOrIdOrConfig === 'string') {
this.modelId = modelName ? `${providerOrIdOrConfig}/${modelName}` : providerOrIdOrConfig;
this.modelConfigObj = undefined;
this.modelConfig = modelName ? `${providerOrIdOrConfig}/${modelName}` : providerOrIdOrConfig;
} else {
this.modelConfigObj = providerOrIdOrConfig;
this.modelId = undefined;
this.modelConfig = providerOrIdOrConfig;
}
return this;
}
@ -211,7 +187,7 @@ export class Agent implements BuiltAgent, AgentBuilder {
return this;
}
/** @internal Read the declared tools (used by the compile step to detect workflow tool markers). */
/** Read the declared tools. Lists only tools added via tool() */
get declaredTools(): BuiltTool[] {
return this.tools;
}
@ -289,54 +265,6 @@ export class Agent implements BuiltAgent, AgentBuilder {
return this;
}
/**
* Declare a credential this agent requires. The execution engine resolves
* the credential name to an API key at build time and injects it into the
* model configuration user code never handles raw keys.
*
* @example
* ```typescript
* const agent = new Agent('assistant')
* .model('anthropic/claude-sonnet-4-5')
* .credential('anthropic')
* .instructions('You are helpful.');
* ```
*/
credential(name: string): this {
this.credentialName = name;
return this;
}
/**
* Attach a credential provider that resolves credential identifiers to
* decrypted API keys at build time. When both `.credential()` and
* `.credentialProvider()` are set, the provider resolves the credential
* before model creation no subclassing required.
*
* @example
* ```typescript
* const agent = new Agent('assistant')
* .model('anthropic', 'claude-sonnet-4')
* .credential('credential-id-123')
* .credentialProvider(myProvider)
* .instructions('You are helpful.');
* ```
*/
credentialProvider(provider: CredentialProvider): this {
this.credProvider = provider;
return this;
}
/** @internal Read the declared credential name (used by the execution engine). */
protected get declaredCredential(): string | undefined {
return this.credentialName;
}
/** @internal Set the resolved API key (called by the execution engine before super.build()). */
protected set resolvedApiKey(key: string) {
this.resolvedKey = key;
}
/**
* Set a structured output schema. When set, the agent's response will be
* parsed into a typed object matching the schema, available as `result.output`.
@ -463,29 +391,10 @@ export class Agent implements BuiltAgent, AgentBuilder {
/**
* Register a handler for an agent lifecycle event.
* Handlers are forwarded into every per-run event bus so they fire for all concurrent runs.
* Use off() to remove the handler when it is no longer needed.
* Handlers are called synchronously during the agentic loop.
*/
on(event: AgentEvent, handler: AgentEventHandler): void {
let set = this.agentHandlers.get(event);
if (!set) {
set = new Set();
this.agentHandlers.set(event, set);
}
set.add(handler);
}
/**
* Remove a previously registered event handler.
* A no-op if the handler was never registered.
*/
off(event: AgentEvent, handler: AgentEventHandler): void {
const set = this.agentHandlers.get(event);
if (!set) return;
set.delete(handler);
if (set.size === 0) {
this.agentHandlers.delete(event);
}
this.eventBus.on(event, handler);
}
/**
@ -550,216 +459,63 @@ export class Agent implements BuiltAgent, AgentBuilder {
}
/**
* Return a schema object describing the agent's declared configuration.
* This is a synchronous introspection method it does not build the agent
* or connect to any external services.
* Return a lightweight read-only snapshot of the agent's configured state.
* Useful for testing and debugging does not trigger a build.
*/
describe(): AgentSchema {
// --- Model ---
let model: AgentSchema['model'];
if (this.modelConfigObj) {
model = { provider: null, name: null, raw: 'object' };
} else if (this.modelId) {
const slashIdx = this.modelId.indexOf('/');
get snapshot(): AgentSnapshot {
let model: AgentSnapshot['model'];
const rawModelId =
typeof this.modelConfig === 'string'
? this.modelConfig
: this.modelConfig && typeof this.modelConfig === 'object' && 'id' in this.modelConfig
? this.modelConfig.id
: undefined;
if (rawModelId) {
const slashIdx = rawModelId.indexOf('/');
if (slashIdx === -1) {
model = { provider: null, name: this.modelId };
model = { provider: null, name: rawModelId };
} else {
model = {
provider: this.modelId.slice(0, slashIdx),
name: this.modelId.slice(slashIdx + 1),
provider: rawModelId.slice(0, slashIdx),
name: rawModelId.slice(slashIdx + 1),
};
}
} else {
model = { provider: null, name: null };
}
// --- Tools (custom / workflow) ---
const toolSchemas: ToolSchema[] = this.tools.map((tool) => {
const isWorkflow = '__workflowTool' in tool && Boolean(tool.__workflowTool);
return {
name: tool.name,
description: tool.description,
type: isWorkflow ? ('workflow' as const) : ('custom' as const),
editable: !isWorkflow,
// Source strings — null, CLI patches with original TypeScript
inputSchemaSource: null,
outputSchemaSource: null,
handlerSource: tool.handler?.toString() ?? null,
suspendSchemaSource: null,
resumeSchemaSource: null,
toMessageSource: null,
requireApproval: tool.withDefaultApproval ?? false,
needsApprovalFnSource: null,
providerOptions: tool.providerOptions ?? null,
// Display fields — JSON Schema for UI rendering
inputSchema: zodToJsonSchema(tool.inputSchema),
outputSchema: zodToJsonSchema(tool.outputSchema),
// UI badge indicators — for approval-wrapped tools, hasSuspend/hasResume
// reflect the approval mechanism, not user-declared suspend/resume
hasSuspend: Boolean(tool.suspendSchema),
hasResume: Boolean(tool.resumeSchema),
hasToMessage: Boolean(tool.toMessage),
};
});
// --- Provider tools ---
const providerToolSchemas: ProviderToolSchema[] = this.providerTools.map((pt) => ({
name: pt.name,
source: '',
}));
// --- Guardrails ---
const guardrails: GuardrailSchema[] = [
...this.inputGuardrails.map((g) => ({
name: g.name,
guardType: g.guardType,
strategy: g.strategy,
position: 'input' as const,
config: g._config,
source: '',
})),
...this.outputGuardrails.map((g) => ({
name: g.name,
guardType: g.guardType,
strategy: g.strategy,
position: 'output' as const,
config: g._config,
source: '',
})),
];
// --- MCP servers ---
let mcp: McpServerSchema[] | null = null;
if (this.mcpClients.length > 0) {
mcp = [];
for (const client of this.mcpClients) {
for (const serverName of client.serverNames) {
mcp.push({
name: serverName,
configSource: '',
});
}
}
}
// --- Telemetry ---
const telemetry = this.telemetryBuilder || this.telemetryConfig ? { source: '' } : null;
// --- Checkpoint ---
const checkpoint = this.checkpointStore === 'memory' ? 'memory' : null;
// --- Memory ---
let memory: MemorySchema | null = null;
if (this.memoryConfig) {
const mc = this.memoryConfig;
let semanticRecall: MemorySchema['semanticRecall'] = null;
if (mc.semanticRecall) {
semanticRecall = {
topK: mc.semanticRecall.topK,
messageRange: mc.semanticRecall.messageRange
? {
before: mc.semanticRecall.messageRange.before,
after: mc.semanticRecall.messageRange.after,
}
: null,
embedder: mc.semanticRecall.embedder ?? null,
};
}
let workingMemory: MemorySchema['workingMemory'] = null;
if (mc.workingMemory) {
workingMemory = {
type: mc.workingMemory.structured ? 'structured' : 'freeform',
...(mc.workingMemory.schema
? { schema: zodToJsonSchema(mc.workingMemory.schema) ?? undefined }
: {}),
...(mc.workingMemory.template ? { template: mc.workingMemory.template } : {}),
};
}
memory = {
// TODO: each BuiltMemory should have describe() method to return a config showing connection params and other metadata
// this config must have enough information to rebuild the memory instance
source: null,
storage: mc.memory instanceof InMemoryMemory ? 'memory' : 'custom',
lastMessages: mc.lastMessages ?? null,
semanticRecall,
workingMemory,
};
}
// --- Evaluations ---
const evaluations: EvalSchema[] = this.agentEvals.map((e) => ({
name: e.name,
description: e.description ?? null,
type: e.evalType,
modelId: e.modelId ?? null,
hasCredential: e.credentialName !== null,
credentialName: e.credentialName,
handlerSource: null,
}));
// --- Structured output ---
// TODO: define structured output schema handling better
const structuredOutput = {
enabled: Boolean(this.outputSchema),
schemaSource: null as string | null,
};
// --- Thinking ---
let thinking: ThinkingSchema | null = null;
if (this.thinkingConfig) {
const provider = this.modelId?.split('/')[0];
if (provider === 'anthropic') {
thinking = {
provider: 'anthropic',
budgetTokens:
'budgetTokens' in this.thinkingConfig
? (this.thinkingConfig as { budgetTokens?: number }).budgetTokens
: undefined,
};
} else if (provider === 'openai') {
thinking = {
provider: 'openai',
reasoningEffort:
'reasoningEffort' in this.thinkingConfig
? String((this.thinkingConfig as { reasoningEffort?: string }).reasoningEffort)
: undefined,
};
}
}
return {
name: this.name,
model,
credential: this.credentialName ?? null,
instructions: this.instructionsText ?? null,
description: null,
tools: toolSchemas,
providerTools: providerToolSchemas,
memory,
evaluations,
guardrails,
mcp,
telemetry,
checkpoint,
config: {
structuredOutput,
thinking,
toolCallConcurrency: this.concurrencyValue ?? null,
requireToolApproval: this.requireToolApprovalValue,
},
tools: this.tools.map((t) => ({ name: t.name, description: t.description })),
hasMemory: this.memoryConfig !== undefined,
thinking: this.thinkingConfig ?? null,
toolCallConcurrency: this.concurrencyValue ?? null,
requireToolApproval: this.requireToolApprovalValue,
};
}
/** Return the latest state snapshot of the agent. Returns `{ status: 'idle' }` before first run. */
getState(): SerializableAgentState {
if (!this.runtime) {
return {
persistence: undefined,
status: 'idle',
messageList: { messages: [], historyIds: [], inputIds: [], responseIds: [] },
pendingToolCalls: {},
};
}
return this.runtime.getState();
}
/**
* Cancel all currently active runs on this agent.
* Synchronous sets an abort flag on every active event bus;
* the agentic loop in each run checks it asynchronously.
* Cancel the currently running agent.
* Synchronous sets an abort flag; the agentic loop checks it asynchronously.
*/
abort(): void {
for (const bus of this.activeEventBuses) {
bus.abort();
}
this.eventBus.abort();
}
/** Generate a response (non-streaming). Lazy-builds on first call. */
@ -767,13 +523,8 @@ export class Agent implements BuiltAgent, AgentBuilder {
input: AgentMessage[] | string,
options?: RunOptions & ExecutionOptions,
): Promise<GenerateResult> {
const config = await this.ensureBuilt();
const { runtime, bus } = this.createRuntime(config);
try {
return await runtime.generate(this.toMessages(input), options);
} finally {
this.cleanupBus(bus);
}
const runtime = await this.ensureBuilt();
return await runtime.generate(this.toMessages(input), options);
}
/** Stream a response. Lazy-builds on first call. */
@ -781,15 +532,8 @@ export class Agent implements BuiltAgent, AgentBuilder {
input: AgentMessage[] | string,
options?: RunOptions & ExecutionOptions,
): Promise<StreamResult> {
const config = await this.ensureBuilt();
const { runtime, bus } = this.createRuntime(config);
try {
const result = await runtime.stream(this.toMessages(input), options);
return { ...result, stream: this.trackStreamBus(result.stream, bus) };
} catch (error) {
this.cleanupBus(bus);
throw error;
}
const runtime = await this.ensureBuilt();
return await runtime.stream(this.toMessages(input), options);
}
/** Resume a suspended tool call with data. Lazy-builds on first call. */
@ -808,23 +552,11 @@ export class Agent implements BuiltAgent, AgentBuilder {
data: unknown,
options: ResumeOptions & ExecutionOptions,
): Promise<GenerateResult | StreamResult> {
const config = await this.ensureBuilt();
const runtime = await this.ensureBuilt();
if (method === 'generate') {
const { runtime, bus } = this.createRuntime(config);
try {
return await runtime.resume('generate', data, options);
} finally {
this.cleanupBus(bus);
}
}
const { runtime, bus } = this.createRuntime(config);
try {
const result = await runtime.resume('stream', data, options);
return { ...result, stream: this.trackStreamBus(result.stream, bus) };
} catch (error) {
this.cleanupBus(bus);
throw error;
return await runtime.resume('generate', data, options);
}
return await runtime.resume('stream', data, options);
}
approve(method: 'generate', options: ResumeOptions & ExecutionOptions): Promise<GenerateResult>;
@ -856,7 +588,7 @@ export class Agent implements BuiltAgent, AgentBuilder {
* concurrent callers share one build operation. On error the promise is
* cleared so the caller can retry.
*/
private async ensureBuilt(): Promise<AgentRuntimeConfig> {
private async ensureBuilt(): Promise<AgentRuntime> {
if (!this.buildPromise) {
const p = this.build();
this.buildPromise = p;
@ -867,90 +599,14 @@ export class Agent implements BuiltAgent, AgentBuilder {
return await this.buildPromise;
}
/**
* Create a fresh AgentRuntime from the shared config, wiring a new event bus
* with all registered agent-level handlers copied in. The bus is registered
* in activeEventBuses so that abort() can reach it. Callers are responsible
* for deregistering the bus when the run finishes.
*
* AgentRuntime is not supposed to be reused across runs, it gets created and destroyed for each run.
*/
private createRuntime(config: AgentRuntimeConfig): { runtime: AgentRuntime; bus: AgentEventBus } {
const bus = new AgentEventBus();
for (const [event, handlers] of this.agentHandlers) {
for (const handler of handlers) {
bus.on(event, handler);
}
}
this.activeEventBuses.add(bus);
const runtime = new AgentRuntime({ ...config, eventBus: bus });
return { runtime, bus };
}
/**
* Wrap a stream so that the bus is deregistered from activeEventBuses
* when the stream closes whether the consumer drains it, cancels it, or
* the source errors.
*
* The bus is cleaned up in all three observable terminal states:
* - `pull` reaches `done` (source finished normally)
* - `pull` throws (source errored)
* - `cancel` is called (consumer explicitly cancelled the stream)
*
* The one case that cannot be detected without GC hooks is a consumer that
* holds a reference to the stream but never reads or cancels it. Callers
* should always drain or cancel the returned stream.
*/
private trackStreamBus(
stream: ReadableStream<StreamChunk>,
bus: AgentEventBus,
): ReadableStream<StreamChunk> {
let cleanedUp = false;
const cleanup = () => {
if (!cleanedUp) {
cleanedUp = true;
this.cleanupBus(bus);
}
};
const reader = stream.getReader();
return new ReadableStream<StreamChunk>({
async pull(controller) {
try {
const { done, value } = await reader.read();
if (done) {
controller.close();
cleanup();
} else {
controller.enqueue(value);
}
} catch (error) {
controller.error(error);
cleanup();
}
},
cancel() {
reader.cancel().catch(() => {});
cleanup();
},
});
}
private cleanupBus(bus: AgentEventBus): void {
this.activeEventBuses.delete(bus);
bus.dispose();
}
private toMessages(input: string | AgentMessage[]): AgentMessage[] {
if (Array.isArray(input)) return input;
return [{ role: 'user', content: [{ type: 'text', text: input }] }];
}
/** @internal Validate configuration and produce a shared AgentRuntimeConfig. Overridden by the execution engine. */
protected async build(): Promise<AgentRuntimeConfig> {
const hasModel = this.modelId ?? this.modelConfigObj;
if (!hasModel) {
/** @internal Validate configuration and produce an AgentRuntime. Overridden by the execution engine. */
protected async build(): Promise<AgentRuntime> {
if (!this.modelConfig) {
throw new Error(`Agent "${this.name}" requires a model`);
}
if (!this.instructionsText) {
@ -1019,28 +675,7 @@ export class Agent implements BuiltAgent, AgentBuilder {
);
}
// Resolve credential via provider before building the model config.
if (this.credProvider && this.credentialName) {
const resolved = await this.credProvider.resolve(this.credentialName);
this.resolvedKey = resolved.apiKey;
}
let modelConfig: ModelConfig;
if (this.modelConfigObj) {
if (
this.resolvedKey &&
typeof this.modelConfigObj === 'object' &&
'id' in this.modelConfigObj
) {
modelConfig = { ...this.modelConfigObj, apiKey: this.resolvedKey };
} else {
modelConfig = this.modelConfigObj;
}
} else if (this.resolvedKey) {
modelConfig = { id: this.modelId!, apiKey: this.resolvedKey };
} else {
modelConfig = this.modelId!;
}
const modelConfig: ModelConfig = this.modelConfig;
let instructions = this.instructionsText;
if (this.workspaceInstance) {
@ -1050,24 +685,7 @@ export class Agent implements BuiltAgent, AgentBuilder {
}
}
// Prefetch model cost once — shared across all per-run runtimes.
let modelCost: ModelCost | undefined;
try {
const modelId =
typeof modelConfig === 'string'
? modelConfig
: 'id' in modelConfig
? modelConfig.id
: undefined;
modelCost = modelId ? await getModelCost(modelId) : undefined;
} catch {
// Catalog unavailable — proceed without cost data
}
// Shared RunStateManager so resume() can find state from a prior stream()/generate() call.
const runState = new RunStateManager(this.checkpointStore);
return {
this.runtime = new AgentRuntime({
name: this.name,
model: modelConfig,
instructions,
@ -1081,11 +699,12 @@ export class Agent implements BuiltAgent, AgentBuilder {
structuredOutput: this.outputSchema,
checkpointStorage: this.checkpointStore,
thinking: this.thinkingConfig,
eventBus: this.eventBus,
toolCallConcurrency: this.concurrencyValue,
titleGeneration: this.memoryConfig?.titleGeneration,
telemetry: this.telemetryConfig ?? (await this.telemetryBuilder?.build()),
modelCost,
runState,
};
});
return this.runtime;
}
}

View File

@ -1,6 +1,7 @@
import { filterLlmMessages } from './message';
import { AgentRuntime } from '../runtime/agent-runtime';
import type { BuiltEval, CheckFn, EvalInput, EvalScore, JudgeFn, JudgeHandlerFn } from '../types';
import type { ModelConfig } from '../types/sdk/agent';
import type { AgentMessage } from '../types/sdk/message';
/** Extract text content from LLM messages (custom messages are skipped). */
@ -54,8 +55,6 @@ export class Eval {
private credentialName?: string;
private _resolvedApiKey?: string;
constructor(name: string) {
this.evalName = name;
}
@ -68,6 +67,7 @@ export class Eval {
/** Set the judge model (LLM-as-judge mode). */
model(modelId: string): this {
// TODO: support full model config
this.modelId = modelId;
return this;
}
@ -78,16 +78,6 @@ export class Eval {
return this;
}
/** @internal Read the declared credential name (used by the execution engine). */
protected get declaredCredential(): string | undefined {
return this.credentialName;
}
/** @internal Set the resolved API key for the judge model. */
protected set resolvedApiKey(key: string) {
this._resolvedApiKey = key;
}
/**
* Set a deterministic check function.
* Mutually exclusive with `.judge()`.
@ -146,9 +136,10 @@ export class Eval {
// LLM-as-judge mode
const judgeFn = this.judgeFn!;
const modelConfig: string | { id: `${string}/${string}`; apiKey: string } = this._resolvedApiKey
? { id: this.modelId! as `${string}/${string}`, apiKey: this._resolvedApiKey }
: this.modelId!;
if (!this.modelId) {
throw new Error(`Eval "${this.evalName}" uses .judge() but no .model() was set`);
}
const modelConfig: ModelConfig = this.modelId;
const runtime = new AgentRuntime({
name: `${name}-judge`,

View File

@ -1,364 +0,0 @@
import type { JSONSchema7 } from 'json-schema';
import type { ZodType } from 'zod';
import type { BuiltEval, BuiltGuardrail, BuiltTelemetry, BuiltTool } from '../types';
import { McpClient } from './mcp-client';
import { Memory } from './memory';
import { wrapToolForApproval } from './tool';
import type { AgentBuilder } from '../types/sdk/agent-builder';
import type { CredentialProvider } from '../types/sdk/credential-provider';
import type { EvalInput, EvalScore, JudgeInput } from '../types/sdk/eval';
import type { HandlerExecutor } from '../types/sdk/handler-executor';
import type { McpServerConfig } from '../types/sdk/mcp';
import type { AgentMessage } from '../types/sdk/message';
import type {
AgentSchema,
EvalSchema,
GuardrailSchema,
McpServerSchema,
ProviderToolSchema,
TelemetrySchema,
ToolSchema,
} from '../types/sdk/schema';
import type { InterruptibleToolContext, ToolContext } from '../types/sdk/tool';
import type { JSONObject } from '../types/utils/json';
export interface FromSchemaOptions {
handlerExecutor: HandlerExecutor;
credentialProvider?: CredentialProvider;
}
/** Sentinel used to signal that a sandboxed handler called ctx.suspend(). */
const SUSPEND_MARKER = Symbol.for('n8n.agent.suspend');
interface SuspendResult {
[key: symbol]: true;
payload: unknown;
}
export function isSuspendResult(value: unknown): value is SuspendResult {
return (
typeof value === 'object' &&
value !== null &&
(value as Record<symbol, unknown>)[SUSPEND_MARKER] === true
);
}
/**
* Reconstruct a live Agent from an AgentSchema JSON.
*
* This is the inverse of `Agent.describe()` it takes a serialised schema
* (produced by `describe()` or stored in the database) and rebuilds a
* fully-configured Agent instance with proxy handlers that delegate tool
* execution to the provided `HandlerExecutor`.
*
* All source expressions in the schema (provider tools, MCP configs,
* telemetry, structured output, suspend/resume schemas) are evaluated
* via `HandlerExecutor.evaluateExpression()` / `evaluateSchema()`.
*
* The `agent` parameter is the Agent instance to configure (avoids circular import).
*/
export async function fromSchema(
agent: AgentBuilder,
schema: AgentSchema,
options: FromSchemaOptions,
): Promise<void> {
const { handlerExecutor } = options;
applyModel(agent, schema.model);
if (schema.credential !== null) {
agent.credential(schema.credential);
}
if (schema.instructions !== null) {
agent.instructions(schema.instructions);
}
await applyTools(agent, schema.tools, handlerExecutor);
await applyProviderTools(agent, schema.providerTools, handlerExecutor);
applyConfig(agent, schema.config);
applyMemory(agent, schema);
applyGuardrails(agent, schema.guardrails);
applyEvals(agent, schema.evaluations, handlerExecutor);
await applyStructuredOutput(agent, schema.config.structuredOutput, handlerExecutor);
if (options.credentialProvider) {
agent.credentialProvider(options.credentialProvider);
}
await applyMcpServers(agent, schema.mcp, handlerExecutor);
await applyTelemetry(agent, schema.telemetry, handlerExecutor);
}
// ---------------------------------------------------------------------------
// Helpers each handles one section of the AgentSchema
// ---------------------------------------------------------------------------
function applyModel(agent: AgentBuilder, model: AgentSchema['model']): void {
if (model.provider && model.name) {
agent.model(model.provider, model.name);
} else if (model.name) {
agent.model(model.name);
}
}
async function applyTools(
agent: AgentBuilder,
tools: ToolSchema[],
executor: HandlerExecutor,
): Promise<void> {
const addedTools = new Set<string>();
for (const ts of tools) {
if (addedTools.has(ts.name)) {
throw new Error(`Schema has multiple definitions of tool ${ts.name}`);
}
addedTools.add(ts.name);
if (!ts.editable) {
agent.tool({
name: ts.name,
description: ts.description,
__workflowTool: true,
workflowName: ts.name,
} as unknown as BuiltTool);
continue;
}
const schemas: { suspend?: ZodType; resume?: ZodType } = {};
if (ts.suspendSchemaSource) {
schemas.suspend = await executor.evaluateSchema(ts.suspendSchemaSource);
}
if (ts.resumeSchemaSource) {
schemas.resume = await executor.evaluateSchema(ts.resumeSchemaSource);
}
const builtTool = buildToolFromSchema(ts, executor, schemas);
agent.tool(builtTool);
}
}
async function applyProviderTools(
agent: AgentBuilder,
providerTools: ProviderToolSchema[],
executor: HandlerExecutor,
): Promise<void> {
for (const pt of providerTools) {
if (pt.source) {
const evaluated = (await executor.evaluateExpression(pt.source)) as {
name: `${string}.${string}`;
args?: Record<string, unknown>;
};
agent.providerTool({
name: evaluated.name,
args: evaluated.args ?? {},
});
} else {
agent.providerTool({
name: pt.name as `${string}.${string}`,
args: {},
});
}
}
}
function applyConfig(agent: AgentBuilder, config: AgentSchema['config']): void {
if (config.thinking !== null) {
const { provider, ...thinkingConfig } = config.thinking;
agent.thinking(provider, thinkingConfig);
}
if (config.toolCallConcurrency !== null) {
agent.toolCallConcurrency(config.toolCallConcurrency);
}
if (config.requireToolApproval) {
agent.requireToolApproval();
}
}
function applyMemory(agent: AgentBuilder, schema: AgentSchema): void {
if (schema.memory !== null) {
const memory = new Memory();
if (schema.memory.lastMessages !== null) {
memory.lastMessages(schema.memory.lastMessages);
}
agent.memory(memory);
}
if (schema.checkpoint !== null) {
agent.checkpoint(schema.checkpoint);
}
}
function applyGuardrails(agent: AgentBuilder, guardrails: GuardrailSchema[]): void {
for (const g of guardrails) {
const builtGuardrail: BuiltGuardrail = {
name: g.name,
guardType: g.guardType,
strategy: g.strategy,
_config: g.config,
};
if (g.position === 'input') {
agent.inputGuardrail(builtGuardrail);
} else {
agent.outputGuardrail(builtGuardrail);
}
}
}
function applyEvals(
agent: AgentBuilder,
evaluations: EvalSchema[],
executor: HandlerExecutor,
): void {
for (const evalSchema of evaluations) {
const builtEval = buildEvalFromSchema(evalSchema, executor);
agent.eval(builtEval);
}
}
async function applyStructuredOutput(
agent: AgentBuilder,
structuredOutput: AgentSchema['config']['structuredOutput'],
executor: HandlerExecutor,
): Promise<void> {
if (structuredOutput.enabled && structuredOutput.schemaSource) {
const outputSchema = await executor.evaluateSchema(structuredOutput.schemaSource);
agent.structuredOutput(outputSchema);
}
}
async function applyMcpServers(
agent: AgentBuilder,
mcp: McpServerSchema[] | null,
executor: HandlerExecutor,
): Promise<void> {
if (!mcp || mcp.length === 0) return;
const mcpConfigs: McpServerConfig[] = [];
for (const m of mcp) {
if (m.configSource) {
const config = (await executor.evaluateExpression(m.configSource)) as McpServerConfig;
mcpConfigs.push(config);
}
}
if (mcpConfigs.length > 0) {
agent.mcp(new McpClient(mcpConfigs));
}
}
async function applyTelemetry(
agent: AgentBuilder,
telemetry: TelemetrySchema | null,
executor: HandlerExecutor,
): Promise<void> {
if (telemetry?.source) {
const built = (await executor.evaluateExpression(telemetry.source)) as BuiltTelemetry;
agent.telemetry(built);
}
}
// ---------------------------------------------------------------------------
// Tool & Eval builders
// ---------------------------------------------------------------------------
/**
* Build a `BuiltTool` from a `ToolSchema` with a proxy handler that
* delegates execution to the `HandlerExecutor`.
*
* For interruptible tools (hasSuspend), the proxy handles ctx.suspend at
* the host level: the sandbox receives a stub suspend that records the
* payload, and the proxy calls the real ctx.suspend on the host.
*/
function buildToolFromSchema(
toolSchema: ToolSchema,
executor: HandlerExecutor,
preEvaluated?: { suspend?: ZodType; resume?: ZodType },
): BuiltTool {
const handler = async (
input: unknown,
ctx: ToolContext | InterruptibleToolContext,
): Promise<unknown> => {
if (toolSchema.hasSuspend && 'suspend' in ctx) {
// Interruptible tool: the real ctx.suspend is a host-side function.
// We pass serialisable ctx data into the sandbox, and the sandbox
// returns a marker if suspend was called. Then we call the real
// ctx.suspend on the host.
const interruptCtx = ctx;
const result = await executor.executeTool(toolSchema.name, input, {
resumeData: interruptCtx.resumeData,
parentTelemetry: ctx.parentTelemetry,
});
if (isSuspendResult(result)) {
return await interruptCtx.suspend(result.payload);
}
return result;
}
// Non-interruptible tool: pass ctx through directly (only serialisable
// fields like parentTelemetry).
return await executor.executeTool(toolSchema.name, input, {
parentTelemetry: ctx.parentTelemetry,
});
};
// toMessage: The runtime calls toMessage synchronously (agent-runtime.ts).
// When the executor provides a sync variant (executeToMessageSync), use it
// directly for an immediate result. Otherwise fall back to async with a
// stale-cache workaround.
let toMessage: ((output: unknown) => AgentMessage | undefined) | undefined;
if (toolSchema.hasToMessage) {
if (executor.executeToMessageSync) {
const syncExecutor = executor.executeToMessageSync.bind(executor);
toMessage = (output: unknown): AgentMessage | undefined => {
return syncExecutor(toolSchema.name, output);
};
} else {
throw new Error('Executor does not support executeToMessageSync');
}
}
const built: BuiltTool = {
name: toolSchema.name,
description: toolSchema.description,
inputSchema: (toolSchema.inputSchema as JSONSchema7) ?? undefined,
handler,
toMessage,
suspendSchema: preEvaluated?.suspend,
resumeSchema: preEvaluated?.resume,
providerOptions: toolSchema.providerOptions
? (toolSchema.providerOptions as Record<string, JSONObject>)
: undefined,
};
// If the tool requires approval, wrap it with the approval gate.
// This re-applies the same wrapping that Tool.build() does at define time.
if (toolSchema.requireApproval) {
return wrapToolForApproval(built, { requireApproval: true });
}
return built;
}
/**
* Build a `BuiltEval` from an `EvalSchema` with a proxy _run function
* that delegates execution to the `HandlerExecutor`.
*/
function buildEvalFromSchema(evalSchema: EvalSchema, executor: HandlerExecutor): BuiltEval {
return {
name: evalSchema.name,
description: evalSchema.description ?? undefined,
evalType: evalSchema.type,
modelId: evalSchema.modelId ?? null,
credentialName: evalSchema.credentialName ?? null,
_run: async (evalInput: EvalInput): Promise<EvalScore> => {
// For judge evals, the llm function is bound inside the module
// when the full module runs in the sandbox. The executor passes
// the input to _run() which already has llm in its closure.
return await executor.executeEval(evalSchema.name, evalInput as EvalInput | JudgeInput);
},
};
}

View File

@ -29,7 +29,9 @@ export const providerCapabilities: Record<
groq: {},
deepseek: {},
mistral: {},
openrouter: {},
cohere: {},
ollama: {},
vercel: {},
openrouter: {},
'azure-openai': {},
'aws-bedrock': {},
};

View File

@ -1,6 +1,6 @@
import type { BuiltProviderTool } from '../types';
interface WebSearchConfig {
interface AnthropicWebSearchConfig {
maxUses?: number;
allowedDomains?: string[];
blockedDomains?: string[];
@ -13,6 +13,30 @@ interface WebSearchConfig {
};
}
interface OpenAIWebSearchConfig {
/**
* When set to `true`, lets the model fetch page content from the open web.
* Defaults to OpenAI's own defaults when omitted.
*/
externalWebAccess?: boolean;
/** Restrict results to the given domains (allow-list). */
filters?: {
allowedDomains?: string[];
};
/**
* How much surrounding page content to include per result. Trades off
* token cost against answer quality. Defaults to OpenAI's own default.
*/
searchContextSize?: 'low' | 'medium' | 'high';
userLocation?: {
type: 'approximate';
city?: string;
region?: string;
country?: string;
timezone?: string;
};
}
/**
* Factory for creating provider-defined tools.
*
@ -36,7 +60,7 @@ export const providerTools = {
* .build();
* ```
*/
anthropicWebSearch(config?: WebSearchConfig): BuiltProviderTool {
anthropicWebSearch(config?: AnthropicWebSearchConfig): BuiltProviderTool {
const args: Record<string, unknown> = {};
if (config?.maxUses !== undefined) {
@ -53,11 +77,57 @@ export const providerTools = {
}
return {
// Intentionally on the pre-dynamic-filtering version. The newer
// `web_search_20260209` forces a server-side code-execution pipeline
// (the AI SDK auto-adds the `code-execution-web-tools-2026-02-09`
// beta) which is slower, emits code_execution tool results on every
// search, and is only officially supported on Sonnet 4.6 / Opus 4.6
// / Opus 4.7. The 20250305 version is fast, stable, and works
// across all Claude models that support web search.
// See https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool
name: 'anthropic.web_search_20250305',
args,
};
},
/**
* OpenAI's web search tool available via the Responses API. Gives the
* model access to real-time web content with automatic citations.
*
* Only works on models that the Responses API supports (e.g. GPT-4o
* and successors). Older chat-completions-only models will reject it.
*
* @example
* ```typescript
* const agent = new Agent('researcher')
* .model('openai/gpt-4o')
* .instructions('Research topics using web search.')
* .providerTool(providerTools.openaiWebSearch({ searchContextSize: 'medium' }))
* .build();
* ```
*/
openaiWebSearch(config?: OpenAIWebSearchConfig): BuiltProviderTool {
const args: Record<string, unknown> = {};
if (config?.externalWebAccess !== undefined) {
args.externalWebAccess = config.externalWebAccess;
}
if (config?.filters) {
args.filters = config.filters;
}
if (config?.searchContextSize) {
args.searchContextSize = config.searchContextSize;
}
if (config?.userLocation) {
args.userLocation = config.userLocation;
}
return {
name: 'openai.web_search',
args,
};
},
openaiImageGeneration(): BuiltProviderTool {
return {
name: 'openai.image_generation',

View File

@ -1,8 +1,11 @@
import type { JSONSchema7 } from 'json-schema';
import { z } from 'zod';
import type { BuiltTool, InterruptibleToolContext, ToolContext } from '../types';
import type { AgentMessage } from '../types/sdk/message';
import type { ToolDescriptor } from '../types/sdk/tool-descriptor';
import type { JSONObject } from '../types/utils/json';
import { isZodSchema, zodToJsonSchema } from '../utils/zod';
const APPROVAL_SUSPEND_SCHEMA = z.object({
type: z.literal('approval'),
@ -14,6 +17,10 @@ const APPROVAL_RESUME_SCHEMA = z.object({
approved: z.boolean(),
});
type ZodOrJsonSchema = z.ZodType | JSONSchema7;
type OutputType<TOutput> = TOutput extends z.ZodType ? z.infer<TOutput> : unknown;
export interface ApprovalConfig {
requireApproval?: boolean;
needsApprovalFn?: (args: unknown) => Promise<boolean> | boolean;
@ -65,8 +72,8 @@ export function wrapToolForApproval(tool: BuiltTool, config: ApprovalConfig): Bu
};
}
type HandlerContext<S, R> = S extends z.ZodTypeAny
? R extends z.ZodTypeAny
type HandlerContext<S, R> = S extends z.ZodType
? R extends z.ZodType
? InterruptibleToolContext<z.infer<S>, z.infer<R>>
: ToolContext
: ToolContext;
@ -90,10 +97,10 @@ type HandlerContext<S, R> = S extends z.ZodTypeAny
* @template TResume - Zod schema type for the resume payload
*/
export class Tool<
TInput extends z.ZodTypeAny = z.ZodTypeAny,
TOutput extends z.ZodTypeAny = z.ZodTypeAny,
TSuspend extends z.ZodTypeAny | undefined = undefined,
TResume extends z.ZodTypeAny | undefined = undefined,
TInput extends ZodOrJsonSchema = z.ZodTypeAny,
TOutput extends ZodOrJsonSchema = z.ZodTypeAny,
TSuspend extends ZodOrJsonSchema | undefined = undefined,
TResume extends ZodOrJsonSchema | undefined = undefined,
> {
private name: string;
@ -103,18 +110,18 @@ export class Tool<
private outputSchema?: TOutput;
private suspendSchemaValue?: z.ZodTypeAny;
private suspendSchemaValue?: TSuspend;
private resumeSchemaValue?: z.ZodTypeAny;
private resumeSchemaValue?: TResume;
private handlerFn?: (
input: z.infer<TInput>,
input: OutputType<TInput>,
ctx: HandlerContext<TSuspend, TResume>,
) => Promise<z.infer<TOutput>>;
) => Promise<OutputType<TOutput>>;
private toMessageFn?: (output: z.infer<TOutput>) => AgentMessage;
private toMessageFn?: (output: OutputType<TOutput>) => AgentMessage;
private toModelOutputFn?: (output: z.infer<TOutput>) => unknown;
private toModelOutputFn?: (output: OutputType<TOutput>) => unknown;
private providerOptionsValue?: Record<string, JSONObject>;
@ -122,6 +129,8 @@ export class Tool<
private needsApprovalFnValue?: (args: unknown) => Promise<boolean> | boolean;
private systemInstructionText?: string;
constructor(name: string) {
this.name = name;
}
@ -132,29 +141,43 @@ export class Tool<
return this;
}
/**
* Attach a behavioural directive to this tool. When the tool is registered
* with an agent, the runtime injects this text into the agent's system
* prompt under a `<built_in_rules>` block, where the LLM weighs it heavily
* for "should I call this tool?" decisions.
*
* Use sparingly only for guidance the description alone doesn't reliably
* convey (e.g. "prefer this tool over plain text when X").
*/
systemInstruction(text: string): this {
this.systemInstructionText = text;
return this;
}
/** Set the input Zod schema. Required before building. */
input<S extends z.ZodTypeAny>(schema: S): Tool<S, TOutput, TSuspend, TResume> {
input<S extends ZodOrJsonSchema>(schema: S): Tool<S, TOutput, TSuspend, TResume> {
const self = this as unknown as Tool<S, TOutput, TSuspend, TResume>;
self.inputSchema = schema;
return self;
}
/** Set the output Zod schema. Optional. */
output<S extends z.ZodTypeAny>(schema: S): Tool<TInput, S, TSuspend, TResume> {
output<S extends ZodOrJsonSchema>(schema: S): Tool<TInput, S, TSuspend, TResume> {
const self = this as unknown as Tool<TInput, S, TSuspend, TResume>;
self.outputSchema = schema;
return self;
}
/** Set the suspend payload schema. Must be paired with .resume(). */
suspend<S extends z.ZodTypeAny>(schema: S): Tool<TInput, TOutput, S, TResume> {
suspend<S extends ZodOrJsonSchema>(schema: S): Tool<TInput, TOutput, S, TResume> {
const self = this as unknown as Tool<TInput, TOutput, S, TResume>;
self.suspendSchemaValue = schema;
return self;
}
/** Set the resume payload schema. Must be paired with .suspend(). */
resume<R extends z.ZodTypeAny>(schema: R): Tool<TInput, TOutput, TSuspend, R> {
resume<R extends ZodOrJsonSchema>(schema: R): Tool<TInput, TOutput, TSuspend, R> {
const self = this as unknown as Tool<TInput, TOutput, TSuspend, R>;
self.resumeSchemaValue = schema;
return self;
@ -166,15 +189,15 @@ export class Tool<
*/
handler(
fn: (
input: z.infer<TInput>,
input: OutputType<TInput>,
ctx: HandlerContext<TSuspend, TResume>,
) => Promise<z.infer<TOutput>>,
) => Promise<OutputType<TOutput>>,
): this {
this.handlerFn = fn;
return this;
}
toMessage(toMessage: (output: z.infer<TOutput>) => AgentMessage): this {
toMessage(toMessage: (output: OutputType<TOutput>) => AgentMessage): this {
this.toMessageFn = toMessage;
return this;
}
@ -186,7 +209,7 @@ export class Tool<
* Useful for truncating large outputs, redacting sensitive data, or reformatting
* the result for better LLM comprehension.
*/
toModelOutput(fn: (output: z.infer<TOutput>) => unknown): this {
toModelOutput(fn: (output: OutputType<TOutput>) => unknown): this {
this.toModelOutputFn = fn;
return this;
}
@ -198,7 +221,7 @@ export class Tool<
}
/** Conditionally require approval based on the tool's input. Mutually exclusive with .suspend()/.resume(). */
needsApprovalFn(fn: (args: z.infer<TInput>) => Promise<boolean> | boolean): this {
needsApprovalFn(fn: (args: OutputType<TInput>) => Promise<boolean> | boolean): this {
this.needsApprovalFnValue = fn as (args: unknown) => Promise<boolean> | boolean;
return this;
}
@ -255,6 +278,7 @@ export class Tool<
const built: BuiltTool = {
name: this.name,
description: this.desc,
systemInstruction: this.systemInstructionText,
suspendSchema: this.suspendSchemaValue,
resumeSchema: this.resumeSchemaValue,
toMessage: this.toMessageFn as (output: unknown) => AgentMessage | undefined,
@ -277,4 +301,36 @@ export class Tool<
return built;
}
/**
* Return a lightweight JSON descriptor of this tool's metadata.
* Does NOT require .build() to be called first.
* Used by the JSON-config flow to store tool metadata without executing the handler.
*/
describe(): ToolDescriptor {
if (!this.name) throw new Error('Tool name is required');
if (!this.desc) throw new Error(`Tool "${this.name}" requires a description`);
if (!this.inputSchema) throw new Error(`Tool "${this.name}" requires an input schema`);
const inputSchema = isZodSchema(this.inputSchema)
? zodToJsonSchema(this.inputSchema)
: this.inputSchema;
const outputSchema = this.outputSchema
? isZodSchema(this.outputSchema)
? zodToJsonSchema(this.outputSchema)
: this.outputSchema
: null;
return {
name: this.name,
description: this.desc,
systemInstruction: this.systemInstructionText ?? null,
inputSchema: inputSchema as JSONSchema7,
outputSchema: outputSchema as JSONSchema7,
hasSuspend: this.suspendSchemaValue !== undefined,
hasResume: this.resumeSchemaValue !== undefined,
hasToMessage: this.toMessageFn !== undefined,
requireApproval: this.requireApprovalValue ?? false,
providerOptions: this.providerOptionsValue ?? null,
};
}
}

View File

@ -45,15 +45,7 @@ export function verify(source: string): VerifyResult {
}
if (/process\.env\b/.test(source)) {
errors.push(
'process.env is not available. Use .credential() for API keys, or const variables for configuration.',
);
}
if (!/\.credential\s*\(/.test(source)) {
errors.push(
"No .credential() found. Every agent must declare a credential (e.g. .credential('anthropic')).",
);
errors.push('process.env is not available. Use const variables for configuration.');
}
return { ok: errors.length === 0, errors };

View File

@ -0,0 +1,93 @@
/* eslint-disable @typescript-eslint/promise-function-async */
import type { BuiltMemory, MemoryDescriptor, Thread } from '../types/sdk/memory';
import type { AgentDbMessage } from '../types/sdk/message';
import type { JSONObject } from '../types/utils/json';
export abstract class BaseMemory<TConstructorOptions extends JSONObject = JSONObject>
implements BuiltMemory
{
constructor(
protected readonly name: string,
protected readonly constructorOptions: TConstructorOptions,
) {}
getThread(_threadId: string): Promise<Thread | null> {
throw new Error('Method not implemented.');
}
saveThread(_thread: Omit<Thread, 'createdAt' | 'updatedAt'>): Promise<Thread> {
throw new Error('Method not implemented.');
}
deleteThread(_threadId: string): Promise<void> {
throw new Error('Method not implemented.');
}
getMessages(
_threadId: string,
_opts?: { limit?: number; before?: Date },
): Promise<AgentDbMessage[]> {
throw new Error('Method not implemented.');
}
saveMessages(_args: {
threadId: string;
resourceId: string;
messages: AgentDbMessage[];
}): Promise<void> {
throw new Error('Method not implemented.');
}
deleteMessages(_messageIds: string[]): Promise<void> {
throw new Error('Method not implemented.');
}
search?(
_query: string,
_opts?: {
scope?: 'thread' | 'resource';
threadId?: string;
resourceId?: string;
topK?: number;
messageRange?: { before: number; after: number };
},
): Promise<AgentDbMessage[]> {
throw new Error('Method not implemented.');
}
getWorkingMemory?(_params: {
threadId: string;
resourceId: string;
scope: 'resource' | 'thread';
}): Promise<string | null> {
throw new Error('Method not implemented.');
}
saveWorkingMemory?(
_params: { threadId: string; resourceId: string; scope: 'resource' | 'thread' },
_content: string,
): Promise<void> {
throw new Error('Method not implemented.');
}
saveEmbeddings?(_opts: {
scope?: 'thread' | 'resource';
threadId?: string;
resourceId?: string;
entries: Array<{ id: string; vector: number[]; text: string; model: string }>;
}): Promise<void> {
throw new Error('Method not implemented.');
}
queryEmbeddings?(_opts: {
scope?: 'thread' | 'resource';
threadId?: string;
resourceId?: string;
vector: number[];
topK: number;
}): Promise<Array<{ id: string; score: number }>> {
throw new Error('Method not implemented.');
}
close?(): Promise<void> {
throw new Error('Method not implemented.');
}
describe(): MemoryDescriptor<TConstructorOptions> {
return {
name: this.name,
constructorName: this.constructor.name,
connectionParams: this.constructorOptions,
};
}
}

View File

@ -1,7 +1,7 @@
import type { Pool, PoolClient } from 'pg';
import type { ConnectionOptions } from 'tls';
import type { BuiltMemory, Thread } from '../types/sdk/memory';
import { BaseMemory } from './base-memory';
import type { Thread } from '../types/sdk/memory';
import type { AgentDbMessage, AgentMessage } from '../types/sdk/message';
interface ThreadRow {
@ -40,24 +40,24 @@ function parseJsonSafe(text: string): unknown {
}
}
export interface PostgresConnectionConfig {
/** Postgres host. Defaults to 'localhost'. */
host?: string;
/** Postgres port. Defaults to 5432. */
port?: number;
/** Database name. */
database?: string;
/** Database user. */
user?: string;
/** Database password. */
password?: string | (() => string | Promise<string>);
}
export interface PostgresMemoryConfig {
// --- Connection ---
/** Connection URL string (e.g. 'postgresql://user:pass@localhost:5432/db') or individual connection parameters. */
connection: string | PostgresConnectionConfig;
export type PostgresConnectionOptions =
| { connectionType: 'url'; connection: { url: string } }
| {
connectionType: 'config';
connection: {
/** Postgres host. Defaults to 'localhost'. */ host?: string;
/** Postgres port. Defaults to 5432. */
port?: number;
/** Database name. */
database?: string;
/** Database user. */
user?: string;
/** Database password. Always credential-backed — never a raw string. */
password?: string;
};
};
export type PostgresMemoryOptions = {
// --- Pool settings ---
/** Connection pool configuration. */
pool?: {
@ -75,32 +75,47 @@ export interface PostgresMemoryConfig {
// --- Security ---
/** SSL configuration. `true` for default SSL, or a TLS ConnectionOptions object. */
ssl?: boolean | ConnectionOptions;
ssl?: boolean;
// --- SDK options ---
/** Table name prefix for multi-tenant isolation. Alphanumeric and underscores only. */
namespace?: string;
}
};
export class PostgresMemory implements BuiltMemory {
export type PostgresConstructorOptions = (
| {
type: 'credential';
credential: string;
}
| {
type: 'connection';
connection: PostgresConnectionOptions;
}
) & {
options?: PostgresMemoryOptions;
};
export class PostgresMemory extends BaseMemory<PostgresConstructorOptions> {
private initPromise: Promise<Pool> | null = null;
private embeddingsInitPromise: Promise<void> | null = null;
private readonly config: PostgresMemoryConfig;
private readonly ns: string;
constructor(config: PostgresMemoryConfig) {
if (config.namespace !== undefined) {
if (!/^[a-zA-Z0-9_]+$/.test(config.namespace)) {
constructor(
protected readonly constructorOptions: PostgresConstructorOptions,
private readonly resolveConfig?: (credential: string) => Promise<PostgresConnectionOptions>,
) {
super('postgres', constructorOptions);
const namespace = constructorOptions.options?.namespace;
if (namespace !== undefined) {
if (!/^[a-zA-Z0-9_]+$/.test(namespace)) {
throw new Error(
`Invalid namespace "${config.namespace}": must be alphanumeric and underscores only`,
`Invalid namespace "${namespace}": must be alphanumeric and underscores only`,
);
}
}
this.config = config;
this.ns = config.namespace ? `${config.namespace}_` : '';
this.ns = namespace ? `${namespace}_` : '';
}
// ── Lazy initialisation ──────────────────────────────────────────────
@ -115,34 +130,53 @@ export class PostgresMemory implements BuiltMemory {
private async _initialize(): Promise<Pool> {
const { Pool: PgPool } = await import('pg');
const conn = this.config.connection;
const connectionOpts =
typeof conn === 'string'
? { connectionString: conn }
: {
...(conn.host && { host: conn.host }),
...(conn.port && { port: conn.port }),
...(conn.database && { database: conn.database }),
...(conn.user && { user: conn.user }),
...(conn.password !== undefined && { password: conn.password }),
};
let connectionOpts: Record<string, unknown>;
if (this.constructorOptions.type === 'credential' && !this.resolveConfig) {
throw new Error('resolveConfig() was not provided in constructor options');
}
const config =
this.constructorOptions.type === 'credential'
? await this.resolveConfig!(this.constructorOptions.credential)
: this.constructorOptions.connection;
if (config.connectionType === 'url') {
const url = config.connection.url;
connectionOpts = { connectionString: url };
} else {
const cfg = config.connection;
const host = cfg.host;
const port = cfg.port;
const database = cfg.database;
const user = cfg.user;
const password = cfg.password;
connectionOpts = {
...(host !== undefined && { host }),
...(port !== undefined && { port }),
...(database !== undefined && { database }),
...(user !== undefined && { user }),
...(password !== undefined && { password }),
};
}
const opts = this.constructorOptions.options;
const pool = new PgPool({
...connectionOpts,
// Pool
...(this.config.pool?.max !== undefined && { max: this.config.pool.max }),
...(this.config.pool?.min !== undefined && { min: this.config.pool.min }),
...(this.config.pool?.idleTimeoutMillis !== undefined && {
idleTimeoutMillis: this.config.pool.idleTimeoutMillis,
...(opts?.pool?.max !== undefined && { max: opts.pool.max }),
...(opts?.pool?.min !== undefined && { min: opts.pool.min }),
...(opts?.pool?.idleTimeoutMillis !== undefined && {
idleTimeoutMillis: opts.pool.idleTimeoutMillis,
}),
...(this.config.pool?.connectionTimeoutMillis !== undefined && {
connectionTimeoutMillis: this.config.pool.connectionTimeoutMillis,
...(opts?.pool?.connectionTimeoutMillis !== undefined && {
connectionTimeoutMillis: opts.pool.connectionTimeoutMillis,
}),
...(this.config.pool?.allowExitOnIdle !== undefined && {
allowExitOnIdle: this.config.pool.allowExitOnIdle,
...(opts?.pool?.allowExitOnIdle !== undefined && {
allowExitOnIdle: opts.pool.allowExitOnIdle,
}),
// Security
...(this.config.ssl !== undefined && { ssl: this.config.ssl }),
...(opts?.ssl !== undefined && { ssl: opts.ssl }),
});
await pool.query(

View File

@ -1,6 +1,8 @@
import type { Client, InArgs } from '@libsql/client';
import { z } from 'zod';
import type { BuiltMemory, Thread } from '../types/sdk/memory';
import { BaseMemory } from './base-memory';
import type { Thread } from '../types/sdk/memory';
import type { AgentDbMessage } from '../types/sdk/message';
/** Safe JSON.parse wrapper — returns undefined on failure. */
@ -18,12 +20,19 @@ function float32ToBuffer(arr: number[]): Buffer {
return Buffer.from(f32.buffer);
}
export interface SqliteMemoryConfig {
url: string; // e.g. 'file:./data.db'
namespace?: string; // table name prefix
}
export const SqliteMemoryConfigSchema = z.object({
/** libsql connection URL. Use `'file:./path/to/db.sqlite'` for a local file. */
url: z.string().min(1),
/** Optional table name prefix for multi-tenant isolation. Alphanumeric and underscores only. */
namespace: z
.string()
.regex(/^[a-zA-Z0-9_]+$/)
.optional(),
});
export class SqliteMemory implements BuiltMemory {
export type SqliteMemoryConfig = z.infer<typeof SqliteMemoryConfigSchema>;
export class SqliteMemory extends BaseMemory<SqliteMemoryConfig> {
private initPromise: Promise<Client> | null = null;
private embeddingsInitPromise: Promise<void> | null = null;
@ -32,16 +41,10 @@ export class SqliteMemory implements BuiltMemory {
private readonly ns: string;
constructor(config: SqliteMemoryConfig) {
if (config.namespace !== undefined) {
if (!/^[a-zA-Z0-9_]+$/.test(config.namespace)) {
throw new Error(
`Invalid namespace "${config.namespace}": must be alphanumeric and underscores only`,
);
}
}
this.config = config;
this.ns = config.namespace ? `${config.namespace}_` : '';
constructor(protected readonly constructorOptions: SqliteMemoryConfig) {
super('sqlite', constructorOptions);
this.config = SqliteMemoryConfigSchema.parse(constructorOptions);
this.ns = constructorOptions.namespace ? `${constructorOptions.namespace}_` : '';
}
// ── Lazy initialisation ──────────────────────────────────────────────

View File

@ -9,7 +9,6 @@ export type {
ContentReasoning,
ContentFile,
ContentToolCall,
ContentToolResult,
ContentInvalidToolCall,
ContentProvider,
Message,
@ -61,6 +60,7 @@ export type {
export type {
Thread,
BuiltMemory,
MemoryDescriptor,
SemanticRecallConfig,
MemoryConfig,
CheckpointStore,

View File

@ -1,5 +1,4 @@
import type { AgentPersistenceOptions } from '../sdk/agent';
import type { AgentMessage, ContentToolResult } from '../sdk/message';
import type { AgentMessage, ContentToolCall } from '../sdk/message';
export const enum AgentEvent {
AgentStart = 'agent_start',
@ -11,16 +10,11 @@ export const enum AgentEvent {
Error = 'error',
}
export type SharedAgentEventData = {
runId: string;
persistence?: AgentPersistenceOptions;
};
export type AgentEventSpecificData =
export type AgentEventData =
| { type: AgentEvent.AgentStart }
| { type: AgentEvent.AgentEnd; messages: AgentMessage[] }
| { type: AgentEvent.TurnStart }
| { type: AgentEvent.TurnEnd; message: AgentMessage; toolResults: ContentToolResult[] }
| { type: AgentEvent.TurnEnd; message: AgentMessage; toolResults: ContentToolCall[] }
| { type: AgentEvent.ToolExecutionStart; toolCallId: string; toolName: string; args: unknown }
| {
type: AgentEvent.ToolExecutionEnd;
@ -31,8 +25,6 @@ export type AgentEventSpecificData =
}
| { type: AgentEvent.Error; message: string; error: unknown };
export type AgentEventData = SharedAgentEventData & AgentEventSpecificData;
export type AgentEventHandler = (data: AgentEventData) => void;
// Can be used for observability or controlling the agent. The idea that HITL, guardrails, logging, etc. can be done as middleware and single point of entry.

View File

@ -1,5 +1,4 @@
import type { ModelConfig } from './agent';
import type { CredentialProvider } from './credential-provider';
import type { BuiltEval } from './eval';
import type { BuiltGuardrail } from './guardrail';
import type { CheckpointStore } from './memory';
@ -16,7 +15,6 @@ import type { BuiltProviderTool, BuiltTool } from './tool';
*/
export interface AgentBuilder {
model(providerOrIdOrConfig: string | ModelConfig, modelName?: string): this;
credential(name: string): this;
instructions(text: string): this;
tool(t: BuiltTool | BuiltTool[]): this;
providerTool(t: BuiltProviderTool): this;
@ -25,7 +23,6 @@ export interface AgentBuilder {
requireToolApproval(): this;
memory(m: unknown): this;
checkpoint(storage: 'memory' | CheckpointStore): this;
credentialProvider(p: CredentialProvider): this;
inputGuardrail(g: BuiltGuardrail): this;
outputGuardrail(g: BuiltGuardrail): this;
eval(e: BuiltEval): this;

View File

@ -4,6 +4,7 @@ import type { JsonSchema7Type } from 'zod-to-json-schema';
import type { AgentMessage, ContentMetadata } from './message';
import type { BuiltTool } from './tool';
import type { ProviderId, ProviderCredentials } from '../../runtime/provider-credentials';
import type { AgentEvent, AgentEventHandler } from '../runtime/event';
import type { SerializedMessageList } from '../runtime/message-list';
import type { BuiltTelemetry } from '../telemetry';
@ -27,12 +28,20 @@ export type TokenUsage<T extends Record<string, unknown> = Record<string, unknow
additionalMetadata?: T;
};
/* eslint-disable @typescript-eslint/no-redundant-type-constituents -- LanguageModel is semantically distinct from string */
/**
* Typed model config for known providers gives IDE autocompletion for
* provider-specific credential fields based on the model id prefix.
*/
export type TypedModelConfig = {
[P in ProviderId]: { id: `${P}/${string}` } & ProviderCredentials<P>;
}[ProviderId];
export type ModelConfig =
| string
| { id: string; apiKey?: string; url?: string; headers?: Record<string, string> }
| TypedModelConfig
| { id: string; [k: string]: unknown }
// eslint-disable-next-line @typescript-eslint/no-redundant-type-constituents -- LanguageModel is semantically distinct from string
| LanguageModel;
/* eslint-enable @typescript-eslint/no-redundant-type-constituents */
export interface AgentResult {
id?: string;
@ -53,6 +62,47 @@ export interface AgentResult {
export type StreamChunk = ContentMetadata &
(
| { type: 'start-step' }
| { type: 'finish-step' }
| { type: 'text-start'; id: string }
| { type: 'text-delta'; id: string; delta: string }
| { type: 'text-end'; id: string }
| { type: 'reasoning-start'; id: string }
| { type: 'reasoning-delta'; id: string; delta: string }
| { type: 'reasoning-end'; id: string }
| { type: 'tool-input-start'; toolCallId: string; toolName: string }
| { type: 'tool-input-delta'; toolCallId: string; delta: string }
| { type: 'tool-call'; toolCallId: string; toolName: string; input: unknown }
| {
/**
* Emitted just before a tool handler starts executing. Bridged from
* the runtime event bus (not part of the AI SDK fullStream). Pairs
* with the subsequent `tool-result` to let consumers show a
* mid-flight indicator between "LLM picked a tool" and "result arrived".
*/
type: 'tool-execution-start';
toolCallId: string;
toolName: string;
}
| {
type: 'tool-result';
toolCallId: string;
toolName: string;
output: unknown;
isError?: boolean;
}
| {
type: 'tool-call-suspended';
runId: string;
toolCallId: string;
toolName: string;
input?: unknown;
suspendPayload?: unknown;
/** JSON Schema describing the shape of data to send when resuming. */
resumeSchema?: JsonSchema7Type;
}
// `message` is reserved for sub-agent / app-defined `CustomAgentMessage`
| { type: 'message'; message: AgentMessage }
| {
type: 'finish';
finishReason: FinishReason;
@ -62,41 +112,7 @@ export type StreamChunk = ContentMetadata &
subAgentUsage?: SubAgentUsage[];
totalCost?: number;
}
| {
type: 'text-delta';
id?: string;
delta: string;
}
| {
type: 'reasoning-delta';
id?: string;
delta: string;
}
| {
type: 'tool-call-delta';
id?: string;
name?: string;
argumentsDelta?: string;
}
| {
type: 'error';
error: unknown;
}
| {
type: 'message';
message: AgentMessage;
id?: string;
}
| {
type: 'tool-call-suspended';
runId?: string;
toolCallId?: string;
toolName?: string;
input?: unknown;
suspendPayload?: unknown;
/** JSON Schema describing the shape of data to send when resuming. */
resumeSchema?: JsonSchema7Type;
}
| { type: 'error'; error: unknown }
);
export interface RunOptions {
@ -167,8 +183,6 @@ export interface GenerateResult {
* callers can handle them without try/catch.
*/
error?: unknown;
/** Return a snapshot of the agent state at the end of this run. */
getState(): SerializableAgentState;
}
export interface StreamResult {
@ -176,12 +190,6 @@ export interface StreamResult {
runId: string;
/** The readable stream of chunks. */
stream: ReadableStream<StreamChunk>;
/**
* Return the current agent state for this run.
* May be called at any time during streaming to observe live status,
* or after the stream closes to confirm the terminal state.
*/
getState(): SerializableAgentState;
}
export interface ResumeOptions {
@ -205,6 +213,8 @@ export interface BuiltAgent {
asTool(description: string): BuiltTool;
getState(): SerializableAgentState;
/** Cancel the currently running agent. Synchronous — sets an abort flag that the agentic loop checks asynchronously. */
abort(): void;
@ -256,6 +266,7 @@ export type PendingToolCall = {
suspended: true;
suspendPayload: unknown;
resumeSchema: JsonSchema7Type;
runId: string;
}
| {
suspended: false;

View File

@ -1,8 +1,7 @@
/** A resolved credential containing at minimum an API key. */
export interface ResolvedCredential {
apiKey: string;
/** A resolved credential. May contain an API key and/or provider-specific fields. */
export type ResolvedCredential = {
[key: string]: unknown;
}
};
/** Summary of a credential available for use. */
export interface CredentialListItem {

View File

@ -2,6 +2,20 @@ import type { z } from 'zod';
import type { ModelConfig, SerializableAgentState } from './agent';
import type { AgentDbMessage } from './message';
import type { JSONObject } from '../utils/json';
/**
* Serializable descriptor returned by BuiltMemory.describe().
* Contains enough information to reconstruct the backend from a schema without exposing secrets.
*/
export interface MemoryDescriptor<TParams extends JSONObject = JSONObject> {
/** Backend name (e.g. 'postgres', 'sqlite', 'memory'). Used as key in memoryRegistry. */
name: string;
/** Constructor name (e.g. 'PostgresMemory', 'SqliteMemory'). Used to construct the backend. */
constructorName: string;
/** Non-secret, serializable connection parameters. CredentialConfig refs are safe to store. */
connectionParams: TParams | null;
}
export interface Thread {
id: string;
@ -84,6 +98,8 @@ export interface BuiltMemory {
// --- Lifecycle (optional) ---
/** Close the connection pool / release resources. No-op for in-memory backends. */
close?(): Promise<void>;
/** Return a serializable descriptor of this backend for schema persistence. */
describe(): MemoryDescriptor;
}
// --- Semantic Recall Config ---
@ -103,6 +119,8 @@ export interface TitleGenerationConfig {
model?: ModelConfig;
/** Custom instructions for the title generation prompt. Replaces the defaults entirely. */
instructions?: string;
/** When true, title generation is awaited before returning the result. Default: false (fire-and-forget). */
sync?: boolean;
}
/** Full memory configuration bundle passed from builder to runtime. */

View File

@ -8,7 +8,6 @@ export type MessageContent =
| ContentText
| ContentToolCall
| ContentInvalidToolCall
| ContentToolResult
| ContentReasoning
| ContentFile
| ContentCitation
@ -90,7 +89,7 @@ export type ContentToolCall = ContentMetadata & {
/**
* The identifier of the tool call. It must be unique across all tool calls.
*/
toolCallId?: string;
toolCallId: string;
/**
* The name of the tool that should be called.
@ -104,31 +103,11 @@ export type ContentToolCall = ContentMetadata & {
input: JSONValue;
providerExecuted?: boolean;
};
export type ContentToolResult = ContentMetadata & {
type: 'tool-result';
/**
* The name of the tool that was called.
*/
toolName: string;
/**
* The ID of the tool call that this result is associated with.
*/
toolCallId: string;
/**
* Result of the tool call. This is a JSON-serializable object.
*/
result: JSONValue;
/**
* Optional flag if the result is an error or an error message.
*/
isError?: boolean;
};
} & (
| { state: 'pending' }
| { state: 'resolved'; output: JSONValue }
| { state: 'rejected'; error: string }
);
export type ContentInvalidToolCall = ContentMetadata & {
type: 'invalid-tool-call';

View File

@ -0,0 +1,21 @@
import type { JSONSchema7 } from 'json-schema';
export interface ToolDescriptor {
name: string;
description: string;
/**
* Behavioural directive paired with the tool. Persisted on the descriptor
* so it survives the JSON-config save publish reconstruct cycle for
* custom tools without this, `Tool.systemInstruction(...)` would only
* apply to in-memory/runtime-injected tools and would silently drop on
* reload.
*/
systemInstruction: string | null;
inputSchema: JSONSchema7 | null;
outputSchema: JSONSchema7 | null;
hasSuspend: boolean;
hasResume: boolean;
hasToMessage: boolean;
requireApproval: boolean;
providerOptions: Record<string, unknown> | null;
}

View File

@ -26,8 +26,16 @@ export interface InterruptibleToolContext<S = unknown, R = unknown> {
export interface BuiltTool {
readonly name: string;
readonly description: string;
readonly suspendSchema?: ZodType;
readonly resumeSchema?: ZodType;
/**
* Behavioural directive paired with the tool, injected into the agent's
* system prompt under a `<built_in_rules>` block when the tool is added.
* Use for guidance the LLM needs to *decide whether to call* the tool
* tool descriptions answer "what does this do?" but are weighted lower
* than system instructions for usage decisions.
*/
readonly systemInstruction?: string;
readonly suspendSchema?: ZodType | JSONSchema7;
readonly resumeSchema?: ZodType | JSONSchema7;
readonly withDefaultApproval?: boolean;
readonly toMessage?: (output: unknown) => AgentMessage | undefined;
/**
@ -44,7 +52,7 @@ export interface BuiltTool {
* (MCP tools). Use `isZodSchema()` to distinguish between the two at runtime.
*/
readonly inputSchema?: ZodType | JSONSchema7;
readonly outputSchema?: ZodType;
readonly outputSchema?: ZodType | JSONSchema7;
/** True for tools sourced from an MCP server. */
readonly mcpTool?: boolean;
/** Name of the MCP server this tool belongs to. Set when mcpTool is true. */
@ -56,6 +64,17 @@ export interface BuiltTool {
* Example: `{ anthropic: { eagerInputStreaming: true } }`
*/
readonly providerOptions?: Record<string, JSONObject>;
/**
* Arbitrary platform-specific metadata attached to the tool.
*/
readonly metadata?: Record<string, unknown>;
/**
* Whether the tool has source code that can be introspected.
* When `false`, the tool is treated as a platform-managed marker (e.g. an
* externally-resolved tool) and its source is not introspected.
* Defaults to `true` when absent.
*/
readonly editable?: boolean;
}
/**

View File

@ -0,0 +1,40 @@
import type AjvType from 'ajv';
import type { JSONSchema7 } from 'json-schema';
import type { ZodType } from 'zod';
import { isZodSchema } from './zod';
export type ParseResult<T = unknown> =
| { success: true; data: T }
| { success: false; error: string };
let ajvInstance: InstanceType<typeof AjvType> | undefined;
function getAjv(): InstanceType<typeof AjvType> {
if (!ajvInstance) {
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { default: Ajv } = require('ajv') as { default: typeof AjvType };
ajvInstance = new Ajv({ strict: false });
}
return ajvInstance;
}
/**
* Validate `data` against a Zod schema or a raw JSON Schema.
* Returns a unified success/failure result, with parsed data on success.
*/
export async function parseWithSchema(
schema: ZodType | JSONSchema7,
data: unknown,
): Promise<ParseResult> {
if (isZodSchema(schema)) {
const result = await schema.safeParseAsync(data);
if (result.success) return { success: true, data: result.data };
return { success: false, error: result.error.message };
}
const ajv = getAjv();
const validate = ajv.compile(schema);
if (validate(data)) return { success: true, data };
return { success: false, error: ajv.errorsText(validate.errors) };
}

View File

@ -2,12 +2,12 @@ import type { JSONSchema7 } from 'json-schema';
import type { ZodType } from 'zod';
import { zodToJsonSchema as zodToJsonSchemaImpl } from 'zod-to-json-schema';
/** Type guard: returns true when a tool input schema is a Zod schema (as opposed to raw JSON Schema). */
export function isZodSchema(schema: ZodType | JSONSchema7): schema is ZodType {
/** Type guard: returns true when a value is a Zod schema (as opposed to raw JSON Schema or any other shape). */
export function isZodSchema(schema: unknown): schema is ZodType {
return (
typeof schema === 'object' &&
schema !== null &&
typeof (schema as ZodType).safeParse === 'function'
typeof (schema as { safeParse?: unknown }).safeParse === 'function'
);
}

View File

@ -1,6 +1,6 @@
{
"name": "@n8n/ai-node-sdk",
"version": "0.10.0",
"version": "0.11.0",
"description": "SDK for building AI nodes in n8n",
"types": "dist/esm/index.d.ts",
"module": "dist/esm/index.js",

View File

@ -1,6 +1,6 @@
{
"name": "@n8n/ai-utilities",
"version": "0.13.0",
"version": "0.14.0",
"description": "Utilities for building AI nodes in n8n",
"types": "dist/esm/index.d.ts",
"module": "dist/esm/index.js",

View File

@ -1,6 +1,6 @@
{
"name": "@n8n/ai-workflow-builder",
"version": "1.19.0",
"version": "1.20.0",
"scripts": {
"clean": "rimraf dist .turbo",
"typecheck": "tsc --noEmit",

View File

@ -115,19 +115,23 @@ export class CodeBuilderNodeSearchEngine {
* @param limit - Maximum number of results to return
* @returns Array of matching nodes sorted by relevance
*/
searchByName(query: string, limit: number = 20): CodeBuilderNodeSearchResult[] {
searchByName(
query: string,
limit: number = 20,
nodeFilter?: (nodeId: string) => boolean,
): CodeBuilderNodeSearchResult[] {
const nodeTypes = nodeFilter
? this.nodeTypes.filter((node) => nodeFilter(node.name))
: this.nodeTypes;
// Use sublimeSearch for fuzzy matching
const searchResults = sublimeSearch<INodeTypeDescription>(
query,
this.nodeTypes,
NODE_SEARCH_KEYS,
);
const searchResults = sublimeSearch<INodeTypeDescription>(query, nodeTypes, NODE_SEARCH_KEYS);
const queryLower = query.toLowerCase().trim();
const fuzzyResultNames = new Set(searchResults.map((r) => r.item.name));
// Direct type name match on all nodeTypes (catches nodes sublimeSearch ranked too low)
const typeNameMatches = this.nodeTypes
const typeNameMatches = nodeTypes
.filter((node) => {
if (fuzzyResultNames.has(node.name)) return false;
return getTypeName(node.name).toLowerCase() === queryLower;

View File

@ -141,6 +141,23 @@ describe('CodeBuilderNodeSearchEngine', () => {
expect(results.length).toBeLessThanOrEqual(2);
});
it('should apply nodeFilter before ranking and limiting results', () => {
const httpToolNode = createNodeType({
name: 'n8n-nodes-base.httpRequestTool',
displayName: 'HTTP Request Tool',
description: 'Makes HTTP requests as an AI tool',
group: ['output'],
inputs: [],
outputs: ['ai_tool'],
});
const engine = new CodeBuilderNodeSearchEngine([...nodeTypes, httpToolNode]);
const results = engine.searchByName('http', 1, (nodeId) => nodeId.endsWith('Tool'));
expect(results).toHaveLength(1);
expect(results[0].name).toBe('n8n-nodes-base.httpRequestTool');
});
it('should combine scores for multiple matches', () => {
const results = searchEngine.searchByName('request');

View File

@ -195,8 +195,8 @@ export class ParseValidateHandler {
builder.generatePinData({ beforeWorkflow: currentWorkflow });
}
// Convert to JSON
const workflowJson: WorkflowJSON = builder.toJSON();
// Convert to JSON with Dagre layout matching the FE's tidy-up
const workflowJson: WorkflowJSON = builder.toJSON({ tidyUp: true });
this.logger?.debug('Parsed workflow', {
id: workflowJson.id,

View File

@ -21,6 +21,7 @@ export { generateCodeBuilderThreadId } from './utils/code-builder-session';
export { NodeTypeParser } from './utils/node-type-parser';
export { ParseValidateHandler, WorkflowCodeParseError } from './handlers/parse-validate-handler';
export { createCodeBuilderSearchTool } from './tools/code-builder-search.tool';
export type { CodeBuilderSearchToolOptions } from './tools/code-builder-search.tool';
export { createCodeBuilderGetTool } from './tools/code-builder-get.tool';
export type { CodeBuilderGetToolOptions } from './tools/code-builder-get.tool';
export { createGetSuggestedNodesTool } from './tools/get-suggested-nodes.tool';

View File

@ -99,15 +99,16 @@ startTrigger.to(sourceA.to(sourceB.to(processResults)));
// Pairs items by index, merging fields from both inputs into one item.
// @example input0: [{ a: 1 }, { a: 2 }] input1: [{ b: 10, c: 'x' }, { b: 20 }]
// output: [{ a: 1, b: 10, c: 'x' }, { a: 2, b: 20, c: undefined }]
// .input(n) is 0-based: .input(0) = first input, .input(1) = second input.
const combineResults = merge({
version: 3.2,
config: { name: 'Combine Results', parameters: { mode: 'combine', combineBy: 'combineByPosition' } }
});
export default workflow('id', 'name')
.add(startTrigger)
.to(sourceA.to(combineResults.input(0)))
.to(sourceA.to(combineResults.input(0))) // first input (index 0)
.add(startTrigger)
.to(sourceB.to(combineResults.input(1)))
.to(sourceB.to(combineResults.input(1))) // second input (index 1)
.add(combineResults)
.to(processResults);
@ -121,9 +122,9 @@ const allResults = merge({
});
export default workflow('id', 'name')
.add(startTrigger)
.to(sourceA.to(allResults.input(0)))
.to(sourceA.to(allResults.input(0))) // first input (index 0)
.add(startTrigger)
.to(sourceB.to(allResults.input(1)))
.to(sourceB.to(allResults.input(1))) // second input (index 1)
.add(allResults)
.to(processResults);
\`\`\`
@ -180,12 +181,13 @@ const branch1 = node({ type: 'n8n-nodes-base.httpRequest', ... });
const branch2 = node({ type: 'n8n-nodes-base.httpRequest', ... });
const processResults = node({ type: 'n8n-nodes-base.set', ... });
// Connect branches to specific merge inputs using .input(n)
// Connect branches to specific merge inputs using .input(n).
// Indices are 0-based: .input(0) is the FIRST input, .input(1) is the SECOND.
export default workflow('id', 'name')
.add(trigger({ ... }))
.to(branch1.to(combineResults.input(0))) // Connect to input 0
.to(branch1.to(combineResults.input(0))) // first input (index 0)
.add(trigger({ ... }))
.to(branch2.to(combineResults.input(1))) // Connect to input 1
.to(branch2.to(combineResults.input(1))) // second input (index 1)
.add(combineResults)
.to(processResults); // Process merged results
\`\`\`

View File

@ -44,9 +44,10 @@ export function isValidPathComponent(component: string): boolean {
export function validatePathWithinBase(filePath: string, baseDir: string): boolean {
const resolvedPath = resolve(filePath);
const resolvedBase = resolve(baseDir);
const separator = process.platform === 'win32' ? '\\' : '/';
// Path must start with base directory (with trailing separator to prevent prefix attacks)
return resolvedPath.startsWith(resolvedBase + '/') || resolvedPath === resolvedBase;
return resolvedPath.startsWith(resolvedBase + separator) || resolvedPath === resolvedBase;
}
/**
@ -64,24 +65,22 @@ function getGeneratedNodesPaths(nodeDefinitionDirs?: string[]): string[] {
/**
* Find the first nodes path that contains the given node directory.
* Returns { nodesPath, nodeDir } or null if not found in any dir.
*
* Tool variants (e.g. "slackHitlTool", "httpRequestTool") have no on-disk type files;
* fall back to the base node by stripping the suffix. The regex tries "HitlTool" before
* "Tool" so "slackHitlTool" resolves to "slack", not "slackHitl".
*/
function findNodeDir(
parsed: { packageName: string; nodeName: string },
nodesPaths: string[],
): { nodesPath: string; nodeDir: string } | null {
for (const nodesPath of nodesPaths) {
const nodeDir = join(nodesPath, parsed.packageName, parsed.nodeName);
if (existsSync(nodeDir)) {
return { nodesPath, nodeDir };
}
}
const candidates = [parsed.nodeName];
const baseName = parsed.nodeName.replace(/(?:HitlTool|Tool)$/, '');
if (baseName !== parsed.nodeName) candidates.push(baseName);
// Tool variant fallback: e.g. "httpRequestTool" -> "httpRequest"
// Tool variants share type definitions with their base node
if (parsed.nodeName.endsWith('Tool')) {
const baseName = parsed.nodeName.slice(0, -4);
for (const candidate of candidates) {
for (const nodesPath of nodesPaths) {
const nodeDir = join(nodesPath, parsed.packageName, baseName);
const nodeDir = join(nodesPath, parsed.packageName, candidate);
if (existsSync(nodeDir)) {
return { nodesPath, nodeDir };
}
@ -355,10 +354,11 @@ function resolveModePath(
}
/**
* Try to resolve file path for a specific node ID
* This is the core resolution logic used by getNodeFilePath
* Get the file path for a node ID, optionally for a specific version and discriminators.
* If no version specified, returns the latest version. If the node uses split structure,
* discriminators are required. Tool-variant fallback is handled in findNodeDir.
*/
function tryGetNodeFilePath(
function getNodeFilePath(
nodeId: string,
version: string | undefined,
nodeDefinitionDirs: string[] | undefined,
@ -455,35 +455,6 @@ function tryGetNodeFilePath(
return { filePath };
}
/**
* Get the file path for a node ID, optionally for a specific version and discriminators
* If no version specified, returns the latest version
* If node uses split structure, discriminators are required
*
* For tool variants (e.g., "googleCalendarTool"), falls back to the base node
* (e.g., "googleCalendar") since tool variants don't have separate type files.
*/
function getNodeFilePath(
nodeId: string,
version?: string,
nodeDefinitionDirs?: string[],
discriminators?: { resource?: string; operation?: string; mode?: string },
): PathResolutionResult {
// Try exact node ID first
let result = tryGetNodeFilePath(nodeId, version, nodeDefinitionDirs, discriminators);
// If not found and node name ends with 'Tool', try base node as fallback
// (e.g., n8n-nodes-base.googleCalendarTool -> n8n-nodes-base.googleCalendar)
// Note: Some nodes legitimately end in Tool (agentTool, mcpClientTool) but those
// have their own type files, so this fallback only triggers when no file is found
if (result.error && nodeId.endsWith('Tool')) {
const baseNodeId = nodeId.slice(0, -4);
result = tryGetNodeFilePath(baseNodeId, version, nodeDefinitionDirs, discriminators);
}
return result;
}
/**
* Get the type definition for a single node ID, optionally for a specific version and discriminators
*/

Some files were not shown because too many files have changed in this diff Show More