mirror of
https://github.com/n8n-io/n8n.git
synced 2026-10-18 05:59:42 +02:00
fix(core): Enable Claude prompt caching for OpenRouter agents (no-changelog)
Share Claude cache capabilities and defaults with Vertex. Forward OpenRouter reasoning settings and correct cache-write token accounting.
This commit is contained in:
parent
a36cff36cd
commit
c8eb21b2a0
|
|
@ -1,8 +1,8 @@
|
|||
# Prompt caching
|
||||
|
||||
`Agent.promptCaching()` generates provider-specific `providerOptions` for
|
||||
Anthropic and OpenAI so callers don't have to hand-write the Vercel AI SDK's
|
||||
raw `cacheControl` / `promptCacheKey` shapes. It layers on top of the existing
|
||||
Anthropic, Claude on OpenRouter and Vertex, and OpenAI. Callers do not have to
|
||||
write the AI SDK's raw `cacheControl` / `promptCacheKey` shapes. It uses the existing
|
||||
`providerOptions` escape hatch — nothing here is required, and anything set
|
||||
explicitly (via `.instructions(text, { providerOptions })`, call-level
|
||||
`providerOptions`, or `Tool.providerOptions(...)`) always wins on conflicts.
|
||||
|
|
@ -29,24 +29,24 @@ new Agent('assistant')
|
|||
The richer SDK shape above (OpenAI key/retention overrides) is for direct SDK
|
||||
callers. The n8n Agent product (JSON config / builder UI) exposes
|
||||
`config.promptCaching: { enabled: boolean; anthropic?: { ttl?: '5m' | '1h' } }`
|
||||
and maps it straight to `agent.promptCaching(config.promptCaching)`. Two
|
||||
differences from the general SDK usage worth knowing:
|
||||
and passes explicit settings to `agent.promptCaching(config.promptCaching)`.
|
||||
|
||||
- **Mandatory, not opt-in or opt-out.** For OpenAI and Anthropic agents, the
|
||||
config is always force-written with `{ enabled: true, ... }` — by the
|
||||
- **Mandatory in the builder.** For OpenAI, Anthropic, and Claude on OpenRouter
|
||||
or Vertex, the config is always force-written with `{ enabled: true, ... }` by the
|
||||
builder agent's write-path normalizer and the model picker — regardless of
|
||||
what the LLM or a prior config said; there is no user-facing way to disable
|
||||
it. Every other provider never gets the field at all. As with the SDK, a
|
||||
missing `promptCaching` field is still just "disabled" at runtime — the
|
||||
mandatory-on behavior lives in the config producers (builder tools, model
|
||||
picker), not in the runtime default.
|
||||
- **TTL is Anthropic-only and defaults to `1h`.** The Advanced panel renders
|
||||
a Cache duration dropdown (`5m` / `1h`) only for Anthropic agents; OpenAI
|
||||
has no sub-config or UI (its caching is fully automatic server-side). The
|
||||
TTL is preserved across a switch between Anthropic and another supported
|
||||
provider and back.
|
||||
it. Other models do not get the field.
|
||||
- **Existing routed Claude agents get the default on their next run.** When a
|
||||
saved `openrouter/anthropic/claude-*` or `google-vertex/claude-*` agent has no
|
||||
cache config, the JSON loader supplies `{ enabled: true }`. This does not
|
||||
change the saved config. Explicit settings, including `enabled: false`, pass
|
||||
through unchanged. Standalone SDK use stays opt-in.
|
||||
- **Claude cache duration defaults to `1h`.** The Advanced panel renders
|
||||
a Cache duration dropdown (`5m` / `1h`) for Anthropic and routed Claude agents.
|
||||
OpenAI has no sub-config or UI (its caching is fully automatic server-side).
|
||||
The TTL is preserved when switching between the supported Claude routes.
|
||||
|
||||
## Prefix stability (always on, both providers)
|
||||
## Prefix stability (always on)
|
||||
|
||||
Unlike everything else in this doc, this part is not gated by
|
||||
`.promptCaching()` — it's baseline hygiene that keeps the system-instructions
|
||||
|
|
@ -90,11 +90,12 @@ exists.
|
|||
The block is not part of the base instructions. `buildSystemMessages` sends it
|
||||
as its own system message, after the base instructions and before the volatile
|
||||
message. The tools and base instructions in front of it stay byte-identical, so
|
||||
they stay cached when a skill moves into the block. The block reuses the
|
||||
instruction cache options, so it gets its own breakpoint and stays cached when
|
||||
memory compacts again later. `buildSkillInstructionCacheOptions` drops that
|
||||
breakpoint when it would leave no slot for the conversation breakpoint next to
|
||||
caller breakpoints on tools, history messages, and their content parts.
|
||||
they stay cached when a skill moves into the block. On the native Anthropic API,
|
||||
the block reuses the instruction cache options. It gets its own breakpoint and
|
||||
stays cached when memory compacts again later.
|
||||
`buildSkillInstructionCacheOptions` drops that breakpoint when it would leave
|
||||
no slot for the conversation breakpoint next to caller breakpoints on tools,
|
||||
history messages, and their content parts.
|
||||
Providers without split system messages get the block merged after the base
|
||||
instructions, as before.
|
||||
|
||||
|
|
@ -133,7 +134,8 @@ the cache breakpoint on instructions survives append-only memory growth).
|
|||
- Anthropic requires a minimum prompt length to cache at all (roughly 1,024
|
||||
tokens for the Sonnet family, 4,096 tokens for Opus 4.5 / Haiku 4.5).
|
||||
Below that, the request still succeeds — it's just never cached.
|
||||
- Only Anthropic models get this option; it's a no-op for other providers.
|
||||
- Direct Anthropic and Claude on Vertex use this path. OpenRouter uses the
|
||||
separate path below.
|
||||
|
||||
## Anthropic: runtime breakpoints (conversation history + tools)
|
||||
|
||||
|
|
@ -175,6 +177,36 @@ breakpoint takes priority over the tool breakpoint — it recurs every turn,
|
|||
while a static tool block is already covered as part of the history
|
||||
breakpoint's prefix.
|
||||
|
||||
## OpenRouter: Claude instructions and automatic conversation caching
|
||||
|
||||
For `openrouter/anthropic/claude-*`, `.promptCaching()` adds two controls:
|
||||
|
||||
- An explicit `cache_control` on the stable system instructions.
|
||||
- A top-level `cache_control` for automatic conversation caching. OpenRouter
|
||||
moves this breakpoint as the conversation grows.
|
||||
|
||||
Both use `anthropic.ttl` from the existing config. The default is `1h`.
|
||||
The runtime adds these controls after it assembles the prompt for each call.
|
||||
It does not change stored messages or tool definitions.
|
||||
|
||||
The four-breakpoint limit includes automatic caching. Caller controls take
|
||||
priority, then stable instructions, then automatic caching. The runtime counts
|
||||
both `anthropic` and `openrouter` options and both cache-control key spellings.
|
||||
A message marker on grouped tool results counts once for each result because
|
||||
the adapter copies it to every tool message. Generated controls are omitted
|
||||
when no slot remains or when they would put a longer TTL after a shorter one.
|
||||
An explicit marker on the final block also suppresses generated automatic
|
||||
caching, which can otherwise conflict with that marker's TTL.
|
||||
|
||||
The OpenRouter adapter does not forward cache markers on tool definitions.
|
||||
The runtime adds no such markers, and it adds no markers to tool results or
|
||||
recovered-skill system messages. Stable tool definitions are still part of the
|
||||
cached prefix before the instruction breakpoint.
|
||||
|
||||
Generic `.reasoning()` settings map to `openrouter.reasoning.effort`.
|
||||
Explicit `openrouter.reasoning` options take precedence, including token
|
||||
budgets. The adapter retains reasoning details through tool calls.
|
||||
|
||||
## OpenAI: cache key + retention
|
||||
|
||||
`.promptCaching()` attaches `providerOptions.openai.promptCacheKey` and
|
||||
|
|
@ -205,15 +237,20 @@ Anthropic's native cache breakdown). A healthy Anthropic
|
|||
means the TTL is too short for the traffic pattern, or the cached prefix
|
||||
isn't byte-stable across calls.
|
||||
|
||||
For OpenRouter, the runtime removes cache-write tokens from the adapter's
|
||||
uncached-input count in both generated and streamed responses. Each input
|
||||
token belongs to one cost bucket. Raw provider usage remains unchanged.
|
||||
|
||||
`GenerateResult.usage.cost` / the `finish` chunk's `usage.cost` factor in the
|
||||
model catalog's cache rates when available. One known approximation: the
|
||||
catalog's `cacheWrite` rate reflects Anthropic's 5-minute write premium
|
||||
(1.25x base input); when the effective TTL is `1h` the runtime scales that
|
||||
rate by ~1.6x (2x/1.25x) to approximate the 1h premium, rather than looking
|
||||
up a separate 1h rate from the catalog. This 1h scaling only applies when
|
||||
`.promptCaching()` is enabled for an Anthropic model — cache writes from a
|
||||
manual, caller-supplied `cacheControl` marker (without `.promptCaching()`)
|
||||
are billed at the catalog's 5-minute rate.
|
||||
`.promptCaching()` is enabled for an Anthropic model, including Claude on
|
||||
OpenRouter and Vertex. Cache writes from a manual, caller-supplied
|
||||
`cacheControl` marker (without `.promptCaching()`) use the catalog's 5-minute
|
||||
rate in the estimate.
|
||||
|
||||
## What this does not do
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import { createGoogleVertex } from '@ai-sdk/google-vertex';
|
||||
import { createOpenRouter } from '@openrouter/ai-sdk-provider';
|
||||
import type { LanguageModel } from 'ai';
|
||||
|
||||
import {
|
||||
|
|
@ -192,14 +193,14 @@ vi.mock('@ai-sdk/azure', () => ({
|
|||
}));
|
||||
|
||||
vi.mock('@openrouter/ai-sdk-provider', () => ({
|
||||
createOpenRouter: (opts?: ProviderOpts) => (model: string) => ({
|
||||
createOpenRouter: vi.fn((opts?: ProviderOpts) => (model: string) => ({
|
||||
provider: 'openrouter',
|
||||
modelId: model,
|
||||
apiKey: opts?.apiKey,
|
||||
baseURL: opts?.baseURL,
|
||||
fetch: opts?.fetch,
|
||||
specificationVersion: 'v3',
|
||||
}),
|
||||
})),
|
||||
}));
|
||||
|
||||
// Entra OAuth2 token mint: stubbed so the model-factory's tokenProvider
|
||||
|
|
@ -585,7 +586,7 @@ describe('createModel', () => {
|
|||
}) as unknown as Record<string, unknown>;
|
||||
expect(model.provider).toBe('openrouter');
|
||||
expect(model.modelId).toBe('openai/gpt-4o');
|
||||
expect(model.apiKey).toBe('or-test');
|
||||
expect(createOpenRouter).toHaveBeenCalledWith(expect.objectContaining({ apiKey: 'or-test' }));
|
||||
});
|
||||
|
||||
it('should create model for nvidia with SDK defaults for usage and structured outputs', () => {
|
||||
|
|
|
|||
|
|
@ -1,5 +1,8 @@
|
|||
import { createRuntimeSkillSource } from '../../../skills/registry';
|
||||
import type { ModelConfig } from '../../../types/sdk/agent';
|
||||
import { getModelIdString } from '../../../utils/model';
|
||||
import { AgentMessageList } from '../../model/message-list';
|
||||
import { ActiveSkills } from '../../skills/active-skills';
|
||||
import { RuntimeContextBuilder } from '../runtime-context';
|
||||
|
||||
describe('getModelIdString', () => {
|
||||
|
|
@ -36,6 +39,53 @@ describe('getModelIdString', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('OpenRouter recovered skills', () => {
|
||||
it('keeps the caller instruction marker without copying it to recovered skills', async () => {
|
||||
const marker = { anthropic: { cacheControl: { type: 'ephemeral', ttl: '1h' } } };
|
||||
const builder = new RuntimeContextBuilder(
|
||||
{
|
||||
name: 'test',
|
||||
model: 'openrouter/anthropic/claude-sonnet-4.5',
|
||||
instructions: 'Base instructions',
|
||||
instructionProviderOptions: marker,
|
||||
promptCaching: { enabled: true },
|
||||
},
|
||||
undefined,
|
||||
);
|
||||
const list = new AgentMessageList();
|
||||
list.addInput([{ role: 'user', content: [{ type: 'text', text: 'Hello' }] }]);
|
||||
const source = createRuntimeSkillSource([
|
||||
{
|
||||
id: 'notes',
|
||||
name: 'Notes',
|
||||
description: 'Summarize notes.',
|
||||
instructions: 'Extract decisions.',
|
||||
},
|
||||
]);
|
||||
const activeSkills = new ActiveSkills(source, 'test');
|
||||
await activeSkills.restore(list);
|
||||
await activeSkills.load('notes');
|
||||
const before = list.serialize();
|
||||
|
||||
const prompt = builder.buildModelPrompt({
|
||||
list,
|
||||
tools: builder.buildToolLoopContext({}),
|
||||
instructionProviderOptions: builder.buildInstructionProviderOptions(),
|
||||
providerOptions: { openrouter: { user: 'test-user' } },
|
||||
hostVolatileInstructions: 'Current context',
|
||||
activeSkills,
|
||||
});
|
||||
|
||||
expect(prompt.system).toHaveLength(3);
|
||||
expect(JSON.stringify(prompt.system).match(/"cacheControl":/g)).toHaveLength(1);
|
||||
expect(prompt.system).toContainEqual(expect.objectContaining({ providerOptions: marker }));
|
||||
expect(prompt.providerOptions).toEqual({
|
||||
openrouter: { user: 'test-user', cacheControl: { type: 'ephemeral', ttl: '1h' } },
|
||||
});
|
||||
expect(list.serialize()).toEqual(before);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Vertex thinking options', () => {
|
||||
it.each([
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1063,6 +1063,7 @@ export class AgentRuntime {
|
|||
list,
|
||||
tools,
|
||||
instructionProviderOptions: ctx.instructionProviderOptions,
|
||||
providerOptions: staticContext.providerOptions,
|
||||
hostVolatileInstructions,
|
||||
activeSkills: this.activeSkills,
|
||||
});
|
||||
|
|
@ -1074,7 +1075,7 @@ export class AgentRuntime {
|
|||
hasTools: options?.toolsEnabled !== false && tools.hasTools,
|
||||
aiTools: prompt.aiTools,
|
||||
reasoning: staticContext.reasoning,
|
||||
providerOptions: staticContext.providerOptions,
|
||||
providerOptions: prompt.providerOptions,
|
||||
outputSpec: staticContext.outputSpec,
|
||||
maxOutputTokens: staticContext.maxOutputTokens,
|
||||
aiSdkOptions: this.buildAiSdkOptions(tools.toolMap, options),
|
||||
|
|
|
|||
|
|
@ -1,5 +1,9 @@
|
|||
import type { ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { getProviderPrefix } from '@n8n/ai-utilities/agent-config';
|
||||
import {
|
||||
getProviderPrefix,
|
||||
isGoogleVertexAnthropicModel,
|
||||
isOpenRouterAnthropicModel,
|
||||
} from '@n8n/ai-utilities/agent-config';
|
||||
import type { LanguageModel, Output } from 'ai';
|
||||
|
||||
import type { AgentRuntimeConfig } from '../../types/runtime/agent-runtime';
|
||||
|
|
@ -21,10 +25,10 @@ import {
|
|||
FLAG_MEMORY_TOOL_NAME,
|
||||
resolveEpisodicMemoryCapture,
|
||||
} from '../memory/episodic-memory-capture';
|
||||
import { isGoogleVertexAnthropicModel } from '../model/google-vertex';
|
||||
import { loadAi } from '../model/lazy-ai';
|
||||
import type { AgentMessageList } from '../model/message-list';
|
||||
import { createModel, supportsSplitSystemMessages } from '../model/model-factory';
|
||||
import { applyOpenRouterPromptCache } from '../model/openrouter-prompt-cache';
|
||||
import {
|
||||
applyRuntimeCacheBreakpoints,
|
||||
buildCallPromptCacheOptions,
|
||||
|
|
@ -154,12 +158,14 @@ export class RuntimeContextBuilder {
|
|||
list,
|
||||
tools,
|
||||
instructionProviderOptions,
|
||||
providerOptions,
|
||||
hostVolatileInstructions,
|
||||
activeSkills,
|
||||
}: {
|
||||
list: AgentMessageList;
|
||||
tools: ReturnType<RuntimeContextBuilder['buildToolLoopContext']>;
|
||||
instructionProviderOptions: ProviderOptions | undefined;
|
||||
providerOptions?: ProviderOptions;
|
||||
hostVolatileInstructions: string | undefined;
|
||||
activeSkills: ActiveSkills | undefined;
|
||||
}) {
|
||||
|
|
@ -176,12 +182,14 @@ export class RuntimeContextBuilder {
|
|||
skillContent
|
||||
? {
|
||||
content: skillContent,
|
||||
cacheOptions: (messages) =>
|
||||
buildSkillInstructionCacheOptions(
|
||||
cacheOptions: (messages) => {
|
||||
if (isOpenRouterAnthropicModel(this.modelId)) return undefined;
|
||||
return buildSkillInstructionCacheOptions(
|
||||
instructionProviderOptions,
|
||||
tools.aiTools,
|
||||
messages,
|
||||
),
|
||||
);
|
||||
},
|
||||
}
|
||||
: undefined,
|
||||
);
|
||||
|
|
@ -194,7 +202,14 @@ export class RuntimeContextBuilder {
|
|||
modelId: this.modelId,
|
||||
staticToolCacheName: tools.staticToolCacheName,
|
||||
});
|
||||
return { system, ...cached };
|
||||
const openRouterCache = applyOpenRouterPromptCache({
|
||||
system,
|
||||
messages: cached.messages,
|
||||
providerOptions,
|
||||
promptCaching: this.config.promptCaching,
|
||||
modelId: this.modelId,
|
||||
});
|
||||
return { ...cached, ...openRouterCache };
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -0,0 +1,182 @@
|
|||
import type { ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { generateText, type ModelMessage } from 'ai';
|
||||
|
||||
import { createModel } from '../model-factory';
|
||||
import { applyOpenRouterPromptCache } from '../openrouter-prompt-cache';
|
||||
|
||||
type CacheParams = Parameters<typeof applyOpenRouterPromptCache>[0];
|
||||
type WireRequest = {
|
||||
cache_control?: { type: string; ttl?: string };
|
||||
messages: Array<{ role: string; content: unknown; cache_control?: unknown }>;
|
||||
};
|
||||
const CACHE_1H = { type: 'ephemeral', ttl: '1h' };
|
||||
const CACHE_5M = { type: 'ephemeral', ttl: '5m' };
|
||||
|
||||
async function prepareRequest(overrides: Partial<CacheParams> = {}): Promise<WireRequest> {
|
||||
const params: CacheParams = {
|
||||
modelId: 'openrouter/anthropic/claude-sonnet-4.5',
|
||||
promptCaching: { enabled: true },
|
||||
system: [
|
||||
{ role: 'system', content: 'Stable instructions' },
|
||||
{ role: 'system', content: 'Recovered skills' },
|
||||
{ role: 'system', content: 'Volatile instructions' },
|
||||
],
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
providerOptions: undefined,
|
||||
...overrides,
|
||||
};
|
||||
const original = structuredClone(params);
|
||||
const prepared = applyOpenRouterPromptCache(params);
|
||||
const fetch = vi.fn<typeof globalThis.fetch>().mockResolvedValue(
|
||||
Response.json({
|
||||
id: 'completion-1',
|
||||
model: 'anthropic/claude-sonnet-4.5',
|
||||
choices: [
|
||||
{ index: 0, message: { role: 'assistant', content: 'Done.' }, finish_reason: 'stop' },
|
||||
],
|
||||
}),
|
||||
);
|
||||
await generateText({
|
||||
model: createModel({ id: params.modelId, apiKey: 'test-key' }, fetch),
|
||||
instructions: prepared.system,
|
||||
messages: params.messages,
|
||||
providerOptions: prepared.providerOptions,
|
||||
maxRetries: 0,
|
||||
});
|
||||
expect(params).toEqual(original);
|
||||
return JSON.parse(fetch.mock.calls[0][1]?.body as string) as WireRequest;
|
||||
}
|
||||
|
||||
function toolHistory(count: number, providerOptions: ProviderOptions): ModelMessage[] {
|
||||
const ids = Array.from({ length: count }, (_, index) => `call-${index}`);
|
||||
return [
|
||||
{ role: 'user', content: 'Look up records' },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: ids.map((id) => ({
|
||||
type: 'tool-call',
|
||||
toolCallId: id,
|
||||
toolName: 'lookup',
|
||||
input: {},
|
||||
})),
|
||||
},
|
||||
{
|
||||
role: 'tool',
|
||||
providerOptions,
|
||||
content: ids.map((id) => ({
|
||||
type: 'tool-result',
|
||||
toolCallId: id,
|
||||
toolName: 'lookup',
|
||||
output: { type: 'text', value: 'Found' },
|
||||
})),
|
||||
},
|
||||
{ role: 'user', content: 'Continue' },
|
||||
];
|
||||
}
|
||||
|
||||
describe('OpenRouter prompt caching', () => {
|
||||
it.each([
|
||||
{ promptCaching: undefined },
|
||||
{ promptCaching: { enabled: false } },
|
||||
{ promptCaching: { anthropic: false } },
|
||||
{ modelId: 'openrouter/google/gemini-2.5-pro' },
|
||||
] satisfies Array<Partial<CacheParams>>)('leaves caching unchanged for %j', async (overrides) => {
|
||||
const request = await prepareRequest(overrides);
|
||||
expect(JSON.stringify(request)).not.toContain('cache_control');
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ count: 2, namespace: 'anthropic', key: 'cacheControl', automatic: true },
|
||||
{ count: 3, namespace: 'openrouter', key: 'cache_control', automatic: false },
|
||||
{ count: 4, namespace: 'anthropic', key: 'cache_control', automatic: false },
|
||||
{ count: 4, namespace: 'openrouter', key: 'cacheControl', automatic: false },
|
||||
])(
|
||||
'keeps four wire markers with $count marked tool results via $namespace.$key',
|
||||
async ({ count, namespace, key, automatic }) => {
|
||||
const request = await prepareRequest({
|
||||
messages: toolHistory(count, { [namespace]: { [key]: CACHE_1H } }),
|
||||
});
|
||||
expect(request.cache_control).toEqual(automatic ? CACHE_1H : undefined);
|
||||
expect(request.messages[0].content).toEqual([
|
||||
{
|
||||
type: 'text',
|
||||
text: 'Stable instructions',
|
||||
...(count < 4 ? { cache_control: CACHE_1H } : {}),
|
||||
},
|
||||
]);
|
||||
expect(JSON.stringify(request).match(/"cache_control":/g)).toHaveLength(4);
|
||||
},
|
||||
);
|
||||
|
||||
it.each(['cacheControl', 'cache_control'])(
|
||||
'reserves a slot for explicit automatic %s',
|
||||
async (key) => {
|
||||
const request = await prepareRequest({
|
||||
messages: toolHistory(3, { openrouter: { cacheControl: CACHE_1H } }),
|
||||
providerOptions: { openrouter: { [key]: CACHE_5M } },
|
||||
});
|
||||
expect(request.cache_control).toEqual(CACHE_5M);
|
||||
expect(request.messages[0].content).toEqual([{ type: 'text', text: 'Stable instructions' }]);
|
||||
expect(JSON.stringify(request).match(/"cache_control":/g)).toHaveLength(4);
|
||||
},
|
||||
);
|
||||
|
||||
it('uses the raw automatic control when both caller keys are present', async () => {
|
||||
const request = await prepareRequest({
|
||||
providerOptions: { openrouter: { cacheControl: CACHE_1H, cache_control: CACHE_5M } },
|
||||
});
|
||||
expect(request.cache_control).toEqual(CACHE_5M);
|
||||
expect(request.messages[0].content).toEqual([
|
||||
{ type: 'text', text: 'Stable instructions', cache_control: CACHE_5M },
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ ttl: '1h', caller: CACHE_5M, trailing: true, instruction: CACHE_1H, automatic: undefined },
|
||||
{ ttl: '5m', caller: CACHE_1H, trailing: true, instruction: undefined, automatic: CACHE_5M },
|
||||
{ ttl: '5m', caller: CACHE_1H, trailing: false, instruction: undefined, automatic: undefined },
|
||||
{ ttl: '1h', caller: CACHE_1H, trailing: false, instruction: CACHE_1H, automatic: undefined },
|
||||
] as const)(
|
||||
'respects TTL order and final markers: %j',
|
||||
async ({ ttl, caller, trailing, instruction, automatic }) => {
|
||||
const messages: ModelMessage[] = [
|
||||
{
|
||||
role: 'user',
|
||||
content: 'Caller prompt',
|
||||
providerOptions: { anthropic: { cacheControl: caller } },
|
||||
},
|
||||
];
|
||||
if (trailing) messages.push({ role: 'user', content: 'Continue' });
|
||||
const request = await prepareRequest({ messages, promptCaching: { anthropic: { ttl } } });
|
||||
expect(request.cache_control).toEqual(automatic);
|
||||
expect(request.messages[0].content).toEqual([
|
||||
{
|
||||
type: 'text',
|
||||
text: 'Stable instructions',
|
||||
...(instruction ? { cache_control: instruction } : {}),
|
||||
},
|
||||
]);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([false, true])(
|
||||
'matches adapter precedence for multipart=%s user content',
|
||||
async (multipart) => {
|
||||
const content: Extract<ModelMessage, { role: 'user' }>['content'] = [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'First part',
|
||||
providerOptions: { openrouter: { cache_control: CACHE_5M } },
|
||||
},
|
||||
];
|
||||
if (multipart) content.push({ type: 'text', text: 'Second part' });
|
||||
const request = await prepareRequest({
|
||||
messages: [
|
||||
{ role: 'user', content, providerOptions: { anthropic: { cacheControl: CACHE_1H } } },
|
||||
{ role: 'user', content: 'Continue' },
|
||||
],
|
||||
});
|
||||
expect(request.cache_control).toEqual(multipart ? undefined : CACHE_1H);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
|
@ -0,0 +1,247 @@
|
|||
import { generateText, streamText } from 'ai';
|
||||
|
||||
import { Tool } from '../../../sdk/tool';
|
||||
import type { StreamChunk, TokenUsage } from '../../../types';
|
||||
import { AgentRuntime } from '../../loop/agent-runtime';
|
||||
import { createModel } from '../model-factory';
|
||||
|
||||
const MODEL_ID = 'openrouter/anthropic/claude-sonnet-4.5';
|
||||
const USAGE = {
|
||||
prompt_tokens: 1000,
|
||||
completion_tokens: 10,
|
||||
total_tokens: 1010,
|
||||
prompt_tokens_details: { cached_tokens: 200, cache_write_tokens: 300 },
|
||||
};
|
||||
const REASONING_DETAILS = [
|
||||
{
|
||||
type: 'reasoning.text',
|
||||
text: 'I will look up the records.',
|
||||
signature: 'reasoning-signature',
|
||||
format: 'anthropic-claude-v1',
|
||||
index: 0,
|
||||
},
|
||||
{
|
||||
type: 'reasoning.encrypted',
|
||||
data: 'opaque-reasoning',
|
||||
id: 'reasoning-1',
|
||||
format: 'anthropic-claude-v1',
|
||||
index: 1,
|
||||
},
|
||||
];
|
||||
|
||||
function response(stream: boolean, withTools = false, usage: object = USAGE): Response {
|
||||
const base = { id: 'completion-1', model: 'anthropic/claude-sonnet-4.5', created: 1 };
|
||||
const message = withTools
|
||||
? {
|
||||
role: 'assistant',
|
||||
content: null,
|
||||
reasoning_details: REASONING_DETAILS,
|
||||
tool_calls: ['a', 'b', 'c', 'd'].map((id, index) => ({
|
||||
id,
|
||||
index,
|
||||
type: 'function',
|
||||
function: { name: 'lookup', arguments: '{}' },
|
||||
})),
|
||||
}
|
||||
: { role: 'assistant', content: 'Done.' };
|
||||
const finishReason = withTools ? 'tool_calls' : 'stop';
|
||||
if (!stream) {
|
||||
return Response.json({
|
||||
...base,
|
||||
provider: 'Anthropic',
|
||||
choices: [{ index: 0, message, finish_reason: finishReason }],
|
||||
usage,
|
||||
});
|
||||
}
|
||||
const chunks = [
|
||||
{ ...base, choices: [{ index: 0, delta: message, finish_reason: null }] },
|
||||
{ ...base, choices: [{ index: 0, delta: {}, finish_reason: finishReason }], usage },
|
||||
];
|
||||
return new Response(
|
||||
chunks.map((chunk) => `data: ${JSON.stringify(chunk)}\n\n`).join('') + 'data: [DONE]\n\n',
|
||||
{ headers: { 'content-type': 'text/event-stream' } },
|
||||
);
|
||||
}
|
||||
|
||||
describe.each(['generate', 'stream'] as const)('OpenRouter %s', (mode) => {
|
||||
it.each([
|
||||
{ ttl: '1h', cost: 0.00244 },
|
||||
{ ttl: '5m', cost: 0.00199 },
|
||||
] as const)(
|
||||
'caches the tool loop and prices each token once with $ttl',
|
||||
async ({ ttl, cost }) => {
|
||||
const fetch = vi.fn<typeof globalThis.fetch>();
|
||||
fetch.mockImplementation(async () =>
|
||||
response(mode === 'stream', fetch.mock.calls.length === 1),
|
||||
);
|
||||
const runtime = new AgentRuntime({
|
||||
name: 'test',
|
||||
model: { id: MODEL_ID, apiKey: 'test-key' },
|
||||
modelFetch: fetch,
|
||||
modelCost: { input: 1, output: 10, cacheRead: 0.1, cacheWrite: 1.25 },
|
||||
instructions: 'Look up the requested records.',
|
||||
promptCaching: { enabled: true, anthropic: { ttl } },
|
||||
reasoning: 'high',
|
||||
tools: [
|
||||
new Tool('lookup')
|
||||
.description('Look up a record')
|
||||
.input({ type: 'object', properties: {} })
|
||||
.handler(async () => ({ found: true }))
|
||||
.build(),
|
||||
],
|
||||
});
|
||||
let usage: TokenUsage | undefined;
|
||||
if (mode === 'generate') {
|
||||
const result = await runtime.generate('Look up four records.');
|
||||
expect(result.finishReason).toBe('stop');
|
||||
expect(result.messages).toContainEqual(
|
||||
expect.objectContaining({
|
||||
role: 'assistant',
|
||||
content: expect.arrayContaining([
|
||||
expect.objectContaining({ type: 'text', text: 'Done.' }),
|
||||
]),
|
||||
}),
|
||||
);
|
||||
usage = result.usage;
|
||||
} else {
|
||||
const result = await runtime.stream('Look up four records.');
|
||||
const chunks: StreamChunk[] = [];
|
||||
for await (const chunk of result.stream) chunks.push(chunk);
|
||||
expect(chunks.filter((chunk) => chunk.type === 'error')).toEqual([]);
|
||||
usage = chunks.filter((chunk) => chunk.type === 'finish').at(-1)?.usage;
|
||||
expect(chunks.some((chunk) => chunk.type === 'text-delta')).toBe(true);
|
||||
}
|
||||
expect(usage).toMatchObject({
|
||||
promptTokens: 2000,
|
||||
completionTokens: 20,
|
||||
inputTokenDetails: { noCache: 1000, cacheRead: 400, cacheWrite: 600 },
|
||||
});
|
||||
expect(usage?.cost).toBeCloseTo(cost, 8);
|
||||
expect(fetch).toHaveBeenCalledTimes(2);
|
||||
const requests = fetch.mock.calls.map(([, init]) => JSON.parse(init?.body as string));
|
||||
const cacheControl = { type: 'ephemeral', ttl };
|
||||
for (const request of requests) {
|
||||
expect(request).toMatchObject({
|
||||
cache_control: cacheControl,
|
||||
reasoning: { effort: 'high' },
|
||||
messages: expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
role: 'system',
|
||||
content: expect.arrayContaining([
|
||||
expect.objectContaining({ cache_control: cacheControl }),
|
||||
]),
|
||||
}),
|
||||
]),
|
||||
});
|
||||
expect(JSON.stringify(request).match(/"cache_control":/g)).toHaveLength(2);
|
||||
}
|
||||
expect(requests[1].messages).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({ role: 'assistant', reasoning_details: REASONING_DETAILS }),
|
||||
...['a', 'b', 'c', 'd'].map((id) =>
|
||||
expect.objectContaining({ role: 'tool', tool_call_id: id }),
|
||||
),
|
||||
]),
|
||||
);
|
||||
expect(fetch.mock.calls[0][0]).toBe('https://openrouter.ai/api/v1/chat/completions');
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
{ details: { cached_tokens: 200 }, noCache: 800, cacheWrite: undefined },
|
||||
{ details: { cached_tokens: 900, cache_write_tokens: 200 }, noCache: 0, cacheWrite: 200 },
|
||||
])(
|
||||
'handles incomplete or overlapping usage: $details',
|
||||
async ({ details, noCache, cacheWrite }) => {
|
||||
const fetch = vi
|
||||
.fn<typeof globalThis.fetch>()
|
||||
.mockResolvedValue(
|
||||
response(mode === 'stream', false, { ...USAGE, prompt_tokens_details: details }),
|
||||
);
|
||||
const model = createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch);
|
||||
const result =
|
||||
mode === 'generate'
|
||||
? await generateText({ model, prompt: 'Hello' })
|
||||
: streamText({ model, prompt: 'Hello' });
|
||||
expect((await result.usage).inputTokenDetails).toEqual({
|
||||
noCacheTokens: noCache,
|
||||
cacheReadTokens: details.cached_tokens,
|
||||
cacheWriteTokens: cacheWrite,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it('preserves raw usage and provider errors', async () => {
|
||||
const fetch = vi
|
||||
.fn<typeof globalThis.fetch>()
|
||||
.mockResolvedValueOnce(response(mode === 'stream'));
|
||||
const model = createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch);
|
||||
if (mode === 'generate') {
|
||||
const result = await generateText({ model, prompt: 'Hello', maxRetries: 0 });
|
||||
expect(result.steps[0].usage.raw).toEqual(USAGE);
|
||||
expect(result.providerMetadata?.openrouter).toMatchObject({ provider: 'Anthropic' });
|
||||
} else {
|
||||
const result = streamText({ model, prompt: 'Hello', maxRetries: 0 });
|
||||
await result.consumeStream();
|
||||
expect((await result.steps)[0].usage.raw).toEqual(USAGE);
|
||||
}
|
||||
fetch.mockResolvedValueOnce(
|
||||
Response.json({ error: { message: 'Provider unavailable', code: 503 } }, { status: 503 }),
|
||||
);
|
||||
if (mode === 'generate') {
|
||||
await expect(generateText({ model, prompt: 'Hello', maxRetries: 0 })).rejects.toThrow(
|
||||
'Provider unavailable',
|
||||
);
|
||||
} else {
|
||||
const result = streamText({ model, prompt: 'Hello', maxRetries: 0 });
|
||||
const chunks = [];
|
||||
for await (const chunk of result.fullStream) chunks.push(chunk);
|
||||
expect(chunks).toContainEqual({
|
||||
type: 'error',
|
||||
error: expect.objectContaining({ message: 'Provider unavailable' }),
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('OpenRouter reasoning options', () => {
|
||||
it.each([
|
||||
'none',
|
||||
'minimal',
|
||||
'low',
|
||||
'medium',
|
||||
'high',
|
||||
'xhigh',
|
||||
'provider-default',
|
||||
undefined,
|
||||
] as const)('forwards %s through the adapter', async (reasoning) => {
|
||||
const fetch = vi.fn<typeof globalThis.fetch>().mockResolvedValue(response(false));
|
||||
await generateText({
|
||||
model: createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch),
|
||||
prompt: 'Hello',
|
||||
reasoning,
|
||||
});
|
||||
const body = JSON.parse(fetch.mock.calls[0][1]?.body as string);
|
||||
const expected =
|
||||
reasoning && reasoning !== 'provider-default' ? { effort: reasoning } : undefined;
|
||||
expect(body.reasoning).toEqual(expected);
|
||||
expect(body.cache_control).toBeUndefined();
|
||||
});
|
||||
|
||||
it('preserves an explicit reasoning budget and other caller options', async () => {
|
||||
const fetch = vi.fn<typeof globalThis.fetch>().mockResolvedValue(response(false));
|
||||
const providerOptions = {
|
||||
openrouter: { reasoning: { max_tokens: 4096, exclude: false }, user: 'test-user' },
|
||||
};
|
||||
await generateText({
|
||||
model: createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch),
|
||||
prompt: 'Hello',
|
||||
reasoning: 'high',
|
||||
providerOptions,
|
||||
});
|
||||
expect(JSON.parse(fetch.mock.calls[0][1]?.body as string)).toMatchObject(
|
||||
providerOptions.openrouter,
|
||||
);
|
||||
expect(providerOptions.openrouter.reasoning).not.toHaveProperty('effort');
|
||||
});
|
||||
});
|
||||
|
|
@ -217,12 +217,13 @@ describe('createOpenAIPromptCacheKey (via buildCallPromptCacheOptions)', () => {
|
|||
});
|
||||
|
||||
describe('getEffectiveAnthropicCacheTtl', () => {
|
||||
it.each(['anthropic/claude-sonnet-4-5', 'google-vertex/claude-opus-5@default'])(
|
||||
'returns the default 1h TTL for %s with caching enabled',
|
||||
(modelId) => {
|
||||
expect(getEffectiveAnthropicCacheTtl({ enabled: true }, modelId)).toBe('1h');
|
||||
},
|
||||
);
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'google-vertex/claude-opus-5@default',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
])('returns the default 1h TTL for %s with caching enabled', (modelId) => {
|
||||
expect(getEffectiveAnthropicCacheTtl({ enabled: true }, modelId)).toBe('1h');
|
||||
});
|
||||
|
||||
it('returns an explicit 5m override for an Anthropic model', () => {
|
||||
expect(
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
/* eslint-disable @typescript-eslint/consistent-type-imports */
|
||||
/* eslint-disable @typescript-eslint/no-require-imports */
|
||||
import type { GoogleVertexProviderSettings } from '@ai-sdk/google-vertex';
|
||||
import { splitModelId } from '@n8n/ai-utilities/agent-config';
|
||||
import { formatPemBlock } from '@n8n/utils/format-pem-block';
|
||||
import { isRecord } from '@n8n/utils/is-record';
|
||||
import type { LanguageModel } from 'ai';
|
||||
|
|
@ -38,15 +37,6 @@ export function resolveGoogleVertexSdk(model: string, sdk?: string): string | un
|
|||
return undefined;
|
||||
}
|
||||
|
||||
/** Identify Claude models that use the shared Vertex provider ID. */
|
||||
export function isGoogleVertexAnthropicModel(modelId: string): boolean {
|
||||
const { provider, model } = splitModelId(modelId);
|
||||
return (
|
||||
provider === 'google-vertex' &&
|
||||
resolveGoogleVertexSdk(model) === '@ai-sdk/google-vertex/anthropic'
|
||||
);
|
||||
}
|
||||
|
||||
/** Select the Vertex API for the model. */
|
||||
export function createGoogleVertexModel(
|
||||
creds: ProviderCredentials<'google-vertex'>,
|
||||
|
|
|
|||
|
|
@ -1,14 +1,12 @@
|
|||
/* eslint-disable @typescript-eslint/consistent-type-imports */
|
||||
/* eslint-disable @typescript-eslint/no-require-imports */
|
||||
import { isGoogleVertexAnthropicModel } from '@n8n/ai-utilities/agent-config';
|
||||
import { ensureUrlPathSuffix, isOpenAiCustomEndpoint } from '@n8n/ai-utilities/model-discovery';
|
||||
import type { EmbeddingModel, LanguageModel } from 'ai';
|
||||
import type * as Undici from 'undici';
|
||||
|
||||
import {
|
||||
createGoogleVertexModel,
|
||||
isGoogleVertexAnthropicModel,
|
||||
parseGoogleVertexAuthOptions,
|
||||
} from './google-vertex';
|
||||
import { createGoogleVertexModel, parseGoogleVertexAuthOptions } from './google-vertex';
|
||||
import { wrapOpenRouterModel } from './openrouter';
|
||||
import {
|
||||
endpointRouteKey,
|
||||
guardOpenAiRoutes,
|
||||
|
|
@ -366,7 +364,9 @@ const LANGUAGE_PROVIDERS: ProviderRegistry = {
|
|||
build: (creds, model, fetch) => {
|
||||
const { createOpenRouter } =
|
||||
require('@openrouter/ai-sdk-provider') as typeof import('@openrouter/ai-sdk-provider');
|
||||
return createOpenRouter({ apiKey: creds.apiKey, baseURL: creds.baseURL, fetch })(model);
|
||||
return wrapOpenRouterModel(
|
||||
createOpenRouter({ apiKey: creds.apiKey, baseURL: creds.baseURL, fetch })(model),
|
||||
);
|
||||
},
|
||||
},
|
||||
nvidia: openAiCompatibleEntry('nvidia', 'https://integrate.api.nvidia.com/v1', {}),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,114 @@
|
|||
import type { ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { isOpenRouterAnthropicModel } from '@n8n/ai-utilities/agent-config';
|
||||
import { isRecord } from '@n8n/utils/is-record';
|
||||
import type { ModelMessage, SystemModelMessage } from 'ai';
|
||||
|
||||
import { getEffectiveAnthropicCacheTtl, mergeProviderOptions } from './prompt-cache';
|
||||
import type { PromptCachingConfig } from '../../types/sdk/agent';
|
||||
|
||||
type CacheControl = Record<string, unknown>;
|
||||
type CacheTtl = '5m' | '1h';
|
||||
const MAX_CACHE_BREAKPOINTS = 4;
|
||||
|
||||
/** Match the namespaces and key precedence in the OpenRouter chat adapter. */
|
||||
function getCacheControl(options: ProviderOptions | undefined): CacheControl | undefined {
|
||||
const control =
|
||||
options?.openrouter?.cacheControl ??
|
||||
options?.openrouter?.cache_control ??
|
||||
options?.anthropic?.cacheControl ??
|
||||
options?.anthropic?.cache_control;
|
||||
return isRecord(control) ? control : undefined;
|
||||
}
|
||||
|
||||
function getCacheTtl(control: CacheControl): CacheTtl {
|
||||
return control.ttl === '1h' ? '1h' : '5m';
|
||||
}
|
||||
|
||||
/** Count wire breakpoints, including the marker copied to each tool result. */
|
||||
function getMessageCacheControls(message: ModelMessage): Array<CacheControl | undefined> {
|
||||
const control = getCacheControl(message.providerOptions);
|
||||
if (message.role === 'system' || message.role === 'assistant') return [control];
|
||||
if (message.role === 'tool') {
|
||||
return message.content
|
||||
.filter((part) => part.type === 'tool-result')
|
||||
.map((part) => control ?? getCacheControl(part.providerOptions));
|
||||
}
|
||||
if (typeof message.content === 'string') return [control];
|
||||
if (message.content.length === 1 && message.content[0].type === 'text') {
|
||||
return [control ?? getCacheControl(message.content[0].providerOptions)];
|
||||
}
|
||||
const lastTextIndex = message.content.map((part) => part.type).lastIndexOf('text');
|
||||
return message.content.map((part, index) => {
|
||||
const partControl = getCacheControl(part.providerOptions);
|
||||
return partControl ?? (index === lastTextIndex ? control : undefined);
|
||||
});
|
||||
}
|
||||
|
||||
function cacheOptions(ttl: CacheTtl): ProviderOptions {
|
||||
return { openrouter: { cacheControl: { type: 'ephemeral', ttl } } };
|
||||
}
|
||||
|
||||
/** Add request-local caching after skills and conversation history are assembled. */
|
||||
export function applyOpenRouterPromptCache(params: {
|
||||
system: SystemModelMessage | SystemModelMessage[];
|
||||
messages: ModelMessage[];
|
||||
providerOptions: ProviderOptions | undefined;
|
||||
promptCaching: PromptCachingConfig | undefined;
|
||||
modelId: string;
|
||||
}): {
|
||||
system: SystemModelMessage | SystemModelMessage[];
|
||||
providerOptions: ProviderOptions | undefined;
|
||||
} {
|
||||
const { system, messages, providerOptions, promptCaching, modelId } = params;
|
||||
const ttl = getEffectiveAnthropicCacheTtl(promptCaching, modelId);
|
||||
if (!isOpenRouterAnthropicModel(modelId) || !ttl) return { system, providerOptions };
|
||||
|
||||
const systemMessages = Array.isArray(system) ? system : [system];
|
||||
const controls = [...systemMessages, ...messages].flatMap(getMessageCacheControls);
|
||||
const callerControls = controls.filter((control) => control !== undefined);
|
||||
const openrouter = providerOptions?.openrouter;
|
||||
const hasCallerAutomaticControl =
|
||||
openrouter !== undefined && ('cache_control' in openrouter || 'cacheControl' in openrouter);
|
||||
const automaticControl =
|
||||
openrouter && 'cache_control' in openrouter
|
||||
? openrouter.cache_control
|
||||
: openrouter?.cacheControl;
|
||||
let used = callerControls.length + (isRecord(automaticControl) ? 1 : 0);
|
||||
|
||||
// Caller controls take priority, followed by stable instructions and automatic caching.
|
||||
const instructionTtl = isRecord(automaticControl) ? getCacheTtl(automaticControl) : ttl;
|
||||
const firstSystem = systemMessages[0];
|
||||
const canCacheInstructions =
|
||||
firstSystem?.content.trim() &&
|
||||
!getCacheControl(firstSystem.providerOptions) &&
|
||||
used < MAX_CACHE_BREAKPOINTS &&
|
||||
(instructionTtl === '1h' || callerControls.every((control) => getCacheTtl(control) === '5m'));
|
||||
let nextSystem = system;
|
||||
if (canCacheInstructions) {
|
||||
const cachedInstruction = {
|
||||
...firstSystem,
|
||||
providerOptions: mergeProviderOptions(
|
||||
firstSystem.providerOptions,
|
||||
cacheOptions(instructionTtl),
|
||||
),
|
||||
};
|
||||
nextSystem = Array.isArray(system)
|
||||
? [cachedInstruction, ...system.slice(1)]
|
||||
: cachedInstruction;
|
||||
used++;
|
||||
}
|
||||
|
||||
// A longer TTL must precede shorter TTLs. A marked final block can also conflict with auto caching.
|
||||
if (
|
||||
hasCallerAutomaticControl ||
|
||||
used >= MAX_CACHE_BREAKPOINTS ||
|
||||
controls.at(-1) ||
|
||||
(ttl === '1h' && callerControls.some((control) => getCacheTtl(control) === '5m'))
|
||||
) {
|
||||
return { system: nextSystem, providerOptions };
|
||||
}
|
||||
return {
|
||||
system: nextSystem,
|
||||
providerOptions: mergeProviderOptions(cacheOptions(ttl), providerOptions),
|
||||
};
|
||||
}
|
||||
66
packages/@n8n/agents/src/runtime/model/openrouter.ts
Normal file
66
packages/@n8n/agents/src/runtime/model/openrouter.ts
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
import type * as AiSdk from 'ai';
|
||||
|
||||
import { loadAi } from './lazy-ai';
|
||||
import { mergeProviderOptions } from './prompt-cache';
|
||||
|
||||
type MiddlewareGenerate = NonNullable<AiSdk.LanguageModelMiddleware['wrapGenerate']>;
|
||||
type CallOptions = Parameters<MiddlewareGenerate>[0]['params'];
|
||||
type ModelUsage = Awaited<ReturnType<MiddlewareGenerate>>['usage'];
|
||||
type StreamResult = Awaited<ReturnType<NonNullable<AiSdk.LanguageModelMiddleware['wrapStream']>>>;
|
||||
type StreamPart = StreamResult['stream'] extends ReadableStream<infer Part> ? Part : never;
|
||||
|
||||
function applyReasoning(params: CallOptions): CallOptions {
|
||||
if (!params.reasoning || params.reasoning === 'provider-default') return params;
|
||||
// Explicit options can use a token budget, which must not be combined with effort.
|
||||
if (params.providerOptions?.openrouter?.reasoning !== undefined) return params;
|
||||
return {
|
||||
...params,
|
||||
providerOptions: mergeProviderOptions(params.providerOptions, {
|
||||
openrouter: { reasoning: { effort: params.reasoning } },
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeUsage(usage: ModelUsage): ModelUsage {
|
||||
const { total, cacheRead, cacheWrite } = usage.inputTokens;
|
||||
if (total === undefined || cacheWrite === undefined) return usage;
|
||||
// The adapter includes cache writes in noCache. Price each token only once.
|
||||
return {
|
||||
...usage,
|
||||
inputTokens: {
|
||||
...usage.inputTokens,
|
||||
noCache: Math.max(total - (cacheRead ?? 0) - cacheWrite, 0),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Adapt generic reasoning and cache usage for the OpenRouter chat adapter. */
|
||||
export function wrapOpenRouterModel(model: Parameters<typeof AiSdk.wrapLanguageModel>[0]['model']) {
|
||||
return loadAi().wrapLanguageModel({
|
||||
model,
|
||||
middleware: {
|
||||
transformParams: async ({ params }) => applyReasoning(params),
|
||||
wrapGenerate: async ({ doGenerate }) => {
|
||||
const result = await doGenerate();
|
||||
return { ...result, usage: normalizeUsage(result.usage) };
|
||||
},
|
||||
wrapStream: async ({ doStream }) => {
|
||||
const result = await doStream();
|
||||
return {
|
||||
...result,
|
||||
stream: result.stream.pipeThrough(
|
||||
new TransformStream<StreamPart, StreamPart>({
|
||||
transform(chunk, controller) {
|
||||
if (chunk.type === 'finish') {
|
||||
controller.enqueue({ ...chunk, usage: normalizeUsage(chunk.usage) });
|
||||
} else {
|
||||
controller.enqueue(chunk);
|
||||
}
|
||||
},
|
||||
}),
|
||||
),
|
||||
};
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
|
|
@ -1,10 +1,13 @@
|
|||
import type { ProviderOptions } from '@ai-sdk/provider-utils';
|
||||
import { getProviderPrefix } from '@n8n/ai-utilities/agent-config';
|
||||
import {
|
||||
getModelPromptCachingCapability,
|
||||
getProviderPrefix,
|
||||
isGoogleVertexAnthropicModel,
|
||||
} from '@n8n/ai-utilities/agent-config';
|
||||
import { isRecord } from '@n8n/utils/is-record';
|
||||
import type { ModelMessage, SystemModelMessage, ToolSet } from 'ai';
|
||||
import { createHash } from 'crypto';
|
||||
|
||||
import { isGoogleVertexAnthropicModel } from './google-vertex';
|
||||
import type { PromptCachingConfig } from '../../types/sdk/agent';
|
||||
import type { JSONObject } from '../../types/utils/json';
|
||||
|
||||
|
|
@ -42,7 +45,10 @@ export function getEffectiveAnthropicCacheTtl(
|
|||
config: PromptCachingConfig | undefined,
|
||||
modelId: string,
|
||||
): '5m' | '1h' | undefined {
|
||||
if (!isAnthropicMessagesProvider(modelId) || !isEnabledForProvider(config, 'anthropic')) {
|
||||
if (
|
||||
getModelPromptCachingCapability(modelId) !== 'ttl' ||
|
||||
!isEnabledForProvider(config, 'anthropic')
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
return getAnthropicCacheTtl(config);
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
import { getProviderPrefix } from '@n8n/ai-utilities/agent-config';
|
||||
import { getProviderPrefix, isGoogleVertexAnthropicModel } from '@n8n/ai-utilities/agent-config';
|
||||
import { isRecord } from '@n8n/utils/is-record';
|
||||
|
||||
import { isGoogleVertexAnthropicModel } from './google-vertex';
|
||||
import { type ProviderId } from './provider-credentials';
|
||||
import type { TokenUsage } from '../../types';
|
||||
|
||||
|
|
|
|||
|
|
@ -521,8 +521,8 @@ export class Agent implements BuiltAgent, AgentBuilder {
|
|||
|
||||
/**
|
||||
* Enable prompt caching with defaults tuned for agent workloads. Anthropic
|
||||
* models get a `1h` instruction-level cache breakpoint; OpenAI models get
|
||||
* `24h` retention plus an auto-generated, per-agent-version `promptCacheKey`.
|
||||
* models, including Claude on OpenRouter and Vertex, get a `1h` instruction breakpoint.
|
||||
* OpenAI gets `24h` retention plus an auto-generated, per-agent-version `promptCacheKey`.
|
||||
* Pass `{ anthropic: false }` / `{ openai: false }` to disable per provider,
|
||||
* or override individual fields (e.g. `{ anthropic: { ttl: '5m' } }`).
|
||||
*
|
||||
|
|
|
|||
|
|
@ -325,7 +325,7 @@ export interface OpenAIPromptCachingConfig {
|
|||
export interface PromptCachingConfig {
|
||||
/** Master switch. Defaults to `true` when this config is set. */
|
||||
enabled?: boolean;
|
||||
/** Anthropic-specific config, or `false` to disable for Anthropic models. */
|
||||
/** Claude cache settings, including OpenRouter and Vertex. Set `false` to disable. */
|
||||
anthropic?: false | AnthropicPromptCachingConfig;
|
||||
/** OpenAI-specific config, or `false` to disable for OpenAI models. */
|
||||
openai?: false | OpenAIPromptCachingConfig;
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ model configuration, and model output.
|
|||
## Public subpaths
|
||||
|
||||
- `agent-config`
|
||||
- `agent-config/model-capabilities` (browser-safe model capability checks)
|
||||
- `fromai-helpers`
|
||||
- `generic-text-editor`
|
||||
- `http-proxy-agent`
|
||||
|
|
|
|||
|
|
@ -22,6 +22,9 @@
|
|||
"agent-config": [
|
||||
"dist/esm/agent-config/index.d.ts"
|
||||
],
|
||||
"agent-config/model-capabilities": [
|
||||
"dist/esm/agent-config/model-capabilities.d.ts"
|
||||
],
|
||||
"fromai-helpers": [
|
||||
"dist/esm/utils/fromai-helpers.d.ts"
|
||||
],
|
||||
|
|
@ -76,6 +79,11 @@
|
|||
"import": "./dist/esm/agent-config/index.js",
|
||||
"require": "./dist/cjs/agent-config/index.js"
|
||||
},
|
||||
"./agent-config/model-capabilities": {
|
||||
"types": "./dist/esm/agent-config/model-capabilities.d.ts",
|
||||
"import": "./dist/esm/agent-config/model-capabilities.js",
|
||||
"require": "./dist/cjs/agent-config/model-capabilities.js"
|
||||
},
|
||||
"./fromai-helpers": {
|
||||
"types": "./dist/esm/utils/fromai-helpers.d.ts",
|
||||
"import": "./dist/esm/utils/fromai-helpers.js",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,22 @@
|
|||
import { getModelPromptCachingCapability } from '../model-capabilities';
|
||||
|
||||
describe('getModelPromptCachingCapability', () => {
|
||||
it.each<[string, 'ttl' | boolean]>([
|
||||
['anthropic/claude-sonnet-4-5', 'ttl'],
|
||||
['google-vertex-anthropic/claude-sonnet-4-5', 'ttl'],
|
||||
['google-vertex/claude-opus-5@default', 'ttl'],
|
||||
['openrouter/anthropic/claude-sonnet-4.5', 'ttl'],
|
||||
['openrouter/anthropic/claude-sonnet-4.5:beta', 'ttl'],
|
||||
['openai/gpt-5', true],
|
||||
['google-vertex/gemini-2.5-pro', false],
|
||||
['google-vertex/meta/llama-4-maas', false],
|
||||
['openrouter/google/gemini-2.5-pro', false],
|
||||
['openrouter/openai/gpt-5', false],
|
||||
['openrouter/claude-sonnet-4.5', false],
|
||||
['custom/anthropic/claude-sonnet-4.5', false],
|
||||
['claude-sonnet-4-5', false],
|
||||
['', false],
|
||||
])('resolves %s to %s', (modelId, expected) => {
|
||||
expect(getModelPromptCachingCapability(modelId)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
|
@ -1,4 +1,10 @@
|
|||
export { getProviderPrefix, splitModelId } from './model-id';
|
||||
export {
|
||||
getModelPromptCachingCapability,
|
||||
isGoogleVertexAnthropicModel,
|
||||
isOpenRouterAnthropicModel,
|
||||
isRoutedAnthropicModel,
|
||||
} from './model-capabilities';
|
||||
export {
|
||||
getNativeWebSearchProviderTools,
|
||||
hasNativeWebSearchProvider,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,25 @@
|
|||
import { PROVIDER_CAPABILITIES, type ProviderCapabilities } from '@n8n/api-types';
|
||||
|
||||
import { getProviderPrefix, splitModelId } from './model-id';
|
||||
|
||||
export function isGoogleVertexAnthropicModel(modelId: string): boolean {
|
||||
const { provider, model } = splitModelId(modelId);
|
||||
return provider === 'google-vertex' && model.startsWith('claude-');
|
||||
}
|
||||
|
||||
export function isOpenRouterAnthropicModel(modelId: string): boolean {
|
||||
const { provider, model } = splitModelId(modelId);
|
||||
return provider === 'openrouter' && model.startsWith('anthropic/claude-');
|
||||
}
|
||||
|
||||
export function isRoutedAnthropicModel(modelId: string): boolean {
|
||||
return isGoogleVertexAnthropicModel(modelId) || isOpenRouterAnthropicModel(modelId);
|
||||
}
|
||||
|
||||
/** Cache support depends on the model when the provider routes to several vendors. */
|
||||
export function getModelPromptCachingCapability(
|
||||
modelId: string,
|
||||
): ProviderCapabilities['promptCaching'] {
|
||||
if (isRoutedAnthropicModel(modelId)) return 'ttl';
|
||||
return PROVIDER_CAPABILITIES[getProviderPrefix(modelId)]?.promptCaching ?? false;
|
||||
}
|
||||
|
|
@ -1436,9 +1436,13 @@ describe('AgentsBuilderToolsService', () => {
|
|||
});
|
||||
|
||||
describe('prompt caching defaults', () => {
|
||||
it('write_config defaults prompt caching to enabled for a supported provider when omitted', async () => {
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
'google-vertex/claude-sonnet-4-5',
|
||||
])('write_config enables caching for %s when omitted', async (model) => {
|
||||
const { service, agentsService } = makeService();
|
||||
const currentConfig = { ...baseConfig, integrations: [] };
|
||||
const currentConfig = { ...baseConfig, model, integrations: [] };
|
||||
// Explicitly disable web search so its own write-path normalizer
|
||||
// doesn't add unrelated config/providerTools keys to the expectation.
|
||||
const updatedConfig: AgentJsonConfig = {
|
||||
|
|
@ -1449,7 +1453,7 @@ describe('AgentsBuilderToolsService', () => {
|
|||
...currentConfig,
|
||||
config: { webSearch: { enabled: false }, promptCaching: { enabled: true } },
|
||||
};
|
||||
agentsService.findById.mockResolvedValue(makeAgent(baseConfig));
|
||||
agentsService.findById.mockResolvedValue(makeAgent(currentConfig));
|
||||
agentsService.updateConfig.mockResolvedValue({
|
||||
config: normalizedConfig,
|
||||
configHash: 'config-hash',
|
||||
|
|
@ -1477,7 +1481,11 @@ describe('AgentsBuilderToolsService', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('write_config strips prompt caching when switching to an unsupported provider', async () => {
|
||||
it.each([
|
||||
'google/gemini-2.5-flash',
|
||||
'openrouter/openai/gpt-5',
|
||||
'google-vertex/gemini-2.5-pro',
|
||||
])('write_config strips caching when switching to %s', async (model) => {
|
||||
const { service, agentsService } = makeService();
|
||||
const baseAgent = {
|
||||
...baseConfig,
|
||||
|
|
@ -1487,7 +1495,7 @@ describe('AgentsBuilderToolsService', () => {
|
|||
const currentConfig = { ...baseAgent };
|
||||
const updatedConfig: AgentJsonConfig = {
|
||||
...currentConfig,
|
||||
model: 'google/gemini-2.5-flash',
|
||||
model,
|
||||
credential: 'Google Key',
|
||||
};
|
||||
const { config: _droppedConfig, ...normalizedConfig } = updatedConfig;
|
||||
|
|
@ -1569,10 +1577,15 @@ describe('AgentsBuilderToolsService', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('write_config preserves an explicit Anthropic ttl', async () => {
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
'google-vertex/claude-sonnet-4-5',
|
||||
])('write_config preserves the cache TTL for %s', async (model) => {
|
||||
const { service, agentsService } = makeService();
|
||||
const baseAgent = {
|
||||
...baseConfig,
|
||||
model,
|
||||
integrations: [],
|
||||
config: { promptCaching: { enabled: true, anthropic: { ttl: '5m' as const } } },
|
||||
};
|
||||
|
|
|
|||
|
|
@ -75,10 +75,28 @@ describe('buildFromJson()', () => {
|
|||
executeToMessage: vi.fn().mockResolvedValue(undefined),
|
||||
});
|
||||
|
||||
const makeMockCredentialProvider = () => ({
|
||||
resolve: vi.fn().mockResolvedValue({ apiKey: 'test-api-key' }),
|
||||
list: vi.fn().mockResolvedValue([]),
|
||||
});
|
||||
const makeMockCredentialProvider = (model = '') => {
|
||||
const isVertex = model.startsWith('google-vertex/');
|
||||
return {
|
||||
resolve: vi.fn().mockResolvedValue(
|
||||
isVertex
|
||||
? {
|
||||
projectId: 'test-project',
|
||||
region: 'global',
|
||||
email: 'test@example.com',
|
||||
privateKey: 'test-key',
|
||||
}
|
||||
: { apiKey: 'test-api-key' },
|
||||
),
|
||||
list: vi
|
||||
.fn()
|
||||
.mockResolvedValue(
|
||||
isVertex
|
||||
? [{ id: 'my-anthropic-key', name: 'Test Vertex', type: 'googleVertexAiApi' }]
|
||||
: [],
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
const makeToolDescriptor = (overrides: Partial<ToolDescriptor> = {}): ToolDescriptor => ({
|
||||
name: 'search',
|
||||
|
|
@ -666,8 +684,13 @@ describe('buildFromJson()', () => {
|
|||
expect(snap.reasoning).toBe('high');
|
||||
});
|
||||
|
||||
it('sets prompt caching config with an Anthropic ttl', async () => {
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
'google-vertex/claude-sonnet-4-5',
|
||||
])('preserves the saved cache TTL for %s', async (model) => {
|
||||
const config = makeConfig({
|
||||
model,
|
||||
config: { promptCaching: { enabled: true, anthropic: { ttl: '1h' } } },
|
||||
});
|
||||
|
||||
|
|
@ -676,7 +699,7 @@ describe('buildFromJson()', () => {
|
|||
{},
|
||||
{
|
||||
toolExecutor: makeMockToolExecutor(),
|
||||
credentialProvider: makeMockCredentialProvider(),
|
||||
credentialProvider: makeMockCredentialProvider(model),
|
||||
memoryFactory: makeMockMemoryFactory(),
|
||||
},
|
||||
);
|
||||
|
|
@ -704,8 +727,13 @@ describe('buildFromJson()', () => {
|
|||
expect(snap.promptCaching).toEqual({ enabled: true });
|
||||
});
|
||||
|
||||
it('passes through promptCaching verbatim even if enabled=false was somehow saved (mapping is not a policy enforcer)', async () => {
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
'google-vertex/claude-sonnet-4-5',
|
||||
])('preserves disabled caching for %s', async (model) => {
|
||||
const config = makeConfig({
|
||||
model,
|
||||
config: { promptCaching: { enabled: false } },
|
||||
});
|
||||
|
||||
|
|
@ -714,7 +742,7 @@ describe('buildFromJson()', () => {
|
|||
{},
|
||||
{
|
||||
toolExecutor: makeMockToolExecutor(),
|
||||
credentialProvider: makeMockCredentialProvider(),
|
||||
credentialProvider: makeMockCredentialProvider(model),
|
||||
memoryFactory: makeMockMemoryFactory(),
|
||||
},
|
||||
);
|
||||
|
|
@ -723,15 +751,19 @@ describe('buildFromJson()', () => {
|
|||
expect(snap.promptCaching).toEqual({ enabled: false });
|
||||
});
|
||||
|
||||
it('does not set prompt caching config when omitted', async () => {
|
||||
const config = makeConfig();
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/openai/gpt-5',
|
||||
'google-vertex/gemini-2.5-pro',
|
||||
])('keeps the runtime cache default for %s when config is omitted', async (model) => {
|
||||
const config = makeConfig({ model });
|
||||
|
||||
const agent = await buildFromJson(
|
||||
config,
|
||||
{},
|
||||
{
|
||||
toolExecutor: makeMockToolExecutor(),
|
||||
credentialProvider: makeMockCredentialProvider(),
|
||||
credentialProvider: makeMockCredentialProvider(model),
|
||||
memoryFactory: makeMockMemoryFactory(),
|
||||
},
|
||||
);
|
||||
|
|
@ -739,6 +771,24 @@ describe('buildFromJson()', () => {
|
|||
expect(agent.snapshot.promptCaching).toBeNull();
|
||||
});
|
||||
|
||||
it.each(['openrouter/anthropic/claude-sonnet-4.5', 'google-vertex/claude-sonnet-4-5'])(
|
||||
'enables caching for an existing %s agent without a config update',
|
||||
async (model) => {
|
||||
const config = makeConfig({ model });
|
||||
const agent = await buildFromJson(
|
||||
config,
|
||||
{},
|
||||
{
|
||||
toolExecutor: makeMockToolExecutor(),
|
||||
credentialProvider: makeMockCredentialProvider(model),
|
||||
},
|
||||
);
|
||||
|
||||
expect(agent.snapshot.promptCaching).toEqual({ enabled: true });
|
||||
expect(config.config).toBeUndefined();
|
||||
},
|
||||
);
|
||||
|
||||
it('builds native provider tools for an inline child model, not the parent model', () => {
|
||||
const config = makeConfig({
|
||||
model: 'openai/gpt-4o',
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ import {
|
|||
} from '@n8n/agents/tool';
|
||||
import {
|
||||
applyNativeWebSearchDefaultOn,
|
||||
getProviderPrefix,
|
||||
getModelPromptCachingCapability,
|
||||
rejectIfDynamicSelectorUsesFromAi,
|
||||
rejectIfEmptyInstructions,
|
||||
rejectIfUnsupportedNativeWebSearch,
|
||||
|
|
@ -24,7 +24,6 @@ import {
|
|||
agentSkillSchema,
|
||||
agentTaskSchema,
|
||||
formatZodErrors,
|
||||
PROVIDER_CAPABILITIES,
|
||||
resolvePromptCaching,
|
||||
AgentJsonConfigSchema,
|
||||
isDraftAgentConfig,
|
||||
|
|
@ -351,17 +350,9 @@ function parseBuilderWriteConfig(incoming: unknown, currentConfig: AgentJsonConf
|
|||
return AgentJsonConfigSchema.safeParse(sanitized);
|
||||
}
|
||||
|
||||
/**
|
||||
* Prompt caching is mandatory for OpenAI/Anthropic: this write-path
|
||||
* normalizer guarantees `config.promptCaching` is force-enabled for those
|
||||
* providers (the user cannot disable it, even if the LLM wrote
|
||||
* `{ enabled: false }`), preserves an explicit Anthropic TTL, and strips the
|
||||
* field entirely for every other provider — regardless of what the builder
|
||||
* LLM wrote.
|
||||
*/
|
||||
/** Enable caching for supported models and preserve an explicit cache TTL. */
|
||||
function applyPromptCachingBuilderDefaults(config: AgentJsonConfig): AgentJsonConfig {
|
||||
const providerPrefix = getProviderPrefix(config.model);
|
||||
const capability = PROVIDER_CAPABILITIES[providerPrefix]?.promptCaching ?? false;
|
||||
const capability = getModelPromptCachingCapability(config.model);
|
||||
const resolved = resolvePromptCaching(config.config?.promptCaching, capability);
|
||||
|
||||
if (!resolved) {
|
||||
|
|
|
|||
|
|
@ -52,10 +52,10 @@ export function getConfigRulesSection(): string {
|
|||
- Sub-agent configuration lives at top level under \`subAgents\`. Load
|
||||
\`agent-builder-sub-agents\` before adding refs or changing
|
||||
\`subAgents.maxChildren\`.
|
||||
- For OpenAI and Anthropic models, always include
|
||||
- For OpenAI, Anthropic, and Claude models on OpenRouter or Google Vertex, always include
|
||||
\`config.promptCaching: { "enabled": true }\` — this is mandatory and must
|
||||
never be disabled, even if the user asks. Omit \`config.promptCaching\`
|
||||
entirely for every other provider. For Anthropic only, you may set
|
||||
entirely for other models. For Anthropic and routed Claude models, you may set
|
||||
\`config.promptCaching.anthropic.ttl\` to \`"5m"\` or \`"1h"\` (default
|
||||
\`"1h"\`) when the user asks to tune cache duration; OpenAI has no
|
||||
sub-config.
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ import {
|
|||
getProviderPrefix,
|
||||
hasNativeWebSearchProvider,
|
||||
isNativeWebSearchRequested,
|
||||
isRoutedAnthropicModel,
|
||||
} from '@n8n/ai-utilities/agent-config';
|
||||
import {
|
||||
AI_GATEWAY_MANAGED_TAG,
|
||||
|
|
@ -251,9 +252,6 @@ export async function buildFromJson(
|
|||
if (config.config.reasoning) {
|
||||
agent.reasoning(config.config.reasoning);
|
||||
}
|
||||
if (config.config.promptCaching) {
|
||||
agent.promptCaching(config.config.promptCaching);
|
||||
}
|
||||
if (config.config.toolCallConcurrency) {
|
||||
agent.toolCallConcurrency(config.config.toolCallConcurrency);
|
||||
}
|
||||
|
|
@ -262,6 +260,13 @@ export async function buildFromJson(
|
|||
}
|
||||
}
|
||||
|
||||
if (config.config?.promptCaching) {
|
||||
agent.promptCaching(config.config.promptCaching);
|
||||
} else if (isRoutedAnthropicModel(config.model)) {
|
||||
// Existing agents receive the cache default without a config update.
|
||||
agent.promptCaching({ enabled: true });
|
||||
}
|
||||
|
||||
return agent;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@
|
|||
"@internationalized/date": "catalog:frontend",
|
||||
"@lezer/common": "catalog:",
|
||||
"@lezer/highlight": "catalog:",
|
||||
"@n8n/ai-utilities": "workspace:*",
|
||||
"@n8n/api-types": "workspace:*",
|
||||
"@n8n/chat": "workspace:*",
|
||||
"@n8n/chat-hub": "workspace:*",
|
||||
|
|
|
|||
|
|
@ -487,8 +487,12 @@ describe('AgentAdvancedPanel', () => {
|
|||
expect(wrapper.emitted('update:config')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('shows the Anthropic ttl dropdown, defaulting to 1h, with no on/off toggle', async () => {
|
||||
const config = makeConfig();
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
'google-vertex/claude-sonnet-4-5',
|
||||
])('shows a default 1h cache duration for %s', async (model) => {
|
||||
const config = makeConfig({ model });
|
||||
const wrapper = mount(AgentAdvancedPanel, {
|
||||
props: { config },
|
||||
global: { stubs: globalStubs },
|
||||
|
|
@ -510,8 +514,12 @@ describe('AgentAdvancedPanel', () => {
|
|||
expect(wrapper.find('[data-testid="agent-prompt-caching-ttl-select"]').exists()).toBe(false);
|
||||
});
|
||||
|
||||
it('hides the prompt-caching row entirely for providers that do not support it', () => {
|
||||
const config = makeConfig({ model: 'google/gemini-pro' });
|
||||
it.each([
|
||||
'google/gemini-pro',
|
||||
'openrouter/google/gemini-2.5-pro',
|
||||
'google-vertex/gemini-2.5-pro',
|
||||
])('hides the cache duration for %s', (model) => {
|
||||
const config = makeConfig({ model });
|
||||
const wrapper = mount(AgentAdvancedPanel, {
|
||||
props: { config },
|
||||
global: { stubs: globalStubs },
|
||||
|
|
@ -520,8 +528,12 @@ describe('AgentAdvancedPanel', () => {
|
|||
expect(wrapper.find('[data-testid="agent-prompt-caching-ttl-select"]').exists()).toBe(false);
|
||||
});
|
||||
|
||||
it('emits { enabled: true, anthropic: { ttl } } when the ttl dropdown changes', () => {
|
||||
const config = makeConfig();
|
||||
it.each([
|
||||
'anthropic/claude-sonnet-4-5',
|
||||
'openrouter/anthropic/claude-sonnet-4.5',
|
||||
'google-vertex/claude-sonnet-4-5',
|
||||
])('saves the selected cache duration for %s', (model) => {
|
||||
const config = makeConfig({ model });
|
||||
const wrapper = mount(AgentAdvancedPanel, {
|
||||
props: { config },
|
||||
global: { stubs: globalStubs },
|
||||
|
|
|
|||
|
|
@ -331,6 +331,37 @@ describe('AgentInfoPanel', () => {
|
|||
expect(events.at(-1)?.[1]).toBeUndefined();
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ provider: 'openrouter', model: 'anthropic/claude-sonnet-4.5', supported: true },
|
||||
{ provider: 'openrouter', model: 'openai/gpt-5', supported: false },
|
||||
{ provider: 'google-vertex', model: 'claude-sonnet-4-5', supported: true },
|
||||
{ provider: 'google-vertex', model: 'gemini-2.5-pro', supported: false },
|
||||
])(
|
||||
'normalizes caching when selecting $provider/$model',
|
||||
async ({ provider, model, supported }) => {
|
||||
credsHolder.value[provider] = 'credential-2';
|
||||
const wrapper = mountModelPanel({
|
||||
name: 'Support agent',
|
||||
model: 'anthropic/claude-sonnet-4-5',
|
||||
credential: 'credential-1',
|
||||
instructions: 'Help users.',
|
||||
config: {
|
||||
toolCallConcurrency: 2,
|
||||
promptCaching: { enabled: true, anthropic: { ttl: '5m' } },
|
||||
},
|
||||
});
|
||||
wrapper.findComponent({ name: 'AgentModelSelector' }).vm.$emit('change', { provider, model });
|
||||
await wrapper.vm.$nextTick();
|
||||
|
||||
const changes = wrapper.emitted('update:config')?.at(-1)?.[0] as Partial<AgentJsonConfig>;
|
||||
expect(changes.model).toBe(`${provider}/${model}`);
|
||||
expect(changes.config?.toolCallConcurrency).toBe(2);
|
||||
expect(changes.config?.promptCaching).toEqual(
|
||||
supported ? { enabled: true, anthropic: { ttl: '5m' } } : undefined,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('forwards the model selector\'s own "auto" source through to update:config', async () => {
|
||||
const config: AgentJsonConfig = {
|
||||
name: 'Support agent',
|
||||
|
|
|
|||
|
|
@ -5,26 +5,18 @@ import { useDebounceFn } from '@vueuse/core';
|
|||
import { AGENT_REASONING_LEVELS, type AgentReasoningLevel } from '@n8n/api-types';
|
||||
import { N8nInputNumber, N8nOption, N8nSelect, N8nSwitch2, N8nText } from '@n8n/design-system';
|
||||
import { useI18n } from '@n8n/i18n';
|
||||
import { getModelPromptCachingCapability } from '@n8n/ai-utilities/agent-config/model-capabilities';
|
||||
import AgentMemoryModelSetting from './AgentMemoryModelSetting.vue';
|
||||
import AgentPanel from './AgentPanel.vue';
|
||||
|
||||
import { useModelCatalog } from '../composables/useModelCatalog';
|
||||
import type { AgentJsonConfig } from '../types';
|
||||
import {
|
||||
PROVIDER_CAPABILITIES,
|
||||
ANTHROPIC_CACHE_TTL_OPTIONS,
|
||||
type AnthropicCacheTtl,
|
||||
} from '../provider-capabilities';
|
||||
import { modelToString, parseModelString, parseProvider } from '../utils/model-string';
|
||||
import { ANTHROPIC_CACHE_TTL_OPTIONS, type AnthropicCacheTtl } from '../provider-capabilities';
|
||||
import { modelToString, parseModelString } from '../utils/model-string';
|
||||
import shared from '../styles/agent-panel.module.scss';
|
||||
|
||||
const i18n = useI18n();
|
||||
const { catalog, ensureLoaded } = useModelCatalog();
|
||||
const DEFAULT_CAPABILITIES = {
|
||||
promptCaching: false,
|
||||
webSearch: false,
|
||||
providerTools: [],
|
||||
} as const;
|
||||
|
||||
function normalizeReasoningLevel(value: unknown): AgentReasoningLevel {
|
||||
return AGENT_REASONING_LEVELS.find((level) => level === value) ?? 'medium';
|
||||
|
|
@ -42,7 +34,6 @@ const props = withDefaults(
|
|||
);
|
||||
const emit = defineEmits<{ 'update:config': [changes: Partial<AgentJsonConfig>] }>();
|
||||
|
||||
const provider = computed(() => parseProvider(props.config?.model));
|
||||
const selectedModel = computed(() => parseModelString(modelToString(props.config?.model)));
|
||||
const selectedCatalogModel = computed(() => {
|
||||
if (!selectedModel.value) return undefined;
|
||||
|
|
@ -58,7 +49,9 @@ const reasoningHintKey = computed(() => {
|
|||
}
|
||||
return 'agents.builder.advanced.reasoning.hint';
|
||||
});
|
||||
const capabilities = computed(() => PROVIDER_CAPABILITIES[provider.value] ?? DEFAULT_CAPABILITIES);
|
||||
const promptCachingCapability = computed(() =>
|
||||
getModelPromptCachingCapability(modelToString(props.config?.model)),
|
||||
);
|
||||
|
||||
watch(
|
||||
() => props.projectId,
|
||||
|
|
@ -298,7 +291,7 @@ function onAnthropicTtlChange(value: AnthropicCacheTtl) {
|
|||
</div>
|
||||
</div>
|
||||
|
||||
<div v-if="capabilities.promptCaching === 'ttl'" :class="$style.settingGroup">
|
||||
<div v-if="promptCachingCapability === 'ttl'" :class="$style.settingGroup">
|
||||
<div :class="shared.settingRow">
|
||||
<div :class="shared.settingLabel">
|
||||
<N8nText step="sm" bold :class="shared.dataEntryLabel">{{
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ import {
|
|||
N8nText,
|
||||
} from '@n8n/design-system';
|
||||
import { useI18n } from '@n8n/i18n';
|
||||
import { getModelPromptCachingCapability } from '@n8n/ai-utilities/agent-config/model-capabilities';
|
||||
|
||||
import { getDebounceTime } from '@n8n/composables/useDebounce';
|
||||
import { DEBOUNCE_TIME } from '@/app/constants/durations';
|
||||
|
|
@ -257,7 +258,7 @@ function onModelChange(selection: AgentModelSelection, source: 'user' | 'auto' =
|
|||
'config' in webSearchChanges ? webSearchChanges.config : props.config?.config;
|
||||
const promptCachingChanges = normalizePromptCachingForModelChange(
|
||||
webSearchConfig,
|
||||
capabilities?.promptCaching ?? false,
|
||||
getModelPromptCachingCapability(model),
|
||||
);
|
||||
const normalizedConfig =
|
||||
'config' in promptCachingChanges ? promptCachingChanges.config : webSearchConfig;
|
||||
|
|
|
|||
|
|
@ -6583,6 +6583,9 @@ importers:
|
|||
'@lezer/highlight':
|
||||
specifier: 'catalog:'
|
||||
version: 1.2.3(patch_hash=97f85e6fe46f23015ea0dd420e33d584bc2dc71633910cf131321da31b27ca8c)
|
||||
'@n8n/ai-utilities':
|
||||
specifier: workspace:*
|
||||
version: link:../../@n8n/ai-utilities
|
||||
'@n8n/api-types':
|
||||
specifier: workspace:*
|
||||
version: link:../../@n8n/api-types
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user