fix(core): Enable Claude prompt caching for OpenRouter agents (no-changelog)

Share Claude cache capabilities and defaults with Vertex. Forward OpenRouter reasoning settings and correct cache-write token accounting.
This commit is contained in:
Robin Braumann 2026-10-12 13:37:37 +02:00
parent a36cff36cd
commit c8eb21b2a0
No known key found for this signature in database
32 changed files with 991 additions and 120 deletions

View File

@ -1,8 +1,8 @@
# Prompt caching
`Agent.promptCaching()` generates provider-specific `providerOptions` for
Anthropic and OpenAI so callers don't have to hand-write the Vercel AI SDK's
raw `cacheControl` / `promptCacheKey` shapes. It layers on top of the existing
Anthropic, Claude on OpenRouter and Vertex, and OpenAI. Callers do not have to
write the AI SDK's raw `cacheControl` / `promptCacheKey` shapes. It uses the existing
`providerOptions` escape hatch — nothing here is required, and anything set
explicitly (via `.instructions(text, { providerOptions })`, call-level
`providerOptions`, or `Tool.providerOptions(...)`) always wins on conflicts.
@ -29,24 +29,24 @@ new Agent('assistant')
The richer SDK shape above (OpenAI key/retention overrides) is for direct SDK
callers. The n8n Agent product (JSON config / builder UI) exposes
`config.promptCaching: { enabled: boolean; anthropic?: { ttl?: '5m' | '1h' } }`
and maps it straight to `agent.promptCaching(config.promptCaching)`. Two
differences from the general SDK usage worth knowing:
and passes explicit settings to `agent.promptCaching(config.promptCaching)`.
- **Mandatory, not opt-in or opt-out.** For OpenAI and Anthropic agents, the
config is always force-written with `{ enabled: true, ... }` — by the
- **Mandatory in the builder.** For OpenAI, Anthropic, and Claude on OpenRouter
or Vertex, the config is always force-written with `{ enabled: true, ... }` by the
builder agent's write-path normalizer and the model picker — regardless of
what the LLM or a prior config said; there is no user-facing way to disable
it. Every other provider never gets the field at all. As with the SDK, a
missing `promptCaching` field is still just "disabled" at runtime — the
mandatory-on behavior lives in the config producers (builder tools, model
picker), not in the runtime default.
- **TTL is Anthropic-only and defaults to `1h`.** The Advanced panel renders
a Cache duration dropdown (`5m` / `1h`) only for Anthropic agents; OpenAI
has no sub-config or UI (its caching is fully automatic server-side). The
TTL is preserved across a switch between Anthropic and another supported
provider and back.
it. Other models do not get the field.
- **Existing routed Claude agents get the default on their next run.** When a
saved `openrouter/anthropic/claude-*` or `google-vertex/claude-*` agent has no
cache config, the JSON loader supplies `{ enabled: true }`. This does not
change the saved config. Explicit settings, including `enabled: false`, pass
through unchanged. Standalone SDK use stays opt-in.
- **Claude cache duration defaults to `1h`.** The Advanced panel renders
a Cache duration dropdown (`5m` / `1h`) for Anthropic and routed Claude agents.
OpenAI has no sub-config or UI (its caching is fully automatic server-side).
The TTL is preserved when switching between the supported Claude routes.
## Prefix stability (always on, both providers)
## Prefix stability (always on)
Unlike everything else in this doc, this part is not gated by
`.promptCaching()` — it's baseline hygiene that keeps the system-instructions
@ -90,11 +90,12 @@ exists.
The block is not part of the base instructions. `buildSystemMessages` sends it
as its own system message, after the base instructions and before the volatile
message. The tools and base instructions in front of it stay byte-identical, so
they stay cached when a skill moves into the block. The block reuses the
instruction cache options, so it gets its own breakpoint and stays cached when
memory compacts again later. `buildSkillInstructionCacheOptions` drops that
breakpoint when it would leave no slot for the conversation breakpoint next to
caller breakpoints on tools, history messages, and their content parts.
they stay cached when a skill moves into the block. On the native Anthropic API,
the block reuses the instruction cache options. It gets its own breakpoint and
stays cached when memory compacts again later.
`buildSkillInstructionCacheOptions` drops that breakpoint when it would leave
no slot for the conversation breakpoint next to caller breakpoints on tools,
history messages, and their content parts.
Providers without split system messages get the block merged after the base
instructions, as before.
@ -133,7 +134,8 @@ the cache breakpoint on instructions survives append-only memory growth).
- Anthropic requires a minimum prompt length to cache at all (roughly 1,024
tokens for the Sonnet family, 4,096 tokens for Opus 4.5 / Haiku 4.5).
Below that, the request still succeeds — it's just never cached.
- Only Anthropic models get this option; it's a no-op for other providers.
- Direct Anthropic and Claude on Vertex use this path. OpenRouter uses the
separate path below.
## Anthropic: runtime breakpoints (conversation history + tools)
@ -175,6 +177,36 @@ breakpoint takes priority over the tool breakpoint — it recurs every turn,
while a static tool block is already covered as part of the history
breakpoint's prefix.
## OpenRouter: Claude instructions and automatic conversation caching
For `openrouter/anthropic/claude-*`, `.promptCaching()` adds two controls:
- An explicit `cache_control` on the stable system instructions.
- A top-level `cache_control` for automatic conversation caching. OpenRouter
moves this breakpoint as the conversation grows.
Both use `anthropic.ttl` from the existing config. The default is `1h`.
The runtime adds these controls after it assembles the prompt for each call.
It does not change stored messages or tool definitions.
The four-breakpoint limit includes automatic caching. Caller controls take
priority, then stable instructions, then automatic caching. The runtime counts
both `anthropic` and `openrouter` options and both cache-control key spellings.
A message marker on grouped tool results counts once for each result because
the adapter copies it to every tool message. Generated controls are omitted
when no slot remains or when they would put a longer TTL after a shorter one.
An explicit marker on the final block also suppresses generated automatic
caching, which can otherwise conflict with that marker's TTL.
The OpenRouter adapter does not forward cache markers on tool definitions.
The runtime adds no such markers, and it adds no markers to tool results or
recovered-skill system messages. Stable tool definitions are still part of the
cached prefix before the instruction breakpoint.
Generic `.reasoning()` settings map to `openrouter.reasoning.effort`.
Explicit `openrouter.reasoning` options take precedence, including token
budgets. The adapter retains reasoning details through tool calls.
## OpenAI: cache key + retention
`.promptCaching()` attaches `providerOptions.openai.promptCacheKey` and
@ -205,15 +237,20 @@ Anthropic's native cache breakdown). A healthy Anthropic
means the TTL is too short for the traffic pattern, or the cached prefix
isn't byte-stable across calls.
For OpenRouter, the runtime removes cache-write tokens from the adapter's
uncached-input count in both generated and streamed responses. Each input
token belongs to one cost bucket. Raw provider usage remains unchanged.
`GenerateResult.usage.cost` / the `finish` chunk's `usage.cost` factor in the
model catalog's cache rates when available. One known approximation: the
catalog's `cacheWrite` rate reflects Anthropic's 5-minute write premium
(1.25x base input); when the effective TTL is `1h` the runtime scales that
rate by ~1.6x (2x/1.25x) to approximate the 1h premium, rather than looking
up a separate 1h rate from the catalog. This 1h scaling only applies when
`.promptCaching()` is enabled for an Anthropic model — cache writes from a
manual, caller-supplied `cacheControl` marker (without `.promptCaching()`)
are billed at the catalog's 5-minute rate.
`.promptCaching()` is enabled for an Anthropic model, including Claude on
OpenRouter and Vertex. Cache writes from a manual, caller-supplied
`cacheControl` marker (without `.promptCaching()`) use the catalog's 5-minute
rate in the estimate.
## What this does not do

View File

@ -1,4 +1,5 @@
import { createGoogleVertex } from '@ai-sdk/google-vertex';
import { createOpenRouter } from '@openrouter/ai-sdk-provider';
import type { LanguageModel } from 'ai';
import {
@ -192,14 +193,14 @@ vi.mock('@ai-sdk/azure', () => ({
}));
vi.mock('@openrouter/ai-sdk-provider', () => ({
createOpenRouter: (opts?: ProviderOpts) => (model: string) => ({
createOpenRouter: vi.fn((opts?: ProviderOpts) => (model: string) => ({
provider: 'openrouter',
modelId: model,
apiKey: opts?.apiKey,
baseURL: opts?.baseURL,
fetch: opts?.fetch,
specificationVersion: 'v3',
}),
})),
}));
// Entra OAuth2 token mint: stubbed so the model-factory's tokenProvider
@ -585,7 +586,7 @@ describe('createModel', () => {
}) as unknown as Record<string, unknown>;
expect(model.provider).toBe('openrouter');
expect(model.modelId).toBe('openai/gpt-4o');
expect(model.apiKey).toBe('or-test');
expect(createOpenRouter).toHaveBeenCalledWith(expect.objectContaining({ apiKey: 'or-test' }));
});
it('should create model for nvidia with SDK defaults for usage and structured outputs', () => {

View File

@ -1,5 +1,8 @@
import { createRuntimeSkillSource } from '../../../skills/registry';
import type { ModelConfig } from '../../../types/sdk/agent';
import { getModelIdString } from '../../../utils/model';
import { AgentMessageList } from '../../model/message-list';
import { ActiveSkills } from '../../skills/active-skills';
import { RuntimeContextBuilder } from '../runtime-context';
describe('getModelIdString', () => {
@ -36,6 +39,53 @@ describe('getModelIdString', () => {
});
});
describe('OpenRouter recovered skills', () => {
it('keeps the caller instruction marker without copying it to recovered skills', async () => {
const marker = { anthropic: { cacheControl: { type: 'ephemeral', ttl: '1h' } } };
const builder = new RuntimeContextBuilder(
{
name: 'test',
model: 'openrouter/anthropic/claude-sonnet-4.5',
instructions: 'Base instructions',
instructionProviderOptions: marker,
promptCaching: { enabled: true },
},
undefined,
);
const list = new AgentMessageList();
list.addInput([{ role: 'user', content: [{ type: 'text', text: 'Hello' }] }]);
const source = createRuntimeSkillSource([
{
id: 'notes',
name: 'Notes',
description: 'Summarize notes.',
instructions: 'Extract decisions.',
},
]);
const activeSkills = new ActiveSkills(source, 'test');
await activeSkills.restore(list);
await activeSkills.load('notes');
const before = list.serialize();
const prompt = builder.buildModelPrompt({
list,
tools: builder.buildToolLoopContext({}),
instructionProviderOptions: builder.buildInstructionProviderOptions(),
providerOptions: { openrouter: { user: 'test-user' } },
hostVolatileInstructions: 'Current context',
activeSkills,
});
expect(prompt.system).toHaveLength(3);
expect(JSON.stringify(prompt.system).match(/"cacheControl":/g)).toHaveLength(1);
expect(prompt.system).toContainEqual(expect.objectContaining({ providerOptions: marker }));
expect(prompt.providerOptions).toEqual({
openrouter: { user: 'test-user', cacheControl: { type: 'ephemeral', ttl: '1h' } },
});
expect(list.serialize()).toEqual(before);
});
});
describe('Vertex thinking options', () => {
it.each([
{

View File

@ -1063,6 +1063,7 @@ export class AgentRuntime {
list,
tools,
instructionProviderOptions: ctx.instructionProviderOptions,
providerOptions: staticContext.providerOptions,
hostVolatileInstructions,
activeSkills: this.activeSkills,
});
@ -1074,7 +1075,7 @@ export class AgentRuntime {
hasTools: options?.toolsEnabled !== false && tools.hasTools,
aiTools: prompt.aiTools,
reasoning: staticContext.reasoning,
providerOptions: staticContext.providerOptions,
providerOptions: prompt.providerOptions,
outputSpec: staticContext.outputSpec,
maxOutputTokens: staticContext.maxOutputTokens,
aiSdkOptions: this.buildAiSdkOptions(tools.toolMap, options),

View File

@ -1,5 +1,9 @@
import type { ProviderOptions } from '@ai-sdk/provider-utils';
import { getProviderPrefix } from '@n8n/ai-utilities/agent-config';
import {
getProviderPrefix,
isGoogleVertexAnthropicModel,
isOpenRouterAnthropicModel,
} from '@n8n/ai-utilities/agent-config';
import type { LanguageModel, Output } from 'ai';
import type { AgentRuntimeConfig } from '../../types/runtime/agent-runtime';
@ -21,10 +25,10 @@ import {
FLAG_MEMORY_TOOL_NAME,
resolveEpisodicMemoryCapture,
} from '../memory/episodic-memory-capture';
import { isGoogleVertexAnthropicModel } from '../model/google-vertex';
import { loadAi } from '../model/lazy-ai';
import type { AgentMessageList } from '../model/message-list';
import { createModel, supportsSplitSystemMessages } from '../model/model-factory';
import { applyOpenRouterPromptCache } from '../model/openrouter-prompt-cache';
import {
applyRuntimeCacheBreakpoints,
buildCallPromptCacheOptions,
@ -154,12 +158,14 @@ export class RuntimeContextBuilder {
list,
tools,
instructionProviderOptions,
providerOptions,
hostVolatileInstructions,
activeSkills,
}: {
list: AgentMessageList;
tools: ReturnType<RuntimeContextBuilder['buildToolLoopContext']>;
instructionProviderOptions: ProviderOptions | undefined;
providerOptions?: ProviderOptions;
hostVolatileInstructions: string | undefined;
activeSkills: ActiveSkills | undefined;
}) {
@ -176,12 +182,14 @@ export class RuntimeContextBuilder {
skillContent
? {
content: skillContent,
cacheOptions: (messages) =>
buildSkillInstructionCacheOptions(
cacheOptions: (messages) => {
if (isOpenRouterAnthropicModel(this.modelId)) return undefined;
return buildSkillInstructionCacheOptions(
instructionProviderOptions,
tools.aiTools,
messages,
),
);
},
}
: undefined,
);
@ -194,7 +202,14 @@ export class RuntimeContextBuilder {
modelId: this.modelId,
staticToolCacheName: tools.staticToolCacheName,
});
return { system, ...cached };
const openRouterCache = applyOpenRouterPromptCache({
system,
messages: cached.messages,
providerOptions,
promptCaching: this.config.promptCaching,
modelId: this.modelId,
});
return { ...cached, ...openRouterCache };
}
/**

View File

@ -0,0 +1,182 @@
import type { ProviderOptions } from '@ai-sdk/provider-utils';
import { generateText, type ModelMessage } from 'ai';
import { createModel } from '../model-factory';
import { applyOpenRouterPromptCache } from '../openrouter-prompt-cache';
type CacheParams = Parameters<typeof applyOpenRouterPromptCache>[0];
type WireRequest = {
cache_control?: { type: string; ttl?: string };
messages: Array<{ role: string; content: unknown; cache_control?: unknown }>;
};
const CACHE_1H = { type: 'ephemeral', ttl: '1h' };
const CACHE_5M = { type: 'ephemeral', ttl: '5m' };
async function prepareRequest(overrides: Partial<CacheParams> = {}): Promise<WireRequest> {
const params: CacheParams = {
modelId: 'openrouter/anthropic/claude-sonnet-4.5',
promptCaching: { enabled: true },
system: [
{ role: 'system', content: 'Stable instructions' },
{ role: 'system', content: 'Recovered skills' },
{ role: 'system', content: 'Volatile instructions' },
],
messages: [{ role: 'user', content: 'Hello' }],
providerOptions: undefined,
...overrides,
};
const original = structuredClone(params);
const prepared = applyOpenRouterPromptCache(params);
const fetch = vi.fn<typeof globalThis.fetch>().mockResolvedValue(
Response.json({
id: 'completion-1',
model: 'anthropic/claude-sonnet-4.5',
choices: [
{ index: 0, message: { role: 'assistant', content: 'Done.' }, finish_reason: 'stop' },
],
}),
);
await generateText({
model: createModel({ id: params.modelId, apiKey: 'test-key' }, fetch),
instructions: prepared.system,
messages: params.messages,
providerOptions: prepared.providerOptions,
maxRetries: 0,
});
expect(params).toEqual(original);
return JSON.parse(fetch.mock.calls[0][1]?.body as string) as WireRequest;
}
function toolHistory(count: number, providerOptions: ProviderOptions): ModelMessage[] {
const ids = Array.from({ length: count }, (_, index) => `call-${index}`);
return [
{ role: 'user', content: 'Look up records' },
{
role: 'assistant',
content: ids.map((id) => ({
type: 'tool-call',
toolCallId: id,
toolName: 'lookup',
input: {},
})),
},
{
role: 'tool',
providerOptions,
content: ids.map((id) => ({
type: 'tool-result',
toolCallId: id,
toolName: 'lookup',
output: { type: 'text', value: 'Found' },
})),
},
{ role: 'user', content: 'Continue' },
];
}
describe('OpenRouter prompt caching', () => {
it.each([
{ promptCaching: undefined },
{ promptCaching: { enabled: false } },
{ promptCaching: { anthropic: false } },
{ modelId: 'openrouter/google/gemini-2.5-pro' },
] satisfies Array<Partial<CacheParams>>)('leaves caching unchanged for %j', async (overrides) => {
const request = await prepareRequest(overrides);
expect(JSON.stringify(request)).not.toContain('cache_control');
});
it.each([
{ count: 2, namespace: 'anthropic', key: 'cacheControl', automatic: true },
{ count: 3, namespace: 'openrouter', key: 'cache_control', automatic: false },
{ count: 4, namespace: 'anthropic', key: 'cache_control', automatic: false },
{ count: 4, namespace: 'openrouter', key: 'cacheControl', automatic: false },
])(
'keeps four wire markers with $count marked tool results via $namespace.$key',
async ({ count, namespace, key, automatic }) => {
const request = await prepareRequest({
messages: toolHistory(count, { [namespace]: { [key]: CACHE_1H } }),
});
expect(request.cache_control).toEqual(automatic ? CACHE_1H : undefined);
expect(request.messages[0].content).toEqual([
{
type: 'text',
text: 'Stable instructions',
...(count < 4 ? { cache_control: CACHE_1H } : {}),
},
]);
expect(JSON.stringify(request).match(/"cache_control":/g)).toHaveLength(4);
},
);
it.each(['cacheControl', 'cache_control'])(
'reserves a slot for explicit automatic %s',
async (key) => {
const request = await prepareRequest({
messages: toolHistory(3, { openrouter: { cacheControl: CACHE_1H } }),
providerOptions: { openrouter: { [key]: CACHE_5M } },
});
expect(request.cache_control).toEqual(CACHE_5M);
expect(request.messages[0].content).toEqual([{ type: 'text', text: 'Stable instructions' }]);
expect(JSON.stringify(request).match(/"cache_control":/g)).toHaveLength(4);
},
);
it('uses the raw automatic control when both caller keys are present', async () => {
const request = await prepareRequest({
providerOptions: { openrouter: { cacheControl: CACHE_1H, cache_control: CACHE_5M } },
});
expect(request.cache_control).toEqual(CACHE_5M);
expect(request.messages[0].content).toEqual([
{ type: 'text', text: 'Stable instructions', cache_control: CACHE_5M },
]);
});
it.each([
{ ttl: '1h', caller: CACHE_5M, trailing: true, instruction: CACHE_1H, automatic: undefined },
{ ttl: '5m', caller: CACHE_1H, trailing: true, instruction: undefined, automatic: CACHE_5M },
{ ttl: '5m', caller: CACHE_1H, trailing: false, instruction: undefined, automatic: undefined },
{ ttl: '1h', caller: CACHE_1H, trailing: false, instruction: CACHE_1H, automatic: undefined },
] as const)(
'respects TTL order and final markers: %j',
async ({ ttl, caller, trailing, instruction, automatic }) => {
const messages: ModelMessage[] = [
{
role: 'user',
content: 'Caller prompt',
providerOptions: { anthropic: { cacheControl: caller } },
},
];
if (trailing) messages.push({ role: 'user', content: 'Continue' });
const request = await prepareRequest({ messages, promptCaching: { anthropic: { ttl } } });
expect(request.cache_control).toEqual(automatic);
expect(request.messages[0].content).toEqual([
{
type: 'text',
text: 'Stable instructions',
...(instruction ? { cache_control: instruction } : {}),
},
]);
},
);
it.each([false, true])(
'matches adapter precedence for multipart=%s user content',
async (multipart) => {
const content: Extract<ModelMessage, { role: 'user' }>['content'] = [
{
type: 'text',
text: 'First part',
providerOptions: { openrouter: { cache_control: CACHE_5M } },
},
];
if (multipart) content.push({ type: 'text', text: 'Second part' });
const request = await prepareRequest({
messages: [
{ role: 'user', content, providerOptions: { anthropic: { cacheControl: CACHE_1H } } },
{ role: 'user', content: 'Continue' },
],
});
expect(request.cache_control).toEqual(multipart ? undefined : CACHE_1H);
},
);
});

View File

@ -0,0 +1,247 @@
import { generateText, streamText } from 'ai';
import { Tool } from '../../../sdk/tool';
import type { StreamChunk, TokenUsage } from '../../../types';
import { AgentRuntime } from '../../loop/agent-runtime';
import { createModel } from '../model-factory';
const MODEL_ID = 'openrouter/anthropic/claude-sonnet-4.5';
const USAGE = {
prompt_tokens: 1000,
completion_tokens: 10,
total_tokens: 1010,
prompt_tokens_details: { cached_tokens: 200, cache_write_tokens: 300 },
};
const REASONING_DETAILS = [
{
type: 'reasoning.text',
text: 'I will look up the records.',
signature: 'reasoning-signature',
format: 'anthropic-claude-v1',
index: 0,
},
{
type: 'reasoning.encrypted',
data: 'opaque-reasoning',
id: 'reasoning-1',
format: 'anthropic-claude-v1',
index: 1,
},
];
function response(stream: boolean, withTools = false, usage: object = USAGE): Response {
const base = { id: 'completion-1', model: 'anthropic/claude-sonnet-4.5', created: 1 };
const message = withTools
? {
role: 'assistant',
content: null,
reasoning_details: REASONING_DETAILS,
tool_calls: ['a', 'b', 'c', 'd'].map((id, index) => ({
id,
index,
type: 'function',
function: { name: 'lookup', arguments: '{}' },
})),
}
: { role: 'assistant', content: 'Done.' };
const finishReason = withTools ? 'tool_calls' : 'stop';
if (!stream) {
return Response.json({
...base,
provider: 'Anthropic',
choices: [{ index: 0, message, finish_reason: finishReason }],
usage,
});
}
const chunks = [
{ ...base, choices: [{ index: 0, delta: message, finish_reason: null }] },
{ ...base, choices: [{ index: 0, delta: {}, finish_reason: finishReason }], usage },
];
return new Response(
chunks.map((chunk) => `data: ${JSON.stringify(chunk)}\n\n`).join('') + 'data: [DONE]\n\n',
{ headers: { 'content-type': 'text/event-stream' } },
);
}
describe.each(['generate', 'stream'] as const)('OpenRouter %s', (mode) => {
it.each([
{ ttl: '1h', cost: 0.00244 },
{ ttl: '5m', cost: 0.00199 },
] as const)(
'caches the tool loop and prices each token once with $ttl',
async ({ ttl, cost }) => {
const fetch = vi.fn<typeof globalThis.fetch>();
fetch.mockImplementation(async () =>
response(mode === 'stream', fetch.mock.calls.length === 1),
);
const runtime = new AgentRuntime({
name: 'test',
model: { id: MODEL_ID, apiKey: 'test-key' },
modelFetch: fetch,
modelCost: { input: 1, output: 10, cacheRead: 0.1, cacheWrite: 1.25 },
instructions: 'Look up the requested records.',
promptCaching: { enabled: true, anthropic: { ttl } },
reasoning: 'high',
tools: [
new Tool('lookup')
.description('Look up a record')
.input({ type: 'object', properties: {} })
.handler(async () => ({ found: true }))
.build(),
],
});
let usage: TokenUsage | undefined;
if (mode === 'generate') {
const result = await runtime.generate('Look up four records.');
expect(result.finishReason).toBe('stop');
expect(result.messages).toContainEqual(
expect.objectContaining({
role: 'assistant',
content: expect.arrayContaining([
expect.objectContaining({ type: 'text', text: 'Done.' }),
]),
}),
);
usage = result.usage;
} else {
const result = await runtime.stream('Look up four records.');
const chunks: StreamChunk[] = [];
for await (const chunk of result.stream) chunks.push(chunk);
expect(chunks.filter((chunk) => chunk.type === 'error')).toEqual([]);
usage = chunks.filter((chunk) => chunk.type === 'finish').at(-1)?.usage;
expect(chunks.some((chunk) => chunk.type === 'text-delta')).toBe(true);
}
expect(usage).toMatchObject({
promptTokens: 2000,
completionTokens: 20,
inputTokenDetails: { noCache: 1000, cacheRead: 400, cacheWrite: 600 },
});
expect(usage?.cost).toBeCloseTo(cost, 8);
expect(fetch).toHaveBeenCalledTimes(2);
const requests = fetch.mock.calls.map(([, init]) => JSON.parse(init?.body as string));
const cacheControl = { type: 'ephemeral', ttl };
for (const request of requests) {
expect(request).toMatchObject({
cache_control: cacheControl,
reasoning: { effort: 'high' },
messages: expect.arrayContaining([
expect.objectContaining({
role: 'system',
content: expect.arrayContaining([
expect.objectContaining({ cache_control: cacheControl }),
]),
}),
]),
});
expect(JSON.stringify(request).match(/"cache_control":/g)).toHaveLength(2);
}
expect(requests[1].messages).toEqual(
expect.arrayContaining([
expect.objectContaining({ role: 'assistant', reasoning_details: REASONING_DETAILS }),
...['a', 'b', 'c', 'd'].map((id) =>
expect.objectContaining({ role: 'tool', tool_call_id: id }),
),
]),
);
expect(fetch.mock.calls[0][0]).toBe('https://openrouter.ai/api/v1/chat/completions');
},
);
it.each([
{ details: { cached_tokens: 200 }, noCache: 800, cacheWrite: undefined },
{ details: { cached_tokens: 900, cache_write_tokens: 200 }, noCache: 0, cacheWrite: 200 },
])(
'handles incomplete or overlapping usage: $details',
async ({ details, noCache, cacheWrite }) => {
const fetch = vi
.fn<typeof globalThis.fetch>()
.mockResolvedValue(
response(mode === 'stream', false, { ...USAGE, prompt_tokens_details: details }),
);
const model = createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch);
const result =
mode === 'generate'
? await generateText({ model, prompt: 'Hello' })
: streamText({ model, prompt: 'Hello' });
expect((await result.usage).inputTokenDetails).toEqual({
noCacheTokens: noCache,
cacheReadTokens: details.cached_tokens,
cacheWriteTokens: cacheWrite,
});
},
);
it('preserves raw usage and provider errors', async () => {
const fetch = vi
.fn<typeof globalThis.fetch>()
.mockResolvedValueOnce(response(mode === 'stream'));
const model = createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch);
if (mode === 'generate') {
const result = await generateText({ model, prompt: 'Hello', maxRetries: 0 });
expect(result.steps[0].usage.raw).toEqual(USAGE);
expect(result.providerMetadata?.openrouter).toMatchObject({ provider: 'Anthropic' });
} else {
const result = streamText({ model, prompt: 'Hello', maxRetries: 0 });
await result.consumeStream();
expect((await result.steps)[0].usage.raw).toEqual(USAGE);
}
fetch.mockResolvedValueOnce(
Response.json({ error: { message: 'Provider unavailable', code: 503 } }, { status: 503 }),
);
if (mode === 'generate') {
await expect(generateText({ model, prompt: 'Hello', maxRetries: 0 })).rejects.toThrow(
'Provider unavailable',
);
} else {
const result = streamText({ model, prompt: 'Hello', maxRetries: 0 });
const chunks = [];
for await (const chunk of result.fullStream) chunks.push(chunk);
expect(chunks).toContainEqual({
type: 'error',
error: expect.objectContaining({ message: 'Provider unavailable' }),
});
}
});
});
describe('OpenRouter reasoning options', () => {
it.each([
'none',
'minimal',
'low',
'medium',
'high',
'xhigh',
'provider-default',
undefined,
] as const)('forwards %s through the adapter', async (reasoning) => {
const fetch = vi.fn<typeof globalThis.fetch>().mockResolvedValue(response(false));
await generateText({
model: createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch),
prompt: 'Hello',
reasoning,
});
const body = JSON.parse(fetch.mock.calls[0][1]?.body as string);
const expected =
reasoning && reasoning !== 'provider-default' ? { effort: reasoning } : undefined;
expect(body.reasoning).toEqual(expected);
expect(body.cache_control).toBeUndefined();
});
it('preserves an explicit reasoning budget and other caller options', async () => {
const fetch = vi.fn<typeof globalThis.fetch>().mockResolvedValue(response(false));
const providerOptions = {
openrouter: { reasoning: { max_tokens: 4096, exclude: false }, user: 'test-user' },
};
await generateText({
model: createModel({ id: MODEL_ID, apiKey: 'test-key' }, fetch),
prompt: 'Hello',
reasoning: 'high',
providerOptions,
});
expect(JSON.parse(fetch.mock.calls[0][1]?.body as string)).toMatchObject(
providerOptions.openrouter,
);
expect(providerOptions.openrouter.reasoning).not.toHaveProperty('effort');
});
});

View File

@ -217,12 +217,13 @@ describe('createOpenAIPromptCacheKey (via buildCallPromptCacheOptions)', () => {
});
describe('getEffectiveAnthropicCacheTtl', () => {
it.each(['anthropic/claude-sonnet-4-5', 'google-vertex/claude-opus-5@default'])(
'returns the default 1h TTL for %s with caching enabled',
(modelId) => {
expect(getEffectiveAnthropicCacheTtl({ enabled: true }, modelId)).toBe('1h');
},
);
it.each([
'anthropic/claude-sonnet-4-5',
'google-vertex/claude-opus-5@default',
'openrouter/anthropic/claude-sonnet-4.5',
])('returns the default 1h TTL for %s with caching enabled', (modelId) => {
expect(getEffectiveAnthropicCacheTtl({ enabled: true }, modelId)).toBe('1h');
});
it('returns an explicit 5m override for an Anthropic model', () => {
expect(

View File

@ -1,7 +1,6 @@
/* eslint-disable @typescript-eslint/consistent-type-imports */
/* eslint-disable @typescript-eslint/no-require-imports */
import type { GoogleVertexProviderSettings } from '@ai-sdk/google-vertex';
import { splitModelId } from '@n8n/ai-utilities/agent-config';
import { formatPemBlock } from '@n8n/utils/format-pem-block';
import { isRecord } from '@n8n/utils/is-record';
import type { LanguageModel } from 'ai';
@ -38,15 +37,6 @@ export function resolveGoogleVertexSdk(model: string, sdk?: string): string | un
return undefined;
}
/** Identify Claude models that use the shared Vertex provider ID. */
export function isGoogleVertexAnthropicModel(modelId: string): boolean {
const { provider, model } = splitModelId(modelId);
return (
provider === 'google-vertex' &&
resolveGoogleVertexSdk(model) === '@ai-sdk/google-vertex/anthropic'
);
}
/** Select the Vertex API for the model. */
export function createGoogleVertexModel(
creds: ProviderCredentials<'google-vertex'>,

View File

@ -1,14 +1,12 @@
/* eslint-disable @typescript-eslint/consistent-type-imports */
/* eslint-disable @typescript-eslint/no-require-imports */
import { isGoogleVertexAnthropicModel } from '@n8n/ai-utilities/agent-config';
import { ensureUrlPathSuffix, isOpenAiCustomEndpoint } from '@n8n/ai-utilities/model-discovery';
import type { EmbeddingModel, LanguageModel } from 'ai';
import type * as Undici from 'undici';
import {
createGoogleVertexModel,
isGoogleVertexAnthropicModel,
parseGoogleVertexAuthOptions,
} from './google-vertex';
import { createGoogleVertexModel, parseGoogleVertexAuthOptions } from './google-vertex';
import { wrapOpenRouterModel } from './openrouter';
import {
endpointRouteKey,
guardOpenAiRoutes,
@ -366,7 +364,9 @@ const LANGUAGE_PROVIDERS: ProviderRegistry = {
build: (creds, model, fetch) => {
const { createOpenRouter } =
require('@openrouter/ai-sdk-provider') as typeof import('@openrouter/ai-sdk-provider');
return createOpenRouter({ apiKey: creds.apiKey, baseURL: creds.baseURL, fetch })(model);
return wrapOpenRouterModel(
createOpenRouter({ apiKey: creds.apiKey, baseURL: creds.baseURL, fetch })(model),
);
},
},
nvidia: openAiCompatibleEntry('nvidia', 'https://integrate.api.nvidia.com/v1', {}),

View File

@ -0,0 +1,114 @@
import type { ProviderOptions } from '@ai-sdk/provider-utils';
import { isOpenRouterAnthropicModel } from '@n8n/ai-utilities/agent-config';
import { isRecord } from '@n8n/utils/is-record';
import type { ModelMessage, SystemModelMessage } from 'ai';
import { getEffectiveAnthropicCacheTtl, mergeProviderOptions } from './prompt-cache';
import type { PromptCachingConfig } from '../../types/sdk/agent';
type CacheControl = Record<string, unknown>;
type CacheTtl = '5m' | '1h';
const MAX_CACHE_BREAKPOINTS = 4;
/** Match the namespaces and key precedence in the OpenRouter chat adapter. */
function getCacheControl(options: ProviderOptions | undefined): CacheControl | undefined {
const control =
options?.openrouter?.cacheControl ??
options?.openrouter?.cache_control ??
options?.anthropic?.cacheControl ??
options?.anthropic?.cache_control;
return isRecord(control) ? control : undefined;
}
function getCacheTtl(control: CacheControl): CacheTtl {
return control.ttl === '1h' ? '1h' : '5m';
}
/** Count wire breakpoints, including the marker copied to each tool result. */
function getMessageCacheControls(message: ModelMessage): Array<CacheControl | undefined> {
const control = getCacheControl(message.providerOptions);
if (message.role === 'system' || message.role === 'assistant') return [control];
if (message.role === 'tool') {
return message.content
.filter((part) => part.type === 'tool-result')
.map((part) => control ?? getCacheControl(part.providerOptions));
}
if (typeof message.content === 'string') return [control];
if (message.content.length === 1 && message.content[0].type === 'text') {
return [control ?? getCacheControl(message.content[0].providerOptions)];
}
const lastTextIndex = message.content.map((part) => part.type).lastIndexOf('text');
return message.content.map((part, index) => {
const partControl = getCacheControl(part.providerOptions);
return partControl ?? (index === lastTextIndex ? control : undefined);
});
}
function cacheOptions(ttl: CacheTtl): ProviderOptions {
return { openrouter: { cacheControl: { type: 'ephemeral', ttl } } };
}
/** Add request-local caching after skills and conversation history are assembled. */
export function applyOpenRouterPromptCache(params: {
system: SystemModelMessage | SystemModelMessage[];
messages: ModelMessage[];
providerOptions: ProviderOptions | undefined;
promptCaching: PromptCachingConfig | undefined;
modelId: string;
}): {
system: SystemModelMessage | SystemModelMessage[];
providerOptions: ProviderOptions | undefined;
} {
const { system, messages, providerOptions, promptCaching, modelId } = params;
const ttl = getEffectiveAnthropicCacheTtl(promptCaching, modelId);
if (!isOpenRouterAnthropicModel(modelId) || !ttl) return { system, providerOptions };
const systemMessages = Array.isArray(system) ? system : [system];
const controls = [...systemMessages, ...messages].flatMap(getMessageCacheControls);
const callerControls = controls.filter((control) => control !== undefined);
const openrouter = providerOptions?.openrouter;
const hasCallerAutomaticControl =
openrouter !== undefined && ('cache_control' in openrouter || 'cacheControl' in openrouter);
const automaticControl =
openrouter && 'cache_control' in openrouter
? openrouter.cache_control
: openrouter?.cacheControl;
let used = callerControls.length + (isRecord(automaticControl) ? 1 : 0);
// Caller controls take priority, followed by stable instructions and automatic caching.
const instructionTtl = isRecord(automaticControl) ? getCacheTtl(automaticControl) : ttl;
const firstSystem = systemMessages[0];
const canCacheInstructions =
firstSystem?.content.trim() &&
!getCacheControl(firstSystem.providerOptions) &&
used < MAX_CACHE_BREAKPOINTS &&
(instructionTtl === '1h' || callerControls.every((control) => getCacheTtl(control) === '5m'));
let nextSystem = system;
if (canCacheInstructions) {
const cachedInstruction = {
...firstSystem,
providerOptions: mergeProviderOptions(
firstSystem.providerOptions,
cacheOptions(instructionTtl),
),
};
nextSystem = Array.isArray(system)
? [cachedInstruction, ...system.slice(1)]
: cachedInstruction;
used++;
}
// A longer TTL must precede shorter TTLs. A marked final block can also conflict with auto caching.
if (
hasCallerAutomaticControl ||
used >= MAX_CACHE_BREAKPOINTS ||
controls.at(-1) ||
(ttl === '1h' && callerControls.some((control) => getCacheTtl(control) === '5m'))
) {
return { system: nextSystem, providerOptions };
}
return {
system: nextSystem,
providerOptions: mergeProviderOptions(cacheOptions(ttl), providerOptions),
};
}

View File

@ -0,0 +1,66 @@
import type * as AiSdk from 'ai';
import { loadAi } from './lazy-ai';
import { mergeProviderOptions } from './prompt-cache';
type MiddlewareGenerate = NonNullable<AiSdk.LanguageModelMiddleware['wrapGenerate']>;
type CallOptions = Parameters<MiddlewareGenerate>[0]['params'];
type ModelUsage = Awaited<ReturnType<MiddlewareGenerate>>['usage'];
type StreamResult = Awaited<ReturnType<NonNullable<AiSdk.LanguageModelMiddleware['wrapStream']>>>;
type StreamPart = StreamResult['stream'] extends ReadableStream<infer Part> ? Part : never;
function applyReasoning(params: CallOptions): CallOptions {
if (!params.reasoning || params.reasoning === 'provider-default') return params;
// Explicit options can use a token budget, which must not be combined with effort.
if (params.providerOptions?.openrouter?.reasoning !== undefined) return params;
return {
...params,
providerOptions: mergeProviderOptions(params.providerOptions, {
openrouter: { reasoning: { effort: params.reasoning } },
}),
};
}
function normalizeUsage(usage: ModelUsage): ModelUsage {
const { total, cacheRead, cacheWrite } = usage.inputTokens;
if (total === undefined || cacheWrite === undefined) return usage;
// The adapter includes cache writes in noCache. Price each token only once.
return {
...usage,
inputTokens: {
...usage.inputTokens,
noCache: Math.max(total - (cacheRead ?? 0) - cacheWrite, 0),
},
};
}
/** Adapt generic reasoning and cache usage for the OpenRouter chat adapter. */
export function wrapOpenRouterModel(model: Parameters<typeof AiSdk.wrapLanguageModel>[0]['model']) {
return loadAi().wrapLanguageModel({
model,
middleware: {
transformParams: async ({ params }) => applyReasoning(params),
wrapGenerate: async ({ doGenerate }) => {
const result = await doGenerate();
return { ...result, usage: normalizeUsage(result.usage) };
},
wrapStream: async ({ doStream }) => {
const result = await doStream();
return {
...result,
stream: result.stream.pipeThrough(
new TransformStream<StreamPart, StreamPart>({
transform(chunk, controller) {
if (chunk.type === 'finish') {
controller.enqueue({ ...chunk, usage: normalizeUsage(chunk.usage) });
} else {
controller.enqueue(chunk);
}
},
}),
),
};
},
},
});
}

View File

@ -1,10 +1,13 @@
import type { ProviderOptions } from '@ai-sdk/provider-utils';
import { getProviderPrefix } from '@n8n/ai-utilities/agent-config';
import {
getModelPromptCachingCapability,
getProviderPrefix,
isGoogleVertexAnthropicModel,
} from '@n8n/ai-utilities/agent-config';
import { isRecord } from '@n8n/utils/is-record';
import type { ModelMessage, SystemModelMessage, ToolSet } from 'ai';
import { createHash } from 'crypto';
import { isGoogleVertexAnthropicModel } from './google-vertex';
import type { PromptCachingConfig } from '../../types/sdk/agent';
import type { JSONObject } from '../../types/utils/json';
@ -42,7 +45,10 @@ export function getEffectiveAnthropicCacheTtl(
config: PromptCachingConfig | undefined,
modelId: string,
): '5m' | '1h' | undefined {
if (!isAnthropicMessagesProvider(modelId) || !isEnabledForProvider(config, 'anthropic')) {
if (
getModelPromptCachingCapability(modelId) !== 'ttl' ||
!isEnabledForProvider(config, 'anthropic')
) {
return undefined;
}
return getAnthropicCacheTtl(config);

View File

@ -1,7 +1,6 @@
import { getProviderPrefix } from '@n8n/ai-utilities/agent-config';
import { getProviderPrefix, isGoogleVertexAnthropicModel } from '@n8n/ai-utilities/agent-config';
import { isRecord } from '@n8n/utils/is-record';
import { isGoogleVertexAnthropicModel } from './google-vertex';
import { type ProviderId } from './provider-credentials';
import type { TokenUsage } from '../../types';

View File

@ -521,8 +521,8 @@ export class Agent implements BuiltAgent, AgentBuilder {
/**
* Enable prompt caching with defaults tuned for agent workloads. Anthropic
* models get a `1h` instruction-level cache breakpoint; OpenAI models get
* `24h` retention plus an auto-generated, per-agent-version `promptCacheKey`.
* models, including Claude on OpenRouter and Vertex, get a `1h` instruction breakpoint.
* OpenAI gets `24h` retention plus an auto-generated, per-agent-version `promptCacheKey`.
* Pass `{ anthropic: false }` / `{ openai: false }` to disable per provider,
* or override individual fields (e.g. `{ anthropic: { ttl: '5m' } }`).
*

View File

@ -325,7 +325,7 @@ export interface OpenAIPromptCachingConfig {
export interface PromptCachingConfig {
/** Master switch. Defaults to `true` when this config is set. */
enabled?: boolean;
/** Anthropic-specific config, or `false` to disable for Anthropic models. */
/** Claude cache settings, including OpenRouter and Vertex. Set `false` to disable. */
anthropic?: false | AnthropicPromptCachingConfig;
/** OpenAI-specific config, or `false` to disable for OpenAI models. */
openai?: false | OpenAIPromptCachingConfig;

View File

@ -6,6 +6,7 @@ model configuration, and model output.
## Public subpaths
- `agent-config`
- `agent-config/model-capabilities` (browser-safe model capability checks)
- `fromai-helpers`
- `generic-text-editor`
- `http-proxy-agent`

View File

@ -22,6 +22,9 @@
"agent-config": [
"dist/esm/agent-config/index.d.ts"
],
"agent-config/model-capabilities": [
"dist/esm/agent-config/model-capabilities.d.ts"
],
"fromai-helpers": [
"dist/esm/utils/fromai-helpers.d.ts"
],
@ -76,6 +79,11 @@
"import": "./dist/esm/agent-config/index.js",
"require": "./dist/cjs/agent-config/index.js"
},
"./agent-config/model-capabilities": {
"types": "./dist/esm/agent-config/model-capabilities.d.ts",
"import": "./dist/esm/agent-config/model-capabilities.js",
"require": "./dist/cjs/agent-config/model-capabilities.js"
},
"./fromai-helpers": {
"types": "./dist/esm/utils/fromai-helpers.d.ts",
"import": "./dist/esm/utils/fromai-helpers.js",

View File

@ -0,0 +1,22 @@
import { getModelPromptCachingCapability } from '../model-capabilities';
describe('getModelPromptCachingCapability', () => {
it.each<[string, 'ttl' | boolean]>([
['anthropic/claude-sonnet-4-5', 'ttl'],
['google-vertex-anthropic/claude-sonnet-4-5', 'ttl'],
['google-vertex/claude-opus-5@default', 'ttl'],
['openrouter/anthropic/claude-sonnet-4.5', 'ttl'],
['openrouter/anthropic/claude-sonnet-4.5:beta', 'ttl'],
['openai/gpt-5', true],
['google-vertex/gemini-2.5-pro', false],
['google-vertex/meta/llama-4-maas', false],
['openrouter/google/gemini-2.5-pro', false],
['openrouter/openai/gpt-5', false],
['openrouter/claude-sonnet-4.5', false],
['custom/anthropic/claude-sonnet-4.5', false],
['claude-sonnet-4-5', false],
['', false],
])('resolves %s to %s', (modelId, expected) => {
expect(getModelPromptCachingCapability(modelId)).toBe(expected);
});
});

View File

@ -1,4 +1,10 @@
export { getProviderPrefix, splitModelId } from './model-id';
export {
getModelPromptCachingCapability,
isGoogleVertexAnthropicModel,
isOpenRouterAnthropicModel,
isRoutedAnthropicModel,
} from './model-capabilities';
export {
getNativeWebSearchProviderTools,
hasNativeWebSearchProvider,

View File

@ -0,0 +1,25 @@
import { PROVIDER_CAPABILITIES, type ProviderCapabilities } from '@n8n/api-types';
import { getProviderPrefix, splitModelId } from './model-id';
export function isGoogleVertexAnthropicModel(modelId: string): boolean {
const { provider, model } = splitModelId(modelId);
return provider === 'google-vertex' && model.startsWith('claude-');
}
export function isOpenRouterAnthropicModel(modelId: string): boolean {
const { provider, model } = splitModelId(modelId);
return provider === 'openrouter' && model.startsWith('anthropic/claude-');
}
export function isRoutedAnthropicModel(modelId: string): boolean {
return isGoogleVertexAnthropicModel(modelId) || isOpenRouterAnthropicModel(modelId);
}
/** Cache support depends on the model when the provider routes to several vendors. */
export function getModelPromptCachingCapability(
modelId: string,
): ProviderCapabilities['promptCaching'] {
if (isRoutedAnthropicModel(modelId)) return 'ttl';
return PROVIDER_CAPABILITIES[getProviderPrefix(modelId)]?.promptCaching ?? false;
}

View File

@ -1436,9 +1436,13 @@ describe('AgentsBuilderToolsService', () => {
});
describe('prompt caching defaults', () => {
it('write_config defaults prompt caching to enabled for a supported provider when omitted', async () => {
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/anthropic/claude-sonnet-4.5',
'google-vertex/claude-sonnet-4-5',
])('write_config enables caching for %s when omitted', async (model) => {
const { service, agentsService } = makeService();
const currentConfig = { ...baseConfig, integrations: [] };
const currentConfig = { ...baseConfig, model, integrations: [] };
// Explicitly disable web search so its own write-path normalizer
// doesn't add unrelated config/providerTools keys to the expectation.
const updatedConfig: AgentJsonConfig = {
@ -1449,7 +1453,7 @@ describe('AgentsBuilderToolsService', () => {
...currentConfig,
config: { webSearch: { enabled: false }, promptCaching: { enabled: true } },
};
agentsService.findById.mockResolvedValue(makeAgent(baseConfig));
agentsService.findById.mockResolvedValue(makeAgent(currentConfig));
agentsService.updateConfig.mockResolvedValue({
config: normalizedConfig,
configHash: 'config-hash',
@ -1477,7 +1481,11 @@ describe('AgentsBuilderToolsService', () => {
);
});
it('write_config strips prompt caching when switching to an unsupported provider', async () => {
it.each([
'google/gemini-2.5-flash',
'openrouter/openai/gpt-5',
'google-vertex/gemini-2.5-pro',
])('write_config strips caching when switching to %s', async (model) => {
const { service, agentsService } = makeService();
const baseAgent = {
...baseConfig,
@ -1487,7 +1495,7 @@ describe('AgentsBuilderToolsService', () => {
const currentConfig = { ...baseAgent };
const updatedConfig: AgentJsonConfig = {
...currentConfig,
model: 'google/gemini-2.5-flash',
model,
credential: 'Google Key',
};
const { config: _droppedConfig, ...normalizedConfig } = updatedConfig;
@ -1569,10 +1577,15 @@ describe('AgentsBuilderToolsService', () => {
);
});
it('write_config preserves an explicit Anthropic ttl', async () => {
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/anthropic/claude-sonnet-4.5',
'google-vertex/claude-sonnet-4-5',
])('write_config preserves the cache TTL for %s', async (model) => {
const { service, agentsService } = makeService();
const baseAgent = {
...baseConfig,
model,
integrations: [],
config: { promptCaching: { enabled: true, anthropic: { ttl: '5m' as const } } },
};

View File

@ -75,10 +75,28 @@ describe('buildFromJson()', () => {
executeToMessage: vi.fn().mockResolvedValue(undefined),
});
const makeMockCredentialProvider = () => ({
resolve: vi.fn().mockResolvedValue({ apiKey: 'test-api-key' }),
list: vi.fn().mockResolvedValue([]),
});
const makeMockCredentialProvider = (model = '') => {
const isVertex = model.startsWith('google-vertex/');
return {
resolve: vi.fn().mockResolvedValue(
isVertex
? {
projectId: 'test-project',
region: 'global',
email: 'test@example.com',
privateKey: 'test-key',
}
: { apiKey: 'test-api-key' },
),
list: vi
.fn()
.mockResolvedValue(
isVertex
? [{ id: 'my-anthropic-key', name: 'Test Vertex', type: 'googleVertexAiApi' }]
: [],
),
};
};
const makeToolDescriptor = (overrides: Partial<ToolDescriptor> = {}): ToolDescriptor => ({
name: 'search',
@ -666,8 +684,13 @@ describe('buildFromJson()', () => {
expect(snap.reasoning).toBe('high');
});
it('sets prompt caching config with an Anthropic ttl', async () => {
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/anthropic/claude-sonnet-4.5',
'google-vertex/claude-sonnet-4-5',
])('preserves the saved cache TTL for %s', async (model) => {
const config = makeConfig({
model,
config: { promptCaching: { enabled: true, anthropic: { ttl: '1h' } } },
});
@ -676,7 +699,7 @@ describe('buildFromJson()', () => {
{},
{
toolExecutor: makeMockToolExecutor(),
credentialProvider: makeMockCredentialProvider(),
credentialProvider: makeMockCredentialProvider(model),
memoryFactory: makeMockMemoryFactory(),
},
);
@ -704,8 +727,13 @@ describe('buildFromJson()', () => {
expect(snap.promptCaching).toEqual({ enabled: true });
});
it('passes through promptCaching verbatim even if enabled=false was somehow saved (mapping is not a policy enforcer)', async () => {
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/anthropic/claude-sonnet-4.5',
'google-vertex/claude-sonnet-4-5',
])('preserves disabled caching for %s', async (model) => {
const config = makeConfig({
model,
config: { promptCaching: { enabled: false } },
});
@ -714,7 +742,7 @@ describe('buildFromJson()', () => {
{},
{
toolExecutor: makeMockToolExecutor(),
credentialProvider: makeMockCredentialProvider(),
credentialProvider: makeMockCredentialProvider(model),
memoryFactory: makeMockMemoryFactory(),
},
);
@ -723,15 +751,19 @@ describe('buildFromJson()', () => {
expect(snap.promptCaching).toEqual({ enabled: false });
});
it('does not set prompt caching config when omitted', async () => {
const config = makeConfig();
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/openai/gpt-5',
'google-vertex/gemini-2.5-pro',
])('keeps the runtime cache default for %s when config is omitted', async (model) => {
const config = makeConfig({ model });
const agent = await buildFromJson(
config,
{},
{
toolExecutor: makeMockToolExecutor(),
credentialProvider: makeMockCredentialProvider(),
credentialProvider: makeMockCredentialProvider(model),
memoryFactory: makeMockMemoryFactory(),
},
);
@ -739,6 +771,24 @@ describe('buildFromJson()', () => {
expect(agent.snapshot.promptCaching).toBeNull();
});
it.each(['openrouter/anthropic/claude-sonnet-4.5', 'google-vertex/claude-sonnet-4-5'])(
'enables caching for an existing %s agent without a config update',
async (model) => {
const config = makeConfig({ model });
const agent = await buildFromJson(
config,
{},
{
toolExecutor: makeMockToolExecutor(),
credentialProvider: makeMockCredentialProvider(model),
},
);
expect(agent.snapshot.promptCaching).toEqual({ enabled: true });
expect(config.config).toBeUndefined();
},
);
it('builds native provider tools for an inline child model, not the parent model', () => {
const config = makeConfig({
model: 'openai/gpt-4o',

View File

@ -14,7 +14,7 @@ import {
} from '@n8n/agents/tool';
import {
applyNativeWebSearchDefaultOn,
getProviderPrefix,
getModelPromptCachingCapability,
rejectIfDynamicSelectorUsesFromAi,
rejectIfEmptyInstructions,
rejectIfUnsupportedNativeWebSearch,
@ -24,7 +24,6 @@ import {
agentSkillSchema,
agentTaskSchema,
formatZodErrors,
PROVIDER_CAPABILITIES,
resolvePromptCaching,
AgentJsonConfigSchema,
isDraftAgentConfig,
@ -351,17 +350,9 @@ function parseBuilderWriteConfig(incoming: unknown, currentConfig: AgentJsonConf
return AgentJsonConfigSchema.safeParse(sanitized);
}
/**
* Prompt caching is mandatory for OpenAI/Anthropic: this write-path
* normalizer guarantees `config.promptCaching` is force-enabled for those
* providers (the user cannot disable it, even if the LLM wrote
* `{ enabled: false }`), preserves an explicit Anthropic TTL, and strips the
* field entirely for every other provider — regardless of what the builder
* LLM wrote.
*/
/** Enable caching for supported models and preserve an explicit cache TTL. */
function applyPromptCachingBuilderDefaults(config: AgentJsonConfig): AgentJsonConfig {
const providerPrefix = getProviderPrefix(config.model);
const capability = PROVIDER_CAPABILITIES[providerPrefix]?.promptCaching ?? false;
const capability = getModelPromptCachingCapability(config.model);
const resolved = resolvePromptCaching(config.config?.promptCaching, capability);
if (!resolved) {

View File

@ -52,10 +52,10 @@ export function getConfigRulesSection(): string {
- Sub-agent configuration lives at top level under \`subAgents\`. Load
\`agent-builder-sub-agents\` before adding refs or changing
\`subAgents.maxChildren\`.
- For OpenAI and Anthropic models, always include
- For OpenAI, Anthropic, and Claude models on OpenRouter or Google Vertex, always include
\`config.promptCaching: { "enabled": true }\` — this is mandatory and must
never be disabled, even if the user asks. Omit \`config.promptCaching\`
entirely for every other provider. For Anthropic only, you may set
entirely for other models. For Anthropic and routed Claude models, you may set
\`config.promptCaching.anthropic.ttl\` to \`"5m"\` or \`"1h"\` (default
\`"1h"\`) when the user asks to tune cache duration; OpenAI has no
sub-config.

View File

@ -22,6 +22,7 @@ import {
getProviderPrefix,
hasNativeWebSearchProvider,
isNativeWebSearchRequested,
isRoutedAnthropicModel,
} from '@n8n/ai-utilities/agent-config';
import {
AI_GATEWAY_MANAGED_TAG,
@ -251,9 +252,6 @@ export async function buildFromJson(
if (config.config.reasoning) {
agent.reasoning(config.config.reasoning);
}
if (config.config.promptCaching) {
agent.promptCaching(config.config.promptCaching);
}
if (config.config.toolCallConcurrency) {
agent.toolCallConcurrency(config.config.toolCallConcurrency);
}
@ -262,6 +260,13 @@ export async function buildFromJson(
}
}
if (config.config?.promptCaching) {
agent.promptCaching(config.config.promptCaching);
} else if (isRoutedAnthropicModel(config.model)) {
// Existing agents receive the cache default without a config update.
agent.promptCaching({ enabled: true });
}
return agent;
}

View File

@ -42,6 +42,7 @@
"@internationalized/date": "catalog:frontend",
"@lezer/common": "catalog:",
"@lezer/highlight": "catalog:",
"@n8n/ai-utilities": "workspace:*",
"@n8n/api-types": "workspace:*",
"@n8n/chat": "workspace:*",
"@n8n/chat-hub": "workspace:*",

View File

@ -487,8 +487,12 @@ describe('AgentAdvancedPanel', () => {
expect(wrapper.emitted('update:config')).toBeUndefined();
});
it('shows the Anthropic ttl dropdown, defaulting to 1h, with no on/off toggle', async () => {
const config = makeConfig();
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/anthropic/claude-sonnet-4.5',
'google-vertex/claude-sonnet-4-5',
])('shows a default 1h cache duration for %s', async (model) => {
const config = makeConfig({ model });
const wrapper = mount(AgentAdvancedPanel, {
props: { config },
global: { stubs: globalStubs },
@ -510,8 +514,12 @@ describe('AgentAdvancedPanel', () => {
expect(wrapper.find('[data-testid="agent-prompt-caching-ttl-select"]').exists()).toBe(false);
});
it('hides the prompt-caching row entirely for providers that do not support it', () => {
const config = makeConfig({ model: 'google/gemini-pro' });
it.each([
'google/gemini-pro',
'openrouter/google/gemini-2.5-pro',
'google-vertex/gemini-2.5-pro',
])('hides the cache duration for %s', (model) => {
const config = makeConfig({ model });
const wrapper = mount(AgentAdvancedPanel, {
props: { config },
global: { stubs: globalStubs },
@ -520,8 +528,12 @@ describe('AgentAdvancedPanel', () => {
expect(wrapper.find('[data-testid="agent-prompt-caching-ttl-select"]').exists()).toBe(false);
});
it('emits { enabled: true, anthropic: { ttl } } when the ttl dropdown changes', () => {
const config = makeConfig();
it.each([
'anthropic/claude-sonnet-4-5',
'openrouter/anthropic/claude-sonnet-4.5',
'google-vertex/claude-sonnet-4-5',
])('saves the selected cache duration for %s', (model) => {
const config = makeConfig({ model });
const wrapper = mount(AgentAdvancedPanel, {
props: { config },
global: { stubs: globalStubs },

View File

@ -331,6 +331,37 @@ describe('AgentInfoPanel', () => {
expect(events.at(-1)?.[1]).toBeUndefined();
});
it.each([
{ provider: 'openrouter', model: 'anthropic/claude-sonnet-4.5', supported: true },
{ provider: 'openrouter', model: 'openai/gpt-5', supported: false },
{ provider: 'google-vertex', model: 'claude-sonnet-4-5', supported: true },
{ provider: 'google-vertex', model: 'gemini-2.5-pro', supported: false },
])(
'normalizes caching when selecting $provider/$model',
async ({ provider, model, supported }) => {
credsHolder.value[provider] = 'credential-2';
const wrapper = mountModelPanel({
name: 'Support agent',
model: 'anthropic/claude-sonnet-4-5',
credential: 'credential-1',
instructions: 'Help users.',
config: {
toolCallConcurrency: 2,
promptCaching: { enabled: true, anthropic: { ttl: '5m' } },
},
});
wrapper.findComponent({ name: 'AgentModelSelector' }).vm.$emit('change', { provider, model });
await wrapper.vm.$nextTick();
const changes = wrapper.emitted('update:config')?.at(-1)?.[0] as Partial<AgentJsonConfig>;
expect(changes.model).toBe(`${provider}/${model}`);
expect(changes.config?.toolCallConcurrency).toBe(2);
expect(changes.config?.promptCaching).toEqual(
supported ? { enabled: true, anthropic: { ttl: '5m' } } : undefined,
);
},
);
it('forwards the model selector\'s own "auto" source through to update:config', async () => {
const config: AgentJsonConfig = {
name: 'Support agent',

View File

@ -5,26 +5,18 @@ import { useDebounceFn } from '@vueuse/core';
import { AGENT_REASONING_LEVELS, type AgentReasoningLevel } from '@n8n/api-types';
import { N8nInputNumber, N8nOption, N8nSelect, N8nSwitch2, N8nText } from '@n8n/design-system';
import { useI18n } from '@n8n/i18n';
import { getModelPromptCachingCapability } from '@n8n/ai-utilities/agent-config/model-capabilities';
import AgentMemoryModelSetting from './AgentMemoryModelSetting.vue';
import AgentPanel from './AgentPanel.vue';
import { useModelCatalog } from '../composables/useModelCatalog';
import type { AgentJsonConfig } from '../types';
import {
PROVIDER_CAPABILITIES,
ANTHROPIC_CACHE_TTL_OPTIONS,
type AnthropicCacheTtl,
} from '../provider-capabilities';
import { modelToString, parseModelString, parseProvider } from '../utils/model-string';
import { ANTHROPIC_CACHE_TTL_OPTIONS, type AnthropicCacheTtl } from '../provider-capabilities';
import { modelToString, parseModelString } from '../utils/model-string';
import shared from '../styles/agent-panel.module.scss';
const i18n = useI18n();
const { catalog, ensureLoaded } = useModelCatalog();
const DEFAULT_CAPABILITIES = {
promptCaching: false,
webSearch: false,
providerTools: [],
} as const;
function normalizeReasoningLevel(value: unknown): AgentReasoningLevel {
return AGENT_REASONING_LEVELS.find((level) => level === value) ?? 'medium';
@ -42,7 +34,6 @@ const props = withDefaults(
);
const emit = defineEmits<{ 'update:config': [changes: Partial<AgentJsonConfig>] }>();
const provider = computed(() => parseProvider(props.config?.model));
const selectedModel = computed(() => parseModelString(modelToString(props.config?.model)));
const selectedCatalogModel = computed(() => {
if (!selectedModel.value) return undefined;
@ -58,7 +49,9 @@ const reasoningHintKey = computed(() => {
}
return 'agents.builder.advanced.reasoning.hint';
});
const capabilities = computed(() => PROVIDER_CAPABILITIES[provider.value] ?? DEFAULT_CAPABILITIES);
const promptCachingCapability = computed(() =>
getModelPromptCachingCapability(modelToString(props.config?.model)),
);
watch(
() => props.projectId,
@ -298,7 +291,7 @@ function onAnthropicTtlChange(value: AnthropicCacheTtl) {
</div>
</div>
<div v-if="capabilities.promptCaching === 'ttl'" :class="$style.settingGroup">
<div v-if="promptCachingCapability === 'ttl'" :class="$style.settingGroup">
<div :class="shared.settingRow">
<div :class="shared.settingLabel">
<N8nText step="sm" bold :class="shared.dataEntryLabel">{{

View File

@ -15,6 +15,7 @@ import {
N8nText,
} from '@n8n/design-system';
import { useI18n } from '@n8n/i18n';
import { getModelPromptCachingCapability } from '@n8n/ai-utilities/agent-config/model-capabilities';
import { getDebounceTime } from '@n8n/composables/useDebounce';
import { DEBOUNCE_TIME } from '@/app/constants/durations';
@ -257,7 +258,7 @@ function onModelChange(selection: AgentModelSelection, source: 'user' | 'auto' =
'config' in webSearchChanges ? webSearchChanges.config : props.config?.config;
const promptCachingChanges = normalizePromptCachingForModelChange(
webSearchConfig,
capabilities?.promptCaching ?? false,
getModelPromptCachingCapability(model),
);
const normalizedConfig =
'config' in promptCachingChanges ? promptCachingChanges.config : webSearchConfig;

View File

@ -6583,6 +6583,9 @@ importers:
'@lezer/highlight':
specifier: 'catalog:'
version: 1.2.3(patch_hash=97f85e6fe46f23015ea0dd420e33d584bc2dc71633910cf131321da31b27ca8c)
'@n8n/ai-utilities':
specifier: workspace:*
version: link:../../@n8n/ai-utilities
'@n8n/api-types':
specifier: workspace:*
version: link:../../@n8n/api-types