30b4d7946c
Three pieces so the surfaces built earlier can actually be judged. The `instance-awareness` skill teaches the ladder rather than the tools: ask about intent, look up state; read cheapest-first and stop at the first rung that answers the question; treat an absence in the node-usage histogram as evidence rather than as a gap. It also says where each surface stops — the log records that a save happened and the node count changed, never which nodes, so anything about contents is a workflow read. Four eval cases on a new `activity` tier, which also carries `memory` so they run under both. Two are authored from real reviewed traces rather than invented: lang-tracer observation 3870 and its siblings, where the agent restarted work it had just finished and re-elicited requirements it already had. Their A/B discriminator is a probe-anchored context assertion — with the feed off nothing carries the prior workflow into the turn, so it can only arrive via a tool call, which is turn-end and not probe. That distinction is the measurement. Every case is scored on behaviour, never on narration. A probe that asks the agent to describe its own context manufactures its own evidence, which is the restatement confound recorded on this branch. So the checks ask whether it re-elicited a known requirement, whether it named the broken workflow, whether it picked the house node — not whether it can recite the block. The fourth case is the negative control, and the one that should stop a rollout: loud recent activity plus an unrelated request. It deliberately does not assert that the recent workflow is absent from context — it will be there, the feed put it there. What is scored is whether the agent acted on it. Having a fact and using it are different, and that difference is the whole point. scripts/ab-activity-eval.sh drives both arms. They differ by a flag on the instance under test, not by a runner argument, so each arm needs its own start — the flag is read once when the relay registers. The script says what to read and in what order, with context-ignored first. Also opens the activity preamble up. It forbade narrating the feed but never granted permission to act on it, which left the agent with a list of work it was implicitly told to ignore. It now says this is work you can pick up, that a vague opener usually means the most recent thing, and points at the skill. Two honest limits recorded in the README. `node-usage` is not gated by the flag, so the preference case measures a capability both arms have and a difference there is noise. And that case has no historical precedent: a sweep of 16,262 traces found no conversation where a user stated a node preference their own workflows already showed. |
||
|---|---|---|
| .. | ||
| docs | ||
| evaluations | ||
| knowledge-base/reference | ||
| scripts | ||
| skills | ||
| src | ||
| .gitignore | ||
| CLAUDE.md | ||
| eslint.config.mjs | ||
| eval-run.log | ||
| eval-smoke.log | ||
| package.json | ||
| README.md | ||
| tsconfig.build.json | ||
| tsconfig.json | ||
| vite.config.ts | ||
@n8n/instance-ai
Instance AI is the agent runtime behind the AI assistant experience in n8n. It lets users ask for help with workflows, executions, credentials, nodes, and workflow building from inside an n8n instance.
The package contains the agent prompts, tool registry, workflow-builder logic,
workspace adapters, tracing helpers, and evaluation harnesses. The HTTP API,
database entities, settings, and n8n service adapters live in
packages/cli/src/modules/instance-ai.
What It Does
Instance AI is built around a deep-agent loop:
- An orchestrator agent receives the user's request and maintains the plan.
- Most work runs in the orchestrator; eval setup uses a focused background agent.
- Domain tools read and update n8n resources through backend adapters.
- Observational memory condenses long conversations.
- Workflow building runs in a sandbox workspace, validates generated TypeScript, and submits the workflow through the n8n backend.
The workflow builder requires sandboxing. The default provider is the n8n sandbox service. Daytona remains an explicit provider for environments that still need it.
Running Locally
Instance AI is a backend module, so run it through n8n rather than this package directly.
1. Start the n8n Sandbox Service
From the repo root:
TESTCONTAINERS_REUSE_ENABLE=true pnpm --filter n8n-containers services \
--services sandbox \
--network n8n-instance-ai-dev \
--name n8n-svc-sandbox
This starts the sandbox API and runner containers and writes the host-reachable
sandbox environment variables to packages/cli/bin/.env. You can verify the
service from any terminal:
SANDBOX_PORT=$(docker port n8n-svc-sandbox-sandbox-api 8080/tcp | sed 's/.*://')
curl "http://localhost:${SANDBOX_PORT}/healthz"
Expected response:
{"status":"ok"}
2. Start n8n With Instance AI Enabled
In a second terminal:
export N8N_ENABLED_MODULES=instance-ai
export N8N_AI_ENABLED=true
export N8N_INSTANCE_AI_MODEL=anthropic/claude-sonnet-4-5
export N8N_INSTANCE_AI_MODEL_API_KEY="$ANTHROPIC_API_KEY"
export N8N_INSTANCE_AI_SANDBOX_ENABLED=true
pnpm dev:ai
The pnpm --filter n8n-containers services command writes
N8N_INSTANCE_AI_SANDBOX_PROVIDER, N8N_SANDBOX_SERVICE_URL, and
N8N_SANDBOX_SERVICE_API_KEY to packages/cli/bin/.env. If you are not using
that generated .env file, export them manually:
export N8N_SANDBOX_PORT=$(docker port n8n-svc-sandbox-sandbox-api 8080/tcp | sed 's/.*://')
export N8N_INSTANCE_AI_SANDBOX_PROVIDER=n8n-sandbox
export N8N_SANDBOX_SERVICE_URL="http://localhost:${N8N_SANDBOX_PORT}"
export N8N_SANDBOX_SERVICE_API_KEY=n8n-sandbox-ci-key
For n8n running inside the same Docker network as the sandbox service, use the internal service URL instead:
N8N_SANDBOX_SERVICE_URL=http://sandbox-api:8080
3. Cleanup
pnpm --filter n8n-containers services:clean
docker network rm n8n-instance-ai-dev 2>/dev/null || true
Useful Commands
Run focused tests:
pnpm --filter @n8n/instance-ai test
Print agent prompts:
pnpm --filter @n8n/instance-ai prompts:print
Run evaluations:
pnpm --filter @n8n/instance-ai eval:instance-ai