390b697cd7
Eval cases had two grading surfaces: `processExpectations`, judged on the build conversation, and `outcomeExpectations`, judged on the built workflow. Neither observes the memory subsystem, so a memory comparison could only be measured end-to-end — a score delta was not attributable to retrieval rather than to a luckier build. Add a third kind, `memoryExpectations`, graded against context state the harness already captured and discarded: run debug is stashed on every build, and `parseSystemPromptForDisplay` / `parseUsageSummary` already parse it. No product change is needed. It gets its own judge call with its own rubric, whose only inputs are the final compressed observation block, the raw message window the model received, and the final system prompt. The conversation judge is handed the full transcript, which would let it satisfy "the agent still knew X" from the turn where X was first said — the confound this kind exists to remove. The raw window is not the transcript: evicted messages are absent from it, which is why it can be trusted as a record of what the agent still had. A fact counts as retained if it appears in any of the three tiers. The Observer only compresses past a token threshold, so on a short thread the observation block is legitimately empty while every fact sits in the raw window; grading on the block alone would fail every memory expectation for a structural reason on exactly the short threads the suite is mostly made of. Verdicts carry `kind: "memory"` on every path — judged, ungraded and judge-error — since the report badge and `eval-results.json` filter on it. `stashBuildExpectations` previously early-returned once any expectation was ungraded. With three kinds a case can legitimately have ungraded memory expectations and gradable outcome ones at once, so the verdict sets are now merged; the early return would have dropped the latter and shrunk the pass-rate denominator. `stashRunDebug` now runs just before it at all three call sites, so the memory judge can read the capture. |
||
|---|---|---|
| .agents/skills | ||
| .claude | ||
| .devcontainer | ||
| .github | ||
| .opencode/skills/setup-mcps | ||
| .vscode | ||
| assets | ||
| docker | ||
| docs | ||
| packages | ||
| patches | ||
| scripts | ||
| security | ||
| .actrc | ||
| .aikido | ||
| .boundaries-baseline.json | ||
| .code-health-baseline.json | ||
| .dockerignore | ||
| .editorconfig | ||
| .env.eval.example | ||
| .env.local.example | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .npmignore | ||
| .npmrc | ||
| .poutine.yml | ||
| .prettierignore | ||
| .prettierrc.js | ||
| .tbls.postgres.yml | ||
| .tbls.sqlite.yml | ||
| AGENTS.md | ||
| biome.jsonc | ||
| CHANGELOG.md | ||
| CLAUDE.md | ||
| CODE_OF_CONDUCT.md | ||
| codecov.yml | ||
| CONTRIBUTING.md | ||
| CONTRIBUTOR_LICENSE_AGREEMENT.md | ||
| cubic.yaml | ||
| lefthook.yml | ||
| LICENSE_EE.md | ||
| LICENSE.md | ||
| package.json | ||
| pnpm-lock.yaml | ||
| pnpm-workspace.yaml | ||
| README.md | ||
| SECURITY.md | ||
| tsconfig.configs.json | ||
| tsconfig.json | ||
| turbo.json | ||
| vitest.workspace.ts | ||
n8n – The Platform for AI Agents and Workflow Automation
Fair-code platform to build and deploy AI agents and workflows. Combine a visual canvas with custom code, run it self-hosted or in the cloud, and connect to 1500+ integrations. AI automation you can trust with real work, from prototype to production.
Key Capabilities
- AI-Native Automation Platform: Build and operationalize AI workflows and multi-step agents using your own data, models, and tools
- Model Flexibility, No Lock-In: Connect to OpenAI, Anthropic, Google, or open-source models and switch providers without changing your architecture
- From Prototype to Production: Design multi-step AI workflows with logic, tool use, human approvals, and full observability
- Code When You Need It: Combine visual building with JavaScript, Python, and npm packages for advanced AI workflows
- Enterprise-Ready AI: Self-host or deploy securely with role-based access, audit trails, and support for sensitive data
- Leverage What Already Exists: 1500+ integrations and 9,000+ workflow templates to connect AI with your existing systems
Quick Start
Try n8n instantly with npx (requires Node.js):
npx n8n
Or deploy with Docker:
docker volume create n8n_data
docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n
Access the editor at http://localhost:5678
Resources
- 📚 Documentation
- 🔧 1500+ Integrations
- 💡 Example Workflows
- 🤖 AI & LangChain Guide
- 👥 Community Forum
- 📖 Community Tutorials
Support
Need help? Our community forum is the place to get support and connect with other users: community.n8n.io
License
n8n is fair-code distributed under the Sustainable Use License and n8n Enterprise License.
- Source Available: Always visible source code
- Self-Hostable: Deploy anywhere
- Extensible: Add your own nodes and functionality
Enterprise Licenses available for additional features and support.
Additional information about the license model can be found in the docs.
Contributing
Found a bug 🐛 or have a feature idea ✨? Check our Contributing Guide for a setup guide & best practices.
Join the Team
Want to shape the future of automation? Check out our job posts and join our team!
What does n8n mean?
Short answer: It means "nodemation" and is pronounced as n-eight-n.
Long answer: "I get that question quite often (more often than I expected) so I decided it is probably best to answer it here. While looking for a good name for the project with a free domain I realized very quickly that all the good ones I could think of were already taken. So, in the end, I chose nodemation. 'node-' in the sense that it uses a Node-View and that it uses Node.js and '-mation' for 'automation' which is what the project is supposed to help with. However, I did not like how long the name was and I could not imagine writing something that long every time in the CLI. That is when I then ended up on 'n8n'." - Jan Oberhauser, Founder and CEO, n8n.io