n8n/packages/testing/playwright/AGENTS.md

AGENTS.md

Commands

# Run tests locally
pnpm --filter=n8n-playwright test:local <file-path>
pnpm --filter=n8n-playwright test:local tests/e2e/credentials/crud.spec.ts

# Run with container capabilities (requires pnpm build:docker first)
pnpm --filter=n8n-playwright test:container:sqlite --grep @capability:email

# Lint and typecheck
pnpm --filter=n8n-playwright lint
pnpm --filter=n8n-playwright typecheck

Always trim output: --reporter=list 2>&1 | tail -50

Test Maintenance (Janitor)

Static analysis for Playwright test architecture. Catches problems before they spread.

CRITICAL: Always use TCR for code changes. When janitor identifies violations and you fix them, use pnpm janitor tcr --execute to safely commit. Never manually commit janitor-related fixes - TCR ensures tests pass before the commit lands.

Golden Rules

1. Analysis only? Run pnpm janitor (no TCR needed)
2. Making code changes? Use TCR: pnpm janitor tcr --execute -m="chore: ..."
3. Never manually git commit janitor-related fixes - always go through TCR
4. Never modify .janitor-baseline.json via TCR - baseline updates must be done manually

When to Use

User Says	Intent	Approach
"Clean up the test codebase"	Incremental cleanup	Create baseline first, then use --max-diff-lines=500 for small PRs.
"Start tracking violations"	Enable incremental cleanup	Run janitor baseline to snapshot current state, commit .janitor-baseline.json.
"Add a test for X"	New test following patterns	After writing, run janitor to verify architecture compliance.
"Fix architecture drift"	Enforce layered architecture	Run selector-purity and no-page-in-flow rules.
"Find dead code"	Remove unused methods	Run dead-code rule with --fix --write for auto-removal.
"Find copy-paste code"	Detect duplicates	Run duplicate-logic rule to find structural duplicates.
"This file is messy"	Targeted cleanup	Analyze specific file, fix issues, TCR to safely commit.
"Refactor this page object"	Safe refactoring	Use TCR - changes commit if tests pass, revert if they fail.
"What tests would break?"	Impact analysis	Run impact command before changing shared code.
"Prepare for PR"	Pre-commit check	Run janitor on changed files to catch violations early.

Architecture Rules

The janitor enforces a layered architecture:

Tests → Flows/Composables → Page Objects → Components → Playwright API

Rule	What It Catches
selector-purity	Raw locators in tests/flows: page.getByTestId(), someLocator.locator()
no-page-in-flow	Flows accessing page directly (should use page objects)
boundary-protection	Pages importing other pages (creates coupling)
scope-lockdown	Unscoped locators that escape their container
dead-code	Unused public methods in page objects
deduplication	Same selector defined in multiple files
duplicate-logic	Copy-pasted code across tests/pages (AST fingerprinting)

Commands

# Analyze entire codebase
pnpm janitor

# Analyze specific file
pnpm janitor --file=pages/CanvasPage.ts --verbose

# Run specific rule
pnpm janitor --rule=dead-code

# Auto-fix (dead-code only)
pnpm janitor:fix --rule=dead-code

# List all rules (short)
pnpm janitor --list

# Show detailed rule info (for AI agents)
pnpm janitor rules --json

# Discover test specs (for orchestration)
pnpm janitor discover

# Distribute specs across shards
pnpm janitor orchestrate --shards=14

Baseline (Incremental Cleanup)

For codebases with existing violations, create a baseline to enable incremental cleanup:

# Create baseline - snapshots current violations
pnpm janitor baseline

# Commit the baseline
git add .janitor-baseline.json && git commit -m "chore: add janitor baseline"

Once baseline exists, janitor and TCR only fail on NEW violations. Pre-existing violations are tracked but don't block work.

Safeguard: TCR blocks commits that modify .janitor-baseline.json. This prevents accidentally "fixing" violations by updating the baseline instead of the actual code. Baseline updates must be done manually after fixing violations.

# Update baseline after fixing violations (manual commit required)
pnpm janitor baseline
git add .janitor-baseline.json && git commit -m "chore: update baseline after cleanup"

Incremental Cleanup Strategy

For large cleanups, keep diffs small and reviewable:

# Show ALL violations (ignoring baseline) for cleanup work
pnpm janitor --ignore-baseline --json

# Find easiest files to fix (lowest violation count)
pnpm janitor --ignore-baseline --json 2>/dev/null | jq '.fileReports | sort_by(.violationCount) | .[:5]'

# TCR with max diff size (skip if changes are too large)
pnpm janitor tcr --max-diff-lines=500 --execute -m="chore: cleanup"

AI Cleanup Workflow:

1. Use --ignore-baseline to see all violations (not just new ones)
2. Pick small fixes from the list
3. Fix violations, then TCR to safely commit
4. After fixing, run pnpm janitor baseline to update the baseline

TCR Workflow (Test && Commit || Revert)

Safe refactoring loop: make changes, run affected tests, auto-commit or auto-revert.

# Dry run - see what would happen
pnpm janitor tcr --verbose

# Execute - actually commit/revert
pnpm janitor tcr --execute -m="chore: remove dead code"

# With guardrails - skip if diff too large
pnpm janitor tcr --execute --max-diff-lines=500 -m="chore: cleanup"

After Writing New Tests

Always run janitor after adding or modifying tests to catch architecture violations early:

pnpm janitor --file=tests/my-new-test.spec.ts --verbose

See packages/testing/janitor/README.md for full documentation.

Entry Points

All tests should start with n8n.start.* methods. See composables/TestEntryComposer.ts.

Method	Use Case
fromHome()	Start from home page
fromBlankCanvas()	New workflow from scratch
fromNewProjectBlankCanvas()	Project-scoped workflow (returns projectId)
fromNewProject()	Project-scoped test, no canvas (returns projectId)
fromImportedWorkflow(file)	Test pre-built workflow JSON
withUser(user)	Isolated browser context per user
withProjectFeatures()	Enable sharing/folders/permissions

Test Isolation

Tests run in parallel. Design tests to be fully isolated so they don't interfere with each other.

Unique Identifiers

Use nanoid for unique test data:

const credentialName = `Test Credential ${nanoid()}`;
const workflow = await api.workflows.createWorkflow({
  name: `Test Workflow ${nanoid()}`,
});

Dynamic User Creation

Create users dynamically via the public API:

const member = await api.publicApi.createUser({
  email: `member-${nanoid()}@test.com`,
  firstName: 'Test',
  lastName: 'Member',
});

Isolated Browser Contexts

For UI tests requiring multiple users, create isolated browser contexts:

// 1. Create users via public API
const member1 = await api.publicApi.createUser({ role: 'global:member' });
const member2 = await api.publicApi.createUser({ role: 'global:member' });

// 2. Get isolated browser contexts
const member1Page = await n8n.start.withUser(member1);
const member2Page = await n8n.start.withUser(member2);

// 3. Each operates independently (no session bleeding)
await member1Page.navigate.toWorkflows();
await member2Page.navigate.toCredentials();

Reference: tests/e2e/building-blocks/user-service.spec.ts

Pattern	Why	Use Instead
test.describe.serial	Creates test dependencies	Parallel tests with isolated setup
Fresh DB per file	Tests need isolated container	test.use({ capability: { env: { TEST_ISOLATION: 'name' } } })
Fresh DB per test	Tests modify shared state	@db:reset tag on describe (container-only, combined with test.use())
n8n.api.signin()	Session bleeding	n8n.start.withUser()
Date.now() for IDs	Race conditions	nanoid()
waitForTimeout()	Flaky	waitForResponse(), toBeVisible()
.toHaveCount(N)	Brittle	Named element assertions
Raw page.goto()	Bypasses setup	n8n.navigate.* methods

Code Style

• Use specialized locators: page.getByRole('button') over page.locator('[role=button]')
• Use nanoid() for unique identifiers (parallel-safe)
• API setup over UI setup when possible (faster, more reliable)

Architecture

Tests (*.spec.ts)
    ↓ uses
Composables (*Composer.ts) - Multi-step business workflows
    ↓ orchestrates
Page Objects (*Page.ts) - UI interactions
    ↓ extends
BasePage - Common utilities

See CONTRIBUTING.md for detailed patterns and conventions.

Debugging

See README.md#debugging for detailed instructions on:

• Keepalive mode - Keep containers running after tests with N8N_CONTAINERS_KEEPALIVE=true
• Victoria exports - Logs/metrics automatically attached on failure, importable locally via scripts/import-victoria-data.mjs

Test Migration & Refactoring

Test Name = Contract

• Name declares intent, assertion proves it, everything else is flow
• Bad: should open W1 as U2 (describes action)
• Good: should allow sharee to edit shared workflow (declares rule)

Coverage Parity Check

1. Read old test name → what was the intent?
2. Find the explicit assertion that proved it
3. Verify new test has equivalent proof
4. No proof found? Document as intentional drop or gap

Legacy Tests (unauditable names/assertions)

• Prioritize clarity over parity - can't audit what you can't read
• Document your best interpretation of intent
• Accept short-term risk, fix regressions forward

See Quality Corner: Test Migration Guide for full rationale and examples.

Reference Files

Purpose	File
Multi-user testing	tests/e2e/building-blocks/user-service.spec.ts
Entry points	composables/TestEntryComposer.ts
Page object example	pages/CanvasPage.ts
Composable example	composables/WorkflowComposer.ts
API helpers	services/api-helper.ts
Capabilities	fixtures/capabilities.ts

const member = await api.publicApi.createUser({...});
const memberN8n = await n8n.start.withUser(member);

await memberN8n.navigate.toWorkflows();
await expect(memberN8n.workflows.cards.getWorkflow(workflowName)).toBeVisible();

Isolated API Contexts

For API-only operations as another user, create isolated API contexts (no browser needed):

const member = await api.publicApi.createUser({...});
const memberApi = await api.createApiForUser(member);

const memberProject = await memberApi.projects.getMyPersonalProject();
await memberApi.credentials.createCredential({...});

Identity-Based Assertions

Assert by identity (name) rather than count for parallel-safe tests:

await expect(credentialDropdown.getByText(testCredName)).toBeVisible();
await expect(credentialDropdown.getByText(devCredName)).toBeHidden();

Worker Isolation (Fresh Database)

Use test.use() at file top-level with unique capability config:

// my-isolated-tests.spec.ts
import { test, expect } from '../fixtures/base';

// Must be top-level, not inside describe block
test.use({ capability: { env: { TEST_ISOLATION: 'my-isolated-tests' } } });

test('test with clean state', async ({ n8n }) => {
  // Fresh container with reset database
});

For per-test database reset (when tests modify shared state like MFA), add @db:reset to the describe. Note: @db:reset is container-only - these tests won't run locally.

test.use({ capability: { env: { TEST_ISOLATION: 'my-stateful-tests' } } });

test.describe('My stateful tests @db:reset', () => {
  // Each test gets a fresh database reset (container-only)
});

Data Setup

Use API helpers for fast, reliable test data setup. Reserve UI interactions for testing UI behavior:

// API for data setup
const credential = await api.credentials.createCredential({
  name: `Test Credential ${nanoid()}`,
  type: 'notionApi',
  data: { apiKey: 'test' },
});

const workflow = await api.workflows.createWorkflow({
  name: `Test Workflow ${nanoid()}`,
  nodes: [...],
});

// UI for verification
await n8n.navigate.toCredentials();
await expect(n8n.credentials.cards.getCredential(credential.name)).toBeVisible();

Feature Enablement

The n8n fixture automatically enables project features. For API-only tests (no n8n fixture), enable features explicitly:

test('API-only test', async ({ api }) => {
  await api.enableProjectFeatures();
  // ...
});

Feature Flag Overrides

To test features behind feature flags (experiments), use TestRequirements with storage overrides:

import type { TestRequirements } from '../config/TestRequirements';

const requirements: TestRequirements = {
  storage: {
    N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ 'your_experiment': true }),
  },
};

test.use({ requirements });

test('test with feature flag enabled', async ({ n8n }) => {
  // Feature flag is now active for this test
});

Common patterns:

// Single experiment
{ storage: { N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ '025_new_canvas': true }) } }

// Multiple experiments
{ storage: { N8N_EXPERIMENT_OVERRIDES: JSON.stringify({
  '025_new_canvas': true,
  '026_another_feature': 'variant_a'
}) } }

// Combined with other requirements
const requirements: TestRequirements = {
  storage: {
    N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ 'your_experiment': true }),
  },
  capability: {
    env: { TEST_ISOLATION: 'my-test-suite' },
  },
};

Reference: config/TestRequirements.ts for full interface definition.

Shard Rebalancing

When refactoring, adding, or moving significant numbers of tests, consider rebalancing test shards to maintain even CI distribution. See docs/ORCHESTRATION.md for details.