mirror of
https://github.com/n8n-io/n8n.git
synced 2026-05-12 16:10:30 +02:00
test: Add e2e test template and improve agents.md (#24275)
This commit is contained in:
parent
206b3f3c97
commit
307b851de4
18
.github/WORKFLOWS.md
vendored
18
.github/WORKFLOWS.md
vendored
|
|
@ -207,8 +207,26 @@ These only run if specific files changed:
|
||||||
|
|
||||||
| Workflow | Purpose |
|
| Workflow | Purpose |
|
||||||
|---------------------------|---------------------------------------------------------|
|
|---------------------------|---------------------------------------------------------|
|
||||||
|
| `util-claude-task.yml` | Run Claude Code to complete a task and create a PR |
|
||||||
| `util-data-tooling.yml` | SQLite/PostgreSQL export/import validation (manual) |
|
| `util-data-tooling.yml` | SQLite/PostgreSQL export/import validation (manual) |
|
||||||
|
|
||||||
|
#### Claude Task Runner (`util-claude-task.yml`)
|
||||||
|
|
||||||
|
Runs Claude Code to complete a task, then creates a PR with the changes. Use for well-specced tasks or simple fixes. Can be triggered via GitHub UI or API.
|
||||||
|
|
||||||
|
Claude reads templates from `.github/claude-templates/` for task-specific guidance. Add new templates as needed for recurring task types.
|
||||||
|
|
||||||
|
**Inputs:**
|
||||||
|
- `task` - Description of what Claude should do
|
||||||
|
- `user_token` - GitHub PAT (PR will be authored by the token owner)
|
||||||
|
|
||||||
|
**Token requirements** (fine-grained PAT):
|
||||||
|
- Repository: `n8n-io/n8n`
|
||||||
|
- Contents: `Read and write`
|
||||||
|
- Pull requests: `Read and write`
|
||||||
|
|
||||||
|
**Governance:** If you provide your personal PAT, you cannot approve the resulting PR. For automated/bot use cases (e.g., dependabot-style updates via n8n workflows), an app token can be used instead.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Workflow Call Graph
|
## Workflow Call Graph
|
||||||
|
|
|
||||||
119
.github/claude-templates/e2e-test.md
vendored
Normal file
119
.github/claude-templates/e2e-test.md
vendored
Normal file
|
|
@ -0,0 +1,119 @@
|
||||||
|
# E2E Test Task Guide
|
||||||
|
|
||||||
|
## Required Reading
|
||||||
|
|
||||||
|
**Before writing any code**, read these files:
|
||||||
|
```
|
||||||
|
packages/testing/playwright/AGENTS.md # Patterns, anti-patterns, entry points
|
||||||
|
packages/testing/playwright/CONTRIBUTING.md # Detailed architecture (first 200 lines)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Spec Validation
|
||||||
|
|
||||||
|
Before starting, verify the spec includes:
|
||||||
|
|
||||||
|
| Required | Example |
|
||||||
|
|----------|---------|
|
||||||
|
| **File(s) to modify** | `tests/e2e/credentials/crud.spec.ts` |
|
||||||
|
| **Specific behavior** | "Verify credential renaming updates the list" |
|
||||||
|
| **Pattern reference** | "Follow existing tests in same file" or "See AGENTS.md" |
|
||||||
|
|
||||||
|
**If missing, ask for clarification.** Don't guess at requirements.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run single test
|
||||||
|
pnpm --filter=n8n-playwright test:local tests/e2e/your-test.spec.ts --reporter=list 2>&1 | tail -50
|
||||||
|
|
||||||
|
# Run with pattern match
|
||||||
|
pnpm --filter=n8n-playwright test:local --grep "should do something" --reporter=list 2>&1 | tail -50
|
||||||
|
|
||||||
|
# Container tests (requires pnpm build:docker first)
|
||||||
|
pnpm --filter=n8n-playwright test:container:sqlite --grep @capability:email --reporter=list 2>&1 | tail -50
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test Structure
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { test, expect } from '../fixtures/base';
|
||||||
|
import { nanoid } from 'nanoid';
|
||||||
|
|
||||||
|
test('should do something @mode:sqlite', async ({ n8n, api }) => {
|
||||||
|
// Setup via API (faster, more reliable)
|
||||||
|
const workflow = await api.workflowApi.createWorkflow(workflowJson);
|
||||||
|
|
||||||
|
// UI interaction via entry points
|
||||||
|
await n8n.start.fromBlankCanvas();
|
||||||
|
|
||||||
|
// Assertions
|
||||||
|
await expect(n8n.workflows.getWorkflowByName(workflow.name)).toBeVisible();
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Entry Points
|
||||||
|
|
||||||
|
Use `n8n.start.*` methods - see `composables/TestEntryComposer.ts`:
|
||||||
|
- `fromBlankCanvas()` - New workflow
|
||||||
|
- `fromImportedWorkflow(file)` - Pre-built workflow
|
||||||
|
- `fromNewProjectBlankCanvas()` - Project-scoped
|
||||||
|
- `withUser(user)` - Isolated browser context
|
||||||
|
|
||||||
|
## Multi-User Tests
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const member = await api.publicApi.createUser({ role: 'global:member' });
|
||||||
|
const memberPage = await n8n.start.withUser(member);
|
||||||
|
await memberPage.navigate.toWorkflows();
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Process
|
||||||
|
|
||||||
|
1. **Validate spec** - Has file, behavior, pattern reference?
|
||||||
|
2. **Read existing code** - Understand current patterns in the file
|
||||||
|
3. **Identify helpers needed** - Check `pages/`, `services/`, `composables/`
|
||||||
|
4. **Add helpers first** if missing
|
||||||
|
5. **Write test** following 4-layer architecture
|
||||||
|
6. **Verify iteratively** - Small changes, test frequently
|
||||||
|
|
||||||
|
## Mandatory Verification
|
||||||
|
|
||||||
|
**Always run before marking complete:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Tests pass (check output for failures - piping loses exit code)
|
||||||
|
pnpm --filter=n8n-playwright test:local <your-test> --reporter=list 2>&1 | tail -50
|
||||||
|
|
||||||
|
# 2. Not flaky (required)
|
||||||
|
pnpm --filter=n8n-playwright test:local <your-test> --repeat-each 3 --reporter=list 2>&1 | tail -50
|
||||||
|
|
||||||
|
# 3. Lint passes
|
||||||
|
pnpm --filter=n8n-playwright lint 2>&1 | tail -30
|
||||||
|
|
||||||
|
# 4. Typecheck passes
|
||||||
|
pnpm --filter=n8n-playwright typecheck 2>&1 | tail -30
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important:** Piping through `tail` loses the exit code. Always check the output for "failed" or error messages rather than relying on exit codes.
|
||||||
|
|
||||||
|
**If any fail, fix before completing.**
|
||||||
|
|
||||||
|
## Refactoring Existing Tests
|
||||||
|
|
||||||
|
**Always verify tests pass BEFORE making changes:**
|
||||||
|
```bash
|
||||||
|
pnpm --filter=n8n-playwright test:local tests/e2e/target-file.spec.ts --reporter=list 2>&1 | tail -50
|
||||||
|
```
|
||||||
|
|
||||||
|
Then make small incremental changes, re-running after each.
|
||||||
|
|
||||||
|
## Done Checklist
|
||||||
|
|
||||||
|
- [ ] Spec had clear file, behavior, and pattern reference
|
||||||
|
- [ ] Read `AGENTS.md` and relevant existing code
|
||||||
|
- [ ] Used `n8n.start.*` entry points
|
||||||
|
- [ ] Used `nanoid()` for unique IDs (not `Date.now()`)
|
||||||
|
- [ ] No serial mode, `@db:reset`, or `n8n.api.signin()`
|
||||||
|
- [ ] Multi-user tests use `n8n.start.withUser()`
|
||||||
|
- [ ] Tests pass with `--repeat-each 3`
|
||||||
|
- [ ] Lint and typecheck pass
|
||||||
47
.github/workflows/util-claude-task.yml
vendored
47
.github/workflows/util-claude-task.yml
vendored
|
|
@ -7,6 +7,10 @@ on:
|
||||||
description: 'Task description - what should Claude do?'
|
description: 'Task description - what should Claude do?'
|
||||||
required: true
|
required: true
|
||||||
type: string
|
type: string
|
||||||
|
user_token:
|
||||||
|
description: 'Your GitHub PAT (required for PR authorship - you cannot approve PRs you author)'
|
||||||
|
required: true
|
||||||
|
type: string
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
run-claude-task:
|
run-claude-task:
|
||||||
|
|
@ -18,17 +22,13 @@ jobs:
|
||||||
issues: write
|
issues: write
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Generate App Token
|
- name: Mask user token
|
||||||
id: app-token
|
run: echo "::add-mask::${{ inputs.user_token }}"
|
||||||
uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1
|
|
||||||
with:
|
|
||||||
app-id: ${{ secrets.N8N_ASSISTANT_APP_ID }}
|
|
||||||
private-key: ${{ secrets.N8N_ASSISTANT_PRIVATE_KEY }}
|
|
||||||
|
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
|
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
|
||||||
with:
|
with:
|
||||||
token: ${{ steps.app-token.outputs.token }}
|
token: ${{ inputs.user_token }}
|
||||||
ref: master
|
ref: master
|
||||||
fetch-depth: 1
|
fetch-depth: 1
|
||||||
|
|
||||||
|
|
@ -45,23 +45,16 @@ jobs:
|
||||||
|
|
||||||
- name: Configure git author
|
- name: Configure git author
|
||||||
env:
|
env:
|
||||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GH_TOKEN: ${{ inputs.user_token }}
|
||||||
ACTOR: ${{ github.actor }}
|
|
||||||
run: |
|
run: |
|
||||||
# Set git author to the user who triggered the workflow
|
# Set git author from the authenticated user (token owner)
|
||||||
USER_DATA=$(gh api "/users/$ACTOR")
|
USER_DATA=$(gh api user)
|
||||||
USER_NAME=$(echo "$USER_DATA" | jq -r '.name // .login')
|
USER_NAME=$(echo "$USER_DATA" | jq -r '.name // .login')
|
||||||
|
USER_LOGIN=$(echo "$USER_DATA" | jq -r '.login')
|
||||||
USER_ID=$(echo "$USER_DATA" | jq -r '.id')
|
USER_ID=$(echo "$USER_DATA" | jq -r '.id')
|
||||||
USER_EMAIL="${USER_ID}+${ACTOR}@users.noreply.github.com"
|
USER_EMAIL="${USER_ID}+${USER_LOGIN}@users.noreply.github.com"
|
||||||
git config user.name "$USER_NAME"
|
git config user.name "$USER_NAME"
|
||||||
git config user.email "$USER_EMAIL"
|
git config user.email "$USER_EMAIL"
|
||||||
# Export for Claude action to use
|
|
||||||
{
|
|
||||||
echo "GIT_AUTHOR_NAME=$USER_NAME"
|
|
||||||
echo "GIT_AUTHOR_EMAIL=$USER_EMAIL"
|
|
||||||
echo "GIT_COMMITTER_NAME=$USER_NAME"
|
|
||||||
echo "GIT_COMMITTER_EMAIL=$USER_EMAIL"
|
|
||||||
} >> "$GITHUB_ENV"
|
|
||||||
echo "Git author configured as: $USER_NAME <$USER_EMAIL>"
|
echo "Git author configured as: $USER_NAME <$USER_EMAIL>"
|
||||||
|
|
||||||
- name: Prepare Claude prompt
|
- name: Prepare Claude prompt
|
||||||
|
|
@ -82,10 +75,9 @@ jobs:
|
||||||
echo "1. Read relevant templates from .github/claude-templates/ first"
|
echo "1. Read relevant templates from .github/claude-templates/ first"
|
||||||
echo "2. Complete the task described above"
|
echo "2. Complete the task described above"
|
||||||
echo "3. Follow the guidelines from the templates"
|
echo "3. Follow the guidelines from the templates"
|
||||||
echo "4. Make commits as you work with descriptive messages"
|
echo "4. Make commits as you work - the last commit message will be used as the PR title"
|
||||||
echo "5. IMPORTANT: End every commit message with: Co-authored-by: Claude <noreply@anthropic.com>"
|
echo "5. IMPORTANT: End every commit message with: Co-authored-by: Claude <noreply@anthropic.com>"
|
||||||
echo "6. Ensure code passes linting and type checks before finishing"
|
echo "6. Ensure code passes linting and type checks before finishing"
|
||||||
echo "7. When done, output a PR title on a single line starting with 'PR_TITLE:' (e.g. 'PR_TITLE: fix: Update smithy dependency to resolve CVE')"
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "# Token Optimization"
|
echo "# Token Optimization"
|
||||||
echo "When running lint/typecheck, suppress verbose output:"
|
echo "When running lint/typecheck, suppress verbose output:"
|
||||||
|
|
@ -99,17 +91,18 @@ jobs:
|
||||||
uses: anthropics/claude-code-action@1b8ee3b94104046d71fde52ec3557651ad8c0d71 # v1
|
uses: anthropics/claude-code-action@1b8ee3b94104046d71fde52ec3557651ad8c0d71 # v1
|
||||||
with:
|
with:
|
||||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||||
github_token: ${{ steps.app-token.outputs.token }}
|
github_token: ${{ inputs.user_token }}
|
||||||
prompt: ${{ env.CLAUDE_PROMPT }}
|
prompt: ${{ env.CLAUDE_PROMPT }}
|
||||||
claude_args: |
|
claude_args: |
|
||||||
--allowedTools Bash,Read,Write,Edit,Glob,Grep,WebFetch,WebSearch,TodoWrite
|
--allowedTools Bash,Read,Write,Edit,Glob,Grep,WebFetch,WebSearch,TodoWrite
|
||||||
|
|
||||||
- name: Extract PR title
|
- name: Extract PR title
|
||||||
env:
|
|
||||||
CLAUDE_CONCLUSION: ${{ steps.claude.outputs.conclusion }}
|
|
||||||
run: |
|
run: |
|
||||||
# Extract PR title from Claude's output (looks for PR_TITLE: prefix)
|
# Use the last commit message as PR title
|
||||||
PR_TITLE=$(echo "$CLAUDE_CONCLUSION" | sed -n 's/.*PR_TITLE:\s*//p' | head -1)
|
PR_TITLE=$(git log -1 --format='%s' 2>/dev/null | head -1)
|
||||||
|
# Strip Co-authored-by suffix if present
|
||||||
|
PR_TITLE="${PR_TITLE%%[Cc]o-[Aa]uthored-[Bb]y:*}"
|
||||||
|
PR_TITLE="${PR_TITLE%% }"
|
||||||
if [ -z "$PR_TITLE" ]; then
|
if [ -z "$PR_TITLE" ]; then
|
||||||
PR_TITLE="chore: Claude automated task (run ${{ github.run_id }})"
|
PR_TITLE="chore: Claude automated task (run ${{ github.run_id }})"
|
||||||
fi
|
fi
|
||||||
|
|
@ -118,7 +111,7 @@ jobs:
|
||||||
|
|
||||||
- name: Push branch and create PR
|
- name: Push branch and create PR
|
||||||
env:
|
env:
|
||||||
GH_TOKEN: ${{ steps.app-token.outputs.token }}
|
GH_TOKEN: ${{ inputs.user_token }}
|
||||||
INPUT_TASK: ${{ inputs.task }}
|
INPUT_TASK: ${{ inputs.task }}
|
||||||
TRIGGERED_BY: ${{ github.actor }}
|
TRIGGERED_BY: ${{ github.actor }}
|
||||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||||
|
|
|
||||||
|
|
@ -2,43 +2,110 @@
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
- Use `pnpm --filter=n8n-playwright test:local <file-path>` to execute tests.
|
```bash
|
||||||
For example: `pnpm --filter=n8n-playwright test:local tests/e2e/credentials/crud.spec.ts`
|
# Run tests locally
|
||||||
|
pnpm --filter=n8n-playwright test:local <file-path>
|
||||||
|
pnpm --filter=n8n-playwright test:local tests/e2e/credentials/crud.spec.ts
|
||||||
|
|
||||||
- Use `pnpm --filter=n8n-playwright test:container:sqlite --grep pattern` to execute the tests using test containers for particular features.
|
# Run with container capabilities (requires pnpm build:docker first)
|
||||||
For example `pnpm --filter=n8n-playwright test:container:sqlite --grep @capability:email`
|
pnpm --filter=n8n-playwright test:container:sqlite --grep @capability:email
|
||||||
Note: This requires the docker container to be built locally using pnpm build:docker
|
|
||||||
|
|
||||||
## Code Styles
|
# Lint and typecheck
|
||||||
|
pnpm --filter=n8n-playwright lint
|
||||||
|
pnpm --filter=n8n-playwright typecheck
|
||||||
|
```
|
||||||
|
|
||||||
- In writing locators, use specialized methods when available.
|
Always trim output: `--reporter=list 2>&1 | tail -50`
|
||||||
For example, prefer `page.getByRole('button')` over `page.locator('[role=button]')`.
|
|
||||||
|
|
||||||
## Concurrency
|
## Entry Points
|
||||||
|
|
||||||
Tests run in parallel. Use `nanoid` (not `Date.now()`) for unique identifiers.
|
All tests should start with `n8n.start.*` methods. See `composables/TestEntryComposer.ts`.
|
||||||
|
|
||||||
|
| Method | Use Case |
|
||||||
|
|--------|----------|
|
||||||
|
| `fromHome()` | Start from home page |
|
||||||
|
| `fromBlankCanvas()` | New workflow from scratch |
|
||||||
|
| `fromNewProjectBlankCanvas()` | Project-scoped workflow (returns projectId) |
|
||||||
|
| `fromNewProject()` | Project-scoped test, no canvas (returns projectId) |
|
||||||
|
| `fromImportedWorkflow(file)` | Test pre-built workflow JSON |
|
||||||
|
| `withUser(user)` | Isolated browser context per user |
|
||||||
|
| `withProjectFeatures()` | Enable sharing/folders/permissions |
|
||||||
|
|
||||||
## Multi-User Testing
|
## Multi-User Testing
|
||||||
|
|
||||||
For tests requiring multiple users with isolated browser sessions:
|
For tests requiring multiple users with isolated browser sessions:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// 1. Create users via public API (owner's API key created automatically)
|
// 1. Create users via public API
|
||||||
const member1 = await api.publicApi.createUser({ role: 'global:member' });
|
const member1 = await api.publicApi.createUser({ role: 'global:member' });
|
||||||
const member2 = await api.publicApi.createUser({ role: 'global:member' });
|
const member2 = await api.publicApi.createUser({ role: 'global:member' });
|
||||||
|
|
||||||
// 2. Get isolated browser contexts for each user
|
// 2. Get isolated browser contexts
|
||||||
const member1Page = await n8n.start.withUser(member1);
|
const member1Page = await n8n.start.withUser(member1);
|
||||||
const member2Page = await n8n.start.withUser(member2);
|
const member2Page = await n8n.start.withUser(member2);
|
||||||
|
|
||||||
// 3. Test interactions between users
|
// 3. Each operates independently (no session bleeding)
|
||||||
await member1Page.navigate.toWorkflows();
|
await member1Page.navigate.toWorkflows();
|
||||||
await member2Page.navigate.toWorkflows();
|
await member2Page.navigate.toCredentials();
|
||||||
```
|
```
|
||||||
|
|
||||||
This approach provides:
|
**Reference:** `tests/e2e/building-blocks/user-service.spec.ts`
|
||||||
- Full browser isolation (separate cookies, storage, state)
|
|
||||||
- Dynamic users with unique emails (no pre-seeded dependencies)
|
|
||||||
- Parallel-safe execution (no serial mode needed)
|
|
||||||
|
|
||||||
Avoid the legacy pattern of `n8n.api.signin('member', 0)` which reuses the same browser context and risks session state bleeding.
|
## Worker Isolation (Fresh Database)
|
||||||
|
|
||||||
|
Use `test.use()` at file top-level with unique capability config:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// my-isolated-tests.spec.ts
|
||||||
|
import { test, expect } from '../fixtures/base';
|
||||||
|
|
||||||
|
// Must be top-level, not inside describe block
|
||||||
|
test.use({ capability: { env: { _ISOLATION: 'my-isolated-tests' } } });
|
||||||
|
|
||||||
|
test('test with clean state', async ({ n8n }) => {
|
||||||
|
// Fresh container with reset database
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
| Pattern | Why | Use Instead |
|
||||||
|
|---------|-----|-------------|
|
||||||
|
| `test.describe.serial` | Creates test dependencies | Parallel tests with isolated setup |
|
||||||
|
| `@db:reset` tag | Deprecated - CI issues | `test.use()` with unique capability |
|
||||||
|
| `n8n.api.signin()` | Session bleeding | `n8n.start.withUser()` |
|
||||||
|
| `Date.now()` for IDs | Race conditions | `nanoid()` |
|
||||||
|
| `waitForTimeout()` | Flaky | `waitForResponse()`, `toBeVisible()` |
|
||||||
|
| `.toHaveCount(N)` | Brittle | Named element assertions |
|
||||||
|
| Raw `page.goto()` | Bypasses setup | `n8n.navigate.*` methods |
|
||||||
|
|
||||||
|
## Code Style
|
||||||
|
|
||||||
|
- Use specialized locators: `page.getByRole('button')` over `page.locator('[role=button]')`
|
||||||
|
- Use `nanoid()` for unique identifiers (parallel-safe)
|
||||||
|
- API setup over UI setup when possible (faster, more reliable)
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Tests (*.spec.ts)
|
||||||
|
↓ uses
|
||||||
|
Composables (*Composer.ts) - Multi-step business workflows
|
||||||
|
↓ orchestrates
|
||||||
|
Page Objects (*Page.ts) - UI interactions
|
||||||
|
↓ extends
|
||||||
|
BasePage - Common utilities
|
||||||
|
```
|
||||||
|
|
||||||
|
See `CONTRIBUTING.md` for detailed patterns and conventions.
|
||||||
|
|
||||||
|
## Reference Files
|
||||||
|
|
||||||
|
| Purpose | File |
|
||||||
|
|---------|------|
|
||||||
|
| Multi-user testing | `tests/e2e/building-blocks/user-service.spec.ts` |
|
||||||
|
| Entry points | `composables/TestEntryComposer.ts` |
|
||||||
|
| Page object example | `pages/CanvasPage.ts` |
|
||||||
|
| Composable example | `composables/WorkflowComposer.ts` |
|
||||||
|
| API helpers | `services/api-helper.ts` |
|
||||||
|
| Capabilities | `fixtures/capabilities.ts` |
|
||||||
|
|
|
||||||
|
|
@ -1,255 +0,0 @@
|
||||||
# 🚀 n8n Playwright Test Writing Cheat Sheet
|
|
||||||
|
|
||||||
> **For AI Assistants**: This guide provides quick reference patterns for writing n8n Playwright tests using the established architecture.
|
|
||||||
|
|
||||||
## Quick Start Navigation Methods
|
|
||||||
|
|
||||||
### **n8n.start.*** Methods (Test Entry Points)
|
|
||||||
```typescript
|
|
||||||
// Start from home page
|
|
||||||
await n8n.start.fromHome();
|
|
||||||
|
|
||||||
// Start with blank canvas for new workflow
|
|
||||||
await n8n.start.fromBlankCanvas();
|
|
||||||
|
|
||||||
// Start with new project + blank canvas (returns projectId)
|
|
||||||
const projectId = await n8n.start.fromNewProjectBlankCanvas();
|
|
||||||
|
|
||||||
// Start with just a new project (no canvas)
|
|
||||||
const projectId = await n8n.start.fromNewProject();
|
|
||||||
|
|
||||||
// Import and start from existing workflow JSON
|
|
||||||
const result = await n8n.start.fromImportedWorkflow('simple-webhook-test.json');
|
|
||||||
const { workflowId, webhookPath } = result;
|
|
||||||
```
|
|
||||||
|
|
||||||
### **n8n.navigate.*** Methods (Page Navigation)
|
|
||||||
```typescript
|
|
||||||
// Basic navigation
|
|
||||||
await n8n.navigate.toHome();
|
|
||||||
await n8n.navigate.toWorkflow('new');
|
|
||||||
await n8n.navigate.toWorkflows(projectId);
|
|
||||||
|
|
||||||
// Settings & admin
|
|
||||||
await n8n.navigate.toVariables();
|
|
||||||
await n8n.navigate.toCredentials(projectId);
|
|
||||||
await n8n.navigate.toLogStreaming();
|
|
||||||
await n8n.navigate.toCommunityNodes();
|
|
||||||
|
|
||||||
// Project-specific navigation
|
|
||||||
await n8n.navigate.toProject(projectId);
|
|
||||||
await n8n.navigate.toProjectSettings(projectId);
|
|
||||||
```
|
|
||||||
|
|
||||||
## Common Test Patterns
|
|
||||||
|
|
||||||
### **Basic Workflow Test**
|
|
||||||
```typescript
|
|
||||||
test('should create and execute workflow', async ({ n8n }) => {
|
|
||||||
await n8n.start.fromBlankCanvas();
|
|
||||||
await n8n.canvas.addNode('Manual Trigger');
|
|
||||||
await n8n.canvas.addNode('Set');
|
|
||||||
await n8n.workflowComposer.executeWorkflowAndWaitForNotification('Success');
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Imported Workflow Test**
|
|
||||||
```typescript
|
|
||||||
test('should import and test webhook', async ({ n8n }) => {
|
|
||||||
const { webhookPath } = await n8n.start.fromImportedWorkflow('webhook-test.json');
|
|
||||||
|
|
||||||
await n8n.canvas.clickExecuteWorkflowButton();
|
|
||||||
const response = await n8n.page.request.post(`/webhook-test/${webhookPath}`, {
|
|
||||||
data: { message: 'Hello' }
|
|
||||||
});
|
|
||||||
expect(response.ok()).toBe(true);
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Project-Scoped Test**
|
|
||||||
```typescript
|
|
||||||
test('should create credential in project', async ({ n8n }) => {
|
|
||||||
const projectId = await n8n.start.fromNewProject();
|
|
||||||
await n8n.navigate.toCredentials(projectId);
|
|
||||||
|
|
||||||
await n8n.credentialsComposer.createFromList(
|
|
||||||
'Notion API',
|
|
||||||
{ apiKey: '12345' },
|
|
||||||
{ name: `cred-${nanoid()}` }
|
|
||||||
);
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Node Configuration Test**
|
|
||||||
```typescript
|
|
||||||
test('should configure HTTP Request node', async ({ n8n }) => {
|
|
||||||
await n8n.start.fromBlankCanvas();
|
|
||||||
await n8n.canvas.addNode('Manual Trigger');
|
|
||||||
await n8n.canvas.addNode('HTTP Request');
|
|
||||||
|
|
||||||
await n8n.ndv.fillParameterInput('URL', 'https://api.example.com');
|
|
||||||
await n8n.ndv.close();
|
|
||||||
await n8n.canvas.saveWorkflow();
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## Test Setup Patterns
|
|
||||||
|
|
||||||
### **Feature Flags Setup**
|
|
||||||
```typescript
|
|
||||||
test.beforeEach(async ({ n8n, api }) => {
|
|
||||||
await api.enableFeature('sharing');
|
|
||||||
await api.enableFeature('folders');
|
|
||||||
await api.enableFeature('projectRole:admin');
|
|
||||||
await api.setMaxTeamProjectsQuota(-1);
|
|
||||||
await n8n.goHome();
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### **API + UI Combined Test**
|
|
||||||
```typescript
|
|
||||||
test('should use API-created credential in UI', async ({ n8n, api }) => {
|
|
||||||
const projectId = await n8n.start.fromNewProjectBlankCanvas();
|
|
||||||
|
|
||||||
// Create via API
|
|
||||||
await api.credentialApi.createCredential({
|
|
||||||
name: 'test-cred',
|
|
||||||
type: 'notionApi',
|
|
||||||
data: { apiKey: '12345' },
|
|
||||||
projectId
|
|
||||||
});
|
|
||||||
|
|
||||||
// Verify in UI
|
|
||||||
await n8n.canvas.addNode('Notion');
|
|
||||||
await expect(n8n.ndv.getCredentialSelect()).toHaveValue('test-cred');
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Error/Edge Case Testing**
|
|
||||||
```typescript
|
|
||||||
test('should handle workflow execution error', async ({ n8n }) => {
|
|
||||||
await n8n.start.fromImportedWorkflow('failing-workflow.json');
|
|
||||||
await n8n.workflowComposer.executeWorkflowAndWaitForNotification('Problem in node');
|
|
||||||
await expect(n8n.canvas.getErrorIcon()).toBeVisible();
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture Guidelines
|
|
||||||
|
|
||||||
### **Four-Layer UI Testing Architecture**
|
|
||||||
```
|
|
||||||
Tests (*.spec.ts)
|
|
||||||
↓ uses
|
|
||||||
Composables (*Composer.ts) - Business workflows
|
|
||||||
↓ orchestrates
|
|
||||||
Page Objects (*Page.ts) - UI interactions
|
|
||||||
↓ extends
|
|
||||||
BasePage - Common utilities
|
|
||||||
```
|
|
||||||
|
|
||||||
### **When to Use Each Layer**
|
|
||||||
- **Tests**: High-level scenarios, readable business logic
|
|
||||||
- **Composables**: Multi-step workflows (e.g., `executeWorkflowAndWaitForNotification`)
|
|
||||||
- **Page Objects**: Simple UI actions (e.g., `clickSaveButton`, `fillInput`)
|
|
||||||
- **BasePage**: Generic interactions (e.g., `clickByTestId`, `fillByTestId`)
|
|
||||||
|
|
||||||
### **Method Naming Conventions**
|
|
||||||
```typescript
|
|
||||||
// Page Object Getters (No async, return Locator)
|
|
||||||
getSearchBar() { return this.page.getByTestId('search'); }
|
|
||||||
|
|
||||||
// Page Object Actions (async, return void)
|
|
||||||
async clickSaveButton() { await this.clickButtonByName('Save'); }
|
|
||||||
|
|
||||||
// Page Object Queries (async, return data)
|
|
||||||
async getNotificationCount(): Promise<number> { /* ... */ }
|
|
||||||
```
|
|
||||||
|
|
||||||
## Quick Reference
|
|
||||||
|
|
||||||
### **Most Common Entry Points**
|
|
||||||
- `n8n.start.fromBlankCanvas()` - New workflow from scratch
|
|
||||||
- `n8n.start.fromImportedWorkflow('file.json')` - Test existing workflow
|
|
||||||
- `n8n.start.fromNewProjectBlankCanvas()` - Project-scoped testing
|
|
||||||
|
|
||||||
### **Most Common Navigation**
|
|
||||||
- `n8n.navigate.toCredentials(projectId)` - Credential management
|
|
||||||
- `n8n.navigate.toVariables()` - Environment variables
|
|
||||||
- `n8n.navigate.toWorkflow('new')` - New workflow canvas
|
|
||||||
|
|
||||||
### **Essential Assertions**
|
|
||||||
```typescript
|
|
||||||
// UI state verification
|
|
||||||
await expect(n8n.canvas.canvasPane()).toBeVisible();
|
|
||||||
await expect(n8n.notifications.getNotificationByTitle('Success')).toBeVisible();
|
|
||||||
await expect(n8n.ndv.getCredentialSelect()).toHaveValue(name);
|
|
||||||
|
|
||||||
// Node and workflow verification
|
|
||||||
await expect(n8n.canvas.getCanvasNodes()).toHaveCount(2);
|
|
||||||
await expect(n8n.canvas.nodeByName('HTTP Request')).toBeVisible();
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Common Composable Methods**
|
|
||||||
```typescript
|
|
||||||
// Workflow operations
|
|
||||||
await n8n.workflowComposer.executeWorkflowAndWaitForNotification('Success');
|
|
||||||
await n8n.workflowComposer.createWorkflow('My Workflow');
|
|
||||||
|
|
||||||
// Project operations
|
|
||||||
const { projectName, projectId } = await n8n.projectComposer.createProject();
|
|
||||||
|
|
||||||
// Credential operations
|
|
||||||
await n8n.credentialsComposer.createFromList('Notion API', { apiKey: '123' });
|
|
||||||
await n8n.credentialsComposer.createFromNdv({ apiKey: '123' });
|
|
||||||
```
|
|
||||||
|
|
||||||
### **Dynamic Data Patterns**
|
|
||||||
```typescript
|
|
||||||
// Use nanoid for unique identifiers
|
|
||||||
import { nanoid } from 'nanoid';
|
|
||||||
const workflowName = `Test Workflow ${nanoid()}`;
|
|
||||||
const credentialName = `cred-${nanoid()}`;
|
|
||||||
|
|
||||||
// Use timestamps for uniqueness
|
|
||||||
const projectName = `Project ${Date.now()}`;
|
|
||||||
```
|
|
||||||
|
|
||||||
## AI Guidelines
|
|
||||||
|
|
||||||
### **✅ DO**
|
|
||||||
- Always use `n8n.start.*` methods for test entry points
|
|
||||||
- Use composables for business workflows, not page objects directly in tests
|
|
||||||
- Use `nanoid()` or timestamps for unique test data
|
|
||||||
- Follow the 4-layer architecture pattern
|
|
||||||
- Use proper waiting with `expect().toBeVisible()` instead of `waitForTimeout`
|
|
||||||
|
|
||||||
### **❌ DON'T**
|
|
||||||
- Use raw `page.goto()` instead of navigation helpers
|
|
||||||
- Mix business logic in page objects (move to composables)
|
|
||||||
- Use hardcoded selectors in tests (use page object getters)
|
|
||||||
- Create overly specific methods (keep them reusable)
|
|
||||||
- Use `any` types or `waitForTimeout`
|
|
||||||
|
|
||||||
### **Test Structure Template**
|
|
||||||
```typescript
|
|
||||||
import { test, expect } from '../../fixtures/base';
|
|
||||||
|
|
||||||
test.describe('Feature Name', () => {
|
|
||||||
test.beforeEach(async ({ n8n, api }) => {
|
|
||||||
// Feature flags and setup
|
|
||||||
await api.enableFeature('requiredFeature');
|
|
||||||
await n8n.goHome();
|
|
||||||
});
|
|
||||||
|
|
||||||
test('should perform specific action', async ({ n8n }) => {
|
|
||||||
// 1. Setup/Navigation
|
|
||||||
await n8n.start.fromBlankCanvas();
|
|
||||||
|
|
||||||
// 2. Actions using composables
|
|
||||||
await n8n.workflowComposer.createBasicWorkflow();
|
|
||||||
|
|
||||||
// 3. Assertions
|
|
||||||
await expect(n8n.notifications.getNotificationByTitle('Success')).toBeVisible();
|
|
||||||
});
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
@AGENTS.md
|
|
||||||
|
|
@ -51,12 +51,28 @@ pnpm test:local --ui # To enable UI debugging and test running mode
|
||||||
```typescript
|
```typescript
|
||||||
test('basic test', ...) // All modes, fully parallel
|
test('basic test', ...) // All modes, fully parallel
|
||||||
test('postgres only @mode:postgres', ...) // Mode-specific
|
test('postgres only @mode:postgres', ...) // Mode-specific
|
||||||
test('needs clean db @db:reset', ...) // Sequential per worker
|
|
||||||
test('chaos test @mode:multi-main @chaostest', ...) // Isolated per worker
|
test('chaos test @mode:multi-main @chaostest', ...) // Isolated per worker
|
||||||
test('cloud resource test @cloud:trial', ...) // Cloud resource constraints
|
test('cloud resource test @cloud:trial', ...) // Cloud resource constraints
|
||||||
test('proxy test @capability:proxy', ...) // Requires proxy server capability
|
test('proxy test @capability:proxy', ...) // Requires proxy server capability
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Worker Isolation (Fresh Database)
|
||||||
|
If tests need a clean database state, use `test.use()` at the top level of the file with a unique capability config instead of the deprecated `@db:reset` tag:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// my-isolated-tests.spec.ts
|
||||||
|
import { test, expect } from '../fixtures/base';
|
||||||
|
|
||||||
|
// Must be top-level, not inside describe block
|
||||||
|
test.use({ capability: { env: { _ISOLATION: 'my-isolated-tests' } } });
|
||||||
|
|
||||||
|
test('test with clean state', async ({ n8n }) => {
|
||||||
|
// Fresh container with reset database
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Deprecated:** `@db:reset` tag causes CI issues (separate workers, sequential execution). Use `test.use()` pattern above instead.
|
||||||
|
|
||||||
## Fixture Selection
|
## Fixture Selection
|
||||||
- **`base.ts`**: Standard testing with worker-scoped containers (default choice)
|
- **`base.ts`**: Standard testing with worker-scoped containers (default choice)
|
||||||
- **`cloud-only.ts`**: Cloud resource testing with guaranteed isolation
|
- **`cloud-only.ts`**: Cloud resource testing with guaranteed isolation
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue
Block a user