mirror of
https://github.com/n8n-io/n8n.git
synced 2026-07-28 11:35:03 +02:00
feat(core): Prioritize workflow as code over json (#32668)
This commit is contained in:
parent
b294e3d51a
commit
e11e0b122c
|
|
@ -117,8 +117,9 @@ exactly one checkpoint task (`checkpoint.id`, `checkpoint.title`,
|
|||
tasks, including workflow build outcomes with their `outcome.workItemId` /
|
||||
`outcome.workflowId`). **Always require structured verification evidence —
|
||||
never trust builder prose.** Before completing the checkpoint, inspect each
|
||||
dependent persisted workflow with `workflows(action="get-json", workflowId)` and
|
||||
compare the actual graph to the build task and checkpoint goal. Build/save
|
||||
dependent persisted workflow with `workflows(action="get-as-code", workflowId)` or
|
||||
the bound workspace source file, and compare the actual graph to the build task
|
||||
and checkpoint goal. Build/save
|
||||
success is not proof of workflow quality. If the saved workflow is only a draft,
|
||||
lacks the requested outcome, or verification evidence is weak, patch the same
|
||||
workflow in this checkpoint turn and re-read/re-verify it. If a dependency outcome
|
||||
|
|
|
|||
|
|
@ -55,9 +55,10 @@ workflow as a precondition for running it.
|
|||
`setupRequirement` from the tool output. If the output is missing a
|
||||
`workflowId`, explain that the build did not submit.
|
||||
- Before treating a saved workflow as done, inspect the persisted workflow
|
||||
with `workflows(action="get-json", workflowId)` and compare the actual
|
||||
graph to the user's requested outcome. Build/save success only means a
|
||||
workflow was saved; it does not prove the saved workflow is good.
|
||||
with `workflows(action="get-as-code", workflowId)` or read the bound
|
||||
workspace source file, and compare the actual graph to the user's requested
|
||||
outcome. Build/save success only means a workflow was saved; it does not
|
||||
prove the saved workflow is good.
|
||||
- If the persisted workflow is missing the requested outcome, has an obvious
|
||||
dead-end draft shape, or the verification evidence is weak, load the
|
||||
`workflow-builder` skill and patch the same workflow with `build-workflow`
|
||||
|
|
|
|||
|
|
@ -23,8 +23,8 @@ recommended_tools:
|
|||
# Workflow Builder
|
||||
|
||||
You are an expert n8n workflow builder. You generate complete, valid
|
||||
TypeScript code using `@n8n/workflow-sdk` for new workflows, and you edit
|
||||
WorkflowJSON workspace files for existing saved workflow changes.
|
||||
TypeScript code using `@n8n/workflow-sdk` for new workflows and for existing
|
||||
saved workflow changes.
|
||||
|
||||
This skill runs inside the orchestrator. It does not introduce a separate
|
||||
builder agent, delegated handoff, or separate tool allowlist. Use the
|
||||
|
|
@ -34,9 +34,10 @@ search, use it when it helps complete the build.
|
|||
|
||||
For clear new single-workflow requests, write or edit a TypeScript SDK source
|
||||
file in the workspace, then build directly with `build-workflow({ filePath })`.
|
||||
For existing saved workflow edits, write the current WorkflowJSON to a
|
||||
`.workflow.json` workspace file, make the requested selective JSON edit there,
|
||||
then call `build-workflow({ filePath, workflowId })` the first time. Do not load
|
||||
For existing saved workflow edits, call
|
||||
`workflows(action="get-as-code", workflowId)`, write the returned code to a
|
||||
`.workflow.ts` workspace file, make the requested edit there, then call
|
||||
`build-workflow({ filePath, workflowId })` the first time. Do not load
|
||||
`planning` or call `create-tasks` first. Only load `planning` when the
|
||||
orchestrator routing rules require coordinated multi-artifact work. Use this
|
||||
skill during an approved `<planned-task-follow-up type="build-workflow">` turn,
|
||||
|
|
@ -50,70 +51,14 @@ Do not call `workflows(action="update")` for workflow-building or existing
|
|||
workflow edits. Existing edits must go through a workspace source file and
|
||||
`build-workflow`.
|
||||
|
||||
## Output Discipline
|
||||
|
||||
- Your text output is visible to the user. Be concise and natural.
|
||||
- Only output text for errors that need attention, or a brief natural completion
|
||||
message.
|
||||
- No emojis, no filler phrases, no markdown headers in your text output.
|
||||
- When conversation context is provided, use it to continue naturally. Do not
|
||||
repeat information the user already knows.
|
||||
|
||||
### No Narration
|
||||
|
||||
Do not announce what you are about to do. The user already sees tool calls in
|
||||
real time. Stay silent while working; speak only on completion or when blocked.
|
||||
|
||||
Bad:
|
||||
|
||||
- "I'll build this workflow. Let me start by discovering credentials..."
|
||||
- "I'll start by reading the current workflow code..."
|
||||
- "I don't see any pinData, so let me check..."
|
||||
|
||||
Good:
|
||||
|
||||
- "Workflow ready: Telegram messages are summarized and added to your table."
|
||||
- "Workflow updated: removed the stale pinData from the weather check node."
|
||||
- "Blocked: the Linear API credential is missing; setup is required before I can
|
||||
continue."
|
||||
|
||||
## Tool Surface
|
||||
|
||||
Tool names are part of the compatibility contract. Keep using the same tool
|
||||
names the old builder used:
|
||||
|
||||
- Runtime workspace file tools, typically `read_file`, `write_file`, and
|
||||
`edit_file`, to read, write, and edit workflow source files. If they are
|
||||
deferred behind tool search, search for the file-reading/writing/editing tool
|
||||
before building.
|
||||
- `build-workflow` to save a workspace source file by `filePath`.
|
||||
- `workflows(action="get")`, `workflows(action="list")`,
|
||||
`workflows(action="get-json")`, and `workflows(action="setup")` when
|
||||
inspection, existing workflow editing, or setup routing is needed.
|
||||
- `credentials(action="list" | "get" | "search-types" | "test")` for credential
|
||||
metadata and connection checks.
|
||||
- `nodes(action="suggested")` for known workflow categories.
|
||||
- `nodes(action="search")` for service-specific node discovery.
|
||||
- `nodes(action="type-definition")` for exact parameter names, enum values,
|
||||
credential types, display conditions, and `@builderHint` annotations.
|
||||
- `nodes(action="explore-resources")` for live credential-backed resource lists.
|
||||
- `data-tables(action="list" | "create" | "schema")` for Data Table work.
|
||||
- `parse-file` for parseable user attachments.
|
||||
- `research` for external documentation when node definitions are insufficient.
|
||||
- `ask-user` only when a human choice is needed.
|
||||
- `executions` and `verify-built-workflow` for verification when the current
|
||||
turn is responsible for verification.
|
||||
- `complete-checkpoint` and `report-verification-verdict` only in checkpoint
|
||||
follow-up turns.
|
||||
|
||||
## Repair Strategy
|
||||
|
||||
When called with failure details for an existing workflow, start from the
|
||||
workspace source file if one is available in the conversation or tool output. If
|
||||
you only have a saved n8n workflow ID, use `workflows(action="get-json")`, write
|
||||
the returned JSON to a stable `src/workflows/<name>.workflow.json` file, make
|
||||
the smallest requested edit in that JSON file, then call `build-workflow` with
|
||||
both `filePath` and `workflowId` once. Later repairs should reuse the same
|
||||
you only have a saved n8n workflow ID, use `workflows(action="get-as-code")`,
|
||||
write the returned code to a stable `src/workflows/<name>.workflow.ts` file, make
|
||||
the smallest requested edit in that file, then call `build-workflow` with both
|
||||
`filePath` and `workflowId` once. Later repairs should reuse the same
|
||||
`filePath`; `build-workflow` remembers the bound workflow ID.
|
||||
|
||||
For repairs, edit the workspace file directly and call `build-workflow` again
|
||||
|
|
@ -185,16 +130,15 @@ When mapping downstream fields from an OpenAI node, read
|
|||
mandatory for calendars, spreadsheets, channels, folders, databases, models,
|
||||
and any other list-backed parameter when a credential is available.
|
||||
6. Pick a stable workspace `filePath` for the source file, typically
|
||||
`src/workflows/main.workflow.ts` for a one-off new workflow, a clearly named
|
||||
`.workflow.ts` file when multiple new source files are useful, or a clearly
|
||||
named `.workflow.json` file for existing saved workflow edits. For an
|
||||
`src/workflows/main.workflow.ts` for a one-off new workflow, or a clearly
|
||||
named `.workflow.ts` file when multiple source files are useful. For an
|
||||
existing workflow with no source file in context, call
|
||||
`workflows(action="get-json", workflowId)`, write the returned JSON to the
|
||||
chosen `.workflow.json` file, and pass the n8n `workflowId` only on the first
|
||||
`workflows(action="get-as-code", workflowId)`, write the returned code to the
|
||||
chosen `.workflow.ts` file, and pass the n8n `workflowId` only on the first
|
||||
`build-workflow` call.
|
||||
7. Write complete TypeScript SDK code to the workspace `filePath` for new
|
||||
workflows, or read and selectively edit the WorkflowJSON file for existing
|
||||
workflow changes. Do not put secrets in the source file.
|
||||
7. Write complete TypeScript SDK code to the workspace `filePath`, or read and
|
||||
selectively edit the existing `.workflow.ts` file for workflow changes. Do
|
||||
not put secrets in the source file.
|
||||
8. Call `build-workflow` with `filePath`.
|
||||
For planned build follow-ups where `buildTask.isSupportingWorkflow === true`,
|
||||
pass `isSupportingWorkflow: true`; that saved supporting workflow is the
|
||||
|
|
@ -207,10 +151,11 @@ When mapping downstream fields from an OpenAI node, read
|
|||
10. Fix errors by editing the same workspace source file and calling
|
||||
`build-workflow` again with the same `filePath`. Save again before any
|
||||
verification step.
|
||||
11. Modify existing workflows by editing the workspace source file. If the file
|
||||
was created from `workflows(action="get-json")`, pass the real n8n
|
||||
`workflowId` on the first `build-workflow` call so the file is bound to the
|
||||
saved workflow. Never pass local SDK workflow IDs as n8n workflow IDs.
|
||||
11. Modify existing workflows by editing the workspace `.workflow.ts` source
|
||||
file. If the file was created from `workflows(action="get-as-code")`, pass
|
||||
the real n8n `workflowId` on the first `build-workflow` call so the file is
|
||||
bound to the saved workflow. Never pass local SDK workflow IDs as n8n
|
||||
workflow IDs.
|
||||
12. Finish with a concise completion message only when the build, required
|
||||
setup routing, or required verification path is complete.
|
||||
|
||||
|
|
@ -230,10 +175,10 @@ Use the current turn's higher-priority instructions to decide who verifies:
|
|||
|
||||
Build/save success is not workflow-quality evidence. When this turn is
|
||||
responsible for verification or repair, inspect the persisted workflow with
|
||||
`workflows(action="get-json", workflowId)` after saving or before reporting a
|
||||
verdict. Judge the saved graph against the user's requested outcome and the
|
||||
current build/checkpoint goal, not a hidden service-specific or topology
|
||||
checklist.
|
||||
`workflows(action="get-as-code", workflowId)` or read the bound workspace
|
||||
source file after saving or before reporting a verdict. Judge the saved graph
|
||||
against the user's requested outcome and the current build/checkpoint goal, not
|
||||
a hidden service-specific or topology checklist.
|
||||
If the saved workflow is only a draft, misses the intended outcome, or has weak
|
||||
evidence, edit the same workflow source file and call `build-workflow` with the
|
||||
same `filePath`, then inspect and verify again.
|
||||
|
|
|
|||
|
|
@ -110,16 +110,14 @@ describe('Instance AI runtime skills', () => {
|
|||
const loaded = await source.loadSkill('workflow-builder');
|
||||
expect(loaded?.instructions).toContain('build-workflow');
|
||||
expect(loaded?.instructions).toContain('filePath');
|
||||
expect(loaded?.instructions).toContain('read_file');
|
||||
expect(loaded?.instructions).toContain('write_file');
|
||||
expect(loaded?.instructions).toContain('edit_file');
|
||||
expect(loaded?.instructions).toContain('runtime workspace file tools');
|
||||
expect(loaded?.instructions).toContain('workspace source file');
|
||||
expect(loaded?.instructions).toContain('nodes(action="suggested")');
|
||||
expect(loaded?.instructions).toContain('nodes(action="search")');
|
||||
expect(loaded?.instructions).toContain("newCredential('Credential Name', 'credential-id')");
|
||||
expect(loaded?.instructions).toContain('Verification');
|
||||
expect(loaded?.instructions).toContain('Build/save success is not workflow-quality evidence');
|
||||
expect(loaded?.instructions).toContain('workflows(action="get-json", workflowId)');
|
||||
expect(loaded?.instructions).toContain('workflows(action="get-as-code", workflowId)');
|
||||
expect(loaded?.instructions).toContain(
|
||||
'knowledge-base/reference/workflow-builder-guardrails.md',
|
||||
);
|
||||
|
|
|
|||
|
|
@ -198,13 +198,13 @@ describe('workflows tool', () => {
|
|||
expect(context.workflowService.publish).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should allow JSON inspection but reject raw update on orchestrator surface', () => {
|
||||
it('should allow code inspection but reject raw update on orchestrator surface', () => {
|
||||
const context = createMockContext();
|
||||
const tool = createWorkflowsTool(context, 'orchestrator');
|
||||
const schema = getInputSchema(tool);
|
||||
|
||||
expect(schema.safeParse({ action: 'get-json', workflowId: 'w1' }).success).toBe(true);
|
||||
expect(schema.safeParse({ action: 'get-as-code', workflowId: 'w1' }).success).toBe(false);
|
||||
expect(schema.safeParse({ action: 'get-as-code', workflowId: 'w1' }).success).toBe(true);
|
||||
expect(
|
||||
schema.safeParse({
|
||||
action: 'update',
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ export const EVAL_SETUP_AGENT_PROMPT = `You are an eval setup specialist for n8n
|
|||
|
||||
## Mandatory Process
|
||||
|
||||
1. **Read the workflow** via \`workflows(action="get-json", workflowId)\` using the workflowId in the task. Identify the AI agent nodes named in the task. Trace the main trigger path.
|
||||
1. **Read the workflow** using the workflows tool with the workflowId in the task. Identify the AI agent nodes named in the task. Trace the main trigger path.
|
||||
2. **Use the DataTable id from the task.** The task always names an existing DataTable id under "Wire the EvaluationTrigger to DataTable id ...". Use it as-is. Do not create, modify rows, or modify schema. If the task says to leave it empty (the \`later\` path), set the \`EvaluationTrigger.dataTableId\` to an empty string and report that the user must wire it manually.
|
||||
3. **Patch the workflow.** Build the eval topology in this order:
|
||||
a. Insert \`EvaluationTrigger\` (\`name: "Eval Trigger"\`).
|
||||
|
|
@ -24,7 +24,7 @@ export const EVAL_SETUP_AGENT_PROMPT = `You are an eval setup specialist for n8n
|
|||
d. After the agent: insert \`Evaluation(checkIfEvaluating)\` (no separate IF node — it has two native main output slots). Slot 0 (Evaluation) → \`Evaluation(setOutputs)\` → one \`Evaluation(setMetrics)\` per metric listed under "Metrics". Slot 1 (Normal) preserves the original production downstream path with side-effects.
|
||||
e. For \`correctness\` and \`helpfulness\` metrics: wire an additional outgoing \`ai_languageModel\` connection from the workflow's existing LLM model node to each setMetrics node that uses an AI-judged metric. The LLM gets reused — same node, additional connection. Without this, AI-judged metrics fail silently.
|
||||
4. **Save** the modified workflow via \`workflows(action="update", ...)\`.
|
||||
5. **Validate**: re-read the workflow via \`workflows(action="get-json", workflowId)\` and assert:
|
||||
5. **Validate**: re-read the workflow using the workflows tool and assert:
|
||||
- EvaluationTrigger → target AI agent (direct \`main\` connection, no intermediate node).
|
||||
- Agent → checkIfEvaluating; slot 0 → setOutputs → setMetrics; slot 1 → original downstream path.
|
||||
- When input columns are non-empty, the agent's parameters contain at least one \`{{ $json.<column> }}\` expression matching a column from "Input columns".
|
||||
|
|
@ -224,7 +224,7 @@ Multiple AI agents: one \`checkIfEvaluating + setOutputs + setMetrics\` block pe
|
|||
## Error Handling & Validation
|
||||
|
||||
After patching:
|
||||
1. Re-read the workflow: \`workflows(action="get-json", workflowId)\`.
|
||||
1. Re-read the workflow using the workflows tool.
|
||||
2. Assert EvaluationTrigger connects DIRECTLY to the target AI agent (no intermediate node on the eval branch).
|
||||
3. Assert connections after the agent: agent → checkIfEvaluating; slot 0 → setOutputs → setMetrics (one per metric); slot 1 → original downstream path.
|
||||
4. When the task contained a \`PRODUCTION ADAPTER\` section: assert
|
||||
|
|
|
|||
|
|
@ -288,7 +288,7 @@ function getSupportedWorkflowActionSchemas(
|
|||
list: listAction,
|
||||
get: getAction,
|
||||
'get-json': getJsonAction,
|
||||
...(surface !== 'orchestrator' ? { 'get-as-code': getAsCodeAction } : {}),
|
||||
'get-as-code': getAsCodeAction,
|
||||
delete: deleteAction,
|
||||
unarchive: unarchiveAction,
|
||||
setup: setupAction,
|
||||
|
|
|
|||
|
|
@ -157,7 +157,7 @@ describe('formatWorkflowLoopGuidance', () => {
|
|||
workflowId: 'wf-789',
|
||||
};
|
||||
const result = formatWorkflowLoopGuidance(action);
|
||||
expect(result).toContain('workflows(action="get-json")');
|
||||
expect(result).toContain('workflows(action="get-as-code", workflowId)');
|
||||
expect(result).toContain('Build/save success only means a workflow was saved');
|
||||
expect(result).toContain('verify-built-workflow');
|
||||
expect(result).toContain('executions(action="run")');
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ export function formatWorkflowLoopGuidance(
|
|||
}
|
||||
case 'verify':
|
||||
return (
|
||||
`VERIFY: Inspect the persisted workflow ${action.workflowId} with \`workflows(action="get-json")\` and compare it to the requested outcome. ` +
|
||||
`VERIFY: Inspect the persisted workflow ${action.workflowId} with \`workflows(action="get-as-code", workflowId)\` or read the bound workspace source file, then compare it to the requested outcome. ` +
|
||||
'Build/save success only means a workflow was saved. ' +
|
||||
`If the build had mocked credentials, use \`verify-built-workflow\` with workItemId "${options.workItemId ?? 'unknown'}". ` +
|
||||
'Otherwise use `executions(action="run")`. ' +
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user