n8n/packages/@n8n/instance-ai/evaluations/data/workflows/dont-ask-technical-implementation-choice.json
José Braulio González Valido 65f375bbd4
feat(ai-builder): Make eval executionScenarios optional for build-only cases (no-changelog) (#33252)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:41:14 +00:00

38 lines
2.5 KiB
JSON

{
"description": "INS-666: When a built-in node can't supply a needed field, the agent must pick the implementation itself (e.g. fall back to an HTTP Request to the service's API) instead of asking the user to approve a technical/node-choice decision. The built-in Linear node's 'Get many issues' doesn't return labels or the issue description, so filtering by the 'bug' label and including descriptions forces an implementation fork the agent should resolve on its own — not surface as a 'is this approach acceptable?' question.",
"conversation": [
{
"role": "user",
"text": "Every weekday at 9am, fetch all my Linear issues that have the 'bug' label, including each issue's description, and post a summary of them to Slack #bugs."
},
{
"role": "assistant",
"text": "I'll build that for you now."
},
{
"role": "user",
"text": "[You are a non-technical user. You don't know n8n's nodes or APIs and won't make build decisions. If the agent asks you to choose or approve anything technical — which node to use, whether to use an HTTP Request vs a built-in node, which API or endpoint (e.g. a GraphQL API), data formats — do not answer it: reply 'You're the expert — just decide for me.' and never pick. Only answer normally if it asks about what you want (your goal, schedule, channel, which Linear team/project).]"
}
],
"messageBudget": 4,
"complexity": "medium",
"tags": [
"behaviour",
"clarification",
"question-tool",
"technical-decision",
"schedule",
"linear",
"slack"
],
"triggerType": "schedule",
"datasets": ["behaviour", "full"],
"processExpectations": [
"The agent did not ask the user to approve or choose a technical implementation detail — for example which node type or API to use, whether to use the built-in Linear node or an HTTP Request to Linear's API (e.g. its GraphQL API), or which endpoint. It made any such decision itself.",
"Any questions the agent did ask were about the user's goal or desired outcome, never about how to build the workflow technically. The agent never described an implementation approach and asked whether it was acceptable."
],
"outcomeExpectations": [
"The built workflow triggers on a weekday 9am schedule, retrieves Linear issues filtered to the 'bug' label and including each issue's description (via whichever node or HTTP Request achieves this), and posts a summary of them to Slack #bugs. Unset or placeholder credential values (e.g. an unset Linear team or Slack channel) are acceptable — they are values to fill in at setup, not build mistakes."
]
}