n8n/packages/@n8n/instance-ai/evaluations/data/workflows/user-impatient.json
Arvin A e6dee743e6
test(ai-builder): Add advanced user-behavior multi-turn eval cases (no-changelog) (#31758)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: José Braulio González Valido <jose.gonzalez@n8n.io>
Co-authored-by: José Braulio González Valido <josebragv@gmail.com>
2026-07-09 18:13:42 +00:00

37 lines
2.4 KiB
JSON

{
"description": "The user is impatient and delegates every decision — they skip clarification on purpose. The agent should pick sensible defaults and build rather than blocking on specifics, and the proxy must keep delegating instead of inventing concrete answers.",
"conversation": [
{
"role": "user",
"text": "I've got an incoming webhook with order data. Set it up to receive and store the orders — build it however makes sense with sensible defaults, don't ask me a bunch of questions."
},
{
"role": "user",
"text": [
"[Stay impatient and hands-off for the whole conversation. If the agent asks you to choose or specify any detail — where to store the orders, the schema, field mappings, which service — don't engage with the specifics.",
"Reply in one short sentence that you trust its judgement and it should just use sensible defaults and build it. Don't enumerate options or invent concrete values, and don't change your mind and start specifying things later.",
"Never supply a credential.]"
]
}
],
"messageBudget": 4,
"complexity": "simple",
"tags": ["build", "webhook", "behaviour", "impatient"],
"triggerType": "webhook",
"datasets": ["behaviour", "full"],
"processExpectations": [
"The agent did not pepper the user with a string of detailed configuration questions — when the user delegated, it proceeded with sensible defaults rather than blocking on specifics."
],
"outcomeExpectations": [
"The agent built a working webhook-to-storage workflow with a reasonable storage choice of its own, since the user explicitly handed it the decision."
],
"executionScenarios": [
{
"name": "happy-path",
"description": "Webhook receives an order payload and the agent's chosen storage persists it",
"dataSetup": "The webhook trigger receives a POST with an order payload: { \"orderId\": \"ORD-1042\", \"customer\": \"alice@example.com\", \"total\": 89.99, \"currency\": \"USD\", \"items\": [{ \"sku\": \"SKU-1\", \"qty\": 2 }] }. Whatever storage node the agent chose (database, Google Sheet, Airtable, etc.) returns a success response.",
"successCriteria": "The workflow executes without errors. The incoming order payload is persisted to whatever storage destination the agent chose with sensible defaults — the exact destination is not prescribed, since the user delegated the decision. The order's identifying fields (e.g. orderId, customer, total) reach the storage node."
}
]
}