From 933113e375fd8889f4dcf552720e02dca0ca516e Mon Sep 17 00:00:00 2001 From: Pierantonio Cangianiello <10360978+pcan@users.noreply.github.com> Date: Tue, 29 Sep 2026 14:12:41 +0000 Subject: [PATCH] docs(engine): Add ADR on deletion of binary files written by Engine v2 runs (no-changelog) (#39617) --- ...iles-with-the-execution-that-wrote-them.md | 163 ++++++++++++++++++ 1 file changed, 163 insertions(+) create mode 100644 packages/@n8n/engine/docs/adr/ADR-20260925-delete-binary-files-with-the-execution-that-wrote-them.md diff --git a/packages/@n8n/engine/docs/adr/ADR-20260925-delete-binary-files-with-the-execution-that-wrote-them.md b/packages/@n8n/engine/docs/adr/ADR-20260925-delete-binary-files-with-the-execution-that-wrote-them.md new file mode 100644 index 00000000000..914f6eca97f --- /dev/null +++ b/packages/@n8n/engine/docs/adr/ADR-20260925-delete-binary-files-with-the-execution-that-wrote-them.md @@ -0,0 +1,163 @@ +# Delete binary files with the execution that wrote them + +Date: 2026-09-25 + +Status: Active + +Decision Owner: Catalysts + +## Context + +A node that produces a file writes it to the binary data store at a location that is +execution-specific for each run. The location is identified by the pair `(workflowId, executionId)`, +and `FileLocation.ofExecution()` in `n8n-core` builds it. In the `filesystem` mode (which is the +default) every file produced during a run is stored under one directory named after that pair. + +In engine v1, the execution row in the CP database is the reference for deletion. The pruning +service soft-deletes old rows. Later, `ExecutionPersistence.hardDelete()` deletes the rows and calls +`BinaryDataService.deleteMany()` with the execution locations. The blob manager deletes the +directory of each location by prefix. Only the filesystem byte store implements prefix deletion. For +`s3` and `azure` the call deletes nothing, and bucket lifecycle rules are the only deletion path. + +In engine v2, the DP stores the execution and its steps in its own database and they are not tracked +in the CP database. The node-engine-compatibility layer runs a v1 node with the v2 execution id & +workflow id under `additionalData`, so a v2 run writes files using the same location pattern as v1, +but no job deletes them. The engine package has no pruning code yet: CAT-2939 (execution pruning and +retention on the DP) is in Backlog. CAT-4584 and CAT-4721 made the Respond to Webhook node work with +binary bodies on v2, so a v2 run in `filesystem` mode would already leave files on disk with no +owner. + +There are three constraints for the design: +1. The engine package must not import `n8n-core` or `n8n-workflow`. The engine calls host + capabilities through `ExternalDependencies`, which today has two optional members: the `v1-node` + step executor and the lifecycle event callback. +2. Lifecycle events are best effort. The batching publisher drops a batch that fails or times out, + and drops new events past a pending cap. Reliable deletion needs at-least-once delivery, instead. +3. The DP exposes `POST /api/workflow-executions/search`. It takes a workflow filter that accepts + all workflows, `createdBefore`, a status list, a cursor and a page limit of 100. The CP has a + client method for it, but the endpoint lists executions that exist and does not report executions + that were pruned. + +The four binary data modes differ. + +1. `default` keeps the bytes inline in the items. It is deprecated and will be removed, so it is + out of scope here. +2. `filesystem` needs the writer and the deleter to see the same disk. CAT-4721 already states this + for the reader. +3. `s3` and `azure` need the same bucket and credentials on both sides to read a file. Deletion is + not affected today, because the call deletes nothing in these modes. +4. `database` stores the bytes in the CP database, so a deleter outside the CP process needs CP + database access. + +## Decision + +1. **Each execution owns its files.** For v2, a binary file only belongs to the execution (no other + refs to it), so when the execution is deleted, its files need to be deleted accordingly. There + should be no separate per-file bookkeeping. +2. **Ownership passes to the execution when the run starts.** The call that hands a run to the DP + is `startExecution`. Before it, the CP owns every file the start node wrote, under the temporary + path or under the execution path, and deletes them when the run does not start: a refused + payload, a failed dispatch, or a start rejected after a rename. After it, the files belong to the + execution, and the retention job of CAT-2939 decides when they go. From then on no file is + distinguished by the process that wrote it: a trigger file the CP renamed and a file a step + wrote on the DP are deleted by the same call, because both are under the execution path. This + holds because CP and DP actions on files do not depend on the deployment model: both planes + write to and delete from the same store. With `filesystem` the operator should mount one shared + network volume on every host, and every process points at the same mount path. Without that + mount the two planes see different disks, and no rule in this document holds. +3. **The DP decides when to delete.** The DP owns the lifetime of a v2 execution, so its retention + job (CAT-2939) is the only component that knows when an execution is pruned. The job selects the + prunable executions and triggers the (idempotent) deletion of their files before it deletes the + rows. +4. **The host performs the deletion.** The DP stays unaware of the store. A new callback is added + to `ExternalDependencies`, that receives the `(workflowId, executionId)` pair of one execution + and deletes the files at that location. In integrated mode, `packages/cli` implements it with + `BinaryDataService.deleteMany()` and `FileLocation.ofExecution()`, the same call the v1 hard + delete makes. The callback is optional, since a host that runs no v1 nodes writes no files and + can leave it out. Every other host supplies it with a `BinaryDataService` for the store its nodes + write to. +5. **At least once, and idempotent.** The job deletes the rows of an execution only after the + callback has returned for it. When the callback throws, the job logs the error, keeps the rows, + and retries that execution on its next run. A repeat call is safe: prefix deletion in the + filesystem byte store removes the directory with `force`, so a missing directory is not an error. + This is stricter than v1, which logs a failed hard delete and does not retry the files, because + v2 has no CP row to retry from. +6. **Per mode.** `filesystem` runs the same call as v1 and deletes the directory of the run. `s3` + and `azure` run the same call, but it deletes nothing today, so those modes depend on bucket + lifecycle rules as in v1. `database` deletes the rows of the run through the database manager. + The `database` mode is supported in integrated mode only. An out-of-process DP in `database` mode + waits for the decision on binary data access from an out-of-process DP. +7. **Files written before this is implemented** stay in the store until an operator deletes them. + Engine v2 is behind a module flag and is in its internal release phase, so no migration or sweep + is planned. + +```mermaid +sequenceDiagram + participant Job as DP retention job (CAT-2939) + participant Host as Host callback (cli) + participant Store as BinaryDataService + participant DB as DP database + Job->>DB: select prunable executions + Job->>Host: delete files of (workflowId, executionId), one call per execution + Host->>Store: deleteMany(FileLocation.ofExecution(...)) + Store-->>Host: ok + Host-->>Job: ok + Job->>DB: delete execution and step rows + Note over Job: on callback error: log, keep rows, retry next run +``` + +## Alternatives Considered + +1. **A control plane job that lists old executions through the search endpoint and deletes their + files.** This needs no engine change, and the client method exists. It was not chosen because the + endpoint lists live executions and cannot name pruned ones. The CP job would need its own + retention window, shorter than the DP window, so that it always runs before the DP prunes. Two + windows in two configurations must stay aligned, and one missed CP run before a DP prune leaks + the files for good. The job would also page through every old execution of every workflow on each + run, one hundred at a time, to find the ones it has not handled. +2. **An `execution:pruned` lifecycle event, deleted on receipt by the control plane.** This fits + the relay that exists. It is the second choice because the publisher drops batches by design when + the host is slow or down. Deletion needs at-least-once delivery, so the event channel would need + a durable outbox and acknowledgements first. That is a larger change than one host callback, and + it would change the guarantee of every other event. If the event channel becomes durable later, + this option can replace the callback without a change to ownership. +3. **The data plane deletes the files with its own store client.** This is rejected because the + engine must not depend on `n8n-core`, and a second store implementation would have to track every + mode and every byte store. +4. **Bucket lifecycle rules only.** This is rejected because the `filesystem` and `database` modes + have no such rule, and `filesystem` is the default install. + +## Consequences + +1. `ExternalDependencies` will have one optional member. CAT-2939 must call it and order the + deletions as described. This decision must be agreed before CAT-2939 is implemented. +2. A slow or failing store slows pruning. An execution whose files cannot be deleted is kept, with + its rows, until the store recovers. The job must bound the batch it retries so that one bad + execution does not stop the others. +3. An out-of-process DP needs a `BinaryDataService` in its host. That requires the split of + `BinaryDataConfig` from `InstanceSettings` and the pending decision on binary data access from an + out-of-process DP. +4. Files written by v2 runs before the callback exists are not deleted. This is accepted for the + internal release. +5. A webhook or trigger node runs before the execution id exists. In v1 the file it stores is + written under a temporary execution id and renamed after the run. On v2 a webhook run can + create the id before the node runs and set it on `additionalData`, so the file is written under + its final path from the start. A trigger node runs on its own schedule, so its file is already + under the temporary path when the control plane receives its items, and the control plane + renames it at dispatch, before the run starts. Both changes are not covered here and belong to + the follow-up ticket that accepts files on v2. +6. Files under the temporary path that no run claimed are the CP's to delete. Every refusal path + deletes what the node stored, as the payload guard does today. A crash between the write and the + dispatch leaves files that nothing deletes. No sweep for those exists today, in v1 or v2. + CAT-4756 tracks it. +7. The retention note ("Retention is unsolved") in + ADR-20260904-store-the-workflow-revision-that-ran-with-the-execution stays open for the + workflow snapshot. This decision covers binary files only. + +## Links + +RFC: - + +Documentation: https://linear.app/n8n/issue/CAT-4727 + +Related ADRs: ADR-20260904-store-the-workflow-revision-that-ran-with-the-execution