The n8n Compendium — Volume D: Shaping the Data

The item model, expressions, the transformation toolkit, code nodes, and files and state.

A Signal Through field guide. Independent, plain-English documentation; not affiliated with or endorsed by n8n GmbH. Approximately 22015 words.

In this volume:

• Reading and Reasoning About Data: The Item Model in Practice
• Expressions: The Double-Curly-Brace Language
• The Transformation Toolkit
• Code and Command: When Clicks Are Not Enough
• Files and Remembered State: Binary Data and Data Tables

Reading and Reasoning About Data: The Item Model in Practice

Chapter 3 (The Core Mental Model) introduced the idea that everything moving through an n8n workflow is a list of items — small parcels of structured data that nodes receive, work on, and pass along. That was theory. This chapter is the practice of looking: opening a node, reading what actually came out of it, and reasoning correctly about what you see before you touch anything.

That last part matters. Everything in this chapter is diagnosis by inspection. You will not change a single value here — no transformations, no expressions that rewrite fields, no code. Those come in Chapters 17 through 20, and every one of them assumes you can already do what this chapter teaches: look at a node's output and answer, with confidence, four questions. How many items are there? What is the shape of each one? Where did each one come from? And is that shape what the next node expects? Most broken workflows are broken because the builder guessed at one of those answers instead of checking.

The Anatomy of an Item

An item is not just "some JSON." It is a small envelope with a defined structure, and n8n cares about that structure more than it first appears. Every item has up to four top-level compartments:

The json compartment is the one you will spend most of your time reading. It must be a JSON object — a set of named fields — not a bare string, number, or array. A contact item might carry { "name": "Dana", "email": "dana@example.com", "tags": ["vip", "newsletter"] }. The fields can nest as deep as they need to; the only rule is that the top level of json is an object.

The binary compartment is the attachment slot. When a node downloads a PDF, receives an email attachment, or generates a spreadsheet, the file does not get stuffed into json — it rides alongside it in binary, under a named property (conventionally data when there is only one file, but any name is legal, and one item can carry several files under several names). Each binary entry stores the file content plus metadata: a MIME type (the standard label describing what kind of file it is, like application/pdf), a filename, and usually a size. Chapter 20 (Files and Remembered State) covers working with binary data in depth; here you only need to recognize it when you see it.

The pairedItem compartment is invisible in normal reading but is the star of a later section: it is how n8n remembers lineage — which input item produced which output item. When it is intact, expressions that reach back to earlier nodes work effortlessly. When it is lost, they fail with some of n8n's most confusing error messages. We will come back to it.

One vocabulary point before moving on, because it trips up nearly everyone once: the difference between one item containing a list and a list of items. If an API returns fifty orders, a node might emit fifty items, each with one order in its json — or one single item whose json contains an array field with fifty orders inside it. Downstream nodes behave completely differently in the two cases, because nodes run their logic once per item. Fifty items means fifty rows processed individually; one item with an embedded array means one execution that sees the whole list as a single blob. Telling these apart at a glance is the single most valuable skill this chapter teaches, and the item count badge described next is how you do it.

Where to Look: The Node's Input and Output Panels

Double-click any node on the canvas and it opens into its detail view (n8n's term for it is the node detail view, or NDV). The layout is consistent across nodes: the node's settings sit in the middle, the data flowing in is shown on one side, and the data flowing out on the other. If the workflow has not run yet, these panels are empty — inspection always works on real data from a real execution, whether that was a test run in the editor or a past production execution you have opened from the executions list (Chapter 21 covers retrieving those). The quickest way to get fresh data to read is the node's own test-run control — the Execute step button in the detail view — which runs just enough of the workflow to feed this node and fills both panels. Remember that a test run is still a real run: a node that sends an email will send it, so for repeated study of the same data, the pinning technique from Chapter 10 (Test, Fix, Activate, and Keep It Tidy) lets you capture output once and reuse it without calling the outside world again.

At the top of the output panel you will find the item count — "5 items," "1 item," "No output data." Read this number first, every time, before you read any values. It answers the fifty-orders question instantly: fifty items or one item? It also catches an entire family of bugs where a filter upstream silently removed everything and the node ran on nothing.

When a node ran more than once within a single execution — which happens inside loops (Chapter 9) — the output panel gains a run selector, letting you flip between "1 of 4," "2 of 4," and so on. Each run has its own item list. If a node's output looks inexplicably wrong, check whether you are looking at the run you think you are.

Tip: The output panel of one node and the input panel of the next node show the same data, but the input panel is where you will later drag fields from when mapping values into parameters (Chapter 8 covers the mechanics, Chapter 17 the expressions it generates). Get in the habit of reading data in the input panel of the node you are configuring — it keeps your eyes and your mapping in the same place.

Both panels offer a search box for finding a value across items — invaluable when you have two hundred items and need the one containing a particular email address — and both let you switch between the three views that are the heart of this chapter.

The Three Views: Table, JSON, and Schema

n8n renders the same items three different ways, and each view answers a different question. Fluent builders flip between them constantly.

Table view: the spreadsheet lens

Table view lays items out as rows and fields as columns, like a spreadsheet. It is the fastest way to answer "do all my items have an email address?" — scan the column, spot the blanks. It is also the honest witness for item counts: each row is one item, no ambiguity.

Its weakness is depth. A field whose value is a nested object or an array cannot be shown in a cell, so Table view abbreviates it — you may see a compressed preview, or the classic placeholder [object Object]. This is not corruption. It is Table view telling you "there is structure here I cannot flatten; switch views to see it."

Watch out: Seeing [object Object] or a truncated blob in a Table cell does not mean the data is broken, and "fixing" it is not the move. The data underneath is intact — Table view simply cannot draw a tree inside a cell. Switch to JSON view before drawing any conclusion about what that field contains.

JSON view: the ground truth

JSON view shows the items exactly as they are: the raw JSON, nesting included. When precision matters — is that field a string "42" or a number 42? is address an object or an array of objects? — JSON view is the only view that tells the whole truth. Type distinctions that Table view visually smooths over are explicit here: quotes mean string, no quotes mean number or boolean, square brackets mean array, curly braces mean object. Chapter 17's expressions live and die on these distinctions, so learn to read them now.

JSON view is also where you copy things. Hovering over parts of the data offers copy actions — the value itself, or the path to it (the dotted route from the item's root to that field, like customer.address.city). Copying the path is the reliable way to get a deeply nested field's exact address without hand-typing it and guessing wrong on a level.

Tip: When a downstream expression is misbehaving, come back to the source node's JSON view and copy the field's path directly rather than transcribing it by eye. A large share of "expression returns nothing" reports are one-character path typos, and copying eliminates the whole category.

Schema view: the map without the territory

Schema view discards the values and shows you the structure: every field name, organized as a collapsible tree that mirrors the nesting, with an indication of each field's type. Where Table view is best for a hundred items and JSON view for one item, Schema view is best for no particular item — it answers "what does data from this node look like in general?"

It is also interactive: fields in the Schema (and Table) presentation of an input panel can be dragged straight into a node's parameter fields to create a mapping, which Chapter 17 explores. For pure reading purposes, Schema view is the fastest way to orient yourself in a big unfamiliar payload (the block of data a system delivers in one go) — an API response with sixty fields across four levels of nesting is unreadable as raw JSON but perfectly navigable as a collapsible tree.

One caution: Schema view is derived from the data actually present in this execution. If a field is optional and none of the current items happen to include it, it will not appear in the schema. Absence from Schema view means "not in this data," not "does not exist."

Reading Nested Objects and Arrays

Real-world payloads are rarely flat. An order from an e-commerce API might carry a customer object, which contains an address object, plus a lineItems array where each element is itself an object with its own fields. Reading these confidently is a matter of knowing what each view does with depth.

In JSON view, nesting is literal: objects inside objects, indented. Count the braces. In Schema view, nesting becomes indentation in the tree — expand customer, then address, and you can see city sitting two levels down. In Table view, nesting is where things get abbreviated, as covered above.

Arrays deserve special attention because they come in two flavors that read very differently:

Arrays of scalars — a scalar being a single plain value like a string or number — look like "tags": ["vip", "newsletter"]. These are simple lists of values; they usually display inline and cause little confusion.

Arrays of objects — "lineItems": [{ "sku": "A-1", "qty": 2 }, { "sku": "B-9", "qty": 1 }] — are lists of records embedded inside a single item. This is the fifty-orders trap in miniature. Those line items are not items in n8n's sense; they are cargo inside one item's json. A downstream node will see one item, not two line items, unless something deliberately splits them out (the transformation toolkit in Chapter 18 has a node for exactly that — here, you only need to recognize the situation).

Watch out: The most common data-reading mistake in n8n is treating an embedded array as if it were multiple items. If the item count badge says "1 item" but you can see twenty records in the JSON, those records are nested inside one item, and per-item operations downstream will run once, not twenty times. Diagnose it here; fix it in Chapter 18's territory.

When you inspect a webhook payload (Chapter 14) or an HTTP Request response (Chapter 13), spend your first thirty seconds on exactly this question: where is the boundary between "items" and "arrays inside an item"? APIs almost always return one big response object with the interesting records embedded in an array field, and different nodes make different choices about whether to split that for you. Never assume — read the count, then read the shape.

Binary Data: Reading the Attachment Slot

When items carry files, the output panel indicates it — items with binary attachments expose a binary section alongside the JSON views, showing each attachment's name, MIME type, filename, and size, usually with the option to preview (for common types like images and PDFs) or download the file for inspection.

Reading binary output well means checking three things. First, which named slot the file is in: nodes that consume binary data ask for the property name, and if the producing node put the file under data while the consuming node is configured to read attachment, you get a "no binary data" style error even though the file is plainly there. The binary section shows you the slot names; trust it over your memory. Second, the MIME type: a file labeled text/html that you expected to be application/pdf usually means the download upstream actually fetched an error page, not the document. Third, the size: a zero-byte or suspiciously tiny file is a failed transfer that only looks like it worked.

Note that an item's json and binary compartments are independent. An item can have rich JSON and no files, files and a nearly empty json, or both. The metadata about a file (its name, its type) lives in the binary entry itself, not in json — though nodes sometimes copy some of it into json for convenience. Chapter 20 takes over from here for actually manipulating files.

Item Linking: How n8n Remembers Where Data Came From

Now for the machinery that makes n8n's expressions feel magical when it works and baffling when it does not.

As items flow through a workflow, each node's output items carry a pairedItem marker recording which input item they were derived from. Item three out of a formatting node points back to item three that came in. If a node splits one input item into five output items, all five point back to that one parent. If a node merges several inputs into one output, the output can point back to several parents. Chained node after node, these markers form an unbroken ancestry line from any item at the end of a workflow all the way back to the trigger.

Why does this exist? Because of a thing you will do constantly from Chapter 17 onward: writing an expression in a late node that references data from an earlier node — not the immediately previous one — with syntax along the lines of $('Fetch Customer').item.json.email. Read that as: "for the item currently being processed, find its own ancestor in the Fetch Customer node, and give me that ancestor's email field." The current node might be six steps downstream, with filters and formatting in between. n8n resolves the request by walking the pairedItem chain backward, hop by hop, until it reaches the named node. That is how the third customer in a batch of fifty gets matched with their email and not the first customer's — per-item ancestry, resolved automatically.

This works only while the chain is intact. Every standard node maintains the linkage as part of its job, and most of the time you never think about it. But the chain can be broken, and when it is, any downstream expression that needs to walk through the broken link fails — typically with an error saying n8n cannot determine which item to use, that paired item information is missing, or wording to that effect.

The usual chain-breakers, so you can recognize the crime scene:

Custom code that builds fresh items. A Code node (Chapter 19) that constructs brand-new output items from scratch, without declaring what they were derived from, produces orphans — items with no ancestry. Everything downstream of them can still reference each other, but nothing can walk back past the Code node.

Aggregation. Nodes that collapse many items into one — summaries, concatenations, batching combinations — inherently blur one-to-one lineage. When fifty items become one, "which single input item is this output's parent?" has no clean answer.

Some merge shapes. Combining two input streams (see the next section) can complicate ancestry, since an output item may descend from an item on each side. n8n's Merge node generally does the right bookkeeping, but merged lineage is more intricate, and expressions reaching back past a merge to one specific side are a classic confusion point.

Here is the diagnostic discipline, and it is pure looking: when you hit a paired-item error, the node showing the error is almost never the node that caused it. The error surfaces where an expression tried to walk the chain; the break happened somewhere upstream where the chain was cut. Open each node between the referenced node and the erroring node and check its output. The last node whose items still resolve cleanly, followed by the first node whose items do not, brackets your break — nearly always a Code node or an aggregation sitting right at that seam.

Watch out: Do not "fix" a paired-item error by changing the expression in the node where the error appears to something that merely stops erroring — like grabbing the first item from the referenced node regardless of ancestry. That silences the message while quietly matching every item with the same ancestor, which is a data-corruption bug wearing a fixed costume. Find the broken link upstream instead; Chapters 17 and 19 show the proper repairs on the expression and code sides respectively.

A related but distinct failure: referencing a node that never executed in this run — because it sits on a branch that was not taken, or downstream of a filter that dropped everything. The error message differs (it points out the node has no data or did not run), but the diagnostic move is the same: look at that node's output panel for this execution. "No output data" on a node you are referencing explains everything.

Many Doors: Multiple Inputs and Outputs

So far we have pictured nodes as pipes: one stream in, one stream out. Several important nodes have more doors than that, and reading their data requires knowing which door you are looking at.

Multiple outputs. An If node (Chapter 9) has two output connectors — one for items that passed its condition, one for items that failed. A Switch node has several, one per routing rule. The Loop Over Items node has one output that fires each batch around the loop and another that fires once when everything is done. And any node can grow an additional error output when configured to route failures instead of halting (Chapter 23's territory). On the canvas, these appear as separate labeled connection points on the node's edge. In the output panel, a multi-output node lets you switch between outputs — each has its own item list and its own count. When a branch seems to be receiving nothing, open the branching node and check the per-output counts: it is common to discover all items took the other door because a condition was inverted or a value was a string where a number was expected (JSON view, again, settles that).

Multiple inputs. The Merge node is the everyday example: it takes two input streams by default — recent versions let you raise the number of inputs — and combines them according to its settings: appending one after the other, pairing them up positionally, or joining them on a matching field. Compare Datasets similarly takes two inputs. When you open a multi-input node, the input panel lets you view each input separately. Read both before reasoning about the output: half of all merge surprises are one input being empty or shaped differently than assumed, which is visible in ten seconds of looking.

A subtlety worth internalizing about multi-input nodes: n8n waits for the data each input needs before the node runs, and an input fed by a branch that never executed contributes nothing. If a Merge output looks like only half your data, check each input's panel — you may find one door never received a delivery. The distinction from earlier bears repeating, because their symptoms are identical downstream but their causes differ: a node that ran and produced zero items shows an empty output ("0 items"), while a node that never ran at all shows no output data whatsoever. The former means your data was filtered away; the latter means the execution path never reached that node. The executions view (Chapter 21) makes the path visible; the panels tell you which case you are in.

Building Items by Hand: The Structure n8n Expects

Everything above reads data other nodes produced. Eventually — in Code nodes (Chapter 19), in custom nodes (Chapter 15), or when returning data from a sub-workflow (Chapter 9) — you will construct items yourself, and n8n will hold you to the envelope structure from the start of this chapter. This section is the reference for those chapters: the rules of a well-formed item, and what n8n does when you bend them.

The canonical shape of a node's output is an array of item objects, each with a json property whose value is an object:

[
  { "json": { "name": "Dana", "score": 91 } },
  { "json": { "name": "Ravi", "score": 84 } }
]

That is the contract. In practice, n8n's Code node is forgiving about some deviations and strict about others:

The auto-wrapping conveniences in that table belong to the Code node specifically. When you build a custom node (Chapter 15), you are expected to return the canonical form yourself — no one wraps for you.

Three rules cover nearly every construction mistake. First, the array is the item list — if you want five items downstream, return an array of five; if you return one object containing an array of five records, you have built the embedded-array situation from earlier, and downstream nodes will see one item. Second, json must be an object — when your result is naturally a bare value (say, a computed total), wrap it in a named field: { "json": { "total": 128 } }, because a field name is the handle every later expression will grab. Third, keep it JSON-representable — anything that cannot survive being written down as JSON text will not survive the trip between nodes; convert rich values (dates especially) into strings or numbers deliberately, on your own terms, rather than letting serialization (the automatic conversion of your data into JSON text) decide for you.

When constructing items in code, you can — and generally should — also declare lineage by setting pairedItem on each output item to the index of the input item it came from. That is what keeps the ancestry chain from the item-linking section unbroken through your custom logic. The mechanics belong to Chapter 19; the reason belongs here: every item you create without ancestry is a future expression error for whoever builds downstream of you.

Binary construction has its own envelope rules — file content plus MIME type, filename, and friends in a named slot — and Chapter 20 owns them.

Tip: Whenever you hand-build items, immediately run the node and read your own output in all three views, exactly as you would a stranger's. Table view confirms the item count you intended; JSON view confirms the wrapper structure and types; Schema view confirms field names came out spelled the way downstream mappings will expect. Thirty seconds of self-inspection here saves a debugging session later.

The Reading Order: A Diagnostic Habit

Everything in this chapter compresses into a sequence you can run in under a minute on any node whose behavior surprises you. It is worth committing to habit, because Chapters 17 through 25 all begin, implicitly, with "first, look at the data."

First, the count. How many items came out — and does the number match your mental model? Zero items, one-where-you-expected-many, and many-where-you-expected-one each point to different suspects before you have read a single value. Distinguish "0 items" from "no output data": filtered empty versus never ran.

Second, the shape. Schema view: do the fields you need exist, at the nesting depth you think, with the types you assume? Then JSON view on one representative item to confirm the details Schema abbreviates — string versus number, object versus array, embedded arrays masquerading as items.

Third, the spread. Table view across all items: is the field present in every item, or only most? Data that is 95 percent clean produces the most confusing failures, because your test item works and some production item does not.

Fourth, the doors. If the node has multiple outputs, are the items leaving through the door you think? If it has multiple inputs, did both actually deliver?

Fifth, the lineage. If downstream expressions reach back through this node, is the paired-item chain intact — or does this node (custom code? aggregation?) cut it?

Only after those five checks are you entitled to an opinion about why the workflow is misbehaving — and by then, you usually do not need the opinion, because the panels have already confessed. The next four chapters put tools in your hands: expressions to reference this data (Chapter 17), nodes to reshape it (Chapter 18), code for the stubborn cases (Chapter 19), and files and durable state (Chapter 20). All four assume you now read items the way a mechanic listens to an engine — before reaching for any tool at all.

Expressions: The Double-Curly-Brace Language

Almost every field in an n8n node can hold either a fixed value or an expression. A fixed value is what it sounds like: you type "Hello" and the node sends "Hello" every time. An expression is a small formula that gets evaluated fresh for every item that flows through the node, so the same field can send "Hello, Amira" to one customer and "Hello, Ben" to the next. Expressions are the connective tissue of every non-trivial workflow: they carry data from one node into the parameters of another, reshape it slightly on the way, and make decisions inline. This chapter takes you from zero to fluent — the syntax, the built-in variables, dates with Luxon, deep queries with JMESPath, the transformation functions, and the two deliverables you will come back to: a gallery of the twenty expressions practitioners write weekly, and a catalog of the ways expressions fail.

One boundary to set immediately: expressions are for one-liners. The moment you find yourself wanting a loop, a second statement, or a temporary variable, you have outgrown the expression field — that logic belongs in the Code node, covered in Chapter 19 (Code and Command: When Clicks Are Not Enough).

The Syntax from Absolute Zero

An expression is anything wrapped in double curly braces: {{ ... }}. Whatever sits between the braces is a single JavaScript expression — JavaScript being the programming language n8n evaluates under the hood. You do not need to know JavaScript to use expressions; you need about a dozen idioms, all shown in this chapter. The braces are the signal to n8n: "do not send this literally — compute it."

To use an expression in a field, hover over the field and switch it from Fixed to Expression (most fields show a small toggle when you hover). A faster route: type an equals sign as the very first character in an empty field, and it flips straight into expression mode. Either way, the field now accepts curly-brace syntax and shows a live preview of the result as you type; an expand control on the field opens the full expression editor — a larger panel with your input data alongside. You can mix plain text and expressions freely in the same field:

Order {{ $json.order_id }} shipped on {{ $today.toFormat('yyyy-MM-dd') }}

Everything outside the braces is sent as-is; everything inside is computed per item. If a node receives five items, the expression is evaluated five times, once per item, and each evaluation can produce a different result. That per-item behavior is the single most important fact about expressions, and it is the direct consequence of the item model described in Chapter 16 (Reading and Reasoning About Data: The Item Model in Practice). One nuance follows from mixing text and braces: when a field contains only a single expression and nothing else, the result keeps its real type (a number stays a number, an array stays an array); as soon as literal text surrounds the braces, everything is flattened into one string.

Tip: You rarely need to type expressions from scratch. Open a field's expression editor and drag a field name from the input panel on the left straight into the text area — n8n writes the correct expression for you, including any awkward bracket notation. Start every expression this way and edit from there. Inside the braces, autocomplete does the rest: type $ or a dot after a value and browse what appears.

Dot notation: walking into the data

Incoming data is JSON (JavaScript Object Notation, the text format almost every API speaks) — nested structures of named fields. You walk into a structure with dots:

{{ $json.customer.address.city }}

Read it left to right: start at the current item's JSON body, go into customer, then into address, then grab city. Two situations require square brackets instead of dots:

• Field names with spaces or special characters: {{ $json['First Name'] }}. Dots only work for names that look like plain words.
• Array positions: {{ $json.line_items[0].sku }} — the first element of the line_items array. Counting starts at zero, so [0] is the first, [1] the second.

One more character worth learning on day one: the question mark in ?., called optional chaining. {{ $json.customer?.email }} means "go into customer if it exists; if it does not, return nothing instead of crashing." Why this matters is the subject of the failure-modes section at the end of the chapter.

The Built-in Variables

Inside the braces, n8n gives you a set of variables that all start with a dollar sign. These are your handles on the current item, other nodes' output, time, and the execution itself.

A few of these deserve more than a table row.

$json — the current item

$json is the variable you will type most. It is shorthand for "the JSON data of the item this node is processing right now." When a node runs against five items, $json means item one during the first evaluation, item two during the second, and so on. You never loop manually; the per-item evaluation is the loop.

$('Node Name') — reaching back to earlier nodes

By default an expression sees only the node's direct input. But workflows are chains, and you often need something from three nodes back — the original webhook payload, say, after two transformations have stripped it away. The $('Node Name') syntax reaches back to any node that has already executed:

{{ $('Webhook').item.json.body.email }}

The name in quotes must match the node's display name on the canvas exactly, including spaces and capitalization. After the parentheses you choose which item you want from that node:

• .item — the item from that node which corresponds to the item currently being processed. n8n tracks lineage between items (Chapter 16 explains this "paired item" bookkeeping); .item follows that lineage backward. This is almost always what you want.
• .first() / .last() — the first or last item that node produced, regardless of lineage.
• .all() — every item that node produced, as an array.

Each of these returns an item wrapper, so you append .json to get at the data: $('Get Customer').first().json.id.

Watch out: .item and .first() look interchangeable when you test with one item, and they are not. .first() returns the same single item for every item being processed; .item returns each item's own ancestor. Use .first() only when the referenced node genuinely produced one item (a single API lookup, a config node). Reaching for .first() out of habit is the classic cause of "every record got the first customer's data" incidents — the single-item assumption discussed at the end of this chapter.

$execution, $workflow, and $prevNode — situational awareness

$execution.id is the unique identifier of the current run — invaluable to stamp into logs, ticket comments, or spreadsheet rows so you can trace any output back to the exact execution that produced it in the executions list (Chapter 21). $execution.mode tells you whether the run is a manual test or a production trigger, which lets you route test runs to a sandbox channel. $workflow.name and $workflow.id label which workflow did the work, useful when many workflows write to a shared log. $prevNode.name tells you which node the data arrived from — handy in a node that several branches merge into.

$vars and $env — configuration without hardcoding

$vars reads custom variables — named values (like an API base URL or a team email) defined once for the whole instance and referenced everywhere as {{ $vars.myVariable }}. Change the value in one place and every workflow picks it up. Variables are managed in the instance's admin area and are read-only inside expressions; their availability and how many you can define depend on your plan or license, so not every instance has them.

$env reads operating-system environment variables on self-hosted instances: {{ $env.MY_SETTING }}. It is not available on n8n Cloud, and self-hosted administrators can disable it.

Watch out: Never stash API keys or passwords in $vars or $env for use in expressions. Anyone who can edit a workflow can print these values into an output field. Secrets belong in n8n's credential system, which stores them encrypted and masks them in the UI — see Chapter 11 (Credentials from Zero) and Chapter 34 (Access, Security, and Secrets for Teams).

Dates and Times: Luxon in Plain English

Dates are where beginners lose the most time, so n8n bundles Luxon, a well-regarded JavaScript date library, and exposes it directly in expressions. $now and $today are not strings — they are Luxon DateTime objects, which means they carry methods for math, formatting, and timezone conversion.

Formatting. .toFormat() turns a DateTime into text using a pattern string:

{{ $now.toFormat('yyyy-MM-dd') }}        → 2026-07-17
{{ $now.toFormat('dd LLL yyyy HH:mm') }} → 17 Jul 2026 14:32

The tokens are mostly guessable — yyyy year, MM two-digit month, LLL short month name, dd day, HH 24-hour, mm minutes — and Luxon's documentation lists them all. For machine-to-machine work, skip patterns entirely: .toISO() emits the standard ISO 8601 timestamp that nearly every API accepts.

Date math. .plus() and .minus() take an object naming the unit:

{{ $now.plus({ days: 7 }) }}
{{ $today.minus({ months: 1 }) }}

Units can be years, months, weeks, days, hours, minutes, seconds, and you can combine them: {{ $now.plus({ days: 1, hours: 6 }) }}.

Parsing. Dates arriving from other systems are strings, not DateTime objects, so convert before doing math. For ISO-shaped strings (2026-07-17T14:00:00Z), use DateTime.fromISO($json.created_at) — Luxon's DateTime class is available inside expressions. For odd formats, DateTime.fromFormat($json.date, 'MM/dd/yyyy') parses against a pattern. n8n also adds a convenience .toDateTime() method on strings.

Differences. .diff() measures the gap between two DateTimes: {{ $now.diff(DateTime.fromISO($json.created_at), 'days').days }} gives the age of a record in days (as a decimal — round it if you need a whole number).

Timezones. $now and $today are computed in the workflow's timezone, which you can set per workflow via the workflow menu under Settings > Timezone; workflows without their own setting inherit the instance default, which may not be your local zone. To present a time in another zone, convert explicitly: {{ $now.setZone('America/New_York').toFormat('HH:mm') }}.

Watch out: If your scheduled workflow fires at a strange hour or your "today" boundary is off, check the workflow timezone before debugging anything else. A workflow inheriting an instance default in another timezone is one of the most common — and most invisible — causes of date bugs. Chapter 7 (Triggers: Deciding When Work Happens) covers the scheduling side of this.

JMESPath: Querying Nested Structures

Dot notation is fine for reaching one known field. It gets miserable when you need to search: "the IDs of all paid orders," "the emails of everyone in Washington." For that, n8n exposes JMESPath (pronounced "James path"), a query language for JSON, through the $jmespath() function. It takes the data first and the query second:

{{ $jmespath($json.orders, "[?status=='paid'].id") }}

That reads: within the orders array, filter ([? ... ]) to elements whose status equals 'paid', then project out (.id) just the id of each — returning an array of IDs in one line. A few building blocks cover most real use:

• people[*].name — the name of every element (a projection).
• [?age > `30`].email — filter by numeric comparison; JMESPath wraps literal numbers in backticks.
• locations[?state=='WA'].name | sort(@) — pipe results into a function; JMESPath has built-ins like sort, length, max_by, and join.
• Keys containing spaces or dashes get double quotes inside the query (use single quotes for the outer JavaScript string so the two do not collide).

JMESPath rewards a scan of its tutorial (linked from n8n's documentation and the Resource Shelf in Chapter 40), but even the filter-and-project pattern above replaces what would otherwise be a Code-node loop. For heavier reshaping — grouping, joining datasets, pivoting — reach for the transformation nodes in Chapter 18 instead of ever-longer queries.

The Built-in Transformation Functions

Beyond standard JavaScript, n8n extends strings, numbers, dates, arrays, and objects with its own convenience methods. You chain them onto a value with a dot, and the expression editor autocompletes them as you type — one of the fastest ways to discover what exists. These are the everyday workhorses:

These chain naturally: {{ $json.line_items.pluck('price').sum().round(2) }} — collect every price, add them up, round to cents. Plain JavaScript string and array methods (.trim(), .toLowerCase(), .split(), .join(), .includes(), .slice(), .map()) work too, and mix freely with n8n's additions.

Alongside the methods sit four standalone helpers: $if(condition, thenValue, elseValue) for inline branching, $ifEmpty(value, fallback) for defaults when a value is empty, and $min(...) / $max(...) across several numbers. $if and the JavaScript ternary (condition ? a : b) do the same job; use whichever reads better to you.

The Expression Editor's Preview: Your Safety Net

The expression editor is not just a bigger text box. Below your expression it shows the evaluated result — computed against real data from the node's input. This preview is the difference between guessing and knowing. Habits that make it work for you:

• Feed it data first. The preview can only evaluate against input that exists. Execute the upstream nodes once (or pin a previous execution's data, as described in Chapter 10) so the input panel is populated. With no input, the preview can only shrug.
• Read the result's type, not just its text. The preview distinguishes a string from a number from an array. If you expected a number and see quotes, you have found a type problem before it shipped — the coercion family described in the failure-modes section below.
• Step through items. The preview evaluates against one item at a time — by default the first. Use the item navigation in the input panel to check the second and third items too; expressions that work on item one and fail on item two are the genre's signature bug.
• Treat red as red. If the preview shows an error or undefined, the production run will too. Never save a node whose preview you have not seen evaluate cleanly.

Tip: Keep a scratch Set node (Chapter 18) at the end of any branch you are developing, with one field per expression you are unsure about. It costs nothing, gives every expression a live preview against real items, and doubles as documentation of what you expect the data to look like. Delete it when the workflow graduates to production, or keep it — Chapter 22 (The Debugging Craft) makes the case for leaving inspection points in place.

The Pattern Gallery: Twenty Expressions You Will Write Weekly

Copy these, adjust the field names, and you have covered the bulk of real-world expression writing.

Moving and defaulting data

1. Pass a field through. {{ $json.email }} — the atom of all expressions: read one field from the current item.

2. Default when a field is missing. {{ $json.name ?? 'there' }} — the ?? operator says "use the left side unless it is null or missing, then use the right." Perfect for greeting lines that must never render blank.

3. Default when a field is empty. {{ $ifEmpty($json.company, 'Independent') }} — like pattern 2, but also catches empty strings, which ?? lets through.

4. Reach back to the trigger. {{ $('Webhook').item.json.body.customer_id }} — recover the original payload after intermediate nodes have transformed the data away. (Webhook payloads nest under body; see Chapter 14.)

5. Pull from a single-item lookup. {{ $('Get Config').first().json.api_base }} — when an earlier node produced exactly one item, .first() grabs it regardless of which item you are currently processing.

Text shaping

6. Clean an email for matching. {{ $json.email.trim().toLowerCase() }} — strip stray whitespace and normalize case before comparing or deduplicating.

7. Build a full name. {{ ($json.first_name + ' ' + $json.last_name).trim() }} — the outer .trim() tidies up when one part is empty.

8. Make a slug or filename. {{ $json.title.toLowerCase().replaceAll(' ', '-') }} — a slug is a URL-safe, lowercase version of a title. Spaces become hyphens; add .replaceSpecialChars() before it for accented input.

9. Extract an email from free text. {{ $json.message.extractEmail() }} — finds the address buried in a reply body.

10. Get the domain for routing or enrichment. {{ $json.email.extractDomain() }} — "ana@acme.io" becomes "acme.io".

11. Digits only. {{ $json.phone.replace(/\D/g, '') }} — the regular expression \D matches every non-digit; replacing all of them with nothing leaves a clean numeric string.

12. Safe URL parameter. {{ $json.search_term.urlEncode() }} — always encode user text before splicing it into a URL in the HTTP Request node (Chapter 13).

Dates

13. Today's date stamp. {{ $today.toFormat('yyyy-MM-dd') }} — the ISO day format that sorts correctly everywhere.

14. A deadline seven days out. {{ $now.plus({ days: 7 }).toISO() }} — date math plus the machine-readable format APIs expect.

15. Reformat an incoming date for humans. {{ DateTime.fromISO($json.created_at).toFormat('LLL d, yyyy') }} — parse first, then format: "Jul 17, 2026".

16. Age of a record in whole days. {{ $now.diff(DateTime.fromISO($json.created_at), 'days').days.floor() }} — the backbone of "escalate anything older than N days" logic.

Numbers, arrays, and logic

17. Sum the line items. {{ $json.line_items.pluck('price').sum().round(2) }} — collect, aggregate, round to cents in one chain.

18. Count what a node produced. {{ $('Fetch Rows').all().length }} — feeds "processed 47 records" into a summary message; pairs with the value-proving patterns in Chapter 25.

19. Inline label from a threshold. {{ $json.score > 75 ? 'hot' : 'warm' }} — a ternary for cosmetic labels; use a real IF node (Chapter 9) when the route through the workflow should change.

20. A traceable log line. {{ $workflow.name + ' #' + $execution.id + ' item ' + ($itemIndex + 1) }} — stamps workflow, execution, and item position (converted to human counting) into anything you write to an external system, so every output can be traced to its run.

When Expressions Go Wrong: The Failure Modes

Every broken expression in the wild falls into one of three families. Learn to recognize them and most debugging sessions shrink to seconds.

Undefined paths

The error reads something like "Cannot read properties of undefined" — or the output silently contains the text undefined. The cause is always the same: the expression walked a path that does not exist in this item. $json.customer.email explodes when an item has no customer at all, because you cannot ask nothing for its email.

The fixes, in order of preference: make the data consistent upstream (a Set node that guarantees the field exists on every item); guard the path with optional chaining — {{ $json.customer?.email ?? '' }} never crashes and yields an explicit fallback; or filter out the incomplete items with an IF node before the field is needed. What you should not do is ignore an occasional undefined in test output — the string "undefined" has a way of ending up in production emails. Remember also that field names are case-sensitive and must match exactly: $json.Email and $json.email are different fields, and a typo produces the same undefined behavior as missing data.

Type coercion

JavaScript converts between types silently, and the results are legal but wrong. The classics:

• String arithmetic. If an API delivers "amount": "5" as text, then {{ $json.amount + 1 }} yields "51" — the + concatenated instead of adding. Fix: {{ $json.amount.toNumber() + 1 }}.
• Truthy strings. The string "false" is not the boolean false; a check like {{ $json.enabled ? 'on' : 'off' }} says "on" for it, because any non-empty string counts as true. Compare explicitly — {{ $json.enabled === true }} or {{ $json.enabled === 'true' }} — depending on what the source actually sends.
• Dates that are secretly strings. $json.created_at.plus({ days: 1 }) fails because strings do not have Luxon methods. Parse first (DateTime.fromISO(...) or .toDateTime()).
• Objects flattened into text. Mix an expression with surrounding text and any object it returns becomes the useless string [object Object]; keep structured values in fields that contain a single bare expression, or serialize deliberately with .toJsonString().

The expression preview is your instrument here: it shows the type of the result, not just its text. When a calculation misbehaves, check what type each ingredient really is before touching the logic.

Single-item assumptions

The subtlest family, because everything works in testing. You build with one test item; production sends twelve; and suddenly every record carries the first record's data, or the node errors with a message about not being able to determine which item to use.

Three habits cause it. First, .first() where .item belongs (see the callout earlier): .first() returns one fixed item for the whole batch. Second, assuming the referenced node has run — $('Node Name') can only read a node that already executed in this execution; referencing a node on a branch that was skipped, or one downstream of the current node, produces an error about the node being unexecuted. Third, broken lineage: .item works by tracing each item's ancestry backward, and some node arrangements — merged streams, aggregate-then-split shapes, items generated fresh mid-workflow — can drop that paired-item bookkeeping, producing an error saying n8n cannot find the paired item. When that happens, either restore the lineage (Chapter 16 shows how), carry the needed fields forward on the items themselves so you never have to reach back, or fall back to an explicit lookup over .all().

The vaccination against all three: always test with several items — realistic, varied ones — before activating. Pin a multi-item execution and step the preview through item two and item three. Chapter 22 turns this into a full debugging discipline, and Chapter 23 (Error Handling) covers what to do when a bad expression does slip through to production anyway.

Where to Go Next

Expressions handle the one-line reshaping that happens inside node parameters. When whole items need restructuring — renaming many fields, filtering, sorting, merging streams — dedicated transformation nodes do it more legibly, and Chapter 18 (The Transformation Toolkit) walks through them. When logic needs multiple steps, real loops, or error handling of its own, graduate to the Code node in Chapter 19; a workflow with three tortured expressions is usually better as one short, honest script. Every variable and function touched here also appears in the Mega-Glossary in Chapter 40 with a one-line reminder of what it does. The braces themselves never get more complicated than what you have seen: one formula, evaluated once per item, previewed before you trust it — and it works exactly the same way in every field of every node.

The Transformation Toolkit

Data almost never leaves one app in the shape the next app wants. A CRM hands you a deeply nested contact record; your spreadsheet wants three flat columns. An order API returns one big object containing a list of line items; your fulfillment system wants one message per line. Somewhere between the trigger and the final action, the data has to be reshaped — and in n8n, most of that reshaping is done not with code but with a family of small, click-configured utility nodes.

This chapter is a tour of that family: Edit Fields (Set), Split Out, Aggregate, Summarize, Sort, Limit, Remove Duplicates, Rename Keys, Compare Datasets, Date & Time, Crypto, and the Merge node's data-joining modes. Each node does one job, does it visibly, and can be understood at a glance by whoever inherits your workflow. That last point matters more than it sounds: a workflow built from named transformation nodes documents itself, while the same logic buried in a Code node is a black box until someone opens it. Reach for these nodes first; save expressions (Chapter 17, Expressions: The Double-Curly-Brace Language) for values inside a field, and the Code node (Chapter 19, Code and Command) for the genuinely gnarly cases.

You will find every node in this chapter by opening the nodes panel — the + button on the canvas — and typing its name. They cluster under the platform's data transformation category, but searching by name is faster and survives menu reshuffles.

A note on how to read the examples. As Chapter 16 (Reading and Reasoning About Data) established, everything that travels between nodes is a list of items, and each item carries a JSON object. So every example below shows a Before snapshot (the items entering the node) and an After snapshot (the items leaving it), written as JSON arrays. When you build these yourself, the input and output panels of the node editor show you exactly these two views side by side.

Tip: While experimenting with any node in this chapter, pin a sample of real input data to the node that feeds it (Chapter 10 and Chapter 22 cover pinning in depth). Pinned data means you can re-run the transformation node instantly, as many times as you like, without re-calling the upstream API.

Edit Fields (Set): The Everyday Reshaper

Edit Fields — long known by its older name, Set, which still appears in many templates and tutorials — is the node you will place more often than any other. It builds each outgoing item from fields you declare: some copied from the input, some computed with expressions, some hard-coded. It is how you flatten nesting, rename awkward fields, add constants, and throw away clutter.

The node offers two working modes. In manual mapping mode you add fields one at a time in a form: give each a name, pick a type (string, number, boolean, array, or object), and supply a value — either typed literally or dragged in from the input panel, which writes the expression for you. In JSON mode you write the entire output object as one JSON template with expressions embedded in it, which is quicker when you are defining many fields at once.

Suppose a CRM gives you this:

[
  {
    "id": 8841,
    "createdAt": "2026-05-02T09:14:00Z",
    "properties": {
      "firstname": "Dana",
      "lastname": "Okafor",
      "company": "Brightline Legal",
      "hs_lead_status": "NEW"
    }
  }
]

An Edit Fields node with three declared fields — fullName set to the expression {{ $json.properties.firstname }} {{ $json.properties.lastname }}, company dragged from properties.company, and status dragged from properties.hs_lead_status — produces:

[
  {
    "fullName": "Dana Okafor",
    "company": "Brightline Legal",
    "status": "NEW"
  }
]

Notice that id and createdAt vanished. By default the node outputs only the fields you set. A toggle — Include Other Input Fields — changes that, letting you keep everything and merely add or override, or keep all fields except a listed few. Field names support dot notation, so setting a field called address.city creates a nested object rather than a field with a dot in its name.

Watch out: The single most common "where did my data go?" moment in n8n is an Edit Fields node quietly dropping every field you did not explicitly set. If a downstream node suddenly cannot see a field you know existed, check the Include Other Input Fields setting on the Edit Fields nodes between here and there before suspecting anything else.

Split Out: One Item per List Element

APIs love to hand you one item containing an array. But n8n's power comes from the item model: downstream nodes run once per item, so an array trapped inside a single item is inert. Split Out liberates it — you name the field that holds the list, and the node emits one item per element.

Before — one order, two lines:

[
  {
    "orderId": "A-1027",
    "customer": "dana@brightline.example",
    "lines": [
      { "sku": "TPL-BASIC", "qty": 1 },
      { "sku": "TPL-PRO", "qty": 2 }
    ]
  }
]

After, with Fields to Split Out set to lines and the include option set to carry all other fields along:

[
  {
    "sku": "TPL-BASIC",
    "qty": 1,
    "orderId": "A-1027",
    "customer": "dana@brightline.example"
  },
  {
    "sku": "TPL-PRO",
    "qty": 2,
    "orderId": "A-1027",
    "customer": "dana@brightline.example"
  }
]

Each element became an item, and because we chose to include the other fields, every child item still knows which order and customer it belongs to — usually exactly what you want, since downstream nodes see only their own item. If the array holds plain values (["red", "blue"]) rather than objects, set a destination field name so each value lands under a key you choose. You can also split several array fields at once; the node pairs them up element by element.

Aggregate: Many Items into One

Aggregate is Split Out's mirror image: it takes many items and returns one. You need it whenever the next step wants the whole collection at once — one email that lists all of today's signups, one Slack message summarizing every failed job, one prompt handing an AI model the full batch (a pattern Chapter 27 leans on heavily).

It has two modes. Individual fields collects the values of fields you name into arrays. All item data tucks every complete item into a single list under one field (named data unless you rename it).

Before:

[
  { "sku": "TPL-BASIC", "qty": 1 },
  { "sku": "TPL-PRO", "qty": 2 },
  { "sku": "TPL-PRO", "qty": 1 }
]

After, aggregating the individual fields sku and qty:

[
  {
    "sku": ["TPL-BASIC", "TPL-PRO", "TPL-PRO"],
    "qty": [1, 2, 1]
  }
]

Options let you merge nested lists into one flat array, keep or discard missing and null values, and carry binary attachments along.

Summarize: The Pivot Table

Where Aggregate merely collects, Summarize computes. It is a pivot table in node form: you pick fields to group by ("split by" in the node's vocabulary) and fields to summarize, choosing an operation for each — sum, count, count unique, average, minimum, maximum, or the text-flavored append and concatenate.

Before — order totals by region:

[
  { "region": "EU", "total": 49 },
  { "region": "US", "total": 99 },
  { "region": "EU", "total": 29 }
]

After, splitting by region and summing total while counting rows:

[
  { "region": "EU", "sum_total": 78, "count_total": 2 },
  { "region": "US", "sum_total": 99, "count_total": 1 }
]

The output field names are generated from the operation and source field, and you can choose whether each group becomes its own item (usual, and friendliest to downstream nodes) or the whole summary lands in one item.

Tip: If you catch yourself opening a Code node to add up numbers or count occurrences, stop and try Summarize first. The distinction to remember: Aggregate gathers values into arrays without doing math; Summarize groups items and computes. Between them they cover almost every "roll this up" request.

Sort and Limit: Ordering and Trimming

These two are simple alone and powerful together. Sort orders your items: in its simple mode you list one or more fields, each ascending or descending, and ties on the first field fall through to the next. It also offers a random mode (shuffle — handy for sampling or A/B assignment) and a code mode accepting a custom comparison function for exotic orderings. Limit then keeps only the first N or last N items and discards the rest.

The classic combination is "top three." Before:

[
  { "deal": "Acme renewal", "amount": 1200 },
  { "deal": "Brightline pilot", "amount": 8600 },
  { "deal": "Corax upsell", "amount": 430 },
  { "deal": "Dorian expansion", "amount": 5100 }
]

After Sort (by amount, descending) then Limit (first 3 items):

[
  { "deal": "Brightline pilot", "amount": 8600 },
  { "deal": "Dorian expansion", "amount": 5100 },
  { "deal": "Acme renewal", "amount": 1200 }
]

One trap: if numbers arrive as text — "9" and "10" rather than 9 and 10 — sorting can go dictionary-style, putting "9" after "10". Convert the field to a number first, either with the type selector in Edit Fields or a .toNumber() expression from Chapter 17.

Remove Duplicates: Once and Only Once

Remove Duplicates does two related but importantly different jobs, chosen by its operation setting.

The first is deduplicating within the current input: compare items across all fields, all fields except some, or only the fields you name, and keep one item per distinct combination. Before:

[
  { "email": "dana@brightline.example", "source": "webinar" },
  { "email": "leo@corax.example", "source": "ebook" },
  { "email": "dana@brightline.example", "source": "ebook" }
]

After, comparing on email only:

[
  { "email": "dana@brightline.example", "source": "webinar" },
  { "email": "leo@corax.example", "source": "ebook" }
]

The second job is deduplicating across executions: the "remove items processed in previous executions" operation remembers a key value (or the highest ID or latest date it has seen) between runs, so a workflow that polls a source every few minutes processes each record exactly once even when the source keeps returning old rows. This is one of the platform's quiet superpowers for reliability — Chapter 24 (Reliability Patterns) builds whole strategies on it for idempotency, the property that processing the same record twice produces no double effect. A companion "clear deduplication history" operation resets the memory.

Watch out: The cross-execution memory is real, persistent state — and it does not distinguish test runs from production runs. If you manually execute a workflow while building it, the keys you tested with are now marked as seen, and the first production run will skip them. When testing burns keys you still need, run the node's clear-history operation (or scope your tests to throwaway data) before activating.

Rename Keys: Fixing Names Without Touching Values

Rename Keys changes field names and nothing else. You add mappings — current name to new name, with dot notation reaching into nested objects — or switch on regular-expression mode to rename in bulk by pattern.

Before:

[
  { "First Name": "Dana", "Last Name": "Okafor", "E-mail": "dana@brightline.example" }
]

After three mappings:

[
  { "firstName": "Dana", "lastName": "Okafor", "email": "dana@brightline.example" }
]

Do this early in the workflow. Field names containing spaces or punctuation work, but every expression that touches them needs bracket syntax and quoting, which is tedious and error-prone. One Rename Keys node right after the trigger buys clean names for the whole rest of the flow. Use the regex mode carefully — a loose pattern renames more keys than you intended, and the node will not warn you.

Compare Datasets: What Changed?

Compare Datasets is the reconciliation specialist. It takes two inputs — labeled A and B — matches items between them on fields you specify, and routes the results to four separate outputs, in the order they appear on the node: items found only in A, items present in both and identical, items present in both but different, and items found only in B.

That four-way split is exactly the shape of a sync job. Say input A is your CRM contact list and input B is the master spreadsheet, matched on email:

A: [ { "email": "dana@brightline.example", "tier": "pro" },
     { "email": "leo@corax.example",  "tier": "basic" } ]

B: [ { "email": "dana@brightline.example", "tier": "enterprise" },
     { "email": "mia@dorian.example", "tier": "basic" } ]

Dana lands in the different output (tiers disagree), Leo in in A only (create him in the spreadsheet), and Mia in in B only (create her in the CRM — or investigate). Nothing lands in same. Wire each output to the appropriate action and you have a two-way sync without writing a line of code. For the different branch you choose how the merged item should look: keep both versions side by side, prefer A's values, prefer B's, or mix field by field. A fuzzy-compare option treats near-equal values (such as the number 1 and the string "1") as matches, which saves grief when one side is a spreadsheet.

Date & Time: Calendar Arithmetic Without Tears

Dates are where hand-rolled logic goes to die, so n8n gives them a dedicated node. Date & Time performs one operation per node: get the current date, add to or subtract from a date, format a date for display, extract a part of it (year, month, day, hour...), round it up or down (to the nearest hour, day, and so on), or compute the time between two dates.

Before — a task created now, needing a due date and a human-friendly label:

[
  { "task": "Send contract", "createdAt": "2026-05-02T09:14:00Z" }
]

One Date & Time node (add 3 days to createdAt, output to dueDate) and a second (format dueDate for display) later:

[
  {
    "task": "Send contract",
    "createdAt": "2026-05-02T09:14:00Z",
    "dueDate": "2026-05-05T09:14:00Z",
    "dueDateDisplay": "May 5, 2026"
  }
]

Two habits keep dates painless. First, keep machine-readable ISO timestamps — the sortable 2026-05-02T09:14:00Z shape used throughout this chapter — flowing between nodes and format only at the very end, for the human-facing surface. Second, know your timezone: each workflow has a timezone setting (in the workflow's settings panel, alongside the instance-wide default), and it governs how ambiguous dates are interpreted. The same operations are available in expression form through the built-in date functions covered in Chapter 17; the node exists so common cases never require expression syntax at all.

Crypto: Hashes, Fingerprints, and Random Values

Despite the intimidating name, the Crypto node mostly does four mundane, useful things: it hashes a value (produces a fixed-length fingerprint — the same input always yields the same output, and the output cannot be reversed back into the input) using algorithms like SHA-256; it computes an HMAC (a hash keyed with a secret, so only someone holding the secret can produce or verify it — the mechanism behind most webhook signatures, as Chapter 14 explains); it signs data with a private key; and it generates random strings and UUIDs (universally unique identifiers).

Before:

[
  { "email": "dana@brightline.example" }
]

After a SHA-256 hash of email written to emailHash:

[
  {
    "email": "dana@brightline.example",
    "emailHash": "d348438e824f06ac619c22832acb2439e19c8e75b0019cfe03db203a51ba9eaf"
  }
]

Three everyday uses: building an idempotency key by hashing the fields that define "the same request" (Chapter 24 shows the full pattern), pseudonymizing personal data such as emails before they reach an analytics tool, and generating unique reference IDs for records you create. Note what the node is not: it is not a place to store or protect secrets — hashing is one-way, and credentials belong in n8n's credential system (Chapter 11) or an external secrets vault (Chapter 34).

Merge: Joining Two Streams of Data

You met Merge in Chapter 9 (Flow Control) in its traffic-management role: rejoining branches after an If or Switch split them. Its second life — the one that belongs to this chapter — is as a genuine data-join tool, the closest thing the canvas has to a database JOIN. It works on two inputs (the Append and SQL Query modes let you add more; Combine always pairs exactly two) and offers four modes: Append, Combine, SQL Query, and Choose Branch. Combine is the one that does real joining, and it in turn offers three ways to pair items — chosen with its own Combine By setting — which the next sections walk through one at a time.

Append simply stacks the streams: all of input 1's items, then all of input 2's. Use it when both branches produce the same kind of record and you want one combined list — say, leads from two different forms heading into one deduplication step.

Combine, matching fields is the star. You name the field (or fields) that link the two sides, and the node pairs items whose values match — exactly like joining two database tables on a key. Given orders in input 1 and customer records in input 2:

Input 1: [ { "orderId": "A-1027", "email": "dana@brightline.example", "total": 148 } ]
Input 2: [ { "email": "dana@brightline.example", "company": "Brightline Legal", "tier": "pro" } ]

After combining on email:

[
  {
    "orderId": "A-1027",
    "email": "dana@brightline.example",
    "total": 148,
    "company": "Brightline Legal",
    "tier": "pro"
  }
]

What happens to items that don't find a partner is controlled by the Output Type setting, and the options map directly onto SQL join types:

When both sides carry a same-named field with different values, a clash-handling option decides whose value survives (or keeps both under distinct names), and a fuzzy-compare option forgives type mismatches in the key, such as an ID that is a number on one side and a string on the other.

Combine by position pairs items by index — first with first, second with second. It asks no questions about content, which makes it fast and dangerous in equal measure.

Combine by all possible combinations produces the cross product: every input 1 item paired with every input 2 item. Three items by four items yields twelve. Useful for generating a test matrix or pairing every message template with every recipient segment; ruinous if you feed it two large lists without meaning to.

SQL query mode lets you drop into real SQL, treating the inputs as tables (input1, input2) and writing SELECT ... FROM input1 LEFT JOIN input2 ON ... — worth knowing when a join needs conditions the click-configured modes cannot express.

Choose branch ignores merging altogether and simply waits until all inputs have arrived, then outputs one chosen input's data — a synchronization tool that belongs to Chapter 9's story more than this one.

Watch out: Combine by position fails silently. If the two sides ever arrive with different item counts — one API paginates differently, one branch filtered an item out — items pair up wrongly or drop off the end, and no error tells you so. If any shared key exists, however awkward, prefer matching fields; reserve position-based combining for streams you generated yourself in lockstep.

Which Node Do I Reach For?

The decision table below is the chapter in one glance. Find the sentence that matches what you are trying to do, and the middle column names the node.

Composing the Toolkit

Real transformations are rarely one node; they are short chains, and the chains follow recognizable rhythms. A reporting flow reads Split Out, then Summarize, then Sort, then Limit: explode the raw records, roll them up by group, rank the groups, keep the top few. An enrichment flow reads Rename Keys, then Merge on a matching field, then Edit Fields: clean the names, join in the reference data, shape the final record. A polling flow reads Remove Duplicates (across executions), then Edit Fields, then the action: skip what you have seen, shape what is new, act on it.

Three habits make these chains a pleasure to maintain. First, rename every transformation node to say what it does in your domain — "Keep top 3 deals" beats "Limit", and on a canvas of thirty nodes the difference is navigability. Second, keep each node small: five simple nodes that each do one visible thing beat one clever node, because the executions view (Chapter 21) will show you the data between every pair of them when something goes wrong. Third, test the ugly shapes on purpose before you activate: an empty array into Split Out, a missing field into Summarize, unequal counts into Merge. Chapter 16 explains why edge-case items behave the way they do, and Chapter 22 shows how to chase them when they surprise you anyway — but a two-minute test with pinned edge-case data is cheaper than either.

Tip: When a transformation chain grows past four or five nodes, run it once and walk the item counts across the top of each node on the canvas. The counts tell the story at a glance — 1 becomes 12 at Split Out, 12 becomes 3 at Summarize, 3 stays 3 through Sort. The first place the count defies your expectation is where your mental model and the data disagree, and it is nearly always where the bug lives.

Everything in this chapter reshapes structured JSON. When the payload is a file — a spreadsheet to parse, a PDF to move, an image to convert — you are in binary territory, and the conversion nodes for that live in Chapter 20 (Files and Remembered State), alongside Data Tables, the platform's built-in way to remember state between runs. And when the reshaping you need is a value-level calculation inside a single field rather than a structural change to the items, that is the province of expressions — the double-curly-brace language of Chapter 17 that quietly powers half the fields you just configured.

Code and Command: When Clicks Are Not Enough

Every visual automation tool eventually meets a problem it cannot express in dropdowns. Maybe you need to reshape a deeply nested API response, compute a running total across a hundred items, or parse a date format so cursed that no built-in option recognizes it. n8n's answer is an escape hatch with three levels: the Code node, which lets you write JavaScript or Python inside a workflow; the Execute Command node, which runs shell commands on the machine hosting n8n; and the SSH node, which runs commands on some other machine entirely. This chapter teaches you all three — and, just as importantly, teaches you when to keep the hatch closed.

The honest framing first: code inside a workflow is a power tool, not a status symbol. A workflow built from well-named standard nodes documents itself. A workflow with a 200-line Code node in the middle documents nothing, breaks silently when an upstream field is renamed, and becomes the one node nobody on your team dares to touch. So before we open the editor, we set the rules for staying out of it.

The When-Not-to-Code Ladder

Treat this as a firm decision framework, not a suggestion. When you hit something the current node cannot do, walk down this ladder and stop at the first rung that works:

1. Check the node itself. Many nodes hide the option you need under Options or an Add Field button, or in a different operation of the same node. Chapter 12 (Navigating and Operating the Integration Catalog) covers how to read a node's full surface.
2. Use an expression. A single field that needs computing — a formatted date, a concatenated name, a conditional label — is a job for the double-curly-brace expression language from Chapter 17 (Expressions: The Double-Curly-Brace Language). Expressions live inside the node parameter they affect, so the logic sits exactly where its result is used.
3. Use a transformation node. Reshaping whole items — renaming fields, filtering, sorting, splitting arrays into items, merging branches, deduplicating — is what the Set/Edit Fields, Filter, Sort, Split Out, Aggregate, Merge, and Remove Duplicates nodes are for. Chapter 18 (The Transformation Toolkit) walks through each one. A chain of three small transformation nodes beats one Code node because each step's output is visible in the execution log, testable in isolation, and legible to a colleague who has never seen JavaScript.
4. Only now, reach for the Code node. Legitimate triggers: logic that needs loops or accumulators across items, recursive restructuring of nested data, algorithms (deduplication with fuzzy matching, custom scoring), or transformations that would take five or more chained nodes and still be awkward.
5. Reach for Execute Command or SSH last. Shell-level work belongs at the bottom because it steps outside n8n's data model and safety rails entirely, and because it couples your workflow to a specific machine's installed software.

Why be so strict? Three reasons that compound over time. Visibility: standard nodes show their configuration on the canvas and their output per step in the execution view; code shows a gray box. Maintainability: when n8n upgrades a node, your workflow inherits the improvement; your custom code inherits nothing and must be maintained by hand. Handoff: the whole promise of visual automation is that an operator who is not a programmer can read, audit, and adjust a workflow. Every Code node you add narrows the set of people who can safely do that.

Tip: A useful team rule: any Code node longer than roughly twenty lines gets a sticky note on the canvas explaining what it does, why a standard node could not do it, and what the input and output shapes look like. Future-you is the most frequent beneficiary.

Meet the Code Node

The Code node is a regular node — you add it from the nodes panel like any other, wire it into your chain, and it receives items from the previous node and passes items to the next. What is different is the parameter panel: instead of dropdowns, you get a full code editor with syntax highlighting, autocomplete for n8n's built-in variables, and a language selector.

Two languages are offered: JavaScript, which is the native, first-class option (n8n itself is built on Node.js, the JavaScript runtime), and Python, which is provided for people who think in Python. JavaScript is the more capable of the two inside n8n: it gets the full set of built-in variables and helpers described below, it is what the AI-assist feature drafts, and it is what most community examples use. Python's story has changed across versions — older releases translated it to run inside the JavaScript environment, while current releases run it natively in its own runner — and the two behave differently enough that scripts written for one may need edits for the other. In current versions, Python code sees essentially just the incoming items (as _items when running once for all items, _item when running once per item, read with bracket access like item["json"]["field"]), not the wider helper set JavaScript enjoys. So: reach for Python when you want Python's way of thinking for self-contained data crunching; reach for JavaScript when your code needs n8n's built-ins. If Python is your preference, confirm what your instance offers before committing a critical workflow to it.

One architectural fact worth knowing even as a beginner: in current versions, your code does not run inside n8n's main process. It runs in a task runner — a separate, sandboxed process whose job is to execute user code safely, so a runaway script cannot take down the editor or other workflows. For you as a workflow author, this is invisible except for two consequences: code cannot reach n8n's internals beyond the variables deliberately exposed to it, and there are limits on execution time and memory. How task runners are deployed, scaled, and configured is an operations topic covered in Chapter 33 (Scaling and Performance: Queue Mode and Task Runners).

Watch out: By default the Code node cannot touch the filesystem, and it never has a browser DOM — the page objects a web script would manipulate — because there is no browser here. It can make HTTP calls through an internal helper — but if you find yourself wanting to call an API from inside a Code node, stop anyway. That is the HTTP Request node's job (Chapter 13), where the call gets credential management, retry options, and visible logging for free; an API call buried in code is invisible to whoever debugs the workflow later.

The Two Execution Modes

The single most important setting on the Code node is Mode, which has two values. Choosing wrongly is the number-one source of beginner confusion, so take a moment here.

Recall from Chapter 3 (and in depth from Chapter 16, Reading and Reasoning About Data: The Item Model in Practice) that data flows between nodes as a list of items — individual records, each carrying a JSON payload. The mode setting decides how your code relates to that list.

Run Once for All Items hands your code the entire batch. You typically start with const items = $input.all();, loop or map over them, and return a new array. This is the mode for anything that needs to see multiple items at once: computing a total, finding duplicates, sorting, collapsing fifty items into one summary item, or exploding one item into many.

Run Once for Each Item runs your code separately for every incoming item, with that item's data available directly as $json (its JSON payload). You return one item, and n8n reassembles the outputs into a list in order. This mode is simpler to reason about — no loops needed — and is the right choice when each record is processed independently.

A practical rule: if your logic contains the phrase "across all the items" or "the total/first/largest," you want All Items. If it contains "for each record," you want Each Item. When in doubt, Each Item mode produces simpler code.

The Shape of Items In and Out

The Code node is fussy about one thing: what you return. n8n's item format is a JavaScript object with a json property holding the record's data, and optionally a binary property holding file attachments (binary data is Chapter 20's territory). So a minimal valid item looks like this:

{ json: { name: "Ada", score: 42 } }

In Run Once for All Items mode, you must return an array of such objects — even if there is only one:

const items = $input.all();
const total = items.reduce((sum, item) => sum + item.json.amount, 0);
return [{ json: { total, count: items.length } }];

In Run Once for Each Item mode, you return a single object with a json property:

return { json: { ...$json, fullName: `${$json.first} ${$json.last}` } };

Modern versions of n8n are forgiving in one direction: if you return plain objects without the json wrapper, it will usually wrap them for you. But learn the canonical shape anyway, because error messages, community examples, and the execution viewer all speak in terms of it, and because the wrapper is where binary and item-linking metadata live.

Two shape mistakes account for most first-week frustration. Returning a bare object in All Items mode (n8n expected an array) produces an error along the lines of "Code doesn't return items properly." And returning an array where json is not an object — say, return [{ json: "hello" }] — fails because the payload must be an object (a set of named fields), not a raw string or number. Wrap scalars in a field: return [{ json: { message: "hello" } }].

Tip: While developing, end your code with return items; (pass-through) and add fields incrementally, running after each change. Pin your input data first (Chapter 10 covers pinning) so every test run uses the same items without re-triggering upstream nodes.

What Is Available Inside Your Code

Inside the Code node you get standard JavaScript (or Python) plus a set of n8n-provided variables and helpers. The essentials:

• $input — the gateway to incoming data. $input.all() returns every item (All Items mode); $input.first() and $input.last() return single items; $input.item is the current item in Each Item mode.
• $json — shorthand for the current item's JSON payload (Each Item mode). In All Items mode, you reach payloads through item.json on each element of $input.all().
• $itemIndex — the position of the current item, in Each Item mode.
• $node["Node Name"] and $("Node Name") — reach back to the output of an earlier, non-adjacent node, exactly as expressions can (Chapter 17). Invaluable when a middle node stripped a field you still need.
• $now and $today — the current moment and the start of today, as Luxon objects. Luxon is a date-and-time library bundled with n8n; its DateTime API ($now.minus({ days: 7 }).toISO()) saves you from JavaScript's rough native date handling.
• $jmespath() — a query helper for plucking values out of deeply nested JSON using the JMESPath query language, often terser than chains of optional property access.
• $workflow, $execution, $prevNode, $runIndex — metadata about the running workflow, the current execution (including its ID, useful for logging), the node that fed you data, and which loop iteration you are on.
• $getWorkflowStaticData() — a small persistent store scoped to the workflow, surviving between executions of an active workflow. Handy for cursors and "last seen ID" checkpoints — though note it persists only for production executions of an active workflow, not manual test runs. Its behavior and limits are discussed with other state options in Chapter 20.
• $env — access to environment variables on self-hosted instances (subject to the instance's security settings; see Chapter 34 for why administrators often restrict this).

Everything in this list is JavaScript. In current versions, Python code receives only its items — _items or _item, as described earlier. If your logic needs the current time, a reach-back to another node's output, or any of the other helpers here, write it in JavaScript.

console.log() works, but its output does not land where beginners expect: it does not appear in the node's output panel. Where it does surface depends on your version and logging configuration — often the browser's developer console for manual test runs, and the server logs for production executions on self-hosted instances. Because that is fiddly, the more dependable inspection trick is to attach what you want to see to the items you return — add a temporary debug field, read it in the output panel like any other data, and delete it when you are done.

What you do not get by default: require() or import of arbitrary packages (see the external libraries section below), file system access, and long-running background work. The sandbox also enforces a time limit; code that loops forever will be terminated, with the details of those limits depending on how your instance's task runners are configured (Chapter 33).

AI-Assisted Drafting

The Code node includes an AI assist feature — in current versions it appears as an Ask AI tab or button in the code editor, availability varying by plan and instance configuration. You describe what you want in plain English ("group the items by customer_id and sum the amount field for each group"), and it drafts JavaScript against your actual input data's shape.

Used well, this is a genuine accelerator, especially for operators who can read code but are slow to write it. Used carelessly, it is a way to install bugs you do not understand. Three habits keep it on the right side of that line:

First, give it real input. The assistant writes better code when the node has actual data to look at, so execute the upstream nodes (or pin sample data) before asking.

Second, read the draft before running it. You are checking two things: that the field names it referenced actually exist in your data, and that the return shape matches the mode you selected. These are exactly the two places drafts most often go wrong.

Third, test with edge-case items. AI drafts are optimized for the happy path visible in your sample. Add an item with a missing field, an empty string, or a null and run again before you trust it. Chapter 30 (Trustworthy AI) develops this "trust but verify" posture for AI features generally.

Watch out: An AI-drafted Code node is still a Code node, and the when-not-to-code ladder still applies. "The AI wrote it for me" does not make a 40-line script more maintainable than the two Set nodes that would have done the same job. If the assistant's draft looks like something the Transformation Toolkit covers, take the hint and use the nodes instead.

Reading Code Errors

When a Code node fails, n8n stops the execution (unless you have configured error handling — Chapter 23) and shows the error on the node in red. Reading these errors is a skill, and it decomposes into three questions: what went wrong, where, and on which item.

What: the error message. JavaScript's standard errors dominate. ReferenceError: x is not defined means you used a variable name that does not exist — usually a typo, or using items without first assigning const items = $input.all(). TypeError: Cannot read properties of undefined (reading 'name') means you drilled into a field on something that was not there: $json.customer.name when customer is missing on this item. SyntaxError means the code itself is malformed — an unclosed bracket or quote — and nothing ran at all.

Where: the line number, which n8n includes in the error details. It points into your code as written in the editor.

Which item: the part beginners miss. A per-item error often names the item index that triggered it. Your code can be correct for 99 items and fail on the one with a null field. When an error appears only sometimes, go look at the specific item that killed it — the execution view (Chapter 21) shows you the input items, and the item index from the error tells you which one to inspect.

Then there are the n8n-specific errors, which are almost always shape problems:

The debugging craft in general — pinning data, bisecting the workflow, comparing a failing execution against a passing one — is Chapter 22's subject, and everything there applies doubly to Code nodes. The one Code-specific technique to internalize now: shrink the code until the error disappears. Comment out the second half and return early. If the error goes away, it lives in what you removed. This bisection converges on the guilty line faster than staring ever does.

External npm Libraries on Self-Hosted Instances

JavaScript's ecosystem lives on npm, the package registry with a library for everything from phone-number parsing to CSV wrangling. Out of the box, the Code node cannot import these — the sandbox blocks require() of anything not explicitly allowed. On n8n Cloud that is where the story ends: you get the bundled helpers (Luxon and friends) and standard JavaScript, and if you need a heavier dependency, the path is a community or custom node (Chapter 15) or an external service called via HTTP.

On a self-hosted instance, you can open the door deliberately. Two environment variables — settings passed to the n8n process when it starts, typically in your Docker configuration — control the allowlist:

• NODE_FUNCTION_ALLOW_BUILTIN names which of Node.js's built-in modules code may import (for example crypto for hashing). A value of * allows all built-ins.
• NODE_FUNCTION_ALLOW_EXTERNAL names which installed npm packages code may import, as a comma-separated list.

Two practical caveats. First, allowing a package is not the same as installing it: the package must actually be present in the environment where the code executes, which for containerized deployments usually means building a small custom Docker image that adds your packages on top of the official one. Second, because Code node execution happens in task runners, the allowlist and the installed packages must be present in the runner's environment — on setups where runners are separate containers, configuring only the main n8n container will not work. (Python libraries are a separate story with their own configuration on the Python runner, and on n8n Cloud the Python option cannot import external libraries at all; the self-hosting docs cover the current mechanism.) The mechanics of runner deployment live in Chapter 33 (Scaling and Performance: Queue Mode and Task Runners), and general self-hosting setup in Chapter 32.

Watch out: Every package you allow is code you are trusting with your workflow data, running on your server. Allowlist specific packages you have vetted rather than using *, and treat "let's just allow everything" as the security smell it is. Chapter 34 (Access, Security, and Secrets for Teams) covers the wider hardening picture.

A worthwhile self-check before you do any of this: if the only reason you want an npm library is date formatting, string manipulation, or JSON querying, the bundled helpers almost certainly already cover it. External libraries earn their setup cost for genuinely specialized work — parsing spreadsheet files, validating phone numbers, heavy cryptography — not for conveniences.

Shell-Level Work: Execute Command and SSH

Below code, there is the shell — the command-line interface of an operating system. n8n exposes it through two nodes with a crucial difference between them: where the command runs.

Execute Command: the local shell

The Execute Command node runs a shell command on the machine hosting n8n itself. Type a command into its Command field, and when the node executes, that command runs and the node emits an item containing the command's stdout (normal output), stderr (error output), and exit code (the number a command returns to signal success — zero — or failure — anything else).

This node is self-hosted only; n8n Cloud disables it, for the obvious reason that Cloud customers share infrastructure and cannot be allowed to run arbitrary commands on it. And on self-hosted Docker deployments, "the machine hosting n8n" means inside the container — a minimal Linux environment that has only the tools baked into the image. The command-line tool you use daily on the host is probably not in there. If your workflow needs a specific binary, you are back to building a custom image that includes it, exactly as with npm packages above.

Legitimate uses are real but narrow: invoking a command-line tool that has no API (a media converter, a specialized file processor), poking at files that n8n itself wrote, or gluing in a script that predates your n8n adoption. For each use, ask whether an HTTP API, a community node, or a Code node could do the same job inside the data model — commands return flat text you must parse, while nodes return structured items.

SSH: the remote shell

The SSH node runs commands on a different machine, connecting over SSH (Secure Shell, the standard encrypted protocol for remote server administration). You create an SSH credential — host, username, and either a password or, much better, a private key — through the standard credential system from Chapter 11 (Credentials from Zero), and the node then executes your command on the remote host and returns its output the same way Execute Command does. It can also upload and download files.

Because the command runs elsewhere, the SSH node works on n8n Cloud as well as self-hosted — the Cloud instance is only the client. This makes it the standard answer for "my automation needs to touch my server": restart a service, run a deployment script, fetch a log file, trigger a backup.

Handling command output and failure

Both nodes hand you raw text, and both deserve defensive habits. Parse stdout deliberately — if you control the remote script, have it print JSON and parse it with a small Code node or the expression JSON.parse(), rather than splitting text on spaces and praying. Check the exit code with an If node (Chapter 9) before proceeding, because a command can "succeed" as far as n8n is concerned while the work itself failed. And quote carefully when interpolating expressions into commands: a filename with a space, or worse, a value that came from an external form, can change the meaning of your command entirely. Never interpolate untrusted input — anything that originated from a webhook or user submission — into a shell command without strict validation; this is called command injection, and it is how servers get taken over.

Watch out: A destructive shell command in a workflow is a loaded weapon on a timer. rm (delete), DROP, restarts, and anything with --force deserve a dry-run first, an If-node guard checking its inputs, and ideally a manual-approval step for the irreversible cases. Workflows run unattended at 3 a.m.; they do not hesitate, and they do not double-check. Reliability patterns for exactly this kind of caution are Chapter 24's subject.

Keeping Code Nodes Maintainable

You will accumulate Code nodes; every real n8n estate does. The difference between an estate that ages well and one that rots is a handful of habits:

One job per Code node. A node called "Normalize phone numbers" that normalizes phone numbers is auditable. A node called "Code" that normalizes phones, dedupes, calls $getWorkflowStaticData(), and reformats dates is a future incident.

Name the node for what it does. "Code" tells a reader nothing. "Compute weighted lead score" tells them almost everything.

Keep the shape boring. Read items, transform, return items. Resist cleverness with side channels, static data, and reach-backs to distant nodes unless there is no alternative — every one of those is invisible coupling.

Comment the why, not the what. // The CRM sends dates as DD.MM.YYYY, which Luxon won't parse without an explicit format will save someone an afternoon.

Revisit the ladder on every edit. Workflows evolve, and n8n ships new nodes and functions steadily. The Code node you legitimately needed a year ago may be replaceable by a standard node today — and each replacement makes the workflow more legible than it was.

The through-line of this chapter is the through-line of the whole volume: n8n gives you a gradient of power, from a dropdown, to an expression, to a transformation node, to code, to a shell. Fluency means moving along that gradient deliberately — as far as the problem demands and not one step further. Chapter 20 completes the data story with the two kinds of persistence you have not met properly yet: binary files and Data Tables.

Files and Remembered State: Binary Data and Data Tables

Everything you have built so far in this volume has worked on structured data: items with fields, flowing from node to node, transformed and gone the moment the execution ends. This chapter closes two gaps that structured, ephemeral data leaves open. The first is files — PDFs, spreadsheets, images, exports — which n8n treats as first-class citizens but handles through a distinct set of nodes and a distinct slot on each item. The second is memory: a workflow forgets everything the instant a run finishes, and yet real automations constantly need to remember things between runs — which records they have already processed, where a sync left off, what a running total stands at. n8n gives you three tools for that, from a built-in mini-database called Data Tables down to a scratch area stapled to the workflow itself, and this chapter ends with a decision table for choosing among them and the one pattern — dedupe (short for de-duplication: making sure each record is processed once) between runs — that you will reuse more than any other.

How n8n Carries a File

Recall from Chapter 16 (Reading and Reasoning About Data: The Item Model in Practice) that every item flowing through a workflow has a JSON side — the structured fields you see in the Table and JSON views. What Chapter 16 only touched on is that each item also has a second, optional side: binary data. Binary data is n8n's term for file content — any blob of bytes that is not itself structured JSON, whether that is a PDF, a JPEG, a ZIP archive, or a plain text file.

Binary data lives in named slots on the item called binary properties. Most nodes that produce a file put it in a property named data by default, but the name is arbitrary and an item can carry several files at once: an email trigger, for example, may attach attachment_0, attachment_1, and so on to a single item. Each binary property carries the raw bytes plus a little metadata — a file name, a MIME type (the standard label that identifies a file's format, such as application/pdf or image/png), and a size.

When a node outputs binary data, the output panel grows a Binary tab next to Table, JSON, and Schema. Open it and you will see each binary property as a card with the file name and type, plus buttons to view the file inline (for images and PDFs) or download it to your computer.

Tip: The Binary tab's download button is your best debugging friend when working with files. If a generated spreadsheet looks wrong three nodes later, download it at each step and open it locally — you will find exactly where it went wrong far faster than by staring at metadata.

The essential mental model: JSON data is what nodes read and transform; binary data is cargo. Most transformation nodes (Edit Fields, Filter, Sort) pass binary cargo through untouched while they work on the JSON side. To actually work with file content — parse it, generate it, rename it — you use the file-handling nodes below.

Extract From File: Turning Files into Data

Extract From File is the node that opens a file and turns its contents into JSON your workflow can reason about. You pick an operation matching the file format, point the node at the binary property holding the file, and it emits items with the parsed content on the JSON side.

The operations cover the formats you will meet most often:

• CSV, XLSX, XLS, ODS — spreadsheet formats. The node reads the sheet and emits one item per row, with column headers becoming field names. Options let you tell it whether the first row is a header row, which sheet to read in a multi-sheet workbook, and whether to keep raw values or formatted ones. This is the single most common file operation in practice: someone emails you a CSV export, and one node later it is rows you can filter and route like any other data.
• JSON — parses a .json file into items. Useful when an API or export hands you a file rather than a response body.
• PDF — extracts the text content of a PDF into a text field, along with basic metadata like the page count. This is text extraction, not layout understanding: you get the words, roughly in reading order, not the tables and columns as visual structures.
• HTML, ICS, RTF, text — parse a saved web page, a calendar file, a rich-text document, or a plain text file respectively into JSON fields. (XML has no Extract From File operation of its own; the separate XML node converts XML to and from JSON when you need it.)
• Move File to Base64 String — converts the binary file into a Base64 string, which is a way of encoding raw bytes as plain text so they can travel inside a JSON field. You will use this when an API expects a file embedded in a JSON request body, and when passing images to AI models that accept Base64 input (see Chapter 27, First AI Steps: Single Calls That Earn Their Keep).

Images deserve a special note. Extract From File does not "read" an image — there is no text to parse. What you can do is convert an image to Base64 for an AI vision model to describe, or manipulate it (resize, crop, composite) with the separate Edit Image node. If you need text out of a picture or a scanned document, that is optical character recognition (OCR), which n8n delegates to external services or AI models rather than doing natively.

Watch out: A scanned PDF is a stack of photographs, not text. Extract From File's PDF operation will return empty or near-empty text for scans, because there are no embedded characters to extract. If your PDFs come from a scanner or a phone camera, plan for an OCR step via an AI model or a dedicated OCR API instead.

Convert to File: Turning Data into Files

Convert to File is the mirror image: it takes items' JSON data and produces a file in a binary property. Its operations mirror most of Extract From File's — CSV, XLSX, XLS, ODS, HTML, ICS, JSON, RTF, plain text — plus Move Base64 String to File, which decodes a Base64 string field back into a real file (the reverse of Extract From File's Move File to Base64 String, and the standard move when an API returns a file embedded in JSON — AI image generators are the classic case). One gap to expect: there is no Convert to PDF operation. Extract From File can read a PDF's text, but producing a PDF means rendering HTML through an external service or a community node, not this node.

Two behaviors matter for spreadsheet and CSV output. First, the node gathers all incoming items into a single file by default — fifty items in, one CSV with fifty rows out, sitting in the binary property of a single output item. That is almost always what you want for reports and exports. Second, you control the output binary property's name (default data) and can set a file name option so the eventual attachment is called weekly-report.csv rather than something generic. Downstream nodes that send files — email nodes, chat nodes, cloud storage uploads — have an input field where you name the binary property to attach, and this is where those names connect.

The classic pairing is Convert to File feeding an email node: query rows from somewhere, shape them with the transformation toolkit from Chapter 18, convert to XLSX, attach, send. You will see the full version of this in Volume H's worked projects.

Read/Write Files from Disk: Local Files, With Caveats

The Read/Write Files from Disk node does what it says: reads files from, or writes files to, the disk of the machine n8n is running on. Reading supports glob patterns — wildcard paths like /data/inbox/*.csv — so one node execution can pick up every matching file, emitting one item per file with the content in a binary property.

The caveat is the phrase "the machine n8n is running on."

On n8n Cloud (the hosted service), that machine is a managed container you do not control. The node runs, but n8n confines disk access to a dedicated files directory under its own home — reach outside it and you get an "access to the file is not allowed" error — and nothing you write there is guaranteed to survive a restart or redeploy. On Cloud, treat this node as a scratch pad at most, and use cloud storage nodes (Google Drive, Dropbox, S3, FTP/SFTP) as your real file system.

On self-hosted n8n, the node is genuinely useful — with two things to get right. First, recent n8n versions restrict disk access to a designated files directory by default; to read or write anywhere else, widen the allow-list with the N8N_RESTRICT_FILE_ACCESS_TO environment variable (semicolon-separated paths). Second, if you run n8n in Docker (the common setup; see Chapter 32, Self-Hosting from Zero), remember that paths are inside the container. A file at C:\exports\report.csv on your server does not exist from n8n's point of view unless you have mounted that folder into the container as a volume. This container-path gotcha is the number-one confusion new self-hosters hit with this node.

Watch out: Two errors dominate this node. A "file not found" for a file you can plainly see on your server is almost always the container-path problem: mount the host folder into the container (a bind mount — Docker's mechanism for making a host folder visible inside a container) and address it by its in-container path. An "access to the file is not allowed" error instead means the path falls outside n8n's permitted file directory — add it to N8N_RESTRICT_FILE_ACCESS_TO. Either way, keep n8n's own application directories (its .n8n state folder) out of your write paths — treat them as off-limits.

Renaming and Moving Binary Properties

Because binary properties are named, and because downstream nodes reference them by name, you will sometimes need to rename or rearrange them. Three common situations:

A node expects a different property name. Most file-consuming nodes have an "input binary field" setting — check there first, because pointing the node at the right name is easier than renaming the property. Likewise, most file-producing nodes let you set the output property name, so you can often make names agree at the source.

You need to merge files from two branches onto one item. The Merge node (Chapter 9, Flow Control) combines items, and binary properties ride along; give the two files different property names before merging so one does not overwrite the other.

You genuinely need to rename or restructure. The Code node (Chapter 19, Code and Command: When Clicks Are Not Enough) can manipulate the binary side directly. Renaming attachment_0 to invoice across all items looks like this:

for (const item of $input.all()) {
  if (item.binary?.attachment_0) {
    item.binary.invoice = item.binary.attachment_0;
    delete item.binary.attachment_0;
  }
}
return $input.all();

The same technique splits multiple attachments into separate items (one item per binary property) when you want to loop over files individually — a recipe Volume H's cookbook covers in full.

Large Files and Memory, in Plain English

Here is the part of file handling nobody warns you about until something falls over. By default, n8n holds binary data in memory — in the working RAM of the n8n process — for the life of the execution. The file also becomes part of the execution data, the saved record of the run you inspect in the executions list (Chapter 21, Executions: The System of Record).

For a 200 KB CSV this is invisible. For a 300 MB video it is a problem, and the problem multiplies in ways that surprise people:

• Every item carries its cargo through every node. A big file passing through ten nodes is held, referenced, and recorded at each step.
• Loops and batches multiply the load. Fifty items each carrying a 20 MB PDF is a gigabyte of file data in one execution.
• Parsing inflates. Extracting a 100 MB XLSX into items does not just hold the file; it also creates JSON for every row, which can dwarf the original.

When memory runs short, the symptoms are ugly: executions that die partway with a crash rather than a tidy error, an editor that becomes unresponsive, or on self-hosted setups a restarting container. On n8n Cloud, memory is one of the real ceilings that separates plan tiers — larger plans run with more headroom.

Self-hosted instances can change where binary data lives. Setting the binary data mode to filesystem (the environment variable N8N_DEFAULT_BINARY_DATA_MODE=filesystem) makes n8n write file cargo to disk and pass lightweight references between nodes instead of hauling bytes through RAM — one setting, and the single biggest stability upgrade for file-heavy self-hosted workflows. There is also an S3 external storage mode — an Enterprise-licensed feature — that offloads binary data to object storage entirely; configuring that belongs to Chapter 33 (Scaling and Performance: Queue Mode and Task Runners), alongside the other capacity levers.

Whatever your hosting, three habits keep file workflows healthy. Pass references, not payloads: if a file already lives in cloud storage, move its URL or ID between workflows and download only where needed. Trim early: filter to the items you need before the step that attaches or generates files, not after. And slim the record: for high-volume file workflows, consider the workflow's execution-saving settings (Chapter 10, Test, Fix, Activate, and Keep It Tidy) so you are not archiving gigabytes of attachments as run history.

The State Problem: Workflows Forget Everything

Now the second gap. Every execution starts from a blank slate: nodes run, data flows, the run ends, and nothing carries forward. This statelessness is a feature — it is why runs are reproducible, why you can retry a failed execution safely, and why executions are easy to reason about. But it collides with reality quickly:

• A workflow polls for new orders every ten minutes. How does it know which orders it already handled, so nobody gets two shipping emails?
• A sync pulls records modified since the last run. Where does "the last run's timestamp" live?
• An escalation rule fires after the third failed check. Who is counting?
• A lookup table maps country codes to account managers. Where does it sit so five workflows can share it?

Each of these needs state — remembered information that outlives a single execution. n8n offers three homes for state, in ascending order of power: workflow static data, Data Tables, and external storage. Take them in order of importance, which happily is not the same order.

Data Tables: The Built-In Memory

Data Tables are n8n's built-in, spreadsheet-like storage: simple tables of rows and columns that live inside your n8n project, persist between executions, and are readable and writable both from workflows and by hand in the UI. They are the default answer to the state problem, and for most operators they remove the need to bolt on an external database at all.

You will find them in the Data tables tab, which sits alongside Workflows and Credentials in your project's navigation. Create a table, give it a name like processed_orders, and define columns, each with a type — text, number, boolean (true/false), or date. n8n adds housekeeping columns of its own (an id and created/updated timestamps). From this tab you can also see and edit rows directly — add one, fix a typo, delete a stray — which turns out to matter operationally more than it sounds.

Tip: The UI editability of Data Tables is a quiet superpower. You can seed a lookup table by hand before any workflow exists, inspect exactly what your dedupe ledger remembers when debugging, and surgically delete one row to make a workflow reprocess one record — no database client, no exports.

Workflows talk to tables through the Data Table node. Its row operations are the verbs of the whole system:

• Insert — add a row.
• Get — fetch rows matching filter conditions you build in the node (field, comparison, value — the same style as the If node's conditions).
• Update — change rows matching a filter.
• Delete — remove rows matching a filter.
• Upsert — the workhorse: update the row if a match exists, otherwise insert it. One operation, no branching, no "did it exist?" check. The name is a contraction of update-plus-insert.
• If Row Exists / If Row Does Not Exist — filters rather than fetchers: each passes an input item through unchanged when a matching row does (or does not) exist in the table, and drops it otherwise. They return nothing from the table — they only gate items — and they exist almost entirely for the dedupe pattern at the end of this chapter.

The node can also create, list, update, and delete tables themselves, which is occasionally handy for provisioning but rarely needed day to day.

Know the boundaries. Storage is capped — by default all Data Tables in an instance share a total allowance of around 50 MB, and n8n warns you as you approach it; hit the cap and inserts start failing, in the UI and in workflows alike. Self-hosted operators can raise the ceiling with an environment variable; on Cloud the allowance is fixed by the platform. Structurally, tables are flat: no joins between tables, no SQL queries, no aggregate functions — filters and the row verbs above are the whole vocabulary. That is not a criticism; it is the design. Data Tables are for operational state — dedupe ledgers, sync cursors (bookmarks recording where a run left off), small queues, lookup tables, counters — not for becoming your data warehouse. One practical note: Data Tables are a relatively recent addition to n8n, so a self-hosted instance that has fallen well behind on upgrades may not show the tab at all.

Watch out: Around 50 MB sounds small, and for files it would be — but state rows are tiny. A dedupe ledger storing an order ID and a timestamp per row fits hundreds of thousands of entries in that space. What blows the cap is storing payloads: full API responses, article text, JSON blobs "just in case." Store keys, timestamps, and status flags; leave the payloads in the system of record they came from.

Workflow Static Data: The Sticky Note

Below Data Tables sits something older and smaller: workflow static data, a scrap of JSON stored on the workflow itself that survives between executions. You reach it only from a Code node, via $getWorkflowStaticData('global') (one shared object for the whole workflow) or $getWorkflowStaticData('node') (a private object per node). Read a property, assign a property, and n8n saves the object when the execution finishes:

const state = $getWorkflowStaticData('global');
const lastRun = state.lastRun ?? '2000-01-01T00:00:00Z';
state.lastRun = new Date().toISOString();
return [{ json: { since: lastRun } }];

This is exactly how n8n's own polling triggers remember what they have seen. It is the right tool for one small marker — a last-run timestamp, a cursor, a counter — when creating a whole table feels like ceremony.

But it comes with a trap that catches nearly everyone once:

Watch out: Workflow static data is only saved after production executions — runs started by a real trigger on an active workflow. Manual test runs from the editor read static data but do not persist changes to it. Test your "remember the timestamp" logic with Execute workflow and it will look broken — the value never sticks — while working perfectly once the workflow is activated. Design for it, test around it, and do not spend an afternoon debugging it.

Beyond that quirk: static data is invisible (no UI shows it — you write debug code to inspect it), it belongs to one workflow (no sharing), it can lose writes if two executions overlap, and it is meant to stay tiny. The honest guidance: use it for polling-style markers, and the moment you want to see your state, share it, or store more than a few values, move to a Data Table.

External Storage: When You Graduate

The third home for state is anything outside n8n: a real database like Postgres or MySQL, a key-value store like Redis, or a human-friendly grid like Google Sheets, Airtable, or NocoDB. Chapter 12 (Navigating and Operating the Integration Catalog) covers the nodes; the question here is when state belongs outside.

The graduation signals are concrete. You need more capacity than the Data Table allowance comfortably holds. You need queries Data Tables cannot express — joins, aggregations, full-text search. Other systems need to read or write the same state — a dashboard, a BI tool, another application — so n8n should be a client of the data rather than its container. You need database-grade guarantees under heavy concurrency. Or non-technical teammates must own and edit the data, in which case a Google Sheet or Airtable base — slower and subject to API rate limits, but visible and editable by anyone — is the pragmatic choice, with the caution from Chapter 24 that spreadsheets make weak high-concurrency ledgers.

Until one of those signals appears, stay native. Every external store adds credentials to manage (Chapter 11), a service to keep alive, and a failure mode your error handling must cover (Chapter 23).

Choosing a Memory: The Decision Table

Default to Data Tables. Drop down to static data for a single invisible marker; step out to Sheets when humans own the data; step up to a database when size, queries, or other systems demand it.

The Dedupe-Between-Runs Pattern

One pattern justifies this whole chapter, because nearly every recurring workflow needs it: process each thing exactly once, across runs. New rows arrive, your schedule fires every ten minutes, and the same records will show up in poll after poll. Without remembered state you re-send, re-create, and re-charge. Chapter 18 covered removing duplicates within one run; this is the cross-run version, and it comes in two strengths.

The one-node version. The Remove Duplicates node (the same node from Chapter 18) has a second operation: Remove Items Processed in Previous Executions. Feed it your items, tell it which field identifies each one — an order ID, an email address, a URL — and it silently keeps a history of seen values, passing through only items whose key is new. Options let you scope the history to the node or share it workflow-wide, cap the history size, and — via a third operation — clear the history when you want a fresh start. For straightforward "only handle new ones" polling, this is the fastest correct answer, and it needs no table at all.

Its limits are the flip side of its convenience: the history is invisible (you cannot browse what it remembers), bounded (a capped history can forget old keys, letting ancient items sneak back in), and single-purpose (it answers "seen before?" and nothing else).

The ledger version. When you want dedupe you can see, share, and extend — status per record, timestamps, retry counts — build it on a Data Table. Create a table such as processed_orders with a key column (plus whatever else earns its place: a status, a processed_at). Then, per batch of candidate items:

1. Filter to new: a Data Table node with the If Row Does Not Exist operation, matching key against each item's ID. Items already in the ledger are dropped; genuinely new items pass through untouched. (When you also need to read what the ledger recorded — a status, a retry count — use a Get operation followed by an If node instead, since the filter operations gate items but return nothing.)
2. Do the work on what passes: send the email, create the invoice, post the message.
3. Record: insert a row with the key and a timestamp, after the work succeeds.

Two design choices deserve a sentence each, with the full treatment deferred to Chapter 24 (Reliability Patterns: Idempotency, Pacing, and Recovery). Choose a stable key: dedupe is only as good as its key, so use the source system's real identifier — an order ID, a message ID — never a fragile stand-in like a name or a subject line. Choose when to record: marking after the work (as above) means a crash mid-run can cause a rare duplicate action; marking before the work — where Upsert shines as a claim-the-key move — means a crash can cause a rare missed action. Duplicates-possible versus misses-possible is a genuine trade-off, and Chapter 24 names it (at-least-once versus at-most-once) and shows how to pick per use case.

The ledger version is the one you will meet again: Chapter 24 hardens it, and both worked projects in Volume H — the Lead Machine and the Support Copilot — carry a dedupe ledger at their core.

Where This Leaves You

You can now move files through workflows deliberately — parsing them with Extract From File, producing them with Convert to File, handling disks and property names without surprises, and respecting memory before it bites. And you have broken statelessness on purpose: static data for the smallest markers, Data Tables as your default remembered state, external stores when the signals say graduate, and a dedupe pattern that turns "runs every ten minutes" into "handles each thing once." That combination — files in motion, state at rest — completes the data toolkit of Volume D. Next, Volume E turns to what happens when workflows meet the real world: executions as the system of record, debugging, and the craft of failing well.