4 Aug 2026 · 6 min read
Node-RED didn't break at 20,000 messages a second. It broke when the database did.
node-red · integration · architecture · mqtt · ot · uns · open-source
Somewhere in your plant a Node-RED instance is moving production data into a database. It has run for two years. Now someone has sent you a link to Redpanda Connect, and the pitch is the one this industry always makes: the thing you are running is a toy, and here is the industrial-strength replacement.
I wanted a number rather than an opinion, so I built the same pipeline twice and ran it until something broke. What broke was not what the pitch says breaks, and the failure that mattered had nothing to do with speed.
THE RIG
Redpanda Connect is the project that used to be called Benthos, acquired and renamed in 2024. If you have not met it, the closest familiar thing is a function block diagram written as text: you declare one input, a list of processors, and one output, and the runtime handles connection, retry and batching for you. It is a single Go binary with no editor and no canvas. The whole pipeline for this test is 48 lines of YAML.
Both lanes do the identical job, which is the job most plants actually need: take JSON off MQTT, attach the asset context, convert vendor units into SI, batch, and insert. Same broker, same QoS, same 500-message batch, same database. I gave the database far more resources than it needed so that it could never become the thing under test.
The finding that wasn't there
I ramped from 500 to 20,000 messages a second, sixty seconds a step, measuring what each pipeline landed against a per-site sequence number so loss would be arithmetic rather than estimation. The publisher genuinely reached 19,894 a second, so the top step is real and not an artefact of a lazy load generator.
EXPERIMENT 1 — THROUGHPUT
Bars are to scale with the rate. Each step ran for sixty seconds with a fifteen-second drain, and every step recorded the exact sequence range it published, so the delivery column is a subtraction rather than a sample.
Both delivered everything. Not "close to everything" — 2,310,000 of 2,310,000 each, no duplicates, no gaps, at every rate. Neither ran more than a second behind the wire, and Node-RED's worst lag was actually lower than Connect's at five of the six steps, because Connect's one-second batch window dominates when traffic is light.
For context, a busy bottling line generates a few hundred messages a second. The throughput case for ripping out Node-RED is, at plant scale, a fantasy. If somebody is selling you a migration on performance grounds, ask them for the rate at which your current thing fails, and watch what happens.
Then I stopped the database
EXPERIMENT 2 — A 60-SECOND DATABASE OUTAGE
Same stream, 2,000 a second. Forty-five seconds in I stopped the database for a minute, started it again, and gave both pipelines nearly two minutes to catch up.
Connect landed all 360,000, with two duplicates. Node-RED landed 240,500 and lost 119,500 readings, a third of the stream, permanently. There is no backlog to replay and no file to recover them from. That hour of production simply has a hole in it.
I ran the whole thing a second time before I believed it, and both numbers came back identical to the decimal place.
THE SHAPE OF WHAT WAS LOST
Bucketing by the moment each reading was produced rather than written shows what actually happened, and it is not a slowdown. Node-RED recorded nothing at all for eleven consecutive windows, a single contiguous block precisely as wide as the outage. Connect's line does not so much as dip.
One caveat that cost me an hour and belongs in anyone's runbook: Connect's recovery is not instant. For several minutes after the database came back it crawled at a couple of messages a second and looked thoroughly wedged, and I had already started writing it up as a failure when it drained the entire backlog in one burst. If you test this yourself, give it far longer than seems reasonable before concluding anything.
THE MECHANISM
Acknowledged on receipt. If the write fails afterwards the broker has already let the message go, so the error handler has nothing to hand back. You never get duplicates. You do get holes.
Acknowledged only after the insert lands. A failed batch stays owed and comes back. You never get holes. You do occasionally get the same reading twice, so whatever is downstream has to tolerate that.
The reason is one design decision, and it is not a bug in either. The Connect output does not acknowledge a message to the broker until the insert has committed, so a failed batch stays owed and the broker hands it back. The Node-RED flow acknowledges on receipt, the moment the message enters the canvas. By the time the insert fails, the broker has already discarded its copy, and the catch node is holding the only remaining reference to data that is now homeless.
You can build retry into a Node-RED flow. People do. But you are then hand-rolling a durable queue inside a process with no durable storage, and the honest version of that work is larger than the flow it protects.
The part I did not expect
Then I killed both processes outright, mid-stream, and left them dead for fifteen seconds. Connect lost 180 messages. Node-RED lost 80, and came out marginally ahead.
That looks like a contradiction until you see the mechanism. A dead consumer is not consuming, so the broker holds everything against its session and redelivers on reconnect. Nothing was acknowledged, so almost nothing was lost. Which leaves the genuinely strange conclusion that Node-RED survives its own death far better than it survives a database hiccup. Killing it is safe. Letting it run while the thing underneath it is broken is what costs you the data.
WHAT EACH ONE COST TO RUN
The resource numbers point the same way. Node-RED tops out at one saturated core, because Node.js is single-threaded and no amount of tuning changes that. Connect spread past 100% and used half the memory. On the six-core box I tested, that difference bought nothing at all. On a single-core edge gateway it is the ceiling.
The thing that actually decides it
EXPERIMENT 3 — PROTOCOL COVERAGE
kafka · mqtt · amqp · nats · sql · s3 · gcp · redis · http · …
opc ua · modbus · s7 · bacnet · dnp3 · sparkplug
Here is my favourite result, and it took one command. Redpanda Connect ships 78 input connectors. The number that speak to a PLC is zero. No OPC UA, no Modbus, no S7, no BACnet, no DNP3, not even Sparkplug.
That is not an oversight, it is a statement of where the project lives. It
begins at the broker. Everything to the left of the broker, which is the half of
the job that involves an actual plant, it does not do. This is precisely why the
United Manufacturing Hub maintains its own fork, benthos-umh, whose main
contribution is adding OPC UA, S7 and Modbus to exactly this engine — the same
engine I found underneath UMH Core when I pulled
it apart.
One more thing worth stealing regardless of what you run. Connect has a unit test runner in the same binary that runs the pipeline. I wrote four tests for my unit conversions and one failed immediately, catching a real bug in my own mapping: an unrecognised tag was landing with a null unit rather than the fallback I thought I had written. All 2.31 million messages of load testing never touched it, because every one of them used a tag I already knew about. The bug would have surfaced six months later, at a plant, when somebody added a sensor.
Where this leaves the Node-RED box in your plant
WHERE THE DECISION ACTUALLY IS
Leave it alone, and stop letting it write to the historian.
Node-RED remains the only one of these two that can talk to your equipment, and its flow is a plain file you can review and rebuild, which is the argument I made when everybody was busy fighting about the editor. Keep it doing acquisition, protocol translation, and the awkward local jobs it is genuinely good at, and let it publish to the broker where an acknowledgement means something.
Put Connect on the other side of that broker, where its at-least-once contract turns a database restart into two duplicate rows instead of a hole in your production record. That means whatever you write into has to tolerate seeing a reading twice, which for a historian keyed on timestamp and tag is usually free.
The pitch says Connect replaces Node-RED. It does not. It replaces the one job Node-RED has always done badly and was never designed to do well, and it leaves the rest exactly where it was.
The full rig, every measurement, and the three harness bugs I had to fix before any of the numbers meant anything, are in the lab write-up.
Keep reading
Newsletter
New essays, by email.
SCADA, cloud, AI, and the plant floor — a short email when something new is published. No noise, unsubscribe anytime.