Node-RED didn't break at 20,000 messages a second. It broke when the database did.

Somewhere in your plant there is a small program moving production data into a database. Somebody set it up years ago. It has quietly worked ever since, and nobody has looked at it since the week it went in.

Now someone has sent you a link to a newer tool and told you the old one is a toy. I wanted to know whether that was true, so I built the same job twice, ran real data through both, and broke things until one of them fell over.

The thing that fell over was not the thing anybody warns you about.

Both sides do the same job, and it is the job most plants actually need: take readings off the network, add the missing information about which machine they came from, convert the units into something sensible, and write them into a database. One side is Node-RED, the tool you probably already have. The other is Redpanda Connect, the newer one being recommended.

Same data in. Same database out. Everything that happened in between is the result.

The speed argument is empty#

I pushed the rate up in steps until I was sending twenty thousand readings a second, which is far more than any real plant produces. A busy bottling line makes a few hundred a second.

Both of them handled all of it. Not most of it. Every single reading, at every rate, with nothing lost and nothing delayed by more than about a second. The old tool was actually the quicker of the two at normal plant rates.

So if someone is selling you a replacement on the grounds that your current setup is too slow, ask them at what rate it fails. In my test it never did.

Then I switched the database off#

This is closer to what actually goes wrong on a plant. Databases do not usually explode. They get restarted for patching, or a disk fills, or the network to the server drops for a minute.

So I left both running and turned the database off for sixty seconds.

The new tool lost nothing. Every reading that was produced during that minute turned up in the database afterwards.

The old one lost a third of them. About an hour's worth of production readings, gone permanently. Not delayed, not sitting in a queue somewhere waiting to be recovered. There is simply a hole in the record where that minute should be.

I ran the whole thing again because I did not believe it, and got the same answer to the decimal point.

Both strips above are the same hour of production. The difference is a piece cut clean out of the middle of one of them.

Why one kept the data and the other didn't#

It comes down to one decision, and it is easiest to think about like signing for a delivery.

When a reading arrives, something has to tell the network it has been dealt with, and after that the network stops keeping a copy.

The new tool signs for the delivery only after the reading is safely written to the database. If the write fails, it never signs, so the network hands the reading back and it tries again.

The old tool signs the moment the reading arrives at its door, before it has tried to write anything. By the time the write fails, the network has already thrown its copy away, and there is nothing left to retry. The error message goes in a log nobody reads.

Neither of these is a bug. Both are working as designed. Only one of those designs survives somebody rebooting a database server.

You can build the retry behaviour into the old tool by hand, and people do. But you are then building a proper holding queue inside a program that has nowhere durable to hold anything, and doing that job properly is bigger than the job it is protecting.

The part I did not expect#

Then I killed both programs outright, mid-flow, and left them dead for fifteen seconds. I expected that to be the worse failure.

Both came through it almost perfectly. The old one actually lost slightly less.

The reason is simple once you see it. A program that has been killed is not signing for anything, so the network holds everything and hands it over when the program comes back.

Which leaves a genuinely strange conclusion: the old tool survives its own death far better than it survives a database hiccup. Killing it is safe. Leaving it running while something underneath it is broken is what destroys your data.

That is worth sitting with, because it is the opposite of where most people point their monitoring. Everyone watches whether the program is running. Almost nobody watches whether the thing it writes into is healthy.

The thing that actually settles it#

Here is the result that took one command and decided the whole question.

The new tool ships with seventy-eight ways to connect to things. The number that can talk to a PLC is zero. No OPC UA, no Modbus, no Siemens S7, nothing that speaks to actual equipment.

That is not an oversight. It is a statement about where the tool starts, which is well above the plant floor. Everything below that line, which is the half of the job involving real machines, it does not do at all.

So what do you do with the box in your plant#

Leave it where it is, and stop letting it write to the historian.

Node-RED is still the only one of the two that can talk to your equipment, and that is not a small thing. Keep it doing what it is good at: getting data out of machines and onto the network.

Then put the new tool on the other side, between the network and your database, where losing a minute of production to a server reboot stops being possible. The trade is that you will occasionally get the same reading written twice, which for a historian is usually harmless.

The pitch says the new tool replaces the old one. It does not. It replaces one job the old one has always done badly, and leaves everything else exactly where it was.

Every measurement, the full setup, and the three mistakes I made in my own test rig before the numbers meant anything are in the lab write-up.

For the wider question this piece keeps brushing against, why a plant would want a broker in the middle at all, see You don't buy a broker for the first consumer.