Ignition 8.3 speaks Kafka now, which deletes the middle of your architecture drawing

Nearly every drawing I have seen for getting plant data onto Kafka has the same shape. The gateway publishes MQTT, something in the middle subscribes to it, and that something writes to Kafka. The middle box is usually a small integration service that exists purely to translate one protocol into another.

I built that stack on Kubernetes to test how it behaves when things fail. Then I listed the modules in the Ignition container and found I had drawn a box that no longer needs to exist.

What is in the gateway already#

$ kubectl -n ignition exec gateway-0 -- \
    ls /usr/local/bin/ignition/user-lib/modules/ | grep -iE "kafka|event"
EventStream-module.modl
Kafka Connector-module.modl

Ignition 8.3 ships a Kafka connector and an Event Streams module in the box. The connector talks to Kafka from the gateway itself, and Event Streams treats Kafka as both a source and a handler — it can consume from a topic and publish to one.

Which changes the shape of the problem. If the only job of your middle box was translating Ignition into Kafka, the gateway can now do it directly, and the box is a component you are maintaining, patching and monitoring for no remaining reason.

The box still earns its place if it does more than translate — buffering while the far end is down, fanning one stream out to several destinations, or handling protocols the gateway does not speak. But "we need something to get this onto Kafka" stopped being a reason on 8.3.

Worth checking your own gateway rather than taking my word for it, because this is recent and most architecture diagrams in circulation predate it.

The Kubernetes part, which behaved#

The reason I built this on a cluster was to find out what happens when a pod dies, so I wrote five hundred known records into Redpanda and deleted its pod outright.

Seven seconds to come back, with all five hundred records still there. Then I did the same to the Ignition gateway: sixteen seconds from deletion to the gateway answering an HTTP status check. The difference between the two is a Java virtual machine starting.

Both unattended. Nobody logged in, nobody restored anything.

That is genuinely the whole Kubernetes story here, and it is a dull one, which is the correct outcome for infrastructure. The StatefulSet promise — stable name, stable storage — held for both.

The detail that will actually bite you#

Look at the second row. Both pods came back with the same name and a different IP address.

That is the one habit that has to change coming from a plant floor. The conventional SCADA instinct is to write the address down: put the historian's IP in a config file, type the broker's address into a gateway screen, note it on the drawing. Every one of those breaks the first time a pod moves, and pods move for entirely routine reasons — a node drains, a memory limit trips, somebody applies an update.

Address things by name and it simply reconnects. The name is the stable thing Kubernetes is offering you; the IP is explicitly not.

Who is actually expensive#

One more result worth having, because it contradicts the usual objection.

When people hear "we are adding a streaming platform to the plant" they picture something heavy. Redpanda idled at 250 MiB and two millicores. The Ignition gateway sitting next to it used 859 MiB. The streaming platform is the cheap tenant; the SCADA gateway you already run is the expensive one.

What I would take from this#

Check your gateway's module list before you commission a bridge. If it is 8.3 and the bridge only translates protocols, you may be about to build and maintain something the platform already does.

If you do run this on Kubernetes, the single most important change is addressing by name rather than by IP, because that is the thing that quietly breaks later.

And the honest limitation on all of it: this measures processes returning, not a plant resuming. A gateway answering HTTP in sixteen seconds still has device connections to re-establish, and anything that was mid-transaction when the pod died is a different question that this test does not answer. Kubernetes will restart your software reliably. It has no opinion about your process.

The manifests, both kill tests and the full module list are in the lab write-up.