Only what somebody wrote down gets checked

Somewhere in your plant is a drawing of how it was built. It was accurate on the day it was signed.

Since then people have changed things. Not carelessly — a vendor recommended a setting, a fault needed chasing, a machine was replaced with a slightly different one. Some of those changes were written down. Some were meant to be written down later. Nobody set out to let the drawing rot, and yet almost every plant I have looked at cannot answer the simple question: does the plant still match it?

There is a family of tools now arriving in industrial work that claims to answer this automatically. You describe the plant in a file, and the tool compares that description against the real thing and tells you where they differ.

It is a genuinely good idea and it does work. I built a small estate to measure it. What I found is that "it works" hides three quite different cases, and only one of them is the case people are imagining.

Three changes, one afternoon#

I built three gateways from a written description, which took about half a minute, and then went and changed things by hand the way a shift actually would.

The first change: somebody gave a machine more memory. Perhaps it was struggling. Noticed in under a second, precisely, with the old and new values side by side. This is the case everyone pictures when they hear about this kind of tool, and it is exactly as good as promised.

The second change: somebody made a gateway read the plant four times as often. Chasing an intermittent fault at three in the morning — an entirely reasonable thing to do. The tool reported no changes at all.

The third change: somebody added a small extra settings file next to the main one, holding a local override. Nothing noticed. Nothing will ever notice.

Three changes. One caught immediately, one invisible to the tool that was supposed to be watching, one invisible to everything for ever.

Why the second one was missed#

Not a bug, and not a bad tool. The tool had been told to look after the machines — how much memory, what they are called, which network they are on. Nobody had told it to look after what the gateway does, which lived in a settings file inside.

Once I put that file on the list as well, the same change was caught in under a second, with the exact line that differed printed out.

So the boundary is not really about the tool at all. It is about which things somebody bothered to write down. Change something on the list, and you get a fast, precise answer. Change something else, and you get a confident silence.

That silence is the dangerous part, because it does not look like ignorance. It looks like a clean bill of health. The tool says everything matches, and it is telling the truth about the list it was given.

The third one has no technical fix#

The extra settings file is the case worth sitting with, because it is the one people skip.

A tool can only compare against a list. A new file that nobody declared is not a difference from the list — it is simply not on it. There is no cleverer product that solves this. It is not a technology gap.

It is the same failure as an undocumented modification to a machine: the drawing does not show it, and the drawing is not wrong, because the drawing never knew. The only fix is the boring one — a habit of writing things down, and periodically looking at the real thing rather than the description of it.

Now the part that would have caused an incident#

These tools usually have a second mode. Having found a difference, they can put it back — restore the machine to what the file says. In an office environment that is the entire point: things drift, the system heals them, nobody thinks about it.

I switched it on.

In four seconds it undid the night shift's work. The memory went back down. A label someone had attached to a machine reading do not reboot was deleted, because it was not in the file.

And here is the detail that turns this from a preference into an argument. It reverted the two changes that were harmless to leave alone — a memory bump and a sticky note. It left untouched the one change that was actually altering how the plant behaved, because that one was never on its list.

An automatic fixer that undoes your careful decisions and preserves your dangerous ones is worse than no fixer at all. In an office, the cost of being wrong is a service restarting. In a plant, a label saying do not reboot is a person telling you something, and deleting it silently at three in the morning is how you end up writing an incident report.

What I would do#

Keep the checking. Turn off the fixing. The value is almost entirely in being told that something changed. Deciding what to do about it is a human judgement with context the tool does not have.

Treat a difference as a question, not a fault. Somebody probably changed it for a reason. The useful next step is finding out what the reason was — which is a conversation, and often reveals that the drawing should change rather than the plant.

Ask what is actually on the list. Not whether you have such a tool — whether the things that change plant behaviour are among the things it watches. Settings usually are not.

Then go and look at something that is not on the list, and see whether it matches what you expected. That is the only way to find the third kind of change, and it is a walk round rather than a purchase.

None of this is an argument against these tools. I would use one. It is an argument against reading "everything matches" as "everything is fine", when what it means is "everything I was told to look at matches".

The estate, the timings, the exact commands and the full matrix of what each layer caught are in the lab notes.

A federal advisory later made the same point with worse consequences attached: actors edited reusable logic blocks inside controllers, and nothing in a normal plant is set up to notice a change to something nobody diffs — they didn't turn anything off.