Two settings decide what your plant remembers

Two numbers in your recording system decide everything you will ever be able to say about your plant's past.

The first is how often you read each value. The second is how much that value has to change before you bother writing it down.

Both usually get set once, during installation, by somebody sizing a disk. Both are then treated as the same kind of knob — turn either one down, store less data. That is the mistake, and it is an expensive one, because the two settings fail in completely different ways.

They are not the same knob#

Reading less often means there are things you will never know happened. If you read a value every ten seconds, a four-second event did not occur as far as your plant is concerned. No later analysis recovers it, because nothing ever saw it.

Writing down less often is different. You still read the value at full speed. You still see everything. You are only choosing which of the things you saw are worth keeping.

That difference sounds academic. It is the whole thing.

Measuring them against each other#

I took eighteen signals from a running system and reduced them two ways.

First by reading less often — keeping every second reading, then every fourth, eighth, sixteenth. Then by the write-it-down threshold, at four settings. For each one I measured how much data survived, and how wrong the stored version ever got.

On average, they look similar. Halving the reading rate leaves you typically wrong by about 0.6% of a signal's range. A threshold that stores roughly the same amount is also wrong by about 0.6%.

If you stop at the average, the two settings look interchangeable, and this article would not exist.

The moment it matters#

Here is the same signal, at the moment something actually happens — a value running up to its limit and then dropping straight back to the bottom. A reset, a trip, a pump stopping. The kind of moment you keep records for.

The reading-less-often version holds the old value straight across the event. For one whole interval, its record says the value is still near the top when in reality it is already at the bottom. Measured across all eighteen signals, its worst error is 80% of the signal's entire range.

The write-threshold version catches the same event immediately, because a large change is exactly what it is watching for. Its worst error is 1.8%.

Same signals. Same day. Roughly the same amount of data stored.

The principle worth taking away#

The write-it-down threshold has an error you choose. Set it to 1% and you have promised yourself you will never be more than about 1% wrong. That promise held across every signal I measured. It is a genuine engineering guarantee.

Reading less often has an error nobody chooses. It is not bounded by your setting. It is bounded by how big the event was that happened while you were not looking — which is unknowable in advance and, by definition, largest exactly when something dramatic occurred.

That is why the two are not interchangeable. One of them has a worst case you can write in a specification. The other has a worst case set by the plant.

At the same storage, one simply wins#

This is the part that surprised me. I expected a trade-off and did not find one.

Match them at similar storage and the write-threshold approach stores fewer readings and is more accurate at its worst — by a factor of forty. It is not a compromise between two reasonable options. On this data, one of them is better on both axes at once.

The reason is that it is adaptive. It drops readings when nothing is happening and keeps them when something is, because that is literally what it tests for. Reading less often drops readings on a fixed rhythm that knows nothing about the plant — and one in every so many of those blind drops lands on something important.

The same shows up in whether the true highest and lowest values survive at all. Reading every eighth value, none of the eighteen signals kept its true extremes. Those are the numbers somebody eventually needs.

What this means in practice#

Read fast. Store selectively. That is the whole recommendation. Read as often as the equipment and network will comfortably allow, then use the write-threshold to control how much you keep. You get the storage saving without buying it with blindness.

Never slow the reading rate to save disk. It is the most expensive way to buy storage that exists. You are not compressing your history, you are choosing not to witness parts of it, and no future decision reverses that.

If you must slow something, slow it where nothing happens. Slow polling is defensible for a value that genuinely moves slowly — an outdoor temperature, a tank level on a weekly cycle. It is indefensible on anything that can step.

Set the threshold from the smallest change worth seeing. Not from disk size. Ask what the smallest movement is that anybody would ever need to notice, set the threshold below it, and the storage question answers itself.

Being fair about the measurement#

These signals come from a generator, not from process instruments, so they step more regularly than a real plant does. What that changes is how often you hit the bad case, not what happens when you do — holding a stale value across a step is structural, and any real trip or valve slam behaves the same way.

It also means the averages here are pessimistic for the threshold approach: real signals sit still for long stretches, where it saves far more than it did in this test.

The detailed measurement behind the threshold half of this — eighteen signals, five settings, and the mistake in my own method that reversed the first result — is in what a deadband actually costs, and the argument for treating that setting seriously is in your historian is throwing readings away on purpose.

Two numbers. One of them decides what you can ever know, and the other decides what you keep of it. Most plants have them the wrong way round.