What a deadband actually costs

The question: what does a historian deadband actually buy, and what does it cost?

Both halves are measurable against stored data, so I measured them rather than reasoning about them.

The rig#

What this data is, honestly. These are continuously-moving generated signals — ramps and sine waves — not process measurements from a real plant. That limits one claim and strengthens another. It means I cannot tell you what fraction of a real historian is redundant, because real tags sit still for long periods and these never do — the trap in testing a historian with fake data, and the reason this piece makes no claim about redundancy. It does mean the compression figures here are a floor: a real plant will do better. And because the underlying shape is exactly known, the reconstruction error is measured against ground truth rather than estimated.

The bug that came first#

The first run produced a clean-looking table that was entirely meaningless, and it is worth recording because the output looked fine.

estate_tag_history is keyed on (site, tag_path, ts). I grouped by tag_path alone. Three sites publishing the same tag name therefore interleaved into one series, so the simulation compared each reading against a reading from a different site.

Every consecutive delta was inflated, so retention looked far higher and savings far lower than the truth. At 1% the broken run reported a 22.1% mean saving; the correct figure is 41.1%. It did not error, and nothing in the numbers looked impossible.

What caught it was drawing the waveform. A sine plotted from the query zigzagged between 31 and 305 — which no sine does. The table was plausible and the picture was not, which is a good argument for always drawing the thing.

Retention against error#

Errors are expressed as a percentage of each series' span.

Two things stand out. Error tracks the threshold almost exactly — a 1% deadband produces a worst-case error a shade under 1%, which is the guarantee the mechanism is supposed to give, and it holds. Saving does not track the threshold at all. At every setting below 5%, at least one series saves nothing.

The spread is the finding#

At a 1% threshold, the eighteen series fall into three groups with nothing in between:

Ramp/Ramp2 has a span of 1.0 and Ramp/Ramp0 a span of 10.0; both traverse their range fast enough that every consecutive sample already exceeds 1% of it. The deadband is not compressing them slightly — it is doing nothing at all, while consuming a config field that suggests otherwise.

Raise those two to 5% and they drop 65.9% and 79.9% respectively. The threshold that is correct for them is nowhere near the one that is correct for Sine/Sine2.

What the reconstruction looks like#

140 consecutive readings from Rotterdam / Sine/Sine0, span 98.99, at a 5% threshold: 19 readings retained. The stored series is a staircase whose corners are correct and whose middles are invented by whatever draws the chart later.

Worth being precise about what is lost: not accuracy at the stored points, which is exact, but everything between them. A four-second excursion occurring between two retained samples leaves no trace whatsoever, and nothing downstream can distinguish "flat" from "not recorded".

Verdict#

The error guarantee holds and is predictable. The saving does not, and is not.

That asymmetry is the practical result: a deadband is safe to set and impossible to justify globally. You can promise a bound on error from the threshold alone, but you cannot promise a byte of saving without measuring the signal — and a third of the signals here return exactly nothing for it.

Set it per tag, against the smallest movement anybody will ever need to see. It is the one setting whose mistakes cannot be corrected afterwards.

The argument for a general audience is in your historian is throwing readings away on purpose.