Four paths into one historian, and what each one remembers about where the data came from

This is a provenance audit of an estate I run: three sites, a redundant central pair, a broker cluster, a licence-free edge site, and a historian with a streaming replica. It writes into one database by four different routes, which makes it a useful place to ask what each route actually remembers.

Nothing here is an attack. Every action is on my own equipment on an isolated network, and the interesting output is a comparison of schemas and configuration files.

The four paths#

The two newest tables are the ones built for this estate, and both are (ts, site, tag_path, value, …) — no quality column, and no second timestamp distinguishing when the value occurred from when it was stored. The older tag_history has both. Nobody removed them; each new path was written to move a value, and a value is what it moves.

What identity looks like on the wire#

The open path publishes JSON over MQTT. A live message, captured with an anonymous subscription from another host on the segment:

spBv1.0/Northwind/DDATA/Seville/edge1  {"site": "Seville", "tag_path": "Ramp/Ramp0",
                                        "value": 0.278272, "ts": 1787119669.6018462}

Note that the site appears twice: once as a topic level, once as a payload field. Two independent assertions of origin, in the same packet, that no component compares.

The sink decides which one becomes the record, and it does so in one line:

args_mapping: |
  root = [ this.ts, this.site, this.tag_path, this.value ]

this.site is the payload. The topic is used only to subscribe (spBv1.0/Northwind/DDATA/+/+) and is discarded on ingest.

Testing it#

Published one message to the Seville topic with a payload declaring "site": "Rotterdam", using a distinctive tag path so the row could be found and removed afterwards. No credentials were supplied; the broker accepts anonymous connections on this segment.

The stored row:

   site    |        tag_path        | value  |             ts
-----------+------------------------+--------+---------------------------
 Rotterdam | PROVENANCE_TEST/marker | 99.999 | 2026-08-19 06:08:20.969+00

The address the message was delivered on is absent from the record. The row is byte-for-byte the shape of a genuine Rotterdam reading. Test row deleted after the check.

The defensive reading of this is not "MQTT is unsafe". It is that an ingest mapping is a trust decision, usually written once by whoever built the pipeline, and rarely revisited: this pipeline chose to believe the payload, and nothing downstream can see that choice.

What identity looks like on the licensed path#

The Event Streams path does not carry a self-declared site field. Its sink derives site and tag path by regex from the message key, and takes the time from message metadata:

root = [
  metadata("kafka_timestamp_unix").number(),
  metadata("kafka_key").string().re_replace_all("^\\[([^\\]]+)\\].*$", "$1"),
  metadata("kafka_key").string().re_replace_all("^\\[[^\\]]+\\]", ""),
  ...
]

This is better — the key is set by the gateway rather than by the sampled data — but the identity is still a string parsed out of a message, and any producer permitted to write to the topic can compose that string.

A hypothesis this lab disproved#

Reading the configuration alone, I expected kafka_timestamp to mean the moment the broker received the message — which would make every row on this path a record of network timing rather than plant timing.

That was wrong, twice.

The topic is configured message.timestamp.type=CreateTime, so the timestamp is set by the producer, not the broker. And the producer sets it from the tag's own timestamp rather than from publish time, which the data confirms: intervals between consecutive rows of a 1 Hz signal measure 1000 ms with a standard deviation of 0.0 ms, min and max both exactly 1000. Publish-time stamping cannot produce that.

Stating it because the configuration looked damning and the measurement acquitted it. A config file shows intent; only the data shows behaviour.

What transfers#

• Ask where the identity in a row was last checked, not where it was set. On most paths it is set once and copied thereafter.
• An ingest mapping is a trust policy. this.site versus a broker-assigned topic is a security decision written in a data-transformation file, usually by someone solving a formatting problem.
• Two assertions of origin in one message are worse than one, because their agreement is assumed and never tested. If both exist, compare them and reject mismatches.
• Quality is provenance. A path that drops the instrument's own confidence makes a faulty reading permanently indistinguishable from a real one.
• Encryption authenticates the hop, not the reading. Both are worth having; only one of them is usually present.
• Check behaviour before publishing a claim about it. The strongest-looking finding in this lab was the one that turned out to be false.

The practitioner argument built on these results is in the companion post. The same two paths measured for time rather than identity — including a 24 ms-per-cycle drift that never self-corrects — are in the clock lab.