Twenty-one hours of plant traffic, and the segmentation policy it implies

This is the observation half of a segmentation exercise on a working estate: three SCADA sites, a redundant central pair, a broker cluster, an open-source collect-only site, and a historian with a streaming replica. No zones existed when this started — the estate ran flat, one subnet, every host reachable from every other. That is the honest starting condition and it is the one most plants are in.

Nothing here is enforced yet. Enforcement is the sequel; this is the part that decides what the rules should say.

Why measure first#

The alternative is drawing zones from a reference model and discovering the traffic you did not know about when the rules start dropping it. Every segmentation project I have watched fail, failed there.

So the rule for this run was: the policy is derived from the capture, and the reference model is a thing to compare against afterwards — never an input.

The rig#

Deliberately not used: conntrack. Bridged traffic bypasses netfilter unless bridge-nf-call-iptables is on, and turning that on changes packet handling on a running plant. The measurement must not perturb the thing it measures.

Known blind spot#

Traffic inside a host — containers on the same machine talking over a local bridge — never reaches the capture point. On this estate that hides the broker-to-database and connector-to-database paths, which conceptually cross a zone boundary. The derived policy therefore covers inter-host conduits only, and a second capture inside those hosts is required before enforcement. Stating it here because it would otherwise be discovered after the rules were written.

The window dominates the answer#

A three-minute survey — which is roughly what a person does when they say they have "looked at the traffic" — sees under a fifth of the internal flows. The missing ones are periodic: the hourly poll, the nightly job, the certificate check. Those are the flows a new rule blocks silently, and they surface days later as an outage nobody connects to the firewall change.

Four defects in the instrument#

Recorded in full because they cost more time than the analysis, and because two of them produced confident, plausible, wrong answers.

1. The parser invented flows. A non-greedy host pattern matched portless lines (IGMP, ICMP) by eating the final octet of the address as a port number, producing entries like 192.168.29 → 239.255.255 tcp/250. Nine fabricated flows in the first pass.

2. The fix did not deploy, and the verification passed anyway. The corrected file failed to transfer. My check confirmed the file parsed — the old one parses fine — and separately tested the corrected patterns inline, not as deployed. Two green checks, neither touching the artefact that was actually running. Caught only because malformed rows persisted in the output. Now verified by checksum against the local copy, and by a field only the new version emits.

3. The schema made the question unanswerable. The aggregator keyed flows by (protocol, source, destination, destination-port) and discarded the source port. Without it, direction is not recoverable, and every heuristic failed differently on the same data:

The generalisable lesson is not about ports. The schema of a measurement decides which questions can be answered afterwards, and you find out which ones you foreclosed at analysis time, when re-running is expensive.

4. I misread my own process count. pgrep -c reported four captures running; it was one pipeline of four processes. Thirty seconds of alarm, listed here because miscounting the rig is the same class of error as miscounting the plant.

The derived policy#

Twenty-seven inter-host conduits, of which twenty are identified services and seven remain unresolved at low packet counts and are marked as such rather than asserted.

The largest, by volume:

Two of those deserve comment. Site → Platform tcp/5544 is an edge historian writing store-and-forward data straight into the central database — a single hop across two levels, and the only path that survives a site link cut. And the udp/4446 multicast is gateway discovery from all five gateways: no unicast destination, no port a policy author would think to allow, and invisible to any rule expressed as source-destination-port.

Against the reference model#

Writing out what a Purdue-style model would prescribe for this same estate and differencing the two sets:

• 1 conduit the model allows that the plant never uses
• 22 conduits the plant requires that the model forbids

The single unused one is Site → Ops tcp/8060 — sites reporting upward. The estate's actual largest flow is the exact reverse, because Gateway Network connections must be initiated by the redundant pair or the backup holds no relationship to any site and the data path does not survive a failover. That was measured separately: eight seconds to move the screens, six and a half minutes with no data, until the direction was reversed.

So the model's one confident instruction, applied here, would reintroduce a known-expensive failure — and it would look correct on the diagram.

The host in two zones#

One machine carries the historian, the broker, the HMI proxy and the operations dashboard, and simultaneously serves an unrelated application to a different user population (27,568 packets inbound on its service port during the window) while accepting unsolicited inbound traffic from the public internet.

By function it belongs in the protected core. By exposure it belongs outside. There is no rule that resolves this; only splitting the host does.

Meanwhile the SCADA gateways themselves are clean: Site and Ops zones together reach exactly one external port across the whole window. All external sprawl — 80 distinct destination ports — belongs to that one platform host.

What transfers#

• Measure for a day, not an hour, and count distinct flows rather than bytes. The dangerous ones are small.
• Derive the policy from the capture; use the reference model as a differencing tool afterwards, and treat each disagreement as a question.
• Check every conduit's direction against how the system fails over, not how it runs. Direction can be a redundancy requirement wearing security clothing.
• Capture where the traffic actually is. Intra-host traffic needs its own capture, and a policy written without it is incomplete.
• Record the schema decisions of your instrument. They silently determine which questions you can still answer at analysis time.

The practitioner argument built on these numbers is in the companion post. The failover result that explains the arrow direction is in the estate lab.