A restore is not a rehearsal

Every audit asks whether you have backups. A better audit asks when you last restored one. The standard advice follows immediately: don't just take the backup, restore it somewhere and see if it comes back.

That advice is right, and it is missing a sentence.

I restored a working gateway's backup this week to check the file was good. It came back in twenty-two seconds, complete. Then, without being asked, it opened a connection to the live process historian and began writing rows into it.

Nothing was misconfigured. Nobody made a mistake. The copy behaved exactly as designed.

What is actually in the file#

The backup was twelve megabytes and took a little over two seconds to produce. Inside were 2,711 items: the gateway's settings, its screens, its shared code.

1,008settings

941screens

752shared code

10keys, misc

0process history

not compressed, not sampled — absent

What is not in there is any process history. Not a compressed version, not a sample — none of it. That is not a defect. The history lives in a database on another machine, and the backup holds the address and the password for that database rather than its contents.

Worth saying plainly, because I have heard people assume otherwise: if the historian burns down, the gateway backup does not help you. Those are two recovery plans, and most plants have written one.

The restore itself was excellent#

I should be fair to the thing before I criticise it.

Twenty-two seconds from a cold start to a running gateway. One second of that was the restore; the rest was the software booting. Afterwards I compared the restored machine against the original file by file — 1,451 files against 1,457 — and the only thing missing was a leftover log from an old upgrade. Every actual setting came back: database connections, security, screens, tag structure, all of it.

clean machine

launch

1 second

settings restored

22 seconds

gateway running

270 rows

writing to production

If your question is "will the file come back", the answer here was yes, completely, in well under a minute. That is a real and good result and I would rather have it than not.

Then it started writing#

Ninety seconds after the copy was running, I counted the rows arriving in the production historian. Two hundred and seventy of them. I stopped the copy and counted again over another ninety seconds: none.

Every row had come from the copy. Not from the original — the original wasn't writing at all. The machine I had built purely to inspect a file had become the only thing feeding the plant's history database.

Nobody typed a password to make that happen. The password was in the backup, encrypted, and the restore put it back exactly where it belonged. That is the whole point of a backup. It is also the reason the copy was not a rehearsal.

What else came back with it#

The database was not the only thing the file remembered.

Gateways in an estate have to trust each other, and that trust is normally granted by a person: a new gateway shows up, somebody looks at it, somebody approves it. My restored copy arrived with both of its central gateways already marked approved.

Of course it did. Approval is a setting, and settings are what a backup restores. But follow it through. Anyone holding that file — a contractor, a laptop in a van, an old drive in a drawer — holds a machine that your estate will recognise and your database will accept. If you have already counted the doors your air gap actually has, add this one: a backup on a vendor's laptop is a key, not a copy. The file is not a photograph of your gateway. It is a kit for building another one.

How to test a restore without joining it to the plant#

None of this argues against testing restores. It argues for testing them somewhere they cannot reach anything.

Restore onto an island. A network with no route to the historian, no route to the other gateways. Not a firewall rule you trust — a network that has nowhere to go.

Assume it will start talking the moment it starts. There is no inspect-only mode. If it can reach production, it will use production, and it will be right to.

Do not assume your command line wins. I told the restore to use different network ports, and it used the ports from the backup instead, which promptly collided with the original. The backup's settings beat your arguments. That is consistent behaviour and it caught me out.

Rotate what the file carries. A backup that has been off site is a credential that has been off site. If your restore test proved the database password still works, that is worth knowing in both directions.

Test the two recoveries separately. "The restore worked" is a statement about configuration. Your process history has a different owner, a different mechanism and a different failure, and it needs its own rehearsal.

The reason people quietly avoid restore tests is not laziness. Somewhere in the back of the mind sits the knowledge that a restored gateway is not an inert copy — it is a second real one. That instinct is correct. The answer is not to skip the test. It is to do the test where a second real gateway cannot do any harm.

I ran the measurements — the timings, the file-by-file comparison, the row counts, and the two ways the restore refused to start — in what a gateway backup actually restores.

This was one vendor's gateway. Yours will differ in the details. I would be surprised if it differs in kind.

Restoring the gateway is only the first stage. Timing the whole thing to the point where the plant is actually recording again — and measuring the hole the outage leaves in the record — is in back online is not the same as recording.