Simulator-First Relay Testing: Rehearse the Plan, Then Prove the Installation

A protection test that fails for the wrong reason is expensive twice: once for the time it wastes, and again for the doubt it leaves behind. Most of those wrong reasons — a mistyped CT ratio, a compensation convention applied on both sides, an expected operating time computed from the wrong curve family — are discoverable without energising anything.

That is the case for simulator-first relay testing: get the plan right where mistakes are cheap, then use the hardware test for what only hardware can settle. This article makes that argument, and then makes the counter-argument, because the failure mode of simulator work is believing it proved more than it did.

Key takeaways

• A simulator-first workflow means the whole test program — settings, expected values, tolerances, sequence, report — is built and run in software before any instrument is connected.
• It genuinely de-risks the desk-error class: wrong ratios, wrong constants, wrong conventions, wrong tolerances, wrong plan order, and reports that are missing fields nobody noticed.
• It proves nothing physical. Wiring, CT and VT circuits and their burden, transient CT saturation, real firmware behaviour, panel and scheme logic, and the installation itself are all outside its reach.
• The two are complementary, not competing: the simulator is where the plan becomes correct, the hardware test is where the installation becomes proven.
• The idea is not novel — incumbent vendors offer office and training modes. What differs is treating the simulated loop as a first-class capability rather than a degraded mode.
• In published ProtectionAI v1.3, the workflow runs against the built-in simulator and vendor hardware drivers are stubs. The v1.4 source candidate adds bounded, not-yet-HIL-qualified OMICRON phasor/binary/auxiliary I/O, manual release, software ramps and buffered events; it remains an evaluation path rather than a replacement for qualified vendor software.

What a simulator-first workflow actually is

The word “simulator” does a lot of work in this industry, so be specific. What is being simulated here is the test loop: a virtual test set that produces the analogue quantities a module would inject, paired with a model of a relay that measures them, applies a settings model and asserts contacts with realistic timing.

Run a pickup ramp against that loop and you get a measured pickup value, a reset ratio, a plot, a tolerance verdict and a report row — the same artefacts a real test produces, from the same code path, with the instrument replaced by a model.

The workflow that follows is not “test in software instead of hardware”. It is:

1. Import the settings — an XRIO or RIO export, or a manual entry into the normalized model.
2. Build the plan: which elements, which multiples, which sequence states, which tolerances.
3. Compute the expected values and check them against the standard’s equations rather than against the last job’s spreadsheet.
4. Run it against the simulator. Read what comes out. Fix the plan.
5. Repeat until the plan runs clean and the report contains what the reviewer needs.
6. Take the plan to site and run it against the relay.

Step 5 is the point. By the time anyone drives to the substation, the plan is not a hypothesis.

What it genuinely de-risks

Data errors, found at a desk. A CT ratio entered as 1200/1 instead of 1200/5 will produce a coherent, plausible, entirely wrong set of expected values. Against a simulator you find that in the first minute, because the numbers are absurd relative to the settings. In an energised bay, you find it after an hour of arguing with a relay that is behaving correctly.

Convention errors. Transformer differential testing is full of them: whether the relay expects compensated or primary-equivalent quantities, which restraint definition the manufacturer uses, whether tap correction is already applied. Getting these wrong is the classic way to “prove” a healthy relay is faulty. A simulator run with a known-good model makes the mismatch obvious, because you can see both what you injected and what the model computed.

Template and plan development. Building a reusable template is iterative work — you run it, you find the sequence state that never transitions, you fix the binary mapping, you run it again. Nobody wants to do that inside a four-hour outage. It is comfortable work at a desk and miserable work in a bay.

Training, without an instrument and without an outage. A new technician can learn what a mho zone boundary search does, what a reset ratio means, why a timing point near pickup is unrepeatable, and what a bad result looks like — repeatedly, at no risk, without occupying a test set that is booked for real work. They can also make mistakes, which is how the learning happens, in a place where mistakes are free.

Plan review before the window. A senior engineer can be handed a plan that has demonstrably executed, with a sample report, and asked whether the coverage is right. That is a much better review conversation than reading a table of intended test points.

Report format, settled early. Discovering that the report is missing the as-found values, or the tolerance basis, or a signature block, is a five-minute fix beforehand and a re-visit afterwards.

What it cannot prove

This is where simulator advocacy usually goes quiet, so let us be blunt. A simulator run is evidence about a plan. It is not evidence about an installation. Specifically, it says nothing about:

Wiring. Terminal connections, ferrule identity, cross-phasing, a lead in the wrong socket, a shorting link left in, a landed conductor that is not actually landed. None of this exists in software.

CT and VT circuits. Ratio and polarity as installed, circuit continuity, secondary burden, spare-core loading, earthing. IEEE C37.110 exists because these are engineering questions with real answers, and none of those answers is in your settings file.

CT saturation. This is worth calling out separately because it is the limitation people forget. In a simulated loop, the current transformer is not in the loop — the test set model produces a current and the relay model measures it. Transient saturation, remanence, the effect of burden and X/R on the reproduced waveform: none of it is exercised. If you want to understand why that matters and what does exercise it, CT saturation and relay testing covers the ground.

The real relay. A simulated relay implements a model of a characteristic. The physical relay implements a specific firmware version’s measuring algorithm, with its own filtering, its own sampling, its own reset behaviour, its own quirks, and occasionally its own bugs. The simulator tells you what a correct relay should do. Only the relay tells you what your relay does.

Panel and scheme logic. Interposing relays, lockouts, trip circuit supervision, blocking and intertripping schemes, breaker fail logic, the actual trip circuit and the actual breaker. A test that ends at a relay contact has not proven a trip.

Everything physical about the installation. Auxiliary supplies, communications paths, labelling, and the plant’s real state on the day.

There is a rule of thumb worth carrying: if a question could be answered by reading files, a simulator can help; if it could only be answered by touching plant, it cannot.

How the two fit together

Put like that, the division is clean.

Question	Answered where
Are the expected values right?	Simulator
Does the plan execute end to end?	Simulator
Is the tolerance basis correct and documented?	Simulator
Does the report contain what a reviewer needs?	Simulator
Does the technician know what they are doing?	Simulator
Is the relay’s measuring element accurate?	Hardware
Is the CT circuit right, and does it hold under fault?	Hardware
Is the wiring right?	Hardware
Does the scheme trip the breaker?	Hardware
Is the installation acceptable?	Hardware

The simulator is where you get the plan right. The hardware test is where you prove the installation. Neither substitutes for the other, and a test program that skips the first spends the second discovering things it should already have known.

Is this a new idea? No.

It should be said clearly, because the industry has heard enough novelty claims. The established test-set vendors have offered instrument-free modes for a long time — office editions, training modes, software that opens modules and builds plans with no hardware attached. Engineers have been using them for exactly this purpose for years, and there is a body of published test-strategy work, including CIGRE’s material on testing in digital substations, that treats pre-hardware verification as ordinary good practice rather than an innovation.

The difference we would actually defend is one of product posture. In an office mode, the simulated path is a degraded version of the real one: the modules open, but there is no relay to respond, so you cannot rehearse a result — only a layout. Treating the loop as first-class means building the other half: a configurable simulated relay with real elements and realistic contact timing, so a plan can be executed and produce results, not merely be authored. That is a design decision about what the product is for, not a claim to have invented rehearsal.

Where ProtectionAI honestly sits

ProtectionAI’s entire test workflow runs against its built-in simulator: a virtual test set, a configurable simulated relay and a network model. 68 test modules, IEC 60255-151 and IEEE C37.112 curve mathematics, pickup and dropout ramps, overcurrent characteristic sweeps, distance zone boundary searches, differential operate-point searches, the multi-state sequencer, COMTRADE read and playback, the settings model, the asset database and the branded report export — all of it works, all of it against the simulator.

Release boundary: published v1.3 has five documented vendor stubs. The v1.4 source candidate adds bounded OMICRON CM Engine discovery/lock, phasors, binary I/O, auxiliary DC, manual release, software ramps and buffered binary events, pending physical qualification; Doble, Megger, ISA/Altanova and EuroSMC remain explicit stubs.

So the honest positioning is this: today ProtectionAI is a rehearsal, planning, training and documentation tool. It is where you build the plan, check the arithmetic, train the technician and settle the report. It is not a replacement for your test set’s own software, and if you install it expecting to inject current you have misunderstood what you downloaded. When the drivers ship, the same plans will run against instruments — the driver abstraction, channel model and sequence contracts were built for that and are exercised by the simulator every day — but “when the drivers ship” is a future tense and we will keep writing it that way.

If you want the mechanics, how ProtectionAI works walks the workflow through, and the survey of relay testing software puts it next to the alternatives without pretending the comparison is flattering in every column.