Relay Testing Software: What It Does, What It Couples You To, and How to Evaluate It

Ask three protection engineers what relay testing software is and you get three answers: the thing that drives the amplifiers, the thing that produces the report, or the thing that gets in the way of finishing before the outage window closes. All three are true. This article describes the category honestly — what the software layer does, what buying it commits you to, which tests prove which things, and what to ask a vendor whose demo has just gone suspiciously smoothly.

Key takeaways

• The test set is amplifiers, binary I/O, timing hardware and network interfaces. The software plans, executes, assesses and documents. Two jobs, almost always sold as one product.
• Every category of test proves something specific and misses something specific. A test plan is a deliberate combination, not a maximum.
• The tolerance basis — percent of expected, absolute milliseconds, or the greater of the two — determines the verdict as much as the measurement does.
• Re-typing settings is the dominant error source; settings-driven plans are the highest-value feature in the category.
• The deliverable is the retrievable record, not the injection.
• The established platforms are mature, hardware-anchored ecosystems with very large model-specific template libraries. That maturity is a real advantage.
• Hardware-neutral, simulation-first tools address a real gap — plan validation, settings interpretation, training — but be precise about what they can drive.

The test set and the software are not the same product

The hardware does a small number of things extremely well: current and voltage amplifiers with defined accuracy, compliance voltage and power output; binary inputs that time a trip contact to microsecond resolution; binary outputs to simulate breaker auxiliary contacts; DC supply; and, increasingly, an Ethernet interface for IEC 61850 GOOSE and Sampled Values. Its specifications are the physical envelope of what can be tested — if the relay’s input burden demands more compliance voltage than the amplifier can deliver at 40x pickup, no software feature will get you that point.

The software does everything else: it holds the test plan, computes expected values from settings, sequences amplifier states, waits on binary transitions, measures and timestamps, compares measurement to expectation against a tolerance, issues a verdict, and manages the as-found/as-left pair and the report. It also holds the template library — in the mature platforms, the part representing the most accumulated work.

The two are bundled: every major platform is built by a test-set manufacturer and its full capability is realised on that manufacturer’s hardware. That couples your test-set fleet to your software fleet, anchors your template library to one vendor’s format — the real switching cost, not the licence — and anchors your technicians’ speed too. It explains why “which software is best” is usually the wrong question and “what am I willing to be locked into, and for how long” is the right one.

The categories of test, and what each one actually proves

Steady-state and manual injection

Set the amplifiers to a state and observe. Proves the relay is alive, wired as expected, and measuring the quantities you think it is. Misses everything dynamic. Its value is diagnostic — it is where you go when an automated test is failing and you do not yet know why. Do not confuse it with a test of the protection function.

Pickup and dropout ramps, and the reset ratio

Ramp a quantity until the element asserts, then back down until it releases: pickup, dropout, and the ratio between them — threshold and hysteresis. Misses timing entirely. A ramp that is too fast reads high, because the relay’s measurement window and filtering lag the input, so the ramp rate is part of the test definition and belongs in the record. Two engineers using different ramp rates will legitimately get different pickup values on the same relay.

Timing against a curve

Inject a defined multiple of pickup as a step and measure to trip. Proves the operating time at one point against a computed expectation, and misses the rest of the curve — which is the whole point of an inverse characteristic. Because the IEC and IEEE families are distinguished mainly by their exponents, a single mid-range point cannot even establish which family the relay is applying; the arithmetic is in IEC 60255-151 inverse-time curve constants and IEEE C37.112 curves explained.

Characteristic sweeps

The software walks a grid or set of search rays across a plane and maps where the element operates: current versus time on log-log axes for overcurrent, the R–X plane for distance zones, the bias/restraint plane for differential. Proves the shape of the characteristic, which is what the setting actually is. Misses dynamic behaviour, and can flatter a relay — a steady-state distance sweep does not exercise memory polarisation, load encroachment, or the directional decision under a realistic transient (distance relay reach testing covers where steady-state checks stop being sufficient). Sweeps also expose systematic plan error: a wrong exponent tilts every point the same way, a wrong time multiplier offsets them uniformly, a minimum-operate-time limit flattens the fast end. Random scatter looks different from all three.

Multi-state dynamic sequences

A scripted series of states with defined durations and transition triggers: prefault, fault, breaker open, dead time, reclose onto fault, lockout. Proves sequence logic — reclose schemes, breaker-failure timing, reset behaviour, logic that depends on order rather than magnitude. Misses waveform realism: the states are synthesised from calculated phasors, so DC offset, CT transient response and travelling-wave content are absent or idealised. This is the category most often skipped, and reset behaviour is the casualty — an element with slow inverse reset carries accumulated timing between reclose shots and operates faster the second time, one with instantaneous reset does not, and no single timing shot sees the difference.

Transient playback (COMTRADE)

Inject a recorded or simulated waveform — a disturbance record from a relay or DFR, or the output of an EMT simulation — via the COMTRADE format. Proves the relay behaves correctly on the actual waveform, including DC offset, CT saturation as recorded, harmonics and the exact fault-inception angle. It is the only way to close the loop on a misoperation: replay the event and watch the relay do it again. What it misses is generality — a playback tests one waveform, not the family of faults it came from. It also depends utterly on the record’s fidelity: wrong scaling factors or a truncated prefault period produce confident nonsense. See CT saturation and relay testing.

System-based and simulation-driven testing

Instead of specifying quantities, specify a network — sources, lines, impedances, fault location and type — and let the software compute what to inject, iterating in closed loop with the relay’s trip and breaker signals. Proves the protection scheme in its system context: that Zone 1 reaches where the study says, that teleprotection logic behaves for a fault at 85 percent of the line, that relays at both line ends interact correctly. The most powerful category and the most demanding. It is far less sensitive to a settings transcription error, because the stimulus comes from the model rather than a pre-calculated phasor. Its limits: the model is now part of the test, so a wrong source impedance produces a valid test of the wrong system.

Assessment, tolerances and the as-found/as-left pair

The measurement is not the result. The result is measurement compared to expectation against a stated tolerance, and there is more judgement in that sentence than most reports admit.

Where does the expected value come from? From whichever copy of the settings the plan was built against. If that copy was typed, the expectation is only as good as the typing.

What is the tolerance basis? Three conventions are common: a percentage of expected value; an absolute figure in milliseconds or amperes; or the greater of the two, which is what most frameworks effectively require, because a pure percentage is meaninglessly tight at short operating times and a pure absolute meaninglessly loose at long ones. IEC 60255-151 frames operating-time accuracy in these terms. A verdict is uninterpretable without knowing which convention produced it, so the basis belongs next to the number.

Whose tolerance is it? The standard’s accuracy class, the manufacturer’s declared accuracy and your utility’s maintenance tolerance are three different numbers, not always in the order you expect — a relay can sit inside the manufacturer’s specification and outside a utility standard written when electromechanical relays set the expectation. Which governs is an engineering decision, made once at the standards level rather than on site.

As-found and as-left. Test the relay as you found it before touching anything, then again after any adjustment, and record both. As-found tells you whether it has drifted, been mis-set or been quietly modified; as-left tells you what you are leaving in service. A report with only as-left values has discarded the only evidence about what the previous interval did.

Templates, and why re-typing settings is the real problem

The largest single source of error in automated relay testing is not amplifier accuracy or timing resolution. It is a human transcribing a settings sheet into a test plan.

The failure mode is silent and confident. A mistyped time multiplier raises no error. It makes the software compute wrong expected values, compare good measurements against them, and issue a clean, precise, authoritative FAIL on every point at once — or, worse, a clean PASS against a target wrong in the same direction as a real relay problem. Automation applies transcription errors consistently rather than catching them.

Settings-driven testing is the answer. XRIO and its predecessor RIO carry a relay’s parameters into a test plan so expected values are derived rather than re-entered, and a settings-linked plan can be re-run against a changed setting without being rebuilt: change the pickup in the imported set and every expected value, sweep grid and tolerance band recomputes. That is the difference between a test plan and a test template; XRIO templates explained covers the mechanics.

Mature template libraries matter for the same reason. A pre-built per-model plan means the tolerance conventions, element mappings and binary assignments were worked out once by someone with time to do it properly, rather than at 2 a.m. by whoever drew the outage.

The record is the deliverable

The injection takes minutes. The record has to survive years.

Under NERC PRC-005, and equivalent obligations elsewhere, what is audited is not whether you tested well. It is whether you can demonstrate that you tested, on the required interval, on the required components, to a defined maintenance activity, with results. An excellent test whose record cannot be retrieved is, for compliance purposes, close to no test at all.

A record that holds up needs: unambiguous device identity including model, firmware and settings identifier or checksum; test date, technician, and test-set serial with calibration status; the settings tested against and where they came from; per-test expected value, measured value, tolerance basis and verdict; as-found and as-left distinguished; deviations and untested elements with reasons; and a path from the summary back to the raw measurement rather than to a rendered PDF.

That last item is where good practice often fails. A directory of PDFs on a shared drive is a document store, not a record system — it cannot answer “show me every 51 element in this substation whose as-found timing drifted more than 5 percent since the previous interval,” which is the question that turns test data into asset knowledge. See PRC-005 evidence checklist.

The landscape, described plainly

These are mature, hardware-anchored ecosystems, each with decades of test-module development and large model-specific template libraries behind it. That maturity is their principal strength and should be stated as such.

OMICRON. Test Universe is the long-established modular suite for conventional secondary testing — a module per test type, the Control Center for assembling them into documents, XRIO for settings-driven parameterisation. RelaySimTest is the separate system-based product: build a network model and test the scheme rather than the element. The Protection Testing Library supplies predefined test plans and nominal characteristics for several hundred relay types.

Doble. Protection Suite is Doble’s relay testing software, and RTS is worth calling out specifically: it is a vendor-neutral driver layer, controlling test sets from several manufacturers — Doble, ISA, Megger, OMICRON and SMC hardware among them — from one package. For a utility with a mixed fleet accumulated over decades that is the main counterexample to the coupling described earlier. PowerBase is the protection asset and test database behind it. Doble also carries the ISA/Altanova TDMS line, developed for ISA’s DRTS test sets.

Megger. RTMS is the relay test and management software for the SMRT series, running on the instrument’s Smart Touch View Interface as well as from a PC — which suits engineers who prefer to work at the test set. PowerDB is the broader acceptance-and-maintenance data layer with compliance scheduling; if you already use it for transformer and breaker testing, relay results in the same database is a real advantage.

EuroSMC. ROOTS is EuroSMC’s object-oriented test software for computer-driven operation of the Mentor and TRES test sets, performing fault calculation, sequential execution and automatic reporting.

If you own one of these and its template library covers your fleet, the honest advice is usually to get more out of it rather than replace it.

On AI, as a dated research finding. In our own review of these five vendors’ publicly documented protection-testing portfolios, conducted in July 2026, we did not find an AI-branded capability. That is our reading of public material at a point in time — not a claim about what any vendor is building, has under NDA, or has shipped since.

Where a hardware-neutral, simulation-first approach fits

There is a real gap, and it is narrower than a sales deck would like it to be.

Much of the work described above — interpreting a settings sheet, deriving expected values, building a plan, validating its arithmetic, checking a coordination assumption, training an engineer on curve behaviour — needs no amplifier. It needs a correct model of the relay and of the test. Today it happens either inside a hardware-coupled package, meaning at the instrument or on a licensed seat, or in a spreadsheet, meaning unversioned and unreviewable.

GridAPM’s ProtectionAI is built for that part. Its limits, exactly, because vague claims here are worse than no claims:

• It runs the full test workflow — plan, execute, assess, document — against a high-fidelity built-in simulator.
• Published v1.3 is simulator-only. The v1.4 source candidate has a bounded OMICRON phasor/binary/auxiliary I/O, manual-release, software-ramp and buffered-event path, not physical-CMC qualified; the other vendor drivers remain stubs. Qualified relay injection still requires the appropriate instrument and approved procedure.
• Published v1.3 IEC 61850 GOOSE and Sampled Values are loopback-only. The v1.4 source candidate adds explicit SharpPcap/Npcap physical-NIC transport, but it is not a substitute for qualified on-network GOOSE testing; see IEC 61850 GOOSE testing explained.
• It is not certified against any compliance standard. NERC compliance is a property of your programme and evidence, not of a tool.

What that leaves is useful and bounded: building and validating test plans before an outage, checking a settings interpretation against both curve standards, exercising scheme logic against a simulated network, producing a queryable record, and training engineers without occupying an instrument or a spare relay. The capabilities page sets out what is implemented. Where this article says a test needs an amplifier, it needs an amplifier.

How to evaluate any vendor, including us

Take these to a demo. All are answerable in a sentence, and the ones that instead produce a paragraph of context are the informative ones.

1. Which of the test sets I already own does this drive, at full capability, today? Not “supports” — drives. Ask by model and firmware, and ask what is reduced on non-native hardware.
2. How is a tolerance defined, and does the basis appear in the report? Percentage, absolute, or greater-of? Settable per test type?
3. Can I import my actual settings, and from what? When a setting changes, does the plan recompute or does someone rebuild it?
4. What does the record look like to an auditor two years from now? Show me a stored result, not a freshly generated PDF. Can I query across it?
5. Does it run without the instrument? Can a plan be authored and reviewed offline, and can a new engineer train against a simulator?
6. What does it refuse to do? A vendor who cannot name a limitation has either not been asked or is not answering.

Related reading: IEC 60255-151 inverse-time curve constants, IEEE C37.112 curves explained, how to test a transformer differential relay, distance relay reach testing, CT saturation and relay testing, XRIO templates explained and PRC-005 evidence checklist.