Cybersecurity and AI Governance for Substation Testing Workflows
A control-oriented guide to data flow, threat modeling, provider boundaries, and human review when AI is introduced around substation testing.
On this page
Relay-testing software touches protection settings, engineering files, network configuration, and sometimes physical I/O. Adding an AI assistant changes the data-flow and threat model even when the assistant is described as “only a copilot.”
A minimum threat model
Document the files, APIs, network interfaces, credentials, providers, logs, and tool calls. Consider malicious or accidental instructions inside a relay manual, report, SCL file, or imported settings package. Test whether the assistant can be induced to reveal data, call a tool outside scope, or convert an untrusted value into a setting or trip action.
The safest default is read-only evidence, allow-listed tools, explicit outbound context, no setting writes, no GOOSE publication, and no equipment control. NIST SP 800-82 and IEC 61850 security guidance should be read together with the real site architecture.
Governance before demo
Name the use case, risk owner, reviewer, acceptance criteria, audit fields, provider, update process, and rollback plan. Test abstention and provider outage. A model that produces a confident answer when the manual is missing is a risk signal, not a usability win.
ProtectionAI positioning
ProtectionAI can be evaluated as a local, evidence-first assistant around relay testing. “Local-first” must be supported by a data-flow diagram and customer review; it is not a substitute for OT security assessment.
Make the boundary testable
“Local-first” is a deployment property only when the team can show what stays local and what leaves the boundary. A pilot diagram should name the workstation, evidence store, test-set interface, identity provider, outbound proxy, AI provider, logs, and administrative path. It should show whether manuals, settings, SCL files, COMTRADE records, or customer names are sent outside the site. The safest initial mode is read-only: the assistant may retrieve approved documents and produce a draft, but it cannot write relay settings, publish GOOSE or Sampled Values, change a test-set configuration, or operate a breaker simulator without an explicitly governed tool path.
Threat testing should include prompt injection hidden in a manual or imported report, poisoned metadata, a deliberately stale procedure, provider unavailability, a malformed SCL file, and an instruction that attempts to bypass the engineer review. The expected response to ambiguity is an abstention with the missing evidence named. Audit records should capture the input identifiers, model/provider version, tools called, output, reviewer, and disposition without copying unnecessary personal or operational data into analytics.
Governance gates for a protection pilot
Before a demonstration, agree on the intended use, prohibited uses, risk owner, reviewer role, retention period, incident path, model-change process, and rollback. Define acceptance measures that are observable: citation completeness, missing-evidence detection, false-flag rate, review time, and successful abstention on unsupported questions. A benchmark of polished prose is not enough. The test set should contain contradictory settings, incomplete records, and cases that require the assistant to say “not enough evidence.”
NIST AI RMF gives a useful vocabulary for governing these risks, while NIST SP 800-82 keeps the OT boundary and operational consequences visible. IEC 61850 test guidance adds a protocol-specific dimension: the team must understand the difference between validating an IED function, a message exchange, and the whole system. ProtectionAI can support the evidence plan; it does not replace the utility’s cyber assessment, change control, or qualified protection work.
References
- National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0). https://doi.org/10.6028/NIST.AI.100-1
- Stouffer, K., et al. (2023). Guide to operational technology (OT) security (NIST SP 800-82 Rev. 3). https://doi.org/10.6028/NIST.SP.800-82r3
- IEC. (2020). IEC TR 61850-90-4: Network engineering guidelines. https://webstore.iec.ch/en/publication/64801
- IEEE PES. (2020). Application testing of IEC 61850 based systems (Technical Report 84). https://resourcecenter.ieee.org/publications/technical-reports/pes_tp_tr84_psrc_120720
References
Questions engineers ask
What should remain outside an AI agent's authority?
Relay settings writes, GOOSE or Sampled Values publication, test-set configuration changes, breaker actions, and final protection acceptance should remain governed engineering actions.
What is the safest first deployment mode?
Start with read-only evidence retrieval, bounded plan review, offline simulation, explicit outbound context, and tests that demonstrate abstention when evidence is missing.
Does local-first automatically mean secure?
No. The deployment must show the workstation, evidence store, network boundary, provider connection, logs, identity path, and change controls so the claim can be reviewed.
