How to Configure a Comms-Fail Alarm in SCADA

Back to Blog

Detect Silence With a Watchdog or Heartbeat

Comms failure is the absence of something, which makes it harder to detect than a bad value - you are alarming on nothing arriving rather than on a wrong number. The standard approach is a watchdog: the system expects contact from the device at a regular interval and starts a timer that the device's activity keeps resetting. If the timer expires without contact, the device is presumed dead and the alarm fires. The mechanism is the same idea as a watchdog timer, applied to comms rather than to a processor.

Use a heartbeat where the protocol or device supports it. A heartbeat is a value the device changes on a known schedule - a counter that increments, a bit that toggles - so the master can confirm not just that bytes are arriving but that the device is actually alive and updating rather than a stale connection returning cached data. Watching a value that should always be changing is a stronger liveness test than watching for any traffic, and it catches the case where the link is up but the device behind it has hung. This is the practical use of a heartbeat signal.

Decide the granularity of detection to match how you would respond. A single alarm per device is usually right - you care that the RTU is gone, not that each of its hundred points went stale - so detect at the device or link level and raise one clear comms-fail alarm rather than a hundred simultaneous point alarms. That one alarm tells the operator exactly what is wrong, where a flood of individual stale-point alarms would bury the real message.

Set the Timeout to Avoid False Trips

The timeout is the setting that makes or breaks a comms-fail alarm, and it comes from the device's normal update rate. Set it too short and normal jitter or a brief link hiccup trips a false comms-fail that trains operators to ignore the alarm; set it too long and a genuinely dead device sits undetected for ages. The timeout should be comfortably longer than the longest normal gap between contacts, with margin for the ordinary variability of the link, so only a real failure trips it.

Account for the link type when you choose the margin. A wired, deterministic connection has little normal variability, so its timeout can be tight; a cellular or radio link has real jitter and occasional brief drops that recover on their own, so its timeout needs more margin to avoid alarming on a hiccup that fixes itself in seconds. Matching the timeout to the link's real behavior is what keeps the alarm meaningful, and it draws on the same understanding of jitter in SCADA networks that shapes reporting settings.

Relate the timeout to how the device reports in the first place. A polled device is contacted on a schedule, so its expected gap is the poll interval; a report-by-exception device may legitimately be quiet for a long time, so a naive timeout would false-trip constantly. On exception-reporting links you need a heartbeat or a slow background poll to give the watchdog something to expect, which is exactly why liveness must be solved separately when you choose report by exception or polling.

Mark Stale Data Bad, Not Just Alarm

Firing the alarm is only half the job; the other half is making the dead device's data stop looking alive. When comms fail, the device's last-received values freeze, and a frozen value on a screen looks exactly like a live steady one unless something marks it. Drive the stale device's tags to bad quality on comms failure so their values are visibly untrustworthy, which is precisely the behavior a quality-flag strategy exists to provide.

Propagate the bad quality so it stops downstream harm. Calculations built on the dead device's values should go bad rather than computing confidently on frozen inputs, alarms on those values should not fire spuriously on a stale number, and trends should show the gap rather than a flat line that looks like a calm process. The comms-fail alarm tells the operator the device is gone; the bad quality stops the frozen data from quietly corrupting everything built on top of it.

Handle recovery so the return to service is clean and visible. When comms come back, the alarm should clear, the quality should return to good, and any store-and-forward backlog should fill the gap correctly. The operator needs to see clearly that the device is alive again and its data is trustworthy once more, rather than being left unsure whether the numbers on screen are current, which is the same clarity the failure alarm provided going the other way.

Verifying the Alarm and the Quality

Test by actually cutting comms, not by assuming. Pull the link or power to a device and confirm three things happen within the expected time: the comms-fail alarm raises, the device's data goes to bad quality and looks untrustworthy on the screen, and calculations and alarms built on that data behave correctly rather than acting on the frozen values. A comms-fail alarm that has never been tested against a real disconnection is an assumption, and the day you find out it does not work is the day a device died silently.

Confirm the timeout is right by watching for false trips over time, especially on jittery links. If the alarm fires during normal operation when the device is actually fine, the timeout is too short and needs more margin, or operators will learn to dismiss it. Then test recovery: restore comms and confirm the alarm clears, quality returns to good, and any buffered data fills the gap. This end-to-end check belongs in your configuration validation so no site goes live with untested failure detection.

Common Mistakes to Avoid

The worst mistake is having no comms-fail detection at all, so a dead device freezes its last values and everyone keeps trusting numbers that stopped being real. Always detect silence with a watchdog or heartbeat. The second mistake is alarming but not marking the data bad, so the operator gets one comms-fail alarm but the frozen values still look live and feed calculations and other alarms as if they were current.

The third mistake is a timeout too short for the link, which false-trips on normal jitter and trains operators to ignore the alarm - match the timeout to the link's real variability with margin. The fourth is a naive timeout on a report-by-exception device that is legitimately quiet, which either false-trips constantly or, if widened to compensate, no longer detects a real failure. Give exception-reporting devices a heartbeat for the watchdog to watch.

Frequently Asked Questions