How to Configure Store-and-Forward Buffering

Back to Blog

Size the Buffer for the Worst Realistic Outage

Buffer size is the first and most important setting, and it comes from two numbers: how much data the site produces per unit time, and how long the link can plausibly be down. Multiply them with margin, because a buffer that covers a typical thirty-minute blip is useless during the multi-day outage that a cellular tower failure or a site power event can cause. Size for the bad day, not the average one, because the whole reason the buffer exists is the bad day.

Account for the outages that are longest, not most common. A brief link flap happens often and needs little buffer; the rare long outage is what actually loses data, and it is the one to size against. Think through the realistic failure modes at that specific site - remote power, weather, carrier maintenance - and pick a buffer that survives the worst of them with room to spare, because a buffer that overflows silently discards the oldest records first, which are often the ones marking when the trouble started.

Reduce the data rate into the buffer with the same tools you use for the link. Report-by-exception and deadband cut how fast the buffer fills during an outage exactly as they cut normal traffic, so a site configured for efficient reporting also buffers longer for the same storage. This is one more reason the update-model choice matters on thin links: it directly sets how long your buffer lasts when the link is gone.

Keep Source Timestamps on Buffered Data

The single behavior that separates good store-and-forward from bad is preserving the original event time. Every record held in the buffer must carry the timestamp of when it actually happened, not when it eventually forwards. If buffered data gets stamped on arrival at the historian, then a two-hour outage compresses two hours of events into a single moment when the link returns, and the timeline is destroyed. Source timestamps are what make recovered data land where it belongs.

Confirm the whole path preserves source time, not just the buffer. The device, the buffer, the forwarding mechanism, and the historian must all agree to keep the original timestamp end to end, because a single link in the chain that re-stamps on receipt undoes the rest. This is the same source-versus-receipt distinction that governs event ordering, and it is why buffering and clocks are inseparable - verifying it is part of verifying time sync across the stack.

Handle the reconnect so the fill is clean. When the link returns, the buffer should forward its backlog in order, without duplicating records the historian already has and without reordering them relative to live data now flowing. Getting this wrong produces either double-counted samples or an interleaved mess where recovered history and current data fight for the timeline. The recovery should look, in the finished archive, exactly as if the link had never dropped.

Set Recovery to Fill Gaps Cleanly

Define what happens at the boundary between buffered and live data. On reconnect, the site has a backlog to send and new data still arriving, and the forwarding logic has to merge them without a seam. Decide whether the buffer drains fully before live data flows or the two interleave by timestamp, and confirm the historian files everything by event time so the join is invisible in the finished trend rather than showing up as a gap or an overlap.

Protect against duplication across the outage. If the historian received some records just before the link dropped, the buffer must not re-send them on recovery, or the archive gets duplicates that corrupt totals and confuse trends. A clean implementation tracks what was acknowledged and forwards only what was not, so recovery is exactly the missing slice and nothing more. Duplicate suppression is as important as gap filling, and both are easy to overlook until a total comes out wrong.

Relate the buffered recovery to the other way gaps get filled. Store-and-forward is the automatic, real-time mechanism; a manual historian backfill from a CSV file is the fallback when buffering was not there or overflowed. Knowing both, and which one applies, means a gap discovered after the fact still has a path to being filled correctly rather than left as a permanent hole in the record.

Verifying Gapless, Correctly-Timed Recovery

Test with a real outage, not a theory. Cut the link for a meaningful duration - long enough to matter, ideally approaching the buffer's rated capacity - while the process runs, then restore it and examine the recovered archive. The data during the outage should be complete, in order, and stamped with the times it actually occurred, so the trend across the outage window looks continuous. Anything less means the configuration is not delivering what buffering promises.

Check the two failure modes specifically: timeline and duplication. Confirm the recovered events sit at their true times, not bunched at the reconnect moment, which proves source timestamps survived. Then confirm there are no duplicated samples at the boundaries where the outage began and ended, which proves the acknowledgment tracking works. If both hold across a full-duration test, the buffering is trustworthy; a store-and-forward you never outage-tested is a store-and-forward you are only hoping works, much like an untested backup.

Common Mistakes to Avoid

The first mistake is sizing the buffer for the common short flap instead of the rare long outage, so it overflows during exactly the event that mattered and silently drops the oldest records. Size for the worst realistic outage at the site's data rate, with margin. The second mistake is letting buffered data be stamped on arrival, which collapses hours of events into one moment and destroys the timeline the whole mechanism exists to preserve.

The third mistake is ignoring duplication at the reconnect boundary, so records already received get re-sent and corrupt totals. Track acknowledgments and forward only the missing slice. The fourth is never testing with a real outage, trusting that the feature works because it is enabled - buffering that has never survived a full-duration link cut in practice is an assumption, and the day you find out it does not work is the day you needed the data it lost.

Frequently Asked Questions