How to Buffer MQTT Telemetry Through a Cellular Outage

Back to Blog

Buffer at the Edge, Not Just at the Broker

The most robust place to hold data during an outage is the edge device itself, in a store-and-forward buffer that keeps timestamped values while the link is down and replays them when it returns. Relying only on broker-side queuing does not help when the problem is that the edge cannot reach the broker at all - during a true outage there is no broker to queue for. The edge buffer is what covers that gap.

Size the buffer for the longest outage you must survive without loss, in terms of the data volume generated during that window. The concept and its tradeoffs are in store and forward telemetry. A buffer sized to your worst realistic outage is the difference between a seamless record and a permanent gap.

Preserve the Original Timestamp on Every Value

Buffered data is only useful if it lands in the historian at the time it actually occurred, not the time it was finally delivered. So every buffered value must carry its own source timestamp, and the consumer must honor that timestamp rather than stamping arrival time. A flush that back-dates correctly turns an outage into an invisible event on the trend; one that stamps arrival time compresses hours of data into the reconnect moment.

This is a payload-design decision made before the outage ever happens: the message format must include a trustworthy timestamp from the edge. Ensure the ingestion side keys on that timestamp, as discussed in MQTT to time-series database ingestion. Without source timestamps, buffering preserves the values but destroys their timing.

Use a Persistent Session With a Long Enough Expiry

On the connection itself, a persistent session lets the broker hold any messages destined for the client and lets the client resume without losing its subscriptions, provided the session expiry outlasts the outage. If the session expires during the outage, the broker discards its state and the client reconnects fresh, which can lose queued messages and force a full resubscribe. Set the expiry longer than your worst outage.

Pair persistence with the reconnect behavior so the client comes back cleanly rather than hammering the tower. The two together define recovery. The session mechanics are in MQTT session expiry and clean start, and the backoff that keeps reconnection sane is in reconnect backoff for telemetry clients.

Publish Buffered Data at a Reliable QoS

Flushing the buffer at QoS 0 risks losing the very data you carefully preserved, because a still-marginal link on recovery can drop messages with no retry. Publish buffered telemetry at QoS 1 so each message is acknowledged and retried until it lands - the buffer's whole purpose is defeated if the flush is lossy. The consumer must tolerate the rare QoS 1 duplicate, which is a fair trade for guaranteed delivery.

Control the flush rate so recovery does not itself overwhelm the just-restored link. Dumping a full buffer at maximum speed onto a marginal cellular connection can saturate it and trigger another drop. Pace the flush within the inflight and queue limits tuned for the link, as in tuning MQTT inflight and queue limits for a slow link.

Verify With a Simulated Outage

Test by cutting the link deliberately for a realistic duration while the edge keeps generating data, then restoring it and confirming the buffer flushes completely, every value landing in the historian at its correct original timestamp with no gap. A payload sequence number across the outage proves nothing was lost. This is the only honest test - a bench with no real dropout proves nothing about field behavior.

Push the test to the edge of your buffer size and expiry to find where it actually fails, so you know the real limit rather than assuming it. Confirm the flush does not saturate the link on recovery and trigger a second outage. A buffering scheme proven against a simulated worst-case outage is one you can trust in the field.

Common Mistakes

The core mistake is relying on broker-side queuing for an outage where the edge cannot reach the broker at all - the queue that matters is at the edge. The second is flushing without source timestamps, so hours of buffered data all land at the reconnect moment and the trend is a spike instead of a record.

Another is a session expiry shorter than the outage, which discards session state mid-outage and loses queued messages. And do not flush at QoS 0 or at full speed: the first loses data on a marginal recovery link, the second saturates it and drops again. Reliable QoS and a paced flush are what make recovery clean.

Frequently Asked Questions