How to Fix an MQTT Broker Dropping Messages Under Load
A broker that runs fine most of the time but drops messages during peaks - a startup transient, a burst of alarms, a mass reconnect - is showing a load problem, not a random fault. The message loss is a symptom of a limit being hit somewhere: a consumer too slow, a queue too small, or the broker host out of resources. This page orders those causes by likelihood and gives the test and fix for each.
Fix an MQTT Broker Dropping Messages Under Load in one line: An MQTT broker dropping messages under load is hitting a limit: most often a slow consumer whose queue fills so the broker discards messages for it, then a configured queue or memory limit, then host resource exhaustion. Identify which by watching which consumers lag under load and whether the broker logs a queue or resource limit at the moment of loss.
First Checks
Confirm the loss is load-correlated by timing it against throughput. If drops only appear during identifiable peaks and never during quiet periods, a limit is being exceeded under load rather than a component failing. Note exactly which topics or consumers lose messages during a peak, because a broker rarely drops uniformly - the loss concentrates where the bottleneck is.
Check the broker's own logs and metrics at the moment of loss. A well-behaved broker records when it discards messages for a client whose queue is full, or when it hits a memory or connection limit. That log line usually names the cause directly, turning a hunt into a fix. If the broker exposes queue depth per client, watch it climb toward the limit during a peak.
Fix the Slow Consumer First
The most common cause is a single slow consumer. When a subscriber cannot keep up, the broker queues messages for it, and when that per-client queue hits its limit, the broker discards - either the new messages or the old, depending on configuration. So a slow consumer causes loss on its topics specifically, and often only under load when the backlog outruns its processing. It looks like a broker problem but is a consumer problem.
Find the lagging consumer by watching per-client queue depth during a peak - the one whose queue climbs is the culprit. Fix the consumer to process faster, or use a shared subscription to spread its load across several instances so no single consumer falls behind. The mechanism for that is in the MQTT shared subscription.
Check the Queue and Flow-Control Limits
If no single consumer is obviously slow, look at the configured queue limits and flow control. A broker caps how many messages it will hold per client and how many unacknowledged messages a client may have in flight; under a burst, a tight limit is reached and messages are dropped. Raising a queue limit can absorb a bigger burst - if the host has the memory to back it.
Flow control is the graceful alternative to dropping: a receiver that advertises how many messages it will accept lets the broker slow down rather than discard. Confirm your clients use it. The mechanism is in MQTT receive maximum and flow control. Backpressure that slows the sender is almost always better than silent loss at the broker.
Rule Out Host Resource Exhaustion
If queues and consumers are healthy but the broker still drops under load, the host may be out of resources - memory, file descriptors, or connection slots - during the peak. A mass reconnect after a network event is a classic trigger: hundreds of clients reconnecting at once exhaust connection handling and the broker sheds load. This shows as loss concentrated at the reconnect moment, not steady-state.
Check host memory and connection counts at peak. If the broker is resource-starved, the fixes are more memory, higher system limits, or spreading load across a broker cluster - infrastructure changes, not client ones. A mass-reconnect trigger is also softened by client-side reconnect backoff so they do not all return at once, as in reconnect backoff for telemetry clients.
Verify the Fix Under Load
Reproduce the peak that caused the loss - a burst generator or the real startup transient - and confirm no messages are dropped now, using a payload sequence number across the burst to prove completeness. Testing at idle proves nothing; the fault only appears under load, so the verification must apply load. Watch the previously lagging consumer's queue stay bounded through the peak.
Confirm the broker logs no queue-full or resource-limit events during the reproduced peak. A clean run through the exact conditions that used to drop messages, verified by an unbroken sequence, is the acceptance test. If a bigger burst still drops, you have found the new ceiling and can decide whether it is high enough for the operation.
Common Mistakes
The main mistake is blaming the broker for a slow consumer's backlog - the broker dropped because one subscriber could not keep up, and scaling the broker will not help. The second is raising queue limits without the host memory to back them, which trades message loss for a broker crash under the next peak.
Another is ignoring flow control and accepting silent drops when backpressure could have slowed the sender gracefully instead. And do not overlook mass reconnects as a load source: a network event that returns hundreds of clients at once is a peak the broker never sees in steady state, and reconnect backoff on the clients is the cheap defense.
Frequently Asked Questions
Why does my MQTT broker drop messages only under load?
Because a limit is exceeded during peaks that steady-state never reaches. The most common is a slow consumer whose per-client queue fills, so the broker discards its messages; then configured queue or memory limits; then host resource exhaustion, often triggered by a mass reconnect. Watch which consumer lags and check the broker log at the moment of loss.
How do I tell a slow consumer from a broker resource problem?
Watch per-client queue depth during a peak. If one consumer's queue climbs toward its limit while others stay bounded, that consumer is the bottleneck and the loss is on its topics. If the broker host runs out of memory, file descriptors, or connection slots with no single consumer lagging - especially during a mass reconnect - the problem is host resources.
Is raising the queue limit enough to stop the drops?
Only if the host has the memory to back a larger queue and the load is a bounded burst rather than a sustained overload. A bigger queue absorbs a bigger spike but does not fix a consumer that is permanently too slow or a sustained rate above capacity. Prefer flow control so the sender slows gracefully, and fix or scale the slow consumer.
Sources and verification
This page references the protocol specifications published by the organizations below. Editions, product capabilities, and documentation change over time - confirm current requirements and specifications directly with the source.
- MQTT Version 5.0 (OASIS Standard) - OASIS (v5.0, 2019)
Merobix is not affiliated with, endorsed by, or sponsored by these organizations; their names are used only to identify the standards and products discussed.
Automation services
Need help turning this into a working system?
Merobix integrates SCADA, programs Allen-Bradley and Siemens PLCs, and designs and fabricates industrial control panels.
Meeting requests are reviewed before confirmation.