Automation Glossary • Historian compression ratio

What Is a Historian Compression Ratio?

Merobix Engineering • • 7 min read

A process historian scans tags constantly but does not store every value it sees, because most consecutive readings are close enough to the last stored point that keeping them would waste space without adding information. The compression ratio is the single number that tells you how aggressively a tag is being filtered: what fraction of the values scanned actually made it into the archive. It is one of the most useful health metrics an operator or administrator can watch, because a ratio that has drifted far from normal is usually the fastest sign that a tag is mis-tuned. This page explains what the compression ratio measures, why the usual range sits high, and how the extremes point straight at a tuning problem.

Back to Blog

Historian compression ratio in one line: A historian compression ratio expresses how much of the incoming data a historian discarded through deadband or swinging-door filtering, usually stated as the percentage of scanned values that were not stored. Typical tags sit in a high range because most readings fall within the deviation limit, and a ratio that collapses toward zero or pins near total compression is the fastest single indicator that a tag's compression settings are wrong. It is a health metric operators watch, distinct from the compression algorithm itself.

What the Ratio Actually Counts

The compression ratio is a comparison between two counts: how many values a tag scanned over a period, and how many of those values the historian actually wrote to the archive. Because a historian only stores a new point when the signal has moved by more than its configured deviation limit, most scans on a steady tag add nothing and are discarded, and only the scans that carry genuinely new information are kept. Expressed as a percentage of values discarded, or equivalently as the ratio of values in to values stored, this number captures in one figure how effectively the compression is doing its job on that tag.

It is worth being precise that the compression ratio is a measure of the amount of data removed, not a measure of accuracy. A high ratio means a lot of scanned values were redundant enough to leave out, which on a well-tuned steady tag is exactly what you want, because those discarded values would have reconstructed to essentially the same trend. The ratio says nothing on its own about whether the reconstructed trend is faithful; that is a separate question about the deviation limit and reconstruction error. Two different metrics are at work, and confusing high compression with high accuracy is a common trap.

Different people express the same idea in slightly different ways, so it helps to know the convention in the tool at hand. Some historians report the percentage of values discarded, where higher is more aggressive compression; others report a ratio of input samples to stored samples, where a larger multiple means the same thing. Whatever the presentation, the underlying quantity is identical: of everything the tag saw, how much survived into the archive. Reading that number correctly for the tool is the first step to using it as a diagnostic.

Why a High Ratio Is Normal and What the Extremes Mean

For most process tags a high compression ratio is entirely expected and healthy. Process values usually sit near a setpoint or drift slowly, so scan after scan falls within the deviation limit and is discarded, and only occasional moves are stored. It is common for well-tuned tags to have most of their scanned values compressed away, which is the whole reason historians can keep years of data at a manageable size. A high ratio, in itself, is a sign the tag is behaving and the compression is working, not a cause for concern.

The alarming cases are the extremes. When a ratio collapses toward zero, meaning almost every scanned value is being stored, the tag has effectively stopped compressing. That usually points to a deviation limit set too tight for the signal's noise, so ordinary instrument jitter constantly exceeds the limit and forces a store on nearly every scan. The consequences are practical and unwelcome: the archive fills far faster than planned, and a single noisy tag can consume a disproportionate share of storage while adding no real information, just captured noise. A ratio near zero on a tag that should be steady is one of the clearest calls to action a historian gives you.

The opposite extreme is just as telling. A ratio pinned at or near total compression, where the tag almost never stores a new value, can mean the deviation limit is set far too wide, so genuine changes in the process are being swallowed and never recorded. This is more dangerous than the noisy case because it fails silently: the trend looks smooth and the archive is small, but real excursions have been compressed out and cannot be recovered. A tag that should show normal process movement but is storing almost nothing deserves immediate suspicion, because over-compression hides exactly the events an operator most needs to see.

Using the Ratio to Diagnose Tags in a SCADA Environment

The reason the compression ratio is such a valued diagnostic is that it turns a subtle tuning problem into a single number you can rank and scan. Rather than opening trends for thousands of tags to judge whether each is over- or under-compressed, an administrator can list tags by compression ratio and immediately see the outliers: the ones storing almost everything, which are wasting space on noise, and the ones storing almost nothing, which may be hiding real change. Watching the ratio at the fleet level makes it a triage tool, directing tuning effort to the handful of tags that are actually misbehaving.

This becomes especially valuable when a historian sits behind a SCADA system pulling data from many field sites, because the tags most likely to be mis-tuned are often the ones no one looks at directly. A remote pressure or flow tag from a distant site might quietly consume storage because its deviation limit never matched the instrument noise, or might quietly hide excursions because its limit was set too generously during commissioning and never revisited. The compression ratio surfaces both conditions without anyone having to inspect the raw signal, which is exactly what you need when the data comes from more sites than any person can review by eye.

In a cloud monitoring context the same principle scales further, and a platform such as Merobix can track the compression yield of tags across many sites so that a ratio drifting out of its normal band becomes a maintenance signal rather than a hidden cost. A tag whose ratio has collapsed is flagged as a storage and noise problem to be retuned; a tag whose ratio has climbed to near-total compression is flagged as a possible loss of fidelity to be checked against its raw signal. Treating the ratio as an ongoing health metric, rather than a number glanced at once during setup, keeps a large historian both compact and trustworthy over the long run.

Frequently Asked Questions

What is a normal historian compression ratio?

For most process tags a high ratio is normal, with the large majority of scanned values compressed away, because process values usually sit near a setpoint or drift slowly so scan after scan falls within the deviation limit. This is exactly what lets historians keep years of data at a manageable size. A high ratio in itself is a sign the tag is behaving and the compression is working, not a problem.

What does it mean when the compression ratio drops toward zero?

A ratio near zero means almost every scanned value is being stored, so the tag has effectively stopped compressing. This usually points to a deviation limit set too tight for the signal's noise, so ordinary instrument jitter constantly exceeds the limit and forces a store on nearly every scan. The result is that the archive fills far faster than planned while capturing mostly noise, which makes a near-zero ratio a clear call to retune the tag.

Is a higher compression ratio always better?

No, because the ratio measures how much data was removed, not how accurate the stored trend is. A ratio pinned near total compression can mean the deviation limit is set too wide, so genuine process changes are being swallowed and never recorded, which fails silently while the trend looks smooth. High compression and high fidelity are separate things, so a very high ratio on a tag that should show movement deserves suspicion rather than praise.

From Definitions to a Live Dashboard

Merobix reads your field devices into a cloud SCADA - the real thing behind these terms, live in days from any browser.

Request a Free Demo +1 (903) 307-7300
More in Automation Glossary
Compression deviation limit tuning  •  Compression reconstruction error  •  Collector store-and-forward buffer  •  Store-and-forward recovery ordering  •  Historian backfill window  •  Backfilling a historian from CSV  •  All Automation Glossary →
Free SCADA operator training
Merobix University - 70 video lessons & 261 quiz questions, from first login to compliance reporting. No demo call required.
Start free →