Not every retention decision is a single number. A historian actually faces two related but separate questions: how long to keep the full-resolution raw samples, and how long to keep the pre-computed aggregates rolled up from them. Because raw data is bulky and aggregates are compact, most teams keep summaries far longer than the raw values they came from. This guide explains the two-track retention decision, why raw and aggregate lifetimes are set independently, and how per-tag policies let compliance and trending needs be met without hoarding raw data that no one queries.
Raw vs Aggregate Retention in one line: Raw versus aggregate retention is the practice of setting two separate retention periods in a historian: one for full-resolution raw samples and another for pre-computed aggregates such as minimum, maximum, and average rollups. Because aggregates are far smaller than raw data, teams typically keep them much longer, dropping bulky raw samples after a shorter window while retaining summaries for long-term trending and compliance.
A historian stores data at two levels of detail. The raw level is every sample as it was recorded, the ground truth from which everything else is derived. The aggregate level is a set of summaries computed from those raw samples over intervals, commonly the minimum, maximum, and average across each period, sometimes with counts or totals. Aggregates answer most trending questions, what was the daily average, how high did it peak, without needing every individual reading, and they occupy a fraction of the space because one summary stands in for many samples.
Because raw and aggregate data differ so sharply in bulk and in how often they are queried, treating their retention as one setting wastes the opportunity the two levels create. The raw layer is heavy and, past a certain age, rarely needed at full detail; the aggregate layer is light and remains useful for years. A two-track policy sets each lifetime on its own terms: a raw retention period after which full-resolution samples are dropped, and a separate, usually much longer, aggregate retention period over which the rollups persist.
The practical result is that a historian can forget the exact reading from a random second three years ago while still knowing that hour's average, peak, and trough. For the vast majority of questions asked of old data, that is exactly the right trade. The raw samples are kept long enough to cover the period during which anyone might need instant-by-instant detail, and the aggregates carry the long memory that trending and reporting actually rely on.
The reason to keep aggregates far longer than raw data is a straightforward balance of value against cost. Raw samples are the most expensive data to store, in the largest volume, yet their marginal value drops steeply with age: the precise value at a specific instant last quarter is rarely needed, while the precise value at a specific instant today is central to operations. Aggregates invert that economics, tiny to store and enduring in usefulness, since a multi-year trend or a monthly summary is built from rollups, not from raw points.
This is why the common pattern is to keep raw for a bounded window and aggregates for a long horizon or indefinitely. Within the raw window, the historian can answer any question at full fidelity, reconstruct a fast transient, or re-derive an aggregate if a computation changes. Once raw data ages past that window and is dropped, the aggregates remain to serve the long view. The team accepts that beyond the raw horizon they can no longer see individual samples, in exchange for keeping the far cheaper summaries available for as long as the business needs them.
There is an important caveat about what aggregates cannot recover. Once raw samples are discarded, any detail finer than the aggregation interval is gone for good; you cannot later ask a question that needs sub-interval resolution over that old period. This is why the raw window must be set to comfortably cover any need for fine detail, and why regulated measurements sometimes demand raw be kept much longer than trending alone would justify. Aggregates are a summary, not a lossless compression, so dropping raw is a deliberate, irreversible choice that the retention window should reflect.
One retention policy rarely fits every tag, because tags differ in why they are kept. A regulated measurement subject to record-keeping requirements may need its raw samples preserved for a long, specified period, while a routine operational point may only need raw for a short window and aggregates thereafter. Per-tag retention lets each point carry its own raw and aggregate lifetimes, so the policy matches the obligation and the usefulness of the specific measurement rather than applying one blunt rule across everything.
In a cloud historian such as Merobix, which holds data from oil and gas, water, power, and manufacturing sites, per-tag policies are what reconcile competing pressures without excess. Compliance-critical tags can keep raw data for as long as the relevant requirement dictates; high-volume noisy points can shed raw quickly while keeping aggregates for trending; and everything in between is tuned to its role. The result is that the historian meets every retention obligation and every trending need while avoiding the cost of hoarding raw data uniformly on points that do not warrant it.
Setting these policies well is an act of matching data lifetime to purpose. For each tag, or each class of tags, you decide how long full detail is genuinely needed and how long the summarized view must persist, and you record those as the raw and aggregate retention. Because a cloud platform scales storage with what is kept, tightening raw retention on the tags that do not need it directly restrains cost, while extending aggregate retention preserves the long history that reports and analysis depend on. The two-track, per-tag approach turns retention from a single compromise into a precise instrument that satisfies compliance and trending without keeping more than each tag actually earns.
Raw retention sets how long a historian keeps every individual full-resolution sample, while aggregate retention sets how long it keeps summaries such as minimum, maximum, and average rollups computed from those samples. Raw data is bulky and detailed; aggregates are compact and cover long spans. Setting the two independently lets a historian drop heavy raw data after a shorter window while keeping lightweight summaries for much longer.
Raw samples are the most expensive data to store and their value drops steeply with age, since the exact value at a specific past instant is rarely needed, whereas aggregates are tiny and remain useful for years of trending and reporting. Keeping aggregates far longer preserves the long-term view cheaply while dropping the bulky raw data that no one queries. The trade is losing sub-interval detail on old periods in exchange for lasting, low-cost summaries.
No. Aggregates are summaries over intervals, so once the underlying raw samples are dropped, any detail finer than the aggregation interval is gone permanently and cannot be reconstructed. That is why the raw retention window must cover any period during which fine, instant-by-instant detail might be needed, and why regulated measurements may require raw to be kept far longer than trending alone would justify.
Merobix reads your field devices into a cloud SCADA - the real thing behind these terms, live in days from any browser.