Automation Glossary • Linearity Error

What Is Instrument Linearity Error?

Merobix Engineering • • 7 min read

An ideal sensor turns input into output along a perfectly straight line - double the pressure, double the signal, all the way up. Real sensors bow. Linearity error is the measure of that bow: the largest gap between the sensor's actual input-output curve and the ideal straight line it is supposed to follow. What makes linearity subtle is that the reported number depends on which straight line you compare against, so two honest datasheets can quote different linearity figures for the same physical curve. This page defines linearity error, explains how the terminal-based and best-fit reference lines change the number, and shows why a two-point calibration can pass while the mid-range reading is still off.

Back to Blog

Linearity Error in one line: Linearity error, or non-linearity, is the maximum deviation of an instrument's actual input-output curve from a specified ideal straight line, usually expressed as a percentage of span. Its value depends on which reference line is used: a terminal-based line drawn through the endpoints, or a best-fit line that minimizes the overall deviation. Because it describes the bow in the middle of the range, it is invisible to a two-point calibration that only checks the endpoints.

Deviation From an Ideal Straight Line

Every analog instrument has a transfer function - the relationship between the physical thing it measures and the signal it outputs. For most transmitters that relationship is designed to be linear, so that a given change in input always produces the same change in output regardless of where you are in the range. Linearity is how faithfully the real device honors that design. No physical sensor is perfectly linear; the sensing element, the electronics, and the mechanics all introduce a gentle curvature, so the actual transfer function is a slightly bowed line rather than a ruler-straight one.

Linearity error captures the worst of that bow in a single number: the maximum distance, anywhere in the range, between the actual curve and the reference straight line, usually stated as a percentage of span. If the curve bows out most in the middle and that maximum gap is 0.2 percent of span, the linearity error is 0.2 percent of span. It is a shape property, distinct from a simple zero or span error, because you cannot remove it by shifting or rescaling the line - the curve is genuinely not straight, and no straight line will lie on top of it everywhere.

Because it is one component of overall accuracy rather than the whole story, linearity is often reported alongside separate figures for hysteresis and repeatability. Together those describe how much the reading can deviate from the ideal for reasons intrinsic to the instrument. Linearity specifically answers the question: if the instrument were perfectly repeatable and free of hysteresis, how far would its smooth curve still stray from a straight line?

Terminal-Based Versus Best-Fit Reference Lines

Here is the catch that trips people up: linearity error is meaningless until you say which straight line you are measuring deviation from, and there is more than one reasonable choice. The terminal-based (or endpoint) method draws the reference line through the two ends of the range - the zero and full-scale points - and measures the largest deviation of the curve from that line. It is intuitive and matches how a simple endpoint calibration thinks, but because the line is pinned to the extremes, a curve that bows in the middle shows its full deviation there, so terminal-based linearity tends to report a larger number.

The best-fit straight line (also called independent linearity) instead positions the reference line to minimize the maximum deviation across the whole range, letting the line float rather than pinning it to the endpoints. Geometrically this splits the bow: the line sits so the curve strays about equally above it and below it, which roughly halves the worst-case gap compared with the endpoint line. The same physical sensor therefore has a smaller best-fit linearity figure than terminal-based figure, not because it is any straighter, but because the yardstick was chosen more generously.

This is why comparing linearity specs across datasheets requires reading the fine print. A vendor quoting best-fit linearity can advertise a tighter number than one quoting terminal-based linearity for an identical curve, and neither is lying. The reference method is part of the specification, not a footnote, and an accuracy comparison that ignores it can rank a worse instrument ahead of a better one. When the method is not stated, the safe assumption is that the more flattering best-fit basis was used.

Why Two-Point Cal Misses It, and Multi-Point Cal in SCADA

A two-point calibration checks the instrument at just two values, typically zero and full scale, and adjusts so those two points read correctly. That is exactly the pattern that hides linearity error. Because a two-point cal only touches the endpoints - the very points the terminal-based line is pinned to - it can leave the instrument reading perfectly at both ends while the bowed middle is still off by the full linearity error. The calibration passes, the paperwork looks clean, and the mid-range reading is quietly wrong. An operator working near the middle of the range is the one who pays for the endpoints looking good.

The remedy is a multi-point calibration, which checks the instrument at several values spread across the range - commonly three or five points, sometimes more - so the curve itself is sampled, not just its ends. With intermediate points you can actually see the bow: if the endpoints read true but the mid-scale point reads high, you have measured the linearity error directly rather than assuming it away. For any measurement whose non-linearity matters, or where operation sits in the middle of the range, a multi-point check is the only calibration that tells the truth about the whole curve.

That multi-point picture is also what a SCADA scaling really needs. When a cloud SCADA platform such as Merobix takes a raw analog input and scales it into engineering units, a plain two-point linear scaling assumes the sensor is straight between its endpoints - which reintroduces exactly the linearity error a two-point cal missed. Capturing the multi-point calibration results lets the mid-range deviation be documented and, where it matters, corrected in the scaling, so the value on the dashboard reflects the sensor's real curve rather than an idealized straight line. On remote sites where a technician cannot easily recheck a reading, knowing the linearity behavior from a proper multi-point cal is what keeps a mid-range value trustworthy long after the endpoints were last verified.

Frequently Asked Questions

What is the difference between terminal-based and best-fit linearity?

Terminal-based linearity measures deviation from a straight line drawn through the zero and full-scale endpoints, while best-fit linearity measures deviation from a line positioned to minimize the maximum deviation across the whole range. For the same bowed curve, the best-fit line roughly halves the worst-case gap, so it yields a smaller reported number. The same sensor can therefore show two different linearity figures depending on which reference line the datasheet used.

Why can a two-point calibration pass while the reading is still off in the middle?

A two-point calibration only checks and corrects the instrument at the endpoints, typically zero and full scale. Linearity error is a bow in the middle of the range, so an instrument can read perfectly at both ends while being off by the full linearity error at mid-scale. Only a multi-point calibration, which samples several values across the range, exposes that mid-range deviation.

How does linearity error affect a SCADA analog input?

SCADA typically scales a raw analog input into engineering units with a linear conversion between two endpoints, which assumes the sensor is perfectly straight in between. If the sensor has real linearity error, that assumption reintroduces the mid-range deviation into the displayed value. Documenting a multi-point calibration lets the bow be recognized and, where it matters, corrected in the scaling so the dashboard value reflects the sensor's true curve.

From Definitions to a Live Dashboard

Merobix reads your field devices into a cloud SCADA - the real thing behind these terms, live in days from any browser.

Request a Free Demo +1 (903) 307-7300
More in Automation Glossary
Error Band  •  Total Probable Error (TPE)  •  Uncertainty Budget  •  Expanded Uncertainty (k)  •  Systematic vs Random Error  •  Sensitivity  •  All Automation Glossary →
Free SCADA operator training
Merobix University - 70 video lessons & 261 quiz questions, from first login to compliance reporting. No demo call required.
Start free →