An SFTP file drop integration is one of the oldest and most durable ways to move SCADA data between systems that do not share a live API. The control system, or a gateway near it, exports its readings to a comma-separated or fixed-width file on a schedule and places that file in a landing folder on an SFTP server, where the receiving system picks it up, parses it, and archives it. It sounds primitive next to streaming APIs, and it is, but it survives across firewalls and vendors that agree on almost nothing else. This page explains the file conventions, the atomic-rename trick, and how to reconcile files that go missing or arrive twice.
SFTP file drop integration in one line: An SFTP file drop SCADA integration exports tag readings from the control system into flat files, usually CSV or fixed-width, and pushes them on a schedule to a folder on an SFTP server. A downstream system watches that landing zone, ingests each completed file, and moves it to an archive, giving two systems a reliable batch interface without a shared live connection.
Flat file exchange predates modern APIs and refuses to die because it asks so little of both sides. The sender only needs to write a file and open an outbound SFTP session; the receiver only needs to watch a folder. There is no shared library, no matching protocol version, and no requirement that both systems be online at the same instant. A file written overnight is simply waiting in the morning, which makes the pattern naturally tolerant of outages on either end.
It also passes cleanly through the kind of tightly controlled network boundaries common around industrial systems. Security teams are far more comfortable allowing a single outbound SFTP connection to a known host on a known port than opening inbound API access to a control network. The file drop becomes a one-way, auditable channel where every exchange leaves a physical artifact on disk that can be inspected, replayed, or retained for compliance.
The trade-off is latency and granularity. A file drop is a batch mechanism, so data arrives in chunks minutes or hours apart rather than as a live stream, and it carries whatever the export query captured at that moment. For daily production totals, meter reads, or shift summaries this is exactly right. For anything needing sub-minute freshness it is the wrong tool, which is why file drops usually coexist with a real-time path rather than replacing one.
The single most important detail in a file drop is avoiding partial reads. If the sender writes directly into the folder the receiver is watching, the receiver can grab the file while it is still being written and parse a truncated, corrupt record. The standard defense is atomic rename: the sender writes to a temporary name or a staging subfolder, for example a .tmp suffix, and only after the file is fully flushed and closed does it rename it into place with its final name. Because rename within the same filesystem is atomic, the receiver never sees a half-written file, only a complete one that appears instantly.
Filenames carry meaning and have to be agreed up front. A good convention encodes the source, the data type, and a timestamp, so a name like site12_dailyflow_20260726T0300.csv sorts naturally, is unique, and tells the receiver exactly what window it covers without opening it. Embedding the timestamp in the name lets the receiver detect gaps, since a missing hour is a missing filename in an otherwise regular sequence. It also makes duplicates obvious, because a file with a name already processed is a repeat rather than new data.
The internal format needs the same discipline. A header row naming the columns, a consistent quoting and escaping rule for text fields, an explicit and unambiguous timestamp format including time zone, and a stable column order all keep the parser from breaking when a value happens to contain a comma or a field is empty. Fixed-width files trade the delimiter problem for strict column positions, which some legacy mainframes still require. Either way, both sides pin the format in writing, because a silent change to the export layout is the most common cause of a file drop suddenly failing.
Because a file drop is loosely coupled, files occasionally go missing or arrive more than once, and a serious integration plans for both. A missed file usually comes from an outage on the sender during its scheduled export window, or a network interruption during upload. The receiver detects this by watching the expected filename sequence, and when the timestamped name for a period never appears it raises an alert and, ideally, requests a re-export of that window rather than silently leaving a hole in the data.
Duplicates are the mirror image and are handled by idempotent ingestion. If the sender retries an upload after an uncertain failure, or an operator re-drops a file by hand, the same records can arrive twice. The receiver guards against double counting by keying each record on a stable identifier, such as tag plus timestamp, and by remembering which filenames it has already processed, so a repeated file is recognized and either skipped or merged rather than appended. Moving each processed file into a dated archive folder, rather than deleting it, gives both a record of what was ingested and a way to replay if a downstream problem is found later.
For an operator running dozens of remote sites, a cloud platform such as Merobix can sit on either end of this exchange, generating tidy timestamped exports from field data or consuming a legacy system's file drops, while quietly enforcing atomic writes, sequence checks, and dedup on the way in. The result is that a mechanism as humble as a CSV on an SFTP server behaves reliably at scale, filling the gap left where a modern API is not available, and doing so with a full, auditable trail of every file that moved.
Atomic rename stops the receiver from reading a file that is still being written, which would produce truncated or corrupt records. The sender writes to a temporary name and only renames the file into the watched folder once it is fully closed. Because a same-filesystem rename is instantaneous, the receiver only ever sees complete files.
When filenames embed a regular timestamp, the expected sequence is predictable, so a period whose file never arrives shows up as a gap in the naming sequence. The receiver watches for that gap, raises an alert, and can request the sender re-export the missing window. This is far more reliable than assuming every scheduled file always shows up.
No. SFTP runs the file transfer over an encrypted SSH connection, so credentials and data are protected in transit, whereas plain FTP sends everything in the clear and is unsuitable for modern industrial networks. When people say file drop today they almost always mean SFTP or another encrypted transport, not legacy FTP.
Merobix reads your field devices into a cloud SCADA - the real thing behind these terms, live in days from any browser.