Tutorial
Wake a desktop agent when a file arrives: a watch plus a sweep you can trust
Use the OS file-watch as a hint, make a fingerprinted reconciliation sweep the source of truth, and never read a half-copied file
Oct 29, 2026 - 10 min read
What You Will Build
A desktop agent that notices a file the moment it lands in a watched folder, waits until the file has finished copying, fingerprints it, and processes it - with a reconciliation sweep underneath that makes being wrong about any single notification harmless.
You will build it in two languages (Python for the examples, .NET for the Windows specifics) and you will wire four triggers into one sweep: start-up catch-up, overflow catch-up, a cadence tick, and a visible Scan button. By the end, dropping a file is fast, dropping it twice counts once, and killing the agent mid-drop loses nothing.
Prerequisites
- Python 3.10+ with
pip install watchdog(the examples use it; any OS works). - For the Windows-specific overflow handling, .NET 6+ and a folder you can watch.
- A place to keep a set of fingerprints you have already processed - a file, a table, or just an in-memory set for the demo.
- Comfort reading short Python and C#; you do not need to be an expert in either.
Tip
Build this against a throwaway folder first. You want to be able to drop fifty files at once, kill the process, and copy a 2 GB file in slowly - all of which you would rather not do against real data on your first run.
Mental Model
The operating system can tell you a file arrived, but it is allowed to miss some and to tell you about others more than once. So you never trust a single notification. You treat it as a hint that makes a sweep run sooner, and you make the sweep - not the hint - the thing that decides what is true.
The sweep is level-triggered: it asks "what is on disk versus what have I already processed?" and converges, rather than trusting it saw every event. The hint is edge-triggered: it fires once, when something changes, and may be dropped under load. You keep both, in that order. This is the same split a Kubernetes controller makes, and the same one Syncthing makes when it runs an hourly full rescan even with its live watcher on, "because it is possible that some changes aren't picked up."
Steps
Follow them in order. The first two are free; the rest unlock with Supporter.
Watch the whole tree
Start a recursive watch on the account folder and push every event onto a queue. Treat what lands on the queue as a hint, not a fact - the real work happens later, in the sweep.
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class Hint(FileSystemEventHandler):
"""Every event is a hint that something may have changed - never a guarantee."""
def __init__(self, queue):
self.queue = queue
def on_created(self, event):
if not event.is_directory:
self.queue.put(event.src_path)
def on_moved(self, event):
if not event.is_directory:
self.queue.put(event.dest_path)
def watch(root, queue):
observer = Observer()
observer.schedule(Hint(queue), root, recursive=True) # the whole tree, one watch
observer.start()
return observer
On Windows the native layer is ReadDirectoryChangesW, which can watch a subtree in a single call; watchdog wraps it for you and falls back to polling where no native observer exists.
Done
Drop a file into the watched folder and its path appears on your queue within a second. Drop one into a subfolder: it appears too, because the watch is recursive.
Wait until the file is finished copying
A notification fires on the first bytes, not the last. If you read immediately you can read half a file. Hold off until the size has stopped changing and the file opens without a sharing conflict - the same "wait until the write is finished" move the chokidar library makes by waiting until "the file size has stayed constant."
import os, time
def settled(path, wait=2.0, tries=30):
"""True once the file has stopped growing and opens cleanly; False if still being written."""
last = -1
for _ in range(tries):
try:
size = os.path.getsize(path)
with open(path, "rb"): # opens only if nothing else holds the file
pass
except OSError:
size = -1 # locked or vanished: not settled yet
if size == last and size >= 0:
return True
last = size
time.sleep(wait)
return False
The size-stability check is the primary signal; the open attempt is a cheap complement. If a file is still copying, settled returns False and you simply leave it for the next pass - the sweep will pick it up once it is done.
Done
Start copying a large file into the folder and call settled on it: it returns False while the copy runs and True only after the size holds steady across two checks.
Sources
- Python watchdog: a cross-platform folder observer with a polling fallbackpython-watchdog.readthedocs.io
- .NET FileSystemWatcher: Error event on buffer overflow; InternalBufferSize up to 64 KBlearn.microsoft.com
- Windows ReadDirectoryChangesW: recursive watch; on overflow the buffer is discarded and you must re-enumeratelearn.microsoft.com
- chokidar awaitWriteFinish: hold the event until the file size stays constantgithub.com
- Git content addressing: a file is identified by a hash of its contentsgit-scm.com
- Amazon S3 event notifications are designed to be delivered at least oncedocs.aws.amazon.com
- Syncthing keeps a live watcher and still runs a full rescan every hour as a fail-safedocs.syncthing.net