Deduplication That Knows a Copy From a Version

Repeat copies are caught as they arrive. Nothing is deleted automatically, so the signed original is never swept up with the clutter.

Catching repeats without losing the record

The risk in cleaning up a repository is not missing a duplicate. It is deleting the one signed copy.

Every cleanup exercise runs into the same problem: the file that looks redundant is sometimes the executed original, and the one that looks authoritative is sometimes a draft somebody renamed. Detection is worth having only if it can tell those apart.

01

Catch repeats on the way in

Repeat copies of a document are detected as it arrives rather than accumulating quietly across folders for someone to find later.

02

Compare what is inside, not the filename

Matching works on document contents, including text read from scanned and photographed pages, so a file renamed on the way between two folders is still recognised as the same document.

03

Separate a copy from a version

Every document carries a draft, active or superseded status. A superseded version is part of the history rather than clutter, and is not treated as a redundant copy of the one that replaced it.

04

Leave the decision to a person

A detected match is surfaced with the context needed to judge it — owner, source, status and where each copy came from. What happens next is chosen by someone with the rights to choose it.

05

Record what was decided

The action taken on a group goes into the same tamper-evident history as everything else, so a cleanup can be explained afterwards rather than merely trusted.

What counts as a duplicate, and what does not

Four groups the system would surface. The two it declines to call duplicates are the ones worth reading — they are what separates a cleanup you can defend from one you cannot.

Duplicate
  • Vendor agreement — uploaded by operations
  • scan0032.pdf — uploaded by finance

The same document, saved twice under different names by two departments that did not know the other held it.

What happens

Both are shown as one group with their owners and sources. Someone decides which is the record to keep; neither disappears on its own.

Duplicate
  • Circular, emailed to all branches
  • The same circular, uploaded at three branches

One document that arrived four times because each recipient filed their own copy of it.

What happens

The repeat arrivals are flagged as they come in rather than after the folder has four of them.

Not a duplicate
  • Agreement, marked superseded
  • Agreement, marked active

Two versions of one agreement, and the older one is the record of what was in force at the time.

What happens

Left alone. A superseded document is history, not clutter, and status is what keeps it from being read as a redundant copy.

Not a duplicate
  • Draft agreement
  • The executed copy carrying signatures

Nearly identical text, but only one of them is evidence of anything.

What happens

Left alone. This is the case that makes automatic deletion the wrong default no matter how confident the match looks.

Technical Details

What is never deleted automatically
Everything. Nothing is removed, merged or overwritten by the system on its own. Detection is automatic; acting on it is not, and the action is taken by a person with the rights to take it.
When detection runs
As documents arrive, so repeat copies are caught on intake rather than accumulating and being discovered during a later cleanup exercise.
What is compared
Document contents, including text read from scanned and photographed pages, rather than filenames alone. A file renamed between folders is still recognised as the same document.
Versions
Draft, active and superseded are explicit statuses on every document. A superseded version is retained as part of the record and is not treated as a redundant copy of the one that replaced it.
Context shown on a match
Owner, source, category, document type and status for each record in the group, plus the dates, reference numbers and parties extracted from them, so the decision is made on evidence rather than on filenames.
Permissions
A group only shows records this reader can already open. A document stays private to whoever added it until it is deliberately shared, so deduplication cannot be used to discover what exists elsewhere.
Audit
Actions taken on a duplicate group are recorded in the same tamper-evident history as everything else, so what was resolved and by whom can be reconstructed afterwards.
Tenancy
Each organisation’s records are stored fully separated from every other organisation’s.

Where copies come from

Nobody sets out to store the same document eleven times. In each of these it happened as a side effect of people doing something reasonable.

A migration from drives nobody has opened in years

An old server, a stack of external drives and a cloud folder all hold overlapping copies of the same archive. Detecting the repeats as material comes in stops the new repository inheriting the old one’s mess.

  • Legacy archives
  • External drives
  • Migrated folders

Every branch keeps its own copy of the same circular

One head-office document ends up filed independently in a dozen branch folders, and none of them knows the others exist. Catching the repeats on arrival keeps one record authoritative instead of twelve equally plausible ones.

  • Circulars
  • Standard forms
  • Internal policies

The same attachment, downloaded eleven times

An attachment gets saved by everyone on the thread, each into their own folder and under whatever name their machine gave it. Comparing contents rather than filenames is the only thing that recognises those as one document.

  • Email attachments
  • Downloaded copies
  • Shared folders

A records team has to justify what was removed

Cleanup is easy to do and hard to defend once someone asks what happened to a particular file. Because resolutions are recorded rather than silent, the answer is retrievable instead of remembered.

  • Retention records
  • Cleanup decisions
  • Audit trails

Frequently Asked Questions

Will it delete our documents?+

No. Nothing is deleted, merged or overwritten automatically, no matter how confident a match looks. The system detects and presents; a person with the rights to act decides. That division is deliberate — the cost of a wrong automatic deletion is unbounded, and the cost of a duplicate sitting there one more day is not.

How does it avoid deleting a signed original?+

The safeguard is not clever matching, it is that removal is never automatic. On top of that, documents carry an explicit draft, active or superseded status, so a version that is part of the record is distinguishable from a redundant copy before anyone is asked to decide anything.

Are superseded versions treated as duplicates?+

No. A superseded document is the record of what was in force at the time, which is exactly what you need when someone asks what the rule was two years ago. Status keeps it in the history rather than in the cleanup pile.

Does it work on scanned documents?+

Yes. Scanned and photographed pages are read on arrival, so matching works on what the page says rather than on the filename. Two photographs of the same register page, saved under different names, are recognisable as the same document.

Can it match files that were renamed?+

Yes. Comparison works on document contents rather than filenames, which is the point — a file that travelled between folders picking up a new name at each stop is the single most common way one document becomes four.

Can someone see a duplicate of a document they have no access to?+

No. A group only contains records that reader can already open. Documents stay private to whoever added them until deliberately shared, so a match cannot be used to work out what is stored elsewhere in the organisation.