dedupe

Fast, secure, efficient backup program

Deduplicating archiver with compression and authenticated encryption.

🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

A powerful duplicate file finder and an enhanced fork of 'fdupes'.

Deduplication tool for yarn.lock files

A powerful and modular toolkit for record linkage and duplicate detection in Python

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Make CSS easier and more maintainable by using JavaScript

Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)

Best-Effort Extent-Same, a btrfs dedupe agent

🆔 Command line tool for deduplicating CSV files

🆔 Examples for using the dedupe library

Finding and deleting near-duplicate images based on perceptual hash.

A fast file deduplicator

Fast block-level out-of-band BTRFS deduplication tool.

📧 CLI to deduplicate mails from mail boxes.

A simple command line interface to the datamade/dedupe library.

Yet Another Dupes Finder

Self-contained C# library for data deduplication using Sqlite

Utilities for de-duping Django model instances