fdupes is a program written by Adrián López to scan directories for duplicate files, with options to list, delete or replace the files with hardlinks pointing to the duplicate. It first compares file sizes, partial MD5 signatures, full MD5... Wikipedia