I'd use perl:
perl -MDigest::SHA -le '
for $f (@ARGV) {
$d = Digest::SHA->new(256);
$d->addfile($f);
push @{$h{$d->digest}}, $f
}
print join ", ", @{$h{$_}} for keys %h' -- *.txt
We're building an associative array whose keys are the sha256 hash of the files and the value the list of files with that hash.
It makes it easy to sort the output by number of occurrences for instance with:
perl -MDigest::SHA -le '
for $f (@ARGV) {
$d = Digest::SHA->new(256);
$d->addfile($f);
push @{$h{$d->digest}}, $f
}
print join ", ", @{$h{$_}} for sort {@{$h{$a}} <=> @{$h{$b}}} keys %h' -- *.txt
Or even sort the list of files in each set by file name:
perl -MDigest::SHA -le '
for $f (@ARGV) {
$d = Digest::SHA->new(256);
$d->addfile($f);
push @{$h{$d->digest}}, $f
}
print join ", ", sort {$a cmp $b} @{$h{$_}} for
sort {@{$h{$a}} <=> @{$h{$b}}} keys %h' -- *.txt