Making md5sum understand file names with spaces

Question

I need to use md5sum in Python by using pipe to calculate checksum for a bunch of .mp3 files... is there a command which ignore whitespaces in filenames on the command line of md5sum program?

For example:

import os
def index(directory):
    stack = [directory]
    files = []
    while stack:
        directory = stack.pop()
        for file in os.listdir(directory):
            fullname = os.path.join(directory, file)
            if fullname.endswith('mp3'):
                files.append(fullname)
            if os.path.isdir(fullname) and not os.path.islink(fullname):
                stack.append(fullname)
    return files

def check(directory):
    files = index(directory)
    hvalues = []
    for x in files:
        cmd = 'md5sum' + ' ' + x
        fp = os.popen(cmd)
        res = fp.readline()
        hvalues.append(res)
        stat = fp.close() # What to do with stat?
    return hvalues

Command cmd = 'md5sum' + ' ' + x won't work as it should on files that include whitespaces or special characters, because 'md5sum' tool lacks the ability of properly handling (hashing) files with whitespaces in filenames.

Answer 1

As @binfalse points out, the problem is not in the md5sum program, but the way you invoked it. Your code is actually bad on several levels:

1. You assembled a shell command without escaping. In the worst case, that could lead to the execution of a completely unintended command if one of the filenames happens to be cleverly crafted. That would be horrible, unless you are writing a single-use throwaway script.

2. The os.popen() function has been deprecated since Python 2.6. The recommended replacement is subprocess.Popen(). Be sure to pass a list as the args parameter, not a concatenated string, to avoid the shell-escaping problem previously mentioned.

   def check(directory):
       files = index(directory)
       hvalues = []
       for f in files:
           cmd = ['md5sum', f]
           proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
           hvalues.append(proc.stdout.readline())
           proc.stdout.close()
           stat = os.waitpid(proc.pid, 0)
       return hvalues
   

3. Better yet, use Python's hashlib to calculate the hashes.

Answer 2

That's not a lack of abilities in the md5tool but a general command line restriction. Arguments are separated by spaces. So if you pass a file name containing spaces to md5sum it will interpret each token as a single file. You can get around that by surrounding the file name with quotation marks. That said, try replacing the line

cmd = 'md5sum' + ' ' + x

with

cmd = 'md5sum' + ' "' + x + '"'

and your command line call will look like

md5sum "file name with spaces.mp3"

Thus, md5sum will calculate the hash without complaining.