A clever trick:
rsync -a --delete empty/ your_folder/
It's super CPU intensive, but really really fast. See http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.htmlhttps://web.archive.org/web/20130929001850/http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html