The easiest way to zero out unused space is to mount the image, fill up the space with zeroes, and remove the zero-filled file.
mount -o loop /path/to/image /mnt
cat /dev/zero >/mnt/zero
rm /mnt/zero
umount /mnt
Depending on what the filesystem is, there may be tools that can work directly on the image. See this question for ext2/ext3/ext4 (answer: zerofree).