You can use either of these two one-liners. Both yield the exact same output on my machine and are more precise than all solutions proposed up until now (July 2014) in this question. They are a combination of the two answers (1)(1) and (2)(2). Note that I originally posted this answer here.
Using apt-mark:
comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)
Using aptitude:
comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)
Very few packages still fall through the cracks, although I suspect these are actually installed by the user, either right after the installation through the language localization setup or e.g. through the Totem codec installer. Also, the linux-header versions also seem to accumulate, even though I've only installed the non version-specific metapackage. Examples:
libreoffice-help-en-gb
openoffice.org-hyphenation
gstreamer0.10-fluendo-mp3
linux-headers-3.13.0-29    
How does it work
-  Get the list of manually installed packages. For aptitude, the additional sedstrips out remaining whitespace at the end of the line.
- Get the list of packages installed right after a fresh install.
- Compare the files, only output the lines in file 1 that are not present in file 2.
Other possibilities don't work as well:
-  Using the ubuntu-14.04-desktop-amd64.manifestfile (here for Ubuntu 14.04) instead of/var/log/installer/initial-status.gz. More packages are shown as manually installed even though they are not.
-  Using apt-mark showautoinstead of/var/log/installer/initial-status.gz.apt-markfor example doesn't include the xserver-xorg package, while the other file does.
Both list more packages than the above solution.