You'll run your things with ssh -X (assuming your laptop runs a Linux with some X11 desktop) and you would only install the packages of client applications, e.g.
sudo aptitude install gedit gnome-terminal emacs24 xterm
I forgot the package manager on Gentoo, so use pacman instead of aptitude
this will pull the required client libraries (e.g. GTK, Xlib, ...), but not the server part.
You don't want any desktop on the remote VPS, only some few GUI applications.