If you're logging into a graphical session, arrange to start ssh-agent during your session startup. Some distributions already do that for you. If yours doesn't, arrange to run ssh-agent from your session startup script or from your window manager. How you do that depends on your desktop environment and your window manager. For example, if you start your window manager manually, simply replace the call to my_favorite_wm by ssh-agent my_favorite_wm.
Do not start ssh-agent from .bashrc or .zshrc, since these files are executed by each new interactive shell. The place to start ssh-agent is in a session startup file such as .profile or .xsession.
If you want to use the same SSH agent on all processes no matter where you logged in from, you can make it always use the same socket name, instead of using a randomly-named socket. For example, you might put this in your ~/.profile:
export SSH_AUTH_SOCK=~/.ssh/ssh-agent.$HOSTNAME.sock
ssh-add -l 2>/dev/null >/dev/null
if [ $? -ge 2 ]; then
ssh-agent -a "$SSH_AUTH_SOCK" >/dev/null
fi