Also sed (here using GNU sed syntax) can do the job:
MYPATH=$(printf '%s\n' "$MYPATH" | sed ':b;s/:\([^:]*\)\(:.*\):\1/:\1\2/;tb')
this one works well only in case first path is . like in dogbane's example.
In general case you need to add yet another s command:
MYPATH=$(printf '%s\n' "$MYPATH" | sed ':b;s/:\([^:]*\)\(:.*\):\1/:\1\2/;tb;s/^\([^:]*\)\(:.*\):\1/:\1\2/')
It works even on such construction:
$ echo "/bin:.:/foo/bar/bin:/usr/bin:/foo/bar/bin:/foo/bar/bin:/bar/bin:/usr/bin:/bin" \
| sed ':b;s/:\([^:]*\)\(:.*\):\1/:\1\2/;tb;s/^\([^:]*\)\(:.*\):\1/\1\2/'
/bin:.:/foo/bar/bin:/usr/bin:/bar/bin
