Reason #1: To save space
The splitting of the distros like this is really more to do with how much cruft you have to download and install on the system. There's really no difference between the server version and the desktop versions except for which software packages are bundled on the media (DVD) + and what is installed by default when you use the particular versions.
As an example you could easily install CentOS using the server or minimal editions and then once the system is up and functioning, run a command such as:
$ yum groupinstall '..Desktop Environment..'
to install all the packages required to make it a fully functioning desktop edition.
Reason #2: To specialize the distro installation
As to what else is missing, well that really depends on each distro. With server distros, they're usually trying to cut out software to make the download smaller and more manageable. So things like a web browser, LibreOffice, and anything GUI related will typically not be included in a server edition. They'll also omit non-GUI software like compilers, such as gcc, and a whole host of other software, which really have no business being included in a truly server install.
Some Examples
On the various distros, such as with anything Red Hat based like CentOS or Fedora, you can use the yum command to get a list of the various groups of software:
$ yum grouplist
Available environment groups:
Fedora Server
Fedora Workstation
Fedora Cloud Server
KDE Plasma Workspaces
Xfce Desktop
LXDE Desktop
Cinnamon Desktop
LXQt Desktop
MATE Desktop
Sugar Desktop Environment
Development and Creative Workstation
Web Server
Infrastructure Server
Basic Desktop
Minimal Install
Installed groups:
C Development Tools and Libraries
Available Groups:
3D Printing
Administration Tools
Audio Production
Authoring and Publishing
Books and Guides
Cloud Infrastructure
Cloud Management Tools
Cloud Server Tools
Container Management
D Development Tools and Libraries
Design Suite
Development Tools
Domain Membership
Editors
Educational Software
Electronic Lab
Engineering and Scientific
Fedora Eclipse
FreeIPA Server
Games and Entertainment
Headless Management
LibreOffice
MATE Applications
MATE COMPIZ
Medical Applications
Milkymist
Network Servers
Office/Productivity
RPM Development Tools
Robotics
Security Lab
Sound and Video
System Tools
Text-based Internet
Window Managers
Done
and then see what's included in each using this command:
$ yum groupinfo "X"
-or- in later versions of Fedora/CentOS, for example:
$ yum group info "DNS Name Server"
Group: DNS Name Server
Group-Id: dns-server
Description: This package group allows you to run a DNS name server (BIND) on the system.
Default Packages:
+bind-chroot
Optional Packages:
bind
dnsperf
ldns
nsd
pdns
pdns-recursor
rbldnsd
unbound
You can also get a summary of what groups you have installed using this command:
$ yum groups
Available environment groups: 15
Installed groups: 1
Available Groups: 35
Done