It seems to me that a lot of the problems using Linux is because of mismatched dependencies. Specifically that you can't install two versions of a library.
So if program A requires "lib1.2" and program B requires "lib1.3" then they cannot run together.
Question:
Having multiple versions of a library in the system is not an issue in most Linux distributions: shared libraries with distinct sonames can coexist within the same /usr/lib directory. Moreover, package managers can be instructed to install several such libraries by specifying correct package names:
apt-get install libjpeg7
apt-get install libjpeg8
In reality you can encounter two flaws with this system:
Most distributions provide only one library version per distribution version to save space. For example, my Debian Wheezy only has libjpeg8. You will still be able to install a particular library version (e.g. libjpeg7) from sources, and it will coexist with other versions without issues. You can even try to install a package from another distribution, but beware as it might pull a whole lot of dependencies.
Sometimes two library versions with the same soname (which are meant to be compatible) work with some programs and not with others. That's the main reason to have multiple versions of distributions in the first place, by the way. The solution in this case is to keep incompatible libraries outside /usr/lib and make them accessible to a particular program via LD_LIBRARY_PATH or chroot.
Short answer:
Longer answer:
Linux is the kernel, not a distribution. If you use a distribution like NixOS, it's easy to have several versions of the same library installed at the same time.
RE: your comment on 'never seen it in Windows'
Windows avoids this because nearly every program installs an extra copy of the major libs that it uses in its own directory, so you end up with dozens of copies of the same libs installed. This is why you often see setup.exe or similar installers re-installing the same libs like Direct X or Dot Net or Visual C.
Windows does support versioned DLLs, similar to how Linux supports versioned shared libraries but Microsoft encourages app developers to include with their app the specific version that their software depends on - something that was perhaps necessary in the pre-internet age where it might be difficult for users to acquire copies of essential libraries.
It's possible to do the same in Linux but a) it's discouraged as being both wasteful and lazy (on the part of both app developers and lib developers) and b) Linux doesn't, by default, look in the same directory as a program for its shared libraries - it only looks in the directories specified by LD_LIBRARY_PATH. Some programs come with shell script wrappers to change LD_LIBRARY_PATH before running the binary executable.
