Ah yes this is a very confusing part if you've dealt with Unixes for any length of time. There is a standard that most Unixes "try" to follow called FHS - Filesystem Hierarchy Standard.
Given I primarily use Red Hat based distros I'm most familiar with their take on FHS for Fedora, CentOS, and RHEL Linux distros. But I've used Debian & BSD based distros as well and they're not all that different in terms of where things are kept, filesystem wise.
Now to your questions. I would take a look at the FHS document, which loosely governs these directory structures. In general:
Directory - /lib
Contains essential shared libraries and kernel modules.
Purpose: The /lib directory contains those shared library images needed to boot the system and run the commands in the root filesystem, ie. by binaries in /bin and /sbin.
Note1: Shared libraries that are only necessary for binaries in /usr (such as any X Window binaries) must not be in /lib. Only the shared libraries required to run binaries in /bin and /sbin may be here.
Note2: Given the primary purpose of /lib is to contain libraries for tools deployed to the directories /bin & /sbin, the libraries in /lib can be either 32-bit or 64-bit.
For example (Fedora 14 64-bit system)
$ uname -a
Linux grinchy 2.6.35.14-106.fc14.x86_64 #1 SMP Wed Nov 23 13:07:52 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
Here's a sampling of the files from my /lib
./libpam.so.0.82.2: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
./libplc4.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
./libidn.so.11.6.1: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
./upstart/telinit: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, stripped
./upstart/runlevel: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, stripped
./upstart/shutdown: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, stripped
./upstart/reboot: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, stripped
./libdb-4.8.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
./firmware/mixart/miXart8.elf: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), statically linked, not stripped
./libtinfo.so.5.7: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
Directory - /lib<qual>
Alternate format essential shared libraries (optional). These would be the directories /lib32, /lib64, etc.
Purpose: There may be one or more variants of the /lib directory on systems which support more than one binary format requiring separate libraries. This is commonly used for 64-bit or 32-bit support on systems which support multiple binary formats, but require libraries of the same name.
Note: In this case, /lib32 and /lib64 might be the library directories, and /lib a symlink to one of them.
Directory - /usr/lib
Libraries for programming and packages.
Purpose: /usr/lib includes object files, libraries, and internal binaries that are not intended to be executed directly by users or shell scripts.
Note1: Miscellaneous architecture-independent application-specific static files and subdirectories must be placed in /usr/share.
Applications may use a single subdirectory under /usr/lib. If an application uses a subdirectory, all architecture-dependent data exclusively used by the application must be placed within that subdirectory.
Note2: For example, the perl5 subdirectory for Perl 5 modules and libraries.
Directory - /usr/lib<qual>
Alternate format libraries (optional).
Purpose: /usr/lib<qual> performs the same role as /usr/lib for an alternate binary format, except that the symbolic links /usr/lib<qual>/sendmail and /usr/lib<qual>/X11 are not required.
Note: The case where /usr/lib and /usr/lib<qual> are the same (one is a symbolic link to the other) these files and the per-application subdirectories will exist.
TLDR;
In general:
If there are libraries that are required by an executable in either the /bin or /sbin directories, those libraries should be going in the /lib* directories.
If there are libraries for use programs and packages they go in /usr/lib/*. If there are executables that are needed by a particular library, but these executables aren't suppose to be called by the users directly or by root, they go in /usr/libexec.