What is a (static) library?
A library is just an object file, that is obtained by pre-compiling several source files together. These source files offer common functionality that may be re-used in a lot of other source files. Thus instead of compiling them each time, with each other source file that requires them, it is more efficient to just pre-compile it, ready to be linked with whichever program needs to use it. Static libraries end with a “.a” extension.
How does the compiler know that a particular static library must be linked to some program?
Well, this is specified by using the -l option with gcc. For instance, if your hello.c program needs to use the libmath.a library, then it would be compiled as
gcc -c hello.c
gcc -o hello hello.o -lmath
Also if the libmath.a library isn’t located in a standard path like say /usr/lib, then it is also necessary to specify the exact path at which it may be found by using the -L switch with gcc
Ok, so what are shared libraries then?
So static libraries are not bad. They keep pre-compiled the commonly used code, so it is efficient and ready to use, we save up on the compile time. But there may be more than 1 program that is using the libmath library. And each such separate program that needs the libmath library simply just links in its own copy. This means that multiple programs running in the memory may have multiple copies of the same library. Clearly this wastes space. What we want to have is, a way of sharing the same library amongst several programs, that may be resident in the memory at the same time. Shared libraries offer this solution. Shared libraries typically end with the “.so” extension
Well that, how do shared libraries work?
First of all, shared libraries are usually compiled with -fPIC option (positionally independent code) – this makes it possible to place it at any virtual memory location, during runtime.
Executables ( which are usually in the ELF format btw) that need to use shared libraries, aren’t directly linked to them, instead the ELF header, contains information about which shared libraries the executable will require.
Now it is the job of the dynamic linker, to locate these libraries at run time ( if some other program is already using these libraries, they might already be in the memory, otherwise they will need to be loaded into the memory) and perform all the symbol resolution. Again, if the libraries aren’t in some standard location like /usr/lib, then the path needs to be specified by setting the LD_LIBRARY_PATH variable. Shared libraries are pretty much the norm in most of the use cases today, with static libraries being used rarely.
Some useful tools while working with shared libraries?
One very useful tool is ldd. Should be run as ldd <name of your executable>. This will tell you all the shared libraries that are used in the executable, and also the path to which they resolve. Often useful in figuring out some weird compilation error.
ldconfig is another tool. This will keep track of all the possible directories in which the shared libraries may be located. The paths can be specified in the /etc/ld.so.conf and it will also search standard paths like /usr/lib. It then creates a list of all the different libraries in /etc/ld.so.cache. This helps the dynamic linker to resolve the paths quickly without wasting too much time in searching all the possible directories.
And finally what is dynamic linking?
Not going into it much here, but essentially instead of loading all the shared libraries that a program depends on upfront, this is a way of lazily loading a library, only when it is required. The dlopen API is provided and used for achieving this.