Static vs Shared libraries and Dynamic Linking

What is a (static) library?

A library is just an object file, that is obtained by pre-compiling several source files together. These source files offer common functionality that may be re-used in a lot of other source files. Thus instead of compiling them each time, with each other source file that requires them, it is more efficient to just pre-compile it, ready to be linked with whichever program needs to use it. Static libraries end with a “.a” extension.

How does the compiler know that a particular static library must be linked to some program?

Well, this is specified by using the -l option with gcc. For instance, if your hello.c program needs to use the libmath.a library, then it would be compiled as 

gcc  -c hello.c

gcc -o hello hello.o -lmath

Also if the libmath.a library isn’t located in a standard path like say /usr/lib, then it is also necessary to specify the exact path at which it may be found by using the -L switch with gcc

Ok, so what are shared libraries then?

So static libraries are not bad. They keep pre-compiled the commonly used code, so it is efficient and ready to use, we save up on the compile time. But there may be more than 1 program that is using the libmath library. And each such separate program that needs the libmath library simply just links in its own copy. This means that multiple programs running in the memory may have multiple copies of the same library. Clearly this wastes space. What we want to have is, a way of sharing the same library amongst several programs, that may be resident in the memory at the same time. Shared libraries offer this solution. Shared libraries typically end with the “.so” extension

Well that, how do shared libraries work?

First of all, shared libraries are usually compiled with -fPIC option (positionally independent code) – this makes it possible to place it at any virtual memory location, during runtime. 

Executables ( which are usually in the ELF format btw) that need to use shared libraries, aren’t directly linked to them, instead the ELF header, contains information about which shared libraries the executable will require.

Now it is the job of the dynamic linker, to locate these libraries at run time ( if some other program is already using these libraries, they might already be in the memory, otherwise they will need to be loaded into the memory) and perform all the symbol resolution. Again, if the libraries aren’t in some standard location like /usr/lib, then the path needs to be specified by setting the LD_LIBRARY_PATH variable. Shared libraries are pretty much the norm in most of the use cases today, with static libraries being used rarely.

Some useful tools while working with shared libraries?

One very useful tool is ldd.  Should be run as ldd <name of your executable>. This will tell you all the shared libraries that are used in the executable, and also the path to which they resolve. Often useful in figuring out some weird compilation error. 

ldconfig is another tool. This will keep track of all the possible directories in which the shared libraries may be located. The paths can be specified in the /etc/ld.so.conf and it will also search standard paths like /usr/lib. It then creates a list of all the different libraries in /etc/ld.so.cache. This helps the dynamic linker to resolve the paths quickly without wasting too much time in searching all the possible directories.

And finally what is dynamic linking?

Not going into it much here, but essentially instead of loading all the shared libraries that a program depends on upfront, this is a way of lazily loading a library, only when it is required. The dlopen API is provided and used for achieving this.

man page of the week : mincore

There’s always some nifty command you can find lurking in the linux manpages. This then helps you instantly and magically do whatever you were trying to do. The caveat is that to do the trick you do sometimes need to remember some elusive combination of options. For instance, I have completely forgotten a neat combination of options I used with iostat during my intern project, that fetched me the right level of detail, for checking the read/write performance.

But before I ramble too far, here’s the point : whenever I can remember, I will let some neat manpage enjoy some limelight on this blog, as a tribute to all the times manpages have saved my life :).

So here I present mincore. The command that lets you check if a page is in memory, or will need to be fetched from the disc! No magic combinations for this one so far! Go man it now!