dlpreopen Loading

On machines which do not have any facility for shared libraries or dynamic modules, libltdl allows an application to lt_dlopen modules, provided that the modules are known at link time. This works by linking the code for the modules into the application in advance, and then looking up the addresses of the already loaded symbols when lt_dlsym is called. We call this mechanism dlpreopening - so named because the modules must be loaded at link time, not because the API to use modules loaded in this way is any different.

This feature is extremely useful for debugging, allowing you to make a fully statically linked application from the executable and module objects, without changing any source code to work around the module loading calls. As far as the code outside the libltdl API can tell, these modules really are being loaded dynamically. Driving a symbolic debugger across module boundaries is however much easier when blocks of code aren't moving in and out of memory during execution.

You may have wondered about the purpose of the following line in the dynamic module code in the section called Dependent Libraries:

#define run ltdl_module_LTX_run

The reason for redefining the entry point symbol in this way is to prevent a symbol clash when two or more modules that provide identically named entry point functions are preloaded into an executable. It would be otherwise impossible to preload both simple-module.c and ltdl-module.c, for example, since each defines the symbol run. To allow us to write dynamic modules that are potentially preloaded, lt_dlsym will first try to lookup the address of a named symbol with a prefix consisting of the canonicalized name of the module being searched, followed by the characters _LTX_. The module name part of this prefix is canonicalized by replacing all non-alphanumeric characters with an underscore. If that fails, lt_dlsym resorts to the unadorned symbol name, which is how run was found in simple-module.la by ltdl-loader earlier.

Supporting this feature in your module loading code is a simple matter of initialising the address lookup table, and ltdl.h defines a convenient macro to do exactly that: - Macro: LTDL_SET_PRELOADED_SYMBOLS ()

Add this macro to the code of your module loading code, before the first call to a libltdl function, to ensure that the dlopen address lookup table is populated.

Now change the contents of ltdl-loader.c, and add a call to this macro, so that it looks like this:

/* Initialise preloaded symbol lookup table. */ LTDL_SET_PRELOADED_SYMBOLS(); /* Initialise libltdl. */ errors = lt_dlinit ();

Libtool will now be able to fall back to using preloaded static modules if you tell it to, or if the host platform doesn't support native dynamic loading.

If you use LTDL_SET_PRELOADED_SYMBOLS in your module loader, you must also specify something to preload to avoid compilation failure due to undefined lt_preloaded_symbols. You can name modules on the Libtool link command line using one of -dlopen or -dlpreopen. This includes support for accessing the symbols of the main executable opened with lt_dlopen(NULL)--you can ask Libtool to fall back to preopening the main modules like this: $ libtool gcc -g -o ltdl-loader -dlopen self -rpath /tmp/lib \ ltdl-loader.c -lltdl rm -f .libs/ltdl-loader.nm .libs/ltdl-loader.nmS \ .libs/ltdl-loader.nmT creating .libs/ltdl-loaderS.c (cd .libs && gcc -c -fno-builtin -fno-rtti -fno-exceptions "ltdl-loaderS.c") rm -f .libs/ltdl-loaderS.c .libs/ltdl-loader.nm .libs/ltdl-loader.nmS .libs/ltdl-loader.nmT gcc -o ltdl-loader .libs/ltdl-loaderS.o ltdl-loader.c -Wl,--export-dynamic /usr/lib/libltdl.so -ldl -Wl,--rpath -Wl,/tmp/lib rm -f .libs/ltdl-loaderS.o

It doesn't make sense to add preloaded module support to a project, when you have no modules to preopen, so the compilation failure in that case is actually a feature of sorts.

The LTDL_SET_PRELOADED_SYMBOLS macro does not interfere with the normal operation of the code when modules are dynamically loaded, provided you use the -dlopen option on the link line. The advantage of referencing the macro by default is that you can recompile the application with or without preloaded module, and all without editing the sources.

If you have no modules to link in by default, you can force Libtool to populate the preload symbol table by using the -dlopen force option. This is the option used to preload the symbols of the main executable so that you can subsequently call lt_dlopen(NULL).

Multiple modules can be preloaded, although at the time of writing only Libtool compiled modules can be used. If there is a demand, Libtool will be extended to include native library preloading in a future revision.

To illustrate, I have recompiled the simple-module.c module with libtool:

$ libtool --mode=compile gcc -c simple-module.c rm -f .libs/simple-module.lo gcc -c simple-module.c -fPIC -DPIC -o .libs/simple-module.lo gcc -c simple-module.c -o simple-module.o >/dev/null 2>&1 mv -f .libs/simple-module.lo simple-module.lo $ libtool --mode=link gcc -g -o simple-module.la -rpath `pwd` -no-undefined -module -avoid-version simple-module.lo rm -fr .libs/simple-module.la .libs/simple-module.* .libs/simple-module.* gcc -shared simple-module.lo -lc -Wl,-soname \ -Wl,simple-module.so -o .libs/simple-module.so ar cru .libs/simple-module.a simple-module.o creating simple-module.la (cd .libs && rm -f simple-module.la && ln -s ../simple-module.la \ simple-module.la)

The names of the modules that may be subsequently lt_dlopened are added to the application link line. I am using the -static option to force a static only link, which must use dlpreopened modules by definition. I am only specifying this because my host has native dynamic loading, and Libtool will use that unless I force a static only link, like this:

$ libtool --mode=link gcc -static -g -o ltdl-loader ltdl-loader.c \ -lltdl -dlopen ltdl-module.la -dlopen simple-module.la rm -f .libs/ltdl-loader.nm .libs/ltdl-loader.nmS \ .libs/ltdl-loader.nmT creating .libs/ltdl-loaderS.c extracting global C symbols from ./.libs/ltdl-module.a extracting global C symbols from ./.libs/simple-module.a (cd .libs && gcc -c -fno-builtin -fno-rtti -fno-exceptions \ "ltdl-loaderS.c") rm -f .libs/ltdl-loaderS.c .libs/ltdl-loader.nm \ .libs/ltdl-loader.nmS .libs/ltdl-loader.nmT gcc -g -o ltdl-loader ltdl-loader.c .libs/ltdl-loaderS.o \ ./.libs/ltdl-module.a -lm ./.libs/simple-module.a \ /usr/lib/libltdl.a -ldl rm -f .libs/ltdl-loaderS.o $ ./ltdl-loader ltdl-module 345 Square root of 345 is 18.574176 => 0 $ ./ltdl-loader simple-module World Hello, World! => 0

Note that the current release of Libtool requires that the pseudo-library be present for any libltdl loaded module, even preloaded ones. Once again, if there is sufficient demand, this may be fixed in a future release. Until then, if the pseudo-library was deleted or cannot be found, this will happen:

$ rm -f simple-module.la $ ./ltdl-loader simple-module World ./ltdl-loader: file not found.

A side effect of using the LTDL_SET_PRELOADED_SYMBOLS macro is that if you subsequently link the application without Libtool, you will get an undefined symbol for the Libtool supplied lt_preloaded_symbols. If you need to link in this fashion, you will need to provide a stub that supplies the missing definition. Conversely, you must be careful not to link the stub file when you do link with Libtool, because it will clash with the Libtool generated table it is supposed to replace:

#include <ltdl.h> const lt_dlsymlist lt_preloaded_symbols[] = { { 0, 0 } };

Of course, if you use this stub, and link the application without the benefits of Libtool, you will not be able to use any preloaded modules - even if you statically link them, since there is no preloaded symbol lookup table in this case.