Using libltdl

Configury

Because libltdl supports so many different platforms [1] it needs to be configured for the host platform before it can be used.

The path of least resistance to successfully integrating libltdl into your own project, dictates that the project use Libtool for linking its module loader with libltdl. This is certainly the method I use and recommend, and is the method discussed in this chapter. However, I have seen projects which did not use Libtool (specifically because Libtool's poor C++ support made it difficult to adopt), but which wanted the advantages of libltdl. It is possible to use libltdl entirely without Libtool, provided you take care to use the configuration macros described here, and use the results of those running these macros to determine how to link your application with libltdl.

The easiest wat to add libltdl support to your own projects is with the following simple steps:

1. You must add the libltdl sources to your project distribution. If you are not already using Libtool in some capacity for your project, you should add AC_PROG_LIBTOOL [2] to your configure.in. That done, move to the top level directory of the project, and execute:

   $ libtoolize --ltdl $ ls -F aclocal.m4 configure.in libltdl/ $ ls libltdl/ COPYING.LIB README aclocal.m4 configure.in stamp-h.in Makefile.am acconfig.h config.h.in ltdl.c Makefile.in acinclude.m4 configure ltdl.h

2. libltdl has its own configuration to run in addition to the configuration for your project, so you must be careful to call the subdirectory configuration from your top level configure.in:

   AC_CONFIG_SUBDIRS(libltdl)

   And you must ensure that Automake knows that it must descend into the libltdl source directory at make time, by adding the name of that subdirectory to the SUBDIRS macro in your top level Makefile.am:

   SUBDIRS = libltdl src

3. You must also arrange for the code of libltdl to be linked into your application. There are two ways to do this: as a regular Libtool library; or as a convenience library (see the section called Creating Convenience Libraries in the chapter called Introducing GNU Libtool). Either way there are catches to be aware of, which will be addressed in a future release. Until libltdl is present on the average user's machine, I recommend building a convenience library. You can do that in configure.in:

   AC_LIBLTDL_CONVENIENCE AC_PROG_LIBTOOL

   The main thing to be aware of when you follow these steps, is that you can only have one copy of the code from libltdl in any application. Once you link the objects into a library, that library will not work with any other library which has also linked with libltdl, or any application which has its own copy of the objects. If you were to try, the libltdl symbol names would clash. The alternative is to substitute AC_LIBLTDL_CONVENIENCE with AC_LIBLTDL_INSTALLABLE. Unfortunately there are currently many potential problems with this approach. This macro will try to find an already installed libltdl and use that, or else the embedded libltdl will be built as a standard shared library, which must be installed along with any libraries or applications that use it. There is no testing for version compatibility, so it is possible that two or more applications that use this method will overwrite one anothers copies of the installed libraries and headers. Also, the code which searches for the already installed version of libltdl tends not to find the library on many hosts, due to the native libraries it depends on being difficult to predict. Both of the AC_LIBLTDL_... macros set the values of INCLTDL and LIBLTDL so that they can be used to add the correct include and library flags to the compiler in your Makefiles. They are not substituted by default. If you need to use them you must also add the following macros to your configure.in:

   AC_SUBST(INCLTDL) AC_SUBST(LIBLTDL)

4. Many of the libltdl supported hosts require that a separate shared library be linked into any application that uses dynamic runtime loading. libltdl is wrapped around this native implementation on these hosts, so it is important to link that library too. Adding support for module loading through the wrapped native implementation is independent of Libtools determination of how shared objects are compiled. On GNU/Linux, you would need to link your program with libltdl and libdl, for example. Libtool installs a macro, AC_LIBTOOL_DLOPEN, which adds tests to your configure that will search for this native library. Whenever you use libltdl you should add this macro to your configure.in before AC_PROG_LIBTOOL:

   AC_LIBTOOL_DLOPEN AC_LIBLTDL_CONVENIENCE AC_PROG_LIBTOOL ... AC_SUBST(INCLTDL) AC_SUBST(LIBLTDL)

   AC_LIBTOOL_DLOPEN takes care to substitute a suitable value of LIBADD_DL into your Makefile.am, so that your code will compile correctly wherever the implementation library is discovered:

   INCLUDES += @INCLTDL@ bin_PROGRAMS = your_app your_app_SOURCES = main.c support.c your_app_LDADD = @LIBLTDL@ @LIBADD_DL@

Libtool 1.4 has much improved inter-library dependency tracking code which no longer requires @LIBADD_DL@ be explicitly referenced in your Makefile.am. When you install libltdl, Libtool 1.4 (or better) will make a note of any native library that libltdl depends on - linking it automatically, provided that you link libltdl.la with libtool. You might want to omit the @LIBADD_DL@ from your Makefile.am in this case, if seeing the native library twice (once as a dependee of libltdl, and again as an expansion of @LIBADD_DL@) on the link line bothers you.

Beyond this basic configury setup, you will also want to write some code to form a module loading subsystem for your project, and of course some modules! That process is described in the section called Module Loader and the section called Dynamic Module respectively.

Memory Management

Internally, libltdl maintains a list of loaded modules and symbols on the heap. If you find that you want to use it with a project that has an unusual memory management API, or if you simply want to use a debugging malloc, libltdl provides hook functions for you to set the memory routines it should call.

The way to use these hooks is to point them at the memory allocation routines you want libltdl to use before calling any of its API functions:

lt_dlmalloc = (lt_prt_t (*) PARAMS((size_t))) mymalloc; lt_dlfree = (void (*) PARAMS((lt_ptr_t))) myfree;

Notice that the function names need to be cast to the correct type before assigning them to the hook symbols. You need to do this because the prototypes of the functions you want libltdl to use will vary slightly from libltdls own function pointer types-- libltdl uses lt_ptr_t for compatibility with K&R compilers, for example.

Module Loader

This section contains a fairly minimal libltdl based dynamic module loader that you can use as a base for your own code. It implements the same API as the simple module loader in the section called A Simple GNU/Linux Module Loader in the chapter called Dynamic Loading, and because of the way libltdl is written is able to load modules written for that loader, too. The only part of this code which is arguably more complex than the equivalent from the previous example loader, is that lt_dlinit and lt_dlexit must be called in the appropriate places. In contrast, The module search path initialisation is much simplified thanks to another relative improvement in the libltdl API: - Function: int lt_dlsetsearchpath (const char *path)

This function takes a colon separated list of directories, which will be the first directories libltdl will search when trying to locate a dynamic module.

Another new API function is used to actually load the module: - Function: lt_dlhandle lt_dlopenext (const char *filename)

This function is used in precisely the same way as lt_dlopen. However, if the search for the named module by exact match against filename fails, it will try again with a .la extension, and then the native shared library extension (.sl on HP-UX, for example).

The advantage of using lt_dlopenext to load dynamic modules is that it will work equally well when loading modules not compiled with Libtool. Also, by passing the module name parameter with no extension, this function allows module coders to manage without Libtool.

#include <stdio.h> #include <stdlib.h> #ifndef EXIT_FAILURE # define EXIT_FAILURE 1 # define EXIT_SUCCESS 0 #endif #include <limits.h> #ifndef PATH_MAX # define PATH_MAX 255 #endif #include <string.h> #include <ltdl.h> #ifndef MODULE_PATH_ENV # define MODULE_PATH_ENV "MODULE_PATH" #endif typedef int entrypoint (const char *argument); /* Save and return a copy of the dlerror() error message, since the next API call may overwrite the original. */ static char *dlerrordup (char *errormsg); int main (int argc, const char *argv[]) { char *errormsg = NULL; lt_dlhandle module = NULL; entrypoint *run = NULL; int errors = 0; if (argc != 3) { fprintf (stderr, "USAGE: main MODULENAME ARGUMENT\n"); exit (EXIT_FAILURE); } /* Initialise libltdl. */ errors = lt_dlinit (); /* Set the module search path. */ if (!errors) { const char *path = getenv (MODULE_PATH_ENV); if (path != NULL) errors = lt_dlsetsearchpath (path); } /* Load the module. */ if (!errors) module = lt_dlopenext (argv[1]); /* Find the entry point. */ if (module) { run = (entrypoint *) lt_dlsym (module, "run"); /* In principle, run might legitimately be NULL, so I don't use run == NULL as an error indicator in general. */ errormsg = dlerrordup (errormsg); if (errormsg != NULL) { errors = lt_dlclose (module); module = NULL; } } else errors = 1; /* Call the entry point function. */ if (!errors) { int result = (*run) (argv[2]); if (result < 0) errormsg = strdup ("module entry point execution failed"); else printf ("\t=> %d\n", result); } /* Unload the module, now that we are done with it. */ if (!errors) errors = lt_dlclose (module); if (errors) { /* Diagnose the encountered error. */ errormsg = dlerrordup (errormsg); if (!errormsg) { fprintf (stderr, "%s: dlerror() failed.\n", argv[0]); return EXIT_FAILURE; } } /* Finished with ltdl now. */ if (!errors) if (lt_dlexit () != 0) errormsg = dlerrordup (errormsg); if (errormsg) { fprintf (stderr, "%s: %s.\n", argv[0], errormsg); free (errormsg); exit (EXIT_FAILURE); } return EXIT_SUCCESS; } /* Be careful to save a copy of the error message, since the next API call may overwrite the original. */ static char * dlerrordup (char *errormsg) { char *error = (char *) lt_dlerror (); if (error && !errormsg) errormsg = strdup (error); return errormsg; }

This file must be compiled with libtool, so that the dependent libraries (libdl.so on my GNU/Linux machine) are handled correctly, and so that the dlpreopen support is compiled in correctly (see the section called dlpreopen Loading):

$ libtool --mode=link gcc -g -o ltdl-loader -dlopen self \ -rpath /tmp/lib ltdl-loader.c -lltdl gcc -g -o ltdl-loader -Wl,--rpath,/tmp/lib ltdl-loader.c -lltdl -ldl

By using both of lt_dlopenext and lt_dlsetsearchpath, this module loader will make a valiant attempt at loading anything you pass to it - including the module I wrote for the simple GNU/Linux module loader earlier (see the section called A Simple GNU/Linux Dynamic Module in the chapter called Dynamic Loading). Here, you can see the new ltdl-loader loading and using the simple-module module from the section called A Simple GNU/Linux Dynamic Module in the chapter called Dynamic Loading:

$ ltdl-loader simple-module World Hello, World! => 0

Dependent Libraries

On modern Unices [3] , the shared library architecture is smart enough to encode all of the other libraries that a dynamic module depends on as part of the format of the file which is that module. On these architectures, when you lt_dlopen a module, if any shared libraries it depends on are not already loaded into the main application, the system runtime loader will ensure that they too are loaded so that all of the module's symbols are satisfied.

Less well endowed systems [4] , cannot do this by themselves. Since Libtool release 1.4, libltdl uses the record of inter-library dependencies in the libtool pseudo-library (see the chapter called Introducing GNU Libtool) to manually load dependent libraries as part of the lt_dlopen call.

An example of the sort of difficulties that can arise from trying to load a module that has a complex library dependency chain is typified by a problem I encountered with GNU Guile a few years ago: Earlier releases of the libXt Athena widget wrapper library for GNU Guile failed to load on my a.out based GNU/Linux system. When I tried to load the module into a running Guile interpreter, it couldn't resolve any of the symbols that referred to libXt. I soon discovered that the libraries that the module depended upon were not loaded by virtue of loading the module itself. I needed to build the interpreter itself with libXt and rely on back-linking to resolve the Xt references when I loaded the module. This pretty much defeated the whole point of having the wrapper library as a module. Had Libtool been around in those days, it would have been able to load libXt as part of the process of loading the module.

If you program with the X window system, you will know that the list of libraries you need to link into your applications soon grows to be very large. Worse, if you want to load an X extension module into a non-X aware application, you will encounter the problems I found with Guile, unless you link your module with libtool and dynamically load it with libltdl. At the moment, the various X Window libraries are not built with libtool, so you must be sure to list all of the dependencies when you link a module. By doing this, Libtool can use the list to check that all of the libraries required by a module are loaded correctly as part of the call to lt_dlopen, like this:

$ libtool --mode=link gcc -o module.so -module -avoid-version \ source.c -L/usr/X11R6/lib -lXt -lX11 ... $ file .libs/module.so .libs/module.so: ELF 32-bit LSB shared object, Intel 80386, version 1, not stripped $ ldd .libs/module.so libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x4012f00) libXt.so.6 => /usr/X11R6/lib/libXt.so.6 (0x4014500)

Or, if you are using Automake:

... lib_LTLIBRARIES = module.la module_la_SOURCES = source.c module_la_LDFLAGS = -module -avoid-version -L$(X11LIBDIR) module_la_LIBADD = -lXt -lX11 ...

It is especially important to be aware of this if you develop on a modern platform which correctly handles these dependencies natively (as in the example above), since the code may still work on your machine even if you don't correctly note all of the dependencies. It will only break if someone tries to use it on a machine that needs Libtool's help for it to work, thus reducing the portability of your project.

Dynamic Module

Writing a module for use with the libltdl based dynamic module loader is no more involved than before: It must provide the correct entry points, as expected by the simple API I designed - the run entry point described in the section called A Simple GNU/Linux Module Loader in the chapter called Dynamic Loading. Here is such a module, ltdl-module.c:

#include <stdio.h> #include <math.h> #define run ltdl_module_LTX_run int run (const char *argument) { char *end = NULL; long number; if (!argument || *argument == '\0') { fprintf (stderr, "error: invalid argument, \"%s\".\n", argument ? argument : "(null)"); return -1; } number = strtol (argument, &end, 0); if (end && *end != '\0') { fprintf (stderr, "warning: trailing garbage \"%s\".\n", end); } printf ("Square root of %s is %f\n", argument, sqrt (number)); return 0; }

To take full advantage of the new module loader, the module itself must be compiled with Libtool. Otherwise dependent libraries will not have been stored when libltdl tries to load the module on an architecture that doesn't load them natively, or which doesn't have shared libraries at all (see the section called dlpreopen Loading).

$ libtool --mode=compile gcc -c ltdl-module.c rm -f .libs/ltdl-module.lo gcc -c ltdl-module.c -fPIC -DPIC -o .libs/ltdl-module.lo gcc -c ltdl-module.c -o ltdl-module.o >/dev/null 2>&1 mv -f .libs/ltdl-module.lo ltdl-module.lo $ libtool --mode=link gcc -g -o ltdl-module.la -rpath `pwd` \ -no-undefined -module -avoid-version ltdl-module.lo -lm rm -fr .libs/ltdl-module.la .libs/ltdl-module.* .libs/ltdl-module.* gcc -shared ltdl-module.lo -lm -lc -Wl,-soname \ -Wl,ltdl-module.so -o .libs/ltdl-module.so ar cru .libs/ltdl-module.a ltdl-module.o creating ltdl-module.la (cd .libs && rm -f ltdl-module.la && ln -s ../ltdl-module.la \ ltdl-module.la)

You can see from the interaction below that ltdl-loader does not load the math library, libm, and that the shared part of the Libtool module, ltdl-module, does have a reference to it. The pseudo-library also has a note of the libm dependency so that libltdl will be able to load it even on architectures that can't do it natively:

$ libtool --mode=execute ldd ltdl-loader libltdl.so.0 => /usr/lib/libltdl.so.0 (0x4001a000) libdl.so.2 => /lib/libdl.so.2 (0x4001f000) libc.so.6 => /lib/libc.so.6 (0x40023000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) $ ldd .libs/ltdl-module.so libm.so.6 => /lib/libm.so.6 (0x40008000) libc.so.6 => /lib/libc.so.6 (0x40025000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) $ fgrep depend ltdl-module.la # Libraries that this one depends upon. dependency_libs=' -lm'

This module is now ready to load from ltdl-loader:

$ ltdl-loader ltdl-module 9 Square root of 9 is 3.000000 => 0