Initialising the Module Loader

Before using this code (or any other libltdl based module loader for that matter), a certain amount of initialisation is required:

• libltdl itself requires initialisation.

  1. libltdl should be told to use the same memory allocation routines used by the rest of Sic.

  2. Any preloaded modules (see the section called dlpreopen Loading in the chapter called Using GNU libltdl) need to be initialised with LTDL_SET_PRELOADED_SYMBOLS().

  3. ltdl_init() must be called.

• The module search path needs to be set. Here I allow the installer to specify a default search path to correspond with the installed Sic modules at compile time, but search the directories in the runtime environment variable SIC_MODULES_PATH first.

• The internal error handling needs to be initialised.

Here is the start of the module loader, sic/module.c, including the initialisation code for libltdl:

#if HAVE_CONFIG_H # include <config.h> #endif #include "common.h" #include "builtin.h" #include "eval.h" #include "ltdl.h" #include "module.h" #include "sic.h" #ifndef SIC_MODULE_PATH_ENV # define SIC_MODULE_PATH_ENV "SIC_MODULE_PATH" #endif int module_init (void) { static int initialised = 0; int errors = 0; /* Only perform the initialisation once. */ if (!initialised) { /* ltdl should use the same mallocation as us. */ lt_dlmalloc = (lt_ptr_t (*) (size_t)) xmalloc; lt_dlfree = (void (*) (lt_ptr_t)) free; /* Make sure preloaded modules are initialised. */ LTDL_SET_PRELOADED_SYMBOLS(); last_error = NULL; /* Call ltdl initialisation function. */ errors = lt_dlinit(); /* Set up the module search directories. */ if (errors == 0) { const char *path = getenv (SIC_MODULE_PATH_ENV); if (path != NULL) errors = lt_dladdsearchdir(path); } if (errors == 0) errors = lt_dladdsearchdir(MODULE_PATH); if (errors != 0) last_error = lt_dlerror (); ++initialised; return errors ? SIC_ERROR : SIC_OKAY; } last_error = multi_init_error; return SIC_ERROR; }

Managing Module Loader Errors

The error handling is a very simplistic wrapper for the libltdl error functions, with the addition of a few extra errors specific to this module loader code [2] . Here are the error messages from module.c:

static char multi_init_error[] = "module loader initialised more than once"; static char no_builtin_table_error[] = "module has no builtin or syntax table"; static char builtin_unload_error[] = "builtin table failed to unload"; static char syntax_unload_error[] = "syntax table failed to unload"; static char module_not_found_error[] = "no such module"; static char module_not_unloaded_error[] = "module not unloaded"; static const char *last_error = NULL; const char * module_error (void) { return last_error; }

Loading a Module

Individual modules are managed by finding specified entry points (prescribed exported symbols) in the module: - Variable: const Builtin * builtin_table

An array of names of built-in commands implemented by a module, with associated handler functions. - Function: void module_init (Sic *sic)

If present, this function will be called when the module is loaded. - Function: void module_finish (Sic *sic)

If supplied, this function will be called just before the module is unloaded. - Variable: const Syntax * syntax_table

An array of syntactically significant symbols, and associated handler functions. - Function: int syntax_init (Sic *sic)

If specified, this function will be called by Sic before the syntax of each input line is analysed. - Function: int syntax_finish (Sic *sic, BufferIn *in, BufferOut *out)

Similarly, this function will be call after the syntax analysis of each line has completed.

All of the hard work in locating and loading the module, and extracting addresses for the symbols described above is performed by libltdl. The module_load function below simply registers these symbols with the Sic interpreter so that they are called at the appropriate times - or diagnoses any errors if things don't go according to plan:

int module_load (Sic *sic, const char *name) { lt_dlhandle module; Builtin *builtin_table; Syntax *syntax_table; int status = SIC_OKAY; last_error = NULL; module = lt_dlopenext (name); if (module) { builtin_table = (Builtin*) lt_dlsym (module, "builtin_table"); syntax_table = (Syntax *) lt_dlsym (module, "syntax_table"); if (!builtin_table && !syntax_table) { lt_dlclose (module); last_error = no_builtin_table_error; module = NULL; } } if (module) { ModuleInit *init_func = (ModuleInit *) lt_dlsym (module, "module_init"); if (init_func) (*init_func) (sic); } if (module) { SyntaxFinish *syntax_finish = (SyntaxFinish *) lt_dlsym (module, "syntax_finish"); SyntaxInit *syntax_init = (SyntaxInit *) lt_dlsym (module, "syntax_init"); if (syntax_finish) sic->syntax_finish = list_cons (list_new (syntax_finish), sic->syntax_finish); if (syntax_init) sic->syntax_init = list_cons (list_new (syntax_init), sic->syntax_init); } if (module) { if (builtin_table) status = builtin_install (sic, builtin_table); if (syntax_table && status == SIC_OKAY) status = syntax_install (sic, module, syntax_table); return status; } last_error = lt_dlerror(); if (!last_error) last_error = module_not_found_error; return SIC_ERROR; }

Notice that the generalised List data type introduced earlier (see the chapter called A Small GNU Autotools Project) is reused to keep a list of accumulated module initialisation and finalisation functions.

Unloading a Module

When unloading a module, several things must be done:

• Any built-in commands implemented by this module must be unregistered so that Sic doesn't try to call them after the implementation has been removed.

• Any syntax extensions implemented by this module must be similarly unregistered, including syntax_init and syntax_finish functions.

• If there is a finalisation entry point in the module, module_finish (see the section called Loading a Module), it must be called.

My first cut implementation of a module subsystem kept a list of the entry points associated with each module so that they could be looked up and removed when the module was subsequently unloaded. It also kept track of multiply loaded modules so that a module wasn't unloaded prematurely. libltdl already does all of this though, and it is wasteful to duplicate all of that work. This system uses lt_dlforeach and lt_dlgetinfo to access libltdls records of loaded modules, and save on duplication. These two functions are described fully inLibltdl interface: (Libtool)Libltdl interface.

static int unload_ltmodule (lt_dlhandle module, lt_ptr_t data); struct unload_data { Sic *sic; const char *name; }; int module_unload (Sic *sic, const char *name) { struct unload_data data; last_error = NULL; data.sic = sic; data.name = name; /* Stopping might be an error, or we may have unloaded the module. */ if (lt_dlforeach (unload_ltmodule, (lt_ptr_t) &data) != 0) if (!last_error) return SIC_OKAY; if (!last_error) last_error = module_not_found_error; return SIC_ERROR; }

This function asks libltdl to call the function unload_ltmodule for each of the modules it has loaded, along with some details of the module it wants to unload. The tricky part of the callback function below is recalculating the ntry point addresses for the module to be unloaded and then removing all matching addresses from the appropriate internal structures. Otherwise, the balance of this callback is involved in informing the calling lt_dlforeach loop of whether a matching module has been found and handled:

static int userdata_address_compare (List *elt, void *match); /* This callback returns 0 if the module was not yet found. If there is an error, LAST_ERROR will be set, otherwise the module was successfully unloaded. */ static int unload_ltmodule (lt_dlhandle module, void *data) { struct unload_data *unload = (struct unload_data *) data; const lt_dlinfo *module_info = lt_dlgetinfo (module); if ((unload == NULL) || (unload->name == NULL) || (module_info == NULL) || (module_info->name == NULL) || (strcmp (module_info->name, unload->name) != 0)) { /* No match, return 0 to keep searching */ return 0; } if (module) { /* Fetch the addresses of the entrypoints into the module. */ Builtin *builtin_table = (Builtin*) lt_dlsym (module, "builtin_table"); Syntax *syntax_table = (Syntax *) lt_dlsym (module, "syntax_table"); void *syntax_init_address = (void *) lt_dlsym (module, "syntax_init"); void **syntax_finish_address = (void *) lt_dlsym (module, "syntax_finish"); List *stale; /* Remove all references to these entry points in the internal data structures, before actually unloading the module. */ stale = list_remove (&unload->sic->syntax_init, syntax_init_address, userdata_address_compare); XFREE (stale); stale = list_remove (&unload->sic->syntax_finish, syntax_finish_address, userdata_address_compare); XFREE (stale); if (builtin_table && builtin_remove (unload->sic, builtin_table) != SIC_OKAY) { last_error = builtin_unload_error; module = NULL; } if (syntax_table && SIC_OKAY != syntax_remove (unload->sic, module, syntax_table)) { last_error = syntax_unload_error; module = NULL; } } if (module) { ModuleFinish *finish_func = (ModuleFinish *) lt_dlsym (module, "module_finish"); if (finish_func) (*finish_func) (unload->sic); } if (module) { if (lt_dlclose (module) != 0) module = NULL; } /* No errors? Stop the search! */ if (module) return 1; /* Find a suitable diagnostic. */ if (!last_error) last_error = lt_dlerror(); if (!last_error) last_error = module_not_unloaded_error; /* Error diagnosed. Stop the search! */ return -1; } static int userdata_address_compare (List *elt, void *match) { return (int) (elt->userdata - match); }

The userdata_address_compare helper function at the end is used to compare the address of recalculated entry points against the already registered functions and handlers to find which items need to be unregistered.

There is also a matching header file to export the module interface, so that the code for loadable modules can make use of it:

#ifndef SIC_MODULE_H #define SIC_MODULE_H 1 #include <sic/builtin.h> #include <sic/common.h> #include <sic/sic.h> BEGIN_C_DECLS typedef void ModuleInit (Sic *sic); typedef void ModuleFinish (Sic *sic); extern const char *module_error (void); extern int module_init (void); extern int module_load (Sic *sic, const char *name); extern int module_unload (Sic *sic, const char *name); END_C_DECLS #endif /* !SIC_MODULE_H */

This header also includes some of the other Sic headers, so that in most cases, the source code for a module need only #include <sic/module.h>.

To make the module loading interface useful, I have added built-ins for load and unload. Naturally, these must be compiled into the bare sic executable, so that it is able to load additional modules:

#if HAVE_CONFIG_H # include <config.h> #endif #include "module.h" #include "sic_repl.h" /* List of built in functions. */ #define builtin_functions \ BUILTIN(exit, 0, 1) \ BUILTIN(load, 1, 1) \ BUILTIN(unload, 1, -1) BUILTIN_DECLARATION (load) { int status = SIC_ERROR; if (module_load (sic, argv[1]) < 0) { sic_result_clear (sic); sic_result_append (sic, "module \"", argv[1], "\" not loaded: ", module_error (), NULL); } else status = SIC_OKAY; return status; } BUILTIN_DECLARATION (unload) { int status = SIC_ERROR; int i; for (i = 1; argv[i]; ++i) if (module_unload (sic, argv[i]) != SIC_OKAY) { sic_result_clear (sic); sic_result_append (sic, "module \"", argv[1], "\" not unloaded: ", module_error (), NULL); } else status = SIC_OKAY; return status; }

These new built-in commands are simply wrappers around the module loading code in module.c.

As with dlopen, you can use libltdl to lt_dlopen the main executable, and then lookup its symbols. I have simplified the initialisation of Sic by replacing the sic_init function in src/sic.c by `loading' the executable itself as a module. This works because I was careful to use the same format in sic_builtin.c and sic_syntax.c as would be required for a genuine loadable module, like so:

/* initialise the module subsystem */ if (module_init () != SIC_OKAY) sic_fatal ("module initialisation failed"); if (module_load (sic, NULL) != SIC_OKAY) sic_fatal ("sic initialisation failed");