17 Modules and Packages

Despite its central role in the implementation of modules in Lua, require is a regular function, with no special privileges. To load a module, we simply call it with a single argument, the module name. Remember that, when the single argument to a function is a literal string, the parentheses are optional, and it is customary to omit them in regular uses of require. Nevertheless, the following uses are all correct, too:

      local m = require('math')
      
      local modname = 'math'
      local m = require(modname)

The function require tries to keep to a minimum its assumptions about what a module is. For it, a module is just any code that defines some values, such as functions or tables containing functions. Typically, that code returns a table comprising the module functions. However, because this action is done by the module code, not by require, some modules may choose to return other values or even to have side effects (e.g., by creating global variables).

The first step of require is to check in the table package.loaded whether the module is already loaded. If so, require returns its corresponding value. Therefore, once a module is loaded, other calls requiring the same module simply return the same value, without running any code again.

If the module is not loaded yet, require searches for a Lua file with the module name. (This search is guided by the variable package.path, which we will discuss later.) If it finds such a file, it loads it with loadfile. The result is a function that we call a loader. (The loader is a function that, when called, loads the module.)

If require cannot find a Lua file with the module name, it searches for a C library with that name.^[17] (In that case, the search is guided by the variable package.cpath.) If it finds a C library, it loads it with the low-level function package.loadlib, looking for a function called luaopen_modname. The loader in this case is the result of loadlib, which is the C function luaopen_modname represented as a Lua function.

No matter whether the module was found in a Lua file or a C library, require now has a loader for it. To finally load the module, require calls the loader with two arguments: the module name and the name of the file where it got the loader. (Most modules just ignore these arguments.) If the loader returns any value, require returns this value and stores it in the package.loaded table, to return the same value in future calls for this same module. If the loader returns no value, and the table entry package.loaded[@rep{modname}] is still empty, require behaves as if the module returned true. Without this correction, a subsequent call to require would run the module again.

To force require into loading the same module twice, we can erase the library entry from package.loaded:

      package.loaded.modname = nil

The next time the module is required, require will do all its work again.

A common complaint against require is that it cannot pass arguments to the module being loaded. For instance, the mathematical module might have an option for choosing between degrees and radians:

      -- bad code
      local math = require("math", "degree")

The problem here is that one of the main goals of require is to avoid loading a module multiple times. Once a module is loaded, it will be reused by whatever part of the program that requires it again. There would be a conflict if the same module were required with different parameters. In case you really want your module to have parameters, it is better to create an explicit function to set them, like here:

      local mod = require "mod"
      mod.init(0, 0)

If the initialization function returns the module itself, we can write that code like this:

      local mod = require "mod".init(0, 0)

In any case, remember that the module itself is loaded only once; it is up to it to handle conflicting initializations.

      ?;?.lua;c:\windows\?;/usr/local/lua/?/?.lua

      sql
      sql.lua
      c:\windows\sql
      /usr/local/lua/sql/sql.lua

      ./?.so;/usr/local/lib/lua/5.2/?.so

      .\?.dll;C:\Program Files\Lua502\dll\?.dll

      > path = ".\\?.dll;C:\\Program Files\\Lua502\\dll\\?.dll"
      > print(package.searchpath("X", path))
      nil
              no file '.\X.dll'
              no file 'C:\Program Files\Lua502\dll\X.dll'

      function search (modname, path)
        modname = string.gsub(modname, "%.", "/")
        local msg = {}
        for c in string.gmatch(path, "[^;]+") do
          local fname = string.gsub(c, "?", modname)
          local f = io.open(fname)
          if f then
            f:close()
            return fname
          else
            msg[#msg + 1] = string.format("\n\tno file '%s'", fname);
          end
        end
        return nil, table.concat(msg)       -- not found
      end

The simplest way to create a module in Lua is really simple: we create a table, put all functions we want to export inside it, and return this table. Figure 17.2, “A simple module for complex numbers” illustrates this approach.

      local M = {}         -- the module
      
      -- creates a new complex number
      local function new (r, i)
        return {r=r, i=i}
      end
      
      M.new = new        -- add 'new' to the module
      
      -- constant 'i'
      M.i = new(0, 1)
      
      function M.add (c1, c2)
        return new(c1.r + c2.r, c1.i + c2.i)
      end
      
      function M.sub (c1, c2)
        return new(c1.r - c2.r, c1.i - c2.i)
      end
      
      function M.mul (c1, c2)
        return new(c1.r*c2.r - c1.i*c2.i, c1.r*c2.i + c1.i*c2.r)
      end
      
      local function inv (c)
        local n = c.r^2 + c.i^2
        return new(c.r/n, -c.i/n)
      end
      
      function M.div (c1, c2)
        return M.mul(c1, inv(c2))
      end
      
      function M.tostring (c)
        return string.format("(%g,%g)", c.r, c.i)
      end
      
      return M

Note how we define new and inv as private functions simply by declaring them local to the chunk.

Some people do not like the final return statement. One way of eliminating it is to assign the module table directly into package.loaded:

      local M = {}
      package.loaded[...] = M
        as before, without the return statement

Remember that require calls the loader passing the module name as the first argument. So, the vararg expression ... in the table index results in that name. After this assignment, we do not need to return M at the end of the module: if a module does not return a value, require will return the current value of package.loaded[modname] (if it is not nil). Anyway, I find it clearer to write the final return. If we forget it, any trivial test with the module will detect the error.

Another approach to write a module is to define all functions as locals and build the returning table at the end, as in Figure 17.3, “Module with export list”.

      local function new (r, i) return {r=r, i=i} end
      
      -- defines constant 'i'
      local i = complex.new(0, 1)
      
        other functions follow the same pattern
      
      return {
        new      = new,
        i        = i,
        add      = add,
        sub      = sub,
        mul      = mul,
        div      = div,
        tostring = tostring,
      }

What are the advantages of this approach? We do not need to prefix each name with M. or something similar; there is an explicit export list; and we define and use exported and internal functions in the same way inside the module. What are the disadvantages? The export list is at the end of the module instead of at the beginning, where it would be more useful as a quick documentation; and the export list is somewhat redundant, as we must write each name twice. (This last disadvantage may become an advantage, as it allows functions to have different names inside and outside the module, but I think programmers seldom do this.)

Anyway, remember that no matter how we define a module, users should be able to use it in a standard way:

      local cpx = require "complex"
      print(cpx.tostring(cpx.add(cpx.new(3,4), cpx.i)))
        --> (3,5)

Later, we will see how we can use some advanced Lua features, such as metatables and environments, for writing modules. However, except for a nice technique to detect global variables created by mistake, I use only the basic approach in my modules.

Lua allows module names to be hierarchical, using a dot to separate name levels. For instance, a module named mod.sub is a submodule of mod. A package is a complete tree of modules; it is the unit of distribution in Lua.

When we require a module called mod.sub, the function require will query first the table package.loaded and then the table package.preload, using the original module name "mod.sub" as the key. Here, the dot is just a character like any other in the module name.

However, when searching for a file that defines that submodule, require translates the dot into another character, usually the system’s directory separator (e.g., a slash for POSIX or a backslash for Windows). After the translation, require searches for the resulting name like any other name. For instance, assume the slash as the directory separator and the following path:

      ./?.lua;/usr/local/lua/?.lua;/usr/local/lua/?/init.lua

The call require "a.b" will try to open the following files:

      ./a/b.lua
      /usr/local/lua/a/b.lua
      /usr/local/lua/a/b/init.lua

This behavior allows all modules of a package to live in a single directory. For instance, if a package has modules p, p.a, and p.b, their respective files can be p/init.lua, p/a.lua, and p/b.lua, with the directory p within some appropriate directory.

The directory separator used by Lua is configured at compile time and can be any string (remember, Lua knows nothing about directories). For instance, systems without hierarchical directories can use an underscore as the “directory separator”, so that require "a.b" will search for a file a_b.lua.

Names in C cannot contain dots, so a C library for submodule a.b cannot export a function luaopen_a.b. Here, require translates the dot into another character, an underscore. So, a C library named a.b should name its initialization function luaopen_a_b.

As an extra facility, require has one more searcher for loading C submodules. When it cannot find either a Lua file or a C file for a submodule, this last searcher searches again the C path, but this time looking for the package name. For example, if the program requires a submodule a.b.c this searcher will look for a. If it finds a C library for this name, then require looks into this library for an appropriate open function, luaopen_a_b_c in this example. This facility allows a distribution to put several submodules together, each with its own open function, into a single C library.

From the point of view of Lua, submodules in the same package have no explicit relationship. Requiring a module does not automatically load any of its submodules; similarly, requiring a submodule does not automatically load its parent. Of course, the package implementer is free to create these links if she wants. For instance, a particular module may start by explicitly requiring one or all of its submodules.