29 Calling C from Lua

As a first example, let us see how to implement a simplified version of a function that returns the sine of a given number:

      static int l_sin (lua_State *L) {
        double d = lua_tonumber(L, 1);  /* get argument */
        lua_pushnumber(L, sin(d));  /* push result */
        return 1;  /* number of results */
      }

Any function registered with Lua must have this same prototype, defined in lua.h as lua_CFunction:

      typedef int (*lua_CFunction) (lua_State *L);

From the point of view of C, a C function gets as its single argument the Lua state and returns an integer with the number of values it is returning on the stack. Therefore, the function does not need to clear the stack before pushing its results. After it returns, Lua automatically saves its results and clears its entire stack.

Before we can call this function from Lua, we must register it. We do this bit of magic with lua_pushcfunction: it gets a pointer to a C function and creates a value of type "function" that represents this function inside Lua. Once registered, a C function behaves like any other function inside Lua.

A quick-and-dirty way to test our function l_sin is to put its code directly into our basic interpreter (Figure 27.1, “A bare-bones stand-alone Lua interpreter”) and add the following lines right after the call to luaL_openlibs:

        lua_pushcfunction(L, l_sin);
        lua_setglobal(L, "mysin");

The first line pushes a value of type function; the second line assigns it to the global variable mysin. After these modifications, we can use the new function mysin in our Lua scripts. In the next section, we will discuss better ways to link new C functions with Lua. Here, we will explore how to write better C functions.

For a more professional sine function, we must check the type of its argument. The auxiliary library helps us with this task. The function luaL_checknumber checks whether a given argument is a number: in case of error, it throws an informative error message; otherwise, it returns the number. The modification to our function is minimal:

      static int l_sin (lua_State *L) {
        double d = luaL_checknumber(L, 1);
        lua_pushnumber(L, sin(d));
        return 1;  /* number of results */
      }

With the above definition, if you call mysin('a'), you get an error like this one:

      bad argument #1 to 'mysin' (number expected, got string)

The function luaL_checknumber automatically fills the message with the argument number (#1), the function name ("mysin"), the expected parameter type (number), and the actual parameter type (string).

As a more complex example, let us write a function that returns the contents of a given directory. Lua does not provide this function in its standard libraries, because ISO C does not offer functions for this job. Here, we will assume that we have a POSIX compliant system. Our function—we will call it dir in Lua, l_dir in C—gets as argument a string with the directory path and returns a list with the directory entries. For instance, a call like dir("/home/lua") may return the table {".", "..", "src", "bin", "lib"}. The complete code for this function is in Figure 29.1, “A function to read a directory”.

      #include <dirent.h>
      #include <errno.h>
      #include <string.h>
      
      #include "lua.h"
      #include "lauxlib.h"
      
      static int l_dir (lua_State *L) {
        DIR *dir;
        struct dirent *entry;
        int i;
        const char *path = luaL_checkstring(L, 1);
      
        /* open directory */
        dir = opendir(path);
        if (dir == NULL) {  /* error opening the directory? */
          lua_pushnil(L);  /* return nil... */
          lua_pushstring(L, strerror(errno));  /* and error message */
          return 2;  /* number of results */
        }
      
        /* create result table */
        lua_newtable(L);
        i = 1;
        while ((entry = readdir(dir)) != NULL) {  /* for each entry */
          lua_pushinteger(L, i++);  /* push key */
          lua_pushstring(L, entry->d_name);  /* push value */
          lua_settable(L, -3);    /* table[i] = entry name */
        }
      
        closedir(dir);
        return 1;  /* table is already on top */
      }

It starts getting the directory path with luaL_checkstring, which is the equivalent of luaL_checknumber for strings. Then it opens this directory with opendir. In case it cannot open the directory, the function returns nil plus an error message that it gets with strerror. After opening the directory, the function creates a new table and populates it with the directory entries. (Each time we call readdir, it returns a next entry.) Finally, it closes the directory and returns 1, in C, meaning that it is returning the value on top of its stack to Lua. (Remember that lua_settable pops the key and the value from the stack. Therefore, after the loop, the element on the top of the stack is the result table.)

In some conditions, this implementation of l_dir may cause a memory leak. Three of the Lua functions that it calls can fail due to insufficient memory: lua_newtable, lua_pushstring, and lua_settable. If any of these functions fails, it will raise an error and interrupt l_dir, which therefore will not call closedir. In Chapter 32, Managing Resources, we will see an alternative implementation for a directory function that corrects this problem.

Through lua_pcall and lua_call, a C function called from Lua can call Lua back. Several functions in the standard library do that: table.sort can call an order function; string.gsub can call a replacement function; pcall and xpcall call functions in protected mode. If we remember that the main Lua code was itself called from C (the host program), we have a call sequence like C (host) calls Lua (script) that calls C (library) that calls Lua (callback).

Usually, Lua handles these sequences of calls without problems; after all, this integration with C is a hallmark of the language. There is one situation, however, where this interlacement can cause difficulties: coroutines.

Each coroutine in Lua has its own stack, which keeps information about the pending calls of the coroutine. Specifically, the stack stores the return address, the parameters, and the local variables of each call. For calls to Lua functions, the interpreter needs only this stack, which we call the soft stack. For calls to C functions, however, the interpreter must use the C stack, too. After all, the return address and the local variables of a C function live in the C stack.

It is easy for the interpreter to have multiple soft stacks, but the runtime of ISO C has only one internal stack. Therefore, coroutines in Lua cannot suspend the execution of a C function: if there is a C function in the call path from a resume to its respective yield, Lua cannot save the state of that C function to restore it in the next resume. Consider the next example, in Lua 5.1:

      co = coroutine.wrap(function ()
                            print(pcall(coroutine.yield))
                          end)
      co()
        --> false    attempt to yield across metamethod/C-call boundary

The function pcall is a C function; therefore, Lua 5.1 cannot suspend it, because there is no way in ISO C to suspend a C function and resume it later.

Lua 5.2 and later versions ameliorated that difficulty with continuations. Lua 5.2 implements yields using long jumps, in the same way that it implements errors. A long jump simply throws away any information about C functions in the C stack, so it is impossible to resume those functions. However, a C function foo can specify a continuation function foo_k, which is another C function to be called when it is time to resume foo. That is, when the interpreter detects that it should resume foo, but that a long jump threw away the entry for foo in the C stack, it calls foo_k instead.

To make things a little more concrete, let us see the implementation of pcall as an example. In Lua 5.1, this function had the following code:

      static int luaB_pcall (lua_State *L) {
        int status;
        luaL_checkany(L, 1);  /* at least one parameter */
        status = lua_pcall(L, lua_gettop(L) - 1, LUA_MULTRET, 0);
        lua_pushboolean(L, (status == LUA_OK));  /* status */
        lua_insert(L, 1);  /* status is first result */
        return lua_gettop(L);  /* return status + all results */
      }

If the function being called through lua_pcall yielded, it would be impossible to resume luaB_pcall later. Therefore, the interpreter raised an error whenever we attempted to yield inside a protected call. Lua 5.3 implements pcall roughly like in Figure 29.2, “Implementation of pcall with continuations”.^[29]

      static int finishpcall (lua_State *L, int status, intptr_t ctx) {
        (void)ctx;   /* unused parameter */
        status = (status != LUA_OK && status != LUA_YIELD);
        lua_pushboolean(L, (status == 0));  /* status */
        lua_insert(L, 1);  /* status is first result */
        return lua_gettop(L);  /* return status + all results */
      }
      
      static int luaB_pcall (lua_State *L) {
        int status;
        luaL_checkany(L, 1);
        status = lua_pcallk(L, lua_gettop(L) - 1, LUA_MULTRET, 0,
                            0, finishpcall);
        return finishpcall(L, status, 0);
      }

There are three important differences from the Lua 5.1 version: first, the new version replaces the call to lua_pcall by a call to lua_pcallk; second, it puts everything done after that call in a new auxiliary function finishpcall; third, the status returned by lua_pcallk can be LUA_YIELD, besides LUA_OK or an error.

If there are no yields, lua_pcallk works exactly like lua_pcall. If there is a yield, however, then things are quite different. If a function called by the original lua_pcall tries to yield, Lua 5.3 raises an error, like Lua 5.1. But when a function called by the new lua_pcallk yields, there is no error: Lua does a long jump and discards the entry for luaB_pcall from the C stack, but keeps in the soft stack of the coroutine a reference to the continuation function given to lua_pcallk (finishpcall, in our example). Later, when the interpreter detects that it should return to luaB_pcall (which is impossible), it instead calls the continuation function.

The continuation function finishpcall can also be called when there is an error. Unlike the original luaB_pcall, finishpcall cannot get the value returned by lua_pcallk. So, it gets this value as an extra parameter, status. When there are no errors, status is LUA_YIELD instead of LUA_OK, so that the continuation function can check how it is being called. In case of errors, status is the original error code.

Besides the status of the call, the continuation function also receives a context. The fifth parameter to lua_pcallk is an arbitrary integer that is passed as the last parameter to the continuation function. (The type of this parameter, intptr_t, allows pointers to be passed as context, too.) This value allows the original function to pass some arbitrary information directly to its continuation. (Our example does not use this facility.)

The continuation system of Lua 5.3 is an ingenious mechanism to support yields, but it is not a panacea. Some C functions would need to pass too much context to their continuations. Examples include table.sort, which uses the C stack for recursion, and string.gsub, which must keep track of captures and a buffer for its partial result. Although it is possible to rewrite them in a “yieldable” way, the gains do not seem to be worth the extra complexity and performance losses.

A Lua module is a chunk that defines several Lua functions and stores them in appropriate places, typically as entries in a table. A C module for Lua mimics this behavior. Besides the definition of its C functions, it must also define a special function that plays the role of the main chunk in a Lua library. This function should register all C functions of the module and store them in appropriate places, again typically as entries in a table. Like a Lua main chunk, it should also initialize anything else that needs initialization in the module.

Lua perceives C functions through this registration process. Once a C function is represented and stored in Lua, Lua calls it through a direct reference to its address (which is what we give to Lua when we register a function). In other words, Lua does not depend on a function name, package location, or visibility rules to call a function, once it is registered. Typically, a C module has one single public (extern) function, which is the function that opens the library. All other functions can be private, declared as static in C.

When we extend Lua with C functions, it is a good idea to design our code as a C module, even when we want to register only one C function: sooner or later (usually sooner) we will need other functions. As usual, the auxiliary library offers a helper function for this job. The macro luaL_newlib takes an array of C functions with their respective names and registers all of them inside a new table. As an example, suppose we want to create a library with the function l_dir that we defined earlier. First, we must define the library functions:

      static int l_dir (lua_State *L) {
         as before
      }

Next, we declare an array with all functions in the module with their respective names. This array has elements of type luaL_Reg, which is a structure with two fields: a function name (a string) and a function pointer.

      static const struct luaL_Reg mylib [] = {
        {"dir", l_dir},
        {NULL, NULL}  /* sentinel */
      };

In our example, there is only one function (l_dir) to declare. The last pair in the array is always {NULL, NULL}, to mark its end. Finally, we declare a main function, using luaL_newlib:

      int luaopen_mylib (lua_State *L) {
        luaL_newlib(L, mylib);
        return 1;
      }

The call to luaL_newlib creates a new table and fills it with the pairs name–function specified by the array mylib. When it returns, luaL_newlib leaves on the stack the new table wherein it opened the library. The function luaopen_mylib then returns 1 to return this table to Lua.

After finishing the library, we must link it to the interpreter. The most convenient way to do it is with the dynamic linking facility, if your Lua interpreter supports this facility. In this case, you must create a dynamic library with your code (mylib.dll in Windows, mylib.so in Linux-like systems) and put it somewhere in the C path. After these steps, you can load your library directly from Lua, with require:

      local mylib = require "mylib"

This call links the dynamic library mylib with Lua, finds the function luaopen_mylib, registers it as a C function, and calls it, opening the module. (This behavior explains why luaopen_mylib must have the same prototype as any other C function.)

The dynamic linker must know the name of the function luaopen_mylib in order to find it. It will always look for luaopen_ concatenated with the name of the module. Therefore, if our module is called mylib, that function should be called luaopen_mylib. (We discussed the details of this function name in Chapter 17, Modules and Packages.)

If your interpreter does not support dynamic linking, then you have to recompile Lua with your new library. Besides this recompilation, you need some way of telling the stand-alone interpreter that it should open this library when it opens a new state. A simple way to do this is to add luaopen_mylib into the list of standard libraries to be opened by luaL_openlibs, in the file linit.c.