There was one pain point during the development of gotobed, errors were not actionable. What I was missing was a stacktrace that would tell me what was going on when an error happened.

So that’s what I implemented recently in plugin handlers tracebacks and plugin loaders tracebacks.

Now raising an error in a plugin will show a stacktrace when debug mode is on. Nice.

The implementation was as easy as switching from protected calls (pcall) to extended (?) protected calls (xpcall). This effort though sparked some interest in me about sandboxing the plugins. In the end I did not implement this but here’s some thoughts for others attemping the same feat.

Lua has first-class support for function environments. Using getfenv and setfevn it is possible to change the bindings to global variables and free bindings in a function scope.

Here’s an example.

function foo ()
    bar()
end

function bar ()
    print("bar")
end

function baz ()
    print("baz")
end

foo() -- outputs "bar"
setfenv(foo, { bar = baz })
foo() -- outputs "baz"

This does not work with functions implemented in C, .e.g require. This means that it is not possible to change the bindings to globals for C functions. If you try to, you get an error.

> setfenv(require, { package = {}})
stdin:1: 'setfenv' cannot change environment of given object
stack traceback:
	[C]: in function 'setfenv'
	stdin:1: in main chunk
	[C]: ?

This problem is exactly where I had landed. I want to load plugins by executing a lua script for each plugin, but each plugin comes with additional scripts that it may load dynamically. To make this possible KOReader modifies the package.path value to also include the plugin directory at runtime. Then after loading the plugin it resets the value of package.path.

This is an example of changing the package.path value to resolve a module in a custom directory.

> local MyModule = require ("my_module")
stdin:1: module 'my_module' not found:
	no field package.preload['my_module']
	no file './my_module.lua'
	no file '/usr/local/share/lua/5.1/my_module.lua'
	no file '/usr/local/share/lua/5.1/my_module/init.lua'
	no file '/usr/local/lib64/lua/5.1/my_module.lua'
	no file '/usr/local/lib64/lua/5.1/my_module/init.lua'
	no file '/usr/share/lua/5.1/my_module.lua'
	no file '/usr/share/lua/5.1/my_module/init.lua'
	no file './my_module.so'
	no file '/usr/local/lib64/lua/5.1/my_module.so'
	no file '/usr/lib64/lua/5.1/my_module.so'
	no file '/usr/local/lib64/lua/5.1/loadall.so'
stack traceback:
	[C]: in function 'require'
	stdin:1: in main chunk
	[C]: ?
> package.path = "path/to/the/module/directory/?.lua;" .. package.path
> local MyModule = require("my_module")

So my solution was as follows. Create a function environment for each plugin, change the package table so that package.path and package.cpath contain the plugin directory. Load the script as a chunk, set the function environment to my custom one and execute the chunk. This is sensible but does not work.

Again an example for you to follow.

local plugin_env = {
    pacakge = {
        path = "path/to/the/module/directory/?.lua",
        ...
    }
}
function foo()
    print(package.path)
    local MyModule = require ("my_module")
end
setfenv(foo, plugin_env)
foo()
-- outputs "path/to/the/module/directory"
-- stdin:1: module 'my_module' not found:
-- 	no field package.preload['my_module']
-- 	no file './my_module.lua'
-- 	no file '/usr/local/share/lua/5.1/my_module.lua'
-- 	no file '/usr/local/share/lua/5.1/my_module/init.lua'
-- 	no file '/usr/local/lib64/lua/5.1/my_module.lua'
-- 	no file '/usr/local/lib64/lua/5.1/my_module/init.lua'
-- 	no file '/usr/share/lua/5.1/my_module.lua'
-- 	no file '/usr/share/lua/5.1/my_module/init.lua'
-- 	no file './my_module.so'
-- 	no file '/usr/local/lib64/lua/5.1/my_module.so'
-- 	no file '/usr/lib64/lua/5.1/my_module.so'
-- 	no file '/usr/local/lib64/lua/5.1/loadall.so'
-- stack traceback:
-- 	[C]: in function 'require'
-- 	stdin:1: in main chunk
-- 	[C]: ?

What did not work? require is implemented in C. setfenv has no effect on it so it is not using the new bindings to package.path.

Now you have two choices. The first one is to trust the plugins, share the global environment with them and get on with your life. Modify package.path before loading the plugin so that it can find the modules it needs and move on with life. This is the current implementation in KOReader.

But this solution has one caveat. The plugins cannot dynamically require modules. Everything must be declared at the plugin top-level. Before executing a plugin main.lua file package.path is set by KOReader to search in the plugin directory. After execution package.path is restored to the initial value. Any require call happening after the plugin returns will not work as expected.

Again example.

-- the plugin we want to load
local dbg = require ("debug") -- this require is executed as plugin load-time 
Plugin = { mt = {}}
function handleEvent (e)
    local log = require ("log") -- this require is executed at plugin runtime
    log.info ("we should really do something nice for the user here")
end

-- the application we load the plugin into

-- this is what a plugin looks like to the application
function plugin_module ()
    local t = {}
    return setmetatable(t, Plugin.mt)
end

function app ()
    local package_path = package.path -- save the value of package.path before
plugin load
    package.path = "path/to/the/plugin/directory"
    local plugin = plugin_module()
    package.path = package_path -- restore package.path after plugin load
    plugin.handleEvent(nil)
end

We can restore the correct behavior. We will need both a function environment for our plugin and a loader.

TODO dynamic require does not work a new global environment between the scripts provided by plugins and

require and setfenv do not mix. You got to write your own loaders.