Haskell, Lua, and Fennel

I find Haskell a fantastic language for almost all of the programming I want to do: a (reasonably) expressive type system, a (reasonably) good library ecosystem, and (reasonably) good tooling together give me a very satisfying local maximum for getting stuff done. I can’t see myself giving up libraries for a more powerful type system, nor giving up Haskell’s guarantees for a larger library ecosystem.

Scripting a larger program is one of the few areas where Haskell struggles. Despite some very impressive efforts like dyre, I think it’s a bit much to require a working Haskell toolchain and a “dump state, exec, load state” cycle just to make a program scriptable. This post discusses why Lua is a great scripting runtime for compiled programs, its shortcomings as a scripting language, how Fennel addresses many of these shortcomings, and demonstrates a Haskell program calling Fennel code which calls back into Haskell functions.

Lua

Lua is a weakly-typed imperative programming language designed to be embedded into larger programs. It has a lot of attractive features:

• It’s written in the common subset of ANSI C and C++, so it compiles on nearly anything;
• Its C API is quite simple, so it’s not too hard to drive from the host program;
• Hot code loading allows for interactive tweaks to a running program;
• It runs reasonably quickly;
• A fresh runtime has almost nothing installed in it, so you have a fighting chance of hardening it to make a “safe” sandbox (though it’s still pretty hard); and
• It has pretty good semantics, including lexical scoping and tail-call optimisation.

Many of these features are inherent to the runtime and not the language. Which is useful, because the language has some undesirable features:

• If you forget to write local when assigning a variable, you set a global variable;
• There is nothing like destructuring/pattern-matching in the language;
• Variables are mutable by default;
• Sequences are indexed starting at 1; and
• It is an imperative language, which is an unintuitive paradigm for solving real-world problems.

Fennel

Fennel is a Lisp which compiles to Lua, and draws some syntactic inspiration from Clojure. For example, the classic factorial function in Fennel:

(fn factorial [n]
  (match n
     0 1
     _ (* n (factorial (- n 1)))))

Would compile to this Lua code:

local function factorial(n)
  local _1_ = n
  if (_1_ == 0) then
    return 1
  elseif true then
    local _ = _1_
    return (n * factorial((n - 1)))
  else
    return nil
  end
end
return factorial

The language has been designed to smoothly interoperate with existing Lua code, while also providing convenience features you’d expect from a Lisp (destructuring binds, macros, etc.).

The compiler is provided in two forms: an ahead-of-time compiler which translates .fnl files to .lua; and a runtime compiler that can hook itself into Lua’s package search mechanism.

HsLua

HsLua (old site) is a fully-featured set of Haskell bindings to Lua. The most recent versions bundle Lua 5.4, so it is both mature and up-to-date. The lua package implements low-level FFI bindings to Lua’s C API, but the hslua package is probably the one you want. It provides idiomatic wrappers for the low-level functions as well as re-exports from all the other hslua-* packages, creating an all-in-one import for most common cases. (Use Hoogle to find out which package actually defines a function or type.)

Putting it all Together

Our goal is to put these pieces together in a way that demonstrates how a larger program might use an embedded interpreter: a Haskell program with a Lua runtime which can load Fennel files, calling back into Haskell functions. Almost all of our work will be performed inside a Lua monad, which creates and destroys an interpreter for us.

The first task is to implement a Lua module in Haskell. Since computing large factorials is the only thing Haskell is any good at, let’s export that capability to Lua:

import qualified HsLua as L

-- | The 'L.DocumentedFunction' machinery is from "hslua-packaging";
-- we can provide to Lua any function returning @'LuaE' e a@, so long
-- as we can provide a 'Peeker' for each argument and a 'Pusher' for
-- each result.
factorial :: L.DocumentedFunction e
factorial =
  L.defun "factorial"
    ### L.liftPure (\n -> product [1 .. n])
    <#> L.integralParam "n" "input number"
    =#> L.integralResult "factorial of n"
    #? "Computes the factorial of an integer."
    `L.since` makeVersion [1, 0, 0]

-- | Also using "hslua-packaging", this registers our
-- (single-function) module into Lua's @package.preload@ table,
-- setting things up such that the first time
-- @require('my-haskell-module')@ is called, the module will be
-- assembled, stored in @package.loaded['my-haskell-module']@ and
-- returned.
--
-- This lazy loading can help with the startup time of larger programs.
--
-- /See:/ http://www.lua.org/manual/5.4/manual.html#pdf-require
registerHaskellModule :: Lua ()
registerHaskellModule =
  L.preloadModule
    L.Module
      { L.moduleName = "my-haskell-module",
        L.moduleDescription = "Functions from Haskell",
        L.moduleFields = [],
        L.moduleFunctions = [factorial],
        L.moduleOperations = []
      }

To add Fennel to our Lua runtime, we need to download and unpack a Fennel tarball, use the file-embed library to store fennel.lua inside our Haskell binary (it is a mere 200K, less if compressed), load it, and install it:

fennelLua :: ByteString
fennelLua = $(embedFile "fennel.lua")

-- | Load our embedded copy of @fennel.lua@ and register it in
-- @package.searchers@.
registerFennel :: Lua ()
registerFennel = do
  L.preloadhs "fennel" $ L.NumResults 1 <$ L.dostring fennelLua

  -- It's often easier to run small strings of Lua code than to
  -- manipulate the runtime's stack with the C API.
  void $ L.dostring "require('fennel').install()"

registerFennel :: Lua ()
registerFennel = do
  L.preloadhs "fennel" $ L.NumResults 1 <$ L.dostring fennelLua

  void $
    L.dostring
      "local fennel = require('fennel');\
      \table.insert(package.searchers, fennel.searcher)"

We also need some Fennel code to run. We want to be able to change which factorials we compute without rebuilding all the Haskell, so fennel-demo.fnl imports our Haskell module and builds a table containing a sequence of factorials:

(local hs (require :my-haskell-module))

(local factorials [])
(for [i 1 10]
  (table.insert factorials (hs.factorial i)))

{ :factorials factorials }

main is all that’s left. Populate a Lua runtime and ask it for our value, then bring it across to Haskell and print it out:

main :: IO ()
main = do
  luaRes <- L.run $ do
    L.openlibs -- Add Lua's (tiny) standard library
    registerHaskellModule
    registerFennel

    -- Call into our fennel module and return a value from it
    L.dostring "local f = require('fennel-demo'); return f.factorials"

    -- From hslua-classes: use the Peekable typeclass to unmarshal and
    -- pop the sequence left on the stack
    L.popValue

  print (luaRes :: [Int])

If you want to see it running for yourself, the code is at https://git.sr.ht/~jack/hslua-fennel-demo .

Final Thoughts

I’d previously played around with embedding Lua as a scripting language in my MudCore project, but the limitations of the language made me disinclined to actually build something on top of the core. Fennel is an interesting little language that’s a lot more appealing to me, and I’m keen to find a use for it to write some scriptable Haskell programs.

I’m also pretty impressed by the thought that’s gone into the HsLua libraries: there are a lot of facilities that make the language boundary fairly convenient to cross, and it doesn’t take long to get a sense of which package actually provides the tool you’re looking for.