What's the alternative to a stack API? If you look at Python and Perl they use a combination of (1) an explicit stack similar to Lua, (2) instantiation of heavy-weight list or array objects for passing arguments, (3) explicit reference counting, and (4) code generation.
Lua's stack protocol can sometimes be verbose, but I find it infinitely more simple and elegant than the existing alternatives.
I think you tend to see long, unwieldy code blocks that invoke Lua functions multiple times because you can, despite how tricky it can be to track the top of your stack. You rarely see this in Perl or Python because it becomes impossible to manage long before that point. So the solution for Lua is the same as for other interfaces--use shorter, simpler functions and compose them. This is usually easier and more performant to do in Lua, anyhow, because there's less fixed boiler plate and indirection necessary; not to mention Lua gives you many more facilities for stashing and managing your C state across calls (like closure upvalues, userdata values, continuation context cookies, etc) without resorting to global variables either inside or outside the VM.
Yeah, implementing a precise GC is a pain, but in Go the hard work of that is already done for you. If I were to write my own Lua-in-Go, my inclination would be to try to make an API something like this
The above is shorter and also significantly more efficient because you're not creating the intermediate table object. Even in Go, given Lua's coroutine and tailcall semantics (assuming they were implemented) the Go compiler would likely be forced to always heap allocate the argument list table. There's a reason PUC Lua is blazingly fast, and its partly the same reason why LuaJIT is so fast--the Lua stack protocol is a thin abstraction that easily admits a simple, efficient implementation yet still managing to provide an effective interface boundary that doesn't leak (beyond the fact of the protocol itself).
I don't think there's any good middle ground here. The best options are (1) Lua's explicit stack protocol or (2) some sort of type inference that permitted a direct call, like lua_tcall(L, "five", 5), into the VM. You can actually implement the latter in C somewhat easily (at least if you stick to simple data types) by using _Generic and __VA_ARG__ macros to generate the stack pushing code.
But I rarely see this done because it's a lot of magic for little gain. And where I have seen it done it's always been annoying because it's too leaky--invariably you have to fallback to the stack pushing code because there's a limit to the types of automagic type coercions you can accomplish between two drastically different languages. So you have to learn their magical wrapper routines in addition to the regular Lua API.
Many years ago I abused __VA_ARG__, __builtin_types_compatible_p, and libffi so I could invoke ad hoc functions as libevent callbacks without having to proxy the call through a function that unwrapped a pointer-to-void cookie. See http://25thandclement.com/~william/projects/delegate.c.html I still think it's kinda clever but I can't remember the last time I actually used it. Even in code bases already using delegate.c I ended up returning to the slightly more repetitive but transparent and idiomatic patterns for installing libevent events.
rather than having to roll your own. Part of the function of the stack protocol is to minimize the number of unique functions needed to load and store values across language boundaries. So there's a single [low-level] function for communicating a number value in a type-safe manner, rather than a multitude of functions--one routine for setting a number value for a table string key, another for a number value for a table number key, another for a string value for a table number key, etc. The permutations explode, and unless the host language has some sort of generics capability then it adds significant complexity both to the interface and to implementation maintenance. (See Perl XS.) Alternatively you can reduce the number of permuted interfaces by using printf-style formatting strings and va_lists, but that's not type safe. (See Py_BuildValue.)
Notably, Lua 5.3 removed lua_pushunsigned, lua_tounsigned, luaL_checkint, luaL_checklong, and many similar routines as there was never an end to the number of precomposed type coercing, constraint checking auxiliary routines people demanded. They decided that the only way to win that game was to not play at all. But with C11 _Generic and a dash of compiler extension magic you can actually implement a richer set of type agnostic (but type safe) numeric load/store interfaces in just a handful of auxiliary routines. For better or worse, though, Lua doesn't target C11 nor wish to depend on compiler extensions. "Batteries not included" is a whole 'nother debate in the Lua universe.
> Lua's stack protocol can sometimes be verbose, but I find it infinitely more simple and elegant than the existing alternatives.
This. TL;DNR, used a C/Lua binding to explain the Lua stack to a fellow engineer. Took a few post-it notes to keep track of 4 levels of stack, 1 pcall back to Lua to sort an array, and forward the results on. It just worked, was elegant, and blew our minds.
Today on Friday at 5pm I was talking about a ‘problem’ we had with a Lua daemons compared to their C brothers. Due to the use of tables, each time you queried them, the output order was always random. Computers don’t care, but a big user is also engineers and finding fields wandering all over the place is a pita. So what if we ordered the keys alphabetically instead? The code is Lua -> C -> IPC -> Output. The order is set in the Lua->C binding, so what if we sorted it in the binding.
But the tables are arbitrary size and you’d have to allocate memory, put them in a structure of some kind, etc, etc. Then we realized we could just put all the keys in a Lua array, and sort them with table.sort(). We could then use the array to get key/values in order, all from within the C binding.
Being late Friday, and having talked it through let’s just do it. Plus it was a handy teaching moment to explain to a fellow engineer how Lua binding worked, including a call back into Lua. By 6pm we had the binding recompiled, tested and it’s just worked.
Lessons learned:
1. The stack can seem confusing, but once you learn how to work with it, it’s pretty amazing.
2. The Lua reference docs are amazing. They clearly list what they pop and what they push onto the stack.
3. Calling back into Lua is.
4. It looks like write only code. Reading the code needs a post it note to keep track of the stack, along with the Lua reference manual. -- comments...
Lua's stack protocol can sometimes be verbose, but I find it infinitely more simple and elegant than the existing alternatives.
I think you tend to see long, unwieldy code blocks that invoke Lua functions multiple times because you can, despite how tricky it can be to track the top of your stack. You rarely see this in Perl or Python because it becomes impossible to manage long before that point. So the solution for Lua is the same as for other interfaces--use shorter, simpler functions and compose them. This is usually easier and more performant to do in Lua, anyhow, because there's less fixed boiler plate and indirection necessary; not to mention Lua gives you many more facilities for stashing and managing your C state across calls (like closure upvalues, userdata values, continuation context cookies, etc) without resorting to global variables either inside or outside the VM.