Lua 5.4

[Falco Girgis]

Greetings! First of all, I'm a big fan of DreamShell as both a user who has enjoyed its features and as a developer who has a lot of respect for how ambitious and impressive the project and its codebase are! The module system is really very well architected, and I find it really impressive work too.

I'm also a fellow Lua developer and enthusiast and wanted to run something by you. I noticed that the Lua version included within DreamShell is only version 5.1, I believe. I'm not sure if you've ever considered upgrading or have had any interest in it. It's true that a few things might have to change, involving how the ENV variable works for sandboxed scripts, but I wanted to make you aware of a few things that I've found really beneficial with current Lua 5.4 and Dreamcast:

1) Performance
It's actually quite hard to find any good benchmarks between Lua versions that aren't just LuaJIT, but in my research, it seems as though Lua5.4 performs the best of the modern Lua revisions, and is definitely the faster than 5.2 and 5.3: https://eklausmeier.goip.de/blog/2020/05...-5-4-vs-c/

2) Builtin Integral Types
Starting with Lua5.3, Lua now supports integers as first-party types, so all numbers no longer have to be floats. This has performance and precision implications, but for us its also very useful for when you're binding low-level code that tends to require bitwise operation and arithmetic. Bitwise operators are now supported within the language itself: https://www.lua.org/manual/5.4/manual.html#3.4.2 and don't require external libraries or bindings to do it in C.

3) List of New Features
Here's a high-level list of new features for each revision from Lua's site which you might be interested in:

Lua 5.2:

• yieldable pcall and metamethods
• new lexical scheme for globals
• ephemeron tables
• light C functions
• emergency garbage collector
• goto statement
• finalizers for tables

Lua 5.3:

• integers (64-bit by default)
• official support for 32-bit numbers
• bitwise operators
• basic utf-8 support
• functions for packing and unpacking values
  

Lua 5.4:

• new generational mode for garbage collection
• to-be-closed variables
• const variables
• userdata can have multiple user values
• new implementation for math.random
• warning system
• debug information about function arguments and returns
• new semantics for the integer 'for' loop
• optional 'init' argument to 'string.gmatch'
• new functions 'lua_resetthread' and 'coroutine.close'
• string-to-number coercions moved to the string library
• allocation function allowed to fail when shrinking a memory block
• new format '%p' in 'string.format'
• utf8 library accepts codepoints up to 2^31
  

4) Emergency Garbage Collection Sweep
One thing that I think is really useful and cool for us on DC and embedded devices is the "emergency GC sweeping" that as introduced in Lua5.2. Now, if the device ever runs out of memory (malloc() call fails and returns NULL), Lua will gracefully attempt to free any unused memory before trying again.

KOS actually had a problem with this not working previously, because we were always aborting the system when we ran out of memory, but we just pushed a PR to stop doing that, so that malloc() will return NULL in C, C++ will raise its std::bad_alloc exceptions, Lua will do its emergency sweeps, etc. Here's the PR on the KOS repo with the change as well as a new KOS example I wrote illustrating gracefully handling allocation failure in C++: https://github.com/KallistiOS/KallistiOS...8e8693f9ea

Anyway, I wasn't sure if you had any compelling reasons to stick to Lua 5.1, or if you just hadn't really look into what the new revisions have to offer. Definitely a few things will have to change from 5.1 to 5.2, but virtually nothing has changed from 5.2 to 5.4 in terms of the C API and user libs. I do believe it could bring you runtime performance improvements and other benefits, but I'm not sure if perhaps the binary file size for the VM might be larger now.

Thank you for your time, and as always, please keep up the amazing work! I'm both a user and a fan!

[SWAT]

Hi! Nice to see you here.
No one has yet praised me for the modular system, it's worth a lot, thank you. It's especially nice to hear that from a good developer.
There is only one reason why I use version 5.1 - I started intergating lua in 2007 Then it was the last one. I would love to switch to a new one, but there is not enough time for all the Wishlist. So much more to do It wasn't a big priority for me. But maybe it's time to do it. I like lua, it's cool.