Luau Recap: February 2020

[zeuxcg]

We continue to iterate on our language stack, working on many features for type checking, performance, and quality of life. Some of them come with announcements, some come with release notes, and some just ship - here we will talk about all things that happened since November last year.

A lot of people work on these improvements; thanks @Apakovtac, @EthicalRobot, @fun_enthusiast, @xyzzyismagic, @zeuxcg!

If you want to catch up on the improvements from last year, you probably want to read these as well!

• Faster Lua VM Released
• Luau Recap: November 2019

Type checking (beta)

We were originally intending to ship the beta last year but had to delay it due to last minute bugs. However, it’s now live as a beta option on production! Go here to learn more:

Luau Type Checking Beta!

EDIT: Please DO NOT publish places with type annotations just yet as they will not work on production! This is why it’s a beta However, please continue to experiment in Studio and give us feedback. We are reading everything and will be fixing reported bugs and discussing syntax / semantics issues some people brought up. Hello! We’ve been quietly working on building a type checker for Lua for quite some time now. It is now far enough along that we’d really like to hear what…

We’re continuing to iterate on the feedback we have received here. Something that will happen next is that we will enable type annotations on live server/clients - which will mean that you will be able to publish source code with type annotations without breaking your games. We still have work to do on the non-strict and strict mode type checking before the feature can move out of beta though, in particular we’ve implemented support for require statement and that should ship next week

We also fixed a few bugs in the type definitions for built-in functions/API and the type checker itself:

• table.concat was accidentally treating the arguments as required
• string.byte and string.find now have a correct precise type
• typeof comparisons in if condition incorrectly propagated the inferred type into elseif branches

We are also making the type checker more ergonomic and more correct. Two changes I want to call out are:

• Type aliases declared with type X = Y are now co-recursive, meaning that they can refer to each other, e.g.

type array<T> = { [number]: T }

type Wheel = { radius: number, car: Car }
type Car = { wheels: array<Wheel> }

• We now support type intersections (A & B) in addition to type unions (A | B). Intersections are critical to modeling overloaded functions correctly - while Lua as a language doesn’t support function overloads, we have various APIs that have complex overloaded semantics - one of them that people who tried the beta had problems with was UDim2.new - and it turns out that to correctly specify the type, we had to add support for intersections. This isn’t really intended as a feature that is used often in scripts developers write, but it’s important for internal use.

Debugger (beta)

When we shipped the original version of the VM last year, we didn’t have the debugger fully working. Debugger relies on low-level implementation of the old VM that we decided to remove from the new VM - as such we had to make a new low-level debugging engine.

This is now live under the Luau VM beta feature, see this post for details:

Luau in Studio beta!

Hi, everyone. In addition to all of the typechecking work that’s currently in beta, we’ve been hard at work on the other side of Luau, which is the faster runtime. The runtime is already used everywhere except in Studio debugging sessions, and we’ve recently released a beta feature called “Enable faster Lua runtime” which turns it on in Studio as well. Please try it out and tell us what you think. Ideally, the only thing you might notice is an improvement in runtime performance when debugging. …

If you use the debugger at all, please enable the beta feature and try it out - we want to fix all the bugs we can find, and this is blocking us enabling the new VM everywhere.

(a quick aside: today the new VM is enabled on all servers and all clients, and it’s enabled in Studio “edit” mode for plugins - but not in Studio play modes, and full debugger support is what prevents us from doing so)

Language

This section is short and sweet this time around:

• You can now use continue statement in for/while/repeat loops.

Please note that we only support this in the new VM, so you have to be enrolled in Luau VM beta to be able to use it in Studio. It will work in game regardless of the beta setting as per above.

Performance

While we have some really forward looking ideas around multi-threading and native code compilation that we’re starting to explore, we also continue to improve performance across the board based on our existing performance backlog and your feedback.

In particular, there are several memory and performance optimizations that shipped in the last few months:

• Checking for truth (if foo or foo and bar) is now a bit faster, giving 2-3% performance improvements on some benchmarks
• table.create (with value argument) and table.pack have been reimplemented and are ~1.5x faster than before
• Internal mechanism for filling arrays has been made faster as well, which makes Terrain:ReadVoxels ~10% faster
• Catching engine-generated errors with pcall/xpcall is now ~1.5x faster (this only affects performance of calls that generated errors)
• String objects now take 8 bytes less memory per object (and in an upcoming change we’ll save a further 4 bytes)
• Capturing local variables that are never assigned to in closures is now much faster, takes much less memory and generates much less GC pressure. This can make closure creation up to 2x faster, and improves some Roact benchmarks by 10%. This is live in Studio and will ship everywhere else shortly.
• The performance of various for loops (numeric & ipairs) on Windows regressed after a VS2017 upgrade; this regression has been fixed, making all types of loops perform roughly equally. VS2017 upgrade also improved Luau performance on Windows by ~10% across the board.
• Lua function calls have been optimized a bit more, gaining an extra 10% of performance in call-heavy benchmarks on Windows.
• Variadic table constructors weren’t compiled very efficiently, resulting in surprisingly low performance of constructs like {...}. Fixing that made {...} ~3x faster for a typical number of variadic arguments.

Diagnostics

We spent some time to improve error messages in various layers of the stack based on the reports from community. Specifically:

• The static analysis warning about incorrect bounds for numeric for loops is now putting squigglies in the right place.
• Fixed false positive static analysis warnings about unreachable code inside repeat…until loops in certain cases.
• Multiline table construction expressions have a more precise line information now which helps in debugging since callstacks are now easier to understand
• Incomplete statements (e.g. foo) now produce a more easily understandable parsing error
• In some cases when calling the method with a . instead of :, we emitted a confusing error message at runtime (e.g. humanoid.LoadAnimation(animation)). We now properly emit the error message asking the user if : was intended.
• The legacy global ypcall is now flagged as deprecated by script analysis
• If you use a Unicode symbol in your source program outside of comments or string literals, we now produce a much more clear message, for example:

local pi = 3․13 -- spoiler alert: this is not a dot!

produces Unexpected Unicode character: U+2024. Did you mean '.'?

LoadLibrary removal

Last but not least, let’s all press F for LoadLibrary:

LoadLibrary is going to be removed on February 3rd

UPDATE This update is live as of February 3rd, 2020. UPDATE This has been postponed once more to February 3rd. UPDATE This has been postponed to January 6th; the post has been updated with the new timeline. What’s going on? LoadLibrary is a legacy mechanism that we previously used to provide access to Roblox-maintained Lua code. However, we haven’t updated the list of libraries or the libraries themselves in ages. There are three libraries: RbxGui, RbxStamper and RbxUtility; the first two are…

It was fun while it lasted, but supporting it caused us a lot of pain over the years and prevented some forward-looking changes to the VM. We don’t like removing APIs from the platform, but in this case it was necessary. Thanks to the passionate feedback from the community we adjusted our initial rollout plans to be less aggressive and batch-processed a lot of gear items that used this function to stop using this function. The update is in effect and LoadLibrary is no more.

---

As usual, if you have any feedback about any of these updates, suggestions, bug reports, etc., post them in this thread or (preferably for bugs) as separate posts in the bug report category.

[Polyheximal]

Big F for LoadLibrary but an even bigger yay for Luau! Glad to see the progress of this and can’t wait to see more in the future.

[Cald_fan (Cald)]

It would be cool to see the | and & changed to|| and && because it is more convenient when you are also coding in JavaScript, it is also more convenient in general. That also goes because the tables are now more like objects.

[qelei]

Epic recap!

Luau is epic
Glad to see these progress a long journey

[buildthomas]

@Cald_fan

It would be cool to see the | and & changed to || and && because it is more convenient when you are also coding in JavaScript

That makes less sense – they are not meant to denote logical operators. It’s to denote type unions and intersections.

[Cald_fan (Cald)]

yes but it is convenient, and developers who might type || instead of |, it also teaches you something about another language in a way.

[Elttob (dphfox)]

That aside, these updates have been awesome! Continue has been a dream for my insane iteration habits, and so have the performance gains

Out of curiosity, which is faster when checking for nil (assuming x is not false)? Do type annotations change the performance of these checks?

if x then print("Not null!") end

if x ~= nil then print("Not null!") end

[sleitnick]

I love that you guys just built your own interpreter and made it so much better. That’s some serious engineering commitment. Everyone benefits from this, including the players.

[ScriptOn (Genya)]

Any updates on multithreading?

[Rocky28447 (Rocky)]

Hooray for Luau! However, I’m quite upset that I’ve gotten no word from staff about this 100% reproducible studio crashing bug that occurs with the “Script Analysis” beta feature enabled, even though it’s been well over a month since I reported it.

Chaining function calls with colon syntax causes 100% Reproducible Crash - Luau Type Checking Beta

I’ve started experiencing a 100% reproducible crash when I have the new Luau type checking enabled. I’ve narrowed it down to a single function in a single script included with @Crazyman32’s AeroGameFramework. Here is what happens: Studio will immediately become and remain unresponsive upon the script being pasted into DataModel Studio will continue to exponentially use more and more memory until killed from task manager I’m including a stripped-down version of the script with the bare minimum…

[Wqund (p1rate)]

Amazing recap! Cant wait to see what improvements are going to take place. A great work for those people who work on that!

Also big F for LoadLibrary.

[zeuxcg]

We’re actively looking into this bug, but it’s a bit non-trivial to resolve.

[Elttob (dphfox)]

Also another question! Here’s my current code for classes within the type system:

--!strict

type MyClass = {
	foo: string,
	
	speak: (MyClass) => nil
}

local MyClass = {}

function MyClass.new(foo: string) => MyClass
	local self = setmetatable({}, {__index = MyClass})
	self.foo = foo
end

function MyClass:speak()
	print("Bar bar",self.foo, "!")
end

local ins: MyClass = MyClass.new("hello!")

ins:speak()

It might be more ergonomic if parts of the class type could be inferred somehow. I’m honestly not sure how you’d pull it off, but is stuff like this under consideration?

[zeuxcg]

@Elttob

Out of curiosity, which is faster when checking for nil (assuming x is not false)? Do type annotations change the performance of these checks?

I’d expect if x to be a bit faster. We have a pending optimization in backlog to optimize comparisons with constants a bit better. Type information currently doesn’t affect code generation.

[nooneisback]

&& is also a double address operator in C++. That has nothing to do with learning another language, it just pushes false sense of familiarity which can result mistakes that are hard to find.

[Rocky28447 (Rocky)]

Agreed, but it’s just something that shouldn’t happen. I shouldn’t have to go and disable type checking, restart studio, reopen the level, make all the necessary changes to the script I’m using, enable the beta feature AGAIN, restart studio AGAIN, then reopen the level AGAIN every time there’s an update to the script just to stop studio from crashing.

[spicychilly2 (lan)]

I never got to know LoadLibrary ;(

[Mystifine]

I’m sorry for being quite confused but is Luau basically another language that’s going to be used for ROBLOX?

Does this mean that ROBLOX programmers will have to change their ways of scripting and learn the next prefixes and syntax?

I’m just quite confused because I’m not sure what’s going on.

[Dekkonot]

Luau is still Lua. It’s just the internal name for the new implementation of Lua they’ve made. You won’t have to worry about learning new syntax, unless you care about type annotations (essentially, there will be a new syntax for adding what type a variable is and what a function returns).

@zeuxcg

in particular we’ve implemented support for require statement and that should ship next week

I assume this means the automatic importing of types via require, or is there some other support that I’ve not noticed?

Also, is the syntax for typed Lua considered stable (only additions, never changes or removals) or should we hold out on using it for serious work still?

[Killian773]

This is a great recap! Nice job.