Luau Recap: August 2020

math.round is very cool, thank you.

With the removal of __meta, is there no way for us to define the result of various operators now? This feels like it would be useful.

Also if we’re approaching the end of the typed Lua beta, does that mean I can start bothering you guys about vararg support again?

5 Likes

Right now the “best” way is to use typeof(setmetatable(...)). This is not perfect but in general right now the best way to specify a type for an object seems to be via typeof. We plan to explore some way to define classes or interfaces or both to fix this in the future, but we decided to start by removing the __meta hack.

Yeah please share your grievances with varargs here, we’d love to make this better.

5 Likes

Last question I promise.

Is there absolutely any difference at the end if I do math.floor(x + .5) rather than math.round(x)

I decided to run a quick test (probably should’ve done this for my previous question too :sweat_smile: ) and I found that math.round(x) is about 3 times slower than math.floor(x + .5). I was hoping it would be faster

do
	local start = tick()
	
	for i = 1, 10, .0001 do
		math.round(i)
	end
	
	print(tick() - start)
end

do
	local start = tick()
	
	for i = 1, 10, .0001 do
		math.floor(i + .5)
	end
	
	print(tick() - start)
end
  0.0032095909118652
  0.0009310245513916
> print(0.0032095909118652/0.0009310245513916)
3.4473751600512
2 Likes

math.floor has some magical optimizations that haven’t been implemented for math.round yet. There’s some difference in behavior, math.round in particular returns 0 correctly for, uh, 0.49999999999999994.

12 Likes

Is it possible to change the math.round function to the nearest even?
I’m used to half even rounding and it just feels natural to me.

2 Likes

WARNING: Do not use round for geometric snapping!!, you should still be using math.floor(x + 0.5) for that.

math.round() makes sense for rounding textual inputs from the user, but is not the correct thing to use for geometric snapping, such as snapping a dragged part or Gui to the nearest grid unit, thanks to the difference in rounding rules:

Good:

math.floor(-1.5 + 0.5) -> -1
math.floor(-0.5 + 0.5) -> 0
math.floor(0.5 + 0.5) -> 1
math.floor(1.5 + 0.5) -> 2

Bad:

math.round(-1.5) -> -2
math.round(-0.5) -> -1
math.round(0.5) -> 1  ... +1 to the input resulted in +2 to the output, yikes!
math.round(1.5) -> 2

TL;DR: The fact that math.floor / math.ceil always have the same relative “bias” in their rounding is desirable for geometric snapping.

27 Likes

Will we ever see syntax for defining metatable types? A built-in type (something like WithMetatable<T, U>) like the ones TypeScript has (i.e. FunctionArguments, ReturnType, Partial) would be helpful.

I guess this is technically possible right now?

type WithMetatable<T, U> = typeof((function()
	local x: T
	local y: U
	return setmetatable(x, y)
end)())

Luau likes to crash when I mess around with this too much though image

edit: This code will 100% of the time crash luau when pasted into a script:
--!strict
type MyType<T, U> = typeof((function(x, y)
	
end))

local x: MyType<{}, {}> = {}
4 Likes

I’m excited to know that parallel Luau is under way. I’m not too knowledgeable on the technical implications of dispatching threads to handle tasks or how it’ll all go down, but I’m keeping posted. If this will allow running of certain expensive tasks in different threads and if this can increase performance depending on how it’s used, then it’s absolutely something I’ll want to have access to.

8 Likes

Am I missing something regarding the “Month following June is August” bit? It feels like I am missing some context regarding that statement. Otherwise, I am really excited about this.

1 Like

Since \0 is allowed in patterns, will utf8.charpattern be changed to use \0 instead of %z?

Format string analysis has some issues:

Treats using %1 with no captures as invalid, when it’s valid.
image
Treats an empty set [] or [^] as valid, the first character isn’t considered for closing the set (so something like []] is valid).
image
Same with frontier patterns %f[] or %f[^]
image
Treats %z and %Z as invalid
image

Some odd cases with ranges

% is part of the range in this case, shouldn’t consider the next character
image
And these too


Similar thing here, probably also related to accepting empty sets but this is valid

Q here should generate a warning, as it’s an invalid character class in a position to be used as a character class.
image
Or better yet, it should warn for creating a range which uses character classes, considering the manual states:

The interaction between ranges and classes is not defined. Therefore, patterns like [%a-z] or [a-%%] have no meaning.

Odd interaction with nested captures, the first is invalid while the second is valid


Doesn’t check if integral size is within [1,16]
image
Doesn’t check for anything to be after X
image

8 Likes

I saw the string.format analysis and that gave me an idea. What if Luau supported compile-time/runtime analysis things like Rust does?

Like, for example, defining a Luau function that actually gets executed during analysis that can access type information / tokens / etc. and can provide warnings/errors (squigglies) with a custom message.

But, of course, the Luau analyzer is probably not written in Luau which would make it very difficult and expensive to translate all of its internal state into a straightforward Luau data structure, so this will probably never be a thing… :(

Things like communicating what types the function takes based on a string you pass to it would be great to support generically rather than as a special feature for string.format.

Yeah that’s probably reasonable.

I believe the fact that %1 with no captures is valid is an accidental result of how the implementation works, not the specified behavior. %0 should be used instead.

More or less intentional to simplify the implementation, although might be worth checking.

The other issues look like bugs, thanks!

3 Likes

The 5.2 reference manual added this text to the description of string.gsub

In any case, if the pattern specifies no captures, then it behaves as if the whole pattern was inside a capture.

So it doesn’t seem like %1 working with no captures is an accident, considering the description was updated to include this case.

Luau’s type system is not sound. Function types’ parameters should be contravariant and generic parameters should be invariant.

--!strict
--- The parameters of function types should be contravariant, not covariant!

function apply(f: (BasePart) -> ())
	f(Instance.new("Part"));
end

-- valid, but should not be
apply(function(obj: SpawnLocation)
	print(obj.TeamColor)
end)

-- not valid, but should be
apply(function(obj: Instance)
	print(obj)
end)


--- Generic parameters should be invariant, not covariant!

type Array<T> = { [number]: T }

local x: Array<BasePart> = {}

-- valid, but should not be
local y: Array<Instance> = x
y[1] = Instance.new("Model")

-- not valid, and shouldn't be
local z: Array<SpawnLocation> = x;
z[1] = Instance.new("SpawnLocation")

Edit: generics should be invariant

2 Likes

In the generics example, this is where type assertions would definitely be useful, cause I can see a case where you get a function that returns a (newly constructed, rather than mutated) Array<BasePart>, and you want to generalize that to an Array<Instance>

Of course type assertions can go wrong if you lie to the type checker, but I think it’s better to have that be up to the user. On the other hand, as it works currently, things can definitely go wrong without type assertions since it lets you mutate x in that example in an invalid way.

Edit: that very last example is both not valid, nor should it be valid, since if you read an element from z, it’s not actually guaranteed to be a SpawnLocation

I have a few questions about this.


image
I think it should be possible to strong-type class properties like this.

could be a good solution for that I think. Would also reduce a lot of boilerplate code developers have to write for OOP.


Will this ever be available to the public?

1 Like

Woo, Type declaration.
Can’t wait to try out the new features.

Yeah you’re absolutely right.

I still believe it should be an error, not up to the user, and Roblox should introduce something akin to typescripts “v as X” that way you can only write potentially unsound code explicitly which will make it harder for the user to write unsafe code.

I have edited my post.

1 Like

I think that for me these luau updates have been the hype of the century. If anyone told me we we’d be able to do parralel processing in Roblox 3 years ago I’d not believe you in the slightest let alone most of the new stuff luau is adding already.

I have the feeling anyone who has seen me post on the topic of luau has probably noticed I have an unnatural obsession with squeezing every little nanosecond out of some of my Roblox code and every time I see lauu performance improvements mentioned I immediately get very excited.

(I’m too lazy to properly quote these right now because Android, and I’m probably missing some critical information on how discourse quoting works so uh)

This sounds like it would’ve been a huge disaster if I’m understanding correctly so it’s great that this isn’t a thing.

This is a piece of info I’m super interested on! Which pieces of info will Roblox’s form of debug.getinfo support?

Lastly, table.pack and table.unpack
[super]thankyouthankyouthankyouthankyouthankyouthankyou[/super]
When I originally looked at the docs for this, I scanned through them, but, I decided to take a more thorough look recently and I’ve realized how relevant this is to just about everything I do. This is absolutely INCREDIBLE for me and I’m sure others as well.

Things I am now considering being relatively feasible (and even performant!):

  1. rbxm in rbxlua :eyes: This would only take a port of lz4, and a port of an existing implementation (@CloneTrooper319 wrote a very nice one in C# and I think this wouldn’t be hugely difficult to port now!). Additionally, a great example of data that is stored in these files that’d be a very quick luau implementation would be the newer (holy cow how isn’t this released yet) attributes
  2. Bytecode interpreters for other languages (e.g. a custom x86 interpreter) There was nothing stopping anyone before, but, now it’s going to be a piece of cake (relatively speaking)
  3. Faster and smaller file format processing for all (File format plugins?)

If anyone wants to look into porting CloneTrooper’s rbxm implementation I’d be super interested! (rbxm in Lua has been one of my all time favorite ideas)

Lastly, (because I’m not sure most people are even still reading at the pretty pointless sheer size of this post) things that I am most definitely legally obliged to mention because I’ve only recently noticed that almost all of my luau posts contain something about my damn compression algorithm:

  1. It’s finally time for me to write the 10th revision (actually technically the 16th which would change any mechanisms if you consider smaller rewrites. I’m unreasonably satisfied that this will be the 16th AND 10th)
  2. I’m pretty sure I can turn about 60 lines give or take in my existing code into about just one. Not only does that imply a hefty performance boost given the expensivity of the code that’s a performance boost on top of the fact that iirc prior I was compressing a little over a quarter of a megabyte of test data in under a millisecond on my hardware (about 0.53 ms on average, which is nearly identical to lz4 speeds, and given what I was seeing with a couple tests with using lz4 data as input data this becomes an enormously useful fact)
  3. Because I’ve never gone into depth on the algorithm’s benefits and I think anyone still reading might be interested I’ll go ahead and talk about it in much much more detail rather than being kind of vague. I compare my algorithm to lz4 a lot but I also wouldn’t say it’s anywhere near as good as lz4 on its own (although to be completely fair I haven’t actually done much output size comparison), lz4 is still way better on its own. The generally best benefit I’ve noticed is that when using the algorithm on lz4 data it can achieve some hilariously small sizes on a majority of readable data. My algorithm is using a method I personally designed (which I have no idea if I’ll describe at any point, I’d like to get the project to a point where I can release it) and due to the mechanisms it uses to actually compress data it is locked to ratios in 8ths. As to why 8ths specifically, this is due to bytes having 8 bits. This number is actually pretty ideal for the algorithm itself I’d think because a number that’s much smaller would result in bad ratios a lot, and a larger number wouldn’t compress well. Additionally, this makes prediction and even guarantees for certain data pretty easy. The behaviour of actually compressing certain data is pretty predictable, for example, if your data is readable ASCII data, it will always compress it to ratios under 7/8ths. I will definitely take advantage of this fact and likely some math for prediction to automatically reencode data for even better ratios than what I have. As for why this currently improves lz4 output sizes? That comes down to the algorithm itself. The thing is, lz4 is a surprisingly simple algorithm at its core. The gist of it is that it mostly just indicates how many times to repeat certain data. So a majority of the old data is still intact. An estimate of the size of the “hybrid” output is lz4ratio * standardRatio. So, if lz4 produces data 2/3rds (~5.3/8ths) the size and my algorithm produces data 6/8ths the rough data size (which will likely be even smaller with any size optimizations of which I already implement a few simple and costless ones, but can still be a bit larger theoretically) the output will be about 4/8ths. That’s half the size vs both outputs being at least 2/3ds!
2 Likes

This bug was exploitable so I’m very happy we found it before exploiters did :smiley:

Still figuring that out. On one hand we could expose everything that Lua 5.x does. On another hand, maybe exposing a function object from the stack frame isn’t a great idea since then you can call it. On the third hand, setfenv() already allows you to mess with your callers so maybe it’s fine.

Our current implementation supports “source” (script path), “what” (C/Lua), “currentline”, “nupvals” (from Lua 5.x, not useful in absence of other debug methods that we don’t want to provide for sandboxing reasons so might go away), “nparams” (number of fixed arguments to the function), “isvararg” (is function variadic), “name” (function name), “namewhat” (historical artifact from Lua 5.x, will be removed), “func” (function object).

2 Likes