New Type Solver [Beta]

Updates Announcements
226

Hey folks,

It’s been a little while now since we first released the beta for Luau’s New Type Solver! First and foremost, we want to thank all of you for trying it out, and the extensive feedback, bug reports, and crash reproductions we’ve received from all of you. It’s a tremendous effort to build something like this, and polishing it towards general release truly takes a village. So, thank you all!

With thanks aside, we wanted to post a quick update to everyone about the work the Luau team has been up to since we made that initial beta release. Overall, there’s been a lot of crash and bug fixes that have landed (you’ve probably seen them scattered throughout Roblox release notes!), and still plenty more to come as we continue to build towards a general release. Some of the major pain points on our radar right now include, but are not limited to:

  • Constraint solving failed to complete: A common and understandable pain point for folks is when type inference is not able to complete. These always represent a bug in the type system, and manifest as the “orange bar on the first line of the script” in Studio. We’ve got a number of big changes coming over the next few weeks that will reduce these errors significantly by avoiding constructing the states that lead to type inference getting stuck and bailing out before it completes.
  • Confusing type errors caused by builtin type functions failing to reduce, typically because of cycles in the types: This is a tricky one, but we’re working on improving our approach to normalizing types to eliminate these cycles and ensure that the type functions reduce!
  • Unexpected type mismatches between a user-annotated type and a table literal expression: These are broadly a consequence of us needing to be more strict around when tables can be used as other tables for type safety, but specifically for defining new tables, we’re working on ensuring that the type can be appropriately promoted.
  • Type mismatches around conditionals, refinements, and assignments: We’ve identified a number of bugs where either we expect assignment to introduce a new type-state (as described in the original release) but it doesn’t happen, or a type refinement from an if statement persists into places where it’s not useful (like the left-hand-side of an assignment).
  • Autocomplete denial of service: Sometimes, performance problems (exponential blowup) in the New Type Solver can lead to an autocomplete denial of service. We’re working at this problem from two angles: (1) improving the performance by reducing the frequency of the types that cause this exponential blowup, and (2) changing the architecture of autocomplete to decouple autocomplete as a service from typechecking performance (currently in a separate Studio Beta called “Incremental typechecking and autocomplete” but we’re streamlining the name to “Incremental Autocomplete” in an upcoming update).

We’ve also had some major feature drops for the new solver, like Improvements to New Non-strict Mode and User-Defined Type Functions. In the case of the former, we’re responding to user feedback and iterating on our design. In the case of the latter, our original announcement discussed how the new type inference engine supports functions over types which allowed our team to more accurately infer the types of overloaded operators. With this new addition, we’re going even further to put this capability into the hands of our Creators.

Improvements to New Non-Strict Mode

As some of you may already know, there’s been a long-standing Studio Beta for “Non-Strict by Default” that has quietly remained a beta as we’ve iterated on the existing strict and non-strict mode. We’ve chosen to leave the beta as-is, rather than move forward to a general rollout of non-strict by default because we feel that the existing product for non-strict mode has not been what we’d want it to be. The new type solver has been a key piece of our effort to redesign that product, as its architecture allows both strict and non-strict mode to share exactly the same type inference system that you’ve all been beta testing for us.

With the initial release of the new type solver beta, we intentionally stripped back the functionality of non-strict mode considerably. Philosophically, we think of non-strict mode as a product for the Creators who do not know what a type system is (or don’t want one), do not write type annotations, and (rightfully!) want the script editor to just work — providing helpful autocompletions, appropriate documentation on hover, and so forth. We want non-strict mode to be as helpful as possible, but we also want to make sure that the errors it produces meet the criteria of being clear, useful, and actionable to users who don’t know what a type is. As such, the new non-strict mode shipped with just one kind of error: if the type solver could prove that a builtin function will always receive a value at runtime that will cause it to raise an error (e.g. passing a string to math.abs), we will surface a static error that explains the runtime error.

That being said, we’ve heard from many of you that this new non-strict mode is missing an essential piece of editor feedback, namely signaling to the user that they are referencing an unknown identifier (typically because of a typo while writing or editing the code). We agree, and we think that this sort of error fits with the overarching direction we see for new non-strict since it is similarly explainable without reference to the type system at large. As such, we’re pleased to announce that the new non-strict mode has been extended to support these unknown identifier errors. We look forward to hearing more about experiences using the new non-strict, and iterating further towards delivering the best possible developer experience for folks who do not want the fully-typed experience offered by strict mode!

User-Defined Type Functions

With user-defined type functions, Luau programmers can write their own custom functions that take type arguments to compute a resulting type. This feature is a pretty advanced one (and one we expect will largely be taken advantage of by major libraries in the Luau and Roblox ecosystems), so it’s understandable if it feels unfamiliar at first, but we’re really excited about the expressiveness that it unlocks. While the details are quite different, for those familiar with TypeScript, user-defined type functions offer the ability to implement behaviors such as mapped types and quite a bit beyond!

As a place to start, we can consider some very simple type functions that have existing equivalents and then build up to reimplementing some of the builtin ones in pure Luau! The most simple type function we could write is just one that immediately returns a builtin type:

-- this type function
type function PrimitiveString()
    return types.string
end
-- is equivalent to:
type PrimitiveString<> = string

In the body of a type function, we have access to this whole types library (whose API you can find today on Luau’s website, and hopefully soon on Creator Hub as well). Besides primitives, we have constructors for making types out of other types. So, for instance, we could produce a union or an intersection:

type function UnionOf(t1, t2)
    return types.unionof(t1, t2)
end
-- is equivalent to:
type UnionOf<T1, T2> = T1 | T2

type function IntersectionOf(t1, t2)
    return types.intersectionof(t1, t2)
end
-- is equivalent to:
type IntersectionOf<T1, T2> = T1 & T2

But the body of these type functions is just Luau, so we can add additional reasoning as well. For instance, we can modify these type functions to versions that perform an immediate simplification, i.e. not producing a union at all if one of the arguments is unknown, or not producing an intersection if one of the arguments is never:

type function SimplifyingUnionOf(t1, t2)
	if t1 == types.unknown or t2 == types.unknown then
		return types.unknown
	end
	
	if t1 == types.never then
		return t2
	end
	
	if t2 == types.never then
		return t1
	end

    return types.unionof(t1, t2)
end

type function SimplifyingIntersectionOf(t1, t2)
	if t1 == types.never or t2 == types.never then
		return types.never
	end
	
	if t1 == types.unknown then
		return t2
	end
	
	if t2 == types.unknown then
		return t1
	end

    return types.intersectionof(t1, t2)
end

These type functions have no existing equivalent since they are not a trivial mapping, but rather use conditionals to perform algebraic simplifications on the union or intersection. User-defined type functions are even powerful enough to implement some of the builtin type functions we originally announced, like keyof and index (though the versions below are simplified in scope):

type function KeyOf(ty)
	if not ty:is("table") then
		error("KeyOf only supports operating on table types!")
	end

	local components = {}
	for key, _ in ty:properties() do
		components.insert(key)
	end

	return types.unionof(table.unpack(components))
end

type function Index(tbl, index)
    if not tbl:is("table") then
        error("Index only supports operating on table types!")
    end

    for key, value in tbl:properties() do
        if key == index then
            if value.read ~= value.write then
                error("mismatched read/write types found for the property")
            end

            return value.read
        end
    end

    error("key not found!")
end

We’re really excited to see how Creators will leverage the extremely powerful functionality of user-defined type functions to build more type-safe libraries for Luau and Roblox.

21 Likes