Typed Lua - Best Syntax and Methods

Help and Feedback Scripting Support
#1

Hello fellow Robloxians! As some of you know, I’m working on extending the Lua syntax to add type checking and inference and call it Luafide. Having put a lot of time and effort into the groundwork for Luafide, I’d like to see it used. At the bottom of this post I ask for feedback, but I’ll first present the major use cases for Luafide and then describe my current plans for it.

Major Use Cases

As we are all aware, Roblox employees do a lot of programming in Lua. Roblox employees have written tens of thousands of lines of code in Lua, which runs on up to millions of devices concurrently. In addition, some developers' games have upwards 30,000+ concurrent players. Many developers' codebases have also grown upward of tens of thousands of lines.
Keeping track of such codebases cannot be done by memory. The issue is further complicated when codebases are developed by teams. Developers on teams must either keep documentation up to date, read the mind of what other developers were thinking when they named their variables, or dive into the source code to try and determine the types of a functions inputs and outputs. Even after having looked at the source code, the type may not be apparent because it may be dependent upon external types. A development environment is needed that will infer types and make this information available to developers.

A strict, explicit type system would be hinder rapid Lua development iteration and introduce maintenance of explicit types, however for a complete, decidable, and accurate type system some types may need to be explicitly defined. The cyclic dependencies present in recursion as well as dynamically modified tables are particularly difficult to type without explicit declarations or making the types involved very general.

For developers who are just starting to develop on Roblox, an inferred type system would provide great help and prevent many beginner’s mistakes. While some functions are well documented, accessing documentation and looking up types is time consuming and at times difficult for the new developer to find. Built in information available on mouse-over and type mismatch warnings would greatly enhance the learning experience.

Developers using Lua outside of Roblox would also benefit from typed Lua. Although there are other typed lua alternatives, some have been discontinued, morphed into entirely different languages, are only available for newer versions of Lua, or require libraries.

Because a single type system is guaranteed to not to be perfect for every use case and satisfy every developer, a type system and syntax that can be relatively easily extended or modified is desirable.

The plan

A type system is almost useless without a development environment that utilizes it. Using a LuaWidget plugin, a syntax highlighter will be developed to work with the Luafide syntax and display type errors. This will be divided into two sections, a data oriented and platform independent backend, and a development environment specific frontend. This will allow the frontend to be swapped to develop in other environments besides Roblox studio. 
-- Proposed Syntax Changes:
TYPENAME: 'Any'
    | 'Number'
    | 'String'
    | 'Boolean'
    | 'Thread'
    | 'Struct'
    | 'Userdata'
    | 'None'
    | name -- must be defined in a typedef
-- Note the missing Function and Table types

TYPE: TYPENAME 
    | TYPE '->' TYPE -- Tables (map key type to value type)
    | '(' TYPELIST ')' -- Functions (input type list -> output type list, or just input type list)
    | TYPE '|' TYPE -- Unions

TYPELIST: TYPE 
    | TYPE ',' TYPELIST

-- Added to list of possible statements
-- Has static local scope
TYPEDEF: 'typedef' name STRUCT_DEF 
    | 'typedef' name TYPE

-- Local definitions (local namelist = explist)
-- and function parameters (namelist | ... | namelist, ...)
TYPED_NAME: name 
    | name ':' TYPE

NAME_LIST: TYPED_NAME
    | TYPED_NAME ',' NAME_LIST -- replaces normal Lua 'namelist'

-- Definitions (varlist = explist)
TYPED_VAR: VAR
    | VAR ':' TYPE

VAR_LIST: TYPED_VAR
    | TYPED_VAR ',' VAR_LIST -- replaces normal Lua 'varlist'

-- Structs
TYPED_FIELD_LIST: TYPED_NAME 
    | TYPED_NAME FIELD_SEP TYPED_FIELD_LIST

STRUCT_DEF: '{' TYPED_FIELD_LIST [FIELD_SEP] '}'

Here is an example:

typedef Tower {
    health: Number;
    maxHealth: Number;
    position; -- types are optional, if you so choose
    owner: Userdata;
}

return function (tower: Tower, at)
    local beam = Instance.new("Part")
    ...
    return beam
end

The system will perform type checking on groups of scripts/modules. It will start with variables of known types, and assume all other variables have type Any. It will then piece together hints of the variable’s type to restrict the variable’s type. When no more hints can be evaluated, the algorithm is finished and all variable types are final until changes occur. It will warn when values can be mutated by code outside the checked scripts/modules to cause unexpected results and understand type checks performed by the developer using type(). It will also inform the developer of cyclic dependencies or tables which may be causing the inferred type to be too general.

Luafide will be guaranteed to run on Lua 5.1 and newer, never throw errors for valid vanilla Lua syntax unless there is a type error, and run without any external libraries. This means that while it could run on Roblox, it could also run in other restricted environments without much effort.

If performance is an issue, prioritizing source files which are open or contain symbols present in the open files can be prioritized.

Feedback

This is the most important part of this post. While I could make a system that I like and would use, I want it to be useful to others as well! My first question is: does this explanation of Luafide make sense? What parts could use a better explanation? Next, have you noticed any problems this addresses or partially addresses that hasn't been mentioned? I'd like to make sure that I'm on target and solving problems.
I based the syntax upon typed languages that I have worked with and seen. I have noticed a downfall though: it is difficulty to type Userdatas. This is in part because the type system is completely static and doesn't allow dynamic types. I'm not sure what way is best to add support for typing Instances like Parts, BaseParts, ext. How do you recommend I add that support?

I’ve noticed that when types are explicitly stated, they are final. It might be nice to give hints to Luafide (may be this type, definitely not one of these types) without forcing the type. I haven’t seen this done in practice, and so question if it is even a good idea. What do you guys think?

Lastly, if you think there is a better way to do something or think another feature should be added (types should always be explicit, explicit casts) then please leave a message describing your idea and why.

Thanks!

17 Likes