A while ago, I created rStack, a library for stack-based purposes. I’ve decided to write a stack-based scripting language inside of Roblox’s Luau. Obviously, it will be useless unless I implement some features that Luau already has, like changing properties, etc. I mostly wrote this out of boredom, but also to see how far I could take it. I’ve written languages like this plenty of times in the past, in several other programming languages. I’ve just never completed them. I’ve based this lexer off a similar one I’ve written in Python.
If you don’t know how a lexer works, allow me to explain. A lexer, or lexical analyzer, takes in a string, goes through that string, and generates a list of tokens, or lexemes. They contain information like the position in the string of that token (the line and column), the file (not important since I’m writing this in Luau and can’t add custom file support, just require the module and type in your code), the type of that token (int, float, string, identifier, operator, etc.), and the value (1, 1.2, “hello world”, helloworld01, +, etc.) Those tokens will later be used by a parser. It’s all fundamentals of language design.
In this example, I’ve inputted this string:
and this is the output I get:
If you have any feedback, feel free to let me know. Thanks for reading!