Oh, sorry I didn’t see it.
I was reading original Luau C++ lexer code to create parser for my project and found many interesting things.
For example, for interpolated strings it is much better to use 4 patterns instead of one:
STRING_INTER = "`[^\n]-`"
STRING_INTER_START = "`[^\n]-[^\\]{"
STRING_INTER_END = "}[^\n]-`"
STRING_INTER_MIDDLE = "}[^\n]-{"
Again, thank you for your great job and good luck in improvement your lexer!
Your patterns will not work, because it needs to be contextual. For example, consider the following case where a string is defined after some table.
foo({"table"}, `string`)
STRING_INTER_END will match the closing brace of the table, along with the comma, space, and backtick. That section of code is not, in fact, a string.
Ok, I’ll try create working code later when get to my PC.
I have found interpolated strings support in your code. I don’t think it needs to be replaced.