Luau modern programming techniques open source library (Slice, Iterator...)

After exploring multiple modern programming languages, I learnt a lot.
And coming to programming on Luau again after a long time surprised me that not a lot have changed since compared to other technologies I’ve explored.
Since I introduce to the library I just started working on for myself and for the community going forward:

I would gladly accept contributions on github or here on the devforum.
Publishing this library is very early but I’d like to get as much feedback and possibly contributions :slight_smile:

Codotaku Luau Library

Luau’s standard libary is raw and minimal, unlike other programming languages like Rust and Zig’s standard library and C++'s STL. So luau is far behind by ages in terms of modern programming techniques.

Examples:

  • Rust and Zig have slices, C++ have views.
  • Rust have iterators, C++ have ranges.

A slice or a fat pointer in lua’s terms is basically a table with a start index and an end index. or a table with an offset and a length. A slice is very useful when you want to pass or receive only a portion of a table, a view into a table (without ownership of the table) without having to copy the table or construct a new one. it does exist in lua’s standard library but in its naked form table.fn(t, i, j). And slices also give the property that if you modify the table directly or through one of its slices, the table and overlapping slices also see the changes, since it doesn’t have a copy of the table, merely a reference to it. Slices are also very cheap and small tables to copy and pass around, unlike copying or creating a new big table just to return one of the portions of the existing table.

An iterator or a generator is very useful as it allows a declarative functional approach into computation and transforming data but its also useful to save on resources, since iterators are usually lazy which means that you only pay for what you want although they come with some overhead for each iteration. and they work by basically requesting the next computation result, which also avoids creating a new table with all results, when the user may be interested only in the first result, here is some pseudo code example:

print(split('hello world, and salam in arabic!')[1])

here split will return all words on the sentence although we’re only printing the first word, but this problem is avoided with an iterator

print(split_iterator('hello world, and salam in arabic!').next())

Soon I’ll edit this post and the github repo with examples of how to use the currently developed module Slice but here is the current API from the script (still highly experimental and not fully tested):

--[[ API(27)
Class(1)
	Slice = {ClassName = 'Slice'}
Metamethod(1)
	__len()
Constructor(1)
	new(t: table, i: number?, j: number?)
Copy Constructor(1)
	clone()
Getter method(1)
	at(index: number)
Self mutating method(3)
	shrink(length: number)
	advance(offset: number)
	sub(offset: number, length: number?)
Functional method(2)
	foreach(fn, ...)
	reduce(fn, ...)
 Mathematical method(5)
	sum()
	product()
	average()
	min()
	max()
Table mutating method(3)
	map(fn, ...)
	fill(value)
	fill_with(fn, ...)
Conversion method(4)
	table(): table
	string(seperator: string?): string
	concat(seperator: string?): string
	unpack()
Iterator method(4)
	filter(predicate)
	chunk_exact(chunk_size: number)
	chunk(chunk_size: number)
	group_by(classifier)
]]

--[[ Goals
	-- Performance before readability
	-- Using the standard library when possible
	-- Independent methods as much as reasonable
]]

--[[ Inspiration
	-- Rust standard library (https://doc.rust-lang.org/std/primitive.slice.html)
]]

Examples:

local Std = require(script.Parent)
local Slice = Std.Slice

local function add(a, b) return a + b end
local function multiply(a, b) return a * b end
local function is_even(x) return x % 2 == 0 end

local slice = Slice.new{104, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100, 44, 32, 115, 97, 108, 97, 109, 33}
local hash_byte = string.byte('#')

assert(slice:string() == 'hello world, salam!')
assert(slice:sum() == slice:reduce(add))
assert(slice:product() == slice:reduce(multiply))
assert(slice:min() == slice:reduce(math.min))
assert(slice:max() == slice:reduce(math.max))
assert(slice:average() == slice:reduce(add)/#slice)
assert(slice:advance(6):string() == 'world, salam!')
assert(slice:shrink(8):string() == 'world')
assert(slice:at(1) == string.byte('w'))
assert(slice:concat(', ') == '119, 111, 114, 108, 100')
assert(slice:map(add, 1):string() == 'xpsme')
assert(slice:concat(', ') == '120, 112, 115, 109, 101')
assert(slice:map(add, -1):string() == 'world')
local slice2 = slice:clone()
assert(slice:string() == slice2:string())
assert(slice2:sub(1, 1):string() == 'orl')
assert(slice:string() == 'world')
assert(slice2:fill(hash_byte):string() == '###')
assert(slice:string() == 'w###d')

local expr = '1234.345 + 235 * 345'
local s = Slice.new{string.byte(expr, 1, #expr)}
assert(s:string() == expr)

local TokenKind = {
	Unknown = 0,
	Operator = 1,
	Number = 2,
	Whitespace = 3,
}

local function classifier(byte: number)
	local b = string.byte
	if byte == b(' ') then return TokenKind.Whitespace end
	if byte == b('+') or byte == b('*') or byte == b('/') or byte == b('') then
		return TokenKind.Operator
	end
	if (byte >= b('0') and byte <= b('9')) or byte == b('.') then
		return TokenKind.Number
	end
	return TokenKind.Unknown
end

--[[ Expected output:
  2 1234.345
  1 +
  2 235
  1 *
  2 345
]]
for kind, token in s:group_by(classifier) do
	if kind == TokenKind.Whitespace then continue end
	print(kind, token:string())
end

local s = Slice.new{1, 2, 3, 4, 5, 6, 7}

--[[ Expected output:
  2
  4
  6
]]
for x in s:filter(is_even) do
	print(x)
end

--[[ Expected output:
  1:2
  3:4
  5:6
  7
]]
for x in s:chunk(2) do
	print(x:concat(':'))
end

--[[ Expected output:
  1:2
  3:4
  5:6
]]
for x in s:chunk_exact(2) do
	print(x:concat(':'))
end
6 Likes

Added Examples (Test) and done a lot of small fixes!
here is the file on github:

1 Like

will there be a filter?

3 Likes

yes, very soon as an iterator!

1 Like

Added 4 useful iterators:

  • filter
  • chunk, chunk_exact
  • group_by
Iterator method(4)
	filter(predicate)
	chunk_exact(chunk_size: number)
	chunk(chunk_size: number)
	group_by(classifier)

@commitblue requested filter