Better standard library support for manipulating lists and tables

Feature Requests Studio Features
#1

I want a function to concatenate two lists.
items = a:Append(b)

I want a function to map a function over a list.
Model:GetChildren():Map(function(n) n:Destroy() end)
or better
Model:GetChildren():Map(n => n:Destroy())

I want a function to map a selector over a list to generate a new list.
items = items:Where(n => onSale == true)

I want to sort this way instead of with fiddly comparators
items = items:OrderByDesc(n => n.Name)

In C# or JS these are all one-liners. In Lua I’m in for loop city constantly. Is there any thought being given to expanding the standard library?

I want to be able to do this:

var title = entries.Where(e => e.Approved)
    .OrderBy(e => e.Rating).Select(e => e.Title)
    .FirstOrDefault();

This would be a page of code today in Roblox Lua and it would not be readable.

Look at this code I had to write today. Yuck. Unfortunately, I have additional lists to merge.

local items = {}
function buildDb()
	local a = game.ReplicatedStorage.RewardItems:GetChildren()
	local b = game.ReplicatedStorage.ShopItems.Hats:GetChildren()
	
	for i = 1,#a do
		items[#items + 1] = a[i]
	end
	
	for i = 1,#b do
		items[#items + 1] = b[i]
	end
end
19 Likes
#2

The table:operation suggestions don’t work because the table could have a value with the same key as the name of the operation, e.g. t.map = true.

Here is the existing proposal for arrow function syntax with response from engineers:

See also:

You can trim down your list-merging code using table.move.

8 Likes
#3

I think the table:operation functions could be made to work in all instances where the table is being used as a list. Lua already has some QoL syntax for tables that look like lists (thinking of the # operator).

Personally I don’t care if everything in Lua is a table, that is an under-the-hood detail. As a developer, I’d like some nice collection primitives for things like Lists and Queues. Those can be built on tables, I just don’t think every Roblox developer needs to implement them individually. There should be a better standard library.

It can be done without the arrow/lambda operator (=>), I just think that is more readable when you are using the modern “fluent” style of chaining different list operations together. More ends make code less readable.

5 Likes
#4

I agree with this. Rewriting the same implementations over and over that should be included by default and most of the time is in other languages is a bit annoying to deal with. A stronger standard library makes the most sense.

For example (in Lua), we have these:
Queues - Programming in Lua : 11.4
Stacks - lua-users wiki: Simple Stack

tl;dr - Code examples

-- Lua docs
--
-- Queue implementation
List = {}
    function List.new ()
      return {first = 0, last = -1}
end

function List.pushleft (list, value)
      local first = list.first - 1
      list.first = first
      list[first] = value
end
    
function List.pushright (list, value)
      local last = list.last + 1
      list.last = last
      list[last] = value
end
    
function List.popleft (list)
      local first = list.first
      if first > list.last then error("list is empty") end
      local value = list[first]
      list[first] = nil        -- to allow garbage collection
      list.first = first + 1
      return value
end
    
function List.popright (list)
      local last = list.last
      if list.first > last then error("list is empty") end
      local value = list[last]
      list[last] = nil         -- to allow garbage collection
      list.last = last - 1
      return value
end

-- Lua docs
--
-- "Simple" Stack implementation
-- Stack Table
-- Uses a table as stack, use <table>:push(value) and <table>:pop()
-- Lua 5.1 compatible

-- GLOBAL
Stack = {}

-- Create a Table with stack functions
function Stack:Create()

  -- stack table
  local t = {}
  -- entry table
  t._et = {}

  -- push a value on to the stack
  function t:push(...)
    if ... then
      local targs = {...}
      -- add values
      for _,v in ipairs(targs) do
        table.insert(self._et, v)
      end
    end
  end

  -- pop a value from the stack
  function t:pop(num)

    -- get num values from stack
    local num = num or 1

    -- return table
    local entries = {}

    -- get values into entries
    for i = 1, num do
      -- get last entry
      if #self._et ~= 0 then
        table.insert(entries, self._et[#self._et])
        -- remove last value
        table.remove(self._et)
      else
        break
      end
    end
    -- return unpacked entries
    return unpack(entries)
  end

  -- get entries
  function t:getn()
    return #self._et
  end

  -- list values
  function t:list()
    for i,v in pairs(self._et) do
      print(i, v)
    end
  end
  return t
end

For the majority of people out there, you’re probably going to end up copying this or developing an implementation that isn’t as good and only you can read. Either way, for a audience that is predominantly young or new to the platform, hell, even experienced users will have trouble with this. Simplifying the process by providing it in the standard library is a much friendlier approach than implementing this yourself.

For Python, you can use collections which is part of the standard library:

-- Quick example
from collections import deque

queue = deque([])
queue.append('a')
queue.append('b')
queue.append('c')

# queue = [a, b, c]
queue.popleft() # 'a' is popped

# queue = [b, c]

It would save a load of time if you could just do something like:

-- Basic examples

-- Queues
local queue = Queue.new() or Queue.new({'a', 'b', 'c'})
queue.pop(0)
queue.popleft()
queue.pushright()

-- Stack
local stack = Stack.new() or Stack.new({'a', 'b', 'c'})
stack.pop()

Slap a couple wiki articles somewhere on the wiki site and problem solved.

Edit: For luau, the most radical step would be to add arraylists / lists and you could easily use that. It wouldn’t be lua-like though.

1 Like
#5

Back here today because I’m trying to sort a table and that’s annoyingly tedious to do in Lua since there is no proper List primitive.

Trying to put game result rows in a sorted ListView.

table.sort(gameresult, function(a,b) a.Hellos < b.Hellos end)

Doesn’t work this way and I have to look it up every 2 weeks because how Lua sorts associative tables is so idiomatic and unlike any other language I use.

3 Likes
#6

Back here today because I’m tired of writing this beauty to append to a list:

ropes[#ropes + 1] = c[i]

2 Likes
#7

table.insert(ropes,c[i])

Having built in methods on the table doesn’t make sense, what happens if an index has the same name as one of these built in methods?

local t = {Append=function(...)return...end}
t:Append(1) -- does this call the built in function?
setmetatable(t,{__index={Map=function(...)return...end}})
t:Map(error) -- does this call the built in function?
local m = t.Where -- does this get nil or the built in function?

local l = {1,2,3,4}
local M = t.Append -- does this get nil or the built in function?

Tables aren’t just lists, and changing the behavior if “it looks like a list” is very confusing. The # operator doesn’t conflict with any other operations on a table, and it works with all tables (but with unexpected results).

A mostly non breaking way of doing it would be to only allow it in the form table:operation(...) and if table.operation is nil, as that would currently generate an error. This would break the rule that a:b(...) is equivalent to a.b(a,...) where a is evaluated once, and would probably be confusing.

If you want these operations, a function which adds a metatable to the table could be used:

Example 
local mt = {
	__index = setmetatable({
		Append = function(t,v)
			table.insert(t,v)
		end
		-- ...
	},{__index=table}),
	__tostring = function(t)
		return "["..t:concat", ".."]"
	end
}
local function array(t)
	return setmetatable(t,mt)
end
local t = array{1,2,3,4}
print(t) --> [1, 2, 3, 4]
t:Append(5)
print(t) --> [1, 2, 3, 4, 5]
t:insert(6)
print(t) --> [1, 2, 3, 4, 5, 6]
t:move(1,6,7)
print(t) --> [1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6]
t:sort()
print(t) --> [1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6]

I dislike functions like Map in Lua: using varargs from the outer function, yielding, and returning from the outer function are all harder. Stopping the loop is also weird, usually implemented by returning a non nil value. These functions will almost always end up as slower because of the extra function calls and common extra allocations from passing temporary functions (the recent optimizations only fix some cases).

If these were added, the functions that return tables couldn’t be easily changed to return these types because some uses may modify them in ways that the news types wouldn’t allow (metatables/different indices than those allowed in arrays).

2 Likes
#8

To be clear, I’d like some collection classes to use that are not raw tables.

2 Likes
#9

Providing collection classes out of the box seems handy. It’s yet another thing I’ve written personally, for example, I have Stack/Queue modules that provide behavior, and I have an extension to table (unfortunately without method support, only static call e.g. only table.where instead of someTable:where).

The thing is, I don’t feel that the use cases on a global perspective are strong enough. While yes, they are extremely handy where necessary, and are a standard across many languages, I don’t think the feature satisfies their use case requirements for Luau (it’s all on the GitHub). I’d personally love to have native support for all this stuff I’ve written but I’m doubtful we’ll actually see it.