Will we ever get something similar to the ffi of LuaJIT?
I want it specifically for statically typed long arrays for both better performance and memory management.
Something like;
local hash = ffi.new("uint8_t[]", 256*256*256) --16 MB big table, 256 MB big in Luau
for creating a hashmap for making a minecraft-like game with voxels where each point could be one of 256 different voxel types.
Memory Benchmarks
local size = 256*256
local ramt = collectgarbage("count")*1024 + 16 --16 byte overhead
local ffi = require"ffi"
local hash = ffi.new("uint8_t[?]", size)
print((collectgarbage("count")*1024 - ramt) / size) --1
local size = 256*256
local ramt = collectgarbage("count")*1024 + 64 --64 byte overhead
local hash = {}
for i = 1,size do
hash[i]=0
end
print((collectgarbage("count")*1024 - ramt) / size) --8 times more space than required
local size = 256*256
local ramt = collectgarbage("count")*1024 + 64 --64 byte overhead
local hash = {}
for i = 1,size do
hash[i]=0
end
print((collectgarbage("count")*1024 - ramt) / size) --16 times more space than required
With Luau, I use 16 times more memory than I be would from using LuaJIT with ffi.
That’s only the memory optimization. Accessing that array is also much faster since LuaJIT doesn’t need to do a typecheck for each access and can atomically optimize the operations.
Speed Benchmarks
local size = 256*256*256
local ffi = require"ffi"
local tick do --Code for a LuaJIT tick() function
ffi.cdef [[typedef unsigned long DWORD, *PDWORD, *LPDWORD;
typedef struct _FILETIME {
DWORD dwLowDateTime;
DWORD dwHighDateTime;
} FILETIME, *PFILETIME;
void GetSystemTimeAsFileTime ( FILETIME* );]]
local ft = ffi.new ("FILETIME[1]")
tick = function()
ffi.C.GetSystemTimeAsFileTime(ft)
return tonumber(ft[0].dwLowDateTime)/1e7 + tonumber(ft[0].dwHighDateTime)*(2^32/1e7) - 11644473600
end
end
local hash = ffi.new("uint8_t[?]", size)
for i = 0,size-1 do --Because C arrays start from 0
hash[i]=i%127
end
local t = tick()
for i = 0,size-1 do
hash[i] = hash[i]*2
end
print(tick()-t) --0.01 seconds
local size = 256*256*256
local ffi = require"ffi"
local tick do --Code for a LuaJIT tick() function
ffi.cdef [[typedef unsigned long DWORD, *PDWORD, *LPDWORD;
typedef struct _FILETIME {
DWORD dwLowDateTime;
DWORD dwHighDateTime;
} FILETIME, *PFILETIME;
void GetSystemTimeAsFileTime ( FILETIME* );]]
local ft = ffi.new ("FILETIME[1]")
tick = function()
ffi.C.GetSystemTimeAsFileTime(ft)
return tonumber(ft[0].dwLowDateTime)/1e7 + tonumber(ft[0].dwHighDateTime)*(2^32/1e7) - 11644473600
end
end
local hash = {}
for i = 1,size do
hash[i]=i%127
end
local t = tick()
for i = 1,size do
hash[i] = hash[i]*2
end
print(tick()-t) --0.02 seconds (twice as slower than with ffi)
local size = 256*256*256
local hash = {}
for i = 1,size do
hash[i]=i%127
end
local t = tick()
local total = 0
for i = 1,size do
hash[i] = hash[i]*2
end
print(tick()-t) --0.36 seconds
Luau is 36 times slower than LuaJIT with ffi and 18 times slower without ffi in this benchmark example on my i5 3570. The potential of optimizations for Luau are still great and some library like ffi would be a great addition to it. It’s not only but custom structs with custom metamethods in long arrays would be even more efficient.