Kinda wished buffers were bit-based. I sometimes dont rlly need 8 bit buffers, but smaller than it, but maybe that kind of optimization is too extreme, either way this is gonna be the golden age of networking in roblox!
6 Likes
Regarding both of these, could we see buffer support for MessagingService? This is pretty similar to DataStore support I can imagine, it would be very cool to see. Compression & encoding designed for storage in DataStores and for transmission over MessagingService would be incredibly useful.
I can forsee a lot of interesting tech utilizing buffers over MessagingService as a capability in addition to DataStores alone, because it’d increase the bandwidth we can get and the amount of data we can theoretically send.
It would be interesting to allow cross-server spectating or match previews for example.
5 Likes
This update allows for 8 bit integers. Do you know the optimisation capabilities here? This has to be the best update to come from Roblox that I have ever encountered. It also allows for over 200 variables (technically), since you can use the offset to read and write to different variables. This. Is. Epic.
4 Likes
Are there plans to add API functions for reading/writing varints to buffers (buffer.readvarint/buffer.writevarint)? I feel this would be a very simple solution to people’s compression demands. While implementing these functions on our own is possible, the algorithm is probably not straightforward to most people and a native implementation would be much faster considering Luau lacks bitwise operations.
5 Likes
It’s a value that we can reconfigure to find the best balance, right now we use 1 (the range is -7…22).
If it’s popular enough we might consider adding it.
We have a fix for this ready, but to reduce risk when multiple features are released, we are only going to enable it on Monday.
11 Likes
Please allow us to manipulate single bits. Some functions to read and write numbers of arbitrary bit size would go along nicely with that. Even if hardware doesnt support it, it’s useful. That extra freedom would be great, but it’s also worth noting that this would lend itself to more dynamic code. The current read/write functions are good but I don’t always want bit size hardcoded.
5 Likes
I don’t mean to ask this in a passive aggressive way, but does “cannot currently” imply that attribute implementation is in-the-works or has attribute compatibility been banished to the backburner?
1 Like
I knew about this update already, I was more interested in creating audio with a script too!
Are there plans to allow for buffer.fill to be able to use values larger than 8 bits?
I’m writing a compression utility that needs to be able to write individual bits, my current approach is to split the bits in N bytes and write each byte in a loop, however I had the idea to use buffer.fill in order to fill in the bits all at once getting rid of the loop but to my disappointment buffer.fill only uses the first 8 bits of the value passed to it.
Attribute implementation is not even in the backburner, there are no plans to add it right now.
I guess ‘currently’ was a bad choice of word to use there.
No.
Filling custom repeated sequences of bytes can be implemented similar to how string.rep works: if the pattern is 4 bytes, first you copy 4 bytes to make 8, then you copy 8 to make 16 and so on. Only a few buffer.copy operations are required to fill megabytes of data this way.
1 Like
I just successfully migrated from the BitBuffer module to this luau’s buffer for my network library.
Thank you Luau team. I hope this improves performance.
Server code:
Net.start("test")
Net.writeTimestamp()
Net.writeUInt32(12345)
Net.writeTable({
abc=true,
roblox=098765
})
Net.broadcast()
Client code:
Net.onReceive("test",function()
warn("delay man:", Net.getServerTime()-Net.readTimestamp())
warn("number:",Net.readUInt32())
warn("table:",Net.readTable())
end)
Client output:
1 Like
What use do you have for writing arbitrarily sized integers that aren’t better served with e.g. native i24/u24 support or native bit-level support?
I am genuinely curious because I’ve never ran into a need for an integer whose width wasn’t a multiple of 8.
1 Like
This issue has been fixed. (some characters)
4 Likes
This is GREAT
I saw it on the list today in my studio. This is going to work wonders for sending positional data with mixed precision if that is to become possible.
I do REALLY agree with the others here, we need more control over the buffers. In unity for example we have complete control over what we put in there. This is the best part about it, please don’t leave it out!
This is an amazing feature, especially for emulation or WebAssembly.
Also, it does account for the system’s memory limits.
--!native
local buffers = {}
for i = 1, 100 do
table.insert(buffers, buffer.create(1073741824))
task.wait(0.5)
end
Edit: In production, this will crash the server.
2 Likes
Here’s a couple use cases for arbitrary bit length operations:
- Replicating a list of boolean values. In my position replication system, I replicate a list of booleans showing whether an object moved during a tick. Since a bit is basically a boolean, I compress every boolean into a single bit.
- Replicating arbitrary precision values. For rotation, 8 bits only gives 255 degrees of rotation, which is noticeably imprecise without even looking very closely. I like to use 10 bits for rotation which gives four times as much precision.
this is really useful but how long until its usable in public games?
1 Like
Hello! Loving buffers so far.
This is somewhat unrelated, but even with buffers and native code me and a friend still can’t get zlib deflate to run fast in Luau, especially for larger data 
Very big issue when sending large chunks of data eg. what you’d be using with editable meshes/images.
There was a post a while back about potentially exposing native compression/decompression functions to Luau. Are there any plans to revisit that, now that Roblox is giving far more flexibility and power to developers?
Being able to not have to reimplement these algorithms ourselves (and avoiding unavoidable performance losses in the process) would be a great boon to developers, especially in the current year where power users are able to do more and more with Luau as time goes on.
2 Likes
I actually really want 24-bit functionality.
The reason why is because I might want to experiment and try writing my own character mover and simple physics engine from scratch and custom replication.
And to replicate data between clients I want to use the smallest amount of data possible.
Vectors use about 12 - 13 bytes of data to replicate through remote events.
By using 24-bit precision I can reduce it to about 9 bytes or 8 bytes if I use 16-bit precision for height.
32-bit numbers use too much memory and 16-bit numbers are too small.
I only need a few decimals of precision and I know how large my maps and environments will be so I want to use 24-bit precision.
This will save lots of bandwidth when there are lots of characters wandering around.
Having bit-level control would benefit me a lot since sometimes I can work with numbers smaller than 8-bit.
I don’t need a full byte if the largest number I need to store or replicate is only going to have a range from 1 to 10.
Having 24-bit read/write operations for buffers would benefit me greatly.
Having bit-level manipulation would be even better.
2 Likes