As a Roblox developer, it is currently too hard to process certain formats or utilize certain algorithms, such as the
If Roblox is able to address this issue, it would improve my development experience because it would allow for all of the benefits of using compression we may never see ported like LZ4.
Below is a very detailed explanation of just how big of an impact native compression support and/or support for processing
rbxms or other formats might have, I really don’t know if many will read it, but, I wrote it regardless. It very thoroughly goes through a lot of use cases of having compression and access to the
rbxm format, and, reading it all isn’t really necessary, I’m just passionate about it, being a nodejs dev and someone who likes Rojo a lot.
Summary: Rojo, node, and all kinds of modern web services, programming languages, and libraries utilize formats that are impossible to use on Roblox due to the resources needed to port some of them. Being able to use some common compression algorithms (and especially having native support for
rbxms) would unlock enormous performance benefits compared to some of the most performant options out there on Roblox, and lots of new potential for devs at almost no cost to Roblox. The work has been done by those who wrote the libraries.
I think native support for compression is long overdue, if you want to interact with web services, or use most file types on the web, compression is an absolute must, and, that’s a severely limiting factor for people who aren’t doing stuff on Roblox.
Compression should be something in the engine, and, its something that was touched on in the past by Roblox staff but afaik its never been publicly concluded whether or not this could be put in the engine, I haven’t been able to find a place where this was explained.
The amount of use cases having compression algorithms or the ability to load
rbxmx content directly are endless and if there are security concerns for this, I think they need to be mentioned again if they haven’t been, in a place they’ll be more easy to find.
With the sunsetting of
:SaveInstance there will now be no more native way of saving and loading instances for real from developer code. What reason is there not to expose the ability to go from a string containing
rbxm data to an instance
Instance to developers? What reason is there not to at least expose
LZ4 to developers so they can utilize the
rbxm format with a lua port?
While developers can technically do all of the instance data storage themselves already, concluding it at that contradicts the fact that there is a format for it in the first place. The
rbxm format is by very definition designed to store instances. Its faster than what developers will ever create when used natively, its more compact than anything else that’s available with all of the effort that’s been put in to processing the
rbxm format, and its portable.
There are also a lot of things you can’t do from lua code that access to the
rbxm format could open up, for example, if developers enable
loadstring on ServerScriptService they could be allowed to load code too. This is something currently impossible in the engine.
There are even more benefits to the
rbxm format that as developers we just completely lack, especially around script stuff. Being able to patch scripts before the game loads is not something we can do but its something necessary for some modern programming languages and libraries. Take typescript for example. A lot of
npm modules too. A lot of things modify code through code, its a really useful thing to be able to do because it allows you to transcend the language.
Currently, its impossible to get 100% of the data 100% perfectly out of Roblox in a form that can be read and modified. Rojo is a perfect example of this. Rojo has lots of issues and bugs around syncing still, and, there are so many things it just can’t sync. Rojo would be a perfect example of a tool that would benefit from being able to utilize the
rbxm format in engine.
For me, I would use the
rbxm format for performance reasons. I can’t feasibly not use the
rbxm format because I need to create and store so many instances that even if I take all of the best compression algorithms under my tool belt and all of the most perfect, well optimized lua code I can come up with, I don’t even come close to creating something as compact, as accurate, or as fast as what I could with the
rbxm format. All of my efforts in the area are effectively useless, and, I feel taunted by the format because its already in the engine, utilized by CoreScripts, and utilized by so many things, but, I just can’t have access to it, and, that’s somewhat of a frustrating thing.
Being able to use the zip format would allow developers to import and export lots and lots of data from outside sources, either over the web, or just in general. Being able to use zip files would allow for me to simulate a file system in cases where I want to emulate, for example, vanilla lua because I am using a vanilla lua module that needs it. I have to resort to hacky solutions and add messy, pointless code that hard codes things and makes it hard to swap out.
Some web services can be used to download or generate lots of data that would be useful in the engine, and, being able to use the zip format to access this data is currently not possible.
The zip format is designed to store lots of files in a compact way, and, I think it would be useful for Roblox, because it would allow for much, much larger, more complex games, and would open the door for many more new projects along the lines of Rojo.
Being able to utilize both
rbxms and zips would be the holy grail of Roblox features because it would allow for a perfect way to generate a massive amount of instance data from outside of Roblox, and then take all of those instances and performantly and quickly load them all in with other bits and pieces of data and metadata that could be included. Again, Rojo is a prime example of something that could benefit from this. Externally aided Roblox development could be expanded even further and Roblox could see so many new things.
Really all it takes is access to a few compression algorithms, and, developers will do the rest of the porting. The
rbxm format is well known, and, lots of tools for it have already been made in JS, C#, etc. If Roblox doesn’t end up adding native support for
rbxms, adding support for common compression algorithms would ultimately fill a lot of use cases.
Being able to process zips,
rbxms, and all kinds of other formats that use compression would unlock such a wide variety of things that its strange to me that not much has been spoken of on the topic of natively supporting compression for years.