As a Roblox developer, it is currently too hard to send node graphs over remotes (e.g. circular tables, or tables which show up multiple times in different places)
If Roblox is able to address this issue, it would improve my development experience because it would allow me to send very complex objects, as well as node graphs, which have tons of applications in programming.
The table references sent over a remote should be preserved so that table data is only sent once, instead of an arbitrary, potentially enormous number of times:
local tab = {abc = 123}
remoteEvent:FireServer(tab, tab, tab) -- Received as three of the same table reference
remoteEvent:FireServer(tab, tab, tab) -- Received as a new set of three of the same table reference
The problem (In more detail)
The way that tables are sent over remotes serializes every table and sub table uniquely. This means that references to duplicate tables are broken and split into different references, and circular references are impossible since it would require infinite data. This also means that sending tables over remotes potentially uses far more data than is required, and much more work is done than is necessary serializing every table.
Demonstration
Consider the following, which would be a way that common object oriented implementations would work when you send them raw over remotes:
remoteEvent:FireServer({SomeData = 123, __index=classIndex}, {SomeData = 234, __index=classIndex}, ...) -- An arbitrary amount of the same class
The classIndex table gets uniquely serialized for every object. The resulting table sent and received contains unique data for every__index field. This uses more CPU, more memory, and bloats the data sent over the remote, resulting in a boundless increase in performance cost depending on the depth & number of duplicated tables.
Fallbacks
In the case that a game is uniquely modifying two of the same tables received through remotes, code might stop behaving as expected. I would call it bad practice, but that doesn’t dismiss it.
Contrary to what you are implying, I don’t think it’s common to send objects over remotes. Normally, you send data back-and-forth and feed that to objects on the other side.
This seems like a niche issue resulting from the fact how you structure your remote payloads. I don’t want Roblox to make assumptions about my data like that.
So fix it: by not sending over the whole class, but only its relevant data for that network request. The engine can’t implement everything for you.
I assume you don’t need the whole object on the other side in order to fulfil the request anyway, so even if they make this optimization, you likely want to restructure your data anyway to reduce the pointless bandwidth usage even further.
All of this makes the assumption that my primary use for this is to send class objects or that I want to send class objects a lot, but that was just an example which demonstrates the issue in an easy to understand way.
I worded the example bit poorly, because I meant to say its a way that common object oriented data would work over remotes, not that its a common way for object oriented data to be sent.
The place where I am encountering problems is around sending arbitrary data, just in general. I am replicating class objects through a UUID, and at one point, contructor data. No extra data is sent. But the main issue is around when the constructor data is complex, and in some cases actually relies on references matching. Its creating a huge amount of bloat that is just completely unnecessary, and I’ve had to implement a serializer in front of Roblox’s to map everything to references and then map them back afterwards.
The thing about this though, is that I initially expected remotes to transmit with references for everything, just like is done with instances, except, they don’t. There’s a lot of times I’ve wanted to send circular tables, or just some big table of data one-time, but I can’t because of the performance & storage cost being too big, or that remotes will just throw an error if you try to send a circular table.
Even more frustrating, this has created some really obnoxious bugs before because I don’t feel like I need to pcall remotes, except, sometimes I do, and this has happened to me a lot before where libraries will just break because a circular table got thrown into the mix and broke everything.
The whole point of remotes is that they offer an easy way to transmit data. In the case of table references, circular tables, etc, they fall apart, and they stop working like you’d expect. The data received on the other end is not even in the same structure as the data that went in a lot of times.
Instance references only work properly when that instance is known to both the server and to the client. If you try to pass a locally created instance to the server, it’ll come out as nil on the other side.
Try to imagine how this would work with table references (instead of deep-copying them when sent over the network). How do you expose a reference to a table on the server to the client? Would this require an additional API? Would it broadcast all table references of anything on the server to the client? What are the performance and networking implications of the latter?
It doesn’t seem practical or useful to me. If you need to pass things by reference, why not store them in attributes and send over references to the instances with those attributes instead? This works as-is with the current remote implementation.
With all due respect, I’ve been programming for quite some time now, and have yet to encounter a case where circular tables are useful with the exception of very specific algorithms, and they are definitely far away from any networking use cases. It seems to me like it is indicative of code smells in your code base if you feel you have a need to send a circular table over the network. There are probably simpler solutions for your problem that wouldn’t require a significant change to how tables are networked.
Here you’ve lost me completely – if you mean they don’t take mixed tables, that’s fair, but otherwise the content of the data should be exactly what you put in if you feed an array or dictionary.
Edit: I updated the post to make it more clear I’m talking about sharing references to duplicate tables within the scope of the request itself (node graphs). If there’s more miscommunication, hopefully the changes I made to the wording clear it up.
I think what I’m saying is being misunderstood here. I am talking about internal references to things, local to the request, not global to all of Roblox. If you pass an instance from the server to the client multiple times in the same remote request, that reference is the same everywhere its received inside that request. It doesn’t change in the middle of the request, e.g. in another argument or nested in a table.
Referring to my example again:
local tab = {abc = 123}
remoteEvent:FireServer(tab, tab, tab) -- Received as three of the same table reference
remoteEvent:FireServer(tab, tab, tab) -- Received as a new set of three of the same table reference
If the event is connected to print, the output might look like this:
This comes up for me when a circular table deeply nested in functions is sent over the network. Circular tables do pop up in programming a lot, especially in object oriented programming, and there are numerous times I’ve had circular tables get sent into remotes while working on my own games, as well as working on other people’s games. Usually you aren’t going into things with the intention of sending a circular table over the network, the problem is when coincidentally you send one, and now you have errors popping up and potentially wrecking important threads. It has in the past taken me hours to track down circular tables that are getting introduced, because I don’t want to do that by code for every request I send because its hacky.
I’m talking about node graph directionality, if you have a node graph that points into the same node from a different position, it gets destroyed. What I mean is you literally can’t send a graph unless they’re a simple tree, because that’s what the serializer turns graphs into, a plain tree.
Say you have a list of possible actions, each action has metadata, and one of those actions results in a past choice, or a choice in a different location in the node graph. A simple example would be the Roblox Dialogue system, where an option can loop back to another. Well, unfortunately for you, your node graph got mutilated by the remote serializer and now you have a generic tree when you didn’t input one or expect one, hence why I said I would expect the serializer to preserve references.
Node graphs have applications in all kinds of places, including in networking, and even if they didn’t, that doesn’t mean they don’t have applications for being replicated.
If I’m understanding this correctly, the suggestion is to consolidate duplicate references to the same table value behind the scene before sending things over remotes, thereby allowing circular tables and making remotes more efficient.
Honestly, it’s a little surprising Roblox doesn’t already do this. I don’t know how high of a priority this is, but it’d definitely be nice to have.
Just to clear it up a little since I clearly didn’t communicate this well, I meant references inside the scope of the request. In other words, preserving node graphs. For example, you might have a node graph like this:
a -> b, d
b -> c, d
c -> 456
d -> 123
Or, as tables like so:
a is {b, d}
b is {c, d}
c is {456}
d is {123}
Table a, b, c, and d sent over a remote as individual arguments become
a is {{{456}, {123}}, {123}} -- Duplicates serialized separately
b is {{456}, {123}} -- Duplicates serialized separately
c is {456}
d is {123}
Each table is unique (b, c, and d are duplicated in each place they come up), thus, the node graph is destroyed. On top of that, each table is a separate section in the remote payload that’s sent to the client or server, and each one is individually allocated in lua.
That means a table that’s 64 bytes large, which shows up one million times in a remote query will be individually serialized into one million unique tables, and now the receiver theoretically has to allocate 64 megabytes of data.
What I am instead asking for is that one copy of that table be serialized in the request, so node graphs can be sent over remotes, and then that one table is referred to 1,000,000 times, resulting in 1,000,000 identical references, and one 64 byte table allocated (rather than 64 kb of copies of that table).
This means that the argument received from remoteEvent:FireServer(tab) does not == the argument received from remoteEvent:FireServer(tab) a second time, but the two arguments received from remoteEvent:FireServer(tab, tab) do == eachother.