I wouldn’t send the JSON, then you have the overhead of the keys, strings, encoded numbers, etc.
Luau just backported string.pack/unpack from Lua 5.3, so binary-encoding data shouldn’t be too bad.
2 Likes
Ooh sounds interesting. I’m not familiar with the different formats, which one is most preferable for universal use? If that is impossible what format is useful for what purpose? How do I check it’s effectiveness?
1 Like
Well, first I would try to get something working with JSON first honestly. I know I said not to… but the hard part of this is going to be correctly syncing up movements at all.
Then, once you get it working with JSON, you can look into places to optimize, like using string.pack. As for the formats, there’s no universal format. You tell it "I want these numbers encoded with these types of bytes (i.e. char, uint32, float, double, etc.), and it spits out a string whos characters (which themselves are just bytes) represent those numbers. Then you can turn that string-of-bytes back into numbers again on the other side.
1 Like
Played around and it seems like floats are the best option for any Roblox replicating: only 4 characters, accurate enough for replication purposes, and I imagine its faster than any compression algorithm Lua can make due to being in C++.
Some small notes:
- numbers in lua are usually doubles (8 chars), but some members of roblox objects (vector3 components for example) are indeed floats or ints
- for some things that you don’t need that much precision for, you can probably get away with storing it in individual bytes (for instance, do you need more than 256 values for the direction a player is facing for rendering purposes?)
- for integers, use a integer type. The uint32 type (or
"I4"in string.pack terms) gives you exact representation of integers up to 4,294,967,295, while the float type stops being exact for integers at 16,777,217
2 Likes
Next I need to test out how much data I can transfer without any performance issues, assuming that each character is 1 byte and that 30 players are present.
How should I measure performance?
Yes but simply testing it would be difficult. You can’t imitate 30 players on a single client due to remote client limits. You also cannot simulate 30 clients in Studio for this because the server is hosted locally and it isn’t a fair test.
Oh gotcha. You could try to use something like clumsy to make localhost slower, not sure if that’s what you’re looking for though. It’s a good question though. I’m not sure how you’d go about getting stress test numbers for something like this.
1 Like
This isn’t enough info but I think that 25% of this should be a safe maximum to aim for.
Well, if you’re not sending physics data, and you’re smart about how often you send data, I suppose that’s doable. As long as that limit’s per-player, anyways.
1 Like
It is indeed per client. Assuming that you want to stay within 50% of the soft cap and you’re running at 60 FPS, you’re limited to 416 characters per replicated frame. Clearly, JSON is not allowed here. A lot of compression needs to be done to efficiently pack all the data into here.
2 Likes
For some reason, in practice, it took about 1000 characters per frame in order to get to 25kb/s. Not sure why.
Just a correction, I believe this measurement is actually 50 kilobits per second not kilobytes (outstanding) at least from testing this is where I seemed to get into throttling… That’s a whopping 8 times slower than I originally thought. At most you can fit about 106 bytes per network frame if this is in fact correct.
Generally, I have now taken into consideration number of requests more so than this theoretical limit because as far as I can tell Roblox will not actually spread out a request over more than one network frame even if its far past the limit. I think throttling comes down to future requests at that point? This leads to more CPU usage on the client for decoding, but, at this point I’m beginning to believe knowing there is a 50kbps limit is no longer helpful.
Generally, you’ll tend to see better network performance under the following conditions (from my testing at least):
- Strings. Strings sort of seem to bypass most of the CPU required for decoding. This makes them ideal if you want to introduce your own remote encoding as well. This is something I wasn’t aware of until recently.
- Minimizing the number of remote requests.
A little piece of useful info that might come in handy: RemoteFunctions will take approximately two network frames to complete a request no matter what. If you send a remote request on the server and have the client simply use an empty callback, basically, it will take one network frame to send the call to the client. It will then take an additional network frame (on the client) for the client to indicate a return.
Then this makes my test have even stranger results. Why would sending a 1000 character string 60 times a second to the server result in only 25kb/s traffic? I ran the test in studio and in-game, same results.
I have no clue, that’s a big reason why I’ve given up on that area. It seems that no matter how much testing I would do I would always end up finding inconsistencies under different circumstances and it honestly makes no sense to me. There are plenty of things you think would be a good measurement, and, they look to match each other, but then you start testing more things and you get wildly different results from your different methods of measurement.
1 Like
If I had to guess, the netcode is very complicated, case-specific, and changes constantly, which is probably why roblox doesn’t document it anywhere. Wish they would, but what can ya do.
2 Likes
The data I sent over consisted of 250 random float values (-10k to +10k, not whole) packed with string.pack(). Switching this over with a string of just the letter A 1000 times made no difference.
1 Like
hey i’m like 4 years late but did any of you guys make it work?
1 Like
See Chickynoid, this is basically a custom replication module founded on the principles of server-authoritative roll-back netcode. Unfortunately it has non-existent documentation and hasn’t been updated in years, but it’s an amazing tool nonetheless.
1 Like