Best way to send large amounts of data (bytecode to be specific) to the client

I am currently working on a system that provides the ability to create and distribute a product for other games to use. Product scripts are loaded dynamically from our servers.

I need to add support for client scripts, so I need to figure out the best way to send the bytecode (what is executed) to the client. The bytecode gets encoded as base64 before being sent, and my current plan is to split it up into packets. Base64 doesn’t tend to compress well, so I assume my best bet is sending it as packets like I just said.

Executing the bytecode on the client is secure because I am using a custom Lua VM. That way, someone with a low-level exploit won’t be able to gain higher-level execution on the client without understanding our bytecode structure (which I plan to be random every time).

Just wanted to get people’s input on how the bytecode should get sent for speed and efficiency.

2 Likes

Your best bet is to just compress the data as much as you can and then send it to the client :slight_smile:

2 Likes

Like I said in the OP - Base64 doesn’t compress well and can sometimes make it take up more data.

1 Like

Base64 isn’t designed for compression, but rather to safely transport bytes/non alphanumeric characters. You should rather use a compression algorithm made specifically for the purpose of making big data smaller, and eventually combine it with base64.

1 Like

Which is exactly why I’m using it. This is why I’m guessing the best option is to split it into packets and reassemble it on the clients end.

1 Like

Pardon my asking but have you thought about using json?

1 Like

JSON wouldn’t help with this unfortunately.

Are you just trying to send over the pure Bytecode? How much different is your custom Bytecode from stock Lua? If you are really trying to do proper compression, structure of your data is vital to how compressible it is going to be. If you don’t care too much about having the best compression, just use zlib or some similar widely used compression format.

2 Likes