if we’re gonna do the optimization argument then I’d bulk serialize this when sending the DataStore update, not make the buffer the format every script has to work with.
the runtime representation should be optimized around how that data is accessed, then dirty tracked. when it’s time to save, you hit the serialization boundary, and that’s where I’d actually pack it into the buffer.
if we’re talking genuinely large-scale data, you’d also separate hot data from cold data instead of pretending every field benefits from the exact same memory layout.
making every script know that offset 4 means Level isn’t some final form of optimization, you just spread the serialization format across the entire runtime.
and again, this is someone learning how DataStores work. you cannot tell me a framework is too complicated and then teach him to maintain a handwritten memory layout everywhere, let alone make up your own serializer – tsk tsk I thought better of you Yarik
so I actually benchmarked the exact thing I’m talking about in Luau Playground.
2,000,000 runtime updates with a normal table + one bulk serialization at the end took ~140.3ms.
keeping the data in a buffer for the entire runtime took ~283.2ms, so about 2.02x slower in this test.
and that’s the point I’m making: buffers are good for storage density and serialization, but that doesn’t mean the serialized representation should also be your hot runtime representation.
if the game touches the data millions of times and saves it once, I’d rather pay the packing cost once instead of paying buffer read/write costs every single mutation.
you optimized the format being sent to storage, then made the entire runtime use that format too.
very optimized Yarik
(if you don’t believe the run, re-order the test runs if you’re worried about warm-cache/order effects)