What do you want to achieve? Keep it simple and clear!
i want save massive data to datastore
What is the issue? Include screenshots / videos if possible!
i cant save data because of 4MB Limit
What solutions have you tried so far? Did you look for solutions on the Creator Hub?
I confirmed that storing data as a buffer results in compression, and after applying it, it worked well. However, when the data to be stored lacks patterns and has low compressibility, saving still fails — which remains an unresolved issue.
I also considered using multiple DataStore keys (2 to 8 keys), but I hesitated to implement it because it increases the chances of load or save failures, and there’s a higher risk of data corruption.
Would using multiple DataStore keys be a reliable approach?
it has the same reliability as long as you dont spam all of them at the same time
you could load one and then wait for a few seconds according to these datastore limits
if it fails then you can call it again using exponential backoff (wait something like 1 second and then double each time it fails)
The question of using multiple DataStore entries is less a question of reliability and more a question of what other choice is there. Beyond setting up something third-party, there really isn’t, but it does indeed increase the complexity of working with save data.
Beyond sharding, you can evaluate if you’re really only saving data that needs to be saved. Without any insight regarding the nature of your data, I can’t provide any recommendations currently, but there may be ways to discard unnecessary data and reduce the need for sharding in the first place.
Also, for what it’s worth, buffer.tostring generates a byte string that is not compatible with DataStores because it uses invalid characters (bytes in the range 128-255). You will need to encode the data in a different format in order to save it if you aren’t already.