Cannot store string in data store. Data stores can only accept valid UTF-8 characters

[IAmBanFor_Devforum (DevforumAccount)]

Some background

I'm currently working on a project that allow players to create games in older time periods of Roblox (Much like RetroStudio).

To store the user generated games without making new keys for the values, I use [BitBuffer](https://github.com/dekkonot/bitbuffer/). (Not to be confused with https://devforum.roblox.com/t/stravants-bitbuffer-module-compact-storage-of-data-tutorial/20189 )

Problem

The main problem I currently have is the way I store the information, it being DataStores.

BitBuffer doesn't output UTF-8 characters. DataStores only accepts UTF-8 characters.

Resulting in a rather, unfavourable state right now. Being that I can't save data without using HTTPService and a Server.

Code and Images

The BitBuffer module: BitBuffer

DataHandler module:

-- We don't need too much bytes for game information!
-- Just cut it down!

local BitBuffer = require(script.Parent:WaitForChild("BitBuffer"))
local BOOL_TO_BIT = { [true] = 1, [false] = 0 }
local BIT_TO_BOOL = { [1] = true, [0] = false }
local DataHandler = {}

DataHandler.Store = function(AllowCustomAvatars, IsSpecial, Name, Version, DataStoreBuffer, DataStoreBufferName)
	local Buffer = BitBuffer()
	
	Buffer.writeUInt8(BOOL_TO_BIT[AllowCustomAvatars])
	Buffer.writeUInt8(BOOL_TO_BIT[IsSpecial])
	Buffer.writeString(Name)
	Buffer.writeUInt16(Version)
	
	local Output = Buffer.dumpString()
	DataStoreBuffer:SetAsync(DataStoreBufferName, Output)
	return Output
end

DataHandler.Restore = function(DataStoreBuffer, DataStoreBufferName, FileContents)
	if not FileContents then
		print("Global restoring mode.")
		FileContents = DataStoreBuffer:GetAsync(DataStoreBufferName)
	else
		print("Local restoring mode.")
		warn("WARNING: Your DATA won't save! This is because we're in local restoring mode.")
	end
	local Buffer = BitBuffer(FileContents)
	local data = {}
	
	data.AllowCustomAvatars = BIT_TO_BOOL[Buffer:readInt8()]
	data.IsSpecial = BIT_TO_BOOL[Buffer:readInt8()]
	data.Name = Buffer:readString()
	data.Version = Buffer:readInt16()
	
	return data
end

return DataHandler

Testing script:

-- Testing!!!!
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local Game = ReplicatedStorage:WaitForChild("Game")
local HandlerFolder = Game:WaitForChild("Handler")
local DataHandler = require(HandlerFolder:WaitForChild("DataHandler"))
local DataStore = game:GetService("DataStoreService")
local DataStoreBuffer = DataStore:GetDataStore("CVENV_Game")


game.Players.PlayerAdded:Connect(function(plr)
	local FileStuff = DataHandler.Store(true, true, "random", math.random(0000, 3000), DataStoreBuffer, plr.Name)

	wait(7) -- Yes I know, but I need the delay
	print(DataHandler.Restore(DataStoreBuffer, plr.Name))
end)

Game folder image:

Error message image:

[Wigglyaa]

This might defeat the purpose of your exercise… but, can you use JSONencode on the data before sending to the datastore.

[IAmBanFor_Devforum (DevforumAccount)]

Now I get Can’t convert to JSON because the output is not a table.

[Wigglyaa]

Try wrapping it in a table? {}
Hope this helps, must reach word limit

[IAmBanFor_Devforum (DevforumAccount)]

I need to convert it back to the original character set. Not UTF-8 or else it doesn’t work.
Error: BitBuffer.readInt8 cannot read past the end of the stream

[Wigglyaa]

JSONdecode will return the original contents, although inside a table.
If this is a single string, passing the first table index should be sufficient.
Else you may need to loop the table.

[IAmBanFor_Devforum (DevforumAccount)]

Turns out I was using JSONEncode to decode, thanks a lot!