How to safely convert Enums to numbers for datastore saving?

[NyrionDev (Nyrion)]

Hello! I’m facing the problem of having to save a lot of information inside a single datastore key(my use case is similar to that of a neighborhood where each user has his own house but all the houses must load when a random user joins the server, but much simpler). One of the very effective methods of data compression I have found so far is converting numbers to base93(datastores support 128 characters from 0 to 127, the not control character ones are 95 and I use 2 for splitting information so 93). Anyways what I’m facing as an obstacle currently is saving Enums without having to save the entire "Enum.Category.SomeEnum" thing in the data store.

One of the methods I thought of is to save the Enum.Category.SomeEnum.Value property to the data store, which is an integer representing the enum(the integer is like a global enum integer, not the index of the enum for that specific category so it can have a value like 1234 instead of for example 12), however, I’m not sure if that int value is a constant and can’t be changed over time if enums are removed or added to the Roblox engine. Also regarding this method, I also don’t know how to reverse the process(basically get the Enum.Category.SomeEnum from the integer) without having to iterate through all the enums which is extremely inefficient.

Another method I have thought of is saving the index of the Enum for that specific subcategory(basically the table.find(Enum.Category:GetEnumItems(), Enum.Category.SomeEnum)), however, I consider this even more vulnerable regarding the issue pointed above(of Roblox adding or removing Enums).

So what I’m looking for? I’m looking for efficient ways of saving Enums that take a small number of bytes and won’t result in wrong data being loaded if Roblox makes any changes to Enums and also a function that gets the enum from its global id(.Value) and also doesn’t on iterating over a bunch of enums. I’m looking for professional answers because I consider this an important scripting issue regarding the compression of large amounts of data that many devs may have faced, are facing, or will face in the future.

[corehimself (core)]

Hey there!

EnumValue.Value

One approach is to use the EnumValue.Value property. This integer is constant and won’t change if new enums are added or existing ones are removed. Therefore, this can be a reliable way of storing enum values in a data store.

However, as you noted, there is no built-in way to reverse the process and get the Enum.Category.SomeEnum from the integer. To overcome this limitation, you could consider creating a lookup table that maps each integer to its corresponding enum value. This lookup table can be generated once during game initialization and cached for fast access at runtime. This way, you can efficiently retrieve enum values from their integer representation without having to iterate over all the enums.

String Representation

Another approach is to use string representation of enums, which can be more human-readable and easier to work with. One option is to use the EnumValue.Name property, which returns the string name of the enum value (e.g., “SomeEnum” in “Enum.Category.SomeEnum”).

However, this approach may not be as efficient in terms of storage size, as string values can take up more space in a data store compared to integers. To optimize storage, you could consider using a custom string encoding scheme, similar to the base93 encoding you mentioned. For example, you could use a lookup table that maps each character in the encoding scheme to its corresponding enum value. This way, you can efficiently store and retrieve enum values using a compact string representation.

My Approach

Module Script:

local EnumEncoder = {}

local ENCODING_SCHEME = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz~!@#$%^&*()-_=+[{]}\\|;:'\",<.>/? "

function EnumEncoder.encode(enumValue)
    local globalId = enumValue.Value
    local encoded = ""
    while globalId > 0 do
        local remainder = globalId % 93
        encoded = ENCODING_SCHEME:sub(remainder + 1, remainder + 1) .. encoded
        globalId = math.floor(globalId / 93)
    end
    return encoded
end

function EnumEncoder.decode(encoded)
    local globalId = 0
    for i = 1, #encoded do
        local char = encoded:sub(i, i)
        local index = ENCODING_SCHEME:find(char, 1, true) - 1
        globalId = globalId + index * (93 ^ (#encoded - i))
    end
    return Enum.ByValue[globalId]
end

return EnumEncoder

To require the script

local EnumEncoder = require(script.EnumEncoder)

-- Encode an enum value
local encoded = EnumEncoder.encode(Enum.Category.SomeEnum)
print(encoded) -- prints something like "9kW%N!>"

-- Decode the encoded string back to an enum value
local decoded = EnumEncoder.decode(encoded)
print(decoded == Enum.Category.SomeEnum) -- prints true

The encode function takes an enum value and returns a string that represents the value in a compressed form. This function works by converting the global ID of the enum value to a base-93 representation using a custom character set. This allows the enum value to be represented using fewer characters than the original name string. The resulting encoded string can then be stored in the data store or passed around as needed.

The decode function takes an encoded string and returns the corresponding enum value. This function works by reversing the encoding process, converting the base-93 representation back to the global ID of the enum value, and then looking up the value using the Enum.ByValue table.

By using these functions to encode and decode enum values, you can reduce the amount of data that needs to be stored in the data store or transmitted over the network, while still being able to easily convert between enum values and their compact string representations.

[JarodOfOrbiter]

@corehimself

However, as you noted, there is no built-in way to reverse the process and get the Enum.Category.SomeEnum from the integer.

There’s not a built-in way to directly convert an integer to the EnumItem (or the name thereof), however in practice an integer can be used in the stead of an enum:

Part.Shape = Enum.PartType.Cylinder
-- same as
Part.Shape = 2

Even though it’s not so human-readable, this all you need for DataStore deserialization.

---

Also what the heck is Enum.ByValue? Are you just plagiarizing ChatGPT stuff without actually verifying that it works? And since this is for serialization purposes, you should really be storing it as an integer anyways. To reverse the integer, this is the function you need.

local function decode(integer, enumType)
    for i, v in enumType:GetEnumItems() do
        if v.Value == integer then
            return v
        end
    end
end

print(decode(2, Enum.PartType)) --> Cylinder

[NyrionDev (Nyrion)]

The approach you provided reminds me of the code I currently use for encoding and decoding numbers(using base 93) and is related to publically available code in the forum with small modifications related to base64 if I’m not mistaken, also Enum.ByValue isn’t a function. As a dev, I have to use ChatGPT frequently to find information and your post has many red flags on it(it makes me come to the conclusion that it’s made out of AI-generated parts that were created through prompts which you made manually for error correction and such). For example, your post has much of the information I had figured out myself rephrased and not much actual useful information, something that chatGPT has a tendency to do. I assume your reply is almost fully AI-generated and thus I can’t consider it a reliable source of information or mark it as a solution. Given the fact that you answered the post almost immediately, I doubt you have fact-checked any of the information you provided(like for example have proof of the fact that EnumValue.Value remains constant)

[NyrionDev (Nyrion)]

@JarodOfOrbiter

local function decode(integer, enumType)
    for i, v in enumType:GetEnumItems() do
        if v.Value == integer then
            return v
        end
    end
end

print(decode(2, Enum.PartType)) --> Cylinder

That’s pretty much the code I came up with(the iteration method). Also, I just figured out that by using 93 characters to represent numeric information numbers can be represented by only 2 digits until the number 8648 which for enums looks like a good ceiling. Now regarding the iteration method, I came up with and that you also provided, I will have to call this method very frequently(basically for each datastore entry which in reality isn’t an entry but a part of one huge entry) so I have to speed it up as much as possible if there are no ways to directly get the value, do you have any ideas? Maybe if the enums are returned sorted it could be sped up using binary search(so instead of checking 37 for materials I can check only like 8-10 each time)?

[JarodOfOrbiter]

If you’re going as far as a binary tree, you should probably just make a dictionary. It would be faster to access by key than a search tree. That said though, is there a reason you can’t use an integer like in my first example?

Also somewhat unrelated, you should probably look into string.pack over base 97 or whatever you’re using. If you’re unfamiliar with it I can provide a brief overview tomorrow.

[NyrionDev (Nyrion)]

@JarodOfOrbiter

That said though, is there a reason you can’t use an integer like in my first example?

I actually didn’t think of that, as far as I’m working with objects through scripts I feel comfortable using Enums rather than memorizing integers, however, for automated processes this isn’t needed at all. Also, the fact that it can be used in practice may be proof that they can’t change, because if Roblox did change them many of the old deprecated scripts would break apart. I’ll mark this as a solution since it solved my current issue of storing enums, however, if anyone finds a solution that includes retrieving the enum as text without iterating(or Roblox adds it as a future) they’re welcome to revive this thread in case anyone is specifically looking for that.

Also, I haven’t actually ever messed with the string.pack function, I guess I’ll have to deep further into the Roblox developer rabbit hole. Thank you