How do I go about creating a public liking system with a single datastore?

[Ethanthegrand14 (Ethanthegrand)]

So, for context, I have a game where you mess around in a 2D sandbox environment, and I want users to be able to publicly upload creations to a public list.

The way how this datastore system currently works, is that the store has keys represented by the page, which means each page can have up to 10 creations max.

But one thing I would like to add is a liking/voting system where users can click on one of these creations and a like to the datastore.

I have already tried this, but there were issues with multiple people liking/unliking the creation which caused things like inaccurate numbers and negative numbers.

How should I go about doing this? And also will I have to worry about data loss to players being able to write to a same public datastore key?

local function LoadFeaturedCreationsData(Player, Page, Order) -- Get community creations
	local LevelSaves

	local Success, ErrorMessage = pcall(function()
		local PageReversed = FeaturedPagesUsed() - Page + 1
		
		if Order == 0 then
			LevelSaves = CreationsStore:GetAsync(Page .. FeaturedCreationsKey)
		else	
			LevelSaves = CreationsStore:GetAsync(PageReversed .. FeaturedCreationsKey)
		end
		
		if not LevelSaves then
			LevelSaves = {}
		end
		
		for i, LevelInfo in pairs(LevelSaves) do
			LevelInfo.TrueIndex = i
			
			if Order == 1 then
				LevelInfo.TruePage = PageReversed
			else
				LevelInfo.TruePage = Page
			end
			
			--LevelInfo.LikedByUser = CheckIfLiked(Player, LevelInfo.CreationId)
		end
		
		if LevelSaves and Order == 1 then -- Newest first
			table.sort(LevelSaves, Pred)
		elseif LevelSaves then
			table.sort(LevelSaves, PredReversed)
		end
	end)

	if Success then
		print("Successfully loaded featured creations data")
		return {Success = true, Data = LevelSaves}
	else
		warn("Failed to load featured creations data: " .. ErrorMessage)
		return {Success = false, Data = "Failed to load data: " .. ErrorMessage}
	end
end

[7z99 (cody)]

What were you doing to increment the data store? I believe if you use UpdateAsync and mutate the likes data inside that mutator function (I’d advise doing it in a batch, ie upon the first request, wait 2 or so minutes, meanwhile for every new like add to a variable to see how many likes/dislikes should be added, then after those 2 minutes call UpdateAsync and increment the value accordingly as to not reach a data store request limit), it would solve the issues related to likes not being accounted for since UpdateAsync calls are called sequentially regardless of server UpdateAsync is called in.

[Ethanthegrand14 (Ethanthegrand)]

I… don’t think im incrementing it at all. I’m really just rewriting the datastore every time a creation get’s uploaded, which I know is bad, but I really don’t know how else i’d do it. Heck, i don’t even know how to use UpdateAsync really, I only just learnt about it today

This is the current code I am using to write to the datastore.

local function PublishFeaturedCreation(
	Player, CellsData, LevelName, LevelCreationDate, AmbientTemperature, LevelDescription 
)

	--if Player.UserId == game.CreatorId or game.CreatorId == 0 then  -- DEV ONLY
	local LevelsDataTable
	local NewLevelInfo

	local Page = 1

	local FeaturedCreationId = 0

	local Success, ErrorMessage = pcall(function()
		NewLevelInfo = {
			Name = LevelName, 
			CreationDate = LevelCreationDate,
			CellsData = CellsData,
			TimeCreated = os.time(),
			AmbientTemperature = AmbientTemperature,
			LevelCreator = Player.Name,
			LevelDescription = LevelDescription,
			Likes = 0,
		}

		for i = 1, FeaturedCreationsPages do
			LevelsDataTable = CreationsStore:GetAsync(Page .. FeaturedCreationsKey)

			if not LevelsDataTable or #LevelsDataTable < FeaturedCreationsPageSize then
				break -- We have found an available page
			else
				Page = i
			end
		end

		local LatestId = CreationsStore:GetAsync(FeaturedCreationsIdCountKey)

		if LatestId then
			NewLevelInfo.CreationId = LatestId + 1
			CreationsStore:SetAsync(FeaturedCreationsIdCountKey, LatestId + 1) -- A new ID has been made
		else
			NewLevelInfo.CreationId = 1
			CreationsStore:SetAsync(FeaturedCreationsIdCountKey, 1) -- Initiate data
		end

		CreationsStore:SetAsync(PagesUsedDatastoreKey, Page)
	end)

	if Success then
		if not LevelsDataTable then
			LevelsDataTable = {}
		end

		table.insert(LevelsDataTable, NewLevelInfo)

		table.sort(LevelsDataTable, PredReversed)
		
		CreationsStore:UpdateAsync(Page .. FeaturedCreationsKey, function(OldData)
			return LevelsDataTable
		end)

		print("Successfully saved " .. LevelName)
		return true
	else
		warn("Failed to get data: " .. ErrorMessage)
		return false
	end
	--end
end

Oh, I also should have mentioned that this game is single player and these likes are cross-server and global. So literally everything needs to be done with datastore

[7z99 (cody)]

Gotcha. If it’s singleplayer and one person can only send one like or one dislike per server then you don’t need to worry about doing things in batches, you should be able to do things on the fly.

There aren’t too many functions that exist that take mutator functions but if you’ve ever used the comp function of table.sort it’s something like that (in that the returned value dictates how to proceed). In the case of :UpdateAsync, instead of returning a boolean you’d return the new value that should be updated in the key.

The benefit of using :UpdateAsync in this case is that each call is queued and done in order globally. You can also use this as a replacement of :GetAsync to ensure the latest version of the data is loaded.

So for liking, you’d do something like:

likeEvent.OnServerEvent:Connect(function(player, levelInfo, liked) -- levelInfo would be some identifier to know which creation they wanted to like or dislike. liked would be a boolean, if true they liked, if false they disliked
    -- get data store key for this level
    local key = getKeyForLevel(levelInfo)

    local success, data, dataStoreKeyInfo = pcall(CreationsStore.UpdateAsync, CreationsStore, key, function(oldData)
        -- in this case, oldData is equal to the table of LevelInfo (so {Name = levelName, CreationDate = creationDate...} and so on for each entry. this is the latest version of the data obtained from the data store
        -- now we mutate the oldData to add or subtract 1 from its Likes entry:
        oldData.Likes += if liked then 1 else -1
        return oldData -- now once the request finishes, the latest version of the data will now be equal to what the previous version's like is, but incremented or decremented
    end)

    -- if you need it, data is equal to the latest version of the data (same as oldData inside of the mutator function), dataStoreKeyInfo is the DataStoreKeyInfo for the key. if request fails then typical rules of pcall apply

    -- continue and do the rest of the function if needed

[Ethanthegrand14 (Ethanthegrand)]

Thanks! This should work great.

Also slightly off-topic, but I am absolutely having trouble with trying to get multiple pages working for my datastore. It’s barely works and uses like 3 datastore keys to make it work.

local PublicCreationsDatastoreKey = "_CommunityCreations" -- This is where the page index would go too (e.g. page 3 is 3_CommunityCreations)
local PagesUsedDatastoreKey = "CommunityPagesUsed" -- Save the amount of pages we have used
local PublicCreationsIdCountKey = "CommunityIdCount" -- Keep track of IDs

I really just want to rework this and make something reliable and easy to apply filters to (newest, oldest, most liked, etc)

currently, I have to go through like many many datastore keys (which are the pages), which each of them containing 10 or less creations, and i absolutely cannot sort them correctly. I have to keep track of the creations true corresponding page, index and ID, and omg sorting is a pain.

The most I have gotten is simply flipping all the pages (exaple, from 1 - 10 to 10 - 1) and the layout order of the UIs (which is hacky and makes the first page look weird, since it’s technically the last page and there can be under 10 creations there)

Is there a better way of handling this?

Also my current method cannot allow for any other type of sorting filters as I can only load one page at a time and is just naturally sorted from oldest to newest, and getting an infinte amount of pages from the datastore with keys just to get say the most liked creations, will not work.

[7z99 (cody)]

I’m a little confused as to how your sort system is laid out specifically but in my opinion the biggest factor to saving data, looking through data etc is to be conservative, save as little data as possible, under as few DataStores as possible.

This is how I’d approach it personally. It’s a bit of me winging it and an awfully long post of me brainstorming but I can clarify details if need be. Note that this probably isn’t super friendly taking into account backwards compatibility, however if you only have a few pages of levels at the moment, migrating it to the new structure should be doable.

Anyway 1, save each creation under a separate key, starting at 0 and it increments by 1 for each new level (for the sake of simplicity I’ll just call the datastore “NewestCreation” for the sake of simplicity). Each time a new level is created and is bound to be save, increment NewestCreation by 1 (by :UpdateAsync). This number would be available globally but would probably have to be requested each time a player submits a new level (so to summarize this point, NewestCreation:UpdateAsync, in your mutator function increment oldData by 1, this new value of NewestCreation will be the key under which the level is saved). This ensures no overlap of IDs as again :IncrementAsync calls are done in order globally.

Since creations are created in order, creation with id 0 is the oldest, creation with id equal to NewestCreation is the newest.

Now to sort from newest to oldest, you would start counting from whatever NewestCreation is, until you eventually hit 0 which is the oldest creation.

Same premise from sorting from oldest to newest but reverse. Start at 0, keep incrementing until you hit what ever NewestCreation is equal to.

For searching by like count, I’m thinking you could create an OrderedDataStore, key would be the level’s ID, value is the number of likes. This would have to be updated every time a player likes or dislikes a level. Do :GetSortedAsync using the ascending/descending param as which ever search order you’re trying to do.

For a fourth feature I was going to say that it could be a good idea to allow users to search by user IDs using metadata but that apparently hasn’t been implemented yet (it’s planned but you know with Roblox it’s probably going to be a good while before it’s properly implemented, but you could set metadata for future implementations that are more conservative wrt data store request limits).

Since obtaining keys by metadata doesn’t exist (yet) and if you want to implement searching by IDs, you can create a third data store to associate certain level IDs with certain user IDs. This I think would be the final data store. The key name would just be the player’s user ID, and the key’s value would be an array of numbers, each of which is equal to a key inside of CommunityCreations.

At this point, for 5 search features, (date ascending, date descending, likes ascending, likes descending, by user ID), and for easy affiliation of user IDs with creations (for GDPR requests) there are 3 data stores.

[Ethanthegrand14 (Ethanthegrand)]

Firstly, thank you so much for your time and information!

But I have a few questions,

@7z99

Anyway 1, save each creation under a separate key, starting at 0 and it increments by 1 for each new level

I thought about doing this, but wouldn’t it take like 10x longer to get? since we are requesting 10 keys per page? Is there a way to speed this up? Or am I stuck with a longer loading time?

@7z99

increment oldData by 1

I’m assuming this will assure that our uploaded creation has an ID of exactly the previous data, plus one?

Oh, and also what if say I had to delete one of the creations for moderation purposes, would this cause any issues? The IDs on the creations shouldn’t change regardless of how many creations have been deleted.

---

[7z99 (cody)]

@Ethanthegrand14

I thought about doing this, but wouldn’t it take like 10x longer to get? since we are requesting 10 keys per page? Is there a way to speed this up? Or am I stuck with a longer loading time?

It will probably be slow, yeah. I’m not sure which method you are using currently but unless you’re saving multiple creations under a single key, I’m not too sure if there’s a way to speed it up. You are still going to have to get the data from the data store regardless of any method you use. It would be useful if somehow we could batch request data but since this doesn’t exist (with the exception of OrderedDataStores), I’m not too sure what other options you could use without using external dependencies.

You could try calling :Get/UpdateAsync 10 times each in its own thread to request the data 10 times all at the same time so all 10 creations load at roughly the same time, but bear in mind that there is still a data store request limit you have to work with, and I’m also frankly not too sure specifically how fast requesting data under multiple keys is.

Another idea, you could possibly cache the data in a large master table on the server and request the likes and other relevant data that needs to be up to date from the DataStore (via :UpdateAsync) after the player requests it. Until the likes load, show a placeholder on screen like 0, underscores, dashes etc, similar to what Roblox does using chat/pending filtering (show underscores in place of characters right off the bat, then when the message finishes filtering, replace the underscores with the actual filtered characters). If calling UpdateAsync fails, show question marks or something.

@Ethanthegrand14

I’m assuming this will assure that our uploaded creation has an ID of exactly the previous data, plus one?

Yeah

@Ethanthegrand14

Oh, and also what if say I had to delete one of the creations for moderation purposes, would this cause any issues? The IDs on the creations shouldn’t change regardless of how many creations have been deleted.

I didn’t take that into account in my original post but if you have to take a creation down, set the key to nil. Then when you are going to request that id and the :Get/UpdateAsync call returns nil, skip to the next id and attempt to load that next id instead, until a creation with the next id is found. Implementing something like that would also have a secondary use, in the case that there’s a gap in any ID (such as if NewestCreation value is somehow incremented twice), it wouldn’t throw any unexpected behaviour.

Also, going back to the caching idea, you could also save in an array which keys are deleted lazily so you’d only have to make the request once. You could also take advantage of memory stores and use those for some caching behaviour between servers, but there are limits for that as well (more so size limits, not rate limits).

[Ethanthegrand14 (Ethanthegrand)]

@7z99

You could try calling :Get/UpdateAsync 10 times each in its own thread to request the data 10 times all at the same time so all 10 creations load at roughly the same time, but bear in mind that there is still a data store request limit you have to work with, and I’m also frankly not too sure specifically how fast requesting data under multiple keys is.

Is this limit per-server? or globally? And what’s the rate? like if i loaded 10 creations at the same time, and then waited like a second or two, would I be able to do it again?

@7z99

Another idea, you could possibly cache the data in a large master table on the server and request the likes and other relevant data that needs to be up to date from the DataStore (via :UpdateAsync) after the player requests it. Until the likes load, show a placeholder on screen like 0, underscores, dashes etc, similar to what Roblox does using chat/pending filtering (show underscores in place of characters right off the bat, then when the message finishes filtering, replace the underscores with the actual filtered characters). If calling UpdateAsync fails, show question marks or something.

While this sounds very optimal, this also sounds like a very complicated process. especially to someone who’s new with datastore management, and I haven’t even heard of this memory store thing.

Also I will let you know how my new datastore management goes with the per-creation keys, as I am currently working on it

[MerlyDev (Woof)]

Sand Powder game, that’s a classic one

[7z99 (cody)]

@Ethanthegrand14

Is this limit per-server? or globally? And what’s the rate? like if i loaded 10 creations at the same time, and then waited like a second or two, would I be able to do it again?

Per server. For a single player server, you have 70 read; 70 write; 7 GetSortedAsync; 7 GetVersionAsync; 7 list, and 7 RemoveAsync requests per minute.

@Ethanthegrand14

While this sounds very optimal, this also sounds like a very complicated process. especially to someone who’s new with datastore management, and I haven’t even heard of this memory store thing.

I think it’s might be more doable to implement than it sounds. Every time you call :GetAsync, check in the master table for an entry with the requested key. Should it not exist, save the result of the GetAsync to the table. Index is the key, value is result of the aforementioned call.

local cache = {} -- laid out as
--[[
{
    [key] = {
        CellsData = {...};
        Likes = 0;
        TimeCreated = 0;
        -- etc
    };
}
]]

local requestDataFunc = path.to.remote.function
local likesLoadedEvent = path.to.remote.event

local function makeRequest(requester, key, checkLikes)
    local hasKeyBeenFound = cache[key]
    
    if hasKeyBeenFound and checkLikes then
        task.spawn(function()
            local success, result = pcall(CreationsStore.GetAsync, CreationsStore, key)
            if success then cache[key] = result end -- update
            remote:FireClient(requester, success, key, if success then result.Likes else nil)
        end)
    
        -- return the data back to the caller
        return hasKeybeenFound
    end

    -- if this point has been reached, we know we need to get the data in the initial thread, so request the data
    local success, result = pcall(CreationsStore.GetAsync, CreationsStore, key)
    if success then cache[key] = result end -- add the value to the cache
    return result
end

function requestDataFunc.OnServerInvoke(player, id, checkLikes)
    local receivedData = makeRequest(player, id, checkLikes)

    local relevantData = {}
    -- add only relevant data that the client needs, as the client has to receive the information from the server which also takes time
    return relevantData -- return it back to the requesting client
end

Then on the client, assuming you have like a secondary panel to show additional info, you could do something like:

local likesLoadedEvent = path.to.remote.event
local requestDataFunc = path.to.remote.function

local currentItem = 0

showInfoButton.MouseButton1Click:Connect(function()
    currentItem = the.creations.id
    openPanel() -- replace this with whichever func is to open the panel, show placeholder labels if need be
    local relevantInfo = requestDataFunc:InvokeServer(currentItem, true)

    if not relevantInfo then -- the data was unavailable in the cache and failed to load
        label.Text = 'failed to load!'
        return
    end
    
    showCreationInfo()
end)

likesLoadedEvent.OnClientEvent:Connect(function(success, key, likesData)
    if currentItem ~= key then -- they closed the panel before the likes info for this loaded so we don't update the label
        return
    end

    likesLabel.Text = if success then likesData else '???'
end)


-- finally, for changing the page, we can just request the relevant data from the server, omitting the checkLikes param
local function onNewPage()
    local relevantData = requestDataFunc:InvokeServer()

    -- finally, update the preview (which would be the page listing)
    updatePreview()
end

If need be, you could also add a time before it requests the likes data again, so if they repeatedly click load, close, load again, and so on, the likes would return the same number up until the timer expires (so like it would only request the likes again from the data store if a new request is made after a minute or so). This could be helpful in not reaching your request limit.

Also I have to get off now, so if you need any additional info I’ll check back here when I wake up if nobody else chimes in.