MemoryStoreQueue:RemoveAsync() doesn't remove queue item when invisibilityTimeout is 0

Bug Reports Cloud Services Bugs
,
#1

The screenshot below shows the logs from the code snippet running on a live server.

  • A dictionary {Hello="World"} is AddAsync() to the queue with invisibilityTimeout=0.
  • The yellow highlight shows the point where RemoveAsync() and ReadAsync() are called.
  • Despite RemoveAsync() being called and no exceptions from pcall(), ReadAsync() still returns the same table from the queue.

Screenshot 2024-04-24 092834
Screenshot 2024-04-24 092834862×519 208 KB

Code Snippet

local queue: MemoryStoreQueue = game:GetService("MemoryStoreService"):GetQueue("test", 0);

local testVal1: {any} = {Hello="World"};

local function retryer(id: string, func)
	for a=1, 8 do
		local s, e = pcall(func);
		if s == true then
			break;
		end
		print("TestScript>> Retrying (",id,") ",a,"/8: ", e);
		task.wait((a-1)^2);
	end
end

retryer("Clear all", function() 
	local d, k = queue:ReadAsync(99, false, 3);
	if k ~= nil then
		queue:RemoveAsync(k);
	end
	print("TestScript>> Clear all read", d, k);
end)

task.wait(1);

retryer("Read 0", function() 
	print("TestScript>> Read 0",queue:ReadAsync(1, false, 3));
end)

retryer("Add", function() 
	queue:AddAsync(testVal1, 300)
	print("TestScript>> Add", testVal1);
end)

retryer("Read 1", function() 
	print("TestScript>> Read 1",queue:ReadAsync(1, false, 3));
end)

local readVal, dataKey;

task.spawn(function()
	retryer("Read 2", function() 
		readVal, dataKey = queue:ReadAsync(1, false, 3);
	end)
	print("TestScript>> Read 2",readVal, dataKey);

	for a=1, 10 do
		retryer("Remove", function() 
			queue:RemoveAsync(dataKey);
			task.wait(1);
		end)
		print("TestScript>> Remove", a);
		retryer("Read Loop", function() 
			readVal, dataKey = queue:ReadAsync(1, false, 3);
			print("TestScript>> Read Loop", readVal, dataKey);
		end)
		if readVal == nil then
			print("Val is removed!");
			break;
		end
	end
end)

retryer("Read 3", function()
	task.wait(1);
	print("TestScript>> Read 3",queue:ReadAsync(1, false, 3));
end)

task.wait(1);
retryer("Read 5", function() 
	print("TestScript>> Read 5",queue:ReadAsync(1, false, 3));
end)
#2

This is by design. RemoveAsync does not remove items whose invisibility timeout has expired. Since the timeout is configured as zero it is guaranteed to be expired. Which problem are you trying to solve by setting it to zero?

#3

It was an attempt at making a caching queue that stores information and action on how a data has/should be changed.

The idea is that I have a shared data that could be modified across multiple servers at low latency. This raw json data is not stored on a SortedMap because of its size and the need to be guaranteed persistent.

So the way I wanted to use timeout here is, by setting it to zero. Whenever I need to read the data, it will first:

  1. Pull the main data from DataStore:GetAsync().
  2. Pull every queued up action from MemoryStoreQueue:ReadAsync().
  3. Process the persistent main data through the list of actions that will modify the data.
  4. If I need to update the data in any way, a new action will be added to the queue. If the queue exceeds a certain amount of items or exceeds a certain amount of time, a service will flush all the queued actions by performing task 1-3 and DataStore:UpdateAsync() the data to persistent storage.
  5. Then all processed items in the queue gets cleared with MemoryStoreQueue:RemoveAsync().

So without invisibility, multiple servers can read the same queued actions, and then when flush is triggered, cache can be flushed and queued items removed with RemoveAsync() .

It’s mostly for concurrency on a large persistent data, especially across servers and in real time.

As for right now, I’m achieving the same solution by creating a pseudo queue system with tables into SortedMap but I’ve been coming across DataUpdateConflict request statuses when trying to MemoryStoreSortedMap:UpdateAsync().

It’s about 10 conflicts per 1.1k successful requests so it’s not major but it is still causing issues and I was wondering what could be causing this and if using MemoryStoreQueue would solve this.

If RemoveAsync does not remove items whose invisibility timeout has expired, is there no other way to remove queued items other than waiting for it to expire from AddAsync’s expiration time?
Is it possible to get some sort of force remove, e.g. MemoryStoreQueue:ForceRemoveAsync()?
Or is there a better way to do all this?

Where server A could make 10 changes in the last 5 seconds and server B could make 5 changes in the last 10 while all being stored on the same data with minimal data loss/desync.

Thanks in advance,
Khronos

2 Likes
#4

Thank you for taking the time to explain the problem thoroughly.

I wouldn’t be too worried about low levels of DataUpdateConflict responses as they are a normal part of UpdateAsync behavior - detecting and automatically retrying conflicting updates. The only downside is it takes longer for UpdateAsync to complete in case of a conflict and that’s why it is a good idea to keep the number of conflicts low.

Currently there’s no other way to delete items from a queue. An option to delete arbitrary items fro a queue is a popular feature request and we will be adding it in the future.

1 Like