MockDataStoreService - seamless local development & testing with datastores

This is an open-source module that emulates datastores in Lua rather than using the actual service. This is useful for testing in offline projects / local place files with code/frameworks that need to have access to datastores.

The MockDataStoreService behaves exactly like DataStoreService – it has the same API, and will also work even if the place is not currently published to a game with Studio API access enabled (it will act as a DataStoreService with no data stored on the back-end, unless you import some data from a json string manually using the module).

A small top-level helper module is provided (DataStoreService) that automatically detects and selects which datastores should be used (real datastores for published games with API access, mock datastores for offline games / published games without API access).

It is recommended to use this code in the structure that it is provided in, and to simply call require(path.to.DataStoreService) instead of game:GetService("DataStoreService") anywhere in your code to use it properly.


Usage:

local MockDataStoreService = require(the.path.to.MockDataStoreService)

-- Use as actual DataStoreService, i.e.:

local gds = MockDataStoreService:GetGlobalDataStore()
local ds = MockDataStoreService:GetDataStore("TestName", "TestScope")
local ods = MockDataStoreService:GetOrderedDataStore("TestName")

local value = ds:GetAsync("TestKey")
ds:SetAsync("TestKey", "TestValue")
local value = ds:IncrementAsync("IntegerKey", 3)
local value = ds:UpdateAsync("UpdateKey", function(oldValue) return newValue end)
local value = ds:RemoveAsync("TestKey")
local connection = ds:OnUpdate("UpdateKey", function(value) print(value) end)

local pages = ods:GetSortedAsync(true, 50, 1, 100)
repeat
	for _, pair in ipairs(pages:GetCurrentPage()) do
		local key, value = pair.key, pair.value
		-- (...)
	end
until pages.IsFinished or pages:AdvanceToNextPageAsync()

local budget = MockDataStoreService:GetRequestBudgetForRequestType(
	Enum.DataStoreRequestType.UpdateAsync
)

-- Import/export data to a specific datastore:

ds:ImportFromJSON({ -- feed table or json string representing contents of datastore
	TestKey = "Hello world!"; -- a key value pair
	AnotherKey = {a = 1, b = 2}; -- another key value pair
	-- (...)
})

print(ds:ExportToJSON())

-- Import/export entirety of MockDataStoreService:

MockDataStoreService:ImportFromJSON({ -- feed table or json string
	DataStore = { -- regular datastores
		TestName = { -- name of datastore
			TestScope = { -- scope of datastore
				TestKey = "Hello world!"; -- a key value pair
				AnotherKey = {1,2,3}; -- another key value pair
				-- (...)
			}
		}
	};
	GlobalDataStore = { -- the (one) globaldatastore
		TestKey = "Hello world!"; -- a key value pair
		AnotherKey = {1,2,3}; -- another key value pair
		-- (...)
	};
	OrderedDataStore = { -- ordered datastores
		TestName = { -- name of ordered datastore
			TestScope = { -- scope of ordered datastore
				TestKey = 15; -- a key value pair
				AnotherKey = 3; -- another key value pair
				-- (...)
			}
		}
	};
}

print(MockDataStoreService:ExportToJSON())


Features:

  • Identical API and near-identical behavior compared to real DataStoreService.
  • Error messages are more verbose/accurate than the ones generated by actual datastores, which makes development/bug-fixing easier.
  • Throws descriptive errors for attempts at storing invalid data, telling you exactly which part of the data is invalid. (credit to @Corecii’s helper function)
  • Emulates the yielding of datastore requests (waits a random amount of time before returning from the call).
  • Extra API for json-exporting/importing contents of one/all datastores for easy testing.
  • All operations safely deep-copy values where necessary (not possible to alter values in the datastore by externally altering tables, etc).
  • Enforces the “6 seconds between writes on the same key” rule.
  • Enforces datastore budgets correctly: budget are set and increased at the rates of the actual service, requests will be throttled if the budget is exceeded, and if there are too many throttled requests in the queue then new requests will error instead of throttling.
  • Functionality for simulating errors at a certain rate (similar to deprecated/removed Diabolical Mode that Studio used to have).

Get the module:

Download the latest release of the module here:

Or get the source code here: (and submit PRs/issues!)

Or grab the module from the library:
(note: probably outdated, please check Github for latest release model)
https://www.roblox.com/library/2014520164/MockDataStoreService


Feel free to leave questions, comments, requests for improvements, and bug reports here, or as issues on the GitHub repository!

92 Likes

This is neat for CI testing, I like it!

3 Likes

This is awesome, thanks!

2 Likes

Thanks for the contribution. This is awesome!

1 Like

It would be good to simulate the request limits for the data store as well.

1 Like

Need some ice for that burn

Thank God this exists though. I was so annoyed I was no joke gonna make one, arguably of worse functionality, at home. Wonderful.

2 Likes

As you can see in the OP, that’s already on the TODO list. It was a non-trivial amount of work to implement so I left it out for v1. If you want you can file a pull request for it if you can manage to edit the code in a clean way to support it!

EDIT: budgeting and throttling is now supported!

1 Like

This has been updated to fully support datastore budgeting and throttling of requests, and the code has been cleaned up (better file structure)!

As far as I am aware, this now perfectly emulates DataStoreService behavior. I spent a lot of time checking scenarios and making sure this module reflects the behavior. If you do spot inconsistencies between this module and the actual service, please let me know.

Thanks @Validark for comments and small improvements on v1 of the code, and @LouieK22 for initial work done on the 6 second write rule.

8 Likes

Wow thanks for all of this, it’s super cool!

1 Like

Updated with a way to simulate errors similar to what Diabolical Mode used to do in Studio:

(Also updated the first post here, since it seems like I hadn’t updated the model file included there since I wrote this thread and missed some description of features)

3 Likes

Do the errors intentionally not include error codes?

I opted not to because datastore errors were not consistent with providing the error code at all, when this module was made.

Feel free to submit a pull request to add them, although make sure it matches the behavior of actual datastores. Some request errors are ambiguous in terms of the code they should intuitively apply (some data checks are done after the remote call is performed and return a 5xx where you would otherwise expect i.e. 1xx).

5 Likes

This is a seriously wonderful thing you have made here. If you do a lot of code and like to test your changes a lot something like this speeds your development significantly by avoiding the multiple second datastore delays you get otherwise.

I have a question, is there a way we can save data ‘Automatically’ when testing? I can print out a jsonencoded string of what I’m saving and manually put it back in but I think it would be nice to have an automated system.

Thanks again!

2 Likes

No, this is purely to mock datastores. Mocks are not meant to persist.

You can add a small bit of code to push the contents of the datastores to some external storage on game:BindToClose and then load it again on game start. This is outside of the scope of the module.

1 Like

The master branch of the module was updated to support new max datastore size:

Thanks @Fractality !

2 Likes

This looks awesome and the fact your still updating it 2 years later as well :+1:t2: thank you.

2 Likes

I removed the model file from this post and will instead be posting them here:

These are automatically generated from the default.project.json whenever master branch updates, then I can describe and publish from there.

3 Likes

Is there a way to reset the state of the service? I am using this for unit tests and I want to reset it so previously cached values and limits don’t carry over across tests.

1 Like

Yes, there is:

MockDataStoreManager.ResetBudget()
MockDataStoreManager.ResetData()
4 Likes

Thanks for showing me this!

Does this support logging what calls were made? Like printing them? I had to make a little thing to print when a script used SetAsync, UpdateAsync, GetAsync and such. If it doesn’t I think it would be a cool addition!

1 Like