** For the most up-to-date version of this tutorial, see Open Cloud Data Stores. **
While you could previously only access Data Stores using Lua APIs in a game server or Studio, you can now use Open Cloud DataStore APIs to access data stores from external scripts and tools. Open Cloud DataStore APIs are RESTful and provide almost the full feature set as Lua APIs.
There are several cases in which you may want to access your data using Open Cloud DataStore APIs, including:
- Customer Support Portal: Rather than using Studio or joining an experience every time, you can build a web application that allows your customer service agents to directly handle customer support requests, such as viewing and modifying player inventory, issuing refunds, etc.
- LiveOps Dashboard: You can code a big event in your experience while hiding it under a feature flag as part of the configuration data in a data store. You can then use your LiveOps dashboard to schedule the in-game events by flipping the flag when it’s time for the event with the Datastore API. The game servers will detect this change by reading the flag and launching the event.
- Data Migration: As your experience evolves, you may want to change your original data schema in a data store to accommodate the new features you want to launch. In order to not lose existing players’ data, you need to migrate your data store from the old schema to a new one. With data store APIs, you can write an external script that reads each entry from current data stores, maps the data to the new schema, and writes the entry back to a new data store.
Differences with Lua APIs
Although the Open Cloud Data Store API is similar to the Lua API, there are a few differences to be aware of:
- Always provide the universe ID: The base URL for Datastore APIs is https://apis.roblox.com/datastores/v1/universes/{universeId}. Unlike Lua APIs, Open Cloud APIs are stateless and can come from anywhere, so you need to always provide the universeId and datastore name when sending the requests.
-
Permissions for creating and updating are different: The Lua API creates new entries if they don’t exist when you call
GlobalDataStore:SetAsync. The permissions for creating and updating entries using Open Cloud APIs are separated, which may be more desirable in certain situations. For example, a customer support tool might be able to edit a player’s profile, but not create a new one. - Data serialization: All data has to be serialized before network transportation. Serialization means to convert an object into that string, and deserialization is its inverse operation (convert string → object). The data store system uses JSON format. The Lua API serializes/deserializes entry content automatically. When using Open Cloud APIs, you need to generate/parse your entry data with JSON.
Security Permissions
Data stores usually store sensitive information, such as user profiles and virtual currency. To maintain security, each Open Cloud API has corresponding required permissions that you must add to your API key in order for the API call to work, such as the ‘List Keys’ permission for the listing API. When you don’t add the required permissions, the API call returns errors. For the specific permissions that are required for each API, refer to the [Data Store API Reference]((DataStore API | Roblox Creator Documentation).
When configuring your API keys, you can set granular permissions (read, write, list entry, etc.) for each data store within a specific experience, or you can give a key to read or write all data stores within an experience.
Select permissions for all data stores in an experience.
Note that permission control on the individual data store level will be available in the middle of the beta program.
Building Tools with DataStore API
This section provides a concrete example using Python to illustrate how you can list and read a subset of your users’ inventory, make edits, and then update back to an experience’s data store.
For this example, assume the following:
- The experience’s name is “MyAwesomeAdventure”, and the universe ID is 123456.
- The data store that stores player inventory is called “player_inventory”.
- The data schema for each data entry is “userId”: {“currency”: number, “weapon”: string, “level”: number}, i.e. the key is just userId.
- The Python script will list a subset of user inventories based on prefixes, increase their virtual currency by 10 for a promotion, and update the data.
From the high level, you can build your Python app to create an API key and write the Python code.
Create API Key
In this example app, you need to use three data store APIs: List Entries, Get Entry, and Set Entry. Referencing the API reference, these three APIs need the following API key permissions:
- List keys
- Read entries
- Update entries
Note that there is no need for the “create entries” API key permission since the app doesn’t need to create new player inventories.
To create an API Key:
-
Go to the Credentials page on the Creator Dashboard.
-
(Optional) If you want to create an API key for a group-owned experience, in the Creator section of the left-hand navigation, select the dropdown arrow, then select the group you would like to access the API key.
-
On the upper-right of the screen, click the Create API Key button.
-
Enter a unique name for your API key. Use a name that helps you recall its purpose later for continuous integration and deployment tooling access. This example uses the name Player-inventory-update.
-
From the Select API System menu in the Access Permissions section, select the Datastore API system, then click the Add API System button.
-
(Optional) In the Datastore API section, select API operations for specific data stores.
A. Enable the Specific Datastore Operations toggle. By default, five data stores automatically load, but you can add additional data stores through the + Add Datastore to List button.
B. Select the dropdown arrow next to a data store’s name, then select the API operations you want the data store to have access to.
-
(Optional) Select API operations for the entire experience.
A. Click the Select Experience to Add dropdown and select an experience.
B. In the Experience Operations, click the dropdown arrow and select the operations you want to add to your API. This example selects Read Entry, Update Entry, and List Entry Keys for the entire experience.
-
In the Security section, explicitly set IP access to the key using CIDR notation, and set an explicit expiration date so your key will automatically stop working after that date. For this example, since you will do local testing first, you can remove the IP restriction by setting it to ‘0.0.0.0/0’ and let it expire in 30 days.
-
Click the Save and Generate key button.
-
Copy and save the API key string to a secure location.
The following section uses “API_KEY” to signify this API key.
Write the Code
Put these files in the same directory:
TutorialFunctions.py
import hashlib
import requests
import json
import base64
class DataStores:
def __init__(self):
self._base_url = "https://apis.roblox.com/datastores/v1/universes/"
# API Key is saved in an environment variable
self._apiKey = str(os.environ['API_KEY'])
self._universeId = "UNIVERSE_ID"
self.ATTR_HDR = 'Roblox-entry-Attributes'
self.USER_ID_HDR = 'Roblox-entry-UserIds'
self._objects_url = self._base_url +self._universeId+'/standard-datastores/datastore/entries/entry'
self._increment_url = self._objects_url + '/increment'
self._version_url = self._objects_url + '/versions/version'
self._list_objects_url = self._base_url +self._universeId+'/standard-datastores/datastore/entries'
def _H(self):
return { 'x-api-key' : self._apiKey }
def _get_url(self, path_format: str):
return f"{self._config['base_url']}/{path_format.format(self._config['universe_id'])}"
return r, attributes, user_ids
def get_entry(self, datastore, object_key, scope = None):
self._objects_url = self._base_url +self._universeId+'/standard-datastores/datastore/entries/entry'
headers = { 'x-api-key' : self._apiKey }
params={"datastoreName" : datastore, "entryKey" : object_key}
if scope:
params["scope"] = scope
r = requests.get(self._objects_url, headers=headers, params=params)
if 'Content-MD5' in r.headers:
expected_checksum = r.headers['Content-MD5']
checksum = base64.b64encode(hashlib.md5(r.content).digest())
#print(f'Expected {expected_checksum}, got {checksum}')
attributes = None
if self.ATTR_HDR in r.headers:
attributes = json.loads(r.headers[self.ATTR_HDR])
user_ids = []
if self.USER_ID_HDR in r.headers:
user_ids = json.loads(r.headers[self.USER_ID_HDR])
return r
def list_entries(self, datastore, scope = None, prefix="", limit=100, allScopes = False, exclusive_start_key=None):
self._objects_url = self._base_url +self._universeId+'/standard-datastores/datastore/entries'
headers = { 'x-api-key' : self._apiKey }
r = requests.get(self._objects_url, headers=headers, params={"datastoreName" : datastore, "scope" : scope, "allScopes" : allScopes, "prefix" : prefix, "limit" : 100, "cursor" : exclusive_start_key})
return r
def increment_entry(self, datastore, object_key, incrementBy, scope = None, attributes=None, user_ids=None):
self._objects_url = self._base_url +self._universeId+'/standard-datastores/datastore/entries/entry/increment'
headers = { 'x-api-key' : self._apiKey, 'Content-Type': 'application/octet-stream' }
params={"datastoreName" : datastore, "entryKey" : object_key, "incrementBy" : incrementBy}
if scope:
params["scope"] = scope
r = requests.post(self._objects_url, headers=headers, params=params)
attributes = None
if self.ATTR_HDR in r.headers:
attributes = json.loads(r.headers[self.ATTR_HDR])
user_ids = []
if self.USER_ID_HDR in r.headers:
user_ids = json.loads(r.headers[self.USER_ID_HDR])
return r
Tutorial.py
import tutorialFunctions
DatastoresApi = tutorialFunctions.DataStores()
# Set up
datastoreName = "PlayerInventory"
# List keys for a subset of users (you may need to use the nextPageCursor to view other entries)
keys = DatastoresApi.list_entries(datastoreName)
print(keys.content)
# Read inventory for each user
for x in range(5):
updatedObjectKey = "User_"+str(x+1)
value = DatastoresApi.get_entry(datastoreName, updatedObjectKey)
# change response type to a string
updatedValue = value.json()
print(updatedObjectKey + " has "+str(updatedValue)+" gems in their inventory")
# Update the currency of each user by 10
for x in range(5):
updatedObjectKey = "User_"+str(x+1)
value = DatastoresApi.increment_entry(datastoreName, updatedObjectKey, 10)
# change response type to a string
updatedValue = value.json()
print(updatedObjectKey + " now has "+str(updatedValue)+" robux in their inventory")
To test, set the API_KEY environment variable and run Tutorial.py file:
$ export API_KEY=...
$ python Tutorial.py
Legacy Scope Support
This section is only for users who use the legacy Scope feature.
Note: It is strongly recommended to deprecate any usage of the legacy Scope feature.
Like the DataStoreService, every key in a data store has a default “global” scope, but you can further organize keys by setting a unique string as a scope that specifies a “subfolder” for the entry. Once you set a scope, it automatically prepends to all keys in all operations done on the data store.
The scope categorizes your data with a string and a separator with “/”, such as:
| Key | Scope |
|---|---|
| houses/User_1 | houses |
| pets/User_1 | pets |
| inventory/User_1 | inventory |
All data store entry operation APIs have a ‘Scope’ parameter for when you need to access the entries stored under a non-default scope. For example, you may have a “1234” key under the default “global” scope, and the same key under “special” scope. You can access the former without using the scope parameter, but to access the latter, you have to specify the scope parameter as “special” in Get Entry or Set Entry API calls.
Additionally, if you want to enumerate all the keys stored in a data store that has one or multiple non-default scopes, you can set the ‘AllScopes’ parameter in List Entries API to be ‘true’, in which case the call returns a tuple with key string and scope. In the previous example, the List Entries API would return both (“1234”, “global”), and (“1234”, “special”) in the response.
You cannot pass ‘Scope’ and ‘AllScopes’ parameters on the same request, otherwise the call returns an error.
Leveraging the helping functions from the DatastoreAPI module, the following code illustrates how you can read every key in a data store with a custom scope:
# Set up
import tutorialFunctions
DatastoresApi = tutorialFunctions.DataStores()
datastoreName = "PlayerInventory"
# List keys for global scope
specialScopeKeys = DatastoresApi.list_entries(datastoreName, scope = "global", allScopes = False)
print(keys.content)
# List keys for special scope
specialScopeKeys = DatastoresApi.list_entries(datastoreName, scope = "special", allScopes = False)
print(keys.content)
# List keys for allScope set to true
specialScopeKeys = DatastoresApi.list_entries(datastoreName, allScopes = True)
print(specialScopeKeys.content)
Response for global scope:
{"keys":[{"scope":"global","key":"User_2"}],"nextPageCursor":""}
Response for special scope:
{"keys":[{"scope":"special","key":"User_6"},{"scope":"special","key":"User_7"}],"nextPageCursor":""}'
Response for AllScopes = True
'{"keys":[{"scope":"global","key":"User_3"},{"scope":"global","key":"User_4"},{"scope":"global","key":"User_5"},{"scope":"special","key":"User_6"},{"scope":"special","key":"User_7"}],"nextPageCursor":""}'