'Exception was thrown by handler' when querying DataStore entries with certain characters

[Arch_Mage]

When querying the following endpoint, an UNKNOWN error response of "Exception was thrown by handler." is returned.
https://apis.roblox.com/cloud/v2/universes/1359573625/data-stores/DATASTORE/scopes/SCOPE/entries/dXNlcl80NDE2NjM0QE0=

Other endpoints for the same DataStore with otherwise the exact same GET query work completely fine, such as
https://apis.roblox.com/cloud/v2/universes/1359573625/data-stores/DATASTORE/scopes/SCOPE/entries/dXNlcl8xNTEyMzk4NzJATw==

The keys in question are Base64 encoded, with = being used for padding.

Escaping/encoding the = with %3D does not remedy this issue. \= also does not work. This seems to be an internal exception from gRPC that has to do with the URI component being decoded before it is parsed as a key. I believe this is something that can be adjusted in the gRPC config for the endpoint.

This also seems to be an issue with other special characters like /, as in the following thread: Unable to get Data/Memory Store entries when key contains "/"

I haven’t been able to find a way to otherwise escape this character, and the DataStore has already been using these keys without issue for years, so changing the key format on our end to something more URI-friendly isn’t ideal.

Expected behavior

Keys should either be escapable or have no limitations on the characters used.

A private message is associated with this bug report

[ChasingSpace (Chasing)]

I noticed this issue when migrating from the V1 DataStore endpoints. It’s a regression from the V1 endpoints because those specify the entry key as a query parameter which has no problems with these characters.

You’re able to use the V1 DataStore endpoint for now, but there’s currently no way to access MemoryStores if your keys are stored with these characters.

[Hooksmith]

Thanks for the feedback, we’re aware of it but not something we can fix on a short time frame due to some underlying technical constraints in how these requests are resolved for the Open Cloud v2 stack.

We’re considering this and some other corner cases in future Open Cloud roadmap development. In the mean-time it seems like the above will unblock you if it’s just related to data stores.

[Arch_Mage]

Thanks for the response!

Thankfully I added a v1 fallback to my requests so it was a simple fix. Glad it’s something being considered, I really love the interface v2 provides so I was a little sad when this edgecase popped up and prevented me from fully converting my webserver to it.

[ahrielia]

Hello I think we may have resolved the issue, could you try url encoding the = again?

[Arch_Mage]

Doesn’t seem to be working still. An input of dXNlcl80NDE2NjM0QE0%3D still fails and returns 'Exception was thrown by handler'.

[ahrielia]

Hmm actually it might be the key value, could you DM me the value? If its something sensitive then could you provide me a mock of the structure?

[ChasingSpace (Chasing)]

I wonder if it could be related to how Infinity/NaN values are parsed. The V1 endpoint would output JSON like {"key":inf,"key2":nan} which would break most JSON parsers I used, so I had to manually account for those cases before I could parse it.

The V2 endpoints seem to parse the value for you instead of handing off JSON.

Edit: I’ve confirmed this (but may be unrelated to the above issue). Roblox accepts Infinity and NaN inside DataStores but they can’t be serialized by JSON. I got {"code":2,"message":"Exception was thrown by handler."} when attempting to get the value of this:

game:GetService("DataStoreService"):GetDataStore("Test"):SetAsync("Test", { key = math.huge, key2 = 0/0 })

[ahrielia]

Yeah that’s my theory aswell, asking the OP to confirm with their entry value

[ahrielia]

Discussed with the OP offline, it was an issue with one of the keys name being empty. Removing it fixed the issue. I will work on a fix to display a better error message for both invalid JSON and invalid structs. Thanks everyone!