Data Stores Wiki Page could use some love

#1

It would be helpful if https://developer.roblox.com/articles/Datastore-Errors was updated with new error codes.


Low hanging fruit first: error code 105 has an incorrect description.

105
Serialized value converted byte size exceeds max size 64*1024 bytes.
Character count in string cannot exceed 65,536 characters.


Also, I run into errors which aren’t in the table.

11:51:11.525 - 508: Exhausted all retries for key: %s 
11:51:11.526 - Stack Begin
11:51:11.526 - Script 'Workspace.Script', Line 3
11:51:11.526 - Stack End
11:51:11.589 - Go to Game Settings and turn on Allow HTTP requests.
How to reproduce...
  • trying to save a value with a single character in ascii range 128-255
  • Trying to save a 43 KB buffer filled with \x00 (null terminator).

I would say that the use of this error code to mean “invalid characters” or “string too long” is faulty, but bugs are more time consuming to fix, and it’s helpful to at least explain workarounds for your bugs as a short-term solution.

I imagine there are error codes in use between 505 and 508, and quite possibly error codes above 508.


Also, it would be very nice if it were clear what the true limits are. In my bench testing, I did notice that you can save any character in ASCII range 0-127 – very cool. You can even save XML without it failing, which is p cool, too.

Unfortunately, there is some encoding happening in order to make this work, and any additional characters required to encode the text eat into the budget. For example, the null terminator counts as 6 bytes when calculating the 260,000 byte maximum.

Summary:

  • In the range 0-31, everything costs 6 bytes with few exceptions (tab, newline, etc.)
  • In the range 32-127, everything costs 1 byte, except for quotes and backslashes, both costing 2.
Full analysis

Obtained with the following script:

local dss = game:GetService("DataStoreService");
local ds = dss:GetDataStore("some-data-store");

KeyIndex = 0;
local function SaveSucceeds(value)
	local key = string.format("key-%d", KeyIndex);
	KeyIndex = KeyIndex + 1;
	local results, returnValue = pcall(ds.SetAsync, ds, key, value);
	if results then
		return true;
	elseif returnValue == "508: Exhausted all retries for key: %s " then
		return false;
	else
		print(string.format("Unexpected error message: %q", returnValue));
		return false;
	end
end

local BYTE_TO_STRING = {
	[string.byte("|")] = "\\|";
	[127] = "\\x7F";
	[32] = "\\x20";
}

for i = 0, 127 do
	--Convert this integer to a character suitable for reading in the output.
	if BYTE_TO_STRING[i] then
		char = BYTE_TO_STRING[i];
	elseif 0 <= i and i <= 31 then
		char = string.format("\\x%02X", i);
	else
		char = string.char(i);
	end

	--Determine how many bytes each character requires.
	local bytes;
	for j = 1, 7 do
		if SaveSucceeds(string.rep(string.char(i), 260e3/j)) then
			bytes = j;
			break;
		end
	end

	assert(bytes)
	print(string.format("| %d | %s | %d |", i, char, bytes));
end
ASCII Value Character Bytes Cost
0 \x00 6
1 \x01 6
2 \x02 6
3 \x03 6
4 \x04 6
5 \x05 6
6 \x06 6
7 \x07 6
8 \x08 2
9 \x09 2
10 \x0A 2
11 \x0B 6
12 \x0C 2
13 \x0D 2
14 \x0E 6
15 \x0F 6
16 \x10 6
17 \x11 6
18 \x12 6
19 \x13 6
20 \x14 6
21 \x15 6
22 \x16 6
23 \x17 6
24 \x18 6
25 \x19 6
26 \x1A 6
27 \x1B 6
28 \x1C 6
29 \x1D 6
30 \x1E 6
31 \x1F 6
32 \x20 1
33 ! 1
34 " 2
35 # 1
36 $ 1
37 % 1
38 & 1
39 1
40 ( 1
41 ) 1
42 * 1
43 + 1
44 , 1
45 - 1
46 . 1
47 / 1
48 0 1
49 1 1
50 2 1
51 3 1
52 4 1
53 5 1
54 6 1
55 7 1
56 8 1
57 9 1
58 : 1
59 ; 1
60 < 1
61 = 1
62 > 1
63 ? 1
64 @ 1
65 A 1
66 B 1
67 C 1
68 D 1
69 E 1
70 F 1
71 G 1
72 H 1
73 I 1
74 J 1
75 K 1
76 L 1
77 M 1
78 N 1
79 O 1
80 P 1
81 Q 1
82 R 1
83 S 1
84 T 1
85 U 1
86 V 1
87 W 1
88 X 1
89 Y 1
90 Z 1
91 [ 1
92 \ 2
93 ] 1
94 ^ 1
95 _ 1
96 ` 1
97 a 1
98 b 1
99 c 1
100 d 1
101 e 1
102 f 1
103 g 1
104 h 1
105 i 1
106 j 1
107 k 1
108 l 1
109 m 1
110 n 1
111 o 1
112 p 1
113 q 1
114 r 1
115 s 1
116 t 1
117 u 1
118 v 1
119 w 1
120 x 1
121 y 1
122 z 1
123 { 1
124 | 1
125 } 1
126 ~ 1
127 \x7F 1

It would be excellent if this table was published on the wiki so the next person doesn’t have to jump through hoops to find out this info like I did.


This is all for now. I’m working on a script to save/load builds, kinda like how Pilgrim Islands Reborn did, so I’ll be pushing datastores to their limit. I’m sure I’ll have more feedback to share when I uncover more exotic bugs. :slight_smile:

6 Likes