SQLite alternative to plugin GetSetting and SetSetting methods that rely on JSON

As a Roblox plugin developer, it is currently too hard to efficiently save large arbitrary data locally.

The only way plugins can save data locally is by using the plugin.SetSetting and plugin.GetSetting methods, which save & load data from a JSON file.

The larger the settings.json file for a plugin is, the longer each Set and Get call takes.

If Roblox is able to address this issue, it would improve my development experience because as a plugin developer, I’ve always wanted and have had use cases for saving RBXM data locally.

Right now the only way to do this is to convert instances to a buffer, compress it, encode it into Base64, and then save it to the settings.json file.

However, you can’t store many instances with this method, and it quickly increases the size of the settings file to the point where there is noticeable lag.

Of course, efficient plugin storage is not just needed for Roblox models, but also for other data formats, one example being EditableImages pixel data.


That’s why I think SQLite solves this problem relatively well:

  1. It stores all data in a single file (can be placed alongside settings.json)
  2. It’s way more efficient than JSON storage
  3. It’s a very light dependency

The plugin API could look something like this:

local SerializationService = game:GetService("SerializationService")

local conn = plugin:GetSQLiteConnection()

local ok, err = pcall(function()
  conn:Execute([[
    CREATE TABLE IF NOT EXISTS models (
      id   INTEGER PRIMARY KEY AUTOINCREMENT,
      data BLOB
    );
  ]])
end)

if not ok then
  error(`failed to execute the initial query: {err}`)
end

local data = SerializationService:SerializeInstancesAsync({ workspace.Test })

conn:Execute("INSERT INTO models (data) VALUES(?)", data)
conn:Commit()

local cursor = conn:Cursor()
cursor:Execute("SELECT data FROM models WHERE id = 1")

local storedData = cursor:FetchOne()

print(data == storedData) -- true
5 Likes

I’m sorry for necroposting, bumping this.

The point when :SetSetting() starts to lag is as small as about 4MB. It would be very useful if we had some alternatives to :SetSetting() and GetSetting(). As of now, it seems like both methods are accessing the general studio settings file, which contains the data stored by every other plugin and even studio settings.

Given this, 4MB is too little.

Apart from the topic, it’d be useful having the ability to write more than 200 000 bytes to Script/ModuleScript/LocalScript .Content
It’s understandable why TextBox.Text has this limit, but .Content of a script can only be set in Roblox Studio so there’s really no reason to cap it at 200K bytes, aka 200K characters (if each character is 1 byte, like all of the english letters, numbers and most of the popular special characters) or 0.2MB

I had experience dealing with both the slow :SetSetting method and the 200K bytes limit. 9 Months ago and now again.

Please Roblox, we need this.

1 Like

That’s what ScriptEditorService:UpdateSourceAsync is for.

1 Like