DataUtils | Comprehensive Table and JSON Utilities

Overview

This module provides enhanced JSON and table manipulation utilities. It extends Roblox’s table and JSON handling with additional functions. Made this for a project and decided to extend it and release it :slight_smile:

Source File DataUtils.lua (20.4 KB)
Model File DataUtils.rbxm (9.2 KB)

Key Features

  • Deep table copying and merging
  • Advanced JSON serialization with custom type handling
  • Nested value manipulation with dot notation
  • Table filtering and mapping
  • JSON schema validation
  • Data compression and encryption
  • Data diffing and patching
  • Data migration and versioning
  • Advanced serialization for Roblox-specific types
  • Table flattening and comparison utilities

API Reference

Table Operations

DeepCopy(original)

Creates a complete independent copy of a table, including nested tables.

local originalTable = {a = 1, b = {c = 2}}
local copy = DataUtils.DeepCopy(originalTable)

MergeTables(table1, table2, deep, overwrite)

Merges two tables with optional deep merge and conflict resolution.

  • deep: Whether to merge nested tables
  • overwrite: Whether to overwrite existing keys
local merged = DataUtils.MergeTables(table1, table2, true, false)

JSON Operations

AdvancedEncode(data, options)

Serializes data to JSON with custom type handling.

local json = DataUtils.AdvancedEncode(data, {
    indent = 2,
    handleSpecialTypes = true
})

JSONToTable(jsonString, options)

Converts JSON string to Roblox table with advanced type handling.

local table = DataUtils.JSONToTable(jsonString, {
    convertRobloxTypes = true,
    nilValue = nil
})

TableToJSON(tbl, options)

Converts Roblox table to JSON with enhanced serialization.

local json = DataUtils.TableToJSON(table, {
    maxDepth = 10,
    handleRobloxTypes = true,
    preserveNil = true
})

Nested Value Operations

GetNestedValue(tbl, path)

Retrieves nested values using dot notation.

local value = DataUtils.GetNestedValue(table, "user.profile.name")

SetNestedValue(tbl, path, value)

Sets nested values using dot notation.

DataUtils.SetNestedValue(table, "user.profile.name", "John")

Table Transformation

FilterTable(tbl, predicate)

Filters table based on a predicate function.

local filtered = DataUtils.FilterTable(table, function(v, k)
    return v > 10
end)

MapTable(tbl, mapper)

Transforms table values using a mapping function.

local mapped = DataUtils.MapTable(table, function(v, k)
    return v * 2
end)

Data Validation

ValidateSchema(data, schema)

Validates JSON data against a schema.

local isValid = DataUtils.ValidateSchema(data, {
    name = "string",
    age = "number"
})

Advanced Features

Compression and Encryption

local compressed = DataUtils.Compress(data)
local decompressed = DataUtils.Decompress(compressed)

Data Diffing

local diff = DataUtils.CreateDataDiff(originalTable, modifiedTable)
local patched = DataUtils.ApplyDataPatch(originalTable, diff)

Data Pipeline

local pipeline = DataUtils.CreateDataPipeline()
    :AddValidator(function(data)
        return data.value > 0, "Value must be positive"
    end)
    :AddTransformer(function(data)
        data.value = data.value * 2
        return data
    end)

local result = pipeline:Process(data)

Data Migration

local migrations = {
    [1] = function(data)
        data.newField = "default"
        return data
    end
}
local migrated = DataUtils.MigrateData(data, migrations)

Best Practices

  1. Always use DeepCopy when you need independent copies of tables
  2. Handle errors appropriately when using JSON operations
  3. Use schema validation for critical data structures
  4. Consider using the data pipeline for complex transformations
  5. Implement proper error handling for all operations
  6. Use compression for large data sets
  7. Implement proper versioning for data migrations

Performance Considerations

  • Deep copying large tables can be expensive
  • Consider the depth of nested structures when using JSON operations
  • Use table flattening for simpler storage of complex structures
  • Implement caching for frequently accessed data
  • Monitor memory usage when handling large data sets

Compatibility Notes

  • Designed for Roblox Lua environment
  • Requires HttpService for JSON operations
  • Supports Roblox-specific types (Vector3, CFrame)
7 Likes

Very cool! I might use this in my game.

2 Likes

Added some comments, added parameters and return types for luau lint (shows up on hover like buil-in functions). Also fixed a lot of bugs and made the code overall a lot better :slight_smile: