What is newproxy() and what is it good for?

I noticed the function newproxy() while browsing through built-in functions.The Developer Hub says that it creates a blank userdata and adds an empty metatable if specified via the addmetatable argument.

The least I know is that you can create a blank userdata object with it, you can modify its metatable and printing its type returns userdata. What I don’t know is how you can use it. Can you add things to it so it looks like some other kind of userdata or object? What can you use it for? What’s the extent of its use? What’s the difference between using newproxy with a metatable and without one?

22 Likes

I asked @Corecii and this is his answer

you can use it when you want to make objects with completely custom behavior

it’s only necessary if you want to make the __len metatable (aka the # operator e.g. #object ) work for some custom object

you can use newproxy(true) to make an empty userdata which you can set the metatable of to customize its behavior

i just use tables. fyi, vanilla/default lua does not have newproxy

you get the advantage of being able to customize and restrict anything and everythign about the object when you use newproxy(true) but this rarely ever matters

and you can’t exactly store data inside a userdata, you have to store it somewhere else and add behavior to get/set it

which is a disadvantage of using newproxy(true) , since you can store data in a table

practically i’d say just avoid newproxy unless you find some niche situation where you absolutely need it

16 Likes

This isn’t true.
newproxy is a undocumented feature of Lua 5.1 but was scrapped in 5.2 onward.

17 Likes

I’ve never had a practical use-case for needing newproxy. In 10 years of programming I have never used it for anything practical.

It’s not that it can’t be useful, but rather there are always other ways to accomplish what you’re trying to do IMO.

13 Likes

So I played around with newproxy() and made a thing, I guess.

ServerStorage.ModuleScript:

local Internal = {
	Name = "lol"
}

local External = newproxy(true)
local Metatable = getmetatable(External)
Metatable.__index = Internal
Metatable.__tostring = function(self)
	return "yeet"
end

return External

ServerScriptService.Script:

local ok = require(game.ServerStorage.ModuleScript)
print(ok, type(ok), typeof(ok), ok.Name) --> yeet userdata userdata lol

Reference: https://devforum.roblox.com/t/does-anyone-use-newproxy-to-create-userdatas/24883/4


I probably wouldn’t have any reason to use newproxy() except forcing myself to use it, now that I look at it. Although not too many of my posed questions have been answered, I see where the previous responses come in as “not much practical use for it”.

9 Likes

It is common practice to make a new table, userdata, or (rarely) a function when you need a new unique value that cannot be obtained from anywhere else except by passing it around. For example, someone were to create a function that prints its input in either the standard output or error output they could naively implement it like so:

local function printMsg(value, ...)
  if value == 'error' then
    error(...)
  else
    print(value, ...)
  end
end

However, if my message that I want to print is ever ‘error’ then this function will actually throw an error and drop that portion of my message! While we could pass in a callback in this example, sometimes it is advantageous to pass in a unique value instead. The function could be implemented like so:

local ERROR = newproxy() -- or {} or function() end
local function printMsg(value, ...)
  if value == ERROR then
    error(...)
  else
    print(value, ...)
  end
end

A more practical use case would be in information hiding and secret exchange. Consider this locked down script:

local PROCESS_ACCESS_KEY = newproxy()
local SERVICE_ACCESS_KEY = newproxy()
local KERNEL_ACCESS_KEY = newproxy()

local ACCESS_LEVEL = {
  [PROCESS_ACCESS_KEY] = 1;
  [SERVICE_ACCESS_KEY] = 2;
  [KERNEL_ACCESS_KEY] = 3;
}

require('scheduler')(KERNEL_ACCESS_KEY)

local services = {
  require 'anti-exploit';
  require 'IPC';
}

local processes = {}

for i, service in ipairs(services) do
  service.setKey(SERVICE_ACCESS_KEY)
end

function shared.startProgram(accessKey, program)
  if ACCESS_LEVEL[accessKey] >= ACCESS_LEVEL[SERVICE_ACCESS_KEY] then
    local process = coroutine.create(program)
    processes[process] = tick()
  else
    error('Access Denied.', 2)
  end
end

function shared.stepScheduler(accessKey)
  if ACCESS_LEVEL[accessKey] >= ACCESS_LEVEL[KERNEL_ACCESS_KEY] then
    for i, service in ipairs(services) do
      service(SERVICE_ACCESS_KEY)
    end
    for i, process in ipairs(processes) do
      coroutine.resume(PROGRAM_ACCESS_KEY)
    end
  end
end

-- In scheduler script 
return function(accessKey)
  spawn(function ()
    while wait() do
      shared.stepScheduler(accessKey)
    end
  end)()
end

Now in the previous examples, the unique value could be table or function without any issues. The advantage of tables over functions is that you can set their metatable. This allows you not only to have unique values, but unique values that print prettily because they have a custom __tostring method on them. You can also define + to do special operations like return another table which represents the union of the two tables.

However, tables do have a problem. Consider this example:

local Set = {}
Set.__index = Set

function Set.new(data)
  return setmetatable(data, Set)
end

function Set:__add(other)
  local result = {}
  for key, value in next, self do
    result[key] = value
  end
  for key, value in next, other do
    result[key] = value
  end
  return result
end

return setmetatable(Set, {
  __metatable = 'This metatable is locked';
  __newindex = {}; -- don't change me
})

-- In another script:
local playerScores = Set.new {}

...

local function adjustScores(adjustment)
  playerScores = playerScores + adjustment
end

-- An in some module that got inserted and required at run time:
local Set = require 'Set'

local Spy = {}

function Spy.new(data)
  data.IdiomicLanguage.moneyz = 9999999999999
  return data
end

function Spy:__add(other)
  other.IdiomicLanguage = nil -- Don't change my moneyz please
  return self, other
end

for key, newValue in next, Spy do
  local oldValue = Set[key]
  rawset(Set, key, function(...)
    return oldValue(newValue(...))
  end)
end

Because rawset can be used on tables, the desired functionality of tables can be changed. Since __index only runs when the key being accessed doesn’t exist in a table, rawset can be used to prevent the __index or __newindex metamethod from running. Eventually you run into this huge problem where even if you don’t have some nefarious code running on your server, if you ever use rawset then you can get into huge trouble with crazy bugs if it is used on the wrong table/key.

Because userdata’s don’t store keys/values, rawset and rawget don’t affect them. With a locked metatable, you have absolute control and security. __index will fire every time the userdata is accessed, and __newindex every time a value is set. The metatable cannot be obtained. The functions and methods on it can’t even be enumerated. It is essentially a black box to any code looking at it.

Another advantage of a userdata is that it isn’t a table. For example, if you have a function that usually takes in tables then if you were to pass in unique tables to perform special actions, you would have to be able to compile a list of special tables before hand and compare the each table you receive to the list. On the other hand, with a userdata value you can simply check the type and if it is a userdata, then call it (the __call metamethod), add it, or whatever, without having to compile a list to know that it was a special value.

The original purpose if userdata values is to provide an interface between Lua and C/C++. That is why all instances are userdatas. I’m not as familiar with this aspect of it, so I’ll forgo expending on this topic (although there are probably some brilliant people here who could).

45 Likes

magic man, thanks for the comprehensive look at newproxy and ways of using it.

For now I’m thinking: would I, in theory, be able to further secure a module using newproxy? There would be changing the script reference to a blank ModuleScript ( script = Instance.new"ModuleScript" ) and of course how I choose to organize my code in the module itself, but then there’s also returning a userdata.

Furthermore, two questions.

  1. Does it matter what type I set to __metatable? It seems from this that I can set a random value so that trying to use getmetatable won’t return the actual metatable but rather that value.
  2. What purpose does setting __newindex as a blank table serve?
4 Likes

newproxy wasnt entirely useful and removed in Lua 5.3
Yes it allows userdata creation but unless you have access to the debug library or you set it up in a library it wouldn’t make sense. Userdata are protected and not modifyable just by using Lua (unless using libs) so it is practically purposeless. Also userdatum have uservalues, similar to whats going on in dictionaries but also different.

1 Like
  1. No, it doesn’t matter what type. I’m not sure if ‘false’ would work though.

  2. I often set __newindex to a blank table when I want to make sure that __index will always fire. Otherwise, someone could set a key on the table, then __index wont fire when that key is grabbed (since __index only fires when a key is missing).

3 Likes

u can use proxy table too bc its not in the object or table itself, it’s in another

Small correction. Calling type on userdata returns "userdata" (a string - not a table in either sense).

1 Like

Thanks, I have corrected it by removing that sentence. Its original purpose was to expand upon the previous statement, but it obviously came out wrong. :stuck_out_tongue:

1 Like

The __len metamethod doesn’t work on tables, so I’ve used newproxy to do extreme tests on how my objects are being used. It can also be used for security, but unless your object interacts with lots of external scripts I wouldn’t recommend using it for that.
Besides those two things there’s nothing that a newproxy(true) can do that a plain table and metatable can’t do better.

2 Likes

The best use case for this API is a sandbox, which is only useful if you plan to allow third party Lua execution. Other than that, there’s no real point. I tried experimenting with an OOP framework called Construct which made using newproxy userdata easy, but it just gets ugly and can actually slow things down in the long run. If no third party is accessing your code, which they shouldn’t be, then you’re golden to use tables.

4 Likes