[SOLVED] Metamethods for when you index any value not just nil values (like with __index)

Help and Feedback Scripting Support
#1

** this post was originally a feature request but it has been solved **

It would be nice to have a metamethod to detect if a table is indexed AT ALL, not just if the value of the index was nil, would be pretty useful in some cases. I have to utilize extremely hacky workarounds to detect if a table was indexed regardless of it’s potential value.

Same goes for __newIndex

(let met also add that the metamethods wouldn’t necessarily have to override assignments/lookups, just detect them)

The hacky workaround in question:

local realTbl = {}--table that actually has values

setmetatable(
   {},--table that will remain permanently empty
   {
      __index = function(tbl, index)
         indexrawDetected(tbl, index)

         return realTbl[index]
      end
   
      __newindex = function(tbl, index, value)
         newindexrawDetected(tbl, index, value)

         realTable[index] = value
      end

   },
)--this code might be wrong, i just wrote it while making this post, but i hope you get the idea

I’m basically creating a mock table that is nil for all values, so the __index and __newindex metamethods will always fire for lookups and assignments on that table.

This workaround isn’t fun because the normal table that has a metatable applied is now is not normal. For example, if you print the example table i used, it will be empty {} and not show the values that its supposed to have. Same goes for iterating it. You will have to use this table in a fundamentally different way.

A metamethod like __indexraw and __newindexraw would be nice.
or (__indexdetected and __newindexdetected)
(idk I’m not good at naming methods)

if anyone else has metamethod ideas you should put them below!! Its about time roblox gets some new metamethods, I’m sure there are plenty that would be useful.

2 Likes
#2

good idea, but you should delete this before the nerd mfers start sliding through and jump you for using the wrong category

4 Likes
#3

__tostring already exists.

This would require changing the indexing system fundamentally.
What Lua does right now when you index a table, is to look through the table that you’re indexing, check if the value within the given index exists, and if it does, return it, if it doesn’t, call __index.

And this makes sense, tables are meant to be this way. Lua shouldn’t have to call two metamethods each time it indexes a table or a single metamethod that gets called when the table gets indexed all the time. That would ruin the performance, and ruin how indexing works at the backend. Lua would first have to make sure that metamethod doesn’t exist if it ever wants to index a table, and then do the indexing.

This can potentially break a ton of systems.

Also, if you wish to create a proxy, use userdata objects with the newproxy() function. If you want to fully have control over indexing, they provide you with the most power. Every single metamethod is available for userdata. (except for __gc)
And __index gets called no matter how you index it.

#4

You’re right my bad. Didn’t see that one.

Sorry, but I don’t understand your point here, I might be misunderstanding. It would only have to call the newly proposed metamethod when it is actually in the metatable (like all other metamethods). It wouldn’t override indexing just detect it.

some theoretical C/C++ psuedocode but in lua for simplicity

function onTableIndexed(luaTbl, index, value)
   if hasNewIndexRawMetaMethod(luaTbl) then
      fireNewIndexRawMetaMethod()
   end

   --normal stuff that was happpening before
end

And in terms of performance, this metamethod wouldn’t be used for normal oop code, just sparingly, and also most likely wouldn’t be used at the same time as __index. And also, metamethods aren’t built for performance to begin with :wink:

#5

Thanks for mentioning this I had no idea this existed.

#6

What I’m trying to say here is that your metamethod is unnecessary and decreases performance.
How Lua handles the indexing is that it first tries to index the table normally, then it calls the metamethod, if the index exists, this doesn’t create any issues even if there’s no metatable attached, it behaves the same. Think of it like this:

  1. Table indexed
  2. Does the table contain the index?
    – Yes? Return the value.
    – No? Call the __index.

Now, even if the table doesn’t have the __index metamethod, the behavior will stay the same. It will cause no issues. And if you try to add the __index metamethod, it will still behave the same, with no issues. Why? Because it’s not doing a logic check constantly. It only does a logic check when the index is finally not found within the table, which causes the __index to be run if it exists.

In your system, that logic check will run every single time a table has been indexed, regardless if the table is a metatable, or contains the __index metamethod. This decreases the performance. Maybe not on small-scale applications, but on a big scale, it changes a lot. It’s completely unnecessary to do this logic check when alternative options already exist. And mostly, your metamethod wouldn’t even get used a lot, since, proxy tables aren’t a big necessity in an average developer’s life.

#7

This sounds like the XY Problem . What are you actually trying to do accomplish with this?

#8

I’m proposing 2 new metamethods, please make sure to read the post next time.

#9

First of all, tables with no metamethods are VERY often indexed with the value being nil, with no impact on performance.
This means in your theoretical scenario, (theoretical because what you describe is probably not how it works internally), the step 2 in that you listed in the case of the lookup being nil is ran very often unnecessarily, because most tables don’t even have metatables. Would this not meet your definition of “decreasing performance”?

Second of all, the system internally could function like this with no realistic hit to performance

  1. Table indexed
  2. Does table have a metatable (boolean)?
    • Yes? do the following.
      1. call proposed index metamethod i described in the post.
      2. Does the table contain the index?
      • Yes? Return the value.
      • No? Call the __index.
    • No? index table like normal.

this probably isn’t even how it works internally and it is most likely the case that entirely different indexing functions are called depending on if the table has a metatable or not.

#10 
void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val)
{
    int loop;
    for (loop = 0; loop < MAXTAGLOOP; loop++)
    {
        const TValue* tm;
        if (ttistable(t))
        { // `t' is a table?
            Table* h = hvalue(t);

            const TValue* res = luaH_get(h, key); // do a primitive get

            if (res != luaO_nilobject)
                L->cachedslot = gval2slot(h, res); // remember slot to accelerate future lookups

            if (!ttisnil(res) // result is no nil?
                || (tm = fasttm(L, h->metatable, TM_INDEX)) == NULL)
            { // or no TM?
                setobj2s(L, val, res);
                return;
            }
            // see if it has an INDEX meta-method to look up the key with
        }
        else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_INDEX)))
            luaG_indexerror(L, t, key);
        if (ttisfunction(tm))
        {
            callTMres(L, val, tm, t, key);
            return;
        }
        t = tm; // else repeat with `tm'
    }
    luaG_runerror(L, "'__index' chain too long; possible loop");
}

From: luau/VM/src/lvmutils.cpp at master · luau-lang/luau · GitHub

It’s not my system I have to make it clear, it’s the current Luau’s system. And it’s the best solution, currently. And the problem is dude, as you can see in the above code, when the index is nil, it throws an error, and the execution is halted. No additional logic checks happened. In yours? The logic check needs to always happen, regardless of if there’s even a metatable attached.

Your system contains 6 logic checks while the above only contains 3.

1 Like
#11

But why do you want two new metamethods? What makes calling another function every time you get something not an option?

#12

If you call a function every time you index something or do something with tables then there’s no point to metatables to begin with.

In other words, the point of metamethods is so you dont have to call a function every time you do something special with a table. aka just syntax sugar

#13

It isn’t about that in this case. This person wants two new completely unnecessary metamethods to fit their needs when there are alternative solutions that can very well benefit them.

#14

I’m going to be honest, I dont use C++ or C so I don’t understand the code you posted, but I’m going to take your word for it that there would be an impact on performance.

But, I will also say that couldn’t you theoretically separate indexing functions for tables with and tables without metatables? That way theres no performance difference for normal tables (90% of all tables) and theres only a very minor difference for tables with a metatable.

#15

Some parts of your suggestions would make sense if Lua didn’t already have userdata already. If you can learn C++ one day and inspect the code above I posted, it already uses a system like that for userdata and normal tables.

#16

In that case, could you describe a use of userdata that works just like the workaround i gave? if you can I will delete the post. Userdata has 0 useful documentation so I’m not sure how to use it. Does it work like a normal table?

What I WANT, is a table that works like normal but also signals when something is indexed or assigned. I’m not sure if userdata can match that.

I mean, this is literally the only information
image

1 Like
#17

It’s essentially like your proxy system. But it’s protected from rawget() and rawset() and additionally allows you to access more metamethods.

local newUserData = newproxy(true) -- This function creates a userdata, with the option to have a metatable attached to it. (the true boolean in this case)
local metatable = getmetatable(newUserData) -- Returns the metatable attached to the userdata.
local proxyTable = {1} -- The table you want to store data in.

metatable.__index = function(t, i) 
      print("This index function will always run when the userdata is indexed.")
      return proxyTable[i]
end

print(newUserData[1]) -- Will print "1"

Now this is technically the same as your proxy system, but it is more secure, and provides you more control over what you want to do with the userdata object. For example, in your proxy system, someone could use the rawset() function towards your proxy table and set a value inside it, causing the __index to never run.

Here’s a post that explains them well: Userdatas, how to use them

1 Like
#18

I did read your post. You never explained why this would be useful.

A metamethod to detect if a table is indexed AT ALL, not just if the value of the index was nil, would be pretty useful in some cases.

You’re suggesting these metamethods as a solution to to your problem instead of explaining your actual problem.

1 Like
#19

This is extremely helpful, thank you for showing me this

I would mark this as the solution but i dont know how :grinning:

I’ll leave this post up just for future people who are curious on the same thing

1 Like
#20

you’re right my bad

But in a nutshell it would be useful for making even more sugary syntax sugar. I’m not going to go in to the specifics of what I’m doing but you can get an idea just from the way metamethods are used in general.

2 Likes