Maliciously crafted OnInvoke callback can blow the C-Stack

If you set the OnInvoke callback of a BindableFunction to a function that calls the BindableFunction, and then invoke that BindableFunction, roblox will crash.

This isn’t a major issue, because you shouldn’t be calling the BindableFunction inside of itself in the first place, but nonetheless, Roblox should probably throw a C-Stack overflow error rather than completely crash.

local bf = Instance.new("BindableFunction")

bf.OnInvoke = function ()
	bf:Invoke()
end

bf:Invoke()

Not to mention
while true do end
lol

Basicly yes.
Normally, if you do something like:

local function a() a() end a()

It would stack overflow, as you would have 200 (I forgot the limit) function calls in the thread’s stack.
When you call bf:Invoke(), the OnInvoke runs in a new thread.
(Use getfenv() or coroutine.running() to check)
The function call stack thingy only has one (or two?) calls, so never overflows.

You can have the same effect by doing the a(), but calling a using pcall(a)

It should still be rather easy to fix; if you tack onto the lua state struct. That’s how 5.2 and above cope with the expensive pattern matches which would normally crash in 5.1