Memory spike detector

Help and Feedback Code Review
#1

What does the code do and what are you not satisfied with? Automatically determine a threshold.
What potential improvements have you considered? DeveloperHub
How (specifically) do you want to improve the code? Automatically determine a threshold.

local memory_spike_detector = {}

local function check_memory()
	local memory_usage = collectgarbage("count")
	if memory_usage > memory_spike_detector.last_memory_usage + memory_spike_detector.threshold then
		print("memory usage increased by " .. memory_usage - memory_spike_detector.last_memory_usage .. " kB")
	end
	memory_spike_detector.last_memory_usage = memory_usage
end

function memory_spike_detector.start()
	memory_spike_detector.last_memory_usage = collectgarbage("count")
	if memory_spike_detector.threshold == nil then
		memory_spike_detector.threshold = 100
	end
	print("memory spike detector started with threshold " .. memory_spike_detector.threshold .. " kB")
end

function memory_spike_detector.stop()
	print("memory spike detector stopped")
end

memory_spike_detector.start()

while time() < 10 do
	check_memory()
	task.wait()
end

memory_spike_detector.stop()
#2

collectorgarbage is deprecated, if all your doing is getting the memory usage its better if you use gcinfo() its still the same as collectorgarbage("count").

#3

It’s not pure lua though. Unless it is?? No its not. 20 CHARS

#4

Actually on lua gcinfo() is deprecated however I’m not sure why on roblox collectorgarbage is deprecated and gcinfo() is not, but it’s better if you use it for now since that is what roblox recommended they might change this soon, not sure. It should still be the same as collectorgarbage(“count”)

#5

If you want to automatically determine a threshold, I would recommend trying a few calculations:

  1. start recording memory usage and time, and after enough data, take slope (y2 - y1) / (x2 - x1) (in this case y would be memory usage and x would be time) and see if it is higher than some threshold
  2. do the same as (1) but instead of a set threshold for slope, see if slope is significantly higher than the average slope (or significantly higher than the current max slope)
  3. If you can, collect memory usage for a number of players (or even one player, although it may not work as well), and calculate the outliers. If you want to do this you’ll need to find the quartiles (take dataset and find the median (Q2), then find median of the upper (Q3) and lower(Q1) halves), then find interquartile range (IQR) which is Q3-Q1, and outliers can be identified if the data point is > Q3+1.5*IQR or < Q1-.5*IQR (See Statology.org and Outlier.org)

I’m not an expert or a statistician, but this may be a place to start

#6

So i did this:

function get_interquartile_range(t)
    local median = get_median(t)
    local lower_half = {}
    local upper_half = {}
    for i, v in ipairs(t) do
        if v < median then
            table.insert(lower_half, v)
        elseif v > median then
            table.insert(upper_half, v)
        end
    end
    return get_median(upper_half) - get_median(lower_half)
end

function get_median(t)
    local sorted = {}
    for i, v in ipairs(t) do
        table.insert(sorted, v)
    end
    table.sort(sorted)
    local middle = math.floor(#sorted / 2)
    if #sorted % 2 == 0 then
        return (sorted[middle] + sorted[middle + 1]) / 2
    else
        return sorted[middle + 1]
    end
end

local t = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
print(get_interquartile_range(t))
#7

Nice, now you just have to collect data on the player(s) memory usage and check if it is an outlier, although I should mention that for small sets of data, it may not be reliable (potentially false positives and false negatives)

#8

I recommend collecting an average of the players memory usage. If the current spike is greater then its average memory usage than i would go based on that. I recommend reducing your numbers of gcinfo as much as possible to get a really fine result. Try sqrt it to get a lower number. This will create a very sensitive data-set. Meaning changes can easily be detected and seen through.

1 Like
#9

Good suggestion, but one question: if you just check against average memory usage, that means half of the players would be flagged as spiked (as they would be above average), how would you filter that out?