So, what I’m trying to do is pretty insane and stupid in terms of optimization. I made a python script that takes a screenshot and puts 50 pixels (for EditableImage) for each panel (terribly implemented “LED screens” that are just surfaceGUIs and an imagelabel with a texture to simulate the “LEDs”) in each panel. Roblox will send the amount of panels to the pyscript and python will take a screenshot at whatever resolution-- it depends on the width and height of the said panels.
Why I’m trying to do it
This was just an experimentation of the things that could be done with EditableImages. And when I find the right solution. I will try to make a knockoff of Resolume Arena.
How I did it
So in an example; here’s a 40 by 19 “screen” that has in total 760 editable images. Obviously that’s a lot and of course there will be some slowdowns.
This video shows how long it would take to render 760 editable images or 2000x950 resolution (38000 pixels in total). I made a terrible implementation in my python script that if a specific panel was requested, it would send back those specific pixels. Meaning that HTTPService is requesting over 700 times in less than 40 seconds, which is obviously not good.
Here’s the output of the python script. How this works is pretty hard to understand. And it’s not good in performance. Basically how it works is HTTPService will GetAsync on “localhost:5000/screenshot/(X)/(Y)” and python will send back 50 of those pixels. It’s pretty hard to explain how it actually works but in example of the last line of the output; /screenshot/38 (width in coordinates)/ 18 (height in coordinates) they start from 0. This isn’t good because it’s making way too many requests in less than a minute and it’s just not optimized.
Some of the Roblox Script:
for i,v in pairs(workspace.test:GetChildren()) do
task.spawn(function()
local result = HttpService:GetAsync("http://localhost:5000/screenshot/"..v:GetAttribute("X").."/"..v:GetAttribute("Y"))
--print("get result")
local pixelData = HttpService:JSONDecode(result)
--print("decode")
v.SurfaceGui.ImageLabel.yes:WritePixels(Vector2.new(0, 0), Vector2.new(50, 50), pixelData)
end)
--print("write")
end
It’s pretty easy to understand what’s happening here and I know I could do better.
What needs to happen
Make it more efficient.
Use Parallel Luau is possible
Dramatically lower the amount of requests that are made.
Use something else instead of the /screenshot/x/y method
Efficiency can be accomplished by studying how video/stream decoding works. For example a simple optimization you could make is only call write pixels on pixels that have changed since the last frame, by creating 2D “groups” of the changed pixels. That way you can also decrease the amount of data you have to send over. Additionally you can compress the data and decompress it back on Roblox(using community compression libraries). Because this is a stream and not a video file, you can use many sneaky ways to decrease the amount of data you have to deal with by creating small room for error(that doesn’t matter much when streaming, since we’re optimizing for speed).
I wouldn’t recommend using threads through task.spawn here, because the task of writing pixels on an editable image is still resource-heavy and the lag would remain. Instead I would go down the Actor path, using multiple threads within the CPU to render different equally split parts of each frame.
For this you can use PNGs and a known community library that can understand and read them. That way python can send over the entire PNG at once(that doesn’t need to be compressed as mentioned above because the file format already does that) and only make a single request per frame. Also you can request multiple frames at once since you control the server. For example you can tell it “give me 60 PNGs representing all frames of a second, in a single request”, that way you only need 60 requests per minute, much fewer than the 500 limit.
I made something a while ago but using a method that isn’t possible anymore.
This was before editable images were a thing so it was using lots of parts
I would have to peek in my old drive for the python (maybe ruby?) scripts but if you want them let me know, it wouldn’t be a solution but a good example.
I’m still not quite sure on how the MicroProfiler works but I can see that it is indeed JSONDecode causing lag spikes, even with actors for parallel (i may be using it entirely wrong).
What I do need to do is completely remake my python script that screenshots and converts it into the format that EditableImage uses. The Python script takes a screenshot and crops them to any specific area that is requested. And I also tried to send the output in whole (every pixel instead of just 50 pixels each) And…
Doing that sometimes causes Roblox Studio to freeze up for 30 seconds, hell, even up to a minute. It can GetAsync without any lag but jsondecode causes way too much lag spikes. Even at half a mb. So I’m kinda just stuck here. Here is my terrible attempt at Parallel Lua
I’m still a bit dumb when it comes to Actors and I know I’m definitely doing this wrong. Though I heard that :WritePixelsBuffer exists but I’m pretty sure that’s now how i think it works.
