It would be nice if GuiService:GetScreenPixelColor(number X, number Y) was added.
GuiService:GetScreenPixelColor(number X, number Y) returns a Color3 value based on the pixel coordinate it is called on.
Currently there is no real way to get the color of a current pixel of the client screen at the time, whether it be ScreenGui or world replication.
It would be cool if it was added as it could allow for split screen cameras to be made along with other cool things like projecting your screen to a part through SurfaceGui (youâll need to replicate all the pixels of each personâs screen for both scenarios though, may kind of be laggy but ok ÂŻ\_(ă)_/ÂŻ) and stuff like that.
(edit: if used like this in split screen camera and stuff it will crash the players Honestly though. It would be good though if GetScreenPixelColor exists because we can possibly create GUI things that color based on the environment, where the player is pointing his/her screen at.)
Using this could cause some unwanted behavior, combining this with ViewportSize would literally allow you to take screenshots of peopleâs ROBLOX windows, which I would personally like, however it would probably violate COPPA because âusers could accidentally paste sensitive info into the chat windowâ or something.
This doesnât match your feature request, did you mean to ask for a âSetScreenPixelâ?
Split screen makes the most sense when both players are local, not really for online multiplayer. Moreover, sending images back and forth to simulate split screen would probably not be the ideal way of implementing this.
âmay kind of be laggyâ is an understatement, I donât think it could be used for this at all, really. Would like a proper separate camera gui feature thing though, it has been mentioned.
It actually does match his request. He wants to get all of the pixels of someoneâs screen, and then project it on a SurfaceGui of a part. SetScreenPixel would be just making a {0,1}{0,1} Frame objectâŚ
Have and 1 x 1 frame for each pixel is going to be a whole world of lag. Most people have 2,073,600 pixels, and setting and getting them one by one is like, actually terrible.
If you want to test if itâll lag or not here is some code to find out!!
local ScreenGui = Instance.new("ScreenGui", game.Players.LocalPlayer.PlayerGui)
local Frames = {}
for i = 1, 2073600 do
local Frame = Instance.new("Frame")
Frame.Parent = ScreenGui
Frame.Size = UDim2.new(0, 1, 0, 1)
Frame.Position = UDim2.new(0, math.random(1, 2073600), 0, math.random(1, 2073600))
table.insert(Frames, Frame)
end
while wait(1/30) do
for i,v in pairs(Frames) do
v.BackgroundColor3 = Color3.new(math.random(), math.random(), math.random())
end
end
Edit: I didnât run this because Iâm 99% sure itâll crash studio
If the use case is screenshots, this is not the right way to implement this
If the use case is shadersâŚthis is not how you do shaders.
If the use case is picture-in-picture for security cameras, this is again not how to implement this.
Dealing with images on the cpu is rather slow so all of the above solutions would be done with a much faster implementation supporting the gpu (shader languages, rendering to a texture for PIP display, etc).
You could actually get the color of a pixel btw. A simple recursive on the GUIs (gets a lot more complicated if you care about Z-Index), and then there is camera api where you could also work this out if you need it for a small thing
For the future: The best way to get heard by engineering is to show them a problem, not a solution. Any situation where you would need to work directly with pixels probably counts as a problem.
Imagine a 1920x1080 monitor at 60 FPS. Youâre looking into manipulating over 124,416,000 (1920x1080x60) Color3 values PER SECOND in Lua. Each Color3 value is represented with doubles (8 bytes, 64 bits) there are 3 doubles used to represent RGB values in Color3, (24 bytes, 192 bits) so you would be consuming over 23,887,872,000 bits, or 23.8 GBs to be short, just to represent all that data on the CPU. And thatâs not even counting the memory youâd be consuming to visualize all of those pixels as frames. Roblox only allows you to have 16,384 GuiObjects on the screen at a time (iirc), so you wouldnât have much luck there.
TL;DR - It wonât work for what youâre thinking. Let the GPU do its job and save the CPU from having a heart attack.
Honestly though. It would be good though if GetScreenPixelColor exists because we can possibly create GUI things that color based on the environment, where the player is pointing his/her screen at.)