GuiService:GetScreenPixelColor(number X, number Y)

Can confirm, we are suckers for a problem to solve.

7 Likes

From what I’ve read this sort of method would be a way off-course solution for what you requested it for.

Why not just add some GUI element that serves as a viewport, where you can specify a Camera and a viewport size/position?

Oh you have no idea.

Imagine a 1920x1080 monitor at 60 FPS. You’re looking into manipulating over 124,416,000 (1920x1080x60) Color3 values PER SECOND in Lua. Each Color3 value is represented with doubles (8 bytes, 64 bits) there are 3 doubles used to represent RGB values in Color3, (24 bytes, 192 bits) so you would be consuming over 23,887,872,000 bits, or 23.8 GBs to be short, just to represent all that data on the CPU. And that’s not even counting the memory you’d be consuming to visualize all of those pixels as frames. Roblox only allows you to have 16,384 GuiObjects on the screen at a time (iirc), so you wouldn’t have much luck there.

TL;DR - It won’t work for what you’re thinking. Let the GPU do its job and save the CPU from having a heart attack.

9 Likes

LPT: If you ever need to measure anything in units of millions of pipeline stalls/frame, just run

2 Likes

Or do it on the GPU which is built for that sort of task.

2 Likes

When will they finally add this man, you could do so much cool things with this and even if “people will be bypassing it” nothing will change people are already bypassing stuff by just using parts as pixels or 1x1 frames. this needs to become a feature it adds so much cool things to do!

1 Like

Not quite in the form suggested here, but the pieces to make this possible will becoming together soon.

CapureService is in development (the final public API will look a lot different than what you see there) as a way to deal with taking screenshots, and combining that with EditableImage you’ll be able to take and read / modify the pixels in a screenshot.

9 Likes

I hope It doesn’t cause much memory to use while RenderStepped, since I’m trying to do if you look at a pixel the more black it is the more visible it becomes like a custom blend mode basically

As discussed above: There’s unfortunately no way you’re applying any full-screen effect at 60fps be reading individual pixels. Also, the screenshot path will require user interaction (you can’t just take a screenshot silently in the background). Either way, screen space effects aren’t happening using this.

4 Likes

Will these screenshots be available in-game? Possibly as some sort of temporary rbxscreenshot content type would make sense.

Will there ever be an api, which will be able to return the pixel color of the render, without any gui elements? This would be really useful to create custom shaders, such as color based lens flares and screen space reflection.

create custom shaders

no that’ll never happen and it’s not even practical to do in the first place

it’s both expensive to pull the viewport’s framebuffer and to perform shader operations entirely in software

gpus specialize doing things in complete parallel and are generally better in arithmetic than cpus

I might /+ctrl+v+enter without looking at the chat bar thinking that i copied something when i forgot that my previous copy was my ip
And oh no turns out something went wrong with copying now i accidentally announced my ip im cooked
Pls roblox remove the chat features its unsafe :frowning:

Search image loader people can put high quality photos using it lol
If you are scared of player fps dropping then no proprem :fire: go to each 9 pixels split them
And get the colour at the middle or make some algorithm that gets the most repeated colour
There you go you made it 9 times smaller now🔥

what about different screen sizes?

That’s something the programmer should care about, not Roblox.

Actually, it looks like you intentionally blocked this:
image

Granted, I can understand not being able to create EditableImages straight from a capture as that would open doors for invasive surveillance; however, I do not understand why the original suggestion of a built-in guiservice (or camera) method to get the color of a single pixel (or averge color of a chunk) isn’t already implemented. We aren’t asking for it to include gui elements on the screen or anything like that, just what the camera sees. Just the rendered color of a pixel or chunk.

Why would this be useful?

  • Dynamically adjust font or UI color based on background color behind it (sampled via pixel color) for better readability.
  • Sample screen pixels for color-based analysis (e.g. adjusting lighting levels when a player is in a dark area or automatically enabling a flashlight).
  • Infinitely more memory-efficient and less computation time compared to raycasting and expensive calculations to factor in lighting/shadows.

We could really use this.

2 Likes

Yes, to avoid capture of problematic information like players Robux balances.

Yes, that wasn’t meant to be a question though - I acknowledged that in the second sentence.
Your thoughts on the rest?

1 Like

That problem can be fixed by just making the function ignore the chat Gui.