Roblox gui that shows computer screen in realtime

Resources Community Resources
,
#1

Introduction

Hello everyone, I was up all night, working on something you guys might like. This is a thing on Roblox I made that will allow you to see your own computers screen in live time in Roblox.

Sending Pixels

To make this work, I have to loop through every pixel on my screen using python and get the color. But the data that has to be sent to Roblox, is way too big to be useful in any way. So I divide the screens X and Y resolution by 20, take a screenshot, read the color of every pixel in the downscaled image, apply it to an array, and send it to Roblox.

Reading Pixels

To read pixels, I have a function that will receive all the colors of every pixel using HttpService. Once I get the table of colors, I run a function that will spawn tons of frames on a surfacegui (each frame will represent a pixel). Next, I loop through every pixel, just like I do in the python code. But instead of getting the color, I set it to the color in the table that represents that pixel.

Final thoughts

I think that this is a pretty cool project, and it was fun making it. I definitely think stuff like this has potential in Roblox, and I might release this as an asset if it’s not against TOS. I will release code if you guys want it :slight_smile: .

(Here’s a little showcase)
Video

Screenshot 2022-02-27 125557
Screenshot 2022-02-27 125557974Ă—549 317 KB

(also credit to this post for giving inspiration and helping me with this)

How to use in your own project

(THIS ONLY WORKS IN STUDIO UNLESS PORTFORWARDED)
Use of this is not recommended in a game you plan on actually making. Unless you optimize this, it’s kinda laggy when receiving pixels. To download this into your own project go to the model and put it into your game. Next, you’re gonna want to download the actual thing that runs on your computer, this python file. Now, go into your game settings, and enable HTTPS requests. Lastly, open the python file, and run your game. You should see your computer screen running on the screen in game. If you run into any problems, message me, or solve it. Thanks :slight_smile:

88 Likes
Proof Of Concept - Using Pixels To Display Images
Proof Of Concept - Using Pixels To Display Images
#2

Very cool! If you are putting this in #resources:community-resources then you need to release code or have a model that we can get. Otherwise, put it into #help-and-feedback:creations-feedback but I would definitely like to mess around with this so please release it along with a tutorial for setup?

3 Likes
#4

I edited the original post to add a source, and how to use it

3 Likes
#5

I honestly thought this was impossible! Thanks for sharing this, I’ll make sure to check out this code…

3 Likes
#6

Nice job, I made something like this a while ago but also added a functioning keyboard and mouse
https://gyazo.com/72bf073bc903bf5ddbc3e0c0e5d9a3d9
https://gyazo.com/b6d5e6b7265d63a1b8862a6b9c87cf73

keyboard:
https://gyazo.com/00be007fa357bd5c667594d76e75d38f

9 Likes
#7

I swear I thought that keyboard was from real footage, dang!!! GG

5 Likes
#8

it’d be cool to see someone make a custom VR desktop monitor app in Roblox

3 Likes
#9

I have seen something like this before, But this time, Open sourced!

But i’ve ran into a problem. Flask is required to install and i installed it on my python program, But it still didnt work.

ERROR ON PYTHON IDLE 3.9:

Traceback (most recent call last):
  File "C:\Users\Usuario\Downloads\rblxdesktopsource-main\rblxdesktopsource-main\RblxDesktop.py", line 1, in <module>
    from flask import Flask, render_template, request
ModuleNotFoundError: No module named 'flask'

im using pip 21.3.1

2 Likes
#10

run
pip install Flask
or
pip3 install Flask

1 Like
#11

I already have executed that command. Still no work.

1 Like
#12

I was going to do something like that but decided to open source. How did you go about sending the screen to roblox? Did you do something similar to me?

1 Like
#13

did you try pip install flask with a lowercase “f”?

2 Likes
#14

Wrong reply, but yeah I did pretty much what you are doing with a flask server

2 Likes
#15

I am studying your code… probs to make something. question doe, does it actually continuously send requests to the server and the server performs a screen shot and turn it into pixels and send it back to roblox?

1 Like
#16

I already have installed flask, but it doesnt just work.

1 Like
#17

This is because you have the wrong interpreter selected. I had this issue a while back, but how I fixed it was changing the interpreter (assuming ur code editor is VS code), go to view → command pallet

image
image1260Ă—94 8.36 KB

image
image359Ă—682 31 KB

Then type >Python: SelectInterpreter. Then, select the latest python interpreter and click enter on it:

image
image738Ă—234 41.1 KB

Then restart VS code

#18

I dont have VSCode open but ill see if it works once i get it open (it takes a long time)

#19

Yes it doesz it’s very unoptimized and lags the game a lot.

#20

Don’t agree with you, It’s very well made. Multiple developers are working on it and making it better every day as it’s a Micorosft app made with electron. It might be your system, I’m not really sure but it runs perfectly fine with studio open on one of my old 2016 Macbooks Airs.

#21

That’s friggin genius! lol, imagine… I would make my own similarly but its kinda different. I can’t say what im making cuz its what others are already making but probably more optimized. Definetly more optimized. Maybe i’ll open source it. Currently its using a combination of an open sourced image processing module and a python script stored within a replit server to be called to do a task.

Then, the roblox script sends a request via HTTPService using RunService and pcall to constantly send it each frame, when the webserver recieves it, it turns the video into an array and sends it back to the roblox script.

1 Like