Screen sharing in Roblox

Help and Feedback Creations Feedback
#104

I’m doing this in JS and you can do this something like this

app.get('/image', async function (req, res) {
    req.headers.xres = req.headers.xres || 100;
    req.headers.yres = req.headers.yres || 100;

    const xres = parseInt(req.headers.xres, 10);
    const yres = parseInt(req.headers.yres, 10);

    console.log(xres, yres);

    try {
        const imgPath = req.headers.url || await screenshot({ filename: 'screenshot.png' });
        const compressedPixels = [];

        const image = await Jimp.read(imgPath);

        let prevColor = null;
        let count = 0;

        for (let y = 0; y < image.bitmap.height; y++) {
            for (let x = 0; x < image.bitmap.width; x++) {
                const { r, g, b } = Jimp.intToRGBA(image.getPixelColor(x, y));
                const currentColor = [r, g, b];

                if (JSON.stringify(currentColor) === JSON.stringify(prevColor)) {

                    count++;
                } else {

                    if (prevColor !== null) {
                        compressedPixels.push([prevColor, count]);
                    }

                    prevColor = currentColor;
                    count = 1;
                }
            }
        }

        if (prevColor !== null) {
            compressedPixels.push([prevColor, count]);
        }

        res.json(compressedPixels);
    } catch (error) {
        console.error(error);
        res.status(500).send({ error: 'An error occurred while processing the image.' });
    }
});
1 Like
#105

Personally, I use one sided plane meshes as I’ve explained above but a thing that matters is the properties of those pixels, since you wont need a touch event or anything similar. So you can just disable those properties.

2023-09-08-16-24-56-23irz
2023-09-08-16-24-56-23irz295×599 14.5 KB

I currently just use this, I have tried many other things such as the material; I’ve found that Neon is the most optimized one but due to the glow (you cannot fully remove the glow even with setting the bloom to 0) it would look weird. I’ve only found that using the neon material be good for darker colors such as another project I’ve done: Procedural grass

I have also looked into a bit with other ways to render it: such as using a color sequence, but unfortunately they are limited to about 20 key points per sequence if I remember correctly, which would mean I would have to rework my compression methods

1 Like
#106

I see. How do you actually use those plane meshes? I have imported the fbx file you gave and I dont understand how you can set each pixel individually. Could you help? Thanks!

EDIT: I am dumb haha. I see you just use normal parts but using the mesh.

1 Like
#107

I haven’t found anything better at this time that could function on large scale besides just cloning each pixel mesh, and then individually changing the colors

You would clone them from top left to bottom right, put them in a table and then in a for loop to change the colors you can just do something like this

for Pixel:number, Color:string in Colors  do
	Pixels[Pixel].Color = Color3.fromHex(Color)
end

I’m happy to help, if you have any more questions, feel free to ask about anything
:happy2:

#108

Yeah I have that down. I see your mesh is huge. Is that normal?

#109

That is most likely because it is an .FBX, they aren’t really good at maintaining a nice size.

That’s why I recommend using .OBJ, but it seems like I didn’t do that there.
Most likely since I was rushing it a bit and wasn’t worried about the sizing of it, since I calculate the size in the actual script so that lower and higher resolutions will maintain an approximately similar size

#110

Ah I see. I just used a single scaled down part and clone it as cmd bar cant set MeshId’s

#111

Wow planes are SOO much faster. Im getting a crisp 60fps now.

1 Like
#112

cool :+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1:

#113

little update: Screen Sharing!

I’ve open sourced it

#114

I made a simmilar system awhile ago with @testingacc333330, based on the video (even though your display is slightly lower res than our tests) you managed to update more often, we had to do some weird janky interlacing to fake 30FPS (our display managed 5-10FPS updates)

#115

Honestly this is pretty cool. It’s nice to see other developers trying to make live screens in roblox.
I’ve managed to so far play 190 X 90 @ 30fps in real time in studio and in roblox. (Up to 530 X 300 @ 30fps is possible, but the game can’t handle it.)

Although that took forever to make an efficient encoding / decoding system.
But the best part about it is that I managed to also get live audio working with it too, but it sounds terrible but voices are somewhat recognisable.

I’d suggest using this UI canvas roblox plans to add in as it seems to be quite efficient, and I suppose wait for roblox to improve sounds.
There are so many limitations with audio and video playback sadly, but none of the less, good luck with your system

#116

Could I know a bit of how you make live audio, is it by sampling the current pitch and volume, and then sending it to Roblox?

#117

Well it is related to pitch and volume. If you upload a sinewave at 200hz, you can adjust the playback speed from 1 - 2 (Anything out of these ranges makes the sinewave sound off)
So you could get frequencies from 200hz to 400hz.
Now with actually getting this information from your pc, you can use pyaudio if I remember correctly and use FFT to get the various frequencies and amplitudes.
I might have some old code from one of the versions I made in python, but my latest system uses javascript.
Hope this helps.

EDIT: Becareful when uploading the sinewaves to roblox. Roblox moderation system likes to flag these sounds as excessively loud when that isn’t the case, but this seems to apply to frequencies below 256Hz

#118

Hey so, ive been trying to make a webserver compatible version of the code, and i was using render.com to do so, however it keeps throwing me server errors and so on and so forth. I spent days trying to figure it out but i just cant, my webserver and roblox just dont want to sit down and communicate.

#119

Me and a friend have been trying to make some sort of program like this for ages. However, the biggest culprit in this is changing the color of objects in Roblox. it’s too slow for resolutions bigger than 144p basically which is really unfortunate. I’d love to see someone figure out audio vis because that would be the best thing to add to games, for certain reasons.

1 Like
#122

Yo i kept this thread alive when it was dying, i am really happy people started to think about making this, i stopped making my project but with more knowledge i should remake it from scractch someday

1 Like
#123

cool thing i like this idea we need audio now

1 Like
#124

Can yalls tell me the code you did to do this? I asked while this post was new and it seems to have grown a lot

1 Like
#125

I’m having problems with the screen. I’m not exactly sure HOW to explain it, but here’s a video.

2 Likes