Converting image to parts - Python

Netpex22 · November 23, 2023, 8:16am

Hey,

Wanted to share some basic code I wrote, I wanted to create a way to display the album cover of the song someone is currently listening to in a plugin using parts as pixels and a viewport frame. I know I could do something like Local File Importer but I want to stick with a pixel(y) or blocky like look.

I use Python to convert images into Lua files. Then using a module create parts that act as pixels to generate something that closely resembles the image.

Python Script:

import os
from PIL import Image

factor = 6.4 #Rate of change to resolution 1: Original resolution. ie: 640x640 - 6.4 - 100x100
rate = 100 #Rate that parts are created at

for filename in os.listdir("input"):
    image = Image.open(os.path.join("input", filename))

    image = image.resize((int(image.size[0]/factor), int(image.size[1]/factor)))
    pixels = image.load()

    with open(f"{os.path.join('output', filename)}.lua", 'w') as f:

        bits = []

        for y in range(image.size[0]):
            for x in range(image.size[1]):
                p = pixels[x, y]
                p = ("{:03d}".format(p[0]), "{:03d}".format(p[1]), "{:03d}".format(p[2]))

                bits.append(''.join(map(str, p)))

        f.write("require(script.Parent.Parent):Draw("+str(rate)+", Vector3.new(0,0,0), {"+str(image.size[0])+","+str(image.size[1])+"}, '"+''.join(bits)+"')")
        f.close()

print("Done!")

Lua Script:

--Netpex 2023

local ImageGenerator = {}

function ImageGenerator:Draw(rate, origin, size, image)
	
	local pos = origin
	local activePixel
	local color = {}
	local bit = ""

	for p = 1, (size[1] * size[2])*9, 1  do
		
		bit = bit..tostring(string.sub(image, p, p))
		if p/3 % 1 == 0 then
			table.insert(color, tonumber(bit))
			bit = ""
		end

		if p/9 % 1 == 0 then
			activePixel = Instance.new("Part", workspace)
			activePixel.TopSurface = Enum.SurfaceType.Smooth
			activePixel.Size = Vector3.new(1, 1, 1)
			activePixel.Anchored = true
			activePixel.Position = pos
			pos -= Vector3.new(1, 0, 0)

			activePixel.Color = Color3.fromRGB(color[1], color[2], color[3])
			table.clear(color)
			bit = ""
		end

		if p/(rate*9) % 1 == 0 then
			wait()
		end

		if p/(size[1]*9) % 1 == 0 then
			pos = origin - Vector3.new(0, 0, p/(size[1]*9))
		end
	end
	
end

return ImageGenerator

Image file example:

require(script.Parent.Parent):Draw(100, Vector3.new(0,0,0), {40,40}, '000000000000000000000000000000000000000000000000000000000000000229025025226034025231035026230035027231035023204051051000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000227033027226035026233036027231036027227035026227034025229035025228035026232037030255000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000225034026235036027226035026226035026228035026231036027235036027233036027226035026227035026228035026229034025229034025000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000226034025234036027226035026226035026226035026226035026226035026226035026228035026232036027235036027232036027227035026227034025228035026227034027234030030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000231046023225034026230036026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026228035026232036027235036027231036027227035025228034026227035025228033026230036024000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000225034024228035026227035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026229035026233036027235036027230036026227035026227035026228035026230037024240030030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000225034025235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026229035026233036027235036027228035026228034025229035025228035026227036027218036018000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000226034026234036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026229036026234036027235036027227035026227035026226035026229034025230035025218036018000000000000000000000000000000000000000000000236036018224034025230036026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026230036026234036027235036027227035026226034026000000000000000000000000000000000000000000000228035026228035026227035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026232036027227034026000000000000000000000000000000000000000000000225034025235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026231036027226035025000000000000000000000000000000000000000000000225034026233036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026226035025227032027000000000000000000000000000000000000226028028225035026229035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026231036027226034026255036036000000000000000000000000000000000000223035027229035026227035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027227034026000000000000000000000000000000000000000000000226034026235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026234036027224035027000000000000000000000000000000000000000000000226035025233036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026232036027235036027232036027228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026228035026227035025221033026000000000000000000000000000000000000231034023225034025229035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026233036027225035026227034024228034026227035026226035026233036027233036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026231036027225035025204051000000000000000000000000000000000000000226035025231036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026234036027226034025000000000000000000255000000233036029226034025227035025226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027226035025000000000000000000000000000000000000000000000226034026235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026228035026227035025226035028000000000000000000000000000000000000000000000227035027230036026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026233036027225035026000000000000000000000000000000000000255000000226035026232036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026231036027225035025255063000000000000000000000000000000000000000000000000226035026235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026228035026226035026224038023000000000000000000000000000000000000225039029225035025228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027226035025000000000000000000000000000000000000000000000255000000226034025232036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026232036027225035026170000000000000000000000000000000000000000000225036026232036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026232036027225035026000000000000000000000000000000000000000000000227036027225034026228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027226035026000000000000000000000000000000000000000000000225034025235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026228036026230036027230032024000000000000000000000000000225035025233036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026232036027226035025000000000000000000000000000000000000255000000226034025232036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026233036027233036027227034026227034025229035025227034024226035025234036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026228035026226034026227036027000000000000000000000000000000000000229034025225034026228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026228035026232036027235036027230036026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026232036027226034025255000000000000000000000000000000000000000000223034024233036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027226035026000000000000000000000000000000000000000000000226035026235036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026231036027227034024000000000000000000000000000000000000191000000226035026232036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026229035026225035025232033022000000000000000000000000000000000000228033026225034026228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026233036027226034025000000000000000000000000000000000000000000000226034026230036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027227035025000000000000000000000000000000000000000000000225035026233036027226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026229035026224034027000000000000000000000000000000000000000000000225035026229035026235036027234036027230036026227035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026229035026226035026226042028000000000000000000000000000000000000000000000231023023228036026229035025228035026227035026227035026235036027233036027229035026227035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026233036027225034026000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000221034034229034025229034025228035026227035026229036026235036027233036027229035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026235036027225034025000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000229038025227034027229035025226034025226035025231036027235036027232036027228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026227035026228035026224034025000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000224030030227034027228035026228035026227035026232036027235036027232036027228035026226035026226035026226035026226035026226035026226035026226035026226035026226035026226035026230036026225034025236036018000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000221034025229035025228035026227035026227035026233036027235036027231036027228035026226035026226035026226035026226035026226035026226035026234036027225034026000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000255000000225037022228035026227035025227035025227035025234036027235036027231036027227035026226035026226035026235036027225034025000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000255063000228033026227036024228034026228035026226035026231036027233036027227035026224035025000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000218036036232033027228035026230036025225035026233042021000000000000000000000000000000000000000000000000000000000000000')

Adding the images to Studio:

Once the Lua files are generated, adding them into Roblox Studio is straightforward:

Importing the Script: Import the generated Lua files into the Roblox Studio module.
Execution: Run the code within Roblox Studio to visualize the images.

I’m sure this poorly written in many regard, please leave any feedback or ideas you have to improve the concept.

Image to roblox.zip (107.7 KB) (Python)
ImageToRoblox.rbxl (104.8 KB) (Roblox)

Thanks for your time
-Netpex

NoxhazeI · November 23, 2023, 8:18am

Amazing python this time? Good thing I know good amount of python so now I can finally understand the logic. Thank you for this! Oh btw its meant to be in #resources:community-resources as its a resource rather than being only a showcase.

Netpex22 · November 23, 2023, 8:25am

Oops! You’re right thank you, switched it over.

VSCPlays · November 23, 2023, 10:32am

Another day, another moderation bypass.

SomeFedoraGuy · November 23, 2023, 10:40am

Changing the resolution means more parts are created, which means better quality recreation?

Great module. You won’t be able to use it in games but it’s fun to make it work.

Netpex22 · November 23, 2023, 7:30pm

Of course, yes the factor effects the resolution; if your image is 640x640 and your factor is 6.4 the out would be 100x100. The original idea was just to display the image in a view port via a plugin widget and keep it when editing in studio.

Netpex22 · November 23, 2023, 7:34pm

Hey, that certainly wasnt the point; however I understand how this could be used to bypass moderation.

optimizedcoder · November 24, 2023, 9:47am

This is an interesting resource; being able to convert a string into an array of parts. I do have a few suggestions though. I think you should use greedy meshing over creating a bunch of parts.

It’s an optimization strategy where you combine parts that are in the same row and or column if they’re the same colour. For example:

Second example:

I personally think greedy meshing would really expand the capability of this resource and allow developers to create more diverse images. Take for example spinning images - Less parts will equate to better model pivoting:

A plugin would also be a nice addition; allowing for easy image integration. But don’t stress it.

Any who, thank you for the contribution

Netpex22 · November 25, 2023, 11:29pm

Hey!

Here’s a version with a form of greedy meshing!

--Netpex 2023

local ImageGenerator = {}

function ImageGenerator:Draw(sc, rate, origin, size, image)
	
	local folder = Instance.new("Folder", workspace)
	folder.Name = sc.Name
	
	
	local pos = origin
	local activePixel = nil
	local lastPixel = nil
	local color = {}
	local bit = ""

	local function pixel(pixel)
		pixel.Color = Color3.fromRGB(color[1], color[2], color[3])
		pixel.Name = pos.X.." "..pos.Z
		lastPixel = pixel
		activePixel = nil
		table.clear(color)
		bit = ""
	end
	
	local function isSimilar(range, color1, color2)
		local status
		color1 = {color1.R, color1.G, color1.B}
		color2 = {color2.R, color2.G, color2.B}
		
		for i, value in pairs(color1) do
			status = false
			for dif = -range, range, 1 do
				if math.floor(value*225)+dif == math.floor(color2[i]*225) then
					status = true
				end
			end
			
			if not status then return end
		end
		
		return true
	end

	for p = 1, (size[1] * size[2])*9, 1  do

		bit = bit..tostring(string.sub(image, p, p))
		if p/3 % 1 == 0 then
			table.insert(color, tonumber(bit))
			bit = ""
		end

		if p/9 % 1 == 0 then
			if lastPixel and isSimilar(5, lastPixel.Color, Color3.fromRGB(color[1], color[2], color[3])) and lastPixel.Position.Z == pos.Z then
				lastPixel.Position = lastPixel:GetAttribute("OriginalPos") - Vector3.new(lastPixel.Size.X/2, 0, 0)
				lastPixel.Size += Vector3.new(1, 0, 0)
				pixel(lastPixel)
			else
				activePixel = Instance.new("Part", folder)
				activePixel.TopSurface = Enum.SurfaceType.Smooth
				activePixel.Size = Vector3.new(1, 1, 1)
				activePixel.Anchored = true
				activePixel.Position = pos
				activePixel:SetAttribute("OriginalPos", pos)
				pixel(activePixel)
			end

			pos -= Vector3.new(1, 0, 0)
		end

		if p/(rate*9) % 1 == 0 then
			wait()
		end

		if p/(size[1]*9) % 1 == 0 then
			pos = origin - Vector3.new(0, 0, p/(size[1]*9))
		end
	end

end

return ImageGenerator

To do change the degree of difference in color change the first value in isSimilar()

if lastPixel and isSimilar(5, lastPixel.Color, Color3.fromRGB(color[1], color[2], color[3])) and lastPixel.Position.Z == pos.Z then

TheRealANDRO · November 25, 2023, 11:44pm

Eh, this still falls under the “Misuse of Roblox Systems” (ToS) though.
I’m not completely against this as this requires using a python script and is not something that in-game players can use, but still bypasses moderation. It can be used greatly but at the same time can be used for many bad purposes.

Magus_ArtStudios · November 26, 2023, 1:42am

I was working on a similar project!
I wanted to be able to allow players to make AI generated art in game! You get a response from the inference API. Probably requires a hosted server to execute a project like that!

local function StableDiffusion()
local API_URL = "https://api-inference.huggingface.co/models/CompVis/stable-diffusion-v1-4"
local headers = {Authorization = Bearerkey}
-- Define a function to query the API with a payload
  local httpService = game:GetService("HttpService")
local function query(payload)
    -- Use HttpService to send a POST request
  
    local response = httpService:RequestAsync({
        Url = API_URL,
        Method = "POST",
        Headers = headers,
        Body = httpService:JSONEncode(payload)
    })
    -- Return the response content
    return response.Body
end

-- Query the API with an input
local image_bytes = query({
    inputs = "Astronaut riding a horse"
})
return image_bytes
end

VSCPlays · November 26, 2023, 7:22am

I agree, Anyone can make a repl to make it work as expected

pixnua · November 28, 2023, 6:51pm

The python script is giving an error on Line 7:
FileNotFoundError: [WinError 3] The system cannot find the path specified: ‘input’

I’m probably doing something wrong, what version of Python is this for?

edit: Nevermind, I assumed that it would take input once ran but I didn’t really look at the script, I now realize that I’m supposed to type the directory in the “input” field.

pixnua · November 28, 2023, 7:13pm

Alright I got the script to work properly

cat

Magus_ArtStudios · December 16, 2023, 7:41pm

Here is the final piece to the puzzle to create a AI image generator in the future when this feature is released!
With this API you can funnel the image json to the image api upload and have ROBLOX handle moderation and the like.
Introducing in-experience Mesh & Image APIs [Studio Beta] - Updates / Announcements - Developer Forum | Roblox

This will also be important if you want to use a AI generated experience using the future AI generated 3-D software.