Introducing in-experience Mesh & Image APIs [Studio Beta]

OMG YES YES YES!

Finally ive been waiting for this update for sooo sooo long, thank you roblox Christmas came early.

5 Likes

How will this look? Specifically the moderation portion. Any sort of real time moderation sort of concerns me- while I see the need for it, I also am curious to know itā€™ll work on the backend / behind the curtain.

Do you plan to incorporate some sort of learning algorithm to prevent profanity, etcā€¦ similar to how current asset moderation works? If so will that not bottleneck people making creations?

4 Likes

Honestly im curious on this too. Iā€™m just concerned it will affect my usage for game engines / real-time renderers that need to run 60+ FPS.

Like, I really donā€™t think itā€™s possible to moderate every frame on a real-time game engine at 60 FPS without some major flaws.

10 Likes

dynamic bullet holes!!!

weird mesh projects!

yesss

12 Likes

Finally! A way to decrease the pixel density and make it seem retro.

6 Likes

Going off this though, one thing that really needs to be added to EditableMesh is batch methods, allowing you to send a whole table to the editable mesh instead of one at a time, you can see in the output log it takes quite some time to put the mesh together using the current API. (It also must be done on the main thread!)

12 Likes

Itā€™d be really awesome if we could read/write to the screen itself!

local cam = workspace.CurrentCamera

cam:GetImageData(Enum.ImageData.Viewport): EditableImage -- Gets image data of the viewport
cam:GetImageData(Enum.ImageData.PlayerGui): EditableImage -- Gets image data of PlayerGui
cam:GetImageData(Enum.ImageData.Both): EditableImage -- Gets image data of the whole screen, excluding CoreGui
cam:GetImageData(GuiObject | LayerCollector): EditableImage -- Gets image data of any even UI object, including LayerCollectors

-- Roblox Script Security:
cam:GetImageData(Enum.ImageData.CoreGui): EditableImage -- Gets image data of CoreGui
cam:GetImageData(Enum.ImageData.All): EditableImage -- Gets image data of the whole screen, including CoreGui 

This would allow for:

  • Proper shaders
  • Color picker tools
  • Taking screenshots so other players can see you play
  • Transition effects
  • Collapse many cloned ViewportFrames into one dynamic image

It would be a huge step if these were added!

(edit): Please give the DrawLine method a thickness argument!

41 Likes

What was performance like using regular decals?

1 Like

This is actually really useful. I have multiple projects which could directly benefit from this. And the UV stuff could really allow some cool stuff. Trying to create large EgoMoose assembled triangles out of edges and then trying to texture them was pretty suboptimal.

2 Likes

I never used decals. I used frames with UIGradients, and the performance wasnā€™t anywhere nearly as nice as this.

Previously, this engine ran around 40-58 FPS at 100x100

3 Likes

Iā€™m imagining it would only apply if you published the decal.

1 Like

Thatā€™s interesting I didnā€™t know that was even possible using UI gradients. And yeah I frames not decals.

Really Iā€™m just trying to figure out how fast editable images are. Because there are definitely some things I want to try, but if theyā€™re not fast enough for doing a lot of real-time shader effects on UI and whatnot, idk.

Has there been any data published about the speed or, whether or not we can set groups of pictures at the same time in an array?

1 Like

uh roblox? i think you made a mistakeā€¦ this is too many good things at once!!! wooo

2 Likes

Quick question for any engineers reading this topic: will EditableImages on mobile be compressed?

Iā€™ve got an open bug report centered around how ImageLabels are compressed when they really shouldnā€™t be on mobile. My use-case is pixelart-centered games.

My first tests with EditableImages have been genuinely mindblowing given how fast it is to render an image.
If the image data isnā€™t ever compressed when displayed to the user, this will act as a complete replacement for ImageLabel spritesheets for everything I could ever need one for, and will make my bug report completely obsolete :slightly_smiling_face:


As a bonus, hereā€™s a clip of me and a friend working on our EditableImage codepath, only to realize we messed up the channels:

19 Likes

Do EditableMeshes work with standard Parts and Unions? Would be even cooler if we could potentially change the geometry of Parts and Unions too.

Also is the 1024x1024 resolution limit on EditableImages permanent? At this point in time i really donā€™t see a reason as to why that limits needs to exist, at least on runtime. I do know that roblox would usually downres any published image on the platform to 1024x1024 though that dosenā€™t entirely mean we could not get some more wiggle room directly at runtimeā€¦

Also are there any performance metrics for these features? Like how much time would it take for EditableImages to update a pixel or how much time would EditableMeshes take to create a triangle and etc etcā€¦ Also with the performance metrics, wonā€™t the additional moderation features just kinda hit performance for these things? I understand the need for some moderation systems though i feel like as if moderation could entirely make these features next to useless if they just are slowed down to an insane extent just so they can keep up with moderation (Or at least force some back-end systems to constantly use resources just so whatever moderation system has the latest up-to-date systems).

As for moderation, what about false positives? These days itā€™s overall pretty easy to get hit by moderation actions for simply uploading assets that your AI systems believe are inappropriate content. Again, i get the need for moderation systems however i believe theres a chance these systems are made entirely useless due to the risk they pose in the face of moderation. Like what if you constantly update a EditableMesh/EditableImage? Would the moderation systems check each new version of said assets? If this is the case then those features would just be overall really dangerous to use as again, the moderation systems already hit people relatively frequently for one-time assets, and with some features that could potentially update assets at runtime (potentially even every single frame) there could be A LOT of situations in which developers get hit by the moderation system unfairly. There is a reason as to why people opt into uploading assets to the platform using alt-accounts.

Also what about players who create inappropriate assets? Right now its only and always the place owner who gets hit by the moderation if roblox deems something in their place as inappropriate, regardless of who made it. I donā€™t think there would be any easy nor any viable way for developers to simply just check and verify what sort of meshes and images playerā€™s make in their games. I personally really cannot put my faith on whatever moderation systems you guys have in place for these things alone. Players will eventually find ways to bypass ā€œsafe replicationā€ and ā€œsafety filteringā€ systems regardless.

Personally i believe that these sort of moderation systems are (and will still be) bottlenecking developer potential just to ensure the absolute ā€œsafetyā€ of other peoples children. I donā€™t think children should be exposed to bad content but i also believe its not my job to ensure the safety of children, nor do i want to be the victim of situations in which whatever moderation ai deems a random mesh/image me or a player made as a random test or just for fun (nothing even against TOS) as inappropriate and therefore should ban/terminate my account. I believe if roblox wants to actually step up with their technologies and systems then one crucial step would have to be to tell parents to actually PARENT their children and not let multi-billion dollar companies or random internet people do it for them.

10 Likes

this is crazy! im working on a 2d game and i have to say this is a literal game changer, can we by any chance get some more functions for editable images? iā€™d love to have a function to get pixels of a specific color on the image? this can be very useful to replace a certain color for another, im sure we can do this using readpixel and iterating through the entire image but i think an internal function for this would probably be more performant, thank you!

2 Likes

the meshes just randomly disappear depending on how i look at them, the terrain is made up of chunks with a editable mesh for each one

14 Likes

THIS. THIS RIGHT HERE. EditableImages are cool as hell already, but holy hell would that make this update so much cooler!

11 Likes

Finally itā€™s here! Now Iā€™m just waiting for you guys to make it so I can mess with EditableImages and SurfaceAppearances/Materials >:)

6 Likes

What is this ā€œsafety filteringā€? Safety in what way? Preventing ā€œinappropriate shapesā€? Or simply preventing saturating the network connection?

7 Likes