[Studio Beta] Major updates to in-experience Mesh & Image APIs

Thanks for pointing this out! It’s fixed in the snippet now

1 Like

Wait, are you telling me we cannot create an editableImage from free toolbox images now? Why is this? I understand misuse with images, but images in the toolbox are moderated anyways. Why change this?

5 Likes

Just to be clear, PromptCreateAssetAsync() would be an in-experience API that we are hoping to ship in the future. There are a number of issues we need to work through before we can get that out the door.

Publishing within Studio is hopefully something that will be out much sooner than that since the issues are a lot fewer within Studio.

  • Nothing planned for right now, but that is an interesting idea
  • Unfortunately, not in the near future. 1024x1024 already uses a ton of memory on clients since we need to make all those pixels available in memory for the non-async APIs to function like you would expect.
  • The group permission check is to ensure you can only load assets that you (or your group) owns to prevent asset misuse. In Studio, the logged in user will be able to access any assets they own or groups they are part of own. Are there valid use cases your plugin would like to achieve that would not work with these limitations?
3 Likes

The EditableImage APIs are not designed nor optimized for this use case (real-time rendering).
Like all APIs in the engine, you can hack your way around them, but YMMV.

That being said, you might want to look at the new WritePixelsBuffer() method since that will let you “batch” write a number of pixels with a buffer.

3 Likes

You’d be surprised by how many people use EditableImage for real time rendering. But as you have said using WritePixelsBuffer is really efficient for this use case

16 Likes

Wow. It’s not very often we get pretty large updates like this. I do have a couple of questions for all of this though.

  1. I question the overall functionality of the memory management behind these systems. Would it be possible for the automatic memory management to not allow any Editable instances from being created for extended periods of it? I believe that devices which suffer from limited memory won’t exactly treat these new Editable systems well at all and potentially cause games who use these features heavily to either break or simply force the user on a very long “retry loop” due to the lack of memory. I believe this issue may be even worse (if it actually IS an issue) considering the fact that roblox has not been very generous when it comes to any previous memory budgeting systems which often butcher game quality (and even functionality) while also restricting any broader use cases for higher end systems (CanvasGroups are a primary example of this).

  2. I understand the need for moderation systems when it comes to these sorts of features however i cannot help but fear that these features may impose a level of “danger” on developers for attempting to use them in real game scenarios. It is no secret that most (if not all) developers on this platform often resort to multiple alternative accounts just to avoid unjust and even random moderation actions on their primary accounts just because they had dared to upload an asset that was considered inappropriate by magical AI moderation systems. I am in no way expecting the same kind of moderation used for standard asset uploads to be used on moderating such high speed and dynamic systems like the Editable instances however i also cannot believe that things will simply just be magically “different” and work flawlessly. To add more to this point, if it’s not the developers who will suffer the consequences of using the Editable instances (since us devs would actually have no way of even attempting to moderate these things) then it’ll likely be the players. Is it unreasonable to expect players sometimes getting potentially moderated for simply using the Editable systems of X Y and Z games without any intent of blatantly breaking ToS?

  3. How far could this brand new Content data type go? Would it be reasonable for us to expect the ability to throw an EditableImage onto something like a Decal or Texture?
    Would this new Content system be capable in handling large scale content swaps (for creating something systems like custom LoD without having to entirely replace entire instances)? Support for high speed Content swapping would help us developers a ton. Hey, what about some sort of addition to the Content data type in which an instance is capable of holding multiple Content’s (like multiple meshes) without having to swap the Contents every time but instead it would let us choose exactly which Content the instance should be using at that given time.
    Would it be possible for the Content data type to let us actually delete non-instance assets (like raw images and mesh geometry) from memory whenever we choose to no longer use said assets? This sort of functionality could let us use more and higher fidelity assets without detonating our performance budgets.

  4. This could have maybe went with question 3 but felt like this is “different enough” to be its own thing. Would it be possible for the Content data type to pull content directly from disk? This functionality could allow us to load our places worlds quicker. Maybe even let us bypass asset uploading so we can iterate on our assets a million times faster without fear of moderation (This should mostly just be a studio only thing of course).

But yeah, this is all i had in mind right now. Im really curious to see what will come next from Object.

5 Likes

I think I found out why. There will be a fix tomorrow. Sorry for the inconvenience.
Update: should be fixed now.

2 Likes

Great, now my rendering plugin is deprecated. They shined :slight_smile:

3 Likes

Is there any guide for using buffers for this sort of situation?
I’ve been wanting to use the WritePixelsBuffer for a while now but couldnt understand how to get the right offset or even turning a color into a series of numbers

I’ve looked in other forums and even the buffer post to understand, but couldnt find the right information for the job

This is the best i could get:

3 Likes

Each pixel takes up 4 bytes, where each byte represents RGBA [0-255] in sequence. They are stored in a scanline pattern, left to right, top to bottom.

You can find a pixel’s starting location in a buffer at:
((y * width * 4) + (x * 4))

Also note that buffers use 0-based indexing. So your x/y values should be clamped between [0, width - 1] and [0, height - 1]. And your initial offset into the buffer should be 0, in case you were using 1-4 instead of 0-3.

5 Likes

Is there going to be some kind of support for bulk methods so it can be used with parallel luau? With this i mean something like this:

local editableMesh = ...

local vertices = getVertices() -- Do wacky stuff in parallel luau
local indices = getIndices()

editableMesh:SetVertices(vertices)
editableMesh:SetIndices(indices)

6 Likes

We really need some bulk feature like this, been trying to implement Parallel Luau into editablemesh programs since I’ve been working on oceans with many vertices.

Currently not really possible if not, quite difficult to implement this.

1 Like

I might’ve misunderstood as im getting an error while using your offset
Screenshot 2024-10-24 at 3.05.51 pm
for the color, I was given a encoder for writing in u32 as I was unsure on what to consider in your explanation. But I dont think ive even used that correctly (rayResult is color3)


I am very unfamiliar with this subject, especially since ive only just started pushing my programming to more than just contributions.

We want to get to bulk operations for EditableMesh as well. Those did not make the cut for v1. We are strongly considering them as priorities for future releases.

@iPeeDev what kind of “wacky stuff” are you thinking of doing in there out of curiousity?

3 Likes

The write methods of buffer write to per byte
→ Comment here, the number at the end of the method name is the number of BITS that it is writing to, writeu8 = writing 1 byte or a single (0-255) value, and writeu32 = writing 4 bytes, or a (0-4,294,967,295) value →
, so one RGBA color would equal 4 bytes, or 32 bits. You would have to use buffer.writeu8 for each color (representing 0 to 255 possible values each for red, green, blue, and alpha). The offset parameters, I believe, step per byte, so to go to up one number, you only need to increment the offset by one.
After figuring out what color the pixel should output, you would call buffer.writeu8 4 times total for RGBA, and those methods do require the value to be between 0 and 255 inclusive.

For when your script errors and says “buffer access out of bounds”, it either means your offset is negative, or it reaches past (width * height * 4 - 1). However, if the encoder you are using does encode in a 1 byte pattern (as in the Color3 encodes into 4 bytes which can be read as 4 number between 0-255), it might be a different issue.
If you need extra help, look up on documentation for possible answers (or this link buffer | Documentation - Roblox Creator Hub)

2 Likes

With wacky stuff i was thinking of some heavy tasks like generate terrain with a lot of vertices, Or updating vertices to add motion to a mesh.

btw, before the update my editable meshes werent having this error, but now that i updated my code, this appears:

image

Before, I created the meshpart and sized it as (1, 1, 1), Like this:

local editableMesh = Instance.new("EditableMesh")
local meshPart = Instance.new("MeshPart")

meshPart.Size = Vector3.one

-- do stuff

editableMesh.Parent = meshPart

And updated the code to this:

local editableMesh = assetService:CreateEditableMesh()

-- do stuff

editableMesh:CreateMeshPartAsync(Vector3.one)

Is this intentional or is this an error, Because if its intentional i personally dont agree adding limits to the mesh sizes, In my case i only used them as LOD so they didnt have any kind of physics or collisions, Only for the visuals.

2 Likes

It is quite disappointing that this is a limitation. This makes using Editable* assets for visual effects extremely difficult. Just because the feature was made for publishing in-experience meshes doesn’t mean that most people aren’t using it for that and would benefit from non-publishing support for meshes they don’t own. This is more specifically an issue for UGC clothing / accessories, which I don’t own the meshes / textures for, yet are forced into my experience anyway. This makes it so I can not apply many visual effects to these items, making them stick out and break my visual style. Please, again, re-consider this restriction for non-publishing use-cases.

8 Likes

I’m having a tough time converting my old code over to this. I was creating the EditableMesh, setting up the vertices, and parenting it to a MeshPart. Can we have a basic code example to do this? Also, is it mandatory to set up collision or can I just skip that for faster mesh rendering?

1 Like

My biggest question is if this will apply to future Sound and Audio instances, as they still seem to use the string ID format. If this only applies to images and meshes; it can get confusing, so why not sound IDs as well?

I also do hope that the older method of using string IDs does not get removed in the future to keep backward compatibility with older content for the far future. Other than this; this seems exciting for the future and I am excited for the future of this!

EditableAudio when?

1 Like

Hey, about content asset moderation, and other factors concerning in-experience asset safety standards: Instead of broadly glossing over utility by threat of harming, damaging, or disrupting user experience by risk of rogue utilizers of such regarded features I think it would be best to establish deeper standards for trust, and safety regarding consistent, and proactive developers on the platform that are quite visibly developing on the platform, that is an easy trust vector to guarantee you don’t have to vehemently impose restrictions on the developer community at large.

People who proactively work on the platform are drastically different than those who create content that goes against the community standards of trust, and safety.

Honestly, there’s room for drastic improvement across the entire strata of the platform, yet again, resources don’t go there, expenditure is tight, budget fixed, and new ideas seem to arise from top->bottom.