[Studio Beta] Major updates to in-experience Mesh & Image APIs

Try it yourself! Guess what the answer is!

print(Instance.new("MeshPart"):IsA("Object"))
5 Likes

I was confused on this too! At the moment we need to put in the object’s size (or any altered Vector3 value for the new size), It should work after that’s done.

If your editable mesh already has a texture you need to re-apply the texture for it, which I personally believe should be automatically set by default instead of needing to re-apply the texture.

3 Likes

It does, I was asking because it looked like the docs were missing information saying what props/methods were inherited from object but I just realized I was on the datatypes page and not the classes page lol

3 Likes

Still waiting for docs updates to publish. Sorry about that.

4 Likes

Doesn’t seem to have gone through, even after restarting my computer and Roblox studio itself, Maybe it just needs more time?

2 Likes

Should be

local editableMesh = AssetService:CreateEditableMeshAsync(Content.fromUri("YOUR_URI_HERE"))

And yes unfortunately you have to populate the mesh with some data before you can create a MeshPart from it. Can’t generate physics data from nothing. At least needs to have non-zero extents…

4 Likes

Got it! A meshSize is required for EditableMesh::CreateMeshPartAsync. For example, you can use local newMeshPart = myEditableMesh:CreateMeshPartAsync(Vector3.one). To use RemoveFace on an EditableMesh loaded from an asset ID, which defaults to a fixed size, you can add option {FixedSize = false}. For instance: AssetService:CreateEditableMeshAsync(Content.fromUri("asset id"), {FixedSize = false});. Stay tuned for more updates on available options!

3 Likes

Keep in mind that the initialSize argument gets assigned to MeshPart.MeshSize, skipping any mesh extents calculation, and the render scale for the mesh will be part.Size / part.MeshSize.

Should have been optional, sorry about that. One of the reasons we’re removing that one.

If you pass Vector3.one but part.Size is larger than that, and your mesh is huge… that’s why.

Here’s a code snippet to calculate mesh size if you need it:

-- We'll add a getter for this in a subsequent update...
local function computeExtents(em: EditableMesh)
    local verts = em:GetVertices()
    if #verts == 0 then
        return Vector3.zero
    end
    local inf = math.huge
    local min = Vector3.new(inf, inf, inf)
    local max = Vector3.new(-inf, -inf, -inf)
    for _, id in verts do
        local v = em:GetPosition(id)
        min = min:Min(v)
        max = max:Max(v)
    end
    return max - min
end

-- ...
local meshPart = em:CreateMeshPartAsync(computeExtents(em))
3 Likes

So, how will the Content Class work for Player Mouse Cursors via the PlayerMouse class and UserInputService? Will these migrate over to the new API automatically too, or do we need to do an alternative approach for mouse cursors?

2 Likes

Creation APIs for new Editable* objects can return nil if the device does not have enough memory budget. You will need to account for this possibility in your scripts.

Not a fan of this. Vague nil returns and errors just encourage developers to write code to brute force the operation instead of intelligently handling the problem and informing the user. I have a sneaking suspicion that memory limitations aren’t the only thing that could potentially trigger a nil return from these APIs.

Other than that though, these changes seem pretty dope.

3 Likes

The general pattern is that we throw for usage errors the dev could have avoided, like invalid arguments etc. Like an invalid option in the creation options table. You can consistently prevent that error. E.g. you should never need to pcall the empty creation AssetService:CreateEditableMesh() because it won’t throw for reasons outside your control.

Async APIs can often throw for reasons outside your control like intermittent web service failures or permissions issues. LIke CreateEditable*Async will throw if you don’t have permission to load the requested asset for editing.

In this case it’s not a preventable error on the developer’s part. It’s expected to happen sometimes and you’re expected to handle that case.

So we return nil if the allocation fails and make that easy to handle. Should be the only reason we’d return nil. We won’t do that for random unspecified reasons otherwise.

Open to feedback.

This is one of the most potentially memory hungry feature we’ve ever shipped, so we’ve spent a lot of time debating the best way to handle that safely.

4 Likes

Are we ever gonna get a DrawPolygon method for editable image? Because editable image is kind of pointless to me if we are unable to draw polygons with it.

3 Likes

Thanks for pointing this out! It’s fixed in the snippet now

1 Like

Wait, are you telling me we cannot create an editableImage from free toolbox images now? Why is this? I understand misuse with images, but images in the toolbox are moderated anyways. Why change this?

5 Likes

Just to be clear, PromptCreateAssetAsync() would be an in-experience API that we are hoping to ship in the future. There are a number of issues we need to work through before we can get that out the door.

Publishing within Studio is hopefully something that will be out much sooner than that since the issues are a lot fewer within Studio.

  • Nothing planned for right now, but that is an interesting idea
  • Unfortunately, not in the near future. 1024x1024 already uses a ton of memory on clients since we need to make all those pixels available in memory for the non-async APIs to function like you would expect.
  • The group permission check is to ensure you can only load assets that you (or your group) owns to prevent asset misuse. In Studio, the logged in user will be able to access any assets they own or groups they are part of own. Are there valid use cases your plugin would like to achieve that would not work with these limitations?
3 Likes

The EditableImage APIs are not designed nor optimized for this use case (real-time rendering).
Like all APIs in the engine, you can hack your way around them, but YMMV.

That being said, you might want to look at the new WritePixelsBuffer() method since that will let you “batch” write a number of pixels with a buffer.

3 Likes

You’d be surprised by how many people use EditableImage for real time rendering. But as you have said using WritePixelsBuffer is really efficient for this use case

16 Likes

Wow. It’s not very often we get pretty large updates like this. I do have a couple of questions for all of this though.

  1. I question the overall functionality of the memory management behind these systems. Would it be possible for the automatic memory management to not allow any Editable instances from being created for extended periods of it? I believe that devices which suffer from limited memory won’t exactly treat these new Editable systems well at all and potentially cause games who use these features heavily to either break or simply force the user on a very long “retry loop” due to the lack of memory. I believe this issue may be even worse (if it actually IS an issue) considering the fact that roblox has not been very generous when it comes to any previous memory budgeting systems which often butcher game quality (and even functionality) while also restricting any broader use cases for higher end systems (CanvasGroups are a primary example of this).

  2. I understand the need for moderation systems when it comes to these sorts of features however i cannot help but fear that these features may impose a level of “danger” on developers for attempting to use them in real game scenarios. It is no secret that most (if not all) developers on this platform often resort to multiple alternative accounts just to avoid unjust and even random moderation actions on their primary accounts just because they had dared to upload an asset that was considered inappropriate by magical AI moderation systems. I am in no way expecting the same kind of moderation used for standard asset uploads to be used on moderating such high speed and dynamic systems like the Editable instances however i also cannot believe that things will simply just be magically “different” and work flawlessly. To add more to this point, if it’s not the developers who will suffer the consequences of using the Editable instances (since us devs would actually have no way of even attempting to moderate these things) then it’ll likely be the players. Is it unreasonable to expect players sometimes getting potentially moderated for simply using the Editable systems of X Y and Z games without any intent of blatantly breaking ToS?

  3. How far could this brand new Content data type go? Would it be reasonable for us to expect the ability to throw an EditableImage onto something like a Decal or Texture?
    Would this new Content system be capable in handling large scale content swaps (for creating something systems like custom LoD without having to entirely replace entire instances)? Support for high speed Content swapping would help us developers a ton. Hey, what about some sort of addition to the Content data type in which an instance is capable of holding multiple Content’s (like multiple meshes) without having to swap the Contents every time but instead it would let us choose exactly which Content the instance should be using at that given time.
    Would it be possible for the Content data type to let us actually delete non-instance assets (like raw images and mesh geometry) from memory whenever we choose to no longer use said assets? This sort of functionality could let us use more and higher fidelity assets without detonating our performance budgets.

  4. This could have maybe went with question 3 but felt like this is “different enough” to be its own thing. Would it be possible for the Content data type to pull content directly from disk? This functionality could allow us to load our places worlds quicker. Maybe even let us bypass asset uploading so we can iterate on our assets a million times faster without fear of moderation (This should mostly just be a studio only thing of course).

But yeah, this is all i had in mind right now. Im really curious to see what will come next from Object.

5 Likes

I think I found out why. There will be a fix tomorrow. Sorry for the inconvenience.
Update: should be fixed now.

2 Likes

Great, now my rendering plugin is deprecated. They shined :slight_smile:

3 Likes