[Client Beta] In-experience Mesh & Image APIs now available in published experiences

Updates Announcements
,
#233

I’ve been messing with the EditableMesh API for a few weeks now and have already done some exciting things!

Decal Projection

I can create square planar projections on the surface of any MeshPart!

Shape Key Morphing

Directly morphing the vertices of one mesh into another by recording their difference as vectors and interpolating the vertices that changed. Vertices are linked together by their shared UV mapping.

I was even able to hack it into the FaceControls system :sunglasses:

API Feedback

I have a few comments that I can make about the API so far.

1. ID collisions exist across multiple EditableMeshes, potentially causing confusion.

When I was working on the Decal Projector there was one bug in my code that had me stumped for a while. I was getting valid ID reads, but sometimes was getting nil for seemingly no reason.

It turned out I was reading data from the wrong EditableMesh and it was giving me data that was sometimes valid because it would collide with an ID in the mesh I was actually trying to use.

If it’s possible, I think the ID ranges should be further scoped to avoid this confusion. I’m guessing there’s a chance it’s impossible because the IDs are actually some integer representation of a byte sequence. I haven’t had a chance to look at how that actually works yet.

2. Converting EditableMeshes into immutable FixedSize Content.

The limit to how many EditableMesh objects can exist at one time is very harsh right now.

I reckon it’s because there’s a lot of memory overhead from all the tracking and I/O bridging needed to make realtime mesh editing possible in a live game, which is fair.

However, we definitely should have some way to convert the EditableMesh into an immutable Content object once we don’t intend to edit the mesh any further (i.e. for decals and other entities that may rapidly accumulate into the dozens)

3. Permissions need to be relaxed.

I’m not the only one who has brought this up so I’d rather not repeat what everyone else has said, but I will say that ultimately the restrictions are moot:

  • There is nothing stopping a user from reuploading someone else’s mesh by simply importing it into Roblox Studio, exporting it as a *.obj file, and then reimporting it.
  • Roblox’s in-house filemesh format has been publicly reverse engineered up to version 5.00. You can just run a proxy server to download the mesh from assetdelivery, pass that binary to the client, decode it as an EditableMesh on the fly, and completely bypass the permission check.
  • WrapDeformer ignores the permission check entirely :kekw:
21 Likes
#234

Hey we’ve just deployed a fix hoping to resolve this error. Please let me know if you still see this or similar error. Thanks a lot!

1 Like
#235

For the ids, we pack the type, an internal index, and an index version into a 53-bit int. We could probably use a few bits for which EditableMesh the id came from, but we’d have to reuse those, so it would only catch an error some of the time. Thanks for the suggestion!

1 Like
#236

Sorry to harp on this, I know what I’m doing may not be an intended use-case, but is the culling behavior in my previous post known? And if so is it considered a bug or is it out of scope?

1 Like
#237

Thanks for your patience and for bringing this to our attention! We understand this culling behavior can be frustrating, even if it’s not a typical use case.

The good news is that fixes for frustum and occlusion culling are in the works. These should address the problems you and other developers are experiencing. We’ll post an update here as soon as we roll out the fixes.

1 Like
#238

Is there any realistic way to get a different ID type (UV, Normal, etc.) of a vertex, given the VertexID? I remember something like this used to be possible, but for the future is there anything that can be done or is it out of scope?

I was looking into the wrap-deformer post and tried doing some shape-key stuff, and noticed that the normals still retained the original mesh.

But I saw in the wrap-deformer post that there was an entire function for getting the position of a vert, given any type of ID

My idea to get those normals adjusted was to save the vertex IDs of the original, compare it to the IDs of the new shape, and then get the normal’s that way but it wasn’t very fruitful for performance.

image

2 Likes
#239

guys i dont think decals and textures will be supported :cry:

#240

We’re working on an API that will make it simpler to do that.

It’s a bit complicated by the fact that a vertex might have multiple normals or other attributes. For now, there are a couple of things you could do.

  1. Find a face, get the matching attributes on the face

For this, you can get the faces that contain a vertex by using :GetFacesWithAttribute(vertex_id). Then on one of those faces, call :GetFaceVertices and :GetFaceNormals, and use the normal at the matching index.

In other words, if faceVertices[i] == vertex_id, then faceNormals[i] is the normal id that matches.

This is the most direct answer to what you’re asking, but to answer your question, we don’t see this is a good solution, and plan to add a new method to make this easier, since it is a common case.

  1. Go from normal id to vertex id

If you iterate through the normal ids, it’s easy to look up the corresponding vertex id, using :GetVerticesWithAttribute(normal_id)

  1. Reset all your normals

If you want the normals to be recomputed, use :ResetNormal(normal_id) on all of the normals. The normal vectors will be automatically calculated each frame.

2 Likes
#241

Oh I didn’t know ResetNormal automatically calculates, I was presuming it would just go back to how the base-mesh had it if you altered it, that’s wonderful news!!

And it’s great to hear there’s gonna be an API to get those different ID types, really awesome to hear!

#242

image
I’ve been experimenting with textured brushes recently and have noticed some very niche needs that would make it far easier for me:

  1. A Tint option, perhaps in :DrawImageTransformed() and :DrawImage(). This would allow for some more advanced features, such as a random color variation.
  2. A better way to get Part and MeshPart UVs. More specifically, a UV return value in RaycastResult. Doing so does undermine some functionality of the EditableMesh, but would also massively save on memory, something notably scarce.
  3. Loading of images that aren’t owned by you (for plugins). This would be great to enable saving of a brush to a single texture, and likewise loading from that texture.
  4. SurfaceAppearance, Decal, and Texture support. Though, this is planned and in-the-works.

That’s it, I hope at least my Tint suggestion gets added, but that’s all the problems I’ve encountered with the API so far, the rest is up to my programming prowess. Great to see something like this finally make it into the engine!

2 Likes
#243

Would editable mesh perform better than bones? Currently, I have a fluid simulation that moves a lot of bones. The simulation runs fine, but the bones tank performance a lot. Now that editablemesh needs ID verification, It is hard as a creator to do cool things.

Largest simulation I can get with 60 fps:

6 Likes
#244

Please give an option to allow EditableMeshes to be generated even if it passes the memory limit. My game uses editable meshes for a pretty crucial feature and just not generating is not a very good option unless I have to.

1 Like
#245

Please tell me I’m doing something wrong

the memory budget is reached after generating 8, simple cubes…

image
image1055×145 31 KB 

local AssetService = game:GetService("AssetService")

-- Given 4 vertex IDs, adds a new normal and 2 triangles, making a sharp quad
local function addSharpQuad(eMesh, vid0, vid1, vid2, vid3)
	local nid = eMesh:AddNormal()  -- This creates a normal ID which is automatically computed
	local fid1 = eMesh:AddTriangle(vid0, vid1, vid2)
	eMesh:SetFaceNormals(fid1, {nid, nid, nid})
	local fid2 = eMesh:AddTriangle(vid0, vid2, vid3)
	eMesh:SetFaceNormals(fid2, {nid, nid, nid})
end


-- Makes a cube with creased edges between the 6 sides
local function makeSharpCube()
	local eMesh = AssetService:CreateEditableMesh()

	local v1 = eMesh:AddVertex(Vector3.new(-1, -1, -1))
	local v2 = eMesh:AddVertex(Vector3.new( 1, -1, -1))
	local v3 = eMesh:AddVertex(Vector3.new(-1,  1, -1))
	local v4 = eMesh:AddVertex(Vector3.new( 1,  1, -1))
	local v5 = eMesh:AddVertex(Vector3.new(-1, -1,  1))
	local v6 = eMesh:AddVertex(Vector3.new( 1, -1,  1))
	local v7 = eMesh:AddVertex(Vector3.new(-1,  1,  1))
	local v8 = eMesh:AddVertex(Vector3.new( 1,  1,  1))
	addSharpQuad(eMesh, v5, v6, v8, v7)  -- Front
	addSharpQuad(eMesh, v1, v3, v4, v2)  -- Back
	addSharpQuad(eMesh, v1, v5, v7, v3)  -- Left
	addSharpQuad(eMesh, v2, v4, v8, v6)  -- Right
	addSharpQuad(eMesh, v1, v2, v6, v5)  -- Bottom
	addSharpQuad(eMesh, v3, v7, v8, v4)  -- Top
	eMesh:RemoveUnused()
	return eMesh
end

local emeshes = {}

for i = 1, 1000 do
	local t = makeSharpCube()
	emeshes[i] = t
	task.wait(0.01)
	print(i, t)
end

for i, v in emeshes do
	v:Destroy()
end
2 Likes
#246

I really like the progress so far, but I cannot wait for an eventual implementation of uploading assets and checking their verification process so you could build things in-game and use them either in-game or outside of the game.

I also would like to see ways to bulk-edit information within the meshes, as it can be really slow to remove, manipulate, or change several parts of a mesh in lua, and I would rather be able to perform operations such as bulk-deleting faces, moving vertices, deleting ‘connected’ faces (or at least getting ‘connected’ faces (share at least one edge)), removing vertices (deleting any attached faces automatically), and so on, to facilitate and simplify the workflows for manipulating meshes.

Hopefully we also get some way to store skinning/morphTarget information for optimized body and facial animations without manual computations in luau

2 Likes
#247

Are there plans to add shadows to editable meshes? I’ve tried everything but I can not get them to render.

#248

Removing id verification by keeping the ability to create editable images but not use them for Content properties would be nice, the apis themselves have a lot of use even if u cant display the end result.

1 Like
#249

8 editable meshes seems to be the limit for most people. In the future they plan on allowing to make more editable meshes by allocating less max triangles per editable mesh.

right now the space for 20k triangles is allocated in memory for each editableimage used, in the future you’ll be able to change that

1 Like
#250

I can’t even test these in studio as my account isn’t id verified :rage: Why does their need to be such a large restriction, apps like photoshop or even paint and blender don’t require you to be above 13

#251

Hello! we are planning to add options in the future that will allow you to specify a maximum number of vertices or triangles for an editable mesh. In the meantime, if you’re concerned about memory usage with unbounded editable meshes, one workaround is to merge multiple editable meshes into a single, larger one.

2 Likes
#252

When are we expected to see the same level of freedom as prior to this change? Previously, for example, we were able to achieve massive expansive terrain generation with custom LoD using Editable Meshes without much issue. With this new “dynamic” memory budgeting system such feats are impossible.

“Dynamic” Memory Budgeting System?
My system has 32GB of memory, is this memory budgeting system truly “dynamic” or is it just a flat limit of 8? I have 16GB+ just sitting there twiddling its thumbs and yet studio is telling me I have reached the limit. When I hear dynamic I expect it to budget from the total available memory and eat that up until it begins to run out. It should behave the same way it would if few thousand parts were created. I want to be able to create as many Editable Meshes as my system allows me to.

FixedSize Content?
When are we expected to get FixedSize Content, and will it be performant enough for the terrain example previously mentioned (E.g. switching between Editable > Fixed > Editable)? It’s very upsetting that such a big limit was imposed without a supported non-hacky method to work around it. I don’t understand why the “dynamic” memory budgeting system pushed out if we don’t have FixedSize Content available to us.

I’m really looking forward to this feature and I appreciate all the work that has gone into it. I really want to push Editable Meshes to its limit. But the “dynamic” memory budgeting system is an artificial roadblock that makes it frustrating to work with Editable Meshes. I really hope this if fixed soon. Thanks!

5 Likes