Update 9/25/2024
TL;DR
-
EditableMeshandEditableImageare now available in Studio-beta and give you direct Luau access to Mesh and Image data at runtime. -
Important Caveats:
-
Publishing edited Meshes / Images from within your experience is not available in this beta but is coming soon.
-
Real-time replication, moderation, and collaborative editing support for meshes / images is not available in this beta but is on the roadmap.
-
During the Studio beta, you can load & edit any Mesh or Image asset using these APIs however, for the final release, there will be checks to ensure your experience has permissions to load & edit the asset.
-
Right now, these APIs are only available on your local Studio client and you cannot publish live experiences with these APIs. They will be released sometime next year once known issues are resolved.
-
We wanted to get these APIs into your hands as soon as possible so you could give us direct feedback on the API surface, the workflows and the use-cases you have in mind.
Hello Creators!
We are very excited to finally get Mesh and Image APIs in your hands as a Studio beta. This has been a highly requested feature and really opens up what you can allow your users to do in your experiences.
For example, this experience illustrates how you can allow your users to customize their car’s hubcaps by modifying the geometry and painting different colors on them.
As we shared at RDC this year (Note: Some APIs have been renamed since this RDC talk), we have a long-term vision for Mesh & Image APIs on the platform. Our end goal here is to enable you to create real-time collaborative experiences that unleash your users’ creativity by giving you direct Luau access to Meshes and Images. We hope to eventually provide APIs that enable publishing Mesh and Image assets directly from within your experience as well as support real-time replication (with moderation) so everyone in your experience can edit meshes and Images collaboratively, in real-time.
As a first step down that path, this Studio Beta aims to get an initial set of Mesh and Image API features in your hands so we can 1) learn how you use them and what use-cases resonate the most with you and 2) get feedback from you on the actual API surface and the overall workflow.
There’s a lot to cover and we can’t wait to see what you build with these new powerful capabilities. So, let’s dive right in!
Usage
As with any Studio beta, your first step will be to enable the beta by going into the File > Beta features window and enabling the EditableImage and EditableMesh checkbox:
Once you have enabled the beta, you should now have access to the new APIs below from any Luau script. Remember, Studio Beta features only work in your own studio client session and will not work in any published places.
API Details
EditableMesh
The core promise of the Mesh API is to give you direct Luau access to any Mesh on the platform that you are allowed to edit. That means direct access to read / write vertices, faces, normals, and attributes on them. This is achieved is through a new instance called EditableMesh.
Creating an EditableMesh instance
An EditableMesh can be created from scratch for procedurally created meshes, by converting an existing MeshPart or directly from a Mesh Asset ID.
To create a new EditableMesh from scratch, you simply need to create a new EditableMesh instance like this:
local newMesh = Instance.new(“EditableMesh”)
To create an EditableMesh from an existing MeshPart, you can use the new CreateEditableMeshFromPartAsync() API on AssetService like this:
local AssetService = game:GetService("AssetService")
local existingMeshPart = workspace:FindFirstChildWhichIsA("MeshPart")
local newMesh = AssetService:CreateEditableMeshFromPartAsync(existingMeshPart)
Finally, if you would like to create a new EditableMesh directly from an Asset Id, you can use the new CreateEditableMeshAsync() API on AssetService and pass it a mesh asset Id directly:
(Remember to replace ASSET_ID below with the actual asset Id that you would like to create an EditableMesh from)
local AssetService = game:GetService("AssetService")
local newMesh = AssetService:CreateEditableMeshAsync("rbxassetid://ASSET_ID")
Keep in mind that creating an
EditableMeshorEditableImagefrom an existing mesh \ image is an asynchronous call since the engine might need to download the actual Mesh \ Image asset and verify that you have permissions to edit it before it is available to your script.
Working with an EditableMesh
Once you have an EditableMesh, you can now get access to a whole slew of APIs that allow you to directly read, manipulate and write vertices, faces, and other attributes of the mesh as well as some helper APIs for common workflows
Direct access APIs:
-
GetTriangles()gets all triangles in a list of stable triangle IDs -
GetVertices()gets all vertices in a list of stable vertex IDs -
Get/SetPosition()allows you to retrieve or set the position of a specific vertex ID respectively -
Get/SetVertexColor()andGet/SetVertexColorAlpha()allows you to retrieve or set the vertex color attribute and Alpha values for individual vertices -
Get/SetUV()lets you retrieve and set the texture UV coordinates at individual vertices -
Add/RemoveVertex()andAdd/RemoveTriangle()allows you to change the topology of the underlying mesh by adding or removing vertices or triangles to it.Note:
EditableMeshmaintains stable vertex and face IDs even if you remove or add vertices / faces. This means you can be confident that IDs will not change underneath you
Helper APIs:
-
FindClosestVertex()returns the closest vertex relative to a point in 3D space and is great for scenarios where you might want to allow your users to do direct manipulation tasks. -
FindClosestPointOnSurface()returns the closest point on a mesh surface. -
FindVerticesWithinSphere()returns all vertices within a sphere and is also great for direct manipulation tasks -
RaycastLocal()returns the first intersection of a ray in mesh local space
For the full list of APIs and further explanations, make sure to explore the docs here: https://create.roblox.com/docs/reference/engine/classes/EditableMesh
Previewing Edits
EditableMesh instances are not rendered in the scene by default. In many creation scenarios, however, it is likely you will want to allow your users to see a live preview of the edits they are making to the underlying mesh.
To preview an EditableMesh, simply place it as a child of a MeshPart instance to override the mesh that is rendered. You can use the following code snippet to re-parent an EditableMesh.
newMesh.Parent = existingMeshPart
Remember, you are simply rendering a visual preview of the EditableMesh and any edits to the Mesh will not reflect in the simulation yet (collisions, mass properties etc.)
Publishing a New Mesh Asset
The publish workflow is not yet available in this Studio beta but it is likely to follow the same pattern as the recently released PromptCreateAssetAsync() API that allows you to publish models from within experiences.
Once they are available, CreateMeshPartAsync and ApplyMesh, will allow you to create or update MeshPart meshes with updated collision geometry without requiring publishing the mesh as an asset immediately, similar to runtime created PartOperations returned from CSG APIs like UnionAsync. However, if you were to shut down your experience, all of these unpublished meshes would be lost! In order to persist the changes, you would need to publish the EditableMesh or the newly created MeshPart instance and get back an asset ID.
EditableImage
Editing images through the new Image API are very similar to the mesh editing workflow described above. Instead of using an EditableMesh, you would simply use an EditableImage instance instead.
Creating an EditableImage instance
Similar to creating an EditableMesh, an EditableImage can be created from scratch for procedurally created images or directly from an image Asset ID.
To create a new EditableImage from scratch, you simply need to create a new EditableImage instance:
local newImage = Instance.new(“EditableImage”)
To create a new EditableImage directly from an asset Id, you can use the new CreateEditableImageAsync() API on AssetService and pass it an image asset Id directly:
(Remember to replace ASSET_ID below with the actual asset Id that you would like to create an EditableImage from)
local AssetService = game:GetService("AssetService")
local newMesh = AssetService:CreateEditableImageAsync("rbxassetid://ASSET_ID")
Working with an EditableImage
Once you have an EditableImage, you can now get access to a whole slew of APIs that allow you to directly read, manipulate and write pixels and other attributes from/to the image as well as a set of higher-level APIs that make it much easier to draw common shapes and accomplish common workflows.
Direct access APIs:
-
EditableImage.Sizeretrieves the width and height of the image in pixels -
Read/WritePixel()allows you to directly read and write pixel values to specific texture coordinates of the image relative to the top-left corner of the image.
Higher-level APIs:
-
Resize(),Rotate()andCrop()allows you to resize, rotate or crop the image using bilinear interpolation where necessary. -
DrawRectangle(),DrawCircle()andDrawLine()can be used to draw a rectangle, circle or line directly onto the image instead of dealing with individual pixels -
DrawImage()allows you to draw anotherEditableImageinto this one at a given position
For the full list of APIs and further explanations, make sure to explore the docs here: https://create.roblox.com/docs/reference/engine/classes/EditableImage
Previewing and Publishing edits
Similar to EditableMesh, EditableImage allows you to preview changes to image if it is a child of the following instances:
-
As a child of a
MeshPart, it will override theTextureIdproperty -
As a child of an
ImageLabelorImageButton, it will override theImageproperty -
As a child of a
Decal, it will override theTextureproperty
Just like EditableMesh we hope to introduce publishing new image assets from EditableImage instances as well using something similar to the PromptCreateAssetAsync API
Combining EditableMesh and EditableImage together
While the new EditableMesh/Image instances are powerful in their own individual rights, combining them can unlock even more powerful scenarios. A common creation scenario might be allowing your users to directly paint onto an object in your experience to customize it. This can now be achieved by combining these APIs together!
Using the EditableMesh:RaycastLocal() API, you can determine the exact point on the EditableMesh that a user is touching, get the UV texture coordinates for that location and then use an EditableImage instance to change the color / draw something on the underlying Image at those coordinates.
For an example, search for the MeshPainting module script in the example place below!
Example place file
Since there are a lot of new APIs we are introducing here, we thought it would be helpful to create a sample place file to show how everything could work together.
The following place file shows a simple experience where you can customize the shape of a wheel hub cap using a set of sliders and the EditableMesh API and then directly paint on it using the EditableImage API
Hubcap_Image_v12.rbxl (426.3 KB)
Note: Please use the latest version of this RBXL file available here: [Client Beta] In-experience Mesh & Image APIs now available in published experiences
Instructions:
- Make sure you have the Studio beta enabled
- Open up the RBXL file attached above in Studio and hit Play
- Walk up to the car and hit
Fto switch to “Hubcap editing mode” - Use the sliders on the left to deform the car’s hub cap, add/remove spokes to it or change the taper
- Use the slider on the right to change the brush size and after picking a color, you should be able to directly paint on the hub cap with your mouse.
- Hit
Fagain to exit “Hubcap editing mode” and you can now enter the car and drive it around with the new hubcaps
Notes:
-
In the RBXL file, all the scripts that make use of
EditableMesh/EditableImagecan be found under:StarterPlayer/StarterPlayerScripts/HubcapScript -
In
HubcapScript, take a look at theInit()function all the way at the bottom of the script for the main entry point. Here you will see howCreateEditableMeshAsyncis used to create anEditableMeshinstance from an existing mesh asset.-
The single hubcap spoke is then duplicated around the hubcap and the sliders are used to perform some straightforward manipulation of the mesh vertices
-
Also In
HubcapScript, take a look at theupdateCurrentDeform(meshInfos, emesh, params)function to see how theEditableMeshinstance is used to get the vertex positions and modify their positions based on the values from the UI sliders.
-
-
Take a look at the
HubcapScript/MeshPaintingModuleScript to see an example of howEditableImagecan be used. The functionMeshPainting.Init()sets up a newEditableImageand sets it as a child of aSurfaceGuiImageLabelso you can preview edits.-
Take a look at the
castRayFromCamera(position)function in theHubcapScript/MeshPaintingModuleScript to see how theEditableMesh:RaycastLocalAPI is used along with theEditableImageto allow the user to paint directly onto the hubcap -
The
doDrawAtPosition()function uses theDrawCircleAPI to mimic a circular “paint brush”
-
-
This place file also contains a number of additional helper ModuleScripts for a variety of common tasks like
- setting up UI sliders (
HubcapScript/SetupSliders), - calculating the mesh deformations necessary (
HubcapScript/MeshMath) - showing a color selection palette (
HubcapScript/SetupPalette)
- setting up UI sliders (
Important caveats
These new APIs come with incredible utility, but also some unusual limitations that are worth highlighting.
Never Saved (By Design)
EditableMesh and EditableImage instances are never saved in places, packages, or model files, even if Archivable is true. This is by design.
These instances represent an editing session and are intended to be eventually moderated and published as assets, or discarded. They’re meant to facilitate asset creation in-engine, not act as an alternative to the asset system.
Never Replicated (Temporary Safety Limitation)
EditableMesh and EditableImages instances currently do not replicate from servers to clients, unlike most instances. This is not by design. The technology to support safe replication isn’t finished yet. Accepting this temporary limitation allows us to get these APIs into your hands and get your feedback sooner! We felt these APIs were already incredibly useful, even without replication, that it justified sharing it, even in this “incomplete” form.
Our plan is for these to eventually replicate from servers to clients like a standard instance by default, with some additional safety filtering. Our goal is to support developers like yourselves building collaboration tools on top of these APIs, and standard replication is important for making that accessible. We’ll share more on the timeline and rollout plans later.
We don’t expect this safety work to be completed before we will first enable use of these editing APIs in experiences next year. This limitation on replication will exist for a while. It might take a year, or more.
EditableMesh:CreateMeshPartAsync is sadly also unavailable at this time. This API was designed to allow you to convert an EditableMesh back into a fully-simulated MeshPart. Without safe mesh replication, any MeshParts created by this method on the server would be invisible to all clients. Also, without special publishing support in Studio, these parts would be saved without meshes and remain invisible forever. We will be working to make this available to Studio plugins soon, but supporting safe runtime use of this method will take longer.
For now, avoid placing EditableMesh or EditableImage instances under Workspace or other replicated services on the server to avoid future compatibility headaches (for both of us!).
Asset Edit Permissions (Incoming Restrictions on Loading)
In this Studio Beta, you will be able to create an EditableMesh\Image from any existing mesh\image asset on Roblox.
However, we will start enforcing asset permissions sometime before these APIs are finally released and publishing support is enabled. You will only be able to load an EditableMesh\Image from an asset that your experience has permission to edit. (e.g. meshes and textures used by items in the Avatar Marketplace will not be able to be loaded into an Editable* unless the experience owner is the original creator of that item.) Stay tuned for more details in the coming months on how asset permissions will work for EditableMesh\Image instances.
For now, we recommend that you only use Mesh / Image assets you currently own even during this Studio beta to make your life easier when permissions are enforced by the API.
Known Issues
Issues that will be addressed before these APIs are released for use in experiences sometime next year:
- [DONE see update] The APIs for Vertex Normals and other attributes will be improved significantly.
- Normals, Colors, UVs, and other vertex attributes will get their own per-face attribute IDs to allow independent normal/color/uv seams.
- Adding or removing vertices and faces will not be supported for skinned meshes until APIs for rigging and skinning are available.
- [DONE see update]The
EditableImageAPI will be updated to have a more unified approach to blend types rather than the currentImageCombineTypeenum that is used in theDrawImageAPI.- This will allow using alternative blend modes in other APIs like
DrawCirclewhich will make creating things like erasers much easier.
- This will allow using alternative blend modes in other APIs like
- [FIXED] EditableMesh preview does not work under Humanoid models. “FastCluster” rendering support is incomplete.
- [FIXED] EditableMesh is currently disabled on devices running Mac lower than MacOs 11.0, subject to change.
- [FIXED] Cloning an
EditableMeshwith 0 triangles can sometimes cause a crash. The fix for this issue will be released in a few weeks. - Meshes with skinning data cannot be loaded (i.e. most avatar meshes). This limitation will be removed once
EditableMeshcan store skinning bind data internally. - EditableMesh doesn’t have batch add/remove/set APIs yet. We’re getting to it!
- In Studio copy & paste does not work because it currently relies on the class being marked as savable. Use
Duplicateinstead. - Live previewing of
EditableMesh/EditableImagewill likely change to use a new multi-owner reference mechanism, instead of supporting EditableMesh/EditableImage as a child.
Feedback
Again, we wanted to get these APIs in your hands as quick as possible to get your feedback. Let us know if you have any thoughts or suggestions regarding the API surface, the workflows, and the use-cases you have in mind.
Made with 
by, @L3Norm, @TheGamer101, @monsterjunjun, @igHunterKiller, @FarazTheGreat, @syntezoid, @theRealLogicalerror, @FGmm_r2, @TigerRabbit2, @NeoBuilder101 and @c0de517e
