Hello Creators,
We are excited to announce that you can now publish experiences with the EditableMesh
and EditableImage
APIs. With this update, these APIs are now available in Client Beta. This means we do not foresee any more breaking API changes and you can confidently publish experiences with these APIs. We are still considering this a beta release as we are going to be tweaking the memory budget and re-evaluating the permissions restrictions over the next few months, and this might impact what you can do with these APIs. Please see the “Best Practices” section for more information. This update also addresses a number of the known issues called out in our previous Developer Forum post.
With that, let’s dive into the details!
Enabling the APIs
Since the EditableImage
and EditableMesh
APIs are powerful new APIs that require extra care to ensure compliance with our Terms of Use, you must be 13+, ID-verified, and explicitly opted-in to using the APIs in published experiences. ID verification allows us to increase creator accountability for APIs that could result in policy violating content on the platform.
- You can visit Roblox.com > Settings to check if you are already ID-verified. If you’re not, complete the ID verification workflow.
- Once you are ID-verified, go to Studio > Game Settings > Security and enable the “Allow Mesh / Image APIs” toggle (see screenshot below). Remember to review the Terms of Use before enabling the toggle.
- If the experience is owned by a group, the group owner will need to be ID-verified and perform the above steps for the experience.
- Please review the Best Practices section below for more details and tips to keep your experience safe
Note: You can use these APIs within Studio plugins (edit mode) without enabling this toggle.
Summary of APIs now available for published experiences
EditableMesh
APIs
EditableMesh
gives you direct access to the vertices, faces and attributes of a given Mesh. They can be procedurally created from scratch or created from an existing asset or MeshPart
instance.
Lifecycle APIs
Click to read more:
-
AssetService:CreateEditableMesh()
creates a new emptyEditableMesh
object for procedural creation. -
AssetService:CreateEditableMeshAsync()
creates anEditableMesh
object from an existing mesh asset or an existingMeshPart
instance’s mesh content -
[New]
AssetService:CreateMeshPartAsync()
converts anEditableMesh
into a newMeshPart
instance -
EditableMesh:Destroy()
marks the object for garbage collection to free up resources -
EditableMesh
objects can only be created from mesh assets that are owned by you (or your group if the experience is owned by a group). Attempting to load an asset into anEditableMesh
that you do not have permissions to will result in a permissions error. -
The above creation APIs are subject to memory usage limits and might return
nil
if the device (or server) is out of memory reserved forEditable*
objects. Always remember to check if yourEditableMesh
object was successfully created before using them in your scripts.
Direct access APIs
Click to read more:
Vertices and Faces:
-
AddVertex
adds a new vertex andAddTriangle
adds a new triangle face to theEditableMesh
object.RemoveFace
removes a specific face, andRemoveUnused
will remove unused vertices and attributes. -
GetVertices
,GetFaces
andGetFaceVertices
retrieves all the vertices or faces of the mesh or the vertices of a specific face that you can then iterate through. Note: Vertex and Face IDs are stable even as you add or remove new vertices or faces. -
Get/SetPosition
allows you to retrieve or update the position of a specific vertex ID.
Attributes:
-
AddColor
,AddNormal
andAddUV
add a new attribute and assign it an id which can then be used to assign the attribute onto a face / vertices. -
Get/SetColor
,Get/SetColorAlpha
,Get/SetNormal
andGet/SetUV
allow you to get or update the respective attribute for a specific Color, Alpha, Normal or UV ID. -
ResetNormal
will reset the normal to be automatically calculated based on the shape of the mesh. -
GetColors
,GetNormals
andGetUVs
retrieves all the instances of the respective attribute so you can iterate through them. Note: Color, Normal and UV attribute IDs are stable even as you add new ones or remove existing ones. -
Get/SetFaceColors
/Get/SetFaceNormals
/Get/SetFaceUVs
are used to get or assign Color, Normals or UV attribute IDs to the vertices of a face. -
GetVerticesWithAttribute
andGetFacesWithAttribute
retrieves a list of all the vertices or faces that use a specific attribute ID.
Helper APIs:
-
FindClosestVertex
returns the closest vertex relative to a point in 3D space and is great for scenarios where you might want to allow your users to do direct manipulation tasks. -
FindClosestPointOnSurface
returns the closest point on a mesh surface. -
FindVerticesWithinSphere
returns all vertices within a sphere and is also great for direct manipulation tasks. -
GetSize
andGetCenter
return the size and extents of the mesh which can be useful as inputs toCreateMeshPartAsync
. -
GetAdjacentFaces
andGetAdjacentVertices
retrieves a list of faces or vertices that are adjacent to a specific face or vertex. -
IdDebugString
retrieves a string describing a stable ID which can be very useful for debugging purposes. -
MergeVertices
will merge all vertices in the mesh that are within a certain tolerance of each other. -
RemoveUnused
will remove all unused vertices, normals, colors and UVs. -
RaycastLocal
returns the first intersection of a ray in mesh local space. -
Triangulate
will split all faces on the mesh to be triangles.
EditableImage
APIs
EditableImage
objects allow for the runtime creation and manipulation of images by giving you direct access to the underlying pixels of the image.
Lifecycle APIs
Click to read more:
-
AssetService:CreateEditableImage()
creates a new emptyEditableImage
object for procedural creation. -
AssetService:CreateEditableImageAsync()
creates anEditableImage
object from an existing image asset or an existingMeshPart
instance’s texture content and now supports an optional options table to customize the createdEditableImage
object. - [New]
EditableImage:Destroy()
marks the object for garbage collection to free up resources. -
EditableMesh
objects can only be created from mesh assets that are owned by you (or your group if the experience is owned by a group). Attempting to load an asset into anEditableMesh
that you do not have permissions to will result in a permissions error. -
The above creation APIs are subject to memory usage limits and might return
nil
if the device (or server) is out of memory reserved forEditable*
objects. Always remember to check if yourEditableMesh
object was successfully created before using them in your scripts.
Direct access APIs
Click to read more:
Pixel APIs:
-
ReadPixelsBuffer
andWritePixelsBuffer
allow you to read and write pixel buffers directly from/to an image.
Drawing APIs:
-
DrawCircle
,DrawRectangle
andDrawLine
allow you to easily draw primitive shapes / lines on an image.DrawRectangle
is especially useful when you are trying to set or clear the entire image at once. -
DrawImage
allows you to draw anotherEditableImage
into anEditableImage
at a given position with a variety of blending options. - New:
DrawImageTransformed
also lets you draw oneEditableImage
onto another but also allows you to specify the position, rotation and scale. In addition to spatial transformations, you can also specify additional options likeCombineType
,SamplingMode
andPivotPoint
to further customize how the image is drawn.
Object and Content
As described in detail in the previous dev forum post, EditableMesh
and EditableImage
inherit from the new Object
class. Along with the new Content
data type, this enables multi-reference workflows where you can drive multiple instances like MeshPart
from the same EditableMesh
or EditableImage
object.
You can read through the following explanations, code snippets and documentation to understand how common workflows like the following can be accomplished:
- Creating a new empty Editable* Object
- Creating An EditableMesh From An Existing MeshPart
- Creating An Editable* From An Asset ID
- Live rendering an EditableMesh
- Multiple MeshParts Referencing The Same EditableImage
Best Practices
With these APIs, our goal is to set the foundation for our vision of collaborative in-experience co-creation, where one or more users come together in an experience and create a piece of artwork or sculpt together.
Thus, these APIs have been designed and optimized specifically for the following:
- A limited number of
Editable*
objects created and being edited at the same time. - Async creation and conversion (e.g.,
CreateEditableMeshAsync
,CreateMeshPartAsync
). - Immediate mode edits (e.g.,
SetPosition
,WritePixelsBuffer
) once you have access to theEditable*
object. - No guarantees that edits will be applied to rendering in the same frame.
The EditableImage
and EditableMesh
are incredibly powerful APIs due to how much flexibility they give you as creators. We recognize that a number of creators will choose to use these APIs for a variety of other use cases like procedural terrain generation, destructible environments, or even special effects. Since the APIs were designed for the above constraints, we recommend following these best practices to get the best results.
Memory Management
As introduced in the previous Studio Beta update post, Editable*
objects provide random read and write access to vertices and pixels, and keeping that data available can be extremely memory-intensive. Roblox supports many platforms, including devices with limited memory. To allow your experiences to scale from low-end devices up to devices with more resources, creation of Editable* objects will be governed by a dynamic memory budgeting system within experiences and when running in Play Mode within Studio. If you are using Editable* objects for plugins in Studio Edit Mode, these budgets will not be applied.
Now that these APIs are available for published experiences, the engine will be enforcing a memory budget when creating new Editable*
objects. Always remember to check if the Editable*
creation APIs returned the object or nil
before continuing to use the APIs.
Using the Destroy APIs to free up memory
Click to read more:
You can use the EditableMesh:Destroy
and EditableImage:Destroy
APIs to explicitly free up memory after you are done using these objects. The follow code snippet shows a simple example:
local myEditableMesh = AssetService:CreateEditableMeshAsync(Content.fromUri("rbxassetid://ASSET_ID"))
If myEditableMesh == nil then
-- Not enough memory to create the EditableMesh. Show fallback UI or skip editing the mesh
return
end
-- myEditableMesh is valid, so safely continue editing the Mesh
-- When all edits are complete you can explicitly call Destroy to free up the reserved memory
myEditableMesh:Destroy()
Instances that reference Editable*
objects when Destroy
is called will fallback to default / empty data. (e.g. empty mesh for EditableMesh
and black image for EditableImage
)
Setting the FixedSize
option
Click to read more:
When using AssetService:CreateEditable*
you can pass in an option that sets FixedSize
. An EditableMesh
object with FixedSize = true
will not allow you to add any more vertices or attributes but will allow you to modify existing ones.
The benefit of setting FixedSize = true
is that the engine knows the maximum memory the underlying object can ever use, so it does not need to reserve any extra memory for the case where more vertices, attributes or pixels are added.
Note:
- When using
AssetService:CreateEditableMeshAsync
to create anEditableMesh
object from an existing asset,FixedSize
defaults to true. -
EditableImage
objects are not resizable and maintain the size they are originally created with by setting theSize
option. Remember,EditableImages
can use up a lot of memory quickly: one 1k x 1k image uses the same budget as four 512 x 512 images.
The following example shows how to explicitly set the FixedSize
option:
local AssetService = game:GetService("AssetService")
-- This EditableMesh can have vertices / attributes added to it
local myEditableMesh = AssetService:CreateEditableMesh({ FixedSize = false})
-- This EditableMesh defaults to `FixedSize = True` because it is created from an asset.
-- The value can be overridden by directly passing in {FixedSize = False} to allow modifications
local myEditableMesh2 = AssetService:CreateEditableMeshAsync(Content.fromUri("rbxassetid://ASSET_ID"), { FixedSize = false})
-- This `EditableImage` will always have its `Size` be 32x32 pixels
local myEditableImage = AssetService:CreateEditableImage({ Size = Vector2.new(32, 32) })
Permissions
As introduced in the previous Studio Beta update post, to prevent misuse of assets using the EditableMesh
and EditableImage
APIs, these APIs will only be allowed to load assets:
-
that are owned by the creator of the experience if the experience is owned by an individual.
-
that are owned by a group, if the experience is owned by the group.
-
that are owned by the logged in Studio user if the place file has not yet been saved or published to Roblox.
The APIs will throw an error if they are used to load an asset ID that does not meet the criteria above.
View assets that you own
Click to read more:
You can see all assets that you own by visiting the Creations page on Creator Hub or by going to the specific page for the asset by visiting: https://www.roblox.com/library/<Insert Asset ID Here>
Use pcall
to catch permissions errors
Click to read more:
You can use the following code snippet to account for cases where you might not have permissions to load an asset into an Editable*
object:
local myEditableMesh
local result, errorMsg = pcall(function()
myEditableMesh = AssetService:CreateEditableMeshAsync(Content.fromUri("rbxassetid://ASSET_ID"))
end)
if result then
-- EditableMesh was created successfully and can now be edited
else
-- EditableMesh was not created successfully due to a permissions issue
print(errorMsg)
end
Ensure you have the correct owner set when importing new assets
Click to read more:
When importing new mesh or texture assets into Studio, remember to verify that you have the correct owner set to prevent having to re-upload all the assets again with the correct owner.
-
When using the Import 3D workflow to import meshes, you can set the appropriate owner by selecting an option from this dropdown:
-
If you are using the new 3D importer queue Studio Beta you can select individual assets to select the appropriate owner
-
When using the existing bulk import tool to import a number of mesh or image assets at once, the owner is always set to the logged in Studio user and you cannot upload assets on behalf of your group from this flow.
Trust and Safety
Safety and civility is foundational to everything we do and these APIs are no exception. EditableMesh
and EditableImage
are powerful APIs that give you direct access to the faces and vertices of meshes and the pixels within images. With this power comes responsibility.
Our long-term vision is for the platform to support “real-time safety for in-experience creation,” so that users feel safe to create and collaborate freely using any tools creators provide. As we work towards that vision, we are taking the following steps to keep the platform safe in the interim:
-
Creators have to be ID-verified to publish experiences with these APIs.
-
Creators have to explicitly opt-in to using the APIs after reviewing the Terms of Use.
-
EditableMesh
andEditableImage
do not automatically replicate (i.e., edits are not automatically shared between clients or the server like other instances and properties on the platform).
As part of our recent Content Labels update (previously Experience Guidelines), certain free-form user creation experiences will be considered Moderate and restricted to 13+ users. If you enable Mesh & Image APIs using 3D assets that have individually gone through Roblox moderation (e.g. building a house with blocks, creating an avatar, customizing a car), then your experience can remain Minimal or Mild (previously All Ages and 9+ respectively).
However, if you allow users to draw or write in 2D and replicate those creations to other users without the completed creation going through Roblox moderation (e.g., writing or drawing on a chalkboard, whiteboard, or with spray paint), then you must update your Content Labels questionnaire (Creator Hub > Creations > Select an experience > Audience > Questionnaire), and your experience will be Moderate and restricted to 13+ users.
Tips for Safer Experiences
Remember, all experiences published on Roblox may be moderated for egregious user behavior within the experience as per the Terms of Use. In general, it is a good idea to:
-
Be careful when creating in-experience tools for free-form creation that could result in policy-violating content.
-
Be careful when enabling custom replication for user creations since you could be sharing policy-violating content.
-
Consider limiting users to UI like sliders or control values that can never result in policy-violating content. It’s much safer than free-form creation.
-
Do not load asset-like content from off-platform sites or unmoderated sources e.g. DataStores, source code, or StringValues. Storing or retrieving settings, slider values, or seed values for procedural generation is generally acceptable.
-
Try to make sure every free-form user action (decal, brush stroke, etc.) can be attributed back to a single user and potentially removed if it is policy-violating. Avoid mixing the work of multiple users into a single asset.
Updated example place file
We’ve updated the Hubcap customizer experience from the original Studio Beta announcement post to reflect the new Object and Content workflow as well as the permissions and memory management best practices described above.
Download the file here: HubCap_Image_v24.rbxl (437.4 KB)
Notes and Instructions:
-
In the RBXL file, all the scripts that make use of EditableMesh / EditableImage can be found under:
StarterPlayer/StarterPlayerScripts/HubcapScript
-
In
HubcapScript
, take a look at theInit()
function all the way at the bottom of the script for the main entry point. Here you will see howCreateEditableMeshAsync
is used to create anEditableMesh
object from an existing mesh asset. Note: This mesh asset has been whitelisted for all clients on Roblox so you do not need to have ownership permissions to load it into anEditableMesh
object -
The single hubcap spoke is then duplicated around the hubcap and the sliders are used to perform some straightforward manipulation of the mesh vertices
-
Also in
HubcapScript
, take a look at theupdateCurrentDeform(meshInfos, emesh, params)
function to see how theEditableMesh
object is used to get the vertex positions and modify their positions based on the values from the UI sliders. -
Take a look at the
HubcapScript/MeshPainting
ModuleScript to see an example of howEditableImage
can be used. The functionMeshPainting.Init()
sets up a newEditableImage
object and sets it as theSurfaceGui
ImageLabelContent
so you can preview edits. -
Take a look at the
castRayFromCamera(position)
function in theHubcapScript/MeshPainting
ModuleScript to see how theEditableMesh:RaycastLocal
API is used along with theEditableImage
object to allow the user to paint directly onto the hubcap -
The
doDrawAtPosition()
function uses theDrawCircle
API to mimic a circular “paint brush” -
This place file also contains a number of additional helper ModuleScripts for a variety of common tasks like
- Setting up UI sliders (
HubcapScript/SetupSliders
), - Calculating the mesh deformations necessary (
HubcapScript/MeshMath
) - Showing a color selection palette (
HubcapScript/SetupPalette
)
- Setting up UI sliders (
Bug fixes and Improvements
-
Improved overall stability
-
Introduced
Content.none
, theContent
equivalent of an empty asset id -
Introduced Destroy() for both EditableImage and EditableMesh
-
Introduced
EditableMesh:GetCenter
&EditableMesh:GetSize
. -
Fixed
LocalRaycast
missing edge hits -
Fixed rendering culling meshes improperly
-
Fixed Luau autocomplete types for
Content
APIs
Known Issues
Here are a list of known issues that are actively working on fixing
- Meshes with skinning information cannot currently be converted to
EditableMesh
. - A perfectly flat axis-aligned plane will not render properly.
- Since
EditableMesh
andEditableImage
objects do not currently replicate, we replicate an unusableObject
of the same type in its place. Any attempt to read or write the contents of these “replicated” objects will throw. We’ll replace this with standard replication behavior in the future. - Setting vertices as transparent currently only works when the MeshPart is at least partially transparent.
- There is a rare chance of the experience crashing during shutdown. The fix for this rare crash is rolling out to clients this week.
- On some Android devices that fall back to OpenGL ES 2.0,
EditableMesh
objects might not be rendered correctly or rendered at all. We estimate that less than 2% of devices should be affected by this. - There are known issues with frustum culling for
EditableMesh
objects and how their bounding box is calculated. This could result in the meshes disappearing at some camera positions and angles. You can work around this by building yourEditableMesh
around the bounding box center and then moving it with theCFrame
instead of moving the vertices themselves. - Plugins that use the
Editable*
APIs in places with Team Create enabled currently require the user to flip on the game Settings > Enable Mesh / Image APIs toggle. This will be fixed soon so any Studio plugin (edit mode) use ofEditable*
APIs will not require flipping on the toggle in Game Settings
What’s Next
-
We’ll keep iterating on the memory budget system to make it more dynamic, and work with other engine systems to scale down to create more headroom as needed.
-
EditableImage
/EditableMesh
and their contents are intended to replicate from server to client whenever anInstance
referencing them would replicate. We’re still working onObject
replication support and efficient image/mesh data replication. -
We plan to provide an efficient option to use runtime-generated images and meshes once you are done reading or editing them, without needing to publish them, similar to in-experience Solid Modeling (CSG).
-
We plan to provide a plugin-security publishing API to publish Mesh and Image assets directly from
Editable*
objects. -
We also eventually plan to provide in-experience publishing APIs to publish Meshes and Images to the Inventory. We are working out details on UX and scalability.
Getting these APIs out to the community has been a long road for us and all the feedback, bug reports, scenarios and workarounds you have shared with us over the months have been invaluable in getting to this state. This is just the start of the road for us though and we still have a number of APIs we would like to add and workflows we would like to enable.
Thanks,
@TheGamer101, @L3Norm, @monsterjunjun, @ContextLost, @LowDiscrepancy, @Penpen0u0, @LowDiscrepancy, @neutrinostack, @c0de517e, @FarazTheGreat, @syntezoid, @portenio, @FGmm_r2 and @gelkros on behalf of the Geometry, Rendering and Avatar teams.