After stumbling upon an interesting interaction that meshes with erased vertex alpha have in a ViewportFrame, I’ve been experimenting with them for a while now and want to share my discoveries.
In this tutorial I’ll be sharing these discoveries and I’ll try to explain how all of it works to the best of my ability.
What Are Meshes With Erased Alpha?
First things first it’s good to know how you can erase vertices’ alpha. Erasing vertex alpha can be done for example in Blender by selecting “Erase Alpha” blend mode in Vertex Paint.
Use this material on your mesh to see vertex alpha:
Blend file for converting selected meshes to have 0 alpha
Mesh_0Alpha_Paint1.blend (855.2 KB)
Press arrow to run script:
0 alpha paint code snippet for Blender
This should work in any Blender project. If the project doesn’t already contain AlphaVisualizer material then the script will create a new one and apply it as each selected meshes’ first material (the script won’t remove the other materials tho they just get moved down):
import bpy
# What alpha level should selected objects have
# 1 fully visible, 0 fully transparent
alpha = 0
# What color should selected objects have
r = 1
g = 1
b = 1
def main():
# Change alpha of all selected meshes to 0
selectedObjects = bpy.context.selected_objects
# Stop script if no objects we're selected
if len(selectedObjects) == 0:
return
# AlphaVisualizer material
alphaMat = bpy.data.materials.get("AlphaVisualizer")
# Create AlphaVisualizer material if it doesn't exist
if alphaMat == None:
mat = bpy.data.materials.new(name = "AlphaVisualizer")
mat.use_nodes = True
# Add Vertex Color node and link its Alpha to BSDF's BaseColor
tree = mat.node_tree
nodes = tree.nodes
node1 = nodes.new("ShaderNodeVertexColor")
node1.location = (-180, 240)
bsdf = nodes["Principled BSDF"]
tree.links.new(node1.outputs["Alpha"], bsdf.inputs["Base Color"])
alphaMat = mat
# Set shading to Material Preview
myAreas = bpy.context.workspace.screens[0].areas
for area in myAreas:
for space in area.spaces:
if space.type == 'VIEW_3D':
space.shading.type = "MATERIAL"
break
for object in selectedObjects:
# Continue to next object if current object isn't a mesh
if object.type != "MESH":
continue
mesh = object.data
# Store current materials except AlphaVisualizer
currentMats = []
for material in mesh.materials:
if material and material.name == "AlphaVisualizer":
continue
currentMats.append(material)
# Clear materials and add them back
mesh.materials.clear()
mesh.materials.append(alphaMat)
for material in currentMats:
mesh.materials.append(material)
# Create new vertex color group for mesh if there isn't one
vertexColorGroup = None
if not mesh.vertex_colors:
vertexColorGroup = mesh.vertex_colors.new()
else:
# If mesh has vertex color groups then use first one
vertexColorGroup = mesh.vertex_colors[0]
vertexColorData = vertexColorGroup.data
# First three floats determine color and 4th is alpha
newColor = (r, g, b, alpha)
vertexCounter = 0
for polygon in mesh.polygons:
for vertex in polygon.vertices:
vertexColorData[vertexCounter].color = newColor
vertexCounter += 1
main()
Masking Out Parts Of a ViewportFrame
Meshes with erased vertex alpha render nothing behind them and in that sense are essentially masks for the ViewportFrame.
With this you can already create some cool effects by masking out parts of a ViewportFrame. However with 0 alpha meshes you can only take away from the end result. Adding to the mask is possible but it further complicates things.
Adding To ViewportFrame Mask
There is a way to add to the end result but its quite hacky and introduces further limitations. This method uses transparent glass parts which will render the otherwise invisible 0 alpha meshes behind them.
The limiting factor of this method is that you can only use 0 alpha meshes behind the glass for the mask addition to work in a desirable way. Instances like Decals and Textures don’t inherently appear invisible like 0 alpha meshes do which breaks the desired effect.
As mentioned before things like Decals, Textures and mesh textures don’t work properly with this method as they don’t inherit the masking capabilities of 0 alpha meshes. SurfaceAppearances in the other hand do which leads to the next topic: using images as masks.
Images As Masks (Supports EditableImages)
Since a SurfaceAppearance parented to a 0 alpha mesh inherits the mesh’s masking ability, it can be used to create custom masks with images. For this method to work AlphaMode of SurfaceAppearance has to be on Transparency.
The opaque parts of the image won’t render the things behind them while the transparent parts do. A limitation though with using SurfaceAppearance is that they only support 5 or 2 levels of transparency which can create ugly transparency banding.
EditableImages can also be used as masks when you parent one under a 0 alpha mesh’s SurfaceAppearance. This way masks can be painted, animated etc.
Diamonplate part being masked by image on left:
Test out ViewportFrame masking examples
After experimenting with this hacky masking method for a while I’ve put together a rbxl place file with a few examples utilizing ViewportFrame masking. I’ll also show short clips of them and share a place file if anyone wants to try them out in game.
Note: In the following examples a blocky character rig is forced for the player. Right now it’s not possible to create a mask dynamically for players as ViewportFrames don’t support EditableMeshes.
If ViewportFrames supported them you could simply create an EditableMesh for each character mesh and set each vertex’s alpha to 0 with SetVertexColorAlpha().
However Character2 example works with the player’s own character as it only sets each character part to be glass so that they can reveal the 0 alpha planes behind them.
(Image and Paint example don’t work in game as they rely on EditableImages)
Test in game: ViewportFrame masking - Roblox
Test in studio: ViewportFrame_Masking_Examples.rbxl (330.1 KB)
List of examples
House example:
In this example the left house, baseplate and Character mask out parts of the other house in the ViewportFrame.
As the two cubes in Workspace aren’t masked out they appear below the ViewportFrame.
Text example:
In this example each letter is a separate glass mesh which reveals the two 0 alpha cubes behind them.
The example also demonstrates how Decals, textures and mesh textures don’t inherit 0 alpha mesh masking capabilities while SurfaceAppearances do.
Wavy text example:
In this example a skinned mesh is used as a mask. This way the mask’s vertices can be transformed with code.
Another way to achieve per vertex transformation would be to use an EditableMesh but they aren’t supported in ViewportFrames yet.
Character1 example:
In this example the character mask is glass with 0.99 transparency which reveals two 0 alpha mask planes behind the player.
The two planes have a stretched UVs and a SurfaceAppearance parented to them which creates sort of a cross hatch effect on the player. This effect however is hard to occlude properly as masks made out of objects in Workspace interfere with the cross hatch planes.
Character2 example:
In this example the character is converted to glass just like in Character1 example but now with the player’s own character. This example only works with R15 characters as you cannot set R6 character parts to glass without extra steps.
The animation within the player is slightly more complicated compared to Character1 example. This time there is two 0 alpha mesh planes with tiling star SurfaceAppearances and another two 0 alpha planes with tiling galaxy cloud SurfaceAppearances.
Image example:
This example is a simple implementation of using images as masks.
Paint example:
This example is the same as the image example but you can draw the mask in and out with the help of EditableImages.
Portal1 example:
In this example glass spheres reveal a the Crossroads map thats made out of 0 alpha meshes.
Since Decals, Textures, mesh textures and part transparency don’t work with this method of masking, the billboards in the Crossroads map are blank and the beacons’ transparent parts aren’t rendered.
I haven’t figured out how to use this method for traversable portals.
Portal2 example:
In this example the Crossroads map is split in half along a 0 alpha plane with a doorway in the middle.
I haven’t figured out a way to use this method for portals but I think it has promise.
Reflection example:
In this example the house is flipped and parts of it are masked out with a puddle mask image.
Using this method for planar reflections can get performance heavy as both the moving objects in the reflected image and the moving mask objects have to be updated manually.
Summary
You can use meshes with erased alpha (0 alpha meshes) to not render anything behind them in a ViewportFrame. Erasing mesh alpha can be done in Blender for example.
With 0 alpha meshes you can mask out parts of a ViewportFrame but not add to it.
However transparent glass parts can be used to in a way add to the mask rather than take away from it by showing the otherwise invisible 0 alpha meshes behind the glass parts.
Other notes
This hacky way of masking out parts of a ViewportFrame can be used for cool effects but as it’s probably not an intended feature of ViewportFrames it might get fixed in a future update. For now though it can be used for all sorts of things despite it’s multiple flaws which might get solutions from combined efforts of other devs experimenting with this further.
I’d be glad to hear any ideas for ways to improve this masking method or use cases for it in the replies!
