Custom HeightMap For SurfaceAppearances

  1. What do you want to achieve?
    -My goal is to have a smooth working heightmap feature that I can use for mesh parts.

  2. What is the issue?
    -The issue in my current setup is that the quality of the heightmap is being limited by the geometry of the mesh. I need a way to subdivide/add more geometry to the mesh without messing up the meshes data (uv’s, normals, ect)

Image Of Issue:


Mesh is very sharp and has an unpleasant look.

Current Setup:

local intensity = 0.25 --> changes how high or som
local heightMap = "rbxassetid://16322097897" --> texture of the height map

--> ⚠️embarrassing math:⚠️

local assetService = game:GetService("AssetService")

local object = script.Parent
local editableMesh = assetService:CreateEditableMeshFromPartAsync(object)
editableMesh.Parent = object

editableMesh.Scale = object.Size

local verts = editableMesh:GetVertices()
local tris = editableMesh:GetTriangles()

local mapImage = assetService:CreateEditableImageAsync(heightMap)
local heightMapResX = mapImage.Size.X - 1
local heightMapResY = mapImage.Size.Y - 1

local vertPositions = {}
local vertUVs = {}
local vertNormals = {}
for _, vert in verts do
	vertPositions[vert] = editableMesh:GetPosition(vert)
	vertUVs[vert] = editableMesh:GetUV(vert)
	vertNormals[vert] = editableMesh:GetVertexNormal(vert)
end

for _, vert in verts do

	local uvPosition = editableMesh:GetUV(vert)

	local pixelPositionX = math.round((uvPosition.X)*heightMapResX)
	local pixelPositionY = math.round((uvPosition.Y)*heightMapResY)
	local pixelPosition = Vector2.new(pixelPositionX,pixelPositionY)

	local vertNormal = editableMesh:GetVertexNormal(vert)
	local vertPosition = vertPositions[vert]

	local height = mapImage:ReadPixels(pixelPosition,Vector2.new(1,1))[1]*intensity

	local newVertPos = (CFrame.lookAt(
		vertPosition,
		vertPosition+vertNormal
		)*CFrame.new(0,0,(-height)*object.Size.Y)).Position

	editableMesh:SetPosition(vert,newVertPos)
end

Just to check:

  1. Do you need to maintain the same number of vertices / edges / faces, or is it acceptable to modify these as long as the normals + UVs etc are recalculated correctly?

  2. By ‘mesh is very sharp and has an unpleasant look’ - do you mean that you want the edges to look smoother, or is your issue that the height differences between verts are too apparent because of the lack of verts?

  3. What methods have you tried if any? Want to avoid suggesting the same one(s)


Also, do you have an asset id for the image you used in the picture of this thread? I downloaded the asset id you had written in your code block but it seems to be a different image entirely

  1. I dont need to keep the same amount of vertices, as long as I have the same UVs ect, then I should be good.

  2. The mesh doesn’t have enough verts in the areas it needs to give a good look, its most visible when the heightmap has a small dark area and it causes the mesh to have a pointy spot poking out of the mesh.

  3. I couldnt think of many ways to do this, (mostly because info on this topic is like non existant), but I did try grabbing each face a dividing between two vertices(to find the center) and adding a new vert. but, that didn’t work

And The Asset Id For The Heightmap In The Photo Is
HeightMap

rbxassetid://16322010839

Thoughts:

1. Subdivision

The quickest & easiest answer is probably subdivision as you’ve mentioned but of course, that has its problems i.e. increasing number of tris.

Though, since we don’t actually care about using the brick mesh per the OP, we can just create our own plane geometry with the desired number of segments, e.g.

Plane Geometry
--[=[
  creates a plane mesh given its w/h and how many segment(s) to create

  @param depth function - (optional) function to compute the height
  @param origin CFrame  - (optional) to reorientate vertices

  @returns mesh EditableMesh
]=]
local function createPlane(w, h, xseg, yseg, depth, origin)
  w = math.max(w or 1, 1)
  h = math.max(h or 1, 1)
  xseg = math.max(xseg or 1, 1)
  yseg = math.max(yseg or 1, 1)
  depth = typeof(depth) == 'function' and depth or nil
  origin = typeof(origin) == 'CFrame' and origin or nil

  local mesh = Instance.new('EditableMesh')

  local w_2 = w*0.5
  local h_2 = h*0.5

  local i_x = 1 / xseg
  local i_y = 1 / yseg

  local vid = 0
  local tid = 0

  local indicesLen = xseg*yseg*6
  local verticesLen = (xseg+1)*(yseg+1)

  local indices = table.create(indicesLen, -1)
  local vertices = table.create(verticesLen, -1)

  local u, v, x, y, d, vertex
  for i = 0, yseg, 1 do
    for j = 0, xseg, 1 do
      u = j*i_x
      v = i*i_y

      x = -w_2 + u*w
      y =  h_2 - v*h

      if depth then
        d = depth(x, y, u, v) or 0
      else
        d = 0
      end

      vertex = Vector3.new(x, d, y)
      if origin then
        vertex = origin * vertex
      end

      vid += 1
      vertices[vid] = vertex
      mesh:AddVertex(vertex)
      mesh:SetUV(vid, Vector2.new(u, 1 - v))
      mesh:SetVertexNormal(vid, Vector3.zero)

      if i < yseg and j < xseg then
        indices[tid + 1] = 1 +      i  * (xseg + 1) +     j
        indices[tid + 2] = 1 +      i  * (xseg + 1) + j + 1
        indices[tid + 3] = 1 + (1 + i) * (xseg + 1) + j + 1

        indices[tid + 4] = 1 + (1 + i) * (xseg + 1) +     j
        indices[tid + 5] = 1 +      i  * (xseg + 1) +     j
        indices[tid + 6] = 1 + (1 + i) * (xseg + 1) + j + 1

        tid += 6
      end
    end
  end

  -- [!] note: normally would need to normalise the normals
  --           but it seems roblox does this C-side
  --           so we can ignore this atm - though this could change in the future

  local a, b, c
  for i = 0, indicesLen - 1, 3 do
    a = indices[i + 1]
    b = indices[i + 2]
    c = indices[i + 3]

    local normal = computeFaceNormal(vertices[a], vertices[b], vertices[c])
    mesh:AddTriangle(a, b, c)
    mesh:SetVertexNormal(a, mesh:GetVertexNormal(a) + normal)
    mesh:SetVertexNormal(b, mesh:GetVertexNormal(b) + normal)
    mesh:SetVertexNormal(c, mesh:GetVertexNormal(c) + normal)
  end

  return mesh
end

2. Calculating normals

It may look like I did but I didn’t really do this in the example code above since it’s late here & it should be fairly simple for you to implement.

What’s important to note here though, is that flat shading is usually the result of calculating the normal from a single triangle but smooth shading can be implemented by taking the sum of the normals of the triangles a vertex is associated with and applying that as the vertex’s normal.

3. Image

Note: I doubt Roblox supports 16-bit image uploads so the only improvement here that you could likely do is increase resolution

Your image’s resolution is teenie tiny, might be a worth taking a look at getting a higher res image and I would recommend you take a look at the bit encoding you used to create the heightmap - you may not be encoding all the data you wanted to.

4. Blur

May or may not be appropriate depending on your usecase but we can apply a blur the heightmap to make it smoother. Several ways to do this, e.g. IIR Gaussian blur etc, but I opted to use Boatbomber’s blur to get this example together quickly for you - that can be found here

5. Smoothing

There are techniques to smooth a mesh but whether these are worth it would really depend on your use case, but an example of one of these techniques would be Laplacian smoothing


Note: The following example only includes (1) and (4) of the above as mentioned previously.

e.g. example output from the following code:
** water added to show depth

example.module.lua
local AssetService = game:GetService('AssetService')
local ReplicatedStorage = game:GetService('ReplicatedStorage')

local packages = ReplicatedStorage.packages

local Maid = require(packages.maid) -- BYO - use any Maid/Janitor _etc_
local Blur = require(packages.blur) -- see comment reference attached, I used Boatbomber's here

--[=[
  safely get normalised vector

  @param vector Vector3 - the vector

  @returns normalisedVector Vector3
]=]
local function getNormalisedVector(normal)
  local d = normal.X*normal.X + normal.Y*normal.Y+normal.Z*normal.Z
  if d > 1 then
    normal = normal.Unit
  end

  return normal
end

--[=[
  computes the luminance
    ref @ https://en.wikipedia.org/wiki/Luma_(video)

  @param r number     - rgb component
  @param g number     - rgb component
  @param b number     - rgb component
  @param gamma number - rgb component

  @returns luma number
]=]
local function computeLuminance(r, g, b, gamma)
  gamma = gamma or 2.2
  return
      0.2126 * math.pow(r, gamma)
    + 0.7152 * math.pow(g, gamma)
    + 0.0722 * math.pow(b, gamma)
end


--[=[
  compute face normal from 3 vertices
  +/- its one of the vertex normals (and/or the average across these)

  @param a Vector3            - vertex position
  @param b Vector3            - vertex position
  @param c Vector3            - vertex position
  @param vertexNormal Vector3 - normalised vertex normal

  @returns tri_normal Vector3
]=]
local function computeFaceNormal(a, b, c, vertexNormal)
  local p0 = b - a
  local p1 = c - a
  local normal = p0:Cross(p1)

  if vertexNormal then
    local d = normal:Dot(vertexNormal) -- OR; normal:Dot((na + nb + nc) / 3)
    normal = d < 0 and -normal or normal
  end

  return getNormalisedVector(normal)
end


--[=[
  creates a plane mesh given its w/h and how many segment(s) to create

  @param depth function - (optional) function to compute the height
  @param origin CFrame  - (optional) to reorientate vertices

  @returns mesh EditableMesh
]=]
local function createPlane(w, h, xseg, yseg, depth, origin)
  w = math.max(w or 1, 1)
  h = math.max(h or 1, 1)
  xseg = math.max(xseg or 1, 1)
  yseg = math.max(yseg or 1, 1)
  depth = typeof(depth) == 'function' and depth or nil
  origin = typeof(origin) == 'CFrame' and origin or nil

  local mesh = Instance.new('EditableMesh')

  local w_2 = w*0.5
  local h_2 = h*0.5

  local i_x = 1 / xseg
  local i_y = 1 / yseg

  local vid = 0
  local tid = 0

  local indicesLen = xseg*yseg*6
  local verticesLen = (xseg+1)*(yseg+1)

  local indices = table.create(indicesLen, -1)
  local vertices = table.create(verticesLen, -1)

  local u, v, x, y, d, vertex
  for i = 0, yseg, 1 do
    for j = 0, xseg, 1 do
      u = j*i_x
      v = i*i_y

      x = -w_2 + u*w
      y =  h_2 - v*h

      if depth then
        d = depth(x, y, u, v) or 0
      else
        d = 0
      end

      vertex = Vector3.new(x, d, y)
      if origin then
        vertex = origin * vertex
      end

      vid += 1
      vertices[vid] = vertex
      mesh:AddVertex(vertex)
      mesh:SetUV(vid, Vector2.new(u, 1 - v))
      mesh:SetVertexNormal(vid, Vector3.zero)

      if i < yseg and j < xseg then
        indices[tid + 1] = 1 +      i  * (xseg + 1) +     j
        indices[tid + 2] = 1 +      i  * (xseg + 1) + j + 1
        indices[tid + 3] = 1 + (1 + i) * (xseg + 1) + j + 1

        indices[tid + 4] = 1 + (1 + i) * (xseg + 1) +     j
        indices[tid + 5] = 1 +      i  * (xseg + 1) +     j
        indices[tid + 6] = 1 + (1 + i) * (xseg + 1) + j + 1

        tid += 6
      end
    end
  end

  -- [!] note: normally would need to normalise the normals
  --           but it seems roblox does this C-side
  --           so we can ignore this atm - though this could change in the future

  local a, b, c
  for i = 0, indicesLen - 1, 3 do
    a = indices[i + 1]
    b = indices[i + 2]
    c = indices[i + 3]

    local normal = computeFaceNormal(vertices[a], vertices[b], vertices[c])
    mesh:AddTriangle(a, b, c)
    mesh:SetVertexNormal(a, mesh:GetVertexNormal(a) + normal)
    mesh:SetVertexNormal(b, mesh:GetVertexNormal(b) + normal)
    mesh:SetVertexNormal(c, mesh:GetVertexNormal(c) + normal)
  end

  return mesh
end


-- main
return function ()
  local maid = Maid.new()

  -- const
  local SIGMA = 1                       -- gaussian blur radius (applicable only if > 0)
  local GAMMA = 0.5                     -- luminance gamma (applicable only if > 0)
  local SCALE_FACTOR = 0.5              -- heightmap scaling

  local SIZE = Vector3.new(50, 10, 50)  -- mesh size
  local WIDTH = 1                       -- plane geom width
  local HEIGHT = 1                      -- plane geom height
  local SEGMENTS = 50                   -- segment(s) of the plane geom

  -- instantiate image from asset
  local heightmap = AssetService:CreateEditableImageAsync('rbxassetid://16322010839')
  maid:addTask(heightmap)

  -- apply blur if applicable
  if SIGMA > 0 then
    -- note: if you had a higher resolution image we probably would want to downscale a little
    --           but since your image was tiny we'll keep it the same size
    Blur({ image = heightmap, blurRadius = SIGMA, downscaleFactor = 1 })
  end

  -- create mesh plane from heightmap
  local xres = heightmap.Size.X - 1
  local yres = heightmap.Size.Y - 1

  local mesh = createPlane(WIDTH, HEIGHT, SEGMENTS, SEGMENTS, function (x, y, u, v)
    local px = math.floor(u*xres)
    local py = math.floor(v*yres)
    local pixel = heightmap:ReadPixels(Vector2.new(px, py), Vector2.one)

    -- you can continue to use pixel[1] if it's grayscale
    -- but i was messing around with normal maps to test
    -- hence why i decided to compute luminance
    local r, g, b = table.unpack(pixel, 1, 3)
    if GAMMA > 0 then
      return computeLuminance(r, g, b, GAMMA)*SCALE_FACTOR
    end

    -- i.e. just use r/g/b components since b/w
    return r*SCALE_FACTOR
  end)

  -- apply to world
  local part = Instance.new('MeshPart')
  part.Size = SIZE
  part.Parent = workspace
  maid:addTask(part)

  mesh.Parent = part
  maid:addTask(mesh)

  -- cleanup
  return function ()
    maid:cleanup()
  end
end

I know you’ve marked as solution but I forgot to mention that another thing you could consider - again this very much depends on your use case - is to simplify / decimate the mesh after applying the heightmap e.g. we’re wasting a lot of tris in the flat areas which could be simplified (such as those under the “water” in the attached image).

You can read about this here and here but I should note that you’ll find a lot of examples on Github relating to this technique - if you struggle to find any just add “unity github” or “js” to your search, have seen several this morning when looking for appropriate papers for you.

There is of course many more in lower level languages though so it may also depend on your understanding of other langs as well. In fact, I think Zeux, an ex-Roblox developer, has a C++ mesh simplification library; though I’ve not actually had time to look at it yet outside of reading the title.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.