Texture Atlases - Creating Foliage Texture and Meshes

Introduction

Good day creators and readers - today we are going to learn about creating foliage for games! Foliage and vegetation are a great way to increase the detail fidelity, improve the art style, and bring a natural believability to your worlds. With a few simple principles, creating foliage for games is easier than it might seem. This article will focus mostly on principles to employ while creating, and will not dive deeply into how textures themselves are actually created from scratch - which is a whole different and much more complex rabbit hole! With that said, let’s get started.

This article focuses on using Texture Atlases for foliage. This is only one type of texture that can be useful in applying in your work. Other types of textures can include:

  • Trim textures, also known as trim sheets, are textures that combine multiple various elements of an object into a single texture, you can reuse trim textures for furniture, walls, and other similarly themed decor.
  • Tiling textures that can tile on both axes. This is useful on a wide variety of meshes and surfaces, such as terrain (grass, sand, mud) or repeatable surfaces like architecture (brick, plaster, marble). See How to Create Tiling Textures for a primer on creating this common texture type.
  • Unique 1-to-1 textures that don’t tile at all. These are completely unique and are designed for a very specific mesh, such as a character, clothing, or something with a uniquely sculpted shape, like a statue or tool.

When creating foliage for games, the first order of business is to understand the components that come together to create these assets; Textures and Meshes.

Textures, the surface appearance over an object, have a few special concepts that we’ll cover, such as:

Meshes, the 3d geometry of an object, will also need some special considerations for their construction for foliage, such as:

Textures

The first portion of the article will cover some principles involving the textures used to create foliage. First, we’ll cover some definitions and technical information to understand how textures work with foliage, then move on to how to arrange your textures, and then some planning and construction tips.

What is a Texture Atlas?

A texture atlas is a single large image that contains many smaller images or textures packed together. Think of it like a collage of all the textures you need for your game objects. Instead of having separate files for each texture, you combine them into one big file. This helps your game run faster because it reduces the number of times the game needs to load different texture files.

In the context of foliage, a texture atlas might include different types of leaves, grass, and flowers all in one image. When you create your foliage models (like trees or plants), you can use parts of this big image to add details to them. This makes your game more efficient and keeps everything organized.


Left - the Foliage Texture Atlas from Racing Template. Right - the Foliage Texture Atlas from an Onboarding Experience.

Alpha Channels and Textures for Games/Foliage

To understand In the context of a game texture, an alpha channel is an additional layer of information that controls the transparency of the texture. It determines which parts of the texture are opaque, partially transparent, or fully transparent. Here’s a simple breakdown:

  • Opaque: Completely solid areas where the texture is fully visible.
  • Transparent: Completely see-through areas where the texture is invisible.
  • Partially Transparent: Areas that are semi-transparent, where you can partially see through the texture.

How It Works:

  • Color Channels: A texture typically has three main color channels (Red, Green, and Blue).
  • Alpha Channel: This is a fourth channel added to the texture that stores transparency information.

Example:

Imagine you have a texture of a leaf. The color channels (RGB) hold the leaf’s colors, while the alpha channel determines the transparency around the leaf. The alpha channel ensures that only the shape of the leaf is visible in the game, while the area around it is transparent, making the leaf look natural when placed on a branch or on the ground.

Why It’s Important:

  • Realism: Alpha channels help create more realistic and detailed objects, like leaves, grass, and hair, by allowing smooth edges and transparency.
  • Performance: Using an alpha channel efficiently can also help with rendering performance, as it reduces the need for complex geometry to achieve the same visual effect.

By using alpha channels in your textures, you can create intricate and visually appealing assets that blend seamlessly into your game world.


Left is the RGB Color/Albedo texture, Right is the Alpha Channel (black and white pixels showing which portions of the RGB will be visible {white}, and which will be transparent {black}). Below each example is a photoshop channels window showing which channels are displayed.

Layout Tips / Arranging your Texture Atlas

When creating a texture atlas for game foliage, it’s important to optimize space usage by packing the textures closely together but leaving a small padding or space between them to avoid blending issues later in the process, when arranging the UV’s of your mesh. Keep in mind the following when arranging your atlas:

  • Use an “invisible rectangle” - It is important to keep in mind that even the most low-poly cards, or flat polygons (like leaves), that will be UV’d on the atlas will be three or four sided - so as a best practice, I recommend imagining an “invisible rectangle” around each element on the atlas, so neighboring polygons that are UV’d on that neighboring element won’t accidentally clip on top of neighboring elements.
  • Maintain texel density - Maintain texel density consistency across each element of the atlas so that each element tries to maintain relative proportions to one another. For example, a palm tree leaf and a flower petal should not be the same size on the texture atlas due to their extreme difference in size.
  • Plan for the future - It is a good practice to plan for future additions by leaving some empty space in the atlas for new textures later, so you don’t have to rearrange everything, and re-UV your meshes.

Orange Lines illustrate the space around the foliage elements to allow for Mesh UV’s to have some room to breath before encountering another element on the sheet, making for easier UV’ing on meshes.

PBR and Foliage

PBR (Physically Based Rendering) Textures for Foliage will typically be authored using the same principles as any other material with a few special things to keep in mind, namely, the unique nature the surfaces of different types of foliage, a wide array of responses from matte, to shiny, to waxy - which can be tightly controlled with a combination of roughness textures (black pixels for matte foliage, brighter gray for pixels for waxy surfaces, white pixels for very shiny surfaces). Also, even though foliage is not reflective, some modulation of the metalness texture can add a bit of extra punch to foliage, if it suits your artistic style.

And as always, lighting information (like shadows and highlights on leaves) should not be represented in the Albedo (color) textures, and should instead be handled by the rendering system itself, so that the foliage appears natural. Contrary to that last statement, however, adding a small amount of Ambient Occlusion to the textures can provide a bit more depth when the foliage is in shadow (because normal maps have no visibility when outside of direct light, adding AO can assist in the illusion).

And of course - the alpha channel of the Albedo (color) texture will contain the transparency mask.


The Ivy Texture Atlas uses PBR Albedo (Color), Normal, and Roughness textures.

Meshes

Creating foliage meshes for games is a subtle art, but ultimately is a fairly straightforward process that takes some imagination and a little bit of technical know-how.

If you don’t know how to model in Blender, or are new to 3d Modeling and looking for more information, please check out this article: Modeling 101 in Blender (Box Modeling) The rest of this article will assume that you have some knowledge of Blender and 3d Modeling.

Basic Construction Principles

When modeling meshes for use as foliage, it is super important to be very efficient with your triangle counts. When scattering foliage around a scene, such as flowers or ferns or grass tufts, there is typically a need to have a larger number of individual placements to create a lush and believable environment, and to fulfill the needs of the composition, so a scene can quickly fill up with many instances that may lead to high levels of transparency overdraw, and a lot of triangles rendered which can lead to bad performance if the meshes aren’t created efficiently.

The following sections will go over some best practices for creating efficient foliage, minimizing overdraw, as well as some concepts to consider when deciding how to create foliage.

Single Leaves vs Clumps/Bunches

When building out a foliage mesh, as an artist you’ll need to decide whether to create the mesh using polygons that display singular leaves, or larger clumps and bunches of leaves on a polygon, and sometimes a combination of both.

This decision will be driven by the specific aesthetic of the foliage, the performance needs described earlier, and the placement context.

This section will demonstrate some techniques for creating assets

Single Leaves

Singular leaves on their own polygon/triangle/element are great for adding more depth to the illusion of larger pieces of foliage, and are usually used sparingly on assets such as Ivy or Bushes that need volume. They can also be used on their own for plant types that have large individual blades. Examples below illustrate.


The ivy has single floating leaves to provide volume, the agave is composed entirely of single blades.

To properly build out meshes for these individual type elements start with a single polygon or triangle and add subdivisions only as-needed.


Starting with one Polygon, subdividing once, then twice, and adjusting verts for depth.

After a single element is completed, it can often be duplicated many times around a central, or logical point to create an entire foliage asset or supplement a larger conglomerate.


Duplicating the single leaf and rotating about center, then applying slight rotations and scales for variety. Stack complexity to complete the plant.

Clumps/Bunches

Other situations call for displaying groups and bunches of leaves and foliage on a single mesh element. This is better for creating foliage with density where using singular leaf elements would be prohibitively expensive from a rendering/performance perspective. You can use similar techniques to those above, simply rearrange the UV’s on the polygons to overlay areas of the texture atlas with content that is authored as clumped sections of greenery.

Bushes, trees, and large growths of ivy are great examples of showing clumps and bunches of greenery that create the illusion of thick growth.

To construct a section of clumped/bunched foliage, follow similar steps as the single leaf section above to create layered cards of depth.


This image, from Blender, shows the UV Atlas on the left and the 3d Viewport on the right. The 3d Viewport shows the progression of a single polygon (right to left) increasing modestly in triangle resolution. Bottom right, all elements are arranged together to create a dense section of Ivy with a total of 46 triangles.

Another creation method is to start with a single flat polygon and UV map it to overlay a full clump of foliage on your texture atlas. Then, duplicate that polygon and rotate it around its central axis, 3 times so that the asset has the foliage clump visible from all angles. Then duplicate that clump multiple times, and arrange them together in a meta-clump of sufficient volume. Finally, scale, rotate, and move each clump so they are all at slightly different sizes, rotations and locations to create an organic arrangement of shapes to generate the illusion of a fully dense bush or tree top section of foliage.


Bottom left is the texture atlas. In the 3d Viewport you can see how the single polygon is duplicated, rotated into a small element visible from all angles, then arrayed in a dense clump of macro-elements that are then repositioned, rotated and scaled differently from one another to complete the illusion of density.

Optimizing Meshes

Mesh optimization is of utmost importance when creating foliage, as mentioned before, because the number of assets on screen when decorating a scene with vegetation can quickly become quite full, so the assets should be optimized with as few triangles as possible on each mesh, and with as little extra transparency included as possible.

When creating foliage cards, try to minimize the amount of “dead space” or blank areas in your textures. This reduces transparency overdraw, which occurs when the renderer processes transparent pixels. Even though these pixels appear invisible, they still take up processing power. Each layer of transparency adds to the workload, as the renderer must calculate every pixel behind it. By reducing dead space, you can optimize performance and reduce the rendering cost.

This example shows the initial leaf on the left, composed of 4 polygons, or 8 triangles. The second leaf has been reduced to 6 triangles by collapsing the outermost vertices into the middle on the tip of the leaf. Then, the final leaf on the right has had the UV’s adjusted to be just outside of the leaf shape on the texture atlas, and had the two vertices on the wide end of the leaf pulled in closer to minimize unused transparency areas, with those UV’s adjusted to match. The final leaf has had the unused transparency areas reduced by ~38% and the triangle count by 25%. The final plant, when fully assembled will be significantly faster to render.

Note: Earlier in this article, it is mentioned that the atlas should be laid out in rectangular sections, you’ll notice that we don’t always want to lay out our UV’s to take over the entire rectangular space. The rectangular layout is simply a method by which you can ensure that your individual texture elements will not overlap one another when building UV’ing your meshes. Experienced creators might ignore this rule as they get more familiar with atlas creation.

This next example shows a slightly more complex asset, used to make a fern plant. More geometry has been used here to provide curvature to the form of the leaves and stem, making a more believable visual result… However, this is only a single stem of a fern, when arrayed around its center to finish the plant, the result relies on a great deal of geometry to create the curved form, and a fair amount of wasted UV space with unused transparent pixels that result in a lot of transparent overdraw during render.

To optimize this mesh, the triangle count can be reduced and the UV’s tightened closer to the visible pixels similar to the first optimization example above.

In this example, the mesh on the left is the leaf from the prior example. To the right, new cuts are made in the mesh to slice away any unneeded transparent pixels. The next mesh deletes the now unneeded faces. The final mesh on the right then collapses some of the excessive vertices into a single point along the spine, reducing the overall curvature of the mesh, but largely retaining artistic intent, in favor of a more optimized mesh.

When the optimized leaf asset is arrayed around the centerpoint to make a new fern asset, notice how fewer triangles are present, and how much less unused transparent pixels exist on the blank areas of the mesh. This optimization reduced unused transparency by about ~33%, and reduced triangle count by 60% (240 down to 144 triangles) without sacrificing hardly ANY visual fidelity.


These simple principles should be applied to every piece of foliage you craft for use in the engine to make an enormous improvement in your rendering performance.

Double Sided vs Single Sided

When bringing your foliage meshes into Studio and applying surfaceAppearance to them, you will notice that by default the foliage cards you created are completely invisible when you look at them from the back side of the face. To resolve this, simply go to the properties tab on the mesh and enable the DoubleSided property. This will make both sides of the polygon render the material on the mesh. Caution, however, because this effectively doubles the cost of your foliage asset, making it ever the more important to ensure your assets are properly optimized!

There may be certain times where you don’t need to see both sides of a foliage asset, and it’s okay to leave this disabled, but it depends entirely on the nature of your asset and the context it is displayed in. For example if you create an ivy asset that hugs tightly against walls and architecture, and the user can never see the back side of the foliage mesh, then it is okay to leave this property disabled to save on performance.

UV’s

This is mostly covered above, but this section will review and summarize proper UV’ing to texture atlases. When aligning UV’s from your mesh to the atlas, make sure the mesh and UV’s conform as closely as possible and while still being efficient with triangle count, to the shape of the underlying foliage element on the texture.

Again - the texture atlas may have large areas of unused space / blank pixels. When mesh UV’s overlay these areas on the atlas, they do not display any pixels when rendered (they are transparent areas). But these transparent pixels still exert a cost during runtime in the rendering process, and the more foliage cards that overlap one another in the viewport, the cost of those overlapped pixels grows exponentially.

Placement in Studio

When your foliage meshes are complete and assembled in Studio, keep in mind the performance cost of placing them into the world. Since transparent assets are more costly to render than others, judicious placement of foliage assets is key.

I recommend trying out some studio plugins to help scatter foliage about the scene - I like to use the Brushtool plugin. This plugin allows you to add individual foliage assets to the tool, and then paint them down en masse with the stroke of a brush to populate a scene quickly with assets. It’s important to tune your brush settings appropriately to avoid painting thousands of assets into the scene, bloating the data model and bogging down the renderer.

Make efforts to compose foliage clumps efficiently, avoiding excessive overlap, and avoiding putting lots of foliage into arrangements where the user camera can look “through” many of them at once (this creates the massive overdraw problems referenced throughout the article).


Fairly sparse placements of foliage above the active gameplay space / sparse placement in planters and along terrain.

Conclusion

Thank you for reading this foliage article. Hopefully this inspires you to take the leap into building your own foliage assets!

Texture atlases are the key to unlocking excellent foliage - making efficient use of the 0-1 texture space, authoring alpha masks to enable transparency, and leaning into PBR textures will be key to creating beautiful and believable vegetation in your creations. Creating meshes should now be within your reach if you keep in mind the basic construction principles, adding geometric complexity that follow the forms of your texture elements while being judicious with tri-count, and eliminating excessive transparent pixels across the polygons that extend beyond the foliage texture element.

Optimization is key for successful performance metrics when rendering in Studio, so be cautious with using too many vertices and placing too many into your scenes.

Hopefully this article serves you well, happy creating! Please add any questions or comments below, as this article is by no means an exhaustive aggregate of all methods for creating foliage, so we can continue the conversation below.

20 Likes

I hope I’m not wrong about this, but in case you want very stylized plants and trees and don’t mind them not being overly dense with leafs.

Computer hardware has actually gotten really good at rendering many polygons efficiently, provided that they’re all opaque, have a simple shader and also that not every single model has like a million polygons each.

Overdraw from transparency is like one of the biggest performance killers.
But if you don’t care that much about realism and want heavily stylized trees, grass, etc instead…

If you model slightly simpler trees and basically make all the leaves squares or pentagons (3 polygons per leaf), this sometimes renders faster than having a tree with very few polygons that uses cards with transparency cut-outs.

Modeling a tree that essentially has like a few thousand polygons might sound crazy and might sound like it’s super slow but modern graphics cards can render large numbers of polygons super fast.

And you can swap the trees out with simple models that don’t have every leaf modeled when it’s far away enough which should decrease total poly count in case a scene has like hundreds of trees.

I suggest testing it though, I have yet to experiment more with it myself.
But it’s wild how lots of polygons are sometimes just faster than a few overlapping ones with transparency.

I saw someone test this on the Nintendo 64.
The N64 is actually capable of rendering 10,000 polygons at 60 FPS, but as soon as you have like a few transparent triangles that overlap, performance tanks.

Apologies if this went slightly off-topic tho, I just hope this is informative.
I encourage researching it as well, it’s really interesting!

2 Likes

This is all true! This tutorial is indeed focused on foliage that uses transparency - and indeed performance is a crucial concern when building foliage this way. Some mitigating advice is listed in the Optimization section.

I will agree that if you take the more stylized approach, and use opaque objects, you have MUCH MUCH more leniency in this regard.

Thanks for the post!

3 Likes

This is an excellent resource! Regarding atlasing/trim sheets, would you recommend keeping all opaque textures and all alpha textures separated into different atlases?

Personally, I do recommend that for the sake of simplicity and cleanliness of project. But a case COULD be made that one colorMap texture with an alpha channel could be used in two different surfaceAppearance objects, where one SA is set to Overlay and another is set to Transparency, and each object could be applied as a child of different meshes for separate lines of reasoning - but this would probably be an edge-case, and the reasoning would probably be for making extreme cuts for memory.

So I guess that the real recommendation I’d make would be to let your content needs drive that decision. Most of the time, I would assume that content would have enough wiggle room to keep them separate though.