Hi NiceDraft,
This is actually expected. FastClusters will be invalidated whenever a part is added or changed, to generate a single mesh and texture for all the parts to improve rendering performance, but apparently in your case, this is causing performance issues. Can I ask what your intention is to add hundreds of parts under an avatar and change them very often? If this is something really common among developers, we could do something handle cases like this in the client. Like you said, some or most developers might not know the difference between adding a part to an avatar and adding it directly to the workspace.
Updating characters this way causes unnecessary lag to games where the character updates frequently. For instance:
Shooters (where you die and respawn often, with animated guns and outfits re-parented to the character)
Anime or physical combat games (where you use a lot of “flashy attacks” and custom part-based effects inside the character)
300 parts were created to simulate the cost of ~20-30 mesh parts in our game (gun MeshParts inside the character). Regular parts are an order of magnitude less expensive than mesh parts in my use case. The invalidated clusters still take up a lot of frame time due to the frequency of player respawns. You can view this in the microprofiler here - make sure to select Competitive when you spawn: [NEW GUN] Aimblox BETA - Roblox
Player character updates can also replicate in batches, so I see problems like this:
And I’m running on an overclocked i7 9700k & RTX 2080S - we can expect double to triple the frametime per invalidated cluster on an average machine, and even more on mobile.
I see a lot of developers:
intuitively parenting assemblies which move around with the character, to the character (e.g. custom accessories, custom tools, custom outfits, custom visual fx, will be parented into the player character model, as opposed to a non-character object like Workspace)
animating parts inside the character (for example, a blast effect in an anime game is parented to the player character, changing rotation, size, and transparency every frame, thus re-uploading the whole player character geometry each frame)
respawning players frequently in PvP games where players die a lot - this causes unnecessary lag spikes when anything relating to visuals is added to the player character
In my specific use case, we parent these to the player character (usually on spawn):
Guns & gun holstering
Red outline parts (for enemy players)
Hitboxes (per-platform)
In one update ~2 months ago, we experienced severe lag spikes when animating out the transparency of the enemy player outlines on death, which dropped playtime by 20%. There is still an issue of frequent lag spikes due to players respawning. I feel like this doesn’t need to be the case, given how simple the workaround is.
Another example I just found on a game with 10k CCUs - Zenkai Origins - Roblox - just open microprofiler and look at any frame, ~4ms on a high-end desktop CPU+GPU
There’s probably many more front-page examples given how common it is for scripts to update sub-parts in characters when relating to part-based effects.
I agree that respawning happens quite often in pvp shooter games, but currently one respawn will lead to about 2 to 3 updateEntity() calls. In games like Arsenal, I think on average, there will be one spawn per second, which shouldn’t put much pressure on CPU, but updateEntity() caused by vfx update might be a different story, if any vfx update will trigger an invalidation, overall invalidation cost will be really high. Does Aimblox have the perf issue with vfx update you mentioned above?
You saw 4ms on updateInvalidatedFastClusters because that is the upper time limit we set for updateInvalidatedFastClusters() each frame. Any remaining invalidated fast clusters will be processed in the next frame if time limit for this frame has been reached. What I mean is that even on a low-end machine, updateInvalidateFastClusters() usually won’t cost more than 4ms, but it will take much longer for all the fast clusters to be updated. However, I think I still need to take a look why there are so many invalidation in a frame, and whether that is something pretty common or not.
I’ll do some investigations and will get back to you soon.
HI NiceDrift,
Sorry for late reply. I did some investigation on the behavior of updateEntity().
First I confirmed that updating properties of particle emitters won’t trigger any updateEntity(). You could try this in an empty map with only one emitter and verify. If you still see updateEntity(), let me know and share with me your test map.
Second, no matter how many parts under a humanoid are updated, the humanoid will only be invalidated once, although updateEntity() could be called 2 to 3 times. (We are working on something to reduce number of updateEntity calls on each invalidation). This means, no matter whether there are 20 or 200 parts added to your avatar, the number of updateEntity() calls will be the same. If in your test with 200 parts, 200 parts are updated one per frame, it won’t actually represent the performance of having 20 parts. I still don’t understand why you are using 200 parts to simulate perf or 20 parts. If you are trying to simulate the physics/animation/simulation cost on a low-spec machine, this is OK, but this won’t increase any cost on updateEntity() calls.
Third, no matter how many humanoids are invalidated, the total budget for all updateEntity() calls in a frames is limited to 4 ms. All remaining invalidations will be handled in the next frame. 4 ms shouldn’t be a burden for most of the games. I played AimBox. There are about 400 kills in a 4 minute game, which means there will be about 1000 updateEntity() calls in 7200 frames (assuming 30 fps). Only 1 out of 7 frames will have 1 updateEntity() on average.
Last, if parts are welded to humanoids through attachments, when attachments are updated, the humanoid will not be invalidated. In your case, movement of guns should be handled by attachments and animation. I’m not sure how you implemented red outline and hitboxes, but I think there will be a way to avoid extra updateEntity() calls when they are updated. Are you sure the lag spikes when animating transparency of outlines are caused by updateEntity(). If you could explain your implementation to me, we may be able to figure out a solution for optimization.
Hello, Sorry for the necro bump but I’m having similar trouble in my project.
I’m having a belt machine gun whose belt is fully animated up to 17 belt sections, with the structure looking roughly like this:
WeaponOrigin → Magazine → BeltLink1 → BeltLink2 → […] → BeltLink17
Whenever the weapon is holstered in and out, the clusters are invalidated and it takes up to 19MS (!) per frames for a total of at least 20 frames, for the clusters to be validated. The weapon is parented in the character, all collisions are set to boxes, no query, no touch, no collide, massless and even in a special collision group that interacts with nothing.
Each belt sections hold 6 meshes and some CSG, which admittedly isn’t a great idea but the cost is absolutely huge regardless, especially considering that I fully turned the physics off for all of these entities.
Removing the belt sections has proven to drastically improve the performances, but it’s a bit of a bummer when the whole point of a belt machine gun is having the belt visible and individually animated.
Replicator ProcessPacket also seems to be struggling along, here’s a screenshot of the microprofiler of one of the frame:
I’ll go ahead and remove the meshes and turn all of the belt sections from meshes to as few CSG parts as possible, but this is pretty concerning if we ever need to add another weapon with large amount of parts / motor6D’s similar to this one.
@fishrotR This issue is made even worse by layered clothing. When testing the repro in the original post, the performance issue becomes less severe when deleting layered clothing:
A lot of games are being affected by this issue, such as Spring’s Rock by AerialsAbove, and Nico’s Nextbots by 1nicopatty. Some of these games are beginning to disable layered clothing because of the issue.
This happens with vanilla camera corescripts as well. Notice the profiler spikes when I zoom in, this is because the built-in camera corescripts change *Part.LocalTransparencyModifier for a few frames when zooming in/out:
This is causing significant issues for me, particularly affecting humanoids. The player character undergoes frequent changes during gameplay, and even a simple weapon switch causes micro stuttering due to our weapon-parenting approach.
We should find a way to avoid this optimization step, or make it an option, as it seems to do more harm than good in our game. Currently, I can force an update by adjusting the transparency of one part of the player character, this SHOULD NOT update the meshes or textures at all! If I try to tween the transparency of something parented to the player character, it causes very unstable framerates, which is something we need to address.
Most shooters on Roblox have players using FPS unlockers, rendering the 4ms budget relatively ineffective in these cases.
I can even cause the game to slow down to a crawl or even freeze for about 1-2 seconds in this repro file by removing or adding a part to the player character. I have exacerbated the issue by parenting 3000 hats in this case.
The total budget for all updateEntity() calls in a frames is limited to 4 ms
This is completely false and not what’s really happening.
In Arsenal (and mainly our other shooters) the cluster gets delayed for more than 5ms. The biggest offender during the heat of combat is equipping and other miscellaneous BasePart setters. All of which gets resolved by parenting out the accoutrements to a separate model.
I don’t see why this system of laggy cluster updater should exist when it does more harm than good and this current workaround with tons of bad practices should not be the solution to the problem.
Pretty annoyed on how nothing has been done. This issue is pretty bad, knowing that ANY visual changes to a humanoid can trigger this frame hiccup. Now that avatars are using complicated skinned meshes with tons of polygons, this completely kills performance and I’m left wondering why we need to keep this. Can we please get a toggle to disable this?
Here is the current workaround list, someone PLEASE tell me there is a better way.
Don’t use a humanoid
Use a humanoid but only have ONE part in it (humanoid root part), create a model with all the character visual’s and weld its torso to that
Do not make any changes to the humanoid during gameplay
Anything you want to change during gameplay, do not parent to the humanoid.
Here is a lovely video I recorded on a RTX 3090 and a Ryzen 5800 X3D, notice the awful frametime spike from 3~4 ms to 16-20 ms.
I dare anyone to play please donate on Oculus Standalone and try walking into someone with a complicated avatar, your headset basically freezes for half a second due to this issue.
I understand this is a bit of a necro and I don’t care. This bug has been wreaking havoc on my game.
I have a character that can switch between two forms. When they do, most parts in their character go invisible and some more become visible. The reverse happens when they switch back. This causes a large lag spike. This happens dozens of times per minute for a single user and can happen multiple times per second in a full lobby. The lag occurs for everyone if any user switches forms.
The transformation between these two forms has animations with multiple stages of parts becoming visible, or invisible, tools being unequipped, or force-equipped and each stage causes lag. The entire process is laggy.
I wanted to tween the transparency of part of the player’s avatar as they enter and exit certain areas (swimming in one of their forms) but this was causing a lot of slowdown.
I have a tank of fluid on the player’s back that moves and gets resized every frame to represent how much of a resource the player has. I use the scale property of a special mesh thinking that it wouldn’t cause lag since it doesn’t affect the colliders and use the transform property of a motor6D instead of a weld to move it by script which Roblox recommends. I lowered this to update once per second instead of every frame which is very low and still causes constant frame drops.
This did eventually force me to implement this mechanic in a better way by making the fluid a skinned mesh with bones I move around with code to represent the fluid level changing but that was work I don’t think I should have needed to do.
I also wanted to animate a Texture object on the player by changing its U and V offset properties by code but guess what? Lag. That’s a cut feature.
I am constantly fighting these invalidatedFastClusters. I didn’t even know what it meant or how to get around it by its name. All I could figure out was that layered clothing was causing it or making it bad enough to notice. It’s helpful that I now understand what causes it, but I really wish this was fixed sometime in the last TWO YEARS.
I’m constantly adjusting transparency on the player’s entire character so it would be a massive pain to move the entire avatar out of the player’s character model to stop this and even then I need tools to stay in the player’s character so their scripts will run.
It’s a huge issue that doesn’t get enough attention, I have to have this weird system creating a folder such as “workspace.PlayerName_Items” and fork the camera controller to always set LocalTransparencyModifier to 1.
This would mean you would have to handle clean up and welding accessories, good luck with player scaling if you use that.
Quick update. I am working on a solution to most FastCluster induced slow downs.
I ran the Reproduction place file that was posted by @NiceDrift against my latest version and it seems the massive slowdowns are gone.
Please note: These are massive core changes I am making and will take time to get into production. But help is under way. Bare with me…
Thank you so much for working on this, I was using microprofiler to try and debug performance issues and everything kept pointing back to the humanoid. Is this at all related to StepHumanoid being slow on the client side as well?
Great news to hear, hope to see this soon as our characters are getting more complicated and we are having humanoid shadow and render distance issues due to us parenting outside the humanoid.
