Changes to parts are not replicating!

Sorry - the place was restricted to group only for testing purposes, I have opened it up to let anyone join.

I can’t give an exact number of how many are seeing this, but anyone with a weak or under-powered device, be it PC, Laptop or Mobile, are reporting this same issue to me. I’m seeing it happen myself on a low-end laptop I picked up specifically to test this issue out.

I do not know how to run a traceroute.

When it happens, does everyone see the same thing, or do some people see the parts correctly deleted on their clients while others do not?

It’s different for all clients.

image

1 Like

OK, that’s pretty unusual. I left a message for devrel to check into this deeper. Thanks in advance for your patience, these issues involving things going wrong over a variety of different connections can take a bit of time to narrow down… lots of possible points of failure to rule out.

Thank you

Are the part Instances in the game tree, but are not rendering?

I once fixed a bug like this where Unions would take a long time to show up or disappear. Their Instances replicated and their physics replicated, but they “lagged behind” in rendering.

The cause, if I recall correctly, had to do with rendering cluster updates. If I recall correctly, there’s two kinds of updates, and one is expensive. Only a certain amount of updates can be completed per frame, and that limitation is based on hardware.

In our case, we were triggering the expensive update type every frame, which caused the updates that would add/remove union rendering to not happen on low-end hardware. Specifically, we were updating a Weld every frame, which is not made for constant, dynamic updates. We had to switch to a Motor6D, which fixed it almost entirely.

Your problem looks and sounds very similar. While it might not be updating Welds every frame, it could be something similar that causes an expensive cluster update every frame, which delays unions/meshes from appearing/disappearing.

They are instances in the game.

The only thing I can think of that I render every frame are the positions of the pets in-game which are basically just moved in respect to character’s positions every render frame.
However, this is an old system I pulled from another game, and the other game does not have this issue.

1 Like

If you are able to reproduce it in Studio, you could disable parts of the game until the issue stops occurring. Whichever part you disabled was the cause. It might be a pain to disable some parts if the rest of the game relies on them, but you should be able to comment out only the sections related to rendering or positioning objects.

That, combined with some research, is how we found out the cause of our bug. The section that we disabled had the Weld update in it, and we were able to narrow down the problem to one single line that set the C0 of a Weld.

Edit:
You can also make use of Find in all Scripts Ctrl+Shift+F and something that logs all instance adds, removes, and updates in order to find out what’s updating often. You can use game.DescendantAdded, game.DescendantRemoving, and game.ItemChanged for that last part. This script uses those to print a log out of all changes every 60 seconds.

1 Like

This other thread by the OP may help provide additional insight. Key takeaways:

  • Screenshots of exact memory usage
  • This happens after 5-10 minutes of play
  • This happens for devices with low VRAM (e.g. 64 MB, 112 MB, 494 MB)
  • StreamingEnabled is not being used
2 Likes

Thanks! I agree with your assessment in that thread. I too was thinking at first it was a networking issue, like a buffer overflow at the client end or something, but now it does look like its possibly a memory issue. Those low VRAM numbers you see are just the tiny amount that is actually on the CPU chip, as part of the Intel integrated graphics processor, but those systems also reserve memory from the computer’s main RAM to use as additional (slow) VRAM. I saw nothing weird at all in Ripull’s mini games, but I’m using a machine with 12GB of VRAM on the graphics card :sunglasses:

These issues aren’t exactly the same but very similar issues, although this was when loading into the game - some of the models didn’t replicate for this player…

and another one here:

The player standing on the hole was inside a building and it was fine - the player who took the screenshot could see this big hole and could fall through!
image

I am also experiencing this when the player respawns in my game, random things go missing or start to flash, which slows their client way down. Their character also doesn’t load so I get people saying they are invisible but they are visible to others.
Looking at the microprofiler when it happens, it looks like it is spending a lot of time invalidating clusters for those frames. I didn’t save the dump but will when I experience it again.

But what I cant understand is why other games (including my own projects) don’t also have this issue occurring.
The three people who’s specs I got on that other thread have also played my other two (more memory hogging) games, and all three of them are not experiencing these issues in my other games.
I too am not experiencing these issues in those other games on the test rig I have, but the game linked in the OP is a whole other story.

If it’s relates to invalidating clusters as pauljkl mentioned, it may be that your other games aren’t changing clusters often while this one is.

The source thread is actually pretty similar to this one. Can you upload a microprofiler dump to Google Drive when this issue happens again and link the file here?

1 Like

That thread was a great read. I think from that I narrowed down that the thing causing the issue was a local-sided animation using TweenService to tween the color3 of a group of 115 parts.
I can’t be sure yet if that is indeed the real cause, but I disabled the animation in real time on the client while in game and instantly the entire workspace cleaned up.

Okay, issue is better but still persists. I’m going to go try get that microprofiler dump file you mentioned.

1 Like

@EchoReaper this is the microprofiler dump. However I don’t know what I’m looking at in it https://drive.google.com/open?id=0B6obnAXuL_Phc0pMRWd3a3Mxbzg

I forget if I mentioned it in the linked threads above, but there’s now a way to check if you’re subject to indefinite update queuing:

In Ctrl+Shift+F2 panel, find the “Geometry Gen” line. Look at the last number. If it never reaches zero and is pretty large then you have an issue where you are updating appearance properties of parts (such as Size, Color, etc.) very frequently; we try to keep up with the updates but fail to do so; and the update system does not have any kind of deterministic order so a few frequently updated clusters could hog the entire update budget - every frame we would update them and then run out of the time allowance for updates per frame; next frame we’ll look at the update queue again and it will already have the same clusters. Repeat ad infinitum.

Can you confirm what numbers you’re seeing in this display panel?

1 Like

Ah yes this was the issues. Geometry gen had a constant queue of like 22p, queue 23c.

I think it’s safe to say the issues I was having in the OP were due to me doing to many local effects to parts in the game.

1 Like

Cool, that’s what I thought.

While new instancing system doesn’t have this problem, it’ll take a while until it’s active on all platforms for all types of parts; we’ll try to fix the queueing behavior to guarantee forward progress for all parts so that you can’t get into a situation where a few frequently updated parts effectively permanently disable updates on all other parts.

4 Likes