Lag caused by "replication send job"

The bug is an issue with Lag. Lately ive been in my testing server, trying to figure out why whenever the player count gets above 8 causes the game to begin to lag heavily. I decided to open a debug profiler to see if I can spot whats causing these lag spikes whenever the player count rises above 8.

Surely enough I did found something in the debug profiler.
image

I was told by a staff member that this isnt a part of my script, and is part of the “replication send job”.
I was then asked if streaming enabled was on or not, and so I decided to try and see if there was any differences in the debug profiler, with it on or off. Sadly there was no real change. Heres an image of how it looks with streaming disabled.

image

It always happens regardless of wether or not the player count rises above 8, however when the player count is above 8, it starts to become extreme. Tanking my laptop down from a solid 60 fps, down to 30, and sometimes 25.

Heres a link to my physics test game, where this has been happening in.

I do not know when this began to happen, as not many people usually are in my testing server. However I can say it was maybe about a week ago. Ive only noticed this today however.

If its something im doing I would love to know how to fix it, however as far as I know, its not something I can really fix.

2 Likes

Second screenshot looks like your rendering is taking longer than usual.

If the issue absolutely gets worse as more players join, then you should think about what your game does with more players that could cause lag.

Examples:

  • Spawning a house per player, could be a lot of parts as you add more players
  • If each player has 1000 ParticleEmitters in them, this will add up
  • If you had a 50 joint pet spawned for each player, this could slow things down
  • Do you start running new loops (while, etc) for new players?

Does the issue get better if people leave the server or does it stay laggy forever?

Anything that would scale with your playercount could help explain why.

2 Likes

Some notes:

  • the render job runs on the server in studio, but doesn’t run on production game servers. You can get equivalent microprofiles from running game servers in the wild via the dev console (streaming may have been the root, but the improvement may not be visible when testing in studio)
  • Looking at your game, your streaming settings set the min and target radius both to 3000. When these are the same, the engine is forced to stutter: the server is forced to gather the min radius all-at-once to minimize time without the minimum radius, and streaming stops as soon as it hits the target radius (meaning that the engine is never building buffer): as the player moves around it is forced to re-gather the minimum. If you set your target radius to be at least 128 larger than your minimum you should see much better performance because the engine will have more latitude to pre-buffer content before it gets to a point where it needs to do a critical send of the min content
  • setting min radius larger than ~1024 has been known to generate some slow behaviors (we are working on making the capacity for this larger, but in the mean time I recommend aiming to keep the minimum smaller than that). Note that this is a radius, so a setting of 3000 means that you have 3000 studs in all directions (it is like you are in a 6000 sized cube)
  • What kinds of behaviors are you seeing on the client when you see the lag? Latency and lag can affect different components of the engine differently and may have slightly different root causes. It could be server frame rate, it could be client frame rate, or it could be some kinds of queuing or buffering (for example, there is a time limit for how long we will spend resuming yielded threads each frame, so if there is a large-and-growing number of yielded lua threads, that resume-time-limit may end up act as a queue and postpone responses until later). Client rendering lag (low client frame rate) should appear in client microprofiles, if the client is responsive but you are seeing other players update slowly then it is likely server frame rate or some kind of queueing (I am guessing that you are seeing this second category).
6 Likes

Hey LordRugdumph and AbstractAlex, I apologize for my late reply. I had an issue of no internet for the past while.

I decided to take both of your advice and go hard into testing on a public server with many people. AbstractAlex was correct. The lag appears to be reaching out from my player models. My player models have too much joints and create too much parts.

I will be working on remaking many models for my game to reduce lag.

Thank you both for helping me with this issue and taking the time out of your day to respond!

Hey LordRugdumph and AbstractAlex, ive currently lost any sight as to what the issue could be.

I followed what you both have stated could be the cause of the lag. I reduced the tri’s massively on my models.

Ive removed many joints from the player model to see if that too would lower the lag count. Which it did indeed help.

However the game still lags insanely when the player count gets to above 10. It also doesnt make much sense to me, seeing as in my other build without the physic’s script in it, the game lags not one bit.

Ive used the microprofiler to see if it was the script, though it doesnt appear to be it. I have no idea what it could be.

I did a public test today and got this from my microprofiler test.

image

I assume this is the issue of the lag, but honestly I have no idea how to minimize it. Ive worked heavily to simplify my scripts, and reduce the polygon count in my models. Im honestly stumped here and would love to get some help in this.

Could the lag be from modules calling to each other a lot? Cause if thats the case ill shove all my code into a single local script again.

I just honestly want this to work and I cant figure it out.

It’s lighting related in that screenshot. If you have any BasePart instances in your player maybe you can play around and tick off CastShadow? Recently we had a uniform stress test in PF and CastShadow property on those caused lighting computations to take a bunch of CPU time. Probably because a bunch of the unions were inside of each other and they were on moving characters. I would try turning off shadows for complex objects.

2 Likes

I will certainly see if that improves performance, thank you for your input!
This game means a lot to me, and any way I can make it more playable for my players, the better!

Really appreciate all of you helping out thanks!

I have discovered the Real Issue. Through some minor testing.
I will be making a thread and linking the link as the solution.

My feelings are hurt.

Here is the thread where I discovered where this lag most likely came from.
This lag would also be depending on wether or not the player model is heavy on tri’s though so previous answers from Devs such as “AbstractAlex”, “LordRugdumph”, and “Raspy_Pi” arent wrong in any way, however the main issue to what I believe, was this.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.