Custom character physics replication stuttering

Bug Reports Engine Bugs
#1

Recently started running into a bug in our game, custom characters (using replicated physics) controlled by other players are stuttering erratically. We started getting reports of this around June 29th or 30th

The bug happens regularly in this game during normal gameplay
https://www.roblox.com/games/3233893879/NEW-MAP-Bad-Business-1-14

The game hasn’t been updated since the 26th, this was never an issue before. Server microprofiler dump looks fine, I’ll attach it just in case.

log_8423B_microprofile_20200701-201232.html (1.6 MB)

13 Likes
#2

Same sort of issue has been appearing with my Planes or recent, I thought it was an issue I caused however realised I hadn’t changed anything in the code that would lead to this.

1 Like
#3

Is it a different game? Does it use regular Humanoids? Same issue, characters moving back (or somewhere) for a frame or so, before resuming normal interpolation? Also started around June 29/30?

#4

Saw this playing 1.15 on Windows.

1 Like
#5

What kind of Architecture are you using for applying positions on your custom characters?

For example:

  • Are you creating a single assembly on the server and letting the Players drive their characters, while using 100% roblox replication and interpolation for this?
  • Are you only replicating using a single proxy-object and using local scripts to update the more complicated animations/functions client-side for every player?
  • Do you have any local scripts that change the position of other players in your view?
1 Like
#6

The server creates a single assembly for the character and sets its network owner to the controlling player.

The client controls the character by setting velocity (for movement) and cframe (for rotation) each step, which is replicated by roblox. There are extra local parts welded to other characters (clothing, weapons, etc) but their position is never changed by code

2 Likes
#7

Is it possible that you have a bug that sets Velocity to a crazy value? If the interpolation system replicates a really insane velocity value that is one of the few ways you could generate a jitter like that.

1 Like
#8

I have checks to make sure velocity doesn’t get too big (or NaN). This also doesn’t affect single players generally, which would be expected if it was just a velocity bug. When it happens everyone replicated will start stuttering.

I also had not changed any code for around 4 days when this first started happening.

2 Likes
#9

NAN or massive checks may not be enough. The issue can happen if the velocity values are highly a-physical. The interpolation system relies on positions and velocities generated by physics, as these are the values that get replicated. If you are manually setting these values and introducing discontinuities in the motion, this kind of effect can happen.

Is any character logic simulated such that the positions and velocities are generated through physics, or the only modifications that happen to positions and velocities only through local scripts?

2 Likes
#10

Collisions are solved through physics, but that’s about it. The cframing just keeps the root part upright and rotated correctly. Velocity is all set through scripts.

Worth noting that all of this was working perfectly for at least a week and a half in our test place + main place beforehand

1 Like
#11

I’ve seen this happening in any game with custom characters.

It makes me horrible at Arsenal.

#12

Hmmm, do you know how often this happens? I’m going to try to play your game a few times a day for a bit to see if I can repro it. I have a special build that lets me see networking data, but when I used it just now everything was smooth as butter.

In the video you will see little green crosses coming out of characters that indicate where the characters will move and interpolate. I can pause this display and playback if I see a problem, its just that I didn’t see any problems when I played. (You can see me playback at 1:25 in the vod)

The reason I was asking about all of these velocities is that the Interpolation System should NEVER rewind something back in time. The only possible way to have an object visually move “backwards” along an interpolation path is if the velocity on one of the two interpolated points points HEAVILY in the opposite direction.

For example:

  • If your character moves at 5 studs per second, and we sample interpolation every 100ms, we would expect that if they are moving in a straight line to have only moved 0.5 studs.
  • If the displacement between two interpolated points doesn’t line up with the velocities, you MAY see weird rubber-banding effect.
  • Basically if you move 0.5 studs over 100ms, but your velocity is set to 20 studs (4x the expected) you can see very strange movement from this character.

Anyway, I’ll play a bit more and see if I can get any repros with debugging build.

3 Likes
#13

So I was able to find a case where it reproed, but around the same time my game stuttering and hanging for a few frames at a time. This raised a few questions for me:

  • Do you ever delete and replace another player’s model on the fly? (If you notice, when I scroll back and forth through this interpolated character occasionally one track disappears and is completely replaced by another track, see 0:55 ).
  • It does seem like there might be some weird improper velocities causing the, when you see me scrolling the the tracks, sometimes you a “red X” above a “Green X” position. This means there are 2 packets with the same Position and Velocity but different timestamps (this can cause jitter). Is it possible for your scripts to hang and cause 2 identical positions and velocities to be set as time goes forward?
  • The interpolation system depends on physics simulation time going forward. If you are generation positions and velocities based on scripts that may be hanging or following timelines that don’t line up with physics you yourself may be generating strange positions and velocities. RunService gameTime is basically what you need to match to make sure interpolation system works well.
1 Like
#14

The only time player models are deleted is when they die/leave

I don’t think this should happen, it’s only ever set by one script in heartbeat

1 Like