Mouse Position is not Accurate

Bug Reports Engine Bugs
#1

Introduction

When getting the mouse position in-game, the position isn’t even close to accurate, usually two to three frames behind the actual position. When getting the position in studio, using the same methods, the position is 100% accurate.

After doing some diving into the deep dark of the devforum, I found that around 2014 there was an update made to studio that several users reported “broke” their sensitivity. There is also the HardwareMouse setting, which I couldn’t find in any menu anywhere, but appears to be an old studio setting to change mouse behavior, perhaps a beta of this mouse change?

Having an accurate mouse position at all times is incredibly important for almost any game. Even a few frames of delay is a big deal.

Reproduction

This bug can be reproduced by running the following code on the client.

local gui = Instance.new("ScreenGui", game.Players.LocalPlayer.PlayerGui)
local frame = Instance.new("Frame", gui)

gui.IgnoreGuiInset = true
frame.Size = UDim2.fromOffset(100, 100)

local uis = game:GetService("UserInputService")

game:GetService("RunService"):BindToRenderStep("MouseMove", 5000, function()
	local mousepos = uis:GetMouseLocation()
	
	frame.Position = UDim2.fromOffset(mousepos.X, mousepos.Y)
end)

For convenience I’ve put this into a copyable game

When running this on the Desktop Client, you will find there is a significant delay between the frame and the cursor. When running this in studio or in the windows app this delay doesn’t exist, and the frame matches the mouse’s movements.

I would get a recording to exemplify this, but something interesting happened.

OBS

Open Broadcast Software is open source screen recording software that is very popular and widely used. This is the software I used when I went to record a video of this bug, but when I enabled OBS Display Capture, the bug stopped. My mouse position matched the frame perfectly, like it does in studio.

Frame Unlocking and VSync

I fully understand “Unlocking” your FPS isn’t supported, but this does contain useful information.

Because of the nature of mouse input, I messed around with “unlocking” my FPS and Enabling and Disabling VSync (using axstin/rbxfpsunlocker and NVIDIA Control Panel). I found that my results were most ideal with a fully unlocked FPS and Disabled VSync.

System Information

Intel(R) Core™ i7-8700K CPU @ 3.70GHz
16 GB
NVIDIA GeForce RTX 2080

2 Likes
#2

This is because the OS mouse cursor updates very quickly. Generally the USB poll rate is set somewhere between 125 and 1000hz, and only at the last possible moment a few microseconds before scanout starts, does the mouse position get locked in for the rest of the frame. It’s called hardware cursor because it actually uses a dedicated feature of graphics hardware, often called planes or sprites, that allows a surface (the mouse cursor icon) to be composited during scan-out to the monitor, which means its position can be updated very late during the rendering process.

In the past, we actually hid the OS cursor and rendered one in-engine. This was changed a few months ago, as a way to significantly improve latency of mouse movements in the client. It feels really terrible when your mouse cursor has 10+ms of input lag. The reason this doesn’t happen in Studio or in the Windows Store app is because those have not yet been updated to use the OS cursor.

Because the mouse cursor is special cased in the OS, there isn’t really a way to fix this. The mouse updates on the screen with far less input latency than anything the engine is rendering. You’ll also observe this effect in other apps and games if you try it.

For context, here’s all the steps that happen in between the engine receiving a mouse event and your UI’s updated position showing on the screen:

  1. Engine receives the event and queues it up to be processed.
  2. User input events fire, like UserInputService.InputBegan.
  3. RenderStepped callbacks run.
  4. The draw commands are submitted to actually draw the next frame.
  5. The graphics hardware renders using the command buffers.
  6. The resulting app window framebuffer is composited by the OS into the main framebuffer.
  7. vsync happens and the final image is actually shown on the screen.

The OS mouse cursor position updates right before step 7, skipping all the other steps in terms of latency. Steps 4 and 5 are the ones that usually take the longest.

6 Likes
#3

Then do you know what OBS is doing that fixes this issue? As well as this, the delay isn’t one frame like this would cause. The delay is 3-4 frames.

#4

I’m not entirely sure, but one thing graphics drivers often do is actually deliberately queueing frames adding 1-2 frames of latency, for smoother rendering and for pipelining. OBS might be forcing the driver to render with a smaller queue.

I know there are some other conditions where the NVidia driver in particular will shorten the queue. It does this for VR, and when Reflex is enabled. It can also apparently be manually controlled in the driver:

773×583 28.6 KB

I think this isn’t enabled by default because it can reduce performance and make stuttering worse.

#5

I changed the Low Latency Mode setting in NVIDIA Control Panel, it didn’t have any effect for me. I’m no expert at low level system controls, I just make games on roblox. Are there any external programs that could have the same effect that OBS does? Having an accurate mouse position is very important to me and others.

#6

Sorry, I’m not sure. There might be programs like that but I don’t know of any myself.

#7