Replicate VR Hands from Client to Server?

Snowlikesgames · July 13, 2021, 11:11am

So, I am trying to make some basic VR support for myself in roblox studio to mess around with. Which is based from nathandecool1234’s community tutorials. And I have some problems.

With it I just want to achieve something I can mess around in for myself with objects and such.
The issue is that it is only client-sided. Which is what I need help with.
I have tried to simply code a few things but they result in some weird errors. For the server I copied the instancing method from the local-script into the server script, and passing the character as the parent through a remote-event. For all of the fails I also passed the hand CFrame from the local-script to the server-script to try set the CFrame.

a) Has one or both server hands spawning inside the default character. Not very visible but its there.

b) There is some hands still inside of the character, but there are some server hands spawning on the start location from the local ones. But they don’t update.

c) Still hands inside of the character, but there is a bunch of hands being instanced in-general.

the client-script

----------------------------
-- Services and gameobjects.
local VRService = game:GetService("VRService")
local RunService = game:GetService("RunService")
local UserInputService = game:GetService("UserInputService")
local StarterGui = game:GetService("StarterGui")
local Players = game:GetService("Players")

local Camera = game.Workspace.CurrentCamera
local Player = Players.LocalPlayer
local Character = Player.Character
local ReplicateHands = game.ReplicatedStorage.ReplicateHands

local TeleportingDistance = 100
local RightSendingCFrame
local LeftSendingCFrame



---------
-- Bools.
local teleportPressed = false



---------------------------------------------------------------------------
-- Camera properties, disable default movement and freeze non-VR character.
Camera.CameraType = Enum.CameraType.Scriptable
Camera.HeadScale = 1
Camera.CFrame = CFrame.new(Camera.CFrame.Position)

StarterGui:SetCore("VRLaserPointerMode", 0)
StarterGui:SetCore("VREnableControllerModels", 0)

Character.HumanoidRootPart.Anchored = true



-----------------------------
-- Creating the player hands.
local function createHand(handType)
	local hand = Instance.new("Part")
	hand.CFrame = Character.HumanoidRootPart.CFrame
	hand.Size = Vector3.new(.4, .4, 1)
	hand.Transparency = 0
	hand.CanCollide = false
	hand.Anchored = true
	hand.Name = handType
	
	hand.Color = Color3.new(1, .72, .6)
	hand.Material = Enum.Material.Plastic
	
	hand.TopSurface = Enum.SurfaceType.Smooth
	hand.BottomSurface = Enum.SurfaceType.Smooth
	
	hand.Parent = Character
	return hand
end

local RightHand = createHand("RightVRHand")
local LeftHand = createHand("LeftVRHand")



--------------------------
-- Functions to be called.
local function TeleportToDirectedPoint()
	local origin = RightHand.Position
	local direction = RightHand.CFrame.LookVector * TeleportingDistance
	local raycastResult = game.Workspace:Raycast(origin, direction, RaycastParams.new())
	
	if raycastResult then
		if raycastResult.Instance.Parent ~= Character and raycastResult.Normal.Y > .4 then
			local cameraAngles = Camera.CFrame - Camera.CFrame.Position
			local headCFrame = VRService:GetUserCFrame(Enum.UserCFrame.Head)
			
			Camera.CFrame = CFrame.new(raycastResult.Position + Vector3.new(0, 5 + headCFrame.Position.Y, 0) - headCFrame.Position) * cameraAngles
		end
	end
end



----------------------------
-- Up and running functions.
function handCorrection(part, move)
	local newCFrame = Camera.CFrame * move
	
	if part == Enum.UserCFrame.RightHand then
		RightHand.CFrame = newCFrame
		
		RightSendingCFrame = RightHand.CFrame
		
	elseif part == Enum.UserCFrame.LeftHand then
		LeftHand.CFrame = newCFrame
		
		LeftSendingCFrame = LeftHand.CFrame
		
	end
	
	ReplicateHands:FireServer(Character, RightSendingCFrame, LeftSendingCFrame)
end


function inputHandling(key, processed)
	if key.UserInputState == Enum.UserInputState.Begin then
		if key.KeyCode == Enum.KeyCode.ButtonB then
			teleportPressed = true
		end
	end
	
	if key.UserInputState == Enum.UserInputState.End then
		if key.KeyCode == Enum.KeyCode.ButtonB then
			teleportPressed = false
			TeleportToDirectedPoint()
		end
	end
end


function rightHandIndicator()
	if teleportPressed then
		local origin = RightHand.Position
		local direction = RightHand.CFrame.LookVector * TeleportingDistance
		local raycastResult = game.Workspace:Raycast(origin, direction, RaycastParams.new())
		
		if raycastResult then
			if raycastResult.Instance.Parent ~= Character then
				local hit = Instance.new("Part")
				hit.Material = Enum.Material.Neon
				hit.Anchored = true
				hit.Position = raycastResult.Position
				hit.Size = Vector3.new(.4, .4, .4)

				hit.Parent = Character

				RunService.Heartbeat:Wait()
				hit:Destroy()
			end
		end
	end
end

function leftHandIndicator()
	if not nil then
		local origin = LeftHand.Position
		local direction = LeftHand.CFrame.LookVector * TeleportingDistance
		local raycastResult = game.Workspace:Raycast(origin, direction, RaycastParams.new())

		if raycastResult then
			if raycastResult.Instance.Parent ~= Character then
				local hit = Instance.new("Part")
				hit.Material = Enum.Material.Neon
				hit.Anchored = true
				hit.Position = raycastResult.Position
				hit.Size = Vector3.new(.4, .4, .4)

				hit.Parent = Character

				RunService.Heartbeat:Wait()
				hit:Destroy()
			end
		end
	end
end



------------------------
-- Function connections.
UserInputService.InputBegan:Connect(inputHandling)
UserInputService.InputEnded:Connect(inputHandling)

UserInputService.UserCFrameChanged:Connect(handCorrection)

RunService.RenderStepped:Connect(rightHandIndicator, leftHandIndicator)



VRService:RecenterUserHeadCFrame()

all the server-sided code. Most of all are the same with minor differences.

a)

local event = game.ReplicatedStorage.ReplicateHands
local Players = game:GetService("Players")
local RunService = game:GetService("RunService")

local RightHand
local LeftHand

local RightHandCFrame
local LeftHandCFrame

event.OnServerEvent:Connect(function(plr, character, RightCFrame, LeftCFrame)
	local function createHand(handType)
		local hand = Instance.new("Part")
		hand.CFrame = character.HumanoidRootPart.CFrame
		hand.Size = Vector3.new(.4, .4, 1)
		hand.Transparency = 0
		hand.CanCollide = false
		hand.Anchored = true
		hand.Name = handType

		hand.Color = Color3.new(1, .72, .6)
		hand.Material = Enum.Material.Plastic
		
		hand.TopSurface = Enum.SurfaceType.Smooth
		hand.BottomSurface = Enum.SurfaceType.Smooth

		hand.Parent = character
		return hand
	end

	RightHand = createHand("RightVRHand")
	LeftHand = createHand("LeftVRHand")
	
	RightHandCFrame = RightCFrame
	LeftHandCFrame = LeftCFrame
end)

RunService.Heartbeat:Connect(function()
	RightHand.CFrame = RightHandCFrame
	LeftHand.CFrame = LeftHandCFrame
	
	RunService.Heartbeat:Wait()
	RightHand:Destroy()
	LeftHand:Destroy()
end)

b)

local event = game.ReplicatedStorage.ReplicateHands
local Players = game:GetService("Players")
local RunService = game:GetService("RunService")

local RightHand
local LeftHand

event.OnServerEvent:Connect(function(plr, character, RightCFrame, LeftCFrame)
	local function createHand(handType)
		local hand = Instance.new("Part")
		hand.CFrame = character.HumanoidRootPart.CFrame
		hand.Size = Vector3.new(.4, .4, 1)
		hand.Transparency = 0
		hand.CanCollide = false
		hand.Anchored = true
		hand.Name = handType

		hand.Color = Color3.new(1, .72, .6)
		hand.Material = Enum.Material.Plastic
		
		hand.TopSurface = Enum.SurfaceType.Smooth
		hand.BottomSurface = Enum.SurfaceType.Smooth

		hand.Parent = character
		return hand
	end

	RightHand = createHand("RightVRHand")
	LeftHand = createHand("LeftVRHand")
	
	RunService.Heartbeat:Connect(function()
		RightHand.CFrame = RightCFrame
		LeftHand.CFrame = LeftCFrame

		RunService.Heartbeat:Wait()
		RightHand:Destroy()
		LeftHand:Destroy()
	end)
end)

c)

local event = game.ReplicatedStorage.ReplicateHands
local Players = game:GetService("Players")
local RunService = game:GetService("RunService")

local RightHand
local LeftHand

event.OnServerEvent:Connect(function(plr, character, RightCFrame, LeftCFrame)
	local function createHand(handType)
		local hand = Instance.new("Part")
		hand.CFrame = character.HumanoidRootPart.CFrame
		hand.Size = Vector3.new(.4, .4, 1)
		hand.Transparency = 0
		hand.CanCollide = false
		hand.Anchored = true
		hand.Name = handType

		hand.Color = Color3.new(1, .72, .6)
		hand.Material = Enum.Material.Plastic
		
		hand.TopSurface = Enum.SurfaceType.Smooth
		hand.BottomSurface = Enum.SurfaceType.Smooth

		hand.Parent = character
		return hand
	end

	RightHand = createHand("RightVRHand")
	LeftHand = createHand("LeftVRHand")
	
	RightHand.CFrame = RightCFrame
	LeftHand.CFrame = LeftCFrame
end)

Sorry if I missed some important info or specifics, this is my first time here.

DutchDeveloper · July 13, 2021, 11:15am

Have you tried setting the network ownership of those parts to the player?

then all you do is change the cframe on the client and it will be replicated to the server.
Thats what i did for my small vr demo that you can check out here:
https://www.roblox.com/games/6974315906/Virtual-hangouts

Snowlikesgames · July 13, 2021, 11:44am

Thanks, I will try this and mark it as the solution if it works.

Snowlikesgames · July 13, 2021, 3:29pm

I can’t seem to figure out how to properly use network ownership stuff, for one is that the hands are anchored and documentation doesn’t have network ownership supported for anchored parts. And I am unsure if this goes in a server-script or a local-script.

EDIT: I have settled to use some different methods including network ownership. Thanks for the help here.

AdagameIsHere · December 31, 2021, 10:17pm

Oi, by any chance could you give a more in-depth explanation as to how you did this. Admittedly, I’m a little confused

Sleazel · January 4, 2022, 2:09pm

Old topic, but since I am currently working on it myself I can give you some hints.
I am assuming you want to replicate “VR Hands” functionality. Here is what you need to do:

(Optional) Disable Character auto-load. This will also prevent StarterGui from being loaded. If you want to have a hybrid game of VR and non-VR players, I suggest to manually load characters for normal players. This way you can put all non-VR scripts in the StarterGui.
Set up your VR environment

--local script
local VRService = game:GetService("VRService")
--Interrupt the script if player is not on VR
if not VRService.VREnabled then
        game.Players.LocalPlayer:LoadCharacter() --optional
	return 
end
--Remove stock pointer and overlays
game.StarterGui:SetCore("VRLaserPointerMode", 0)
game.StarterGui:SetCore("VREnableControllerModels", false)
game.StarterGui:SetCore("TopbarEnabled", false)

--This periodic check removes annoying cursor that keeps popping up.
task.spawn(function()
	while true do
		local VRFolder = game.Workspace.CurrentCamera:FindFirstChild("VRCorePanelParts")
		pcall(function()
			VRFolder:WaitForChild("UserGui", 5).Size *= 0;
		end)
		task.wait(10)
	end
end)

Few notes about the camera.

Camera in the Roblox DOES NOT rotate with the headset. It stays fixed with the initial coordinates and orientation. For example to move camera forward in the direction that the player is looking, you need to use camera:GetRenderCFrame() method and not the camera CFrame itself.
Feel free to move camera, but be careful with rotation. It is very nauseating. I suggest to do implement snap turning instead.
You can also opt for for snap movements, but in my experience smooth movement is not that much of an issue for most people.
Headscale is the special VR only attribute. It makes the map seem smaller or bigger for the VR user. But probably most importantly it increases the reach of player’s hands. That is why VR Hands has such huge VR players. I am not sure, but I think they set it to 10. (10 times bigger than regular R15 avatar)

--local script
--Get the camera ready
local HEADSCALE = 10
local cam = workspace:WaitForChild("Camera")
cam.CameraType = Enum.CameraType.Scriptable
cam.CFrame = CFrame.new(0,0,0) --important, make sure the camera is not tilted
cam.HeadScale = HEADSCALE

Get hands and head ready on server and make local VR script to wait for these. I leave to you to figure out how to make them look like player’s avatar. For now we just ask server to clone those for us. Put Your models in game.ServerStorage.Models folder.

--server script
local SetUpVRModels = Instance.new("RemoteFunction")
SetUpVRModels.Name = "SetUpVRModels"
SetUpVRModels.Parent = game.ReplicatedStorage

local VRTable = {}

local function OnSetUpVRModels(player)
   VRTable[player] = {}
   VRTable[player]["L"] = game.ServerStorage.Models.LeftHand:Clone()
   VRTable[player]["R"] = game.ServerStorage.Models.RightHand:Clone()
   VRTable[player]["H"] = game.ServerStorage.Models.Head:Clone()
   VRTable[player]["L"].Parent = workspace
   VRTable[player]["R"].Parent = workspace
   VRTable[player]["H"].Parent = workspace
   return VRTable[player]["L"], VRTable[player]["R"], VRTable[player]["H"]
end  
SetUpVRModels.OnServerInvoke = OnSetUpVRModels

--local script
local SetUpVRModels = game.ReplicatedStorage:WaitForChild("SetUpVRModels")
local LH, RH, Head = SetUpVRModels:InvokeServer()

I suggest to make players own head transparent.

--local script
for _, part in pairs(Head:GetDescendants()) do
  if part:IsA("BasePart") or part:IsA("Decal") then
     part.Transparency = 1
   end
end

We now need some kind of basic movement script. I use left thumbstick for forward/backward and strafing and right thumbstick for up/down and snap turning.

local thumbX, thumbY, thumbZ = 0,0,0
local rotated = false

UserInputService.InputChanged:Connect(function(input)
	
	if input.UserInputType == Enum.UserInputType.Gamepad1 then
		
		if input.KeyCode == Enum.KeyCode.Thumbstick1 then
		
			thumbX = math.floor(input.Position.X  * 10) / 20
			thumbY = math.floor(input.Position.Y  * 10) / 20
			
		elseif input.KeyCode == Enum.KeyCode.Thumbstick2 then
			
			if math.abs(input.Position.Y) > math.abs(input.Position.X) then
				thumbZ = math.floor(input.Position.Y * 10) / 20
			elseif not rotated and math.abs(input.Position.X) > 0.9 then
				rotated = true
				cam.CFrame = cam.CFrame * CFrame.Angles(0,-input.Position.X/3,0)
			end
			
		end
	

	end
end)

---reset variables to avoid drift
UserInputService.InputEnded:Connect(function(input)
	
	if input.KeyCode == Enum.KeyCode.Thumbstick1 or input.KeyCode == Enum.KeyCode.Thumbstick2 then
		
		thumbX = 0
		thumbY = 0
		thumbZ = 0
		rotated = false
		
	end
			
end)

To make sure hands movement is as smooth as possible, you should connect to RunService. While Heartbeat is better in most cases, to make sure server will not overwrite hands position with old data, we will use RenderStepped at this time. This way even if server intervenes, we will always have accurate position on the client. Since head is transparent, you can ignore it on client.

--server script
local VRUpdate = Instance.new("RemoteEvent")
VRUpdate.Name = "VRUpdate"
VRUpdate.Parent = game.ReplicatedStorage

local function OnVRUpdate(player,LH,RH,Head)
   VRTable[player]["L"].CFrame = LH
   VRTable[player]["R"].CFrame = RH
   VRTable[player]["H"].CFrame = Head
end
VRUpdate.OnServerEvent:Connect(OnVRUpdate)

--local script

local RunService = game:GetService("RunService")
local VRUpdate = game.ReplicatedStorage:WaitForChild("VRUpdate")

RunService.RenderStepped:Connect(function()
 
	camRender =  cam:GetRenderCFrame()
	cam.CFrame = (cam.CFrame + camRender.LookVector * thumbY + camRender.RightVector * thumbX + camRender.UpVector * thumbZ)
	--Head.CFrame = camRender --no need for transparent heads 

	local cfRH = VRService:GetUserCFrame(Enum.UserCFrame.RightHand)
	local rhrot = cfRH - cfRH.p
	local camrot = cam.CFrame - cam.CFrame.p
	rightHandCFrame =  camrot * rhrot + cam.CFrame*(cfRH.p*HEADSCALE)
	RH.CFrame = rightHandCFrame
	
	local cfLH = VRService:GetUserCFrame(Enum.UserCFrame.LeftHand)
	local lhrot = cfLH - cfLH.p
	leftHandCFrame =  camrot * lhrot + cam.CFrame*(cfLH.p*HEADSCALE)
	LH.CFrame = leftHandCFrame

       VRUpdate:FireServer(leftHandCFrame,rightHandCFrame,camRender)

end)

Also do not worry about firing the server that frequently. Remotes are being sent in packets and all you really need to keep an eye out on is the high bitrate rather than amount of events itself. And since FilteringEnabled is a thing, there is no really other way to do this anyway.

And this is basically it. Have fun with VR.