Mapping real-life body joints to a Roblox rig (Roblox VR)

I have a dictionary called data containing all the 3D joints in the human body, and I’m trying to map them to a Roblox rig. As stated by the documentation:

POSE_WORLD_LANDMARKS

Another list of pose landmarks in world coordinates. Each landmark consists of the following:

  • x , y and z : Real-world 3D coordinates in meters with the origin at the center between hips.

In this example, I’m moving my left shoulder by calculating where it’s facing my left elbow, by taking two Vector3's and using the CFrame.lookAt function where it returns a CFrame facing the elbow (if I’m understanding this function correctly).

--LeftROShoulder is a Motor6D in the LeftUpperArm
local LeftUpperArm = chr.LeftUpperArm
local LeftROShoulder = LeftUpperArm.LeftShoulder

--IRL stands for in real life
local IRLLeftShoulder = Vector3.new(data.left_shoulder.x, data.left_shoulder.y, data.left_shoulder.z)
local IRLLeftElbow = Vector3.new(data.left_elbow.x, data.left_elbow.y, data.left_elbow.z)
			
--find direction to look at
local LeftShoulderLookAt = CFrame.lookAt(IRLLeftShoulder, IRLLeftElbow)
--execute tween
local LeftShoulderTween = TweenC0(LeftROShoulder, LeftShoulderLookAt)

This actually works, but my body positions aren’t correct. This is me trying to reach out in front of my body, but I also used the lower parts of my arm, and both the right and left arms.

Does anyone know what else I can try or know what I could be doing wrong?

Okay and how does this help? Can you please read the post before you reply?

If the movements are occurring reverse to how they should be then just flip everything.