I’m working on a project where I need to map MediaPipe Pose landmarks onto a Roblox character’s Motor6D joints to animate poses based on detected body landmarks. However, I’m encountering difficulties aligning the character’s limbs with the detected landmarks.
Context:
Pose detection algorithms typically provide two key arrays:
- Pose Landmarks: An array containing the x, y, and z positions of all detected joints. Each entry corresponds to a specific landmark (e.g., shoulder, elbow).
-
Pose Connections: A 2D array specifying which landmarks are connected. For example,
[11, 13]in the connections array indicates that landmark 11 (left shoulder) and landmark 13 (left elbow) are connected.
I have the pose landmarks and connections, but I’m struggling to map these onto the Roblox character’s Motor6D joints. My math skills aren’t very strong, and I find it challenging to compute the necessary transformations and apply them correctly.
Questions:
- Understanding the Arrays: Can someone clarify how to interpret the pose landmarks and connections arrays effectively?
- Mapping to Motor6D: What is the best approach to convert these landmarks into angles and apply them to the Motor6D joints in Roblox? Any tips or examples would be greatly appreciated.
- Resources or Examples: If you have any resources, tutorials, or examples related to mapping poses from real-time pose detection algorithms onto a character, please share them!
I couldn’t find relevant discussions on the Dev Forum about this topic, and while I’ve tried using ChatGPT for help, it hasn’t fully resolved the issue.
Heres an example of how the arrays are structured for a better understanding
-- Example of how the arrays are structured:
local Pose_Landmarks = {[1]={x=0.245, y=0.654, z=0.876}, [2]=0.271, y=0.454, z=0.176}...}
--Signifies that indices 1 & 2 and indices 3 & 5 in Pose_Landmarks are connected
local Pose_Connections = {{1, 2}, {3, 5}...}