I am hoping to finish the live webcam body tracking tomorrow and open source the code. Right now the way it works is very simple, you upload an image and once you upload it and press “Send” tensorflowjs will find the body parts and give coordinations of them. Then I get those coordinations and convert it in a JSON list. In a script in ROBLOX studio I fetch all of the coordinations and create lines between the points accordingly to the body parts. Opinions?
9 Likes
The tracker is amazing! Even though I personally don’t know what I could do with this right now, one thing that did come to my mind is if these points are able to be translated into say CFrame Positions.
2 Likes
An awesome start, so many possibilities with this. Please keep me updated on any progress you make!
1 Like
You can do this 3D too but I found 2D easier to deal with. The tracker already provides you with 2 coordinates (x and y), you might as well add a 3rd one yourself depending on how high up you want it to show.
1 Like
This looks like it has amazing potential, one thing that immediately came to mind for me was VR full body tracking 
1 Like
This is pretty cool, but the issue with it that I can see is that Roblox will never let you access a users web cam. This means that the player would have to install an external program.
Using this for VR would be very efficient because as soon as any body part goes out of view it’d stop working.
Most VR solutions that use external sensors let you buy extra sensors for full body tracking, however, I don’t know if roblox supports them yet.