I can see the potential of this thing.
This project has definitely potential. Imagine animating using your webcam (and exporting to the animation editor), instead of having to do all the keypoints yourself 
Additionally, roleplay & social games may really benefit from this! It’s a very low-cost option to make motion tracking available to all.
Firstly, you’re mixing personal opinions into your statement. You find $300 expensive for a VR headset, that’s subjective.
Secondly, controllers are even more expensive. Thanks to this project, the webcam will be able to ‘replace’ them as free alternative.
Agreed, “motion tracking” would be more descriptive.
Do you have more than one webcam connected, or maybe OBS Studio installed? It could be using the webcam driver for that. If possible, could you show me the output from the console in the dark command prompt when you ran the program?
Hey, I took the latest files version on the github and when the main.py download all the files, at the end it download something in a loop, without skipping and its continue every time.
Here the output log I got every time :
Requirement already satisfied: mediapipe in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 1)) (0.8.10.1)
Requirement already satisfied: PyDirectInput in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 2)) (1.0.4)
Requirement already satisfied: flask in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 3)) (2.1.2)
Requirement already satisfied: waitress in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from -r requirements.txt (line 4)) (2.1.2)
Requirement already satisfied: matplotlib in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from mediapipe->-r requirements.txt (line 1)) (3.5.2)
Requirement already satisfied: protobuf<4,>=3.11 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from mediapipe->-r requirements.txt (line 1)) (3.20.1)
Requirement already satisfied: opencv-contrib-python in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from mediapipe->-r requirements.txt (line 1)) (4.6.0.66)
Requirement already satisfied: absl-py in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from mediapipe->-r requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: attrs>=19.1.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from mediapipe->-r requirements.txt (line 1)) (21.4.0)
Requirement already satisfied: numpy in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from mediapipe->-r requirements.txt (line 1)) (1.23.1)
Requirement already satisfied: itsdangerous>=2.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from flask->-r requirements.txt (line 3)) (2.1.2)
Requirement already satisfied: click>=8.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from flask->-r requirements.txt (line 3)) (8.1.3)
Requirement already satisfied: Jinja2>=3.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from flask->-r requirements.txt (line 3)) (3.1.2)
Requirement already satisfied: Werkzeug>=2.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from flask->-r requirements.txt (line 3)) (2.1.2)
Requirement already satisfied: colorama in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from click>=8.0->flask->-r requirements.txt (line 3)) (0.4.5)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from Jinja2>=3.0->flask->-r requirements.txt (line 3)) (2.1.1)
Requirement already satisfied: cycler>=0.10 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (0.11.0)
Requirement already satisfied: pyparsing>=2.2.1 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: pillow>=6.2.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (9.2.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (1.4.3)
Requirement already satisfied: packaging>=20.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from matplotlib->mediapipe->-r requirements.txt (line 1)) (4.34.4)
Requirement already satisfied: six>=1.5 in c:\users\pc\appdata\local\programs\python\python310\lib\site-packages (from python-dateutil>=2.7->matplotlib->mediapipe->-r requirements.txt (line 1)) (1.16.0)
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the 'C:\Users\pc\AppData\Local\Programs\Python\Python310\python.exe -m pip install --upgrade pip' command.
Missing 1 or more dependencies. No module named 'requests'
Downloading dependencies now.
This message appear every seconds.
(I tried updating pip and it doesn’t solve my problem)
Could you check if any of these commands work?
pip install requests
pip3 install requests
1 Like
Hey Certurix, did you fix the issue? This is very strange.
AR is virtual things being presented or projected into reality (or vice versa as seen in the OP)
By this logic VR headsets aren’t VR headests but AR headsets since they’re projecting your head motion data into the software (game etc), i do not think “AR” applies backwards, considering anything that isn’t our reality isn’t reality. Plus if we’re talking about the terms, then here’s the entry from the Oxford dictionary:
Augmented reality 1. a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view.
not backwards.
Headsets are expensive. Lowest price you can get in the US that’s actually reliable is $300.
If the price of a cheap vr is too big then what’s stopping you from using a cheap phone with a Gyroscope and virtual desktop + this for a cheap tracking+view solution?
It’s mainly clickbaity because it’s talking about “virtual reality without a VR device” which is the exact opposite of what the post is talking about.
I think you’ve mistaken what he’s trying to propose in this thread, you can use this to play vr games on roblox without a vr device (a tracker, such as htc vive trackers or kinect+kinect2vr)
I’m sorry that some of you disagree with the naming of this project. It was inspired by real Roblox virtual reality which is why I named it that way. If you don’t like how it is, then that’s okay, but you don’t need to argue because I’m not going to change the name. That’s my decision. Please keep this thread on-topic.
Hey sorry i’m in vacation and can’t fix the issue for now. I’ll tell you more once I came back from vacation!
This is a cool idea, but what is the average latency of it?
1 Like
Thats cool! I made something similar back in 2019 thru NodeJS and tensorflowjs but I was too lazy to pursue it.
TIP: tween the cframes for a smoother effect
Hmm…
It’s hard to see this as “VR”. You don’t see the game any differently, nor does it allow you to actually interact with the game. Mediapipe is very limited in terms of movement, so it’s not practical.
The best way to accomplish a full body tracking solution – would just be buying some SlimeVR trackers and modifying the server code to allow Roblox to fetch from it, and then doing things from there.
(By the way, I’ve never seen anyone use full body trackers on Roblox before, I’d like to see that.)
I don’t think there’s any way that you could allow for all degrees of freedom (without having four different webcams on each side of the room and a hacky solution, which isn’t practical for the common player) so this just isn’t very useful, nor practical.
How do you even move around while you’re using this? Do you have to have an XBox controller in your hand at all times? Or do you have to reach down for your keyboard just to walk around?
1 Like
It’s not VR it is just a system that converts body tracker obtained data from the video feed to cframes. This is just a cool open-sourced creation and it is not intended for use for many reasons, the main one being that this is against the TOS.
Then why is it called “Virtual Reality”? It seriously seems
and I’m clearly not the only person who believes that, too.
probably just misworded by OP, it is self-explanatory when you read what the post is about it is really not that deep
1 Like
It is tweened so it doesn’t appear laggy.
Less than 1 second reaction time. Latency depends on the specs of the third-party server. By default the Python script updates the pose every 0.3 seconds but the animation appears smooth because it’s tweened.
1 Like
You’ve inspired me to make a project of my own.
I was thinking of using MediaPipe in conjunction with Godot to make a VR style ping pong game.
Im using the Godot engine for this project.
Im thinking of using ur code in order to make this work but thats me being lazy. Maybe you could spend some time making a tutorial on how you made it?
1 Like
There is no shame in using anyone’s code from open-source project. The point is for people to learn from others’ projects by reading their code, and making your own creations.
If you read the pose estimation section of the mediapipe documentation, there’s a pretty good explanation on how to use this with Python (it’s pretty simple to setup).
Here’s an example of a game I made with Pygame and Mediapipe:
1 Like
You still working in this project?