NASA’s Jet Propulsion Lab team is testing a combination of the Xbox One Kinect sensor and the Oculus Rift headset to control robots:
Using the new Xbox One Kinect sensor, we are able to manipulate the JACO robot arm in real time. By combining position tracking from the Kinect and rotational tracking with the Oculus, we provide a first-person view for the operator. Future work will include integrating sensor array data into the scene and translating our research to the Robonaut 2 humanoid on the International Space Station.
From the research side of things, this is a great proof-of-concept to show another application of consumer technology in a research environment. This sort of thing could be applied anywhere humans shouldn’t be going but autonomous or controller-operated robots aren’t quite nimble enough to work. It could also decrease the need extra-vehicular activities during space missions or even future landings on the Moon and Mars.