Microsoft Research has published two new videos demonstrating the science behind the new Kinect sensor, which is to be bundled with the Xbox One. Microsoft's Scott Evans and Kareem Choudhry are the hosts, taking it in turns being the presenter and subject of the videos. ‘Inside the brains’ and ‘Behind the eyes’ show you what the new Kinect camera can see and how that data is used to form an understanding of the human subject’s body and movement.
‘Behind the eyes’
We are told that, thanks to the IR vision of the new Kinect sensor, it has “three times the fidelity of 3D vision we had in the first generation sensor.” Details such as individual fingers and wrinkles in clothes can be detected by the sensor. The field of view (FOV) is also increased so users can stand a lot closer to the sensor than with the old version – people can enjoy using this camera within modestly sized houses and apartments. Also the scene of your room taken into the Kinect is also much wider.
Active IR works in dim and low light situations to show the Kinect user in excellent detail. Another benefit is that if there are disruptive lighting sources in or coming into the room, they can be almost completely ignored by the new sensor. This facility should improve reliability of motion detection.
The researchers also point out that the colour feed from the camera is now 1080p which is good for uses such as Skype.
‘Inside the brains’
We are told that “The first version of Kinect was really about understanding gross human motion; we have true human understanding here and fidelity of expression.” The video shows Microsoft’s software working out the skeleton position and motion of the subject in front of the camera. Now hands are more detailed, its is possible to use them for manipulation and not just as a pair of shovels.
The Kinect software can add to the skeleton to make ‘block man’, understanding the block orientation of the body quite accurately. Furthermore ‘muscle man’ knows where forces are being applied to the human body, it senses the degree of force on each part of the body, indicated by various colours in the video above. This translates into screen actions like throwing a punch, balancing on one leg and so on – these kinds of aspects can be utilised by games and fitness apps and more.
Another impressive feat is that the Kinect can take the HD colour feed and IR feed combination to read the user’s pulse/heart rate from their face. All in all it will be interesting to see how these capabilities are applied to entertainment software and if they are much better than those of the PlayStation camera.
