Researchers at the MIT Media Lab Tangible Media Group have designed a new 'interaction system' to make smartphones and PC UIs work together in a 'seamless' fashion. THAW allows "hybrid interactions" with smartphones near your computer screen. So basically by hovering your smartphone near your computer screen you can interact with what's going on on the big screen.
The video starts by showing some desktop UI file management interactions. Using a smartphone near the computer screen a user employs drag drop techniques to pick up files and folders to deposit elsewhere on the screen. It goes further than being a simple pasteboard though and we are told the system offers more advanced clipboard features. In another example showing the user 'picks up' a webpage from the big screen and carries it off on his smartphone. One other interesting use of the smartphone was as a slider to adjust photo effects on the PC screen.
The next stage of the THAW demonstration takes us into gaming. This part of the video shows a lot of imagination in smartphone/PC interaction but while the demo is games based some of the techniques could easily overlap into creative content making uses. In the 'game' the smartphone is used as:
- A viewport revealing hidden game elements or information
- The smart phone can act like a screen object - a barrier in a game, a straight edge or a containment box
- It can alter some screen behaviours when near that part of the screen
- The smartphone touchscreen works upon game elements and you can scale the character, for example, and send it back to PC or transport the character to a different part of the screen
The video ends back with more content creation type use cases. We are told the built-in sensors in the smartphone can help us manipulate objects and views - in 3D programs for example. This looked both quick and intuitive.
We are told that the techniques shown in the video don't require any extra hardware, just your smartphone and computer and of course - the THAW software special sauce...
