Spatial Augmented Reality (SAR) uses projectors to add computer generated images into the real world. For SAR systems to be useful to people, new methods of interacting with the computer need to be developed. In an immersive SAR environment, the user is no longer sitting at a desk; a keyboard and mouse are nolonger usable as input devices. This research is all about finding new and better ways to interact with Spatial Augmented Reality.
Physical / Virtual Tools
Physical tools are a compelling interaction method for interaction. SAR requires physical surfaces to project onto, so real objects are there anyway. By using physical tools, users get passive haptic feedback. More importantly, the tool becomes another surface in the environment, and can also have graphics projected onto it. We can overload the functionality of tools by changing the projection, meaning the system needs fewer devices in order to be usable. Below is a simple airbrushing application that demonstrates this concept.
Tools for Industrial Designers
Industrial Design is a domain well suited to SAR tools and interfaces. The key will be to develop tools that fit in with existing workflows and practices.
One example application is Augmented Foam Sculpting. Currently, industrial designers create foam mockups of their designs to test different design concepts. From there, they create CAD models and further refine their designs. Augmented Foam Sculpting combines these two steps. As a designer sculpts the foam, the system records their actions and produces a matching 3D model that can be further refined in a CAD system. This is compelling, as it uses the sculpting skills industrial designers already have, and fits in with current processes. The video below shows Augmented Foam Sculpting in action.