Embodied interactions for spatial design ideation: Symbolic, geometric, and tangible approaches

FNU Vinayak, Purdue University

Abstract

Computer interfaces are evolving from mere aids for number crunching into active partners in creative processes such as art and design. This is, to a great extent, the result of mass availability of new interaction technology such as depth sensing, sensor integration in mobile devices, and increasing computational power. We are now witnessing the emergence of maker culture that can elevate art and design beyond the purview of enterprises and professionals such as trained engineers and artists. Materializing this transformation is not trivial; everyone has ideas but only a select few can bring them to reality. The challenge is the recognition and the subsequent interpretation of human actions into design intent. Taking inspiration and guidance from embodied interactions, our focus in this work is to design, develop, and evaluate interfaces and interaction techniques to support idea generation for the design of three-dimensional (3D) shapes. Grounded in the phenomenological tradition of philosophy, embodied interactions represent a general framework to understand the transformation of action into meaning - in our case, design intent. We present concrete instances of this framework by investigating symbolic, geometric, and tangible approaches for transforming human action into design intent. We begin with the symbolic transformation of action to intent by introducing shape-gesture-context interplay (SGC-I), a framework that enables the expression of design intent in the virtual environment through a prescribed set of hand gestures combined with arm motion. Here, the main idea is to automatically deduce the nature and extent of geometric constraints by interpreting human gestures and motions in the context of a given modeling operation . We explore the geometric transformation of action to intent by introducing proximal attraction, a dwell-time approach that interprets a shape deformation as a progressive convergence of the shape of the manipulated object (such as a lump of clay) to that of the manipulating object (such as a hand or a tool). We demonstrate our approach using a modeling metaphor inspired by pottery, which offers an unambiguous interaction context via a well-defined and intuitive relationship between the use of hands and the shaping of pots. We implement and evaluate a novel algorithm that uses kernel-density estimation to characterize the contact between the hand and a 3D shape. In contrast to the symbolic nature of SGC-I, the geometric approach does not compute any finite set of gestures or hand skeleton. Instead, it implicitly extracts the grasp and motion from the raw point-cloud (PCL) of the user's hand for deforming the shape of a pot in 3D space. This feature of our method directly allows a user to shape pots by using physical artifacts as tools adding tangibility to the process of shape ideation. In the final stage of this work, we introduce a novel interaction metaphor for using a hand-held smartphone as a medium for spatial design ideation. The core goal here is to enable direct externalization of spatial design concepts by embedding the geometric representation of the artifact within the physicality and tangibility of the creation process itself. To achieve this goal, we re-purpose a smartphone as a hand-held reference plane for creating, modifying, and manipulating 3D sweep surfaces. We implement MobiSweep , a prototype application to explore a new design space of constrained spatial interactions that combine direct orientation control with indirect position control via well-established multi-touch gestures. MobiSweep leverages kinesthetically aware interactions for the creation of a sweep surface by utilizing the spatial relationship between the physical action of sweeping and the creation of the resulting swept surface. The design concepts generated by users, in conjunction with their feedback, demonstrate the potential of such interactions in enabling spatial ideation.

Degree

Ph.D.

Advisors

Ramani, Purdue University.

Subject Area

Mechanical engineering|Computer science

COinS