Interactive Sketching Framework for Static and Dynamic 3D Content Creation

Gen Nishida, Purdue University

Abstract

In this dissertation, we propose a sketching framework for static and dynamic 3D content creation that uses a digital sketching interface which allows the user to intuitively design the 3D model and automatically infers the modeling parameters from the user sketch to generate the output 3D model. Creating a 3D digital model from a 2D sketch is notoriously difficult due to the lack of depth in the 2D sketch. For the static content, our approach uses the combination of deep learning and procedural modeling techniques to tackle this problem. Deep learning techniques quickly recognize the shape of the user sketch and infer the modeling parameters while procedural modeling works as a regularizer to reduce the search space and guarantee the validity of the output 3D model. While we focus on 3D building in this dissertation, our approach can be applied to other types of contents. Also, we extend this approach to 3D modeling from a single photograph. The user still has full controllability by specifying the silhouette of the target building in the photograph while reducing the user interaction to create a desired 3D model. For dynamic content creation, our approach uses a combination of mechanical engineering and machine learning techniques. The user draws a few rigid bodies and their motions. Then, the mechanical engineering techniques quickly find initial candidate mechanisms that achieve the desired motion without any collision, and the machine learning technique refines the solutions and finds the best one in terms of our soft-constraints including the accuracy and smoothness of the trajectory, and the location and size of the linkage. Our results and qualitative and quantitative evaluations show the effectiveness of our framework.

Degree

Ph.D.

Advisors

Aliaga, Purdue University.

Subject Area

Computer science

COinS