Abstract
Tactile sensors are crucial in providing contact geometry information necessary for object manipulation. However, it remains highly nontrivial for a tactile sensor to be able to perceive a distant object before manipulation. In this work, we present an innovative robotic finger, VisTac, which seamlessly combines high-resolution tactile and visual perception in a single unified device, while adhering to essential mechanical constraints, such as a human-finger-like wedge-shaped top, vital for manipulation tasks. Furthermore, we discuss the fabrication of a key component in the device; the semitransparent membrane with light-dependent opacity, thus enabling the contact surface to effortlessly transition between visual and tactile modes. With a compact, two-camera configuration, we demonstrate the sensor’s multimodal sensing capabilities through 3-D reconstruction of the tactile imprints, distant object localization via vision, and pose estimation using both visual and tactile feedback. VisTac is capable of delivering the vision and touch capabilities to accurately locate an object in the 3-D space, using vision, and then adeptly manipulate it according to the task at hand, relying on tactile feedback. To demonstrate the remarkable abilities of the sensor, we carry out a peg-in-hole insertion task. This work primarily seeks to pave the way for future research toward developing unified visual–tactile sensors.
Keywords
Sensors; Visualization; Tactile sensors; Robots; Cameras; Task analysis; Robot vision systems; Manipulation; optical tactile sensor; robotics; tactile sensing; vision-based tactile sensor
Date of this Version
10-15-2023
Recommended Citation
Athar, Sheeraz; Patel, Gaurav; Xu, Zhengtong; Qiu, Qiang; and She, Yu, "VisTac Toward a Unified Multimodal Sensing Finger for Robotic Manipulation" (2023). School of Industrial Engineering Faculty Publications. Paper 16.
https://docs.lib.purdue.edu/iepubs/16
Comments
This is the author-accepted manuscript of S. Athar, G. Patel, Z. Xu, Q. Qiu and Y. She, "VisTac Toward a Unified Multimodal Sensing Finger for Robotic Manipulation," in IEEE Sensors Journal, vol. 23, no. 20, pp. 25440-25450, 15 Oct.15, 2023. (c) 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. The version of record is available at DOI: 10.1109/JSEN.2023.3310918.