Tactual transmission of phonetic features

Ali Israr, Purdue University

Abstract

A human's ability to transmit speech through touch has been demonstrated by a natural communication method, the Tadoma method, in which a deaf-and-blind individual places the hands on the speaker's face, and integrates movements, vibrations and air flows from the articulation process for speech perception. In contrast, the achievable information transmission rates with tactile hearing aids have been limited, largely due to the homogeneous nature of the stimulation pattern. It is the goal of this research to broaden the range of tactual stimuli to achieve more effective tactual speech communication. A multi-finger tactual display was developed that is capable of stimulating both the kinesthetic and the cutaneous components of the somatosensory system. A novel two-degree-of-freedom (2-dof) controller consisting of a feedback controller and a prefilter was developed to preserve the relative spectral intensities present in a speech signal in terms of the perceived intensities once the signal has been delivered to a human hand. To gain a better understanding of the masking effects of presenting multiple tactual signals simultaneously to the hand, frequency and amplitude discrimination thresholds were estimated for six target frequencies at two amplitude levels in the absence and presence of masking stimuli. An important finding is that the low-frequency large-amplitude motions and high-frequency small-amplitude vibrations do not interfere significantly. They are later used for redundant coding of speech information. The speech-to-touch coding scheme extracts acoustic features from the speech and presents them as multidimensional tactual waveforms, stimulating both kinesthetic and cutaneous sensory systems. Evaluation of our speech communication system is conducted by measuring performance in both phoneme discrimination and identification tasks. Normal-hearing participants can effectively distinguish between two consonants that differ in voicing (94%), place (88%) or manner of articulation (93%). Performance in a vowel identification task is slightly superior to that reported in prior studies (PC = 42.6%; IT=1.16 bits post training). Training effects are significant in both discrimination and identification tasks. Our results show that by delivering a rich set of multidimensional tactual stimulation, minimizing masking effects, and redundant coding of speech information, we can achieve performance levels that are similar to or more superior than those in the literature.

Degree

Ph.D.

Advisors

Tan, Purdue University.

Subject Area

Mechanical engineering

COinS