delivering electrotactile stimuli to the dorsum of the tongue via a flexible electrode array placed in the mouth. this electrode array is connected to a tongue display unit via a ribbon cable passing out of the mouth. a video camera records a picture, transfers it to the tdu for conversion into a tactile image. the tactile image is then projected onto the tongue via the ribbon cable where the tongue's receptors pick up the signal. after training, subjects are able to associate certain types of stimuli to certain types of visual images. in this way, tactile sensation can be used for visual perception. sensory substitutions have also been successful with the emergence of wearable haptic actuators like vibrotactile motors, solenoids, peltier diodes, etc. at the center for cognitive ubiquitous computing at arizona state university, researchers have developed technologies that enable people who are blind to perceive social situational information using wearable vibrotactile belts (haptic belt) and gloves (vibroglove). both technologies use miniature cameras that are mounted on a pair of glasses worn by the user who is blind. the haptic belt provides vibrations that convey the direction and distance at which a person is standing in front of a user, while the vibroglove uses spatio-temporal mapping of vibration patterns to convey facial expressions of the interaction partner. alternatively, it has been shown that even very simple cues indicating the presence or absence of obstacles (through small vibration modules located at strategic places in the body) can be useful for navigation, gait stabilization and reduced anxiety when evolving in an unknown space. this approach, called the "haptic radar" has been studied since 2005 by researchers at the university of tokyo in collaboration with the university of rio de janeiro. similar products include the eyeronman vest and belt, and the forehead retina system. tactile–auditory neuroscientist david eagleman presented a new device for sound-to-touch hearing at ted in 2015; his laboratory research then expanded into a company based in palo alto, california, called neosensory. neosensory devices capture sound and turn them into high-dimensional patterns of touch on the skin. experiments by schurmann et al. show that tactile senses can activate the human auditory cortex. currently vibrotactile stimuli can be used to facilitate hearing in normal and hearing-impaired people. to test for the auditory areas activated by touch, schurmann et al. tested subjects while stimulating their fingers and palms with vibration bursts and their fingertips with tactile pressure. they found that tactile stimulation of the fingers lead to activation of the auditory belt area, which suggests that there is a relationship between audition and tactition. therefore, future research can be done to investigate the likelihood of a tactile–auditory sensory substitution system. one promising invention is the 'sense organs synthesizer' which aims at delivering a normal hearing range of nine octaves via 216 electrodes to sequential touch nerve zones, next to the spine. tactile–vestibular some people with balance disorders or adverse reactions to antibiotics develop bilateral vestibular damage (bvd). they experience difficulty maintaining posture, unstable gait, and oscillopsia. tyler et al. studied the restitution of postural control through a tactile for vestibular sensory substitution. because bvd patients cannot integrate visual and tactile cues, they have a lot of difficulty standing. using a head-mounted accelerometer and a brain–computer interface that employs electrotactile stimulation on the tongue, information about head-body orientation was relayed to the patient so that a new source of data is available to orient themselves and maintain good posture. tactile–tactile to restore peripheral sensation touch to touch sensory substitution is where information from touch receptors of one region can be used to perceive touch in another. for example, in one experiment by bach-y-rita, the touch perception was restored in a patient who lost peripheral sensation from leprosy. for example, this leprosy patient was equipped with a glove containing artificial contact sensors coupled to skin sensory receptors on the forehead (which was stimulated). after training and acclimation, the patient was able to experience data from the glove as if it was originating in the fingertips while ignoring the sensations in the forehead. after two days of training one of the leprosy subjects reported "the wonderful sensation of touching his wife, which he had been unable to experience for 20 years." tactile feedback system for prosthetic limbs the development of new technologies has now made it plausible to provide patients with prosthetic arms with tactile and kinesthetic sensibilities. while this is not purely a sensory substitution system, it uses the same principles to restore perception of senses. some tactile feedback methods of restoring a perception of touch to amputees would be direct or micro stimulation of the tactile nerve afferents. other applications of sensory substitution systems can be seen in function robotic prostheses for patients with high level quadriplegia. these robotic arms have several mechanisms of slip detection, vibration and texture detection that they relay to the patient through feedback. after more research and development, the information from these arms can be used by patients to perceive that they are holding and manipulating objects while their robotic arm actually accomplishes the task. auditory systems auditory sensory substitution systems like the tactile sensory substitution systems aim to use one sensory modality to compensate for the lack of another in order to gain a perception of one that is lacking. with auditory sensory substitution, visual or tactile sensors detect and store information about the external environment. this information is then transformed by interfaces into sound. most systems are auditory-vision substitutions aimed at using the sense of hearing to convey visual information to the blind. the voice auditory display "the voice" converts live camera views from a video camera into soundscapes, patterns of scores of different tones at different volumes and pitches emitted simultaneously. the technology of the voice was invented in the 1990s by peter meijer and uses general video to audio mapping by associating height to pitch and brightness with loudness in a left-to-right scan of any video frame. eyemusic the eyemusic user wears a miniature camera connected to a small computer (or smartphone) and stereo headphones. the images are converted into "soundscapes". the high locations on the image are projected as high-pitched musical notes on a pentatonic scale, and low vertical locations as low-pitched musical notes. the eyemusic conveys color information by using different musical instruments for each of the following five colors: white, blue, red, green, yellow. the eyemusic employs an intermediate resolution of 30×50 pixels. libreaudioview this project, presented in 2015, proposes a new versatile mobile device and a sonification method specifically designed to the pedestrian locomotion of the visually impaired. it sonifies in real-time spatial information from a video stream acquired at a standard frame rate. the device is composed of a miniature camera integrated into a glasses frame which is connected to a battery-powered minicomputer worn around the neck with a strap. the audio signal is transmitted to the user via running headphones. this system has two operating modes. with the first mode, when the user is static, only the edges of the moving objects are sonified. with the second mode, when the user is moving, the edges of both static and moving objects are sonified. thus, the video stream is simplified by extracting only the edges of objects that can become dangerous obstacles. the system enables the localization of moving objects, the estimation of trajectories, and the detection of approaching objects. psva another successful visual-to-auditory sensory substitution device is the prosthesis substituting vision for audition (psva). this system utilizes a head-mounted tv camera that allows real-time, online translation of visual patterns into sound. while the patient moves around, the device captures visual frames at a high frequency and generates the corresponding complex sounds that allow recognition. visual stimuli are transduced into auditory stimuli with the use of a system that uses pixel to frequency relationship and couples a rough model of the human retina with an inverse model of the cochlea. the vibe the sound produced by this software is a mixture of sinusoidal sounds produced by virtual "sources", corresponding each to a "receptive field" in the image. each receptive field is a set of localized pixels. the sound's amplitude is determined by the mean luminosity of the pixels of the corresponding receptive field. the frequency and the inter-aural disparity are determined by the center of gravity of the co-ordinates of the receptive field's pixels in the image (see "there is something out there: distal attribution in sensory substitution, twenty years later"; auvray m., hanneton s., lenay c., o'regan k. journal of integrative neuroscience 4 (2005) 505–21). the vibe is an open source project hosted by sourceforge. other systems other approaches to the substitution of hearing for vision use binaural directional cues, much as natural human echolocation does. an example of the latter approach is the "seehear" chip from caltech. other visual-auditory substitution devices deviate from the voice's greyscale mapping of images. zach capalbo's kromophone uses a basic color spectrum correlating to different sounds and timbres to give users perceptual information beyond the voice's capabilities. nervous system implants by means of stimulating electrodes implanted into the human nervous system, it is possible to apply current pulses to be learned and reliably recognized by the recipient. it has been shown successfully in experimentation, by kevin warwick, that signals can be employed from force/touch indicators on a robot hand as a means of communication. terminology it has been argued that the term "substitution" is misleading, as it is merely an "addition" or "supplementation" not a substitution of a sensory modality. sensory augmentation building upon the research conducted on sensory substitution, investigations into the possibility of augmenting the body's sensory apparatus are now beginning. the intention is to extend the body's ability to sense aspects of the environment that are not normally perceivable by the body in its natural state. moreover, such new informations about the environment could be used not to directly replace a sensory organ but to offer a sensory information usually perceived via another, potentially harmed, sensory modality. thus, also sensory augmentation is widely used for rehabilitation purposes as well as for investigating perceptive and cognitive neuroscience active work in this direction is being conducted by, among others, the e-sense project of the open university and edinburgh university, the feelspace project of the university of osnabrück, and the hearspace project at university of paris. the findings of research into sensory augmentation (as well as sensory substitution in general) that investigate the emergence of perceptual experience (qualia) from the activity of neurons have implications for the understanding of consciousness.