MIT Developing Devices To Help The Blind See

Great dome of MIT image via shutterstock

Robots have always been a symbol for the future, and it looks like Boston is ready for that future sooner rather than later. From computers creating avatars for physical therapy to robot hands restoring thumb function in children, the Boston medical research world is making computerized futuristic technology useful for everyday applications. And for researchers at MIT and Northeastern University, there is no goal out of reach for technology, not even giving sight to the blind.

The collaborative research project is called The Fifth Sense Project, and it aims to bring sight to blind people through computers. These computerized devices can be worn on a jacket or as part of a suit, and can non-visually transmit information about upcoming trip hazards, approaching people, and important signage to the blind person who is wearing it.

The project has four main objectives: to provide safe mobility and orientation, dynamic social context, the decoding of text and symbols, and timely information delivery.” When a blind person is traveling, they are often at the mercy of others,” says David Hayden, an MIT PhD student who works on the project. “Others have to reach out, or the person has to wait until they hear a voice they recognize to interact with them. They miss out on serendipitous meetings.”

The system is comprised of a camera, a microphone, and motion sensors called accelerometers. The computer uses the camera and microphone to identify approaching acquaintances by voice, face, and context (who is likely to show up at a certain destination), and it conveys this information via personal, privatized audio, or a vibration on the wrist through a type of Morse code.

David Hayden wearing the Fifth Sense Project’s wearable computer jacket. Image provided

The code can only convey 20 words per minute as opposed to 200 via audio, so Northeastern University researchers are working to develop a tactile display. Braille and forms of letters can convey more detailed information than vibrations or audio, Hayden says, like specific obstacles in an environment or the contours of an acquaintance’s face. Ideally, this device would be small enough for the wearer to put in their pocket or hold in their hand. When information is being conveyed, the wearer could casually run a hand over the device and read what is happening around them.

The other parts of the project are focusing on developing technologies that can also spot text and signage and trip hazards. This will help navigate tricky places like airports and busy streets.

Recently, MIT played host to the Challenges Workshop where the Fifth Sense Project was highlighted and Andrea Bocelli spoke to the crowd. The famous opera singer’s foundation is a major funder for the project.

“The workshop was great,” Hayden says. “We discussed a lot of things about assisted technology related to low vision. We talked about everything from retinal prosthetics to video games based on familiar environments that blind people can use to memorize the doctor’s office or their classroom for easier navigation later on.”

Andrea Bocelli, who is blind, has a personal connection to the work the Fifth Sense Project is doing, and so does Hayden. “I’m legally blind,” Hayden says, “which means I have sufficient vision to not run into obstacles, but not enough to recognize people who are walking by and not enough to drive a car.” Determined not to let his disability get in the way of his life, Hayden founded Essistive Technologies in 2011 which produces the NoteTaker, a device of Hayden’s own design which allows blind and low-vision people to simultaneously view lectures and take notes in class. Hayden continues to use the NoteTaker in his PhD studies at MIT.

The Fifth Sense Project will have a similar beneficial outcome for Hayden as the NoteTaker. “It’s nice to work on a project that can help a large population and also have a direct application in your own life. It’s very rewarding for me.”

Hayden says that the Fifth Sense Project is strictly an academic research project at the moment, but in as little as five years, these wearable devices could be available commercially.