What If Books Physically Replicated the Protagonist’s Emotions?

Sensory Fiction photo via Felix Heibeck

Imagine that as you’re reading the scene from Stephen King’s The Shining, where the unwanted inhabitant of Room 217 slowly closes her hands around young Danny Torrance’s neck, a vest strapped to your body syncs up with the frightening encounter and mimics the actions happening in the book, bringing the novel to life.

With a new technology that relies on sensors, developed by researchers at MIT’s Media Lab last semester, that sort of interactive reading experience isn’t far off.

Felix Heibeck, Alexis Hope, and Julie Legault are behind what they dubbed the “Sensory Fiction” book, which uses lights that change color, vibrations, and a wearable vest affixed to the reader that reacts to certain scenes in the literature, based on the mood to enhance the reading experience.

“Traditionally, fiction creates and induces emotions and empathy through words and images,” explain the creators of the project. “The ‘augmented’ book portrays the scenery and sets the mood, and the wearable [technology] allows the reader to experience the protagonist’s physiological emotions.”

In September, instructors Dan Novy and Sophia Brueckner launched “Science Fiction to Science Fabrication,” a course that welded together what Sci-Fi authors envisioned for the future, with today’s high-tech capabilities. In doing that, students and researchers in the class built “code-based interpretations” derived from technologies the authors described in their writing. The purpose of the course was to dissect some of the most thought-provoking classic and modern science fiction literature out there, and “focus on the creation of functional prototypes.”

“We felt it was important to show students at the Media Lab, and MIT in general, that there was a lot of far reaching and critical thinking about the effects of technology being explored in the genre of Science Fiction. Here at the Lab we are literally creating the technologies of tomorrow but very few students ever give a thought to the continuum their technology exists within, or the far extended ethical implications their technology may have,” the course instructors said in an email. “Science Fiction often acts as cautionary tales…so one of the things we tried to do was also take technologies that were presented as dystopian or malevolent and spin them toward a positive outcome. This is the exact opposite of how evil technologies are usually presented in fiction.”

As part of the course curriculum, there were regular, weekly talks about the literature covered throughout the semester, as well as a mandatory final project. The Sensory Fiction project, inspired by a book read for the class, was what Heibeck, Hope, and Legault came up with.

To make it, the trio relied on networked sensors and actuators to create a “new means of conveying plot, mood, and emotion” through physical experiences “… while still allowing space for the reader’s imagination.” They used the novella The Girl Who Was Plugged In by James Tiptree Jr. as the backbone for their creation.

As the book’s protagonist experiences emotional or physical changes from page to page, the vest triggers the actuators, and can cause shifts in the reader’s heart beat and temperature. The vest can also inflate using pressurized airbags, and constrict to create a feeling of anxiety, or pressure.

“Sensory fiction is about new ways of experiencing and creating stories,” researchers said.

The lights on the outside of the book’s cover also change as the overall theme of the protagonist’s mood fluctuates. “The book cover animates to reflect the book’s changing atmosphere, while certain passages trigger vibration patterns,” the team wrote.

While this sort of technology could be sourced out to a range of different novels, taking the reading experience to a level that would make the story come alive, it’s not likely it will hit the market anytime soon.

In an email, one of the creators of the Sensory Fiction book said the project is merely a prototype meant to provoke discussion. “While the project explores new ways of reading with digital augmentations, this is not a product idea but rather an exploration in the context of Science Fiction stories,” said Heibeck.

One of the other prototypes that came out of the class included bionic arms similar to the one Luke Skywalker lost in his battle against Darth Vader.

A video by Heibeck and his team was put together at the end of the semester, and can be seen below:

SENSORY FICTION from Felix on Vimeo.