An App from MIT Can Program Everything Around You

Photo courtesy of Valentin Heun

Imagine you’re just about to go to sleep when you realize you’ve left the light on. The switch is on the other side of the room, so instead of getting out of bed, you close the drawer on your nightstand and in turn, the light is switched off.

Programming a drawer to shut off a light isn’t all that far-fetched. Using objects to control other objects is the idea behind the Reality Editor, a project led by Valentin Heun at MIT’s Fluid Interfaces Group. Through a mobile app, users can “edit reality” by drawing virtual lines to and from objects with their smartphone’s camera to alter their uses and capabilities. Heun compares his technology to a screwdriver—a digital tool for fixing things around you.

From turning on a lamp with an item on your desk to creating an alternative way to open your car windows, the Reality Editor has seemingly endless possibilities. They’ve yet to be discovered in part because the technology is a work in progress.

“It’s an early version research stage,” says Heun about the app, available in the App Store. “There will be many, many more iterations coming. I put it out so people can start using it.”

Heun says he wants to reach three groups of people there: consumers, designers, and programmers. The use of the app itself seems complex for the everyday user, but he hopes that by keeping its code available on an open source platform called Open Hybrid, the latter two groups can build upon and adapt his technology. And much like a screwdriver is only useful when there is a screw to tighten, the Reality Editor’s potential stems from loose screws identified by these designers and programmers.

The app uses the same method for building experiences in web browsers to build its interface. It also requires the use of QR code-like stickers called FingerPrints, so a device can recognize patterns between objects. Once a virtual connection is made between two objects, it stays connected, so users don’t have to pull out their phones to operate their objects each time.

Heun’s first inklings of the Reality Editor arose when he was creating 3D animations on his computer. He realized he could make clay models with his hands much faster than virtual models, and sought to find a better way to connect humans and the digital world. The Reality Editor is a starting point for this—a way to give users more control in an environment filled with smart objects.

“An internet-connected object [think: Fitbit or Nest] talks to its maker, and you as its user are outside of the conversation. You are not in charge of it,” says Heun. “The Reality Editor is basically a tool that allows you to take part in that conversation, and become the strongest point in that conversation.”