Other Users Who Read This Article Also Enjoyed. . .
Companies like Netflix and Amazon.com have no problem telling you what someone else likes. Cambridge firm ChoiceStream is working to figure out what you want—even before you do.
If you’ll indulge me a minute, I’ve got a few recommendations for you. There’s this new album by the Fratellis, a glam-rock band from Scotland—it kicks ass. You should also grab Jim Harrison’s novel A Good Day to Die, about a violent, drug-fueled quest to blow up a dam. I never miss NBC’s Studio 60, starring the goofy guy from Friends and the smarmy guy from The West Wing, and neither should you. And you’ve got to preorder the DVD of Casino Royale and bookmark treehugger.com.
Everywhere you turn these days, from online stores to the menu at the corner restaurant, someone—or something—wants to make a recommendation. But I’m going to stop now. Because, really, like everyone else doing it, I have no idea if my picks are right for you. Maybe you’re an opera lover, in which case the Fratellis would sound like an anvil chorus. If you’re a Jane Austen fanatic, Jim Harrison’s sensibilities won’t make much sense. Maybe you hate the goofy guy from Friends. And the smarmy guy from The West Wing. And James Bond. And bleeding-heart save-the-whale types.
My recommendations are useless because I don’t know anything about you. And the world today is all about you. Your specific You. The plural We of “We Are the World” has been replaced by iThis and MyThat, YouTube and movies-on-demand. As you’re no doubt aware, You were Time’s Person of the Year. (Congratulations.)
Now that you’re in control, the question is, What are you in control of? There’s an infinite amount of stuff out there—movies, TV shows, music, books, blogs, podcasts, video games—and thrashing around to find what you need or want or just might love would take too much time. Time none of us has anymore. If someone—or, again, something—could make sense of it all, act as a kind of filter, it might just feel like divine intervention.
Behold the information age’s latest Holy Grail: the Better Recommender. Because, frankly, Google search results are too random. Amazon.com’s “Customers who bought this item also bought…” doesn’t cut it anymore. TiVo, which automatically records TV shows it thinks you’ll like, is even worse. And don’t get me started on e-dating sites.
But Steve Johnson, CEO of the Cambridge tech firm ChoiceStream, believes he has the solution—so much so, that he doesn’t even think in terms of recommendation. Instead Johnson sees personalization, and he sees it on a previously unimagined scale. On the Web, on TV, on the phone, his company watches you to infer your likes and dislikes. Then it matches these to the qualities of the content itself, skipping all that “other users” crap to figure out what you want even before you do. Connect the dots, and voilà: Here’s the movie you should watch, the music you should download, the digital camera you should buy.
At least that’s how it’s supposed to work. Problem is, you’re not hard-wired. You shift and develop, bombarded by a million life vectors that continuously redefine you. Can a computer keep up? Can it ever really understand you? And more importantly, do you want it to?
Back in 2000, when Steve Johnson cofounded ChoiceStream, recommendation software wasn’t quite cutting-edge stuff (the first examples were incubated at the MIT Media Lab in the early ’90s); in practice, it was little more than a gimmick. Seven years later, recommenders are now ubiquitous, but still drastically flawed—and the companies that use them know it. Last October, online DVD-rental company Netflix put a spotlight on the problem when it announced the Netflix Prize: $1 million to anyone who could improve the accuracy of its Cinematch system by 10 percent.
“Yeah, we’re already past that,” says Johnson, when I sit down with him in the conference room at ChoiceStream’s Kendall Square offices. Dressed casually and always on point, he carries himself with the assurance of someone who knows he’s on to something big. And he is. Consumer technology is on the verge of integrating like never before. Television systems like DirecTV and Comcast are preparing for the coming Internet TV explosion, which will increase your current 400 channels to 4,000—or even 40,000. Mobile phones, such as Apple’s shiny new iPhone, aren’t really phones anymore, but mini computers that can interact with your desktop and TV. Soon you won’t be able to escape being “connected” even if you wanted to.
“The convergence is finally happening,” Johnson says, “the convergence of multiple devices so that consumers have access not only to a nearly unlimited amount of content, but also from pretty much anywhere they want it. There’s so much content available that it’s not enough to expect consumers to know what they’re looking for.”
In other words, the content soup is getting thicker. Exponentially so. Which is just fine with Johnson, who’s always relished having a hill to climb. In fact, Johnson, who could pass for Robert Redford’s cousin (with a little Dick Gephardt thrown in), spends his vacations snowboarding off-piste and trekking up serious mountains like Kilimanjaro. A former vice president of AOL’s software and technology development division and founder of image-compression pioneer Johnson-Grace, he’s a throwback to the dot-com days when suits were for CPAs and new software could be worth billions practically overnight.
Investors are already buzzing around ChoiceStream and the 100 million consumers it reaches. Johnson himself has ponied up more than $10 million, and in the past two years the company has pulled in another $20.1 million from venture capitalists. With all that money floating around, it’s understandable why Johnson is reluctant to dish about his client list, which includes AOL, Yahoo, DirecTV, and Overstock.com (whose ChoiceStream-powered “gift finder” increased the average visitor’s spending by 250 percent during last year’s holiday rush). ChoiceStream recently announced a partnership with Blockbuster Online, as the bricks-and-mortar video-rental company looks to horn in on Netflix’s turf, and rumor has it that something is in the works with Apple, as well as Google and its recently acquired YouTube—which ChoiceStream could help build into a YourTube. “Their demographic right now is 25 and under,” Johnson says. “They could easily extend that to other demographics if users could immediately find the content they’re interested in.”
The most vexing thing about first-generation recommenders is their inability to keep up with a rapidly expanding media universe. Their process of “collaborative filtering”—making recommendations based on past user behavior, such as ratings or purchases—pushes the already popular stuff: the most viewed show, the most listened-to song, the most purchased book. But it doesn’t deal well with brand-new content, thanks to the so-called cold-start problem: Because such content hits the market “cold,” i.e., with no previous user input, the recommenders don’t even include it. At the same time, some very good older stuff is quietly rotting on the back shelves. And the countless niches at the end of the “Long Tail”—the term coined by Wired editor Chris Anderson to refer to highly narrowed personal tastes—go completely unnoticed. Consequently, first-generation recommenders have gotten hammered for offering ridiculous suggestions (no, Amazon.com, I would not also like to buy a fishing memoir, thank you very much) and for homogenizing our culture by simply touting what’s already mainstream.
ChoiceStream’s innovation is to cut those other users out of the equation. After all, this is about you, not some guy in Albuquerque. What do you want?
The key to answering that is a complex algorithm that requires lots of degrees to truly understand. Basically, the ChoiceStream recommender breaks down every bit of content, from movies to TV shows, books to video games (and other retail products), into its attributes. For example, a movie like Casino Royale is tagged with attributes that go beyond just “James Bond,” “thriller,” “Daniel Craig.” It might include descriptions you’d use if you were telling your friend about the film: “violence,” “witty,” “sexual banter,” “cool cars,” “betrayal,” “explosive,” “performance art” (like parkour, the breathtaking artistic style of vaulting over obstacles, employed during one of the film’s chase scenes). Choice-Stream editors predefine these general attributes. Then the company’s software crawls through online reviews, blogs, press releases, and myriad other sources to figure out which content has what attributes. By examining the content itself in real time, rather than waiting for user input, ChoiceStream eliminates the cold-start problem: When a new movie comes out or a new TV show comes on, it has an equal opportunity to be recommended. Same with back-
catalog and niche content.
Finally, to make the match between all that content and you, the consumer, ChoiceStream simply layers the matrix of content attributes over the matrix of your preferences (compiled from the usual questionnaires, ratings, similar-users data, and, when allowed, Big Brother–ish analysis of how you consume content online, on your TV, on your phone, etc.). Where they overlap, there’s your recommendation.
In the center of ChoiceStream’s logo is an eye. It’s cartoonish and friendly, reminiscent of a teddy bear’s lazy gaze, but once you notice it, the Orwellian significance is striking. Johnson would tell you it’s symbolic of his company’s ability to scan the world of content. But come on: It’s an eye, staring out at you 24/7.
What’s it really watching? Your consumer habits. Because, ultimately, this whole personalization thing is about making you consume more. It’s about selling you more stuff.
“It’s a question of value,” Johnson says of the gray area between recommendation and advertisement. “People will get over the nervousness, if there’s a value to it. They not only won’t care if it’s an ad, they’ll prefer it because it’s a value-added service. A technology that can do a really good job of anticipating and leading consumers to new areas that they’ll get excited about is great for the consumer. It’s also great for business.”
Spoken like a true CEO. But he’s right. The ad-based possibilities of personalization are fascinating. Consider cable TV: If your system knew your personal preferences, it not only could suggest other shows to watch, but also tailor the ads you see. During Studio 60 I might see ads for Scottish glam-rock bands, while an opera lover who’s also watching Studio 60 might see ads for a production of La Bohème. And from there, it gets really wild. Instead of “If I like Bond movies, what other movies would I like?” the question becomes “If I like Bond movies, Jim Harrison novels, and eco-friendly websites, what kind of car should I buy?”
“Today, predominantly, recommenders stay within their own domain, so they don’t make inferences about your movie preferences based on the type of music you like,” Johnson says. “But now we’re ready to take it to the next step, to make cross-domain inferences. It’s important not only to help consumers broaden their horizons and find an area they’ve never been exposed to, but it’s also important for advertisers. Take the preference data, and make educated guesses about what car you might want to drive. That’s the big payoff.”
Dan Ariely is one of those supersmart professors who make you wish you’d paid more attention in class when you were in college. Not only can he break down complex ideas into layman’s terms, but he also possesses an endearing, nerdy sense of humor (his website has a “riddles” page, which leads with “If a man talks in the forest and his wife is not there to hear him, is he still wrong?”). As head of something called the eRationality Research Group at MIT, he studies how people make decisions, particularly purchasing decisions.
It turns out our preferences aren’t as set as we think. They’re evolving all the time. So no matter how good Choice-Stream’s recommender is, it can never truly anticipate what we will like, because we can’t even do that ourselves. A computer can analyze only what you have done—past tense—and has no way of accounting for emotions and moods and context and personal development.
“Think about someone learning to appreciate wine,” Ariely says. “You develop your taste going from lighter to heavier wines, whites to reds. So if I start drinking white wine, an electronic recommender system will help me find a great white wine. But it will never know when I’m ready to go to a light red and then to a heavier, better red.”
Also lost in a world of recommenders is one of life’s beautiful quirks: serendipity. The element of surprise. How often have you been surprised by the opening band at a concert, or by a book your friend left at your apartment, or a TV show you stumble across while grazing through your 500 channels? (I discovered my beloved Fratellis on a TV in a London airport.) In an algorithm-based world like the one Johnson and Choice-Stream envision, that couldn’t happen, because nothing is left to chance. That’s the whole point of personalization: Remove the surprise. Reduce the mistakes. And maybe, never learn.
“Don’t get me wrong—this technology is essential,” says Ariely. “It’s the next search engine. The ideal recommender system is like a spouse. It would work for you and make life easier. But you can’t get a complete electronic spouse. And who would want one?”