Netflix discusses its recommendation engine

One of the things that really put Netflix on the map was its recommendations and the publicity for the million dollar Netflix prize certainly helped garner the company a lot of attention. I think the recommendations of Netflix have been taken for granted for a while. People subscribe to Netflix for its DVD by mail or streaming selection. Useful or not, hearing the story behind Netflix’s recommendation engine and the popular contest are still quite interesting.

In 2006 we announced the Netflix Prize, a machine learning and data mining competition for movie rating prediction. We offered $1 million to whoever improved the accuracy of our existing system called Cinematchby 10%. We conducted this competition to find new ways to improve the recommendations we provide to our members, which is a key part of our business. However, we had to come up with a proxy question that was easier to evaluate and quantify: the root mean squared error (RMSE) of the predicted rating. The race was on to beat our RMSE of 0.9525 with the finish line of reducing it to 0.8572 or less.

A year into the competition, the Korbell team won the first Progress Prize with an 8.43% improvement. They reported more than 2000 hours of work in order to come up with the final combination of 107 algorithms that gave them this prize. And, they gave us the source code. We looked at the two underlying algorithms with the best performance in the ensemble: Matrix Factorization (which the community generally called SVD,Singular Value Decomposition) and Restricted Boltzmann Machines (RBM). SVD by itself provided a 0.8914 RMSE, while RBM alone provided a competitive but slightly worse 0.8990 RMSE. A linear blend of these two reduced the error to 0.88. To put these algorithms to use, we had to work to overcome some limitations, for instance that they were built to handle 100 million ratings, instead of the more than 5 billion that we have, and that they were not built to adapt as members added more ratings. But once we overcame those challenges, we put the two algorithms into production, where they are still used as part of our recommendation engine.

You can continue reading the story at the Netflix Tech Blog to see how the contests ended and how things changed as the company transition from DVD to streaming and opened up to new markets. Fun fact: the US is the only country Netflix doesn’t operate its Facebook connect feature due to VPPA law concerns.

Like every Netflix blog post in recent years, the comments are filled with unhappy customers. People pick on certain elements like the interface or the fact that the recommendations are completely muddled by input from multiple family members.

It’s a two-part story, so be sure to check back for the second part.

In Part 2, we will describe some of the data and models that we use and discuss our approach to algorithmic innovation that combines offline machine learning experimentation with online AB testing.



Categories : Web
Posted by Jason Hamilton | April 10, 2012  |  No Comment

Leave a Comment