Summary - Recommendation Systems

What a recommendation system does

A recommendation system finds new items (movies, books, music, …) that you are likely to like, based on what you have liked before. Shopping sites use it to show products you didn’t know existed, streaming services like Netflix and Spotify to suggest movies and songs you have never heard of, and sites like Reddit to suggest new topics.

The simplest version is asking friends which movies they liked, and listening more to the friends who usually share your taste. It works, but it is slow and the data is very limited: you miss every movie none of your friends has seen.

Collaborative filtering and user preferences

Collaborative filtering is a set of techniques for making recommendations automatically. The term was first used by David Goldberg at Xerox PARC in 1992, for a system that recommended documents based on what a user had marked as interesting or uninteresting.

The first step is to store the user preferences, and they must be numeric, for example a rating from 1 to 5. Non-numeric preferences can be translated: bought = 1 and not bought = 0, or liked = 1, disliked = −1 and no vote = 0.

The lecture uses a small dataset of seven users (Lisa, Gene, Mike, Claudia, Mick, Jack and Toby) who have rated six movies. An empty cell means that the user hasn’t seen the movie. The same dataset is the movies example on the Datasets page, used to verify A1.

Averaging is not enough

The simplest way to recommend a movie to Toby is to average the other users’ ratings of the movies Toby hasn’t seen. The Night Listener gets the highest average (3.42). The problem is that an average treats every user equally, even those whose taste is nothing like Toby’s. A better recommendation gives more weight to users who are similar to Toby.

Euclidean distance

To find similar users, every user is compared with every other user and given a similarity score. If you plot two users’ ratings of the movies they have both rated as points in space, the distance between the points shows how different their taste is:

distance = sqrt((x_Toby - x_Mick)^2 + (y_Toby - y_Mick)^2)

A short distance means similar taste, but a similarity score should be high for similar users, so the distance is inverted, with 1 added to avoid division by zero:

similarity = 1 / (1 + distance)

In code, you loop over both users’ ratings, and for every movie they have both rated you add the squared difference to a sum and count the match. With no movies in common the similarity is 0. The square root is skipped for performance: it doesn’t change which users are closest, only the actual values.

Compare with the same formula

The scores in the lecture (for example 0.3077 between Toby and Mick) are calculated without the square root, as 1 / (1 + sum of squared differences). Use the same formula when you compare your A1 results with the test cases.

Pearson correlation

The Pearson correlation score is a more sophisticated similarity measure. It measures how well two users’ ratings fit on a straight line: if all points are on the line the score is 1, the best possible match. Gene and Mick have a fairly bad fit (0.4), Lisa and Jack a fairly good one (0.75).

Pearson handles data that isn’t well normalized better than Euclidean distance. A harsh user who always rates lower than others, or a user who consistently rates higher (grade inflation), can still be a perfect match if the ratings go up and down together. For the movies two users have both rated, with n as the number of common ratings:

num = pSum - (sum1 * sum2 / n)
den = sqrt((sum1sq - sum1^2 / n) * (sum2sq - sum2^2 / n))
r   = num / den

sum1/sum2 are the sums of each user’s ratings, sum1sq/sum2sq the sums of the squared ratings, and pSum the sum of the products of the two users’ ratings. With no ratings in common, return 0.

Finding the top matches

To find the top three matches for a user, calculate the similarity to every other user, sort the list with the highest score first and return the first three. The choice of measure matters: with Euclidean distance Toby’s best matches are Mick (0.3077), Mike (0.2857) and Claudia, with Pearson Lisa (0.9912), Mick (0.9245) and Claudia.

There are many other measures, such as Manhattan distance and the Jaccard coefficient. There is no universal best choice, it depends on the application, so try at least Euclidean and Pearson and see which works best for your data.

Recommending movies with weighted scores

Similar users are only the first step; what we really want is a recommendation. For each movie Toby hasn’t seen:

1. Multiply each other user’s rating of the movie by that user’s similarity to Toby (the weighted score).
2. Sum the weighted scores.
3. Sum the similarities of the users who have rated the movie.
4. Divide the sum of weighted scores by the sum of similarities.

Users with a similarity of 0 or below are left out. With Pearson, Mike has a similarity of −1 to Toby and doesn’t count; with Euclidean the similarity is always positive, so everyone is included. The Night Listener is the top recommendation either way (3.35 with Pearson, 3.50 with Euclidean), but now users who are more like Toby have more influence on the result. Dividing by the sum of similarities matters: otherwise a movie rated by many users would win just because more scores are added up.

Item-based collaborative filtering

Transposing the dataset, so that movies take the place of users, makes it possible to find movies that are similar to a movie, with exactly the same similarity functions. Transposing is slow for large datasets, though.

The approach so far, comparing a user with every other user, is called user-based collaborative filtering. It works for small datasets but becomes very slow for large ones, and with many users there is usually little overlap between them. Item-based collaborative filtering builds on the assumption that similarities between movies change much less than similarities between users. The top matching movies for every movie can then be pre-calculated and stored in a new dataset, and recommendations come from that table instead: for each movie Toby hasn’t seen, Toby’s own ratings of the movies he has seen are weighted by how similar those movies are to it. User similarities are not involved at all.

For Toby, item-based filtering also puts The Night Listener on top (3.183), but the order of the other two movies and the scores differ slightly from the user-based result.

Item-based is faster at producing recommendations for large datasets, but the similar-items table has to be generated and updated regularly, which is slow. It is usually more accurate on sparse datasets, with little overlap between users. The example dataset is dense: almost every user has rated almost every movie.

A real-world dataset

The GroupLens project at the University of Minnesota publishes MovieLens, a dataset collected from the movie recommendation service movielens.org, in a full version and a smaller one with 100,000 ratings. With a dataset of that size, some extra tricks are needed to get sensible scores; P1 uses the small MovieLens dataset and describes what to do.