Recommendation system

General purpose recommendation system based on LSTM and Deep learning was developed.

Deep learning
CUDA
LSTM
Torch

2015

About project

Task

Develop software to produce new recommendations based on collaborative user activity.

Solution

  • LSTM-based model was developed
  • Implemented software for data grabbing and processing
  • We published web solution to analyze the statistics and get recommendations online

See Movix.ai

About recommendations

Usually, when people consume content in any form, recommendations of the new material - a key element of any system. Indeed, the increase in sales of even a few percent can end up in millions of dollars.

Typical example is Netflix Prize. In 2006, one of the largest VOD provider Netflix launched an contest in which participants were trying to advise next movie to watch based on user ratings. The task was to beat current algorithm by 10%. And the prize was $1 000 000. It's not hard to calculate how much company with $3,61 billion turnover will benefit from 10% sales increase.

Pic 1. Netflix.com

Of course, it is possible to recommend more that just movies:

  • Movies, TV series
  • Music
  • Products
  • Books
  • New
  • Web sites

Methods

There are a few general methods to build recommendation system: content filtering and collaborative filtering.

Content filtering is a simple and well-understood method, in which we consider content parameters of item and look for similar items. For example, if user has seen «The Terminator», we can advise other Arnold Schwarzenegger action-movies.

However, the most popular method for today is collaborative filtering. Collaborative filtering use information of every user activity in past — like ratings or views. For example, if user has seen movie «Home Alone», recommendation system will look for movies, other people have seen along with «Home Alone». For example, «Christmas Carol» can be such a movie, because both movies take place on Christmas. Dependencies like this are very hard to program, but such a task can be solved using complex computer models.

Pic 2. Collaborative filtering

Model

In our solution we use recurrent neural networks, LSTM (Long short term memory), to be exact. Such networks work great when the task is to predict next event based on the chain of previous events. In our case, it can be next movie to watch, considering user view history.

This method is really flexible and allows us to use any kind of information about items. Moreover, order of events is considered so it is possible to make recent events more valuable. Another great thing is that we don't have to rebuild model from scratch to obtain new recommendation from new events.

Final models are used in production software to increase recommendations quality. To make analysis easier a web dashboard was developed with help of Angular and Node.JS.

Next project

Airports runway defects detection