Building a PAI-based Recommender System within 10 Minutes

Image for post
Image for post

Are you an AI enthusiast with a keen eye on innovation? Sign up for the Alibaba Cloud Global AI Innovation Challenge and win big! Sign Up Here >>

By GarvinLi

In this article, Alibaba technical expert Aohai introduces how to establish a simple recommender system based on Machine Learning Platform for AI (PAI) within 10 minutes. This article focuses on four parts: the personalized recommendation process, collaborative filtering algorithm, architecture of the recommender system, and practices.

1) Personalized Recommendation Process

We will introduce how to build a simple recommender system based on PAI. This recommender system has two characteristics. First, the system is convenient to build, with the help of tools that we have developed. Second, this system is scalable. We all know that apps today have many feed streaming recommendations, such as advertising and content recommendations. Such an app is essentially a recommender system. A recommender system can be divided into two main modules. The first is the matching module, which is responsible for preliminary filtering. For example, the module can narrow down 100,000 candidate news feeds to only 500. The second is the ranking module, which ranks the 500 news feeds based on the user’s preference and then generates the final recommended rank. A simple recommender system with only the matching module can also make recommendations. If we narrow this scope to as small as possible, such as 10 news feeds, we can push all of the 10 feeds to users without ranking. Therefore, we want to use this video to illustrate how to establish a simple recommender system with the matching module only.

Image for post
Image for post

2) Collaborative Filtering

Many algorithms are available for both the matching and ranking modules. For example, multiple matrix factorization algorithms and the collaborative filtering algorithm are available for the matching module. The most classic one is collaborative filtering, which is easy to understand. We will illustrate it. For example, the following figure shows the preferences of users A, B, and C. Users A and C have similar tastes. Specifically, both users A and C like rice and milk. In addition, user A likes lamb but user C does not like it. We assume that user C also likes lamb and regard lamb as a matching result of user C. This is standard collaborative filtering based on data statistics. This figure can help you understand how collaborative filtering works. In other words, we first find similar items or users, and then identify a recommendation logic based on the correlation between the similar users or items. The recommender system discussed today is based on the collaborative filtering algorithm.

Image for post
Image for post

3) Architecture of the Recommender System

The following figure shows the overall architecture of the recommender system, which includes DataWorks, PAI-Studio, Tablestore, PAI-AutoLearning, and PAI-EAS.

Image for post
Image for post

4) Practices

First, we generate your collaborative filtering result data based on your raw data in the PAI-Studio, and obtain two tables.

Image for post
Image for post

Next, we create the two tables in Tablestore in the required format.

Image for post
Image for post

Then, we use DataWorks to migrate the data of the two tables from PAI-Studio to Tablestore.

Image for post
Image for post

Then, in PAI-AutoLearning, we can configure a policy for the two tables in Tablestore.

Image for post
Image for post

Finally, we convert this policy to a PAI-EAS service. You can check how to call this service here. The result obtained by the user is this service.

Image for post
Image for post

To update and iterate all the data involved, you can modify the original table. The whole set of services can be automated. Therefore, you can use the DataWorks data scheduling system to make an automated system. Then, you only need to update the raw data on a daily basis, and all the remaining services can be automatically completed.

Image for post
Image for post

Learn more about Alibaba Cloud Machine Learning Platform for AI (PAI) at https://www.alibabacloud.com/product/machine-learning

The views expressed herein are for reference only and don’t necessarily represent the official views of Alibaba Cloud.

Original Source:

Gain Access to Expert View — Subscribe to DDI Intel

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store