Building a PAI-based Recommender System within 10 Minutes
Are you an AI enthusiast with a keen eye on innovation? Sign up for the Alibaba Cloud Global AI Innovation Challenge and win big! Sign Up Here >>
By GarvinLi
In this article, Alibaba technical expert Aohai introduces how to establish a simple recommender system based on Machine Learning Platform for AI (PAI) within 10 minutes. This article focuses on four parts: the personalized recommendation process, collaborative filtering algorithm, architecture of the recommender system, and practices.
1) Personalized Recommendation Process
We will introduce how to build a simple recommender system based on PAI. This recommender system has two characteristics. First, the system is convenient to build, with the help of tools that we have developed. Second, this system is scalable. We all know that apps today have many feed streaming recommendations, such as advertising and content recommendations. Such an app is essentially a recommender system. A recommender system can be divided into two main modules. The first is the matching module, which is responsible for preliminary filtering. For example, the module can narrow down 100,000 candidate news feeds to only 500. The second is the ranking module, which ranks the 500 news feeds based on the user’s preference and then generates the final recommended rank. A simple recommender system with only the matching module can also make recommendations. If we narrow this scope to as small as possible, such as 10 news feeds, we can push all of the 10 feeds to users without ranking. Therefore, we want to use this video to illustrate how to establish a simple recommender system with the matching module only.
2) Collaborative Filtering
Many algorithms are available for both the matching and ranking modules. For example, multiple matrix factorization algorithms and the collaborative filtering algorithm are available for the matching module. The most classic one is collaborative filtering, which is easy to understand. We will illustrate it. For example, the following figure shows the preferences of users A, B, and C. Users A and C have similar tastes. Specifically, both users A and C like rice and milk. In addition, user A likes lamb but user C does not like it. We assume that user C also likes lamb and regard lamb as a matching result of user C. This is standard collaborative filtering based on data statistics. This figure can help you understand how collaborative filtering works. In other words, we first find similar items or users, and then identify a recommendation logic based on the correlation between the similar users or items. The recommender system discussed today is based on the collaborative filtering algorithm.
3) Architecture of the Recommender System
The following figure shows the overall architecture of the recommender system, which includes DataWorks, PAI-Studio, Tablestore, PAI-AutoLearning, and PAI-EAS.
4) Practices
First, we generate your collaborative filtering result data based on your raw data in the PAI-Studio, and obtain two tables.
Next, we create the two tables in Tablestore in the required format.
Then, we use DataWorks to migrate the data of the two tables from PAI-Studio to Tablestore.
Then, in PAI-AutoLearning, we can configure a policy for the two tables in Tablestore.
Finally, we convert this policy to a PAI-EAS service. You can check how to call this service here. The result obtained by the user is this service.
To update and iterate all the data involved, you can modify the original table. The whole set of services can be automated. Therefore, you can use the DataWorks data scheduling system to make an automated system. Then, you only need to update the raw data on a daily basis, and all the remaining services can be automatically completed.
Learn more about Alibaba Cloud Machine Learning Platform for AI (PAI) at https://www.alibabacloud.com/product/machine-learning
The views expressed herein are for reference only and don’t necessarily represent the official views of Alibaba Cloud.
Original Source:
Gain Access to Expert View — Subscribe to DDI Intel
