---
title: 3 Recommendation Systems
updated: 2021-09-07 18:33:42Z
created: 2021-09-06 16:26:21Z
latitude: 52.09370000
longitude: 6.72510000
altitude: 0.0000
---

# Cloud SQL and Cloud Dataproc 

- Cloud SQL: managed relational database
- Cloud Dataproc: managed environment on which you can run Apache Spark

### Why moving from on-premisis to cloud
- utilizing and tuning on-premise clusters is dfficult
- but also moving dedicated storage to off-cluster storage 

A core aspect of a **recommendation system** is that you need to train and serve it at scale. 

### what is managed?
................

### Recommendation Systems
The core pieces are:
- data
- the model
- infrastructure
to train and serve recommendations to users. 

A core tenet of machine learning is to let the model learn for itself what
the relationship is between the data that you have, like user preferences (labeled data), and the data that you don't have. A history of good labeled data is important.

Machine learning scales much better because it doesn't require hard-coded rules. It's all automated. Learning from data in an automated way, that's what machine learning is.

Machine learning recommentation model is essentially asking "Who is this user like?" Secondly, is this subjectively a iyrm that people tend to rate highly? The predicted rating is a combination of both these factors. 
All things considered, the rating of an item for a particular user will be the average of the ratings of users like this user but it's calibrated with the quality of the item itself.
Updating the data can be in **batch**, because ratings of the items doesn't change on eq a daily basis. On the otherhand there is a lot of data that has to be updated in a **fault tolerant way** that can scale to **large datasets** ==> Apache Hadoop.
When the user logs on, we want to show that user the recommendations that we precomputed specifically for them. So we need a transactional way (so that while the user is reading these predictions, we can update the predictions table as well) to store the predictions. eg 1 million user with each 5 predictions = 5 miljion rows. A MySQL database is sufficient.