Supervised learning – Applying Machine Learning Algorithms – MLS-C01 Study Guide

Supervised learning

AWS provides supervised learning algorithms for general purposes (regression and classification tasks) and more specific purposes (forecasting and vectorization). The list of built-in algorithms that can be found in these sub-categories is as follows:

  • Linear learner algorithm
  • Factorization machines algorithm
  • XGBoost algorithm
  • KNN algorithm
  • Object2Vec algorithm
  • DeepAR forecasting algorithm

You will start by learning about regression models and the linear learner algorithm.

Working with regression models

Looking at linear regression models is a nice way to understand what is going on inside regression models in general (linear and non-linear regression models). This is mandatory knowledge for every data scientist and can help you solve real challenges as well. You will now take a closer look at this in the following subsections.

Introducing regression algorithms

Linear regression models aim to predict a numeric value (y) according to one or more variables (x). Mathematically, such a relationship can be defined as y = f(x), where y is known as the dependent variable and x is known as the independent variable.

With regression models, the component that you want to predict (y) is always a continuous number – for example, the price of houses or the number of transactions. You saw this in Chapter 1, Machine Learning Fundamentals, in Figure 1.2, when you were learning about the right type of supervised learning algorithm, given the target variable. Please feel free to go back and review it.

When you use just one variable to predict y, this problem is referred to as simple linear regression. On the other hand, when you use more than one variable to predict y, you have a multiple linear regression problem.

There is also another class of regression models, known as non-linear regression. However, let us put that aside for a moment and understand what simple linear regression means.

Regression models belong to the supervised side of machine learning (the other side is non-supervised) because algorithms try to predict values according to existing correlations between independent and dependent variables.

But what does f mean in y=f(x)? Here, f is the regression function responsible for predicting y based on x. In other words, this is the function that you want to find. When talking about simple linear regression, pay attention to the next three questions and answers:

  • What is the shape of f in linear regression?

Linear, of course!

  • How can you represent a linear relationship?

Using a straight line (you will understand why in a few minutes).

  • So what is the function that defines a line?

ax + b (just check any mathematics book).

That is it! Linear regression models are given by y = ax + b. When you are trying to predict y given x, you just need to find out the values of a and b. You can adopt the same logic to figure out what is going on inside other kinds of regression.

Finding out the values of a and b is the only thing you are going to do. It is nice to know that a is also known as the alpha coefficient, or slope, and represents the line’s inclination, while b is also known as the beta coefficient, or y intercept, and represents the place where the line crosses the y axis (into a two-dimensional plane consisting of x and y). You will learn about these two terms in a later subsection.

It is also nice to know that there is a bias (e) associated with every predictor that you do not have control over. That being said, the formal definition of simple linear regression is given by y = ax + b + e.

In the next subsection, you will learn how to find alpha and beta to solve a simple linear regression problem.