Lecture 28 - Linear Regression and Tricks

import seaborn as sns

Announcements:

• Quiz 7 graded
  • Question 2 - PCA cannot reduce any dataset to its intrinsic dimensionality =(
• I recommend running your final project idea by me to see if you're on a good path.
  • End of today's class
  • Office hours
  • Email

Goals

• Understand the setup of the linear regression problem in linear algebra notation.
• Know how to fit nonlinear functions with linear regression using feature expansions
• Know about some of the limitations and failure modes of linear regression:
  • Outliers
  • Linearly dependent columns
• Understand the definition and purpose of regularization, and see a couple examples (ridge regression (L2), LASSO (L1))

Linear Regression

What is it?

• It's a way to solve the regression problem in which you assume the target variable is a linear function of the input feature(s).
• It's how you compute a "best fit line"

sns.lmplot(x="body_mass_g", y="flipper_length_mm", data=sns.load_dataset("penguins"), scatter_kws={"s": 4}, height=8)

<seaborn.axisgrid.FacetGrid at 0x16dacb3d0>

"The target variable is a linear function of the input features" means... $$ y = ax$$