3 Nonparametric Regression

  • Start: Monday, February 8
  • End: Friday, February 12

3.1 Summary

This week we will continue discussing the regression task. We will introduce to new methods, k-nearest neighbors and decisions trees, which will serve as examples of nonparametric modeling techniques. We will also discuss model flexibility and how it relates to the overfitting and bias-variance tradeoff.

  • Keywords: Nonparametric Regression, k-Nearest Neighbors, Decision Trees, Model Flexibility, Tuning Parameters, Bias-Variance Tradeoff, Overfitting, No Free Lunch, Curse of Dimensionality

3.2 Learning Objectives

After completing this week, you are expected to be able to:

  • Differentiate between parametric and nonparametric regression.
  • Understand how model flexibility relates to the bias-variance tradeoff and thus model performance.
  • Use R packages and functions to fit KNN and decision tree models and make predictions or estimate conditional means.
  • Select models by manipulating their flexibility through the use of a tuning parameter.
  • Avoid overfitting by selecting an a model of appropriate flexibility through the use of a validation set.

3.4 Video

Title Link Mirror
3.1 - Welcome to Week 03 3.1 - YouTube 3.1 - ClassTranscribe
3.2 - Nonparametric Regression 3.2 - YouTube 3.2 - ClassTranscribe
3.3 - Nonparametric Regression in R 3.3 - YouTube 3.3 - ClassTranscribe
3.4 - Supervised Learning Concepts 3.4 - YouTube 3.4 - ClassTranscribe

3.5 Assignments

Assignment Deadline Credit
Quiz 01 Monday, February 15 85%
Quiz 02 Monday, February 15 100%
Quiz 03 Monday, February 15 105%

3.6 Office Hours

Staff and Link Day Time
Zoom with David Monday 8:00 PM - 9:00 PM
Zoom with Tianyi Monday 9:00 PM - 10:00 PM
Zoom with David Thursday 8:00 PM - 9:00 PM
Zoom with Tianyi Thursday 9:00 PM - 10:00 PM
Piazza Any! Any!