Advanced Regression Models
Objectives
In statistical modelling, the usual framework of a parametric model is most often a conve- nient simplification of the “true model” which describes the random phenomenon of interest. The parametric framework is typically sufficient for the description of simple situations, but quickly loses relevance when the focus is on modelling more complex situations such as regression problems. The opposite approach consists in defining a much more general model, called “nonparametric model”, where a distribution is characterised by a function and not by a vector of real-valued parameters. Identifying the underlying distribution is then equivalent to estimating this function.
In this course we discuss popular methods for estimating the regression function beyond linearity. The first topic discusses local linear regression and local polynomial regression that achieve flexibility in estimating the regression function. Next, we study how to estimate the regression function using basis expansions and regularisation such as smoothing splines Finally, we will work on additive models and on semiparametric models. A semiparametric model contains a finite-dimensional parameter and a functional parameter. This kind of model allows to keep both a potential for interpretability of estimates and prediction of future events, due to the presence of the parametric component, and good flexibility brought by its functional part. The module focuses on two such models: partially linear regression models and single-index models.
Course outline
Smoothing in nonparametric regression: general concepts (such as bias-variance tradeoff, kernels).
Basis expansion, penalized regression and regularization. Definition, statistical properties (bias, variance, consistency), choice of tuning parameters. Implementation in R.
Kernel Smoothing Methods such as Local Linear Regression and Local polynomial regression: definition, statistical properties (bias, variance, consistency), choice of tuning parameters. Implementation in R. Limits of nonparametric approaches.
Additive models, semi-parametric models (partial linear models and single-index models) Implementation in R
Prerequisites
Good understanding of regression and generalized linear model courses