How I Perform Ridge Regression in R [Update 2023]

5 min readSep 14, 2023

Key points

Ridge regression is a method of regularization that can help you deal with multicollinearity, improve the accuracy of your predictions, and reduce the complexity of your model.
Ridge regression adds a penalty term to the ordinary least squares objective function, which is proportional to the sum of squared coefficients of the regression model.
The penalty term is controlled by a lambda parameter, which determines how much the coefficients are shrunk towards zero.
To implement ridge regression in R, you need to use the glmnet package, which provides functions for fitting generalized linear models with various types of regularization.
To choose the optimal value of lambda, you need to use cross-validation, a technique that splits the data into several subsets and uses some for training and some for testing.

Ridge regression is a method of regularization that can help you deal with multicollinearity, improve the accuracy of your predictions, and reduce the complexity of your model.

In this article, you will learn how to use ridge regression in R, how it works, and how to compare it with other forms of regression. This article is worth reading because it will teach you a useful data analysis and machine learning technique and how to implement it using R.

What is Ridge Regression?

Ridge regression is a form of regression that adds a penalty term to the ordinary least squares (OLS) objective function. The penalty term is proportional to the sum of the squared coefficients of the regression model.

It means that ridge regression tries to minimize the sum of squared residuals (SSR) and the sum of squared coefficients (SSC) simultaneously. The penalty term is controlled by a lambda parameter, which determines how much the coefficients are shrunk towards zero. The higher the lambda, the more the coefficients are shrunk, and the lower the lambda, the more the coefficients are similar to the OLS estimates.

Ridge regression is also known as L2 regularization because it is equivalent to the square of the magnitude of the coefficients. Ridge regression can help you deal with multicollinearity, where some predictor variables are highly correlated.

Multicollinearity can cause the OLS estimates to be unstable, have high variance, and be sensitive to small changes in the data. By shrinking the coefficients, ridge regression reduces the estimates’ variance and makes them more robust to multicollinearity.

However, ridge regression also has some drawbacks. One of them is that it cannot perform variable selection, meaning it cannot reduce the number of predictors in the model. All the coefficients are shrunk, but none are set to zero. It can make the model more complex and harder to interpret.

Another drawback is that ridge regression can introduce some bias in the estimates, meaning they can deviate from the true values. The bias increases as the lambda increases, and the model becomes less flexible and more prone to underfitting.

How to Implement Ridge Regression in R?

Function/Library Description glmnet A package that provides functions for fitting generalized linear models with various types of regularization, such as ridge, lasso, and elastic net. lm.ridge A function from the MASS package that performs ridge regression using the method of ordinary ridge regression or generalized cross-validation. ridge A package that provides functions for linear and logistic ridge regression. Additionally, it includes special functions for genome-wide single-nucleotide polymorphism (SNP) data. cv.glmnet A function from the glmnet package that performs k-fold cross-validation for ridge regression models and returns the optimal value of lambda that minimizes the test error. predict A generic function that generates the predicted values of the response variable for a given model and new data. plot A generic function that produces a graphical display of a model or an object. summary A generic function that returns a summary of a model or an object, such as the coefficients, the lambda values, the degrees of freedom, and the deviance. tidy

A function from the broom package that converts a model or an object into a tidy data frame, with one row per term and one column per attribute.

Libraries and Functions used in this Tutorial

To implement ridge regression in R, you need to use the glmnet package, which provides functions for fitting generalized linear models with various types of regularization. You can install the package using the following command. After installing, load the package using the following command:

install.packages("glmnet")
library(glmnet)

?glmnet

The main function for fitting ridge regression models is glmnet, which takes the following arguments:

x: a matrix of predictor variables
y: a vector of response values
alpha: a parameter that controls the type of regularization. alpha = 0 corresponds to ridge regression, alpha = 1 corresponds to lasso regression and 0 < alpha < 1 corresponds to elastic net regression, a combination of ridge and lasso.
lambda: a parameter that controls the amount of regularization. You can specify a single value of lambda or a sequence of values. If you do not specify lambda, the function will automatically generate a sequence of 100 values, ranging from a very large value (corresponding to many shrinkages) to a very small value (corresponding to no shrinkage).

The function returns an object of class glmnet, which contains information about the fitted model, such as the coefficients, the lambda values, the degrees of freedom, and the deviance. You can access the elements of the object using the $ operator. For example, if you name the object model, you can access the coefficients using model$beta.

To illustrate how to use the glmnet function, we will use a built-in dataset in R called mtcars, which contains information about 32 cars, such as their miles per gallon (mpg), number of cylinders (cyl), displacement (disp), horsepower (hp), and weight (wt). We will try to predict the cars’ mpg using the other variables as predictors.

Split the dataset

First, we must split the dataset into a training set and a test dataset using the sample function. We will use 80% of the data for train and 20% for testing.

Standardize the Predictor Variables

Next, we must standardize the predictor variables to have mean zero and unit variance. This is important for ridge regression because it ensures that the penalty term is applied equally to all the coefficients. We can use the scale function to do this.

Ridge Regression Model

We are ready to fit the ridge regression model using the glmnet function. We will use the default values of alpha and lambda and let the function choose the optimal values for us.

model <- glmnet(x_train, y_train) # fit the ridge regression model

We can inspect the model object using the summary function, which will show us the length, class, mode, and dimensions of the elements.

summary(model)

Our Social Media Handles
Facebook: https://www.facebook.com/DataAnalysis03
Instagram: https://www.instagram.com/dataanalysis03/
Twitter: https://www.twitter.com/Zubair01469079/
Youtube: https://www.youtube.com/@data.03?sub-confirmation=1
Whatsapp Community: https://chat.whatsapp.com/IGjjvZVsGCiLn5jSmrFQaJ
Telegram Channel: https://t.me/dataanalysis03
Medium: https://medium.com/@zubairishaq8305
Quora: https://www.quora.com/profile/Muhammad-Zubair-Ishaq
Google News: https://news.google.com/publications/CAAqBwgKMIaV0QswxbDoAw?hl=en-PK&gl=PK&ceid=PK%3Aen