Regularization Code – Line by Line

Step 1: Imports

We load the libraries we need.

import pandas as pd
import seaborn as sns
from sklearn.linear_model import LinearRegression, Ridge, Lasso
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings("ignore")

What each line does

import pandas as pd — Lets us use DataFrames and read CSV files.
import seaborn as sns — For nicer plots (optional).
from sklearn.linear_model import ... — Brings in LinearRegression, Ridge, and Lasso so we can fit models.
import matplotlib.pyplot as plt — For drawing graphs.
import warnings — So we can turn off warning messages.
warnings.filterwarnings("ignore") — Hides warnings so the output is easier to read.

Step 2: Load the data

Read the CSV file into a DataFrame.

data = pd.read_csv("auto_mpg.csv")

What this line does

data = pd.read_csv("auto_mpg.csv") — Reads the file auto_mpg.csv from the current folder and stores it in a variable called data. Each row is a car; columns are mpg, cylinders, displacement, horsepower, weight, acceleration, model year, origin, car name.

Step 3: Quick look at the data

See the first rows and column types.

data.head(15)   # First 15 rows
data.info()    # Column names, types, and non-null counts

What each line does

data.head(15) — Shows the first 15 rows so you can see sample values.
data.info() — Prints how many rows, column names, data types (int, float, object), and how many non-null values each column has.

Step 4: Clean the horsepower column

In this dataset, missing values in horsepower are stored as ?. We replace them with the average and convert to numbers.

data['horsepower'] = data['horsepower'].str.replace('?', 'NaN').astype(float)
data['horsepower'].fillna(data['horsepower'].mean(), inplace=True)
data['horsepower'] = data['horsepower'].astype(int)

What each line does

Line 1 — Replaces every ? in the horsepower column with NaN (missing), then converts the column to float so we can do math.
Line 2 — Fills every NaN in horsepower with the mean (average) of the column. inplace=True means we change the column in place.
Line 3 — Converts horsepower to integers (e.g. 104) so it's ready for the model.

Step 5: Prepare features and target, then split

We predict mpg (miles per gallon) from the other columns. We drop the car name (not useful for prediction), then split into train and test.

data.drop(columns=['car name'], axis=1, inplace=True)

from sklearn.model_selection import train_test_split
train, test = train_test_split(data, test_size=0.20, random_state=0)
y_train = train.pop('mpg')
y_test = test.pop('mpg')
X_train = train
X_test = test

What each line does

data.drop(columns=['car name'], ...) — Removes the "car name" column; we don't use it to predict mpg.
train_test_split(data, test_size=0.20, random_state=0) — Splits data into 80% train and 20% test. random_state=0 makes the split the same every time.
y_train = train.pop('mpg') — Takes the mpg column out of the training set and saves it as the target we want to predict.
y_test = test.pop('mpg') — Same for the test set.
X_train = train — What's left in train (all columns except mpg) are the features for training.
X_test = test — Same for test.

Step 6: Ridge regression (L2)

Ridge shrinks all weights. alpha is the strength of the penalty (lambda).

from sklearn.linear_model import Ridge
ridge = Ridge(alpha=10.0)
ridge.fit(X_train, y_train)
print(f"Ridge R² on test: {ridge.score(X_test, y_test):.3f}")

What each line does

Ridge(alpha=10.0) — Creates a Ridge model. alpha=10.0 means a strong penalty; try 0.1 or 1.0 for weaker regularization.
ridge.fit(X_train, y_train) — Trains the model using training features and target.
ridge.score(X_test, y_test) — Returns R² on the test set (how well the model predicts).

Step 7: Lasso regression (L1)

Lasso can set some weights to exactly zero, so it also does feature selection.

from sklearn.linear_model import Lasso
lasso = Lasso(alpha=0.1)
lasso.fit(X_train, y_train)
print(f"Lasso R² on test: {lasso.score(X_test, y_test):.3f}")
# See which features Lasso kept (non-zero coefficients)
for name, coef in zip(X_train.columns, lasso.coef_):
    if coef != 0:
        print(f"  {name}: {coef:.3f}")

What each line does

Lasso(alpha=0.1) — Creates a Lasso model. Small alpha = weaker penalty; large alpha = more coefficients become zero.
lasso.fit(X_train, y_train) — Trains the model.
lasso.score(X_test, y_test) — R² on test set.
for name, coef in zip(...) — Loops over each feature name and its coefficient.
if coef != 0 — Lasso sets unused features to 0; we only print the ones it kept.

Impact of alpha (lambda)

Try different alpha values and see how R² and the number of non-zero coefficients change. Higher alpha → stronger regularization → simpler model.

# Try different alphas
for alpha in [0.01, 0.1, 1.0, 10.0]:
    m = Lasso(alpha=alpha)
    m.fit(X_train, y_train)
    nz = sum(1 for c in m.coef_ if c != 0)
    print(f"alpha={alpha}: R²={m.score(X_test, y_test):.3f}, non-zero coefs={nz}")

What each line does

for alpha in [0.01, 0.1, 1.0, 10.0] — We try four different regularization strengths.
m = Lasso(alpha=alpha) — Create a Lasso model with this alpha.
m.fit(X_train, y_train) — Train it.
nz = sum(1 for c in m.coef_ if c != 0) — Count how many coefficients are non-zero (how many features Lasso kept).
print(...) — Print alpha, R², and number of non-zero coefficients so you can see how alpha changes the model.