polynomial regression torch

3 min read 10-03-2025

Polynomial regression extends linear regression by adding polynomial terms to the model. This allows it to capture non-linear relationships between variables. PyTorch, a powerful deep learning framework, provides the tools to implement and train these models efficiently. This guide will walk you through the process, from the basics to advanced techniques.

Understanding Polynomial Regression

In simple linear regression, we model the relationship between variables as a straight line (y = mx + c). Polynomial regression generalizes this by adding higher-order terms (x², x³, etc.), allowing the model to fit curves instead of straight lines. The degree of the polynomial determines the complexity of the curve. A higher degree means a more complex curve, potentially overfitting the data.

Example: A second-degree polynomial regression model would look like this: y = a + bx + cx²

This equation represents a parabola, capable of modeling U-shaped or inverted U-shaped relationships.

Implementing Polynomial Regression in PyTorch

Here's how to implement polynomial regression using PyTorch, step-by-step:

1. Data Preparation

First, we need our data. Let's generate some synthetic data for demonstration:

import torch
import torch.nn as nn
import matplotlib.pyplot as plt
import numpy as np

# Generate synthetic data
np.random.seed(0)
X = np.linspace(-1, 1, 100)
y = 2*X**2 + X + 1 + np.random.normal(0, 0.2, 100) #add some noise

X = torch.tensor(X, dtype=torch.float32).reshape(-1,1)
y = torch.tensor(y, dtype=torch.float32).reshape(-1,1)

This code generates 100 data points following a quadratic relationship with added Gaussian noise to make it more realistic. Remember to always normalize or standardize your data for optimal model performance.

2. Creating the Model

We'll define a simple neural network to represent our polynomial regression model. For a polynomial of degree 'n', we'll need 'n+1' weights.

class PolynomialRegression(nn.Module):
    def __init__(self, degree):
        super(PolynomialRegression, self).__init__()
        self.degree = degree
        self.linear = nn.Linear(1, degree + 1)

    def forward(self, x):
        # Add polynomial terms
        x_poly = torch.cat([x**i for i in range(self.degree + 1)], dim=1)
        return self.linear(x_poly)

# Instantiate the model with desired polynomial degree (e.g., 2 for quadratic)
model = PolynomialRegression(degree=2)

This PolynomialRegression class takes the polynomial degree as input during initialization. The forward method creates the polynomial terms and passes them to a linear layer for prediction.

3. Training the Model

We'll use Mean Squared Error (MSE) as our loss function and Stochastic Gradient Descent (SGD) as our optimizer.

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

# Training loop
epochs = 1000
for epoch in range(epochs):
    # Forward pass
    y_pred = model(X)
    loss = criterion(y_pred, y)

    # Backward pass and optimization
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (epoch+1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}')

This training loop iteratively updates the model's weights to minimize the MSE loss. We print the loss every 100 epochs to monitor progress.

4. Evaluation and Visualization

Finally, let's evaluate the model and visualize the results:

# Make predictions on the training data
y_pred = model(X).detach().numpy()

# Plot the results
plt.figure(figsize=(10, 6))
plt.scatter(X.numpy(), y.numpy(), label='Data')
plt.plot(X.numpy(), y_pred, color='red', label='Polynomial Regression')
plt.xlabel('X')
plt.ylabel('y')
plt.legend()
plt.title('Polynomial Regression with PyTorch')
plt.show()

This code plots the original data and the model's predictions, allowing you to visually assess the model's fit.

Choosing the Polynomial Degree

The choice of polynomial degree is crucial. A low degree might underfit the data, failing to capture its complexity. A high degree, on the other hand, might overfit the data, performing well on the training set but poorly on unseen data. Techniques like cross-validation can help find the optimal degree.

Advanced Techniques

Regularization: Techniques like L1 or L2 regularization can help prevent overfitting by penalizing large weights. You can add these to your loss function.
Different Optimizers: Experiment with different optimizers like Adam or RMSprop for potentially faster convergence.
Feature Scaling: Always standardize or normalize your features for better performance.

This comprehensive guide provides a solid foundation for implementing polynomial regression using PyTorch. Remember to adapt the code and techniques to your specific dataset and problem. By experimenting with different polynomial degrees and optimization strategies, you can build powerful models that accurately capture complex relationships in your data.