# Training Pipeline - PyTorch Beginner 06

Learn all the basics you need to get started with this deep learning framework! In this part we improve the code from the last part and will learn how a complete training pipeline is implemented in PyTorch. We replace the manually computed loss and weight updates with a loss and an optimizer from the PyTorch framework, which can do the optimization for us. We will then see how a PyTorch model is implemented and used for the forward pass.

- Training Pipeline in PyTorch
- Model Design
- Loss and Optimizer
- Automatic Training steps with forward pass, backward pass, and weight updates

All code from this course can be found on GitHub.

## Introduce Loss and Optimizer

```
# 1) Design model (input, output, forward pass with different layers)
# 2) Construct loss and optimizer
# 3) Training loop
# - Forward = compute prediction and loss
# - Backward = compute gradients
# - Update weights
import torch
import torch.nn as nn
# Linear regression
# f = w * x
# here : f = 2 * x
# 0) Training samples
X = torch.tensor([1, 2, 3, 4], dtype=torch.float32)
Y = torch.tensor([2, 4, 6, 8], dtype=torch.float32)
# 1) Design Model: Weights to optimize and forward function
w = torch.tensor(0.0, dtype=torch.float32, requires_grad=True)
def forward(x):
return w * x
print(f'Prediction before training: f(5) = {forward(5).item():.3f}')
# 2) Define loss and optimizer
learning_rate = 0.01
n_iters = 100
# callable function
loss = nn.MSELoss()
optimizer = torch.optim.SGD([w], lr=learning_rate)
# 3) Training loop
for epoch in range(n_iters):
# predict = forward pass
y_predicted = forward(X)
# loss
l = loss(Y, y_predicted)
# calculate gradients = backward pass
l.backward()
# update weights
optimizer.step()
# zero the gradients after updating
optimizer.zero_grad()
if epoch % 10 == 0:
print('epoch ', epoch+1, ': w = ', w, ' loss = ', l)
print(f'Prediction after training: f(5) = {forward(5).item():.3f}')
```

## Introduce the model class

```
# 1) Design model (input, output, forward pass with different layers)
# 2) Construct loss and optimizer
# 3) Training loop
# - Forward = compute prediction and loss
# - Backward = compute gradients
# - Update weights
import torch
import torch.nn as nn
# Linear regression
# f = w * x
# here : f = 2 * x
# 0) Training samples, watch the shape!
X = torch.tensor([[1], [2], [3], [4]], dtype=torch.float32)
Y = torch.tensor([[2], [4], [6], [8]], dtype=torch.float32)
n_samples, n_features = X.shape
print(f'#samples: {n_samples}, #features: {n_features}')
# 0) create a test sample
X_test = torch.tensor([5], dtype=torch.float32)
# 1) Design Model, the model has to implement the forward pass!
# Here we can use a built-in model from PyTorch
input_size = n_features
output_size = n_features
# we can call this model with samples X
model = nn.Linear(input_size, output_size)
'''
class LinearRegression(nn.Module):
def __init__(self, input_dim, output_dim):
super(LinearRegression, self).__init__()
# define diferent layers
self.lin = nn.Linear(input_dim, output_dim)
def forward(self, x):
return self.lin(x)
model = LinearRegression(input_size, output_size)
'''
print(f'Prediction before training: f(5) = {model(X_test).item():.3f}')
# 2) Define loss and optimizer
learning_rate = 0.01
n_iters = 100
loss = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)
# 3) Training loop
for epoch in range(n_iters):
# predict = forward pass with our model
y_predicted = model(X)
# loss
l = loss(Y, y_predicted)
# calculate gradients = backward pass
l.backward()
# update weights
optimizer.step()
# zero the gradients after updating
optimizer.zero_grad()
if epoch % 10 == 0:
[w, b] = model.parameters() # unpack parameters
print('epoch ', epoch+1, ': w = ', w[0][0].item(), ' loss = ', l)
print(f'Prediction after training: f(5) = {model(X_test).item():.3f}')
```

**Join My Newsletter! **Get Python and ML tips emailed directly to your inbox. Each month you’ll get a summary of all the content I created, including the newest videos, articles, promotions, tips, and more.

# ML From Scratch

Implement popular Machine Learning algorithms from scratch using only built-in Python modules and numpy.

# Advanced Python

Advanced Python Tutorials. It covers topics like collections, decorators, generators, multithreading, logging, and much more.

# PyTorch Beginner

Learn all the necessary basics to get started with this deep learning framework.