Learn all the basics you need to get started with this deep learning framework! In this part we improve the code from the last part and will learn how a complete training pipeline is implemented in PyTorch. We replace the manually computed loss and weight updates with a loss and an optimizer from the PyTorch framework, which can do the optimization for us. We will then see how a PyTorch model is implemented and used for the forward pass.

- Training Pipeline in PyTorch
- Model Design
- Loss and Optimizer
- Automatic Training steps with forward pass, backward pass, and weight updates

All code from this course can be found on GitHub.

## Introduce Loss and Optimizer

```
# 1) Design model (input, output, forward pass with different layers)
# 2) Construct loss and optimizer
# 3) Training loop
# - Forward = compute prediction and loss
# - Backward = compute gradients
# - Update weights
import torch
import torch.nn as nn
# Linear regression
# f = w * x
# here : f = 2 * x
# 0) Training samples
X = torch.tensor([1, 2, 3, 4], dtype=torch.float32)
Y = torch.tensor([2, 4, 6, 8], dtype=torch.float32)
# 1) Design Model: Weights to optimize and forward function
w = torch.tensor(0.0, dtype=torch.float32, requires_grad=True)
def forward(x):
return w * x
print(f'Prediction before training: f(5) = {forward(5).item():.3f}')
# 2) Define loss and optimizer
learning_rate = 0.01
n_iters = 100
# callable function
loss = nn.MSELoss()
optimizer = torch.optim.SGD([w], lr=learning_rate)
# 3) Training loop
for epoch in range(n_iters):
# predict = forward pass
y_predicted = forward(X)
# loss
l = loss(Y, y_predicted)
# calculate gradients = backward pass
l.backward()
# update weights
optimizer.step()
# zero the gradients after updating
optimizer.zero_grad()
if epoch % 10 == 0:
print('epoch ', epoch+1, ': w = ', w, ' loss = ', l)
print(f'Prediction after training: f(5) = {forward(5).item():.3f}')
```

## Introduce the model class

```
# 1) Design model (input, output, forward pass with different layers)
# 2) Construct loss and optimizer
# 3) Training loop
# - Forward = compute prediction and loss
# - Backward = compute gradients
# - Update weights
import torch
import torch.nn as nn
# Linear regression
# f = w * x
# here : f = 2 * x
# 0) Training samples, watch the shape!
X = torch.tensor([[1], [2], [3], [4]], dtype=torch.float32)
Y = torch.tensor([[2], [4], [6], [8]], dtype=torch.float32)
n_samples, n_features = X.shape
print(f'#samples: {n_samples}, #features: {n_features}')
# 0) create a test sample
X_test = torch.tensor([5], dtype=torch.float32)
# 1) Design Model, the model has to implement the forward pass!
# Here we can use a built-in model from PyTorch
input_size = n_features
output_size = n_features
# we can call this model with samples X
model = nn.Linear(input_size, output_size)
'''
class LinearRegression(nn.Module):
def __init__(self, input_dim, output_dim):
super(LinearRegression, self).__init__()
# define diferent layers
self.lin = nn.Linear(input_dim, output_dim)
def forward(self, x):
return self.lin(x)
model = LinearRegression(input_size, output_size)
'''
print(f'Prediction before training: f(5) = {model(X_test).item():.3f}')
# 2) Define loss and optimizer
learning_rate = 0.01
n_iters = 100
loss = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)
# 3) Training loop
for epoch in range(n_iters):
# predict = forward pass with our model
y_predicted = model(X)
# loss
l = loss(Y, y_predicted)
# calculate gradients = backward pass
l.backward()
# update weights
optimizer.step()
# zero the gradients after updating
optimizer.zero_grad()
if epoch % 10 == 0:
[w, b] = model.parameters() # unpack parameters
print('epoch ', epoch+1, ': w = ', w[0][0].item(), ' loss = ', l)
print(f'Prediction after training: f(5) = {model(X_test).item():.3f}')
```