Become A Patron and get exclusive content! Get access to ML From Scratch notebooks, join a private Slack channel, get priority response, and more! I really appreciate the support!

back to course overview

# Tensor Basics - PyTorch Beginner 02

16 Dec 2019

Learn all the basics you need to get started with this deep learning framework! This part covers the basics of Tensors and Tensor operations in PyTorch. Learn also how to convert from numpy data to PyTorch tensors and vice versa!

All code from this course can be found on GitHub.

## Tensor

Everything in PyTorch is based on Tensor operations. A tensor can have different dimensions, so it can be 1d (scalar), 2d (vector), or even 3d (matrix) and higher. Let's have a look how we can create a tensor in PyTorch.

``````import torch

# torch.empty(size): uninitiallized
x = torch.empty(1) # scalar
print(x)
x = torch.empty(3) # vector, 1D
print(x)
x = torch.empty(2,3) # matrix, 2D
print(x)
x = torch.empty(2,2,3) # tensor, 3 dimensions
#x = torch.empty(2,2,2,3) # tensor, 4 dimensions
print(x)

# torch.rand(size): random numbers [0, 1]
x = torch.rand(5, 3)
print(x)

# torch.zeros(size), fill with 0
# torch.ones(size), fill with 1
x = torch.zeros(5, 3)
print(x)

# check size
print(x.size())

# check data type
print(x.dtype)

# specify types, float32 default
x = torch.zeros(5, 3, dtype=torch.float16)
print(x)

# check type
print(x.dtype)

# construct from data
x = torch.tensor([5.5, 3])
print(x.size())
``````

## `requires_grad` argument

This will tell PyTorch that it will need to calculate the gradients for this tensor later in your optimization steps, i.e., this is a variable in your model that you want to optimize. This will be very important later in our training!

``````x = torch.tensor([5.5, 3], requires_grad=True)
``````

## Operations on Tensors

``````y = torch.rand(2, 2)
x = torch.rand(2, 2)

z = x + y

# in place addition, everythin with a trailing underscore is an inplace operation
# i.e. it will modify the variable

# subtraction
z = x - y
z = torch.sub(x, y)

# multiplication
z = x * y
z = torch.mul(x,y)

# division
z = x / y
z = torch.div(x,y)
``````

## Slicing

``````x = torch.rand(5,3)
print(x)
print(x[:, 0]) # all rows, column 0
print(x[1, :]) # row 1, all columns
print(x[1,1]) # element at 1, 1

# Get the actual value if only 1 element in your tensor
print(x[1,1].item())
``````

## Reshape with `torch.view()`

``````x = torch.randn(4, 4)
y = x.view(16)
z = x.view(-1, 8)  # the size -1 is inferred from other dimensions
# if -1 it pytorch will automatically determine the necessary size
print(x.size(), y.size(), z.size())
``````

## Numpy

Converting a Torch Tensor to a NumPy array and vice versa is very easy

### torch to numpy with `.numpy()`

``````a = torch.ones(5)
print(a)

b = a.numpy()
print(b)
print(type(b))
``````

### Careful!

If the Tensor is on the CPU (not the GPU), both objects will share the same memory location, so changing one will also change the other:

``````a.add_(1)
print(a)
print(b)
``````

### numpy to torch with `.from_numpy(x)

``````import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
print(a)
print(b)
``````

### Again be careful when modifying

``````a += 1
print(a)
print(b)
``````

## Move Tensors to GPU

By default all tensors are created on the CPU, but you can also move them to the GPU (only if it's available )

``````if torch.cuda.is_available():
device = torch.device("cuda")          # a CUDA device object
y = torch.ones_like(x, device=device)  # directly create a tensor on GPU
x = x.to(device)                       # or just use strings ``.to("cuda")``
z = x + y
# z = z.numpy() # not possible because numpy cannot handle GPU tenors
# move to CPU again
z.to("cpu")       # ``.to`` can also change dtype together!
# z = z.numpy()
``````