Skip to content

Tensor Basics - PyTorch Beginner 02

This part covers the basics of Tensors and Tensor operations in PyTorch. Learn also how to convert from numpy data to PyTorch tensors and vice versa!

Learn all the basics you need to get started with this deep learning framework! This part covers the basics of Tensors and Tensor operations in PyTorch. Learn also how to convert from numpy data to PyTorch tensors and vice versa!

All code from this course can be found on GitHub.


Everything in PyTorch is based on Tensor operations. A tensor can have different dimensions, so it can be 1d (scalar), 2d (vector), or even 3d (matrix) and higher. Let's have a look how we can create a tensor in PyTorch.

import torch

# torch.empty(size): uninitiallized
x = torch.empty(1) # scalar
x = torch.empty(3) # vector, 1D
x = torch.empty(2,3) # matrix, 2D
x = torch.empty(2,2,3) # tensor, 3 dimensions
#x = torch.empty(2,2,2,3) # tensor, 4 dimensions

# torch.rand(size): random numbers [0, 1]
x = torch.rand(5, 3)

# torch.zeros(size), fill with 0
# torch.ones(size), fill with 1
x = torch.zeros(5, 3)

# check size

# check data type

# specify types, float32 default
x = torch.zeros(5, 3, dtype=torch.float16)

# check type

# construct from data
x = torch.tensor([5.5, 3])

requires_grad argument

This will tell PyTorch that it will need to calculate the gradients for this tensor later in your optimization steps, i.e., this is a variable in your model that you want to optimize. This will be very important later in our training!

x = torch.tensor([5.5, 3], requires_grad=True)

Operations on Tensors

y = torch.rand(2, 2)
x = torch.rand(2, 2)

# elementwise addition
z = x + y
# torch.add(x,y)

# in place addition, everythin with a trailing underscore is an inplace operation
# i.e. it will modify the variable
# y.add_(x)

# subtraction
z = x - y
z = torch.sub(x, y)

# multiplication
z = x * y
z = torch.mul(x,y)

# division
z = x / y
z = torch.div(x,y)


x = torch.rand(5,3)
print(x[:, 0]) # all rows, column 0
print(x[1, :]) # row 1, all columns
print(x[1,1]) # element at 1, 1

# Get the actual value if only 1 element in your tensor

Reshape with torch.view()

x = torch.randn(4, 4)
y = x.view(16)
z = x.view(-1, 8)  # the size -1 is inferred from other dimensions
# if -1 it pytorch will automatically determine the necessary size
print(x.size(), y.size(), z.size())


Converting a Torch Tensor to a NumPy array and vice versa is very easy

torch to numpy with .numpy()

a = torch.ones(5)

b = a.numpy()


If the Tensor is on the CPU (not the GPU), both objects will share the same memory location, so changing one will also change the other:


numpy to torch with `.from_numpy(x)

import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)

Again be careful when modifying

a += 1

Move Tensors to GPU

By default all tensors are created on the CPU, but you can also move them to the GPU (only if it's available )

if torch.cuda.is_available():
    device = torch.device("cuda")          # a CUDA device object
    y = torch.ones_like(x, device=device)  # directly create a tensor on GPU
    x =                       # or just use strings ``.to("cuda")``
    z = x + y
    # z = z.numpy() # not possible because numpy cannot handle GPU tenors
    # move to CPU again"cpu")       # ``.to`` can also change dtype together!
    # z = z.numpy()

FREE VS Code / PyCharm Extensions I Use

鉁 Write cleaner code with Sourcery, instant refactoring suggestions: Link*

PySaaS: The Pure Python SaaS Starter Kit

馃殌 Build a software business faster with pure Python: Link*

* These are affiliate link. By clicking on it you will not have any additional costs. Instead, you will support my project. Thank you! 馃檹