7.2 KiB
PyTorch Fundamentals
# Import pytorch in our codespace
import torch
print(torch.__version__)
Output
2.3.0+cu121
2.3.0 is the pytorch version and 121 is the cuda version
Now you have already seen how to create a tensor in pytorch. In this notebook i am going to show you the operations which can be applied on a tensor with a quick previous revision.
1. Creating tensors
Scalar tensor ( a zero dimension tensor)
scalar = torch.tensor(7)
print(scalar)
Output
tensor(7)
Check the dimension of the above tensor
print(scalar.ndim)
Output
0
To retrieve the number from the tensor we use item()
print(scalar.item())
Output
7
Vector (It is a single dimension tensor but contain many numbers)
vector = torch.tensor([1,2])
print(vector)
Output
tensor([1, 2])
Check the dimensions
print(vector.ndim)
Output
1
Check the shape of the vector
print(vector.shape)
Output
torch.Size([2])
The above returns torch.Size([2]) which means our vector has a shape of [2]. This is because of the two elements we placed inside the square brackets ([1,2])
Note: I'll let you in on a trick.
You can tell the number of dimensions a tensor in PyTorch has by the number of square brackets on the outside ([) and you only need to count one side.
# Let's create a matrix
MATRIX = torch.tensor([[1,2],
[4,5]])
print(MATRIX)
Output
tensor([[1, 2],
[4, 5]])
There are two brackets so it must be 2 dimensions , lets check
print(MATRIX.ndim)
Output
2
# Shape
print(MATRIX.shape)
Output
torch.Size([2, 2])
It means MATRIX has 2 rows and 2 columns.
Let's create a TENSOR
TENSOR = torch.tensor([[[1,2,3],
[4,5,6],
[7,8,9]]])
print(TENSOR)
Output
tensor([[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]])
Let's check the dimensions
print(TENSOR.ndim)
Output
3
shape
print(TENSOR.shape)
Output
torch.Size([1, 3, 3])
The dimensions go outer to inner.
That means there's 1 dimension of 3 by 3.
Let's summarise
- scalar -> a single number having 0 dimension.
- vector -> have many numbers but having 1 dimension.
- matrix -> a array of numbers having 2 dimensions.
- tensor -> a array of numbers having n dimensions.
Random Tensors
We can create them using torch.rand() and passing in the size parameter.
Creating a random tensor of size (3,4)
rand_tensor = torch.rand(size = (3,4))
print(rand_tensor)
Output
tensor([[0.7462, 0.4950, 0.7851, 0.8277],
[0.6112, 0.5159, 0.1728, 0.6847],
[0.4472, 0.1612, 0.6481, 0.3236]])
Check the dimensions
print(rand_tensor.ndim)
Output
2
Shape
print(rand_tensor.shape)
Output
torch.Size([3, 4])
Datatype
print(rand_tensor.dtype)
Output
torch.float32
Zeros and ones
Here we will create a tensor of any shape filled with zeros and ones
# Create a tensor of all zeros
zeros = torch.zeros(size = (3,4))
print(zeros)
Output
tensor([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]])
Create a tensor of ones
ones = torch.ones(size = (3,4))
print(ones)
Output
tensor([[1., 1., 1., 1.],
[1., 1., 1., 1.],
[1., 1., 1., 1.]])
Create a tensor having range of numbers
You can use torch.arange(start, end, step) to do so.
Where:
- start = start of range (e.g. 0)
- end = end of range (e.g. 10)
- step = how many steps in between each value (e.g. 1)
Note: In Python, you can use range() to create a range. However in PyTorch, torch.range() is deprecated show error, show use
torch.arange()
zero_to_ten = torch.arange(start = 0,
end = 10,
step = 1)
print(zero_to_ten)
Output
tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
2. Manipulating tensors (tensor operations)
The operations are :
- Addition
- Substraction
- Multiplication (element-wise)
- Division
- Matrix multiplication
1. Addition
tensor = torch.tensor([1,2,3])
print(tensor+10)
Output
tensor([11, 12, 13])
We have add 10 to each tensor element.
tensor1 = torch.tensor([4,5,6])
print(tensor+tensor1)
Output
tensor([5, 7, 9])
We have added two tensors , remember that addition takes place element wise.
2. Subtraction
print(tensor-8)
Output
tensor([-7, -6, -5])
We've subtracted 8 from the above tensor.
print(tensor-tensor1)
Output
tensor([-3, -3, -3])
3. Multiplication
# Multiply the tensor with 10 (element wise)
print(tensor*10)
Output
tensor([10, 20, 30])
Each element of tensor gets multiplied by 10.
Note:
PyTorch also has a bunch of built-in functions like torch.mul() (short for multiplication) and torch.add() to perform basic operations.
# let's see them
print(torch.add(tensor,10))
Output
tensor([11, 12, 13])
print(torch.mul(tensor,10))
Output
tensor([10, 20, 30])
Matrix multiplication (is all you need)
One of the most common operations in machine learning and deep learning algorithms (like neural networks) is matrix multiplication.
PyTorch implements matrix multiplication functionality in the torch.matmul() method.
The main two rules for matrix multiplication to remember are:
The inner dimensions must match:
- (3, 2) @ (3, 2) won't work
- (2, 3) @ (3, 2) will work
- (3, 2) @ (2, 3) will work The resulting matrix has the shape of the outer dimensions:
- (2, 3) @ (3, 2) -> (2, 2)
- (3, 2) @ (2, 3) -> (3, 3)
Note: "@" in Python is the symbol for matrix multiplication.
# let's perform the matrix multiplication
tensor1 = torch.tensor([[[1,2,3],
[4,5,6],
[7,8,9]]])
tensor2 = torch.tensor([[[1,1,1],
[2,2,2],
[3,3,3]]])
print(tensor1) , print(tensor2)
Output
tensor([[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]])
tensor([[[1, 1, 1],
[2, 2, 2],
[3, 3, 3]]])
Let's check the shape
print(tensor1.shape) , print(tensor2.shape)
Output
torch.Size([1, 3, 3])
torch.Size([1, 3, 3])
Matrix multiplication
print(torch.matmul(tensor1, tensor2))
Output
tensor([[[14, 14, 14],
[32, 32, 32],
[50, 50, 50]]])
Can also use the "@" symbol for matrix multiplication, though not recommended
print(tensor1 @ tensor2)
Output
tensor([[[14, 14, 14],
[32, 32, 32],
[50, 50, 50]]])
Note:
If shape is not perfect you can transpose the tensor and perform the matrix multiplication.