Tensors

What is a tensor? A tensor is a generalization of vectors and matrices and is easily understood as a multidimensional array. A tensor can be a number, a vector, a matrix, or any n-dimensional array.

Examples

  • 0-d tensor: scalar
  • 1-d tensor: vector
  • 2-d tensor: matrix
  • 3-d tensor: cube
  • n-d tensor: n-d array

Rank

The rank of a tensor is the number of dimensions present within the tensor. For example, the tensor [1, 2, 3] is a rank 1 tensor, while the tensor [[1, 2, 3], [4, 5, 6], [7, 8, 9]] is a rank 2 tensor.

Shape

The shape of a tensor is the number of elements in each dimension. For example, the tensor [[1, 2, 3], [4, 5, 6], [7, 8, 9]] has a shape of (3, 3).

PyTorch tensors

PyTorch tensors are similar to NumPy arrays, but can also be used on a GPU to accelerate computing.

Creating tensors

Directly from data

1
2
3
4
5
6
7


    import torch
    
    t = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float32)
    print(t)
    print(f"shape: {t.shape}")
    print(f"rank: {t.ndimension()}")

gives

1
2
3
4
5
6


tensor([[1., 2., 3.],
        [4., 5., 6.],
        [7., 8., 9.]])
shape: torch.Size([3, 3])
rank: 2

Random

1
2
3
4
5


    import torch
    
    t = torch.rand(3, 3)
    print(t)

gives

1
2
3
4
5


tensor([[0.6242, 0.6879, 0.8598],
        [0.7353, 0.8342, 0.7261],
        [0.3044, 0.2123, 0.7667]])

Operations

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


	import torch
	
	t1 = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float32)
	t2 = torch.tensor([[9, 8, 7], [6, 5, 4], [3, 2, 1]], dtype=torch.float32)
	
	print(t1 + t2)
	print(t1 - t2)
	print(t1 * t2)
	print(t1 / t2)
	
	print(t1 @ t2) # matrix multiplication (dot product)

gives

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


tensor([[10., 10., 10.],
        [10., 10., 10.],
        [10., 10., 10.]])
tensor([[-8., -6., -4.],
        [-2.,  0.,  2.],
        [ 4.,  6.,  8.]])
tensor([[ 9., 16., 21.],
        [24., 25., 24.],
        [21., 16.,  9.]])
tensor([[0.1111, 0.2500, 0.4286],
        [0.6667, 1.0000, 1.5000],
        [2.3333, 4.0000, 9.0000]])
tensor([[ 30.,  24.,  18.],
        [ 84.,  69.,  54.],
        [138., 114.,  90.]])