## Take advantage of the overloaded operators Just like NumPy, PyTorch overloads a number of python operators to make PyTorch code shorter and more readable. The slicing op is one of the overloaded operators that can make indexing tensors very easy: ```python z = x[begin:end] # z = torch.narrow(0, begin, end-begin) ``` Be very careful when using this op though. The slicing op, like any other op, has some overhead. Because it's a common op and innocent looking it may get overused a lot which may lead to inefficiencies. To understand how inefficient this op can be let's look at an example. We want to manually perform reduction across the rows of a matrix: ```python import torch import time x = torch.rand([500, 10]) z = torch.zeros([10]) start = time.time() for i in range(500): z += x[i] print("Took %f seconds." % (time.time() - start)) ``` This runs quite slow and the reason is that we are calling the slice op 500 times, which adds a lot of overhead. A better choice would have been to use `torch.unbind` op to slice the matrix into a list of vectors all at once: ```python z = torch.zeros([10]) for x_i in torch.unbind(x): z += x_i ``` This is significantly (~30% on my machine) faster. Of course, the right way to do this simple reduction is to use `torch.sum` op to this in one op: ```python z = torch.sum(x, dim=0) ``` which is extremely fast (~100x faster on my machine). PyTorch also overloads a range of arithmetic and logical operators: ```python z = -x # z = torch.neg(x) z = x + y # z = torch.add(x, y) z = x - y z = x * y # z = torch.mul(x, y) z = x / y # z = torch.div(x, y) z = x // y z = x % y z = x ** y # z = torch.pow(x, y) z = x @ y # z = torch.matmul(x, y) z = x > y z = x >= y z = x < y z = x <= y z = abs(x) # z = torch.abs(x) z = x & y z = x | y z = x ^ y # z = torch.logical_xor(x, y) z = ~x # z = torch.logical_not(x) z = x == y # z = torch.eq(x, y) z = x != y # z = torch.ne(x, y) ``` You can also use the augmented version of these ops. For example `x += y` and `x **= 2` are also valid. Note that Python doesn't allow overloading `and`, `or`, and `not` keywords.