## Take advantage of the overloaded operators
Just like NumPy, TensorFlow overloads a number of python operators to make building graphs easier and the code more readable.
The slicing op is one of the overloaded operators that can make indexing tensors very easy:
z = x[begin:end] # z = tf.slice(x, [begin], [end-begin])
Be very careful when using this op though. The slicing op is very inefficient and often better avoided, especially when the number of slices is high. To understand how inefficient this op can be let's look at an example. We want to manually perform reduction across the rows of a matrix:
import tensorflow as tf
x = tf.random.uniform([500, 10])
z = tf.zeros()
start = time.time()
for i in range(500):
z += x[i]
print("Took %f seconds." % (time.time() - start))
On my MacBook Pro, this took 0.045 seconds to run which is quite slow. The reason is that we are calling the slice op 500 times, which is going to be very slow to run. A better choice would have been to use tf.unstack op to slice the matrix into a list of vectors all at once:
z = tf.zeros()
for x_i in tf.unstack(x):
z += x_i
This took 0.01 seconds. Of course, the right way to do this simple reduction is to use tf.reduce_sum op:
z = tf.reduce_sum(x, axis=0)
This took 0.0001 seconds, which is 100x faster than the original implementation.
TensorFlow also overloads a range of arithmetic and logical operators:
z = -x # z = tf.negative(x)
z = x + y # z = tf.add(x, y)
z = x - y # z = tf.subtract(x, y)
z = x * y # z = tf.mul(x, y)
z = x / y # z = tf.div(x, y)
z = x // y # z = tf.floordiv(x, y)
z = x % y # z = tf.mod(x, y)
z = x ** y # z = tf.pow(x, y)
z = x @ y # z = tf.matmul(x, y)
z = x > y # z = tf.greater(x, y)
z = x >= y # z = tf.greater_equal(x, y)
z = x < y # z = tf.less(x, y)
z = x <= y # z = tf.less_equal(x, y)
z = abs(x) # z = tf.abs(x)
z = x & y # z = tf.logical_and(x, y)
z = x | y # z = tf.logical_or(x, y)
z = x ^ y # z = tf.logical_xor(x, y)
z = ~x # z = tf.logical_not(x)
You can also use the augmented version of these ops. For example `x += y` and `x **= 2` are also valid.
Note that Python doesn't allow overloading "and", "or", and "not" keywords.
Other operators that aren't supported are equal (==) and not equal (!=) operators which are overloaded in NumPy but not in TensorFlow. Use the function versions instead which are `tf.equal` and `tf.not_equal`.