Thinking of a neural network like a mathematical equation, a neural network is simply a list of mathematical operations to be applied to an input. The input and output of each operation is a tensor (or more specifically a vector or matrix). Each pair of layers is connected by a list of weights. Each layer has several tensors stored in it. An individual tensor in a layer is called a node. Each node is connected to some or all of the nodes in the next layer by a weight. Each node also has a list of values called biases. The value of each layer is then the out of the activation function of the values of the current layer (called X) multiplied by the weights.