Forwardpropagation

Forwardpropagation is a supervised learning algorithm and describes the "flow of information" through a neural net from its input layer to its output layer.

The algorithm works as follows:

1. Set all weights to random values ranging from -1.0 to +1.0
2. Set an input pattern (binary values) to the neurons of the net's input layer
3. Activate each neuron of the following layer: - Multiply the weight values of the connections leading to this neuron with the output values of the preceding neurons - Add up these values - Pass the result to an activation function, which computes the output value of this neuron
4. Repeat this until the output layer is reached
5. Compare the calculated output pattern to the desired target pattern and compute an error value
6. Change all weights by adding the error value to the (old) weight values
7. Go to step 2
8. The algorithm ends, if all output patterns match their target patterns

Example:

Suppose you have the following 2-layered Perceptron:

Patterns to be learned:

First, the weight values are set to random values (0.35 and 0.81).

The learning rate of the net is set to 0.25.

Next, the values of the first input pattern (0 1) are set to the neurons of the input layer (the output of the input layer is the same as its input).

The neurons in the following layer (only one neuron in the output layer) are activated:

Input 1 of output neuron:       0 * 0.35 = 0
Input 2 of output neuron:       1 * 0.81 = 0.81
Add the inputs:                 0 + 0.81 = 0.81            (= output)
Compute an error value by
subtracting output from target: 0 - 0.81 = -0.81
Value for changing weight 1:    0.25 * 0 * (-0.81) = 0     (0.25 = learning rate)
Value for changing weight 2:    0.25 * 1 * (-0.81) = -0.2025
Change weight 1:                0.35 + 0         = 0.35    (not changed)
Change weight 2:                0.81 + (-0.2025) = 0.6075

Now that the weights are changed, the second input pattern (1 1) is set to the input layer's neurons and the activation of the output neuron is performed again, now with the new weight values:

Input 1 of output neuron:       1 * 0.35   = 0.35
Input 2 of output neuron:       1 * 0.6075 = 0.6075
Add the inputs:                 0.35 + 0.6075 = 0.9575     (= output)
Compute an error value by
subtracting output from target: 1 - 0.9575 = 0.0425
Value for changing weight 1:    0.25 * 1 * 0.0425 = 0.010625
Value for changing weight 2:    0.25 * 1 * 0.0425 = 0.010625
Change weight 1:                0.35   + 0.010625 = 0.360625
Change weight 2:                0.6075 + 0.010625 = 0.618125

That was one learning step. Each input pattern had been propagated through the net and the weight values were changed.

The error of the net can now be calculated by adding up the squared values of the output errors of each pattern:

Compute the net error:          (-0.81)2 + (0.0425)2 = 0.65790625

By performing this procedure repeatedly, this error value gets smaller and smaller.

The algorithm is successfully finished, if the net error is zero (perfect) or approximately zero.