Forwardpropagation is a supervised learning algorithm and describes the "flow of information" through a neural net from its input layer to its output layer.
The algorithm works as follows:
Example:
Suppose you have the following 2-layered Perceptron:
Patterns to be learned:
input | target |
0 1 | 0 |
1 1 | 1 |
First, the weight values are set to random values (0.35 and 0.81).
The learning rate of the net is set to 0.25.
Next, the values of the first input pattern (0 1) are set to the neurons of the input layer (the output of the input layer is the same as its input).
The neurons in the following layer (only one neuron in the output layer) are activated:
Input 1 of output neuron: 0 * 0.35 = 0 Input 2 of output neuron: 1 * 0.81 = 0.81 Add the inputs: 0 + 0.81 = 0.81 (= output) Compute an error value by subtracting output from target: 0 - 0.81 = -0.81 Value for changing weight 1: 0.25 * 0 * (-0.81) = 0 (0.25 = learning rate) Value for changing weight 2: 0.25 * 1 * (-0.81) = -0.2025 Change weight 1: 0.35 + 0 = 0.35 (not changed) Change weight 2: 0.81 + (-0.2025) = 0.6075
Now that the weights are changed, the second input pattern (1 1) is set to the input layer's neurons and the activation of the output neuron is performed again, now with the new weight values:
Input 1 of output neuron: 1 * 0.35 = 0.35 Input 2 of output neuron: 1 * 0.6075 = 0.6075 Add the inputs: 0.35 + 0.6075 = 0.9575 (= output) Compute an error value by subtracting output from target: 1 - 0.9575 = 0.0425 Value for changing weight 1: 0.25 * 1 * 0.0425 = 0.010625 Value for changing weight 2: 0.25 * 1 * 0.0425 = 0.010625 Change weight 1: 0.35 + 0.010625 = 0.360625 Change weight 2: 0.6075 + 0.010625 = 0.618125
That was one learning step. Each input pattern had been propagated through the net and the weight values were changed.
The error of the net can now be calculated by adding up the squared values of the output errors of each pattern:
Compute the net error: (-0.81)2 + (0.0425)2 = 0.65790625
By performing this procedure repeatedly, this error value gets smaller and smaller.
The algorithm is successfully finished, if the net error is zero (perfect) or approximately zero.