Learning in neural networks

Gradual change in weights
Unsupervised learning of weights
Supervised (error-driven) learning of weights
- Change in a weight depends on
  - The error for the output unit: usually the difference between the target value and the activation of that unit
  - The activation of the input unit
  - A learning rate (η)
- The delta rule for updating the weight from unit i to unit j in a network with linear output units:
  `w_{ji}^{text{new}} = w_{ji}^{text{old}} + eta (t_j - x_j) x_i`
  or
  `Delta w_{ji} = eta (t_j - x_j) x_i`

Example: AND

The problem
- Activation function: identity function
- Learning rate: 0.5
- Targets: 1 for input [1, 1]; 0 or negative for other inputs
- Initial weights
  
  0 1 bias
  
  0.0 0.0 0.0
Training
- Input pattern: 0 0 1, output: 0, target: 0; error: 0; no weight changes
- Input pattern: 1 0 1, output: 0, target: 0; error: 0; no weight changes
- Input pattern: 1 1 1, output: 0, target: 1; error: 1
  
  0 1 bias
  
  0.5 0.5 0.5
- Input pattern: 0 1 1, output: 1, target: 0; error: -1
  
  0 1 bias
  
  0.5 0.0 0.0
- Input pattern: 1 0 1, output: .5, target: 0; error: -0.5
  
  0 1 bias
  
  0.25 0.0 -0.25
- Input pattern: 0 0 1, output: -0.25, error: 0
- Input pattern: 1 1 1, output: 0, target: 1; error: 1
  
  0 1 bias
  
  0.75 0.75 0.25
- Input pattern: 0 0 1, output: 0.25, target: 0; error: -0.25
  
  0 1 bias
  
  0.75 0.75 0.125
- Input pattern: 1 0 1, output: 0.875, target: 0; error: -0.875
  
  0 1 bias
  
  0.31 0.75 -0.32

0	1	bias
0.0	0.0	0.0

0	1	bias
0.5	0.5	0.5

0	1	bias
0.5	0.0	0.0

0	1	bias
0.25	0.0	-0.25

0	1	bias
0.75	0.75	0.25

0	1	bias
0.75	0.75	0.125

0	1	bias
0.31	0.75	-0.32

An example of a set of weights

You want a network that can take inputs representing objects in a scene and count them. Say each object is represented as the activation of a single unit at its center of mass, and the output numbers are represented using the "thermometer" encoding described here. Say the output units are binary threshold units which turn on when their input is greater than 0.
Since there are no distinctions among the input units, the weights out of each input unit should be the same.
The output unit representing "one" should turn on only when one or more input units are on. Therefore the weights into it from the input units should be positive, say, +1.
The second output unit should turn on only when two or more input units are on. Again all of the weights from the input units should be positive, say, +1. But now we have to prevent the unit from turning on when only one unit is on. With the weight from the bias unit set to -1.5, this will happen.
Similarly, with weights of +1 from all of the input units to all of the output units, the bias weight into the third unit could be -2.5, into the fourth unit -3.5, etc.

Activation functions

The simplest activation function is the identity function: the activation is just the input. The derivative of this function is 1.
Another possibility is a threshold function which converts inputs above some threshold to a maximum value (usually 1.0) and inputs below the threshold to a minimum values (usually -1.0 or 0.0). But this function is not differentiable everywhere.
A third possibility is a "soft" threshold function which smooths out the region near the threshold.
- The sigmoid function has a minimum of 0.0 and a maximum of 1.0; note that these are asymptotes. (h is the input to the unit; β the gain controlling the steepness of the slope.)
  `g(h) = 1 / (1 + e^{-beta h})` \begin{graph} width=300; height=200; xmin=-5.2; xmax=5.2; ymin=-0.1; ymax=+1.1; xscl=1; yscl=.2; plot(1/(1+e^(-x))); plot(1/(1+e^(-.5x))); plot(1/(1+e^(-5x))) \end{graph}
- An alternative to the sigmoid, which ranges from -1 to +1, is the hyperbolic tangent:
  `g(h) = tanh(h) = (e^{beta h} - e^{-beta h}) / (e^{beta h} + e^{-beta h})`
  \begin{graph} width=300; height=200; xmin=-5.2; xmax=5.2; ymin=-1.1; ymax=+1.1; xscl=1; yscl=.2; plot(tanh(.5x)); plot(tanh(x)); plot(tanh(5x)) \end{graph}
- Both the sigmoid and the hyperbolic tangent are differentiable and have the useful property that their derivatives are functions of their outputs.
  `sigma prime(h) = (e^{-h})/(1 + e^{-h})^2 = sigma(h)(1 - sigma(h))`
  `tanh prime(h) = 4/(e^h + e^{-h})^2 = 1 - tanh^2(h)`

Derivation of the delta rule

Gradient descent learning: learning by moving in the direction which looks locally to be the best
- For supervised neural network learning, the best direction to move in "weight space": for each weight, how the global error changes with respect to that weight
- A global error function: for each pattern the sum of the errors over all of the output units
  `E = sum_j 1/2 (t_j - x_j)^2`
- Move in "weight space" in a direction opposite that of the slope of the error function with respect to each weight
- Size of the move should be proportional to the magnitude of the slope.
- Slope: partial derivative of the error with respect to the weight.
Derivation
- Unit activation
  `x_j(t+1) = g(h_j(t)) = g[h(vec x(t), vec w_j(t))]`
- Only element in the sum of error terms that depends on the weight is the one for the output unit where that weight ends (j).
  (1) `(del E) / (del w_{ji})`
- Using the chain rule, we can decompose this derivative into two that are easier to calculate:
  (2) `(del E)/(del h_j) (del h_j)/(del w_{ji})`
- The first derivative can also be decomposed:
  (3) `(del E)/(del h_j) = (del E)/(del x_j) (del x_j)/(del h_j)`
- The first derivative in 3 is easy to figure; it's just
  (4) `(del E)/(del x_j) = -(t_j - x_j)`
- Since the activation of an output unit is the activation function g applied to the input h, the second derivative in 3 is just
  (5) `(del x_j)/(del h_j) = g prime(h_j)`
  that is, the derivative of whatever the activation function is at the value of the current input to unit j.
- The second derivative in (2) can be figured as follows:
  (6) `(del h_j)/(del w_{ji}) = (del sum_k x_k w_{jk})/(del w_{ji}) = x_i`
  because none of the other weights or input activations depend on w_ji.
- Putting all of the parts together, we get
  (7) `(del E)/(del w_{ji}) = -(t_j - x_j) g prime(h_j) x_i`
- Remember that we want the weight change to be proportional to the negative of the derivative with respect to the weight. So with a learning rate to control the step size for weight changes, we get the more general delta (least mean squares) learning rule
  (8) `Delta w_{ji} = eta (t_j - x_j) g prime(h_j) x_i`
- With a linear activation function, the weight change reduces to `eta (t_j - x_j) x_i`. With the sigmoid, it becomes `eta (t_j - x_j) x_j(1 - x_j) x_i`, with tanh, `eta (t_j - x_j) (1 - x_j^2) x_i`.

Back-propagation

Back-propagation (Parker; Werbos; Rumelhart, Hinton, and Williams)
- A gradient descent algorithm for learning the weights into hidden units as well as output units
- A network with hidden units with linear activation functions is equivalent to a network with no hidden units
The learning rule:
`Delta w_{ji} = eta delta_j x_i`
- For output units:
  `delta_j = (t_j - x_j) g prime(h_j)`
- For hidden units (k indexes the units in the next highest layer):
  `delta_j = (sum_k delta_k w_{kj}) g prime(h_j)`
Derivation
- As with the delta rule, we take the partial derivative of the error with respect to the weight and decompose into two derivatives using the chain rule:
  (1) `(del E)/(del w_{ji}) = (del E)/(del h_j) (del h_j)/(del w_{ji}) = (del E)/(del h_j) x_i`
- As before, the first derivative can also be decomposed; we'll call the negative of this δ:
  (2) `delta_j -= -(del E)/(del h_j) = -(del E)/(del x_j) (del x_j)/(del h_j) = -(del E)/(del x_j) g prime(h_j)`
- For output units,
  (3) `(del E)/(del x_j) = (del (sum_j 1/2 (t_j - x_j)^2))/(del x_j) = -(t_j - x_j)`
- For hidden units, we can use the chain rule once more:
  (4) `(del E)/(del x_j) = sum_k [(del E)/(del h_k) (del h_k)/(del x_j)] = sum_k [(del E)/(del h_k) w_{kj}] = -sum_k delta_k w_{kj}`