Now suppose that for a given input the final output ought to be +1 but
actually is -1. Assume that b is then raised so high that this final
output is corrected. Then commence a gradual decline in b. Various
elements may revert to -1, but until the final output does, no weights
are changed. When the final output does revert to -1, it is due to an
element’s having a sum (weighted sum plus bias) which just passed down
through zero. This then caused a chain effect of changing elements
up to the final element, but presumably this element is the only one
possessing a zero sum. This can then be the signal for the weights
on an element to change—a change of final output from right to wrong
accompanied simultaneously by a zero sum in the element itself.
After such a weight change, the final output will be correct once more
and the bias can again proceed to fall. Before it reaches zero, this
process may occur a number of times throughout the network. When the
bias finally stands at zero with the final output correct, the network
is ready for the next input. Of course if -1 is desired, the bias will
change in the opposite direction.
It is possible that extending the weight change process a little past
the zero bias level may have beneficial results. This might increase
the life expectancy of each learned input-output combination and
thereby reduce the total number of errors. This is because the method
used above can stop the weight correction process so that even though
the final output is correct, some elements whose output are essential
to the final output have sums close to zero, which are easily changed
by subsequent weight changes.
It will be noted that this method conforms to all three considerations
mentioned previously. First, by furnishing each element the same bias,
and by not changing weights until the final output becomes incorrect
with dropping bias, there is a strong tendency to select elements
which, with b = 0, would have sums close to zero. But the size of the
sum in an element is a good measure of the amount of damage done to
an element for other inputs if its current output is to be changed.
Second, it is obvious that each element changed has had a demonstrable
effect on the final output. Finally, there will be a clear tendency to
change only a minimum of elements because changes never occur until the
output clearly requires a change.
Public-domain text, read in full here on John Shaqi.
Reviews
Reviews
No reviews yet
Be the first to share your thoughts on this work.
Join the Discussion
Join the discussion
Sign in to leave a comment or review.
Sign InorCreate an account