On the other hand this method requires little more added complexity to
the network than it already has. Each element requires a bias, an error
signal, and the desired final output, these things being uniform for
all elements in a network. Some external device must manipulate the
bias properly, but this is a simple behavior depending only on an error
signal and the desired final output—not on the state of individual
elements in the network. What one has, then, is a network consisting
of elements which are nearly autonomous as regards their decisions
to change weights. Such a scheme appears to be the only way to avoid
constructing a central weight-change decision apparatus of great
complexity. This rather sophisticated decision is made possible by
utilizing the computational capabilities the network already possesses
in producing outputs from inputs.
It should be noted here that this varying bias method requires that
the variable bias be furnished to just those elements which have
variable weights and to no others. Any fixed portion of the network,
such as preliminary layers or final majority function for example,
must operate independently of the variable bias. Otherwise, the final
output may go from right to wrong as the bias moves towards zero and no
variable-weight element be to blame. In such a case the network would
be hung up.
Logical Redundancy in the Network
A third aspect of the network model is that for all the care taken
in the previous steps, they will not suffice in settling quickly to
a set of weights that will generate the required logical function
unless there is a great multiplicity of ways in which this can be done.
This is to say that a learning network needs to have an excess margin
of weights and elements beyond the minimum required to generate the
functions which are to be learned.
This is analogous to the situation that prevails for a single element
as regards the allowed range of values on its weights. It can be shown
for example, that any function for n=6 that can be generated by a
single element can be obtained with each weight restricted to the range
of integer values -9,-8, ..., +9. Yet no modification of the stated
weight change rule is known which restricts weight values to these and
yet has any chance of ever being learned for most functions.
Fatigued Elements
Public-domain text, read in full here on John Shaqi.
Reviews
Reviews
No reviews yet
Be the first to share your thoughts on this work.
Join the Discussion
Join the discussion
Sign in to leave a comment or review.
Sign InorCreate an account