It would appear from some of the preliminary results of network
simulations that it may be useful to have elements become “fatigued”
after undergoing an excessive number of weight changes. Experiments
have been performed on simplifications of the model described so far
which had the occasional result that a small number of elements came
to a state where they received most of the weight increments, much
to the detriment of the learning process. In such cases the network
behaves as if it were composed of many fewer adjustable elements. In a
sense this is asking each element to maintain a record of the data it
is being asked to store so that it does not attempt to exceed its own
information capacity.
It is not certain just how this fatigue factor should enter in the
element’s actions, but if it is to be compatible with the variable bias
method, this fatigue factor must enter into the element’s response to
a changing bias. Once an element changes state with zero sum at the
same time that the final output becomes wrong, incrementing must occur
if the method is to work. Hence a “fatigued” element must respond less
energetically to a change of bias, perhaps with a kind of variable
factor to be multiplied by the bias term.
NETWORK STRUCTURE
It is felt that the problem of selecting the structure of
interconnections for a network is intimately connected to the
previously mentioned problem of generalization. Presumably a given
type of generalization can be obtained by providing appropriate fixed
portions of the network and an appropriate interconnection structure
for the variable portion. However, for very large networks, it is
undoubtedly necessary to restrict the complexity so that it can be
specified by relatively simple rules. Since very little is known about
this quite important problem, no further discussion will be attempted
here.
COMPUTER SIMULATION RESULTS
A computer simulation of some of the network features previously
described has been made on an IBM 7090. Networks with an excess of
elements and with only positive interconnecting weights were used.
However, in place of the variable bias method, a simple choice of the
element of sum closest to, and on the wrong side of, zero was made
without regard to the effectiveness of the element in correcting the
final output. No fatigue factors were used.
The results of these simulations are very encouraging, but at the same
time indicate the need for the more sophisticated methods. No attempt
will be made here to describe the results completely.
Public-domain text, read in full here on John Shaqi.
Reviews
Reviews
No reviews yet
Be the first to share your thoughts on this work.
Join the Discussion
Join the discussion
Sign in to leave a comment or review.
Sign InorCreate an account