alle RETI NEURALI per neofiti (vai a 2)
is a Neural Network?
||First of all, when we are talking about a neural network, we should
more properly say "artificial neural network" (ANN), because that
is what we mean most of the time. Biological neural networks are
much more complicated than the mathematical models we use for ANNs.
But it is customary to be lazy and drop the "A" or the "artificial".
An Artificial Neural Network (ANN) is an information processing
paradigm that is inspired by the way biological nervous systems,
such as the brain, process information. The key element of this
paradigm is the novel structure of the information processing system.
It is composed of a large number of highly interconnected processing
elements (neurons) working in unison to solve specific problems.
ANNs, like people, learn by example. An ANN is configured for a
specific application, such as pattern recognition or data classification,
through a learning process. Learning in biological systems involves
adjustments to the synaptic connections that exist between the neurons.
This is true of ANNs as well.
Some Other Definitions of a Neural Network include:
According to the DARPA Neural Network Study (1988, AFCEA
International Press, p. 60):
... a neural network is a system composed
of many simple processing elements operating in parallel whose function
is determined by network structure, connection strengths, and the
processing performed at computing elements or nodes.
According to Haykin, S. (1994), Neural Networks: A Comprehensive
Foundation, NY: Macmillan, p. 2:
A neural network is a massively parallel
distributed processor that has a natural propensity for storing
experiential knowledge and making it available for use. It resembles
the brain in two respects:
1.Knowledge is acquired by the network through
a learning process.
2.Interneuron connection strengths known
as synaptic weights are used to store the knowledge.
ANNs have been applied to an increasing number of real-world problems
of considerable complexity. Their most important advantage is in
solving problems that are too complex for conventional technologies
-- problems that do not have an algorithmic solution or for which
an algorithmic solution is too complex to be found. In general,
because of their abstraction from the biological brain, ANNs are
well suited to problems that people are good at solving, but for
which computers are not. These problems includes pattern recognition
and forecasting (which requires the recognition of trends in data).
Why use a neural network?
||Neural networks, with their remarkable ability to derive meaning
from complicated or imprecise data, can be used to extract patterns
and detect trends that are too complex to be noticed by either humans
or other computer techniques. A trained neural network can be thought
of as an "expert" in the category of information it has been given
to analyze. This expert can then be used to provide projections
given new situations of interest and answer "what if" questions.
Other advantages include:
- Adaptive learning: An ability to learn how to do tasks based
on the data given for training or initial experience.
- Self-Organisation: An ANN can create its own organisation
or representation of the information it receives during learning
- Real Time Operation: ANN computations may be carried out in
parallel, and special hardware devices are being designed and
manifactured which take advantage of this capability.
- Fault Tolerance via Redundant Information Coding: Partial
destruction of a network leads to the corresponding degradation
of performance. However, some network capabilites may be retained
even with major network damage.
Neural Networks in Practice
||Given this description of neural networks and how they work, what
real world applications are they suited for? Neural networks have
broad applicability to real world business problems. In fact, they
have already been successfully applied in many industries.
Since neural networks are best at identifying patterns or trends
in data, they are well suited for prediction or forecasting needs
- sales forecasting
- industrial process control
- customer research
- data validation
- risk management
- target marketing
But to give you some more specific examples; ANN are also used in
the following specific paradigms: recognition of speakers in communications;
diagnosis of hepatitis; recovery of telecommunications from faulty
software; interpretation of multimeaning Chinese words; undersea
mine detection; texture analysis; three-dimensional object recognition;
handwritten word recognition; and facial recognition.
Historical Background of Neural Networks
||Neural network simulations appear to be a recent development.
However, this field was established before the advent of computers,
and has survived at least one major setback and several eras. Many
importand advances have been boosted by the use of inexpensive computer
emulations. Following an initial period of enthusiasm, the field
survived a period of frustration and disrepute. During this period
when funding and professional support was minimal, important advances
were made by relatively few reserchers. These pioneers were able
to develop convincing technology which surpassed the limitations
identified by Minsky and Papert. Minsky and Papert, published a
book (in 1969) in which they summed up a general feeling of frustration
(against neural networks) among researchers, and was thus accepted
by most without further analysis. Currently, the neural network
field enjoys a resurgence of interest and a corresponding increase
The history of neural networks that was described above can be divided
into several periods:
First Attempts: There were some initial simulations using
formal logic. McCulloch and Pitts (1943) developed models of neural
networks based on their understanding of neurology. These models
made several assumptions about how neurons worked. Their networks
were based on simple neurons which were considered to be binary
devices with fixed thresholds. The results of their model were simple
logic functions such as "a or b" and "a and b". Another attempt
was by using computer simulations. Two groups (Farley and Clark,
1954; Rochester, Holland, Haibit and Duda, 1956). The first group
(IBM reserchers) maintained closed contact with neuroscientists
at McGill University. So whenever their models did not work, they
consulted the neuroscientists. This interaction established a multidiscilinary
trend which continues to the present day.
Promising & Emerging Technology: Not only was neroscience
influential in the development of neural networks, but psychologists
and engineers also contributed to the progress of neural network
simulations. Rosenblatt (1958) stirred considerable interest and
activity in the field when he designed and developed the Perceptron.
The Perceptron had three layers with the middle layer known as the
association layer.This system could learn to connect or associate
a given input to a random output unit. Another system was the ADALINE
(ADAptive LInear Element) which was developed in 1960 by Widrow
and Hoff (of Stanford University). The ADALINE was an analogue electronic
device made from simple components. The method used for learning
was different to that of the Perceptron, it employed the Least-Mean-Squares
(LMS) learning rule.
Period of Frustration & Disrepute: In 1969 Minsky and
Papert wrote a book in which they generalised the limitations of
single layer Perceptrons to multilayered systems. In the book they
said: "...our intuitive judgment that the extension (to multilayer
systems) is sterile". The significant result of their book was to
eliminate funding for research with neural network simulations.
The conclusions supported the disenhantment of reserchers in the
field. As a result, considerable prejudice against this field was
Innovation: Although public interest and available funding
were minimal, several researchers continued working to develop neuromorphically
based computaional methods for problems such as pattern recognition.
During this period several paradigms were generated which modern
work continues to enhance.Grossberg's (Steve Grossberg and Gail
Carpenter in 1988) influence founded a school of thought which explores
resonating algorithms. They developed the ART (Adaptive Resonance
Theory) networks based on biologically plausible models. Anderson
and Kohonen developed associative techniques independent of each
other. Klopf (A. Henry Klopf) in 1972, developed a basis for learning
in artificial neurons based on a biological principle for neuronal
learning called heterostasis. Werbos (Paul Werbos 1974) developed
and used the back-propagation learning method, however several years
passed before this approach was popularized. Back-propagation nets
are probably the most well known and widely applied of the neural
networks today. In essence, the back-propagation net. is a Perceptron
with multiple layers, a different thershold function in the artificial
neuron, and a more robust and capable learning rule. Amari (A. Shun-Ichi
1967) was involved with theoretical developments: he published a
paper which established a mathematical theory for a learning basis
(error-correction method) dealing with adaptive patern classification.
While Fukushima (F. Kunihiko) developed a step wise trained multilayered
neural network for interpretation of handwritten characters. The
original network was published in 1975 and was called the Cognitron.
Re-Emergence: Progress during the late 1970s and early 1980s
was important to the re-emergence on interest in the neural network
field. Several factors influenced this movement. For example, comprehensive
books and conferences provided a forum for people in diverse fields
with specialized technical languages, and the response to conferences
and publications was quite positive. The news media picked up on
the increased activity and tutorials helped disseminate the technology.
Academic programs appeared and courses were inroduced at most major
Universities (in US and Europe). Attention is now focused on funding
levels throughout Europe, Japan and the US and as this funding becomes
available, several new commercial with applications in industry
and finacial institutions are emerging.
Today: Significant progress has been made in the field of
neural networks-enough to attract a great deal of attention and
fund further research. Advancement beyond current commercial applications
appears to be possible, and research is advancing the field on many
fronts. Neurally based chips are emerging and applications to complex
problems developing. Clearly, today is a period of transition for
neural network technology.
Are there any limits to Neural Networks?
||The major issues of concern today are the scalability problem,
testing, verification, and integration of neural network systems
into the modern environment. Neural network programs sometimes become
unstable when applied to larger problems. The defence, nuclear and
space industries are concerned about the issue of testing and verification.
The mathematical theories used to guarantee the performance of an
applied neural network are still under development. The solution
for the time being may be to train and test these intelligent systems
much as we do for humans. Also there are some more practical problems
- the operational problem encountered when attempting to simulate
the parallelism of neural networks. Since the majority of neural
networks are simulated on sequential machines, giving rise to
a very rapid increase in processing time requirements as size
of the problem expands. Solution: implement neural networks
directly in hardware, but these need a lot of development still.
- instability to explain any results that they obtain. Networks
function as "black boxes" whose rules of operation are completely
||Because gazing into the future is somewhat like gazing into a
crystal ball, so it is better to quote some "predictions". Each
prediction rests on some sort of evidence or established trend which,
with extrapolation, clearly takes us into a new realm.
Neural Networks will fascinate user-specific systems for education,
information processing, and entertainment. "Alternative ralities",
produced by comprehensive environments, are attractive in terms
of their potential for systems control, education, and entertainment.
This is not just a far-out research trend, but is something which
is becoming an increasing part of our daily existence, as witnessed
by the growing interest in comprehensive "entertainment centers"
in each home. This "programming" would require feedback from the
user in order to be effective but simple and "passive" sensors (e.g
fingertip sensors, gloves, or wristbands to sense pulse, blood pressure,
skin ionisation, and so on), could provide effective feedback into
a neural control system. This could be achieved, for example, with
sensors that would detect pulse, blood pressure, skin ionisation,
and other variables which the system could learn to correlate with
a person's response state.
Neural networks, integrated with other artificial intelligence technologies,
methods for direct culture of nervous tissue, and other exotic technologies
such as genetic engineering, will allow us to develop radical and
exotic life-forms whether man, machine, or hybrid.
Neural networks will allow us to explore new realms of human capabillity
realms previously available only with extensive training and personal
discipline. So a specific state of consiously induced neurophysiologically
observable awareness is necessary in order to facilitate a man machine
Klimasauskas, CC. (1989). The 1989 Neuro Computing Bibliography.
Hammerstrom, D. (1986). A Connectionist/Neural Network Bibliography.
DARPA Neural Network Study (October, 1987-February, 1989). MIT Lincoln
Lab. Neural Networks, Eric Davalo and Patrick Naim. Prof. Aleksander.
articles and Books. (from Imperial College) WWW pages through out
the internet Assimov, I (1984, 1950), Robot, Ballatine, New York.
current news from multimedia services (Tv)