What is an Artificial Neural Network (ANN)?

- The artificial neural network or ANN (sometimes also called as just neural network) is a mathematical model that has got its inspiration from the biological neural networks. 

- This network is supposed to consist of several artificial neurons that are interconnected. 

- This model works with a connectionist approach for computing and thus processes information based up on this only. 

- In a number of cases, the neural network can act as an adaptive system that has the ability of making changes in its structure while it is in some learning phase. 

- These networks are particularly used in searching patterns in data and for modeling the complex relationships that exist between the outputs and inputs. 

- An analogy to artificial neural network is the neuron network of the human brain. 

- In an ANN, the artificial nodes are termed as the neurons or sometimes as neurodes or units or the ‘processing elements’. 

- They are interconnected in such a way that they resemble a biological neural network. 

- Till now, no formal definition has been given for the artificial neural networks. - These processing elements or the neurons show a complex global behavior. 

- The connections between the neurons and their parameters is what that determines this behavior.

- There are certain algorithms that are designed for altering the strength of these connections in order to produce the desired flow of the signal. 

- The ANN operates up on these algorithms. 

- As in biological neural networks, in ANN also functions are performed in parallel and collectively by the processing units.

- Here, there is no delineation of the tasks that might be assigned to different units. 

- These neural networks are employed in various fields such as:

1. Statistics
2. Cognitive psychology
3. Artificial intelligence

- There are other neural network models that emulate biological CNS and are part of the following:

1. Computational neuroscience
2. Theoretical neuroscience

- The modern software implementation of the ANNs prefers a more practical approach than biologically inspired approach. 

- This practical approach is based up on the signal processing and statistics. The former approach has been largely abandoned. 

- Many times parts of these neural networks serve as components for the other larger systems that are a combination of non – adaptive and adaptive elements.

- Even though a more practical approach for solving the real world problems is the latter one, the former has more to do with the connectionist models of the traditional artificial intelligence. 

- Well the common thing between them is the principle of distributed, non – linear, local and parallel processing and adaptation. 

- A paradigm shift was marked by the use of neural networks during the late eighties. 

- This shift was from the high level artificial intelligence (expert systems) to low level machine learning (dynamical system). 

- These models are very simple and define functions such as:

f: X à Y

- Three types of parameters are used for defining an artificial neural network:

a)   The interconnection pattern between neuron layers

b)   The learning process

c)   The activation function

- The second parameter updates the weights of the connections and the third one converts the weighted input in to output. 

- Learning is the thing that has attracted many towards it. 

- There are 3 major learning paradigms that are offered by ANN:

1. Supervised learning
2. Un – supervised learning
3. Reinforcement learning

- Training a network requires selecting from a set of models that would best minimize the cost.

- A number of algorithms are available for training purpose where gradient descent is employed by most of the algorithms.

- Other methods available are simulated annealing, evolutionary methods and so on.