What are types of artificial neural networks?

In this article we discuss the types of artificial neural networks. These models simulate the real life biological system of nervous system.

1. Feed forward neural network: 

- This is the simplest type of neural network that has been ever devised. 

- In these networks the information flow is unidirectional; therefore the data moves only in forward direction. 

- From input nodes data flows to the output nodes via hidden nodes (if there are any). 

- In this model there are no loops or cycles. 

- Different types of units can be used for constructing feed forward networks for example, McCulloch – pitts neurons.

- Continuous neurons are used in error back propagation along with the sigmoidal activation.

2. Radial basis function network: 

- For interpolating in a multi – dimensional space radial basis functions are the most powerful tools. 

- These functions can be built in to criterion of distance with respect to some center.

- These functions can be applied in the neural networks. 

- In these networks, sigmoidal hidden layer transfer characteristic can be replaced by these functions.

3. Kohonen self–organization network: 

- Un–supervised learning is performed with the help of self – organizing map or SOM. 

- This map was an invention of Teuvo Kohonen.

- Few neurons learn mapping points in the input space that could not coordinate in the output space. 

- The dimensions and topology of the input space can be different from those of the output space. SOM makes an attempt for preserving these.

4. Learning vector quantization or LVQ: 

- This can also be considered as neural network architecture. 

- This one also was a suggestion of Teuvo Kohonen.  

- In these prototypical representatives are parameterized along with two important things namely, a classification scheme based - up on distance and a distance measure.

5. Recurrent neural network: 

- These networks are somewhat contrary to the feed forward networks. 

- They offer a bi–directional flow of data.

- On a feed forward network data is propagated linearly from input to output. 

- Data from later stages of processing is also transferred to its earlier stages by this network. 

- Sometimes these also double up as the general sequence processors. 

- Recurrent neural networks have a number of types as mentioned below:

Ø  Fully recurrent network

Ø  Hopfield network

Ø  Boltzmann machine

Ø  Simple recurrent networks

Ø  Echo state network

Ø  Long short term memory network

Ø  Bi – directional RNN

Ø  Hierarchical RNN

Ø  Stochastic neural networks

6. Modular neural networks: 

- As per the studies have shown that human brain works actually as a collection of several small networks rather than as just one huge network, this ultimately helped in realizing the modular neural networks where smaller networks cooperate in solving a problem. 

- Modular networks are also of many types such as:

Ø  Committee of machines: Different networks that work together on a given problem are collectively termed as the committee of machines. The result achieved through this kind of networking is quite better than what is achieved with the others. The result is highly stabilized.

Ø  Associative neural network or ASNN: This is an extension of the previous one. And extends a little beyond the weighted average of various models. This one is a combined form of the k- nearest neighbor technique (kNN) and the feed forward neural networks. Its memory is coincident with that of the training set.

7. Physical neural network: 

- It consists of some resistance material that is electrically adjustable and capable of simulating the artificial synapses.

There are other types of ANNs that do not fall in any of the above categories:

Ø  Holographic associative memory

Ø  Instantaneously trained networks

Ø  Spiking neural networks

Ø  Dynamic neural networks

Ø  Cascading neural networks

Ø  Neuro – fuzzy networks

Ø  Compositional pattern producing networks

Ø  One – shot associative memory