Neural Network - Statistical perspective

Neural Networks: A Review from a statistical perspective
Bing Cheng and D.M. Titterington
Statistical Science
1994, Vol 9, No. 1, pgs 2-54

A excellent paper that introduces the connections between statistical methods and neural networks.

* Introduces NN jargon and, to some extent, statistical jargon to the reader. Good as a reference for and introduction to FFNN, i.e., multi layer perceptron.

* Mentions concerns with back propagation algorithm, namely, speed and debates relevance of various quasi-newton techniques - which by not evaluating second derivative speed up the training (relevant to me because that is what i am using).

* Gives good examples of successful NN - in one case of NN that did not need any training. other examples are way too complicated as opposed to generalised techniques.

* Section 4 'Multilayer Perceptron' is very relevant tho sometimes decends to gibberish considering that I am not so well acquinted with the statistical jargon. A few revisits would be able to improve that situation - which would be very much worth it.

* Section 5 discusses Hopfield network - for associative memories (i.e. cluster analysis) but this is too much of jargon and gibberish for me - at this point.

* Section 6 discusses 'Associative networks with unsupervised learning' in lesser detail; but I have not dwelled too much with this section either.
Section 7 talks about the 'Future' - raises some good questions.

* The paper also references to a few really good papers.

Note: use this paper while introducing terms like multi layer perceptron; training algorithm. you would find these terms being defined from the more conservative statistical background useful.