PRINCIPLES OF ARTIFICIAL NEURAL NETWORKS
CONTENTS CHAPTER 1 INTRODUCTION AND ROLE OF ARTI?CIAL NEURAL NETWORKS CHAPTER 2 FUNDAMENTALS OF BIOLOGICAL NEURAL NETWORKS CHAPTER 3 BASIC PRINCIPLES OF ANNS AND THEIR EARLY STRUCTURES 3 1 BASIC PRINCIPLES OF ANN DESIGN 3 2 BASIC NETWORK STRUCTURES 3 3 THE PERCEPTRON’S INPUT-OUTPUT PRINCIPLES 3 4 THE ADALINE (ALC) CHAPTER 4 THE PERCEPTRON 4 1 THE BASIC STRUCTURE 4 2 THE SINGLE-LAYER REPRESENTATION PROBLEM 4 3 THE LIMITATIONS OF THE SINGLE-LAYER PERCEPTRON 4 4 MANY-LAYER PERCEPTRONS 4 A PERCEPTRON CASE STUDY: IDENTIFYING AUTOREGRESSIVE PARAMETERS OF A SIGNAL (AR TIME SERIES IDENTI?CATION) CHAPTER 5 THE MADALINE 5 1 MADALINE TRAINING 5 A MADALINE CASE STUDY: CHARACTER RECOGNITION CHAPTER 6 BACK PROPAGATION 6 1 THE BACK PROPAGATION LEARNING PROCEDURE 6 2 DERIVATION OF THE BP ALGORITHM 6 3 MODI?ED BP ALGORITHMS 6 A BACK PROPAGATION CASE STUDY: CHARACTER RECOGNITION 6 B BACK PROPAGATION CASE STUDY: THE EXCLUSIVE-OR (XOR) PROBLEM (2-LAYER BP) 6 C BACK PROPAGATION CASE STUDY: THE XOR PROBLEM — 3 LAYER BP NETWORK CHAPTER 7 HOP?ELD NETWORKS 7 1 INTRODUCTION 7 2 BINARY HOP?ELD NETWORKS 7 3 SETTING OF WEIGHTS IN HOP?ELD NETS — BIDIRECTIONAL ASSOCIATIVE MEMORY (BAM) PRINCIPLE 7 4 WALSH FUNCTIONS 7 5 NETWORK STABILITY 7 6 SUMMARY OF THE PROCEDURE FOR IMPLEMENTING THE HOP?ELD NETWORK 7 7 CONTINUOUS HOP?ELD MODELS 7 8 THE CONTINUOUS ENERGY (LYAPUNOV) FUNCTION 7 A HOP?ELD NETWORK CASE STUDY: CHARACTER RECOGNITION 7 B HOP?ELD NETWORK CASE STUDY: TRAVELING SALESMAN PROBLEM CHAPTER 8 COUNTER PROPAGATION 8 1 INTRODUCTION 8 2 KOHONEN SELF-ORGANIZING MAP (SOM) LAYER 8 3 GROSSBERG LAYER 8 4 TRAINING OF THE KOHONEN LAYER 8 5 TRAINING OF GROSSBERG LAYERS 8 6 THE COMBINED COUNTER PROPAGATION NETWORK 8 A COUNTER PROPAGATION NETWORK CASE STUDY: CHARACTER RECOGNITION CHAPTER 9 ADAPTIVE RESONANCE THEORY 9 1 MOTIVATION 9 2 THE ART NETWORK STRUCTURE 9 3 SETTING-UP OF THE ART NETWORK 9 4 NETWORK OPERATION 9 5 PROPERTIES OF ART 9 6 DISCUSSION AND GENERAL COMMENTS ON ART-I AND ART-II 9 A ART-I NETWORK CASE STUDY: CHARACTER RECOGNITION 9 B ART-I CASE STUDY: SPEECH RECOGNITION CHAPTER 10 THE COGNITRON AND THE NEOCOGNITRON 10 1 BACKGROUND OF THE COGNITRON 10 2 THE BASIC PRINCIPLES OF THE COGNITRON 10 3 NETWORK OPERATION 10 4 COGNITRON’S NETWORK TRAINING 10 5 THE NEOCOGNITRON CHAPTER 11 STATISTICAL TRAINING 11 1 FUNDAMENTAL PHILOSOPHY 11 2 ANNEALING METHODS 11 3 SIMULATED ANNEALING BY BOLTZMAN TRAINING OF WEIGHTS 11 4 STOCHASTIC DETERMINATION OF MAGNITUDE OF WEIGHT CHANGE 11 5 TEMPERATURE-EQUIVALENT SETTING 11 6 CAUCHY TRAINING OF NEURAL NETWORK 11 A STATISTICAL TRAINING CASE STUDY — A STOCHASTIC HOP?ELD NETWORK FOR CHARACTER RECOGNITION 11 B STATISTICAL TRAINING CASE STUDY: IDENTIFYING AR SIGNAL PARAMETERS WITH A STOCHASTIC PERCEPTRON MODEL CHAPTER 12 RECURRENT (TIME CYCLING) BACK PROPAGATION NETWORKS 12 1 RECURRENT/DISCRETE TIME NETWORKS 12 2 FULLY RECURRENT NETWORKS 12 3 CONTINUOUSLY RECURRENT BACK PROPAGATION NETWORKS 12 A RECURRENT BACK PROPAGATION CASE STUDY: CHARACTER RECOGNITION CHAPTER 13 LARGE SCALE MEMORY STORAGE AND RETRIEVAL (LAMSTAR) NETWORK 13 1 BASIC PRINCIPLES OF THE LAMSTAR NEURAL NETWORK 13 2 DETAILED OUTLINE OF THE LAMSTAR NETWORK 13 3 FORGETTING FEATURE 13 4 TRAINING VS OPERATIONAL RUNS 13 5 ADVANCED DATA ANALYSIS CAPABILITIES 13 6 CORRELATION, INTERPOLATION, EXTRAPOLATION AND INNOVATION-DETECTION 13 7 CONCLUDING COMMENTS AND DISCUSSION OF APPLICABILITY 13 A LAMSTAR NETWORK CASE STUDY: CHARACTER RECOGNITION 13 B APPLICATION TO MEDICAL DIAGNOSIS PROBLEMS PROBLEMS REFERENCES