PRINCIPAL MANIFOLDS FOR DATA VISUALIZATION AND DIMENSION REDUCTION
CONTENTS 1 DEVELOPMENTS AND APPLICATIONS OF NONLINEAR PRINCIPAL COMPONENT ANALYSIS A REVIEW 1 1 1 INTRODUCTION 1 1 2 PCA PRELIMINARIES 3 1 3 NONLINEARITY TEST FOR PCA MODELS 6 1 3 1 ASSUMPTIONS 7 1 3 2 DISJUNCT REGIONS 7 1 3 3 CON DENCE LIMITS FOR CORRELATION MATRIX 8 1 3 4 ACCURACY BOUNDS 10 1 3 5 SUMMARY OF THE NONLINEARITY TEST 11 1 3 6 EXAMPLE STUDIES 12 1 4 NONLINEAR PCA EXTENSIONS 15 1 4 1 PRINCIPAL CURVES AND MANIFOLDS 16 1 4 2 NEURAL NETWORK APPROACHES 24 1 4 3 KERNEL PCA 29 1 5 ANALYSIS OF EXISTING WORK 31 1 5 1 COMPUTATIONAL ISSUES 31 1 5 2 GENERALIZATION OF LINEAR PCA 33 1 5 3 ROADMAP FOR FUTURE DEVELOPMENTS BASICS AND BEYOND 37 1 6 CONCLUDING SUMMARY 38 REFERENCES 39 2 NONLINEAR PRINCIPAL COMPONENT ANALYSIS NEURAL NETWORK MODELS AND APPLICATIONS 44 2 1 INTRODUCTION 44 2 2 STANDARD NONLINEAR PCA 47 2 3 HIERARCHICAL NONLINEAR PCA 48 2 3 1 THE HIERARCHICAL ERROR FUNCTION 49 2 4 CIRCULAR PCA 51 2 5 INVERSE MODEL OF NONLINEAR PCA 52 2 5 1 THE INVERSE NETWORK MODEL 53 2 5 2 NLPCA MODELS APPLIED TO CIRCULAR DATA 55 2 5 3 INVERSE NLPCA FOR MISSING DATA 56 2 5 4 MISSING DATA ESTIMATION 57 2 6 APPLICATIONS 58 2 6 1 APPLICATION OF HIERARCHICAL NLPCA 59 2 6 2 METABOLITE DATA ANALYSIS 60 2 6 3 GENE EXPRESSION ANALYSIS 62 2 7 SUMMARY 64 REFERENCES 65 3 LEARNING NONLINEAR PRINCIPAL MANIFOLDS BY SELF ORGANISING MAPS 68 3 1 INTRODUCTION 68 3 2 BIOLOGICAL BACKGROUND 69 3 2 1 LATERAL INHIBITION AND HEBBIAN LEARNING 69 3 2 2 FROM VON MARSBURG AND WILLSHAW S MODEL TO KOHONEN S SOM 72 3 2 3 THE SOM ALGORITHM 75 3 3 THEORIES 76 3 3 1 CONVERGENCE AND COST FUNCTIONS 76 3 3 2 TOPOLOGICAL ORDERING MEASURES 79 3 4 SOMS MULTIDIMENSIONAL SCALING AND PRINCIPAL MANIFOLDS 80 3 4 1 MULTIDIMENSIONAL SCALING 80 3 4 2 PRINCIPAL MANIFOLDS 82 3 4 3 VISUALISATION INDUCED SOM VISOM 84 3 5 EXAMPLES 86 3 5 1 DATA VISUALISATION 87 3 5 2 DOCUMENT ORGANISATION AND CONTENT MANAGEMENT 88 REFERENCES 91 4 ELASTIC MAPS AND NETS FOR APPROXIMATING PRINCIPAL MANIFOLDS AND THEIR APPLICATION TO MICROARRAY DATA VISUALIZATION 96 4 1 INTRODUCTION AND OVERVIEW 96 4 1 1 FR ECHET MEAN AND PRINCIPAL OBJECTS K MEANS PCA WHAT ELSE 96 4 1 2 PRINCIPAL MANIFOLDS 98 4 1 3 ELASTIC FUNCTIONAL AND ELASTIC NETS 100 4 2 OPTIMIZATION OF ELASTIC NETS FOR DATA APPROXIMATION 103 4 2 1 BASIC OPTIMIZATION ALGORITHM 103 4 2 2 MISSING DATA VALUES 105 4 2 3 ADAPTIVE STRATEGIES 106 4 3 ELASTIC MAPS 109 4 3 1 PIECEWISE LINEAR MANIFOLDS AND DATA PROJECTORS 109 4 3 2 ITERATIVE DATA APPROXIMATION 109 4 4 PRINCIPAL MANIFOLD AS ELASTIC MEMBRANE 110 4 5 METHOD IMPLEMENTATION 112 4 6 EXAMPLES 112 4 6 1 TEST EXAMPLES 112 4 6 2 MODELING MOLECULAR SURFACES 113 4 6 3 VISUALIZATION OF MICROARRAY DATA 114 4 7 DISCUSSION 125 REFERENCES 127 5 TOPOLOGY PRESERVING MAPPINGS FOR DATA VISUALISATION 131 5 1 INTRODUCTION 131 5 2 CLUSTERING TECHNIQUES 132 5 2 1 K MEANS 132 5 2 2 K HARMONIC MEANS 133 5 2 3 NEURAL GAS 135 5 2 4 WEIGHTED K MEANS 136 5 2 5 THE INVERSE WEIGHTED K MEANS 137 5 3 TOPOLOGY PRESERVING MAPPINGS 138 5 3 1 GENERATIVE TOPOGRAPHIC MAP 138 5 3 2 TOPOGRAPHIC PRODUCT OF EXPERTS TOPOE 140 5 3 3 THE HARMONIC TOPOGRAPIC MAP 141 5 3 4 TOPOGRAPHIC NEURAL GAS 143 5 3 5 INVERSE WEIGHTED K MEANS TOPOLOGY PRESERVING MAP 143 5 4 EXPERIMENTS 144 5 4 1 PROJECTIONS IN LATENT SPACE 144 5 4 2 RESPONSIBILITIES 144 5 4 3 U MATRIX HIT HISTOGRAMS AND DISTANCE MATRIX 145 5 4 4 THE QUALITY OF THE MAP 147 5 5 CONCLUSIONS 149 REFERENCES 149 6 THE ITERATIVE EXTRACTION APPROACH TO CLUSTERING 151 6 1 INTRODUCTION 151 6 2 CLUSTERING ENTITY TO FEATURE DATA 152 6 2 1 PRINCIPAL COMPONENT ANALYSIS 152 6 2 2 ADDITIVE CLUSTERING MODEL AND ITEX 154 6 2 3 OVERLAPPING AND FUZZY CLUSTERING CASE 156 6 2 4 K MEANS AND IK MEANS CLUSTERING 157 6 3 ITEX STRUCTURING AND CLUSTERING FOR SIMILARITY DATA 162 6 3 1 SIMILARITY CLUSTERING A REVIEW 162 6 3 2 THE ADDITIVE STRUCTURING MODEL AND ITEX 163 6 3 3 ADDITIVE CLUSTERING MODEL 165 6 3 4 APPROXIMATE PARTITIONING 166 6 3 5 ONE CLUSTER CLUSTERING 168 6 3 6 SOME APPLICATIONS 171 REFERENCES 174 7 REPRESENTING COMPLEX DATA USING LOCALIZED PRINCIPAL COMPONENTS WITH APPLICATION TO ASTRONOMICAL DATA 178 7 1 INTRODUCTION 178 7 2 LOCALIZED PRINCIPAL COMPONENT ANALYSIS 181 7 2 1 CLUSTER WISE PCA 181 7 2 2 PRINCIPAL CURVES 185 7 2 3 FURTHER APPROACHES 188 7 3 COMBINING PRINCIPAL CURVES AND REGRESSION 189 7 3 1 PRINCIPAL COMPONENT REGRESSION AND ITS SHORTCOMINGS 189 7 3 2 THE GENERALIZATION TO PRINCIPAL CURVES 190 7 3 3 USING DIRECTIONS OTHER THAN THE LOCAL PRINCIPAL COMPONENTS 192 7 3 4 A SIMPLE EXAMPLE 193 7 4 APPLICATION TO THE GAIA SURVEY MISSION 194 7 4 1 THE ASTROPHYSICAL DATA 194 7 4 2 PRINCIPAL MANIFOLD BASED APPROACH 196 7 5 CONCLUSION 198 REFERENCES 199 8 AUTO ASSOCIATIVE MODELS NONLINEAR PRINCIPAL COMPONENT ANALYSIS MANIFOLDS AND PROJECTION PURSUIT 202 8 1 INTRODUCTION 202 8 2 AUTO ASSOCIATIVE MODELS 203 8 2 1 APPROXIMATION BY MANIFOLDS 203 8 2 2 A PROJECTION PURSUIT ALGORITHM 205 8 2 3 THEORETICAL RESULTS 206 8 3 EXAMPLES 207 8 3 1 LINEAR AUTO ASSOCIATIVE MODELS AND PCA 207 8 3 2 ADDITIVE AUTO ASSOCIATIVE MODELS AND NEURAL NETWORKS 208 8 4 IMPLEMENTATION ASPECTS 209 8 4 1 ESTIMATION OF THE REGRESSION FUNCTIONS 209 8 4 2 COMPUTATION OF PRINCIPAL DIRECTIONS 211 8 5 ILLUSTRATION ON REAL AND SIMULATED DATA 213 REFERENCES 216 9 BEYOND THE CONCEPT OF MANIFOLDS PRINCIPAL TREES METRO MAPS AND ELASTIC CUBIC COMPLEXES 219 9 1 INTRODUCTION AND OVERVIEW 219 9 1 1 ELASTIC PRINCIPAL GRAPHS 221 9 2 OPTIMIZATION OF ELASTIC GRAPHS FOR DATA APPROXIMATION 222 9 2 1 ELASTIC FUNCTIONAL OPTIMIZATION 222 9 2 2 OPTIMAL APPLICATION OF GRAPH GRAMMARS 223 9 2 3 FACTORIZATION AND TRANSFORMATION OF FACTORS 224 9 3 PRINCIPAL TREES BRANCHING PRINCIPAL CURVES 225 9 3 1 SIMPLE GRAPH GRAMMAR ADD A NODE BISECT AN EDGE 225 9 3 2 VISUALIZATION OF DATA USING METRO MAP TWO DIMENSIONAL TREE LAYOUT 226 9 3 3 EXAMPLE OF PRINCIPAL CUBIC COMPLEX PRODUCT OF PRINCIPAL TREES 227 9 4 ANALYSIS OF THE UNIVERSAL 7 CLUSTER STRUCTURE OF BACTERIAL GENOMES 229 9 4 1 BRIEF INTRODUCTION 230 9 4 2 VISUALIZATION OF THE 7 CLUSTER STRUCTURE 232 9 5 VISUALIZATION OF MICROARRAY DATA 232 9 5 1 DATASET USED 232 9 5 2 PRINCIPAL TREE OF HUMAN TISSUES 234 9 6 DISCUSSION 235 REFERENCES 235 10 DI USION MAPS A PROBABILISTIC INTERPRETATION FOR SPECTRAL EMBEDDING AND CLUSTERING ALGORITHMS 238 10 1 INTRODUCTION 238 10 2 DI USION DISTANCES AND DI USION MAPS 240 10 2 1 ASYMPTOTICS OF THE DI USION MAP 245 10 3 SPECTRAL EMBEDDING OF LOW DIMENSIONAL MANIFOLDS 246 10 4 SPECTRAL CLUSTERING OF A MIXTURE OF GAUSSIANS 251 10 5 SUMMARY AND DISCUSSION 258 REFERENCES 258 11 ON BOUNDS FOR DI USION DISCREPANCY AND FILL DISTANCE METRICS 261 11 1 INTRODUCTION 261 11 2 ENERGY DISCREPANCY DISTANCE AND INTEGRATION ON MEASURABLE SETS IN EUCLIDEAN SPACE 262