VECTORS AND MATRICES
CONTENTS I VECTORS 1 1 VECTORS 3 1 1 VECTORS 3 1 2 VECTOR ADDITION 11 1 3 SCALAR VECTOR MULTIPLICATION 15 1 4 INNER PRODUCT 19 1 5 COMPLEXITY OF VECTOR COMPUTATIONS 22 EXERCISES 25 2 LINEAR FUNCTIONS 29 2 1 LINEAR FUNCTIONS 29 2 2 TAYLOR APPROXIMATION 35 2 3 REGRESSION MODEL 38 EXERCISES 42 3 NORM AND DISTANCE 45 3 1 NORM 45 3 2 DISTANCE 48 3 3 STANDARD DEVIATION 52 3 4 ANGLE 56 3 5 COMPLEXITY 63 EXERCISES 64 4 CLUSTERING 69 4 1 CLUSTERING 69 4 2 A CLUSTERING OBJECTIVE 72 4 3 THE K MEANS ALGORITHM 74 4 4 EXAMPLES 79 4 5 APPLICATIONS 85 EXERCISES 87 5 LINEAR INDEPENDENCE 89 5 1 LINEAR DEPENDENCE 89 5 2 BASIS 91 5 3 ORTHONORMAL VECTORS 95 5 4 GRAM SCHMIDT ALGORITHM 97 EXERCISES 103 II MATRICES 105 6 MATRICES 107 6 1 MATRICES 107 6 2 ZERO AND IDENTITY MATRICES 113 6 3 TRANSPOSE ADDITION AND NORM 115 6 4 MATRIX VECTOR MULTIPLICATION 118 6 5 COMPLEXITY 122 EXERCISES 124 7 MATRIX EXAMPLES 129 7 1 GEOMETRIC TRANSFORMATIONS 129 7 2 SELECTORS 131 7 3 INCIDENCE MATRIX 132 7 4 CONVOLUTION 136 EXERCISES 144 8 LINEAR EQUATIONS 147 8 1 LINEAR AND A NE FUNCTIONS 147 8 2 LINEAR FUNCTION MODELS 150 8 3 SYSTEMS OF LINEAR EQUATIONS 152 EXERCISES 159 9 LINEAR DYNAMICAL SYSTEMS 163 9 1 LINEAR DYNAMICAL SYSTEMS 163 9 2 POPULATION DYNAMICS 164 9 3 EPIDEMIC DYNAMICS 168 9 4 MOTION OF A MASS 169 9 5 SUPPLY CHAIN DYNAMICS 171 EXERCISES 174 10 MATRIX MULTIPLICATION 177 10 1 MATRIX MATRIX MULTIPLICATION 177 10 2 COMPOSITION OF LINEAR FUNCTIONS 183 10 3 MATRIX POWER 186 10 4 QR FACTORIZATION 189 EXERCISES 191 11 MATRIX INVERSES 199 11 1 LEFT AND RIGHT INVERSES 199 11 2 INVERSE 202 11 3 SOLVING LINEAR EQUATIONS 207 11 4 EXAMPLES 210 11 5 PSEUDO INVERSE 214 EXERCISES 217 III LEAST SQUARES 223 12 LEAST SQUARES 225 12 1 LEAST SQUARES PROBLEM 225 12 2 SOLUTION 227 12 3 SOLVING LEAST SQUARES PROBLEMS 231 12 4 EXAMPLES 234 EXERCISES 239 13 LEAST SQUARES DATA TTING 245 13 1 LEAST SQUARES DATA TTING 245 13 2 VALIDATION 260 13 3 FEATURE ENGINEERING 269 EXERCISES 279 14 LEAST SQUARES CLASSI CATION 285 14 1 CLASSI CATION 285 14 2 LEAST SQUARES CLASSI ER 288 14 3 MULTI CLASS CLASSI ERS 297 EXERCISES 305 15 MULTI OBJECTIVE LEAST SQUARES 309 15 1 MULTI OBJECTIVE LEAST SQUARES 309 15 2 CONTROL 314 15 3 ESTIMATION AND INVERSION 316 15 4 REGULARIZED DATA TTING 325 15 5 COMPLEXITY 330 EXERCISES 334 16 CONSTRAINED LEAST SQUARES 339 16 1 CONSTRAINED LEAST SQUARES PROBLEM 339 16 2 SOLUTION 344 16 3 SOLVING CONSTRAINED LEAST SQUARES PROBLEMS 347 EXERCISES 352 17 CONSTRAINED LEAST SQUARES APPLICATIONS 357 17 1 PORTFOLIO OPTIMIZATION 357 17 2 LINEAR QUADRATIC CONTROL 366 17 3 LINEAR QUADRATIC STATE ESTIMATION 372 EXERCISES 378 18 NONLINEAR LEAST SQUARES 381 18 1 NONLINEAR EQUATIONS AND LEAST SQUARES 381 18 2 GAUSS NEWTON ALGORITHM 386 18 3 LEVENBERG MARQUARDT ALGORITHM 391 18 4 NONLINEAR MODEL TTING 399 18 5 NONLINEAR LEAST SQUARES CLASSI CATION 401 EXERCISES 412 19 CONSTRAINED NONLINEAR LEAST SQUARES 419 19 1 CONSTRAINED NONLINEAR LEAST SQUARES 419 19 2 PENALTY ALGORITHM 421 19 3 AUGMENTED LAGRANGIAN ALGORITHM 422 19 4 NONLINEAR CONTROL 425 EXERCISES 434 PART I VECTORS