REAL-TIME OPTIMIZATION BY EXTREMUM-SEEKING CONTROL

By WILLIAM MARTIN

CONTENTS INTRODUCTION TO AVERAGING AVERAGING FOR ORDINARY DIFFERENTIAL EQUATIONS AVERAGING FOR GLOBALLY LIPSCHITZ SYSTEMS AVERAGING FOR LOCALLY LIPSCHITZ SYSTEMS STOCHASTIC AVERAGING AVERAGING FOR STOCHASTIC PERTURBATION PROCESS AVERAGING FOR STOCHASTIC DIFFERENTIAL EQUATIONS INTRODUCTION TO EXTREMUM SEEKING MOTIVATION AND RECENT REVIVAL WHY STOCHASTIC EXTREMUM SEEKING? A BRIEF INTRODUCTION TO STOCHASTIC EXTREMUM SEEKING A BASIC DETERMINISTIC ES SCHEME A BASIC STOCHASTIC ES SCHEME A HEURISTIC ANALYSIS OF A SIMPLE STOCHASTIC ES ALGORITHM STOCHASTIC AVERAGING FOR ASYMPTOTIC STABILITY PROBLEM FORMULATION MAIN THEOREMS UNIFORM STRONG ERGODIC PERTURBATION PROCESS ?-MIXING PERTURBATION PROCESS PROOFS OF THE THEOREMS PROOFS FOR THE CASE OF UNIFORM STRONG ERGODIC PERTURBATION PROCESS PROOFS FOR THE CASE OF ?-MIXING PERTURBATION PROCESS EXAMPLES PERTURBATION PROCESS IS ASYMPTOTICALLY PERIODIC PERTURBATION PROCESS IS ALMOST SURELY EXPONENTIALLY STABLE PERTURBATION PROCESS IS BROWNIAN MOTION ON THE UNIT CIRCLE NOTES AND REFERENCES STOCHASTIC AVERAGING FOR PRACTICAL STABILITY GENERAL STOCHASTIC AVERAGING STATEMENTS OF GENERAL RESULTS ON STOCHASTIC AVERAGING PROOFS OF THE GENERAL THEOREMS ON STOCHASTIC AVERAGING PROOF OF LEMMA 4 1 PROOF OF APPROXIMATION RESULT (4 22) OF THEOREM 4 1 PRELIMINARY LEMMAS FOR THE PROOF OF APPROXIMATION RESULT (4 23) OF THEOREM 4 1 PROOF OF APPROXIMATION RESULT (4 23) OF THEOREM 4 1 PROOF OF THEOREM 4 2 PROOF OF (4 45) DISCUSSIONS OF THE EXISTENCE OF SOLUTION SINGLE-PARAMETER STOCHASTIC EXTREMUM SEEKING EXTREMUM SEEKING FOR A STATIC MAP STOCHASTIC EXTREMUM SEEKING FEEDBACK FOR GENERAL NONLINEAR DYNAMIC SYSTEMS STOCHASTIC SOURCE SEEKING FOR NONHOLONOMIC VEHICLES VEHICLE MODEL AND PROBLEM STATEMENT STOCHASTIC SOURCE SEEKING CONTROLLER STABILITY ANALYSIS CONVERGENCE SPEED SIMULATIONS AND DEPENDENCE ON DESIGN PARAMETERS BASIC SIMULATIONS DEPENDENCE OF ANNULUS RADIUS ? ON PARAMETERS DEPENDENCE ON DAMPING TERM D0 NO DAMPING (D0 = 0) EFFECT OF DAMPING (D0 > 0) EFFECT OF CONSTRAINTS OF THE ANGULAR VELOCITY AND DESIGN ALTERNATIVES EFFECT OF CONSTRAINTS OF THE ANGULAR VELOCITY ALTERNATIVE DESIGNS SYSTEM BEHAVIOR FOR ELLIPTICAL LEVEL SETS STOCHASTIC SOURCE SEEKING WITH TUNING OF FORWARD VELOCITY THE MODEL OF AUTONOMOUS VEHICLE SEARCH ALGORITHM AND CONVERGENCE ANALYSIS SIMULATION MULTI-PARAMETER STOCHASTIC EXTREMUM SEEKING AND SLOPE SEEKING MULTI-INPUT STOCHASTIC AVERAGING MULTI-PARAMETER STOCHASTIC ES FOR STATIC MAPS ALGORITHM FOR MULTI-PARAMETER STOCHASTIC ES CONVERGENCE ANALYSIS STOCHASTIC GRADIENT SEEKING SINGLE-PARAMETER STOCHASTIC SLOPE SEEKING MULTI-PARAMETER STOCHASTIC GRADIENT SEEKING STOCHASTIC NASH EQUILIBRIUM SEEKING FOR GAMES WITH GENERAL NONLINEAR PAYOFFS STOCHASTIC NASH EQUILIBRIUM SEEKING ALGORITHM PROOF OF THE ALGORITHM CONVERGENCE NUMERICAL EXAMPLE NASH EQUILIBRIUM SEEKING FOR QUADRATIC GAMES AND APPLICATIONS TO OLIGOPOLY MARKETS AND VEHICLE DEPLOYMENT 10 1 N-PLAYER GAMES WITH QUADRATIC PAYOFF FUNCTIONS 10 1 1 GENERAL QUADRATIC GAMES 10 1 2 SYMMETRIC QUADRATIC GAMES 10 2 OLIGOPOLY PRICE GAMES 10 3 MULTI-AGENT DEPLOYMENT IN THE PLANE 10 3 1 VEHICLE MODEL AND LOCAL AGENT COST 10 3 2 CONTROL DESIGN 10 3 3 STABILITY ANALYSIS 10 3 4 SIMULATION 10 4 NOTES AND REFERENCES NEWTON-BASED STOCHASTIC EXTREMUM SEEKING 11 1 SINGLE-PARAMETER NEWTON ALGORITHM FOR STATIC MAPS 11 2 MULTI-PARAMETER NEWTON ALGORITHM FOR STATIC MAPS 11 2 1 PROBLEM FORMULATION 11 2 2 ALGORITHM DESIGN AND STABILITY ANALYSIS 11 3 NEWTON ALGORITHM FOR DYNAMIC SYSTEMS 11 4 SIMULATION 11 5 NOTES AND REFERENCES