ASYMPTOTIC THEORY OF STATISTICS AND PROBABILITY

By P. MAKAR

CONTENTS BASIC CONVERGENCE CONCEPTS AND THEOREMS SOME BASIC NOTATION AND CONVERGENCE THEOREMS THREE SERIES THEOREM AND KOLMOGOROV’S ZERO-ONE LAW CENTRAL LIMIT THEOREM AND LAW OF THE ITERATED LOGARITHM FURTHER ILLUSTRATIVE EXAMPLES EXERCISES REFERENCES METRICS, INFORMATION THEORY, CONVERGENCE, AND POISSON APPROXIMATIONS SOME COMMON METRICS AND THEIR USEFULNESS CONVERGENCE IN TOTAL VARIATION AND FURTHER USEFUL FORMULAS INFORMATION-THEORETIC DISTANCES, DE BRUIJN’S IDENTITY, AND RELATIONS TO CONVERGENCE POISSON APPROXIMATIONS MORE GENERAL WEAK AND STRONG LAWS AND THE DELTA THEOREM GENERAL LLNS AND UNIFORM STRONG LAW MEDIAN CENTERING AND KESTEN’S THEOREM THE ERGODIC THEOREM DELTA THEOREM AND EXAMPLES APPROXIMATION OF MOMENTS TRANSFORMATIONS VARIANCE-STABILIZING TRANSFORMATIONS EXAMPLES BIAS CORRECTION OF THE VST SYMMETRIZING TRANSFORMATIONS VST OR SYMMETRIZING TRANSFORM? MORE GENERAL CENTRAL LIMIT THEOREMS THE INDEPENDENT NOT IID CASE AND A KEY EXAMPLE CLT WITHOUT A VARIANCE COMBINATORIAL CLT CLT FOR EXCHANGEABLE SEQUENCES CLT FOR A RANDOM NUMBER OF SUMMANDS IN?NITE DIVISIBILITY AND STABLE LAWS MOMENT CONVERGENCE AND UNIFORM INTEGRABILITY BASIC RESULTS THE MOMENT PROBLEM SAMPLE PERCENTILES AND ORDER STATISTICS ASYMPTOTIC DISTRIBUTION OF ONE ORDER STATISTIC JOINT ASYMPTOTIC DISTRIBUTION OF SEVERAL ORDER STATISTICS BAHADUR REPRESENTATIONS CON?DENCE INTERVALS FOR QUANTILES REGRESSION QUANTILES SAMPLE EXTREMES SUF?CIENT CONDITIONS CHARACTERIZATIONS LIMITING DISTRIBUTION OF THE SAMPLE RANGE MULTIPLICATIVE STRONG LAW ADDITIVE STRONG LAW DEPENDENT SEQUENCES CENTRAL LIMIT THEOREMS FOR DEPENDENT SEQUENCES STATIONARY M-DEPENDENCE SAMPLING WITHOUT REPLACEMENT MARTINGALES AND EXAMPLES THE MARTINGALE AND REVERSE MARTINGALE CLTS CENTRAL LIMIT THEOREM FOR MARKOV CHAINS NOTATION AND BASIC DE?NITIONS NORMAL LIMITS NONNORMAL LIMITS CONVERGENCE TO STATIONARITY: DIACONIS-STROOCK-FILL BOUND ACCURACY OF CENTRAL LIMIT THEOREMS UNIFORM BOUNDS: BERRY-ESSEEN INEQUALITY LOCAL BOUNDS THE MULTIDIMENSIONAL BERRY-ESSEEN THEOREMS OTHER STATISTICS INVARIANCE PRINCIPLES MOTIVATING EXAMPLES TWO RELEVANT GAUSSIAN PROCESSES THE ERD¨OS-KAC INVARIANCE PRINCIPLE INVARIANCE PRINCIPLES, DONSKER’S THEOREM, AND THE KMT CONSTRUCTION INVARIANCE PRINCIPLE FOR EMPIRICAL PROCESSES EXTENSIONS OF DONSKER’S PRINCIPLE AND VAPNIK-CHERVONENKIS CLASSES GLIVENKO-CANTELLI THEOREM FOR VC CLASSES CLTS FOR EMPIRICAL MEASURES AND APPLICATIONS NOTATION AND FORMULATION ENTROPY BOUNDS AND SPECI?C CLTS DEPENDENT SEQUENCES: MARTINGALES, MIXING, AND SHORT-RANGE DEPENDENCE WEIGHTED EMPIRICAL PROCESSES AND APPROXIMATIONS EDGEWORTH EXPANSIONS AND CUMULANTS EXPANSION FOR MEANS USING THE EDGEWORTH EXPANSION EDGEWORTH EXPANSION FOR SAMPLE PERCENTILES EDGEWORTH EXPANSION FOR THE T-STATISTIC CORNISH-FISHER EXPANSIONS CUMULANTS AND FISHER’S K-STATISTICS SADDLEPOINT APPROXIMATIONS APPROXIMATE EVALUATION OF INTEGRALS DENSITY OF MEANS AND EXPONENTIAL TILTING DERIVATION BY EDGEWORTH EXPANSION AND EXPONENTIAL TILTING SOME EXAMPLES APPLICATION TO EXPONENTIAL FAMILY AND THE MAGIC FORMULA TAIL AREA APPROXIMATION AND THE LUGANNANI-RICE FORMULA EDGEWORTH VS SADDLEPOINT VS CHI-SQUARE APPROXIMATION TAIL AREAS FOR SAMPLE PERCENTILES QUANTILE APPROXIMATION AND INVERTING THE LUGANNANI-RICE FORMULA THE MULTIDIMENSIONAL CASE U-STATISTICS ASYMPTOTIC DISTRIBUTION OF U-STATISTICS MOMENTS OF U-STATISTICS AND THE MARTINGALE STRUCTURE EDGEWORTH EXPANSIONS MAXIMUM LIKELIHOOD ESTIMATES INCONSISTENT MLES MLES IN THE EXPONENTIAL FAMILY MORE GENERAL CASES AND ASYMPTOTIC NORMALITY OBSERVED AND EXPECTED FISHER INFORMATION EDGEWORTH EXPANSIONS FOR MLES ASYMPTOTIC OPTIMALITY OF THE MLE AND SUPEREF?CIENCY HA´JEK-LECAM CONVOLUTION THEOREM LOSS OF INFORMATION AND EFRON’S CURVATURE M ESTIMATES CONSISTENCY AND ASYMPTOTIC NORMALITY BAHADUR EXPANSION OF M ESTIMATES THE TRIMMED MEAN ASYMPTOTIC DISTRIBUTION AND THE BAHADUR REPRESENTATION LOWER BOUNDS ON EF?CIENCIES MULTIVARIATE TRIMMED MEAN THE 10-20-30-40 RULE MULTIVARIATE LOCATION PARAMETER AND MULTIVARIATE MEDIANS NOTIONS OF SYMMETRY OF MULTIVARIATE DATA MULTIVARIATE MEDIANS ASYMPTOTIC THEORY FOR MULTIVARIATE MEDIANS THE ASYMPTOTIC COVARIANCE MATRIX ASYMPTOTIC COVARIANCE MATRIX OF THE L1 MEDIAN BAYES PROCEDURES AND POSTERIOR DISTRIBUTIONS BERNSTEIN-VON MISES THEOREM POSTERIOR EXPANSIONS EXPANSIONS FOR POSTERIOR MEAN, VARIANCE, AND PERCENTILES THE TIERNEY-KADANE APPROXIMATIONS FREQUENTIST APPROXIMATION OF POSTERIOR SUMMARIES CONSISTENCY OF POSTERIORS THE DIFFERENCE BETWEEN BAYES ESTIMATES AND THE MLE USING THE BROWN IDENTITY TO OBTAIN BAYESIAN ASYMPTOTICS TESTING INTERVAL AND SET ESTIMATION IN?NITE-DIMENSIONAL PROBLEMS AND THE DIACONIS-FREEDMAN RESULTS TESTING PROBLEMS LIKELIHOOD RATIO TESTS ASYMPTOTIC THEORY OF LIKELIHOOD RATIO TEST STATISTICS DISTRIBUTION UNDER ALTERNATIVES BARTLETT CORRECTION THE WALD AND RAO SCORE TESTS LIKELIHOOD RATIO CON?DENCE INTERVALS ASYMPTOTIC EF?CIENCY IN TESTING PITMAN EF?CIENCIES BAHADUR SLOPES AND BAHADUR EF?CIENCY BAHADUR SLOPES OF U-STATISTICS SOME GENERAL LARGE-DEVIATION RESULTS GENERALIZATION OF THE CRAM´ER-CHERNOFF THEOREM THE G¨ARTNER-ELLIS THEOREM LARGE DEVIATION FOR LOCAL LIMIT THEOREMS CLASSICAL NONPARAMETRICS SOME EARLY ILLUSTRATIVE EXAMPLES SIGN TEST CONSISTENCY OF THE SIGN TEST WILCOXON SIGNED-RANK TEST ROBUSTNESS OF THE T CON?DENCE INTERVAL THE BAHADUR-SAVAGE THEOREM KOLMOGOROV-SMIRNOV AND ANDERSON CON?DENCE INTERVALS HODGES-LEHMANN CON?DENCE INTERVAL POWER OF THE WILCOXON TEST TWO-SAMPLE PROBLEMS BEHRENS-FISHER PROBLEM WILCOXON RANK SUM AND MANN-WHITNEY TEST TWO-SAMPLE U-STATISTICS AND POWER APPROXIMATIONS HETTMANSPERGER’S GENERALIZATION THE NONPARAMETRIC BEHRENS-FISHER PROBLEM ROBUSTNESS OF THE MANN-WHITNEY TEST GOODNESS OF FIT KOLMOGOROV-SMIRNOV AND OTHER TESTS BASED ON FN COMPUTATIONAL FORMULAS SOME HEURISTICS ASYMPTOTIC NULL DISTRIBUTIONS OF DN, CN, AN, AND VN CONSISTENCY AND DISTRIBUTIONS UNDER ALTERNATIVES FINITE SAMPLE DISTRIBUTIONS AND OTHER EDF-BASED TESTS THE BERK-JONES PROCEDURE ?-DIVERGENCES AND THE JAGER-WELLNER TESTS THE TWO-SAMPLE CASE TESTS FOR NORMALITY CHI-SQUARE TESTS FOR GOODNESS OF FIT THE PEARSON ?2 TEST ASYMPTOTIC DISTRIBUTION OF PEARSON’S CHI-SQUARE ASYMPTOTIC DISTRIBUTION UNDER ALTERNATIVES AND CONSISTENCY CHOICE OF K RECOMMENDATION OF MANN AND WALD POWER AT LOCAL ALTERNATIVES AND CHOICE OF K GOODNESS OF FIT WITH ESTIMATED PARAMETERS PRELIMINARY ANALYSIS BY STOCHASTIC EXPANSION ASYMPTOTIC DISTRIBUTION OF EDF-BASED STATISTICS FOR COMPOSITE NULLS CHI-SQUARE TESTS WITH ESTIMATED PARAMETERS AND THE CHERNOFF-LEHMANN RESULT CHI-SQUARE TESTS WITH RANDOM CELLS THE BOOTSTRAP BOOTSTRAP DISTRIBUTION AND THE MEANING OF CONSISTENCY CONSISTENCY IN THE KOLMOGOROV AND WASSERSTEIN METRICS DELTA THEOREM FOR THE BOOTSTRAP SECOND-ORDER ACCURACY OF THE BOOTSTRAP SOME NUMERICAL EXAMPLES FAILURE OF THE BOOTSTRAP M OUT OF N BOOTSTRAP BOOTSTRAP CON?DENCE INTERVALS BOOTSTRAP CON?DENCE INTERVALS FOR QUANTILES BOOTSTRAP IN REGRESSION RESIDUAL BOOTSTRAP CON?DENCE INTERVALS DISTRIBUTION ESTIMATES IN REGRESSION BOOTSTRAP FOR DEPENDENT DATA CONSISTENT BOOTSTRAP FOR STATIONARY AUTOREGRESSION BLOCK BOOTSTRAP METHODS OPTIMAL BLOCK LENGTH JACKKNIFE NOTATION AND MOTIVATING EXAMPLES BIAS CORRECTION BY THE JACKKNIFE VARIANCE ESTIMATION DELETE-D JACKKNIFE AND VON MISES FUNCTIONALS A NUMERICAL EXAMPLE JACKKNIFE HISTOGRAM PERMUTATION TESTS GENERAL PERMUTATION TESTS AND BASIC GROUP THEORY EXACT SIMILARITY OF PERMUTATION TESTS POWER OF PERMUTATION TESTS DENSITY ESTIMATION BASIC TERMINOLOGY AND SOME POPULAR METHODS MEASURES OF THE QUALITY OF DENSITY ESTIMATES CERTAIN NEGATIVE RESULTS MINIMAXITY CRITERION PERFORMANCE OF SOME POPULAR METHODS: A PREVIEW RATE OF CONVERGENCE OF HISTOGRAMS CONSISTENCY OF KERNEL ESTIMATES ORDER OF OPTIMAL BANDWIDTH AND SUPERKERNELS THE EPANECHNIKOV KERNEL CHOICE OF BANDWIDTH BY CROSS VALIDATION MAXIMUM LIKELIHOOD CV LEAST SQUARES CV STONE’S RESULT COMPARISON OF BANDWIDTH SELECTORS AND RECOMMENDATIONS L1 OPTIMAL BANDWIDTHS VARIABLE BANDWIDTHS STRONG UNIFORM CONSISTENCY AND CON?DENCE BANDS MULTIVARIATE DENSITY ESTIMATION AND CURSE OF DIMENSIONALITY KERNEL ESTIMATES AND OPTIMAL BANDWIDTHS ESTIMATING A UNIMODAL DENSITY AND THE GRENANDER ESTIMATE THE GRENANDER ESTIMATE MODE ESTIMATION AND CHERNOFF’S DISTRIBUTION MIXTURE MODELS AND NONPARAMETRIC DECONVOLUTION MIXTURES AS DENSE FAMILIES Z DISTRIBUTIONS AND OTHER GAUSSIAN MIXTURES AS USEFUL MODELS ESTIMATION METHODS AND THEIR PROPERTIES: FINITE MIXTURES MAXIMUM LIKELIHOOD MINIMUM DISTANCE METHOD MOMENT ESTIMATES ESTIMATION IN GENERAL MIXTURES STRONG CONSISTENCY AND WEAK CONVERGENCE OF THE MLE CONVERGENCE RATES FOR FINITE MIXTURES AND NONPARAMETRIC DECONVOLUTION NONPARAMETRIC DECONVOLUTION HIGH-DIMENSIONAL INFERENCE AND FALSE DISCOVERY CHI-SQUARE TESTS WITH MANY CELLS AND SPARSE MULTINOMIALS REGRESSION MODELS WITH MANY PARAMETERS: THE PORTNOY PARADIGM MULTIPLE TESTING AND FALSE DISCOVERY: EARLY DEVELOPMENTS FALSE DISCOVERY: DE?NITIONS, CONTROL, AND THE BENJAMINI-HOCHBERG RULE DISTRIBUTION THEORY FOR FALSE DISCOVERIES AND POISSON AND FIRST-PASSAGE ASYMPTOTICS NEWER FDR CONTROLLING PROCEDURES STOREY-TAYLOR-SIEGMUND RULE HIGHER CRITICISM AND THE DONOHO-JIN DEVELOPMENTS FALSE NONDISCOVERY AND DECISION THEORY FORMULATION GENOVESE-WASSERMAN PROCEDURE ASYMPTOTIC EXPANSIONS LOWER BOUNDS ON THE NUMBER OF FALSE HYPOTHESES B¨UHLMANN-MEINSHAUSEN-RICE METHOD THE DEPENDENT CASE AND THE HALL-JIN RESULTS INCREASING AND MULTIVARIATE TOTALLY POSITIVE DISTRIBUTIONS HIGHER CRITICISM UNDER DEPENDENCE: HALL-JIN RESULTS A COLLECTION OF INEQUALITIES IN PROBABILITY, LINEAR ALGEBRA, AND ANALYSIS PROBABILITY INEQUALITIES IMPROVED BONFERRONI INEQUALITIES CONCENTRATION INEQUALITIES TAIL INEQUALITIES FOR SPECI?C DISTRIBUTIONS INEQUALITIES UNDER UNIMODALITY MOMENT AND MONOTONICITY INEQUALITIES INEQUALITIES IN ORDER STATISTICS INEQUALITIES FOR NORMAL DISTRIBUTIONS INEQUALITIES FOR BINOMIAL AND POISSON DISTRIBUTIONS INEQUALITIES IN THE CENTRAL LIMIT THEOREM MARTINGALE INEQUALITIES MATRIX INEQUALITIES RANK, DETERMINANT, AND TRACE INEQUALITIES EIGENVALUE AND QUADRATIC FORM INEQUALITIES SERIES AND POLYNOMIAL INEQUALITIES INTEGRAL AND DERIVATIVE INEQUALITIES