DEVELOPMENTS IN STATISTICS
1 Direct Gibbs posterior inference on risk minimizers: Construction, concentration, and calibration 1 1 Introduction 1 2 Gibbs posterior distributions 4 21 Problem setup 4 22 Definition 6 23 FAQs 8 24 Illustrations 11 3 Asymptotic theory 14 31 Objectives and general strategies 14 32 Consistency 15 33 Concentration rates 16 34 Distributional approximations 18 4 Learning rate selection 21 5 Numerical examples 24 51 Quantile regression 24 52 Classification 26 53 Nonlinear regression 28 6 Further details 30 61 Things we did not discuss 30 62 Open problems 31 7 Conclusion 32 Acknowledgments 33 Appendix 34 A1 Proofs 34 References 38 2 Bayesian selective inference 43 ´ 1 Introduction 43 2 Bayes and selection 45 21 Fixed and random parameters 48 3 Noninformative priors for selective inference 50 31 Noninformative priors for exponential families 56 4 Discussion 64 References 65 3 Dependent Bayesian multiple hypothesis testing 67 1 Introduction 67 2 Bayesian multiple hypothesis testing 69 21 Preliminaries and setup 69 22 The decision problem 71 3 Dependent multiple testing 72 31 New error based criterion 74 32 Choice of G1,…,Gm 75 4 Simulation study 76 41 The postulated Bayesian model 76 42 Comparison criteria 77 43 Comparison of the results 77 5 Discussion 78 References 79 4 A new look at Bayesian uncertainty 83 1 Introduction 83 2 Missing data 86 3 Parametric martingale sequences 88 31 Langevin posterior 90 4 Nonparametric martingale distributions 92 5 Illustrations 93 51 Parametric case 93 52 Nonparametric case 94 6 Mathematical theory 96 7 Discussion 98 Acknowledgments 100 References 100 5 50 shades of Bayesian testing of hypotheses 103 1 Introduction 103 2 Bayesian hypothesis testing 104 3 Improper priors united against hypothesis testing 107 4 The Jeffreys–Lindley paradox 109 5 Posterior predictive p-values 110 6 A modest proposal 111 7 Conclusion 116 Acknowledgments 116 References 117 6 Inference approach to ground states of quantum systems 121 1 Introduction 121 2 The Jaynes’ maximum entropy methodology: Brief resume 122 3 The quantum maximum entropy approach 123 31 Preliminaries 123 4 Properties of SQ that make our approximate maximum entropy approach wave functions reasonable ones 124 41 SQ is a true Shannon’s ignorance function 124 42 Subject to the known quantities bk, the maximum value of SQ is unique 125 43 The entropy SQ obeys an H-theorem 125 44 Our SQ-ground-state wave functions respect the virial theorem 126 45 The SQ-ground-state wave functions respect hypervirial theorems 127 46 Saturation 127 47 Speculation 127 5 Coulomb potential 127 51 Harmonic oscillator 128 52 Morse potential 128 53 Ground state of the quartic oscillator 129 54 A possible MEM extension 129 6 Noncommuting observables 129 7 Other entropic or information measures 130 8 Conclusions 133 References 133 7 MCMC for GLMMs 135 1 Introduction 135 2 Likelihood function for GLMMs 136 3 Conditional simulation for GLMMs 139 31 MALA for GLMMs 139 32 HMC for GLMMs 141 33 Data augmentation for GLMMs 143 4 MCMC for Bayesian GLMMs 147 41 MALA and HMC for Bayesian GLMMs 148 42 Data augmentation for Bayesian GLMMs 150 5 A numerical example 154 6 Discussion 156 References 157 8 Sparsity-aware Bayesian inference and its applications 161 1 Introduction 162 11 Quick summary of existing methods for sparse signal recovery 163 12 Bayesian approaches: Motivation and related literature 164 2 The hierarchical Bayesian framework 165 21 Gaussian scale mixtures and sparse Bayesian learning 166 22 SBL framework 167 23 Case study: Wireless channel estimation and SBL 169 3 Joint-sparse signal recovery 173 31 The MSBL algorithm 173 32 Expectation maximization in MSBL 174 33 An interesting interpretation of the MSBL cost function 174 34 A Covariance-matching framework for sparse support recovery using MMVs 175 35 Examples of covariance-matching algorithms for sparse support recovery 177 4 Exploiting intervector correlation 178 41 Intervector correlation: The Kalman SBL algorithm 179 42 Online sparse vector recovery 181 43 Case study (continued): Wireless channel estimation and the KSBL algorithm 185 5 Intravector correlations: The nested SBL algorithm 186 51 Nested SBL (B 6¼ IB) 189 6 Quantized sparse signal recovery 192 7 Other extensions 198 71 Decentralized SBL 198 72 Dictionary learning 199 73 Relationship with robust principal component analysis and sparse + low-rank decomposition 200 74 Deep unfolded SBL 202 8 Discussion and future outlook 202 References 203 9 Mathematical theory of Bayesian statistics where all models are wrong 209 1 Introduction 209 2 Mathematical theory of Bayesian statistics 211 21 DGP, model, and prior 211 22 Generalization loss and free energy 213 23 Regular theory 217 24 Singular theory 220 25 Phase transitions 225 3 Applications to statistics and machine learning 227 31 Model evaluation 227 32 Prior evaluation 230 33 Not iid cases 232 4 Conclusion 235 References 236 10 Geometry in sampling methods: A review on manifold MCMC and particle-based variational inference methods 239 1 Geometry consideration in sampling: Why bother? 239 2 Manifold and related concepts 242 21 Manifold 242 22 Tangent vector and vector field 244 23 Cotangent vector and differential form 246 24 Riemannian manifold 247 25 Measure 248 26 Divergence and Laplacian 249 27 Manifold embedding 250 3 Markov chain Monte Carlo on Riemannian manifolds 251 31 Technical description of general MCMC dynamics 252 32 Riemannian MCMC in coordinate space 253 33 Riemannian MCMC in embedded space 258 4 Particle-based variational inference methods 260 41 Stein variational gradient descent 261 42 The Wasserstein space 262 43 Geometric view of particle-based variational inference methods 266 44 Geometric view of MCMC dynamics and relation to ParVI methods 270 45 Variants and Techniques Inspired by the Geometric View 277 5 Conclusion 286 Acknowledgments 286 References 286