PRACTICAL STATISTICS

By KAREN MARTINEZ

1 An Overview of Sample Design and Weighting 1 11 Background and Terminology 1 12 Chapter Guide 7 Part I Designing Single-Stage Sample Surveys 2 Project 1: Design a Single-Stage Personnel Survey 15 21 Specifications for the Study 15 22 Questions Posed by the Design Team 16 23 Preliminary Analyses 18 24 Documentation 21 25 Next Steps 23 3 Sample Design and Sample Size for Single-Stage Surveys 25 31 Determining a Sample Size for a Single-Stage Design 26 311 Simple Random Sampling 28 312 Stratified Simple Random Sampling 43 32 Finding Sample Sizes When Sampling with Varying Probabilities 51 321 Probability Proportional to Size Sampling 51 322 Regression Estimates of Totals 59 33 Other Methods of Sampling 63 34 Estimating Population Parameters from a Sample 64 35 Special Topics 68 351 Rare Characteristics 68 352 Domain Estimates 70 36 More Discussion of Design Effects 75 37 Software for Sample Selection 76 371 R Packages 76 372 SAS PROC SURVEYSELECT 81 Exercises 84 4 Power Calculations and Sample Size Determination 91 41 Terminology and One-Sample Tests 92 42 Power in a One-Sample Test 97 43 Two-Sample Tests 103 431 Differences in Means 103 432 Differences in Proportions 108 433 Special Case: Relative Risk 112 434 Special Case: Effect Sizes 113 44 R Power Functions 113 45 Power and Sample Size Calculations in SAS 122 Exercises 125 5 Mathematical Programming 129 51 Multicriteria Optimization 130 52 Microsoft Excel Solver 133 53 SAS PROC NLP 145 54 SAS PROC OPTMODEL 150 55 R alabama Package 155 56 Accounting for Problem Variations 159 Exercises 160 6 Outcome Rates and Effect on Sample Size 163 61 Disposition Codes 164 62 Definitions of Outcome Rates 166 63 Sample Units with Unknown AAPOR Classification 171 64 Weighted Versus Unweighted Rates 173 65 Accounting for Sample Losses in Determining Initial Sample Size 174 651 Sample Size Inflation Rates at Work 174 652 Replicates 176 Exercises 178 7 The Personnel Survey Design Project: One Solution 185 71 Overview of the Project 185 72 Formulate the Optimization Problem 186 721 Objective Function 186 722 Decision Variables 187 723 Optimization Parameters 187 724 Specified Survey Constraints 188 73 One Solution 189 731 Power Analyses 189 732 Optimization Results 191 74 Additional Sensitivity Analysis 193 75 Conclusion 195 Part II Multistage Designs 8 Project 2: Designing an Area Sample 199 9 Designing Multistage Samples 203 91 Types of PSUs 204 92 Basic Variance Results 205 921 Two-Stage Sampling 205 922 Nonlinear Estimators in Two-Stage Sampling 212 923 More General Two-Stage Designs 215 924 Three-Stage Sampling 218 93 Cost Functions and Optimal Allocations for Multistage Sampling 225 931 Two-Stage Sampling When Numbers of Sample PSUs and Elements per PSU Are Adjustable 225 932 Three-Stage Sampling When Sample Sizes Are Adjustable 229 933 Two- and Three-Stage Sampling with a Fixed Set of PSUs 231 94 Estimating Measures of Homogeneity and Variance Components 235 941 Two-Stage Sampling 235 942 Three-Stage Sampling 239 943 Using Anticipated Variances 243 95 Stratification of PSUs 249 96 Identifying Certainties 250 Exercises 251 10 Area Sampling 257 101 Census Geographic Units 258 102 Census Data and American Community Survey Data 261 103 Units at Different Stages of Sampling 262 1031 Primary Sampling Units 263 1032 Secondary Sampling Units 264 1033 Ultimate Sampling Units 266 104 Examples of Area Probability Samples 266 1041 Current Population Survey 267 1042 National Survey on Drug Use and Health 270 1043 Panel Arbeitsmarkt und Soziale Sicherung 271 105 Composite MOS for Areas 273 1051 Designing the Sample from Scratch 273 1052 Using the Composite MOS with an Existing PSU Sample 277 106 Effects of Population Change: The New Construction Issue 282 107 Special Address Lists 288 Exercises 290 11 The Area Sample Design: One Solution 293 Part III Survey Weights and Analyses 12 Project 3: Weighting a Personnel Survey 303 13 Basic Steps in Weighting 307 131 Overview of Weighting 308 132 Theory of Weighting and Estimation 309 133 Base Weights 311 134 Adjustments for Unknown Eligibility 314 135 Adjustments for Nonresponse 316 1351 Weighting Class Adjustments 319 1352 Propensity Score Adjustments 321 1353 Classification Algorithms 338 136 Collapsing Predefined Classes 342 137 Weighting for Multistage Designs 343 138 Next Steps in Weighting 345 Exercises 346 14 Calibration and Other Uses of Auxiliary Data in Weighting 349 141 Weight Calibration 351 142 Poststratified and Raking Estimators 353 143 GREG and Calibration Estimation 361 1431 Links Between Models, Sample Designs, and Estimators—Special Cases 363 1432 More General Examples 365 144 Weight Variability 374 1441 Quantifying the Variability 375 1442 Methods to Limit Variability 381 Exercises 390 15 Variance Estimation 397 151 Exact Methods 398 152 Linear Versus Nonlinear Estimators 400 153 Linearization Variance Estimation 402 1531 Estimation Method 402 1532 Confidence Intervals and Degrees of Freedom 406 1533 Accounting for Non-negligible Sampling Fractions 408 1534 Domain Estimation 410 1535 Assumptions and Limitations 411 1536 Special Cases: Poststratification and Quantiles 412 1537 Handling Multiple Weighting Steps with Linearization 417 154 Replication 418 1541 Jackknife Replication 418 1542 Balanced Repeated Replication 426 1543 Bootstrap 430 155 Combining PSUs or Strata 437 1551 Combining to Reduce the Number of Replicates 437 1552 How Many Groups and Which Strata and PSUs to Combine 441 1553 Combining Strata in One-PSU-per-Stratum Designs 443 156 Handling Certainty PSUs 444 Exercises 448 16 Weighting the Personnel Survey: One Solution 453 161 The Data Files 454 162 Base Weights 455 163 Disposition Codes and Mapping into Weighting Categories 456 164 Adjustment for Unknown Eligibility 459 165 Variables Available for Nonresponse Adjustment 460 166 Nonresponse Adjustments 462 167 Calibration to Population Counts 466 168 Writing Output Files 474 169 Example Tabulations 475 Part IV Other Topics 17 Multiphase Designs 479 171 What is a Multiphase Design? 480 172 Examples of Different Multiphase Designs 482 1721 Double Sampling for Stratification 482 1722 Nonrespondent Subsampling 485 1723 Responsive Designs 491 1724 General Multiphase Designs 494 173 Survey Weights 494 1731 Base Weights 494 1732 Analysis Weights 498 174 Estimation 502 1741 Descriptive Point Estimation 502 1742 Variance Estimation 504 1743 Generalized Regression Estimator (GREG) 510 175 Design Choices 513 1751 Multiphase Versus Single Phase 514 1752 Sample Size Calculations 515 176 R Software 523 Exercises 527 18 Process Control and Quality Measures 531 181 Design and Planning 532 182 Quality Control in Frame Creation and Sample Selection 534 183 Monitoring Data Collection 536 184 Performance Rates and Indicators 540 185 Data Editing 543 1851 Editing Disposition Codes 544 1852 Editing the Weighting Variables 545 186 Quality Control of Weighting Steps 546 187 Specification Writing and Programming 549 188 Project Documentation and Archiving 551