Social Science Job Candidate

The 2012 presidential election saw an explosion of publicly available polling conducted by a wide variety of organizations ranging from university survey centers to large news organizations. More restrictive laws on predictive dialing of cell phones, the rise of a cell phone only population, and behavioral changes in voters' willingness to respond to such polls all increase the risk of unit non-response bias, which has the potential to severely bias the findings of these polls. This was seen in the 2013 Gubernatorial election in Virginia, where public polls were roughly five points more Democratic, on average, than the final election results. There was also large variance in the findings of individual polls conducted in the final weeks preceding the election. This paper first considers sources of bias caused by unit non-response, breaking down the bias into features that can be addressed in the design and post hoc adjustment stage, and unobservable behavioral bias that must be assumed away. Following the bias decomposition, current design phase approaches are considered. The paper then considers a new sampling method, response rate sampling, that leverages past data on responsiveness to surveys to alleviate bias due to observable imbalance resulting from unit non-response. Post hoc weighting methods for correcting for observed imbalances are then considered, including a recommendation for a novel way of using traditional raking methods. The paper concludes with an application of response rate sampling and a new weighting method used in Obama for America's (OFA) analytics polling methodology, one of the largest and most accurate polling operations in democratic politics.  OFA's internal analytics polling served as the data collection method for the campaign's successful predictive modeling operation.