What Would it take to Change an Inference? Quantifying the Robustness of Causal Inferences in Adolescent Research

  • Half-Day Pre-Conference
  • 9:30 AM - 2:30 PM
  • $35.00

The phrase “But have you controlled for …” is fundamental to inference but can also create a quandary for those social scientists, especially those who study adolescents. Even after controlling for the most likely alternative explanations there may be some alternative explanation that cannot be ruled out with observed data. For example, in studying the effects of peers on adolescent behavior one may not be able to account for all the factors that affect selection of peers. In response, this workshop helps those who research adolescents use sensitivity analyses to quantify the unobserved conditions necessary to nullify an inference. It draws on innovations across the social and behavioral sciences, including sociology, psychology, economics, and epidemiology. In part I, we will use Rubin’s causal model to interpret how much bias there must be to invalidate an inference in terms of replacing observed cases with counterfactual cases or cases from an unsampled population. This supports statements such as “to nullify the inference, one would have to replace XX% of the data with cases for which there was no effect of the focal predictor.” This defines the Robustness of Inference to Replacement which can be extended to a variety of analyses (e.g., logistic regression and multilevel models). In part II, we will quantify the robustness of causal inferences in terms of correlations associated with unobserved variables or in unsampled populations. This leverages a Directed Acyclic Graph (DAG) conceptualization that supports statements such as “an omitted variable would have to be correlated at qqq with both the focal predictor and outcome to change the inference.” Calculations for bivariate and multivariate analysis will be presented in the app: http://konfound-it.com as well as the konfound suite in Stata and R. The format will be a mixture of presentation, individual exploration, and group work. Participants should be comfortable with basic regression and multiple regression or should bring their own laptop or be willing to work with another who has a laptop. Participants may choose to bring to the course an example of an inference from a published study or their own work, as well as data analyses they are currently conducting.

Kenneth Frank received his Ph.D. in measurement, evaluation and statistical analysis from the School of Education at the University of Chicago in 1993. He is a member of the National Academy of Education and MSU Foundation professor of Sociometrics, professor in Counseling, Educational Psychology and Special Education; and adjunct (by courtesy) in Fisheries and Wildlife and Sociology at Michigan State University. His substantive interests include the study of schools as organizations, social structures of students and teachers and school decision-making, and social capital. His substantive areas are linked to several methodological interests: social network analysis, sensitivity analysis and causal inference (http://konfound-it.com), and multi-level models. His publications include quantitative methods for representing relations among actors in a social network, robustness indices for sensitivity analysis for causal inferences, and the effects of social capital in schools, natural resource management, and other social contexts.

Dr. Frank’s current research projects include:
Sensitivity Analysis http://konfound-it.com;
Study of Ambitious Math Instruction: https://www.studyofelemmath.org/;
The Balancing Voices framework for school governance https://www.balancingvoices.org/;
Teachers’ use of social media (https://www.teachersinsocialmedia.com/);
Implementation of the Carbon-Time science curriculum (http://carbontime.bscs.org/);
Social network intervention in natural resources and construction management;
Effects of teaches’ critical race consciousness on disciplinart gaps;

His work can be found at: https://sites.google.com/msu.edu/kenfrank