Title: Butterfly Effect in Claims Data: Small Changes in Design Elements, Large Impacts on Causal Inference
Presenter: Rima Izem, PhD
LUNCH WILL BE PROVIDED
Abstract: Cohort studies using real world evidence from claims databases have been part of medical product post-market safety assessment for over a decade. In these studies, design elements are tailored to the main inference question of whether a drug exposure causes an adverse outcome. Some design elements are universal to all data sources while others are unique to claims data (e.g., the pharmacy dispensing record stockpiling algorithm). Our study investigated whether small changes in small design elements, coming for example from different interpretations of the same published information, can impact causal inference. This study used a multi-factorial design to assess impact of co-varying multiple design elements on different stages of the estimation process, from cohort identification to risk assessment. Data source and main design elements of a test case remained fixed but some elements co-varied across different study designs. Our results show that small changes in use of the Index Date and the stockpiling algorithm impact cohort size, length of follow-up and causal estimates. Standardizing definitions of these design elements will help minimize a study’s potential bias and facilitate replication of study findings.