Difference-in-differences in depth: A field guide to modern methods

Description

Course overview

Textbook difference-in-differences (DiD) can conveniently be estimated using a simple least squares regression. Until 10 years ago, this approach was also assumed to work in more complicated settings, e.g. where treatment is non-binary or rolled out over time, or where parallel trends only holds conditional on covariates.

How wrong we were! Over the last decade, many important problems with this approach have been uncovered, affecting almost every departure from the textbook setting. At the same time, new methods have been developed that address these challenges, carefully defining parameters of interest and ensuring methods successfully recover them.

This course will provide detailed, hands-on training of the key developments in this new DiD literature, ensuring participants leave well equipped to tackle the wide range of practical challenges encountered implementing DiD in the wild. It will be particularly suited to civil servants and practitioners working in policy evaluation, those in industry applying causal inference techniques, and PhD students and junior faculty who need a thorough grounding for their research.

Course content

The course will cover the following six topics

Fundamentals A review of the textbook setting including identification and alternative approaches to estimation

Covariates Why throwing covariates into a linear regression doesn’t work, and what to do instead depending on what form the covariates take

Functional form What functional form assumptions DiD makes, why this matters, and implications for how to choose functional form

Treatment A detailed exploration of different treatment variants – both in terms of timing (staggered adoption and non-absorbing), and type (multi-valued discrete and continuous) – the problems they create, and new methods developed for each

Standard errors Why packaged standard errors can be wildly misleading and how to deal with different error structures, including serial correlation and clustering

Assumptions What restrictions parallel trends imposes, challenges with common approaches to validate parallel trends, and more robust alternatives

Course format

The course runs in-person over three consecutive days with around seven hours of lectures, hands-on exercises and Q&A each day.

Breakdown of topics

Indicative class timings

Trainer bio

Jonathan Shaw is an applied economist with over 20 years’ experience using quasi-experimental methods to evaluate labour market policies and study the impact of financial products. He has published in some of the top economics and finance journals as well as providing technical advice for government departments and central banks in the UK and abroad. He currently works at the Financial Conduct Authority and is a Research Associate at the Institute for Fiscal Studies. His website is here.

Course prerequisites

This is a graduate-level course is targeted towards those with a solid grasp of the textbook DiD model e.g. at the level of the IFS-Cemmap Policy Evaluation Methods training course or Mostly Harmless Econometrics (Angrist and Pischke, 2008)

The material assumes a working knowledge of causal inference and coding and includes a mixture of econometric principles, simple proofs and hands-on exercises in Stata or R. You need to know potential outcomes notation and how to work with conditional expectations, as well as concepts like identification, estimation, consistency and efficiency.

Participants must bring their own laptop with either Stata or R installed, along with all relevant packages (a list of which will be provided in advance).

Learning outcomes

After completing the training, participants will

· Understand the problems with the standard regression-based approach to DiD

· Have a solid grasp of new methods that address these shortcomings

· Be ready to implement these methods using either Stata or R

· Be equipped to think through other settings not covered in the course

Background reading

No preparatory reading is required for this course, so long as the course prerequisites above are satisfied.

The following two resources give a reasonable sense of the type of material to be covered, though the course will be at a somewhat more technical level.

Cunningham, S. (2021). Causal inference: The mixtape. Yale University Press. [DiD chapters only]

Roth, J., Sant’Anna, P. H., Bilinski, A., & Poe, J. (2023). What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. Journal of Econometrics, 235(2), 2218-2244.

Questions to include on registration form

In a sentence, describe your level of experience with DiD

In a sentence, what do you hope to get out of the course?

Given the topics the course will cover, are there any specific questions you would like to see covered? (No guarantees but we will incorporate where possible) [optional]

Have you seen any particularly interesting or innovative applications of DiD? Please give the references here [optional]