5.2 — Difference-in-Differences

ECON 480 • Econometrics • Fall 2022

Dr. Ryan Safner
Associate Professor of Economics

safner@hood.edu
ryansafner/metricsF22
metricsF22.classes.ryansafner.com

Contents

Difference-in-Differences Models

Example I: HOPE in Georgia

Generalizing DND Models

Example II: “The” Card-Kreuger Minimum Wage Study

Clever Research Designs Identify Causality

Again, this toolkit of research designs to identify causal effects is the economist’s comparative advantage that firms and governments want!

Difference-in-Differences Models

Natural Experiments

Difference-in-Differences Models I

  • Often, we want to examine the consequences of a change, such as a law or policy intervention

Difference-in-Differences Models I

  • Often, we want to examine the consequences of a change, such as a law or policy intervention

Example

  • How do States that implement policy X see changes in Y
    • Treatment: States that implement X
    • Control: States that did not implement X

  • If we have panel data with observations for all states before and after the change…

  • Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Differences Models I

  • Often, we want to examine the consequences of a change, such as a law or policy intervention

Example

  • How do States that implement policy X see changes in Y
    • Treatment: States that implement X
    • Control: States that did not implement X

  • If we have panel data with observations for all states before and after the change…

  • Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Differences Models II

  • The difference-in-differences (aka “diff-in-diff” or “DND”) estimator identifies treatment effect by differencing the difference pre- and post-treatment values of Y between treatment and control groups

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • Treatedi={1 if i is in treatment group0 if i is not in treatment groupAftert={1 if t is after treatment period0 if t is before treatment period
Control Treatment Group Diff (ΔYi)
Before β0 β0+β1 β1
After β0+β2 β0+β1+β2+β3 β1+β3
Time Diff (ΔYt) β2 β2+β3 β3 Diff-in-diff (ΔiΔt)

Example: Hot Dogs

  • Is there a discount when you get cheese and chili?
ABCDEFGHIJ0123456789
price
<dbl>
cheese
<dbl>
chili
<dbl>
2.0000
2.3510
2.3501
2.7011
4 rows

Example: Hot Dogs

  • Is there a discount when you get cheese and chili?
lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
(Intercept)2.00
cheese0.35
chili0.35
cheese:chili0.00
4 rows
  • Diff-n-diff is just a model with an interaction term between two dummies!

Visualizing Diff-in-Diff

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • Control group (Treatedi=0)

  • ^β0: value of Y for control group before treatment

  • ^β2: time difference (for control group)

Visualizing Diff-in-Diff

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • Control group (Treatedi=0)

  • ^β0: value of Y for control group before treatment

  • ^β2: time difference (for control group)

  • Treatment group (Treatedi=1)

Visualizing Diff-in-Diff

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • Control group (Treatedi=0)

  • ^β0: value of Y for control group before treatment

  • ^β2: time difference (for control group)

  • Treatment group (Treatedi=1)

  • ^β1: difference between groups before treatment

Visualizing Diff-in-Diff

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • Control group (Treatedi=0)

  • ^β0: value of Y for control group before treatment

  • ^β2: time difference (for control group)

  • Treatment group (Treatedi=1)

  • ^β1: difference between groups before treatment

  • ^β3: difference-in-differences (treatment effect)

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

  • ¯Yi for Control group after: ^β0+^β2

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

  • ¯Yi for Control group after: ^β0+^β2

  • ¯Yi for Treatment group before: ^β0+^β1

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

  • ¯Yi for Control group after: ^β0+^β2

  • ¯Yi for Treatment group before: ^β0+^β1

  • ¯Yi for Treatment group after: ^β0+^β1+^β2+^β3

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

  • ¯Yi for Control group after: ^β0+^β2

  • ¯Yi for Treatment group before: ^β0+^β1

  • ¯Yi for Treatment group after: ^β0+^β1+^β2+^β3

  • Group Difference (before): ^β1

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

  • ¯Yi for Control group after: ^β0+^β2

  • ¯Yi for Treatment group before: ^β0+^β1

  • ¯Yi for Treatment group after: ^β0+^β1+^β2+^β3

  • Group Difference (before): ^β1

  • Time Difference: ^β2

Visualizing Diff-in-Diff II

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • ¯Yi for Control group before: ^β0

  • ¯Yi for Control group after: ^β0+^β2

  • ¯Yi for Treatment group before: ^β0+^β1

  • ¯Yi for Treatment group after: ^β0+^β1+^β2+^β3

  • Group Difference (before): ^β1

  • Time Difference: ^β2

  • Difference-in-differences: ^β3 (treatment effect)

Comparing Group Means (Again)

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

Control Treatment Group Diff (ΔYi)
Before β0 β0+β1 β1
After β0+β2 β0+β1+β2+β3 β1+β3
Time Diff (ΔYt) β2 β2+β3 Diff-in-diff ΔiΔt:β3

Key Assumption: Counterfactual

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • Key assumption for DND: time trends (for treatment and control) are parallel

  • Treatment and control groups assumed to be identical over time on average, except for treatment

  • Counterfactual: if the treatment group had not recieved treatment, it would have changed identically over time as the control group (^β2)

Key Assumption: Counterfactual

ˆYit=β0+β1Treatedi+β2Aftert+β3(Treatedi×Aftert)+uit

  • If the time-trends would have been different, a biased measure of the treatment effect (^β3)!

Example I: HOPE in Georgia

Diff-in-Diff Example I

Example

In 1993 Georgia initiated a HOPE scholarship program to let state residents with at least a B average in high school attend public college in Georgia for free. Did it increase college enrollment?

  • Micro-level data on 4,291 young individuals
  • InCollegeit={1 if i is in college during year t0 if i is not in college during year t1
  • Georgiai={1 if i is a Georgia resident0 if i is not a Georgia resident
  • Aftert={1 if t is after 19920 if t is after 1992

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

  1. Note: With a dummy dependent (Y) variable, coefficients estimate the probability Y=1, i.e. the probability a person is enrolled in college

Diff-in-Diff Example II

  • We can use a DND model to measure the effect of HOPE scholarship on enrollments

  • Georgia and nearby States, if not for HOPE, changes should be the same over time

  • Treatment period: after 1992
  • Treatment: Georgia
  • Difference-in-differences: ΔiΔtEnrolled=(GAafter−GAbefore)−(neighborsafter−neighborsbefore)
  • Regression equation:

^Enrolledit=β0+β1Georgiai+β2Aftert+β3(Georgiai×Aftert)

Example: Data

hope
ABCDEFGHIJ0123456789
StateCode
<fct>
Age
<dbl>
Year
<fct>
Weight
<dbl>
Age18
<dbl>
56198913960
56198910800
5618899241
5619898910
56198913950
56188911061
5619899650
5618899581
56198910060
56198911830
Next
123456
...
430
Previous
1-10 of 4,291 rows | 1-5 of 11 columns

Example: Data

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Example: Regression

DND_reg <- lm(InCollege ~ Georgia + After + Georgia*After, data = hope)
DND_reg %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
(Intercept)0.4057826520.0109239037.14631824.221545e-262
Georgia-0.1052362040.03778114-2.78541655.369384e-03
After-0.0044596090.01585224-0.28132357.784758e-01
Georgia:After0.0893298280.048893291.82703646.776378e-02
4 rows

^Enrolledit=0.406−0.105Georgiai−0.004Aftert+0.089(Georgiai×Aftert)

Example: Interpretting the Regression

^Enrolledit=0.406−0.105Georgiai−0.004Aftert+0.089(Georgiai×Aftert)

  • β0: A non-Georgian before 1992 was 40.6% likely to be a college student
  • β1: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
  • β2: After 1992, non-Georgians are 0.4% less likely to be college students

  • β3: After 1992, Georgians are 8.9% more likely to enroll in colleges than neighboring states

  • Treatment effect: HOPE increased enrollment likelihood by 8.9%

Example: Comparing Group Means

^Enrolledit=0.406−0.105Georgiai−0.004Aftert+0.089(Georgiai×Aftert)

  • A group mean for a dummy Y is E[Y=1], i.e. the probability a student is enrolled:

  • Non-Georgian enrollment probability pre-1992: β0=0.406

  • Georgian enrollment probability pre-1992: β0+β1=0.406−0.105=0.301
  • Non-Georgian enrollment probability post-1992: β0+β2=0.406−0.004=0.402
  • Georgian enrollment probability post-1992: β0+β1+β2+β3=0.406−0.105−0.004+0.089=0.386

Example: Comparing Group Means in R

# group mean for non-Georgian before 1992
hope %>%
  filter(Georgia == 0,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.4057827
1 row
# group mean for non-Georgian AFTER 1992
hope %>%
  filter(Georgia == 0,
         After == 1) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.401323
1 row

Example: Comparing Group Means in R

# group mean for Georgian before 1992
hope %>%
  filter(Georgia == 1,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3005464
1 row
# group mean for Georgian AFTER 1992
hope %>%
  filter(Georgia == 1,
         After == 1) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3854167
1 row

Example: Diff-in-Diff Summary

^Enrolledit=0.406−0.105Georgiai−0.004Aftert+0.089(Georgiai×Aftert)

Neighbors Georgia Group Diff (ΔYi)
Before 0.406 0.301 −0.105
After 0.402 0.386 0.016
Time Diff (ΔYt) −0.004 0.085 Diff-in-diff: 0.089

ΔiΔtEnrolled=(GAafter−GAbefore)−(neighborsafter−neighborsbefore)=(0.386−0.301)−(0.402−0.406)=(0.085)−(−0.004)=0.089

Diff-in-Diff Summary & Data

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Example: Diff-in-Diff Graph

Example: Diff-in-Diff Graph

Generalizing DND Models

Generalizing DND Models

  • DND can be generalized with a two-way fixed effects model:

ˆYit=β1(Treatedi×Aftert)+αi+θt+νit

  • αi: group fixed effects (treatments/control groups)
  • θt: time fixed effects (pre/post treatment)
  • β1: diff-in-diff (interaction effect, β3 from before)
  • Flexibility: many periods (not just before/after), many different treatment(s)/groups, and treatment(s) can occur at different times to different units (so long as some do not get treated)
  • Can also add control variables that vary within units and over time

ˆYit=β1(Treatedi×Aftert)+β2Xit+⋯+αi+θt+νit

Our Example, Generalized I

^Enrolledit=β1(Georgiai×Aftert)+αi+θt+

  • StateCode is a variable for the State ⟹ create State fixed effect (αi)

  • Year is a variable for the year ⟹ create year fixed effect (θt)

Our Example, Generalized II

Using LSDV method:

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
(Intercept)0.4180574780.0226113318.48885171.734550e-73
Georgia-0.1415012550.03936119-3.59494363.281224e-04
After0.0753405940.031280212.40857061.605717e-02
factor(StateCode)57-0.0141811120.02739708-0.51761406.047544e-01
factor(StateCode)58NANANANA
factor(StateCode)59-0.0623785400.01954266-3.19191721.423556e-03
factor(StateCode)62-0.1326502710.02806143-4.72713832.350298e-06
factor(StateCode)63-0.0051038680.02627780-0.19422748.460071e-01
factor(Year)900.0466088450.028336251.64484861.000745e-01
factor(Year)910.0322757890.028568771.12975772.586417e-01
Next
12
Previous
1-10 of 17 rows

Our Example, Generalized II

Using fixest

library(fixest)
DND_fe_2 <- feols(InCollege ~ Georgia*After | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_2 %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
Georgia:After0.09142020.00564329816.199781.633762e-05
1 row

^InCollegeit=0.091(Georgiai×Afterit)+αi+θt

Our Example, Generalized, with Controls II

Using LSDV Method

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
(Intercept)0.7355742220.0299071024.59530371.155308e-121
Georgia-0.1089402760.04765262-2.28613422.231699e-02
After-0.0057535530.03737027-0.15396078.776512e-01
factor(StateCode)57-0.0434060730.03047696-1.42422571.544869e-01
factor(StateCode)58NANANANA
factor(StateCode)59-0.0531756450.02306160-2.30580922.119033e-02
factor(StateCode)62-0.1161046150.03283310-3.53620604.121675e-04
factor(StateCode)630.0073898660.030564440.24177998.089675e-01
factor(Year)900.0393643150.033262911.18342972.367342e-01
factor(Year)910.0292279690.033478500.87303703.827140e-01
Next
12
Previous
1-10 of 19 rows

Our Example, Generalized, with Controls II

Using fixest

DND_fe_controls_2 <- feols(InCollege ~ Georgia*After + Black + LowIncome | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_controls_2 %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
Black-0.093987150.01273233-7.3817690.0007172767
LowIncome-0.301724260.03066188-9.8403690.0001846526
Georgia:After0.023436790.012818381.8283740.1270331132
3 rows

^InCollegeit=0.023(Georgiai×Afterit)−0.094Blackit−0.302LowIncomeit

Our Example, Generalized, with Controls III

No FE TWFE TWFE
Constant 0.40578***
(0.01092)
Georgia −0.10524***
(0.03778)
After −0.00446
(0.01585)
Georgia x After 0.08933* 0.09142*** 0.02344
(0.04889) (0.00564) (0.01282)
Black −0.09399***
(0.01273)
LowIncome −0.30172***
(0.03066)
n 4291 4291 2967
Adj. R2 0.00
SER 0.49 0.49 0.47
* p < 0.1, ** p < 0.05, *** p < 0.01

The Findings

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Intuition behind DND

  • Diff-in-diff models are the quintessential example of exploiting natural experiments

  • A major change at a point in time (change in law, a natural disaster, political crisis) separates groups where one is affected and another is not—identifies the effect of the change (treatment)

  • One of the cleanest and clearest causal identification strategies

Example II: “The” Card-Kreuger Minimum Wage Study

Example: ”The” Card-Kreuger Minimum Wage Study I

Example

The controversial minimum wage study, Card & Kreuger (1994) is a quintessential (and clever) diff-in-diff. ]

Card, David, Krueger, Alan B, (1994), “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania,” American Economic Review 84 (4): 772–793

Card & Kreuger (1994): Background I

  • Card & Kreuger (1994) compare employment in fast food restaurants on New Jersey and Pennsylvania sides of border between February and November 1992.

  • Pennsylvania & New Jersey both had a minimum wage of $4.25 before February 1992

  • In February 1992, New Jersey raised minimum wage from $4.25 to $5.05

Card & Kreuger (1994): Background II

  • If we look only at New Jersey before & after change:
    • Omitted variable bias: macroeconomic variables (there’s a recession!), weather, etc.
      • Including PA as a control will control for these time-varying effects if they are national trends
  • Surveyed 400 fast food restaurants on each side of the border, before & after min wage increase
    • Key assumption: Pennsylvania and New Jersey follow parallel trends,
      • Counterfactual: if not for the minimum wage increase, NJ employment would have changed similar to PA employment

Card & Kreuger (1994): Comparisons

Card & Kreuger (1994): Summary I

Card & Kreuger (1994): Summary II

Card & Kreuger (1994): Model

^Employmentit=β0+β1NJi+β2Aftert+β3(NJi×Aftert)

  • PA Before: β0

  • PA After: β0+β2

  • NJ Before: β0+β1

  • NJ After: β0+β1+β2+β3

  • Diff-in-diff: (NJafter−NJbefore)−(PAafter−PAbefore)

PA NJ Group Diff (ΔYi)
Before β0 β0+β1 β1
After β0+β2 β0+β1+β2+β3 β1+β3
Time Diff (ΔYt) β2 β2+β3 Diff-in-diff ΔiΔt:β3

Card & Kreuger (1994): Results

ECON 480 — Econometrics

1
5.2 — Difference-in-Differences ECON 480 • Econometrics • Fall 2022 Dr. Ryan Safner Associate Professor of Economics safner@hood.edu ryansafner/metricsF22 metricsF22.classes.ryansafner.com

  1. Slides

  2. Tools

  3. Close
  • Title Slide
  • Contents
  • Clever Research Designs Identify Causality
  • Difference-in-Differences Models
  • Natural Experiments
  • Difference-in-Differences Models I
  • Difference-in-Differences Models I
  • Difference-in-Differences Models I
  • Difference-in-Differences Models II
  • Example: Hot Dogs
  • Example: Hot Dogs
  • Visualizing Diff-in-Diff
  • Visualizing Diff-in-Diff
  • Visualizing Diff-in-Diff
  • Visualizing Diff-in-Diff
  • Visualizing Diff-in-Diff II
  • Visualizing Diff-in-Diff II
  • Visualizing Diff-in-Diff II
  • Visualizing Diff-in-Diff II
  • Visualizing Diff-in-Diff II
  • Visualizing Diff-in-Diff II
  • Visualizing Diff-in-Diff II
  • Comparing Group Means (Again)
  • Key Assumption: Counterfactual
  • Key Assumption: Counterfactual
  • Example I: HOPE in Georgia
  • Diff-in-Diff Example I
  • Diff-in-Diff Example II
  • Example: Data
  • Example: Data
  • Example: Regression
  • Example: Interpretting the Regression
  • Example: Comparing Group Means
  • Example: Comparing Group Means in R
  • Example: Comparing Group Means in R
  • Example: Diff-in-Diff Summary
  • Diff-in-Diff Summary & Data
  • Example: Diff-in-Diff Graph
  • Example: Diff-in-Diff Graph
  • Generalizing DND Models
  • Generalizing DND Models
  • Our Example, Generalized I
  • Our Example, Generalized II
  • Our Example, Generalized II
  • Our Example, Generalized, with Controls II
  • Our Example, Generalized, with Controls II
  • Our Example, Generalized, with Controls III
  • The Findings
  • Intuition behind DND
  • Example II: “The” Card-Kreuger Minimum Wage Study
  • Example: ”The” Card-Kreuger Minimum Wage Study I
  • Card & Kreuger (1994): Background I
  • Card & Kreuger (1994): Background II
  • Card & Kreuger (1994): Comparisons
  • Card & Kreuger (1994): Summary I
  • Card & Kreuger (1994): Summary II
  • Card & Kreuger (1994): Model
  • Card & Kreuger (1994): Results
  • f Fullscreen
  • s Speaker View
  • o Slide Overview
  • e PDF Export Mode
  • b Toggle Chalkboard
  • c Toggle Notes Canvas
  • d Download Drawings
  • ? Keyboard Help