Difference-in-Differences (DID) is a popular technique used to estimate treatment effects when treatment status is not assigned by the researcher. For example, if we want to know the effect of a minimum wage increase on the unemployment rate, we are unable to randomly increase or decrease the minimum wage in all 50 states. Instead, some states would naturally increase their minimum while other states leave their minimum wage at the current level. We assume those states that increased the minimum wage are in the treatment group and those states that did not change the minimum wage are used as controls. If we observe states before and after the treated states changed the wage, we can use DID to estimate the effect of the minimum wage on the unemployment rate.

The DID regression equation is:

\(Outcome_{it}=\beta_0+\beta_1Treated_i+\beta_2Post_t+\beta_3Treated_i*Post_t+\sum_{j=4}^{J}\beta_jx_{jit}+\epsilon_{it}\)

\(Outcome_{it}\) is the outcome of interest for group \(i\) at time \(t\). Groups could measure states, countries, or group of individuals. For example, \(Outcome_{it}\) could measure the unemployment rate in state \(i\) at time \(t\). \(Treated_i\) is a dummy variable equal to one if group \(i\) ever receives the treatment and 0 otherwise. \(Post_t\) is a dummy variable equal to one after the treated states began treatment and 0 otherwise. The variables in \(\sum_{j=4}^{J}\beta_jx_{jit}\) are additional control variables that may be correlated with the outcome of interest and treatment status.

I will work through an example that estimates the impact of a change in worker’s compensation benefits on time out of work. This data is from Meyer, Viscusi, & Durban (1995).

# load packages
library(readxl)
library(tidyverse)
library(ggplot2)
library(stargazer)

# loading data
data<- data.frame(read_excel('injury.xlsx')) %>%
  na.omit()

# Kentucky sample
data_ky<- data %>%
  filter(ky==1)

# Michigan sample
data_mi<- data %>%
  filter(mi==1)

The dependent variable is \(durat\) and measures time taken off work for an injury. \(afchange\) is equal to one after the policy change. This is the equivalent of \(Post_t\) in the DID equation above. Since the policy only impacted high-earning individuals, the variable \(highearn\) indicates treatment status. \(highearn\) is equal to one of the individual is a high earner (treated) and zero otherwise (low earner). The variables \(male\) - \(construc\) are control variables that may be correlated with both earnings and time taken off work for an injury (\(x_{jit}\)).

We can compare the earnings of high- and low-earners before and after the policy change.

# summary statistics by group and by time

means<-round(data %>%
  group_by(afchnge, highearn) %>%
  summarise_at(vars(durat), list('Average' = mean)), digits=2)

# displaying table
data.frame(means)
##   afchnge highearn Average
## 1       0        0    7.52
## 2       0        1   10.71
## 3       1        0    8.59
## 4       1        1   14.04

On average, low-earners were off work 7.52 weeks before the policy change and 8.59 weeks after. High earners were off work an average of 10.71 weeks before the policy change and 14.04 weeks after. I will estimate the basic DID model without additional controls to show how the regression equation estimates the group means above.

# regression
# the : multiplies the two variables together so you do not have 
# create the interaction term before estimating the regression
reg1<-lm(durat~highearn+afchnge+highearn:afchnge, data=data)
stargazer(reg1, type='text',omit.stat=c('f','ser'), digits=2)
## 
## ============================================
##                      Dependent variable:    
##                  ---------------------------
##                             durat           
## --------------------------------------------
## highearn                   3.19***          
##                            (0.81)           
##                                             
## afchnge                     1.07            
##                            (0.74)           
##                                             
## highearn:afchnge            2.26*           
##                            (1.17)           
##                                             
## Constant                   7.52***          
##                            (0.50)           
##                                             
## --------------------------------------------
## Observations                6,824           
## R2                          0.01            
## Adjusted R2                 0.01            
## ============================================
## Note:            *p<0.1; **p<0.05; ***p<0.01

The constant (\(\hat{\beta_0}\)) is 7.52. Notice that this is the average time off work for the control group (highearn=0) pre-treatment (afchnge=0). The coefficient for \(highearn\) (\(\hat{\beta_1}\)) is 3.19. This is equal to the difference in average for the treatment and control groups before the policy change (10.71-7.52). The coefficient for \(afchnge\) (\(\hat{\beta_2}\)) is equal to 1.07. This is equal to the difference in averages for the control group pre- and post-treatment (8.59 - 7.52).

The coefficient on the interaction term (\(\hat{\beta_3}\)) is the DID treatment effect (2.26). This is equal to the difference in averages for the treatment and control groups post-treatment and the difference in averages for the treatment and control group pre-treatment ((14.04 - 8.59 ) - (10.71 - 7.52)).

Why does this matter? If we only compare the difference in time off work for high- and low-earners after the policy change we would estimate a treatment effect of 14.04 - 8.59= 5.45. This is too large because we have ignored the difference in time off work for high- and low-earners that existed before the policy change. The DID model accounts changes in the outcome for the control group pre- and post- treatment, and differences that existed in the treatment and control group prior to the treatment. Ignoring any of these differences will lead to a biased estimate of the treatment effect.

I will estimate several different specifications to see if the treatment effect is robust.

# ln(duration)
reg2<-lm(log(durat)~highearn+afchnge+highearn:afchnge, data=data)

# controls
reg3<-lm(log(durat)~highearn+afchnge+highearn:afchnge+male
         +married+head+neck+upextr+trunk+lowback+lowextr+occdis
         +manuf+construc, data=data)
# ky only
reg4<-lm(log(durat)~highearn+afchnge+highearn:afchnge+male
         +married+head+neck+upextr+trunk+lowback+lowextr+occdis
         +manuf+construc, data=data_ky)
# mi only
reg5<-lm(log(durat)~highearn+afchnge+highearn:afchnge+male
         +married+head+neck+upextr+trunk+lowback+lowextr+occdis
         +manuf+construc, data=data_mi) 

# combining output
stargazer(reg1, reg2, reg3, reg4, reg5, type='text', 
          omit.stat = c('f','ser'))
## 
## ================================================================
##                                Dependent variable:              
##                  -----------------------------------------------
##                   durat                 log(durat)              
##                    (1)      (2)       (3)       (4)       (5)   
## ----------------------------------------------------------------
## highearn         3.189*** 0.193*** 0.146***  0.176***    0.128  
##                  (0.814)  (0.044)   (0.047)   (0.052)   (0.111) 
##                                                                 
## afchnge           1.066    0.023     0.024     0.011     0.095  
##                  (0.737)  (0.040)   (0.040)   (0.045)   (0.085) 
##                                                                 
## male                               -0.119*** -0.098**  -0.352***
##                                     (0.041)   (0.045)   (0.097) 
##                                                                 
## married                            0.112***  0.122***    0.089  
##                                     (0.035)   (0.039)   (0.077) 
##                                                                 
## head                               -0.605*** -0.514*** -0.779***
##                                     (0.117)   (0.129)   (0.272) 
##                                                                 
## neck                                 0.151    0.270*    -0.236  
##                                     (0.147)   (0.161)   (0.343) 
##                                                                 
## upextr                             -0.212**   -0.179*   -0.209  
##                                     (0.090)   (0.101)   (0.189) 
##                                                                 
## trunk                                0.106     0.126     0.083  
##                                     (0.097)   (0.109)   (0.203) 
##                                                                 
## lowback                             -0.122    -0.009   -0.430** 
##                                     (0.090)   (0.102)   (0.192) 
##                                                                 
## lowextr                            -0.191**   -0.120    -0.343* 
##                                     (0.091)   (0.102)   (0.193) 
##                                                                 
## occdis                              0.379**    0.273     0.445  
##                                     (0.176)   (0.211)   (0.328) 
##                                                                 
## manuf                              -0.113*** -0.161***  -0.084  
##                                     (0.036)   (0.041)   (0.079) 
##                                                                 
## construc                           0.207***   0.110**  0.438*** 
##                                     (0.046)   (0.052)   (0.102) 
##                                                                 
## highearn:afchnge  2.260*  0.223*** 0.215***  0.231***    0.143  
##                  (1.174)  (0.064)   (0.064)   (0.070)   (0.154) 
##                                                                 
## Constant         7.524*** 1.202*** 1.380***  1.246***  1.870*** 
##                  (0.502)  (0.027)   (0.095)   (0.106)   (0.203) 
##                                                                 
## ----------------------------------------------------------------
## Observations      6,824    6,824     6,824     5,349     1,475  
## R2                0.010    0.016     0.037     0.041     0.056  
## Adjusted R2       0.010    0.016     0.035     0.039     0.047  
## ================================================================
## Note:                                *p<0.1; **p<0.05; ***p<0.01

The DID treatment effect in column 1 is 2.26. On average, time off work due to injury was 2.26 weeks longer for high earners compared to low earners after the policy change. Since the dependent variable in columns 2 and 3 is in logs, the treatment effect is now measured in percents. On average, time off work due to injury was 22.3% and 21.5% longer for high earners compared to low earners after the policy change. When the sample was limited to KY, the treatment effect was about 23.1%. No statistically significant treatment effect was found in Michigan.

Plotting the fitted values from regression 1 helps visualize the DID results.

ggplot() +
  geom_line(aes(x=reg1$model$afchnge, y=reg1$fitted.values,
                group=reg1$model$highearn,
                color=as.factor(reg1$model$highearn))) +
  labs(x='After Change',
       y='Average Time off Work',
       color='High Earner') +
  theme_minimal()

Notice time off work increased for high- and low-earners but the increase was larger for the high earners. The DID treatment effect is the difference in the lines when \(afchnge\)=1 minus the difference in the lines when \(afchnge\)=0.