* Load the data
clear*
*cd ""
use fake_data, clear
* Set as panel data
xtset workerid year, yearly
Panel variable: workerid (unbalanced)
Time variable: year, 1995 to 2011, but with gaps
Delta: 1 year16 - Differences-in-Differences Analysis
econ 490
stata
difference-in-differences
panel data
regression
parallel trends
event study
This notebook introduces difference-in-difference analysis. We look the assumptions required to perform this type of analysis, how to run the regressions, how to run event studies, and some common mistakes to avoid.
Prerequisites
- Run OLS regressions.
- Run panel data regressions.
Learning Outcomes
- Understand the parallel trends (PT) assumption.
- Run the according OLS regression that retrieves the causal estimand.
- Implement these regressions in the two-period case and in multiple time periods (a.k.a event studies).
- Conduct a test on the plausibility of the PT whenever there are more than 1 pre-treatment periods.
16.1 Difference-in-differences
Difference-in-differences (diff-in-diff) is a research design used to estimate the causal impact of a treatment by comparing the changes in outcomes over time between a treated group and an untreated (or control) group. By comparing changes in outcomes over time, it relies on the use of multiple (at least two) time periods. Therefore, there is a link between diff-in-diff designs and panel data. Every time we want to use a diff-in-diff design, we will always have to make sure that we have panel data.
Why are panel datasets crucial in diff-in-diff research designs? The idea is that panel data allows us to control for heterogeneity that is both unobserved and time invariant.
Consider the following example. Earnings \(y_{it}\) of worker \(i\) at time \(t\) can be split into two components:
\[ y_{it} = e_{it} + \alpha_{i} \]
where \(\alpha_i\) is a measure of worker quality and \(e_{it}\) are the part of earnings not explained by \(\alpha_i\). This says that a bad quality worker (low \(\alpha_i\)) will receive lower earnings at any time period, since \(\alpha_i\) is time invariant. Notice that worker quality is typically unobserved and is usually part of our error term, which should not be correlated with treatment. In many cases though, this invariant heterogeneity (in our case, worker quality) is the cause of endogeneity bias. In this example, it can be that workers who attend a training program also tend to be the ones that perform poorly at their job and select into this program.
However, notice that if we take time differences, we get rid of this heterogeneity. Suppose we subtract earnings at time \(1\) from earnings at time \(0\), thus obtaining:
\[ y_{i1} - y_{i0} = e_{i1} - e_{i0} \]
where our new equation no longer depends on \(\alpha_i\)! However, see how we are now measuring \(y_{i1} - y_{i0}\) instead of \(y_{it}\)? Our model now has changes rather than levels. This is going to be the trick used implicitly throughout this module.
For this module, we will keep working on our fake data set. Recall that this data is simulating information of workers in the years 1982-2012 in a fake country where a training program was introduced in 2003 to boost their earnings.
Let’s start by loading our data and letting Stata know that it is panel data with panel variable workerid and time variable year. We’ve seen how to do this in Module 15.
16.2 Parallel Trends Assumption
When using a diff-in-diff design, we first need to make sure our data has a binary treatment variable which takes the value 1 when our unit of observation is treated and 0 otherwise. In the example above, let’s denote such a binary treatment variable as \(D_i\). It takes value 1 if a worker \(i\) is enrolled in the training program at some point in time.
In our fake data set, the binary treatment variable already exists and is called treated. Let’s check that it takes values 0 or 1.
describe, full
summarize treated, detail
Contains data from fake_data.dta
Observations: 138,138
Variables: 11 16 Jul 2023 17:25
-------------------------------------------------------------------------------
Variable Storage Display Value
name type format label Variable label
-------------------------------------------------------------------------------
workerid long %12.0g Worker Identifier
year int %ty Calendar Year
sex str1 %9s Sex
age byte %9.0g Age (years)
start_year int %9.0g Initial year worker is observed
region byte %9.0g group(prov)
treated byte %8.0g Treatment Dummy
earnings float %9.0g Earnings
sample_weight float %9.0g
quarter_birth float %9.0g Quarter of birth
schooling float %9.0g Years of schooling
-------------------------------------------------------------------------------
Sorted by: workerid year
Treatment Dummy
-------------------------------------------------------------
Percentiles Smallest
1% 0 0
5% 0 0
10% 0 0 Obs 138,138
25% 0 0 Sum of wgt. 138,138
50% 0 Mean .2943144
Largest Std. dev. .4557356
75% 1 1
90% 1 1 Variance .2076949
95% 1 1 Skewness .9026566
99% 1 1 Kurtosis 1.814789The aim of diff-in-diff analysis is to estimate the causal impact of a treatment by comparing the changes in outcomes over time between a treated group and an untreated group.
A crucial assumption needed to claim causal impact is that, in the absence of treatment, the treatment and control groups would follow similar trends over time. This assumption is called parallel trends assumption. Whenever we adopt a diff-in-diff design in our research, the first thing we need to check is that this assumption is satisfied.
How do we do that?
A common approach to check for parallel trends is to plot the mean outcome for both the treated and untreated group over time.
Do you recall how to make these plots from Module 9? We start by generating the average log-earnings for each group in each year.
* Generate log-earnings
generate logearn = log(earnings)
* Take the average by group and year
bysort year treated: egen meanearn = mean(logearn)Next, we plot the trend of average earnings by each group. It is common practice to add a vertical line in the period just before the treatment is assigned. In our case, that would be year 2002. The idea is that the treated workers receive the treatment between years 2002 and 2003.
* Make graph
twoway (line meanearn year if treated == 1, lcolor(gs12) lpattern(solid)) || ///
(line meanearn year if treated == 0, lcolor(gs6) lpattern(dash)), ///
graphregion(color(white)) ///
legend(label(1 "Treated") label(2 "Control")) ///
ytitle("Average earnings") xtitle("Year") ///
xline(2002, lpattern(dash) lcolor(black))
graph export graph1.jpg, as(jpg) replacefile graph1.jpg written in JPEG format
Remember that we care about the two variables having similar trends before the year of the treatment. By looking at the graph, it seems that the average earnings of the two groups had similar trends up until year 2002, just before the treatment. This makes us confident that the parallel trends assumption is satisfied.
This test for parallel trends assumption is very rudimentary, but perfectly fine for the early stage of our research project. In the next sections, we will see how to estimate the diff-in-diff design, and there we will see a more formal test for the parallel trends assumption.
16.3 Difference-in-Differences and Regression
Whenever we talk about diff-in-diff, we refer to a research design that relies on some version of the parallel trends assumption. To connect this design to regressions, we need to first build a model. To begin, we will assume a case where no control variables are involved.
For simplicity, suppose there are only two periods: a period \(t=0\) when no one is treated, and a period \(t=1\) when some workers receive the treatment.
We would then rely on a linear model of the form:
\[ y_{it} = \beta D_i \mathbf{1}\{t=1\} + \lambda_t + \alpha_i + e_{it} \tag{1} \]
where \(y_{it}\) is earnings while \(\lambda_t\) and \(\alpha_i\) are year and worker fixed-effects.
The key element in this linear model is the interaction between \(D_i\) and \(\mathbf{1}\{t=1\}\).
Recall that \(D_i\) is a dummy variable taking value 1 if worker \(i\) receives the treatment at any point in time, and \(\mathbf{1}\{t=1\}\) is an indicator function taking value 1 when \(t=1\).
Therefore, the interaction term \(D_i \mathbf{1}\{t=1\}\) will take value 1 for treated workers only when the year is \(t=1\), or when the treated workers are treated.
The parameter \(\beta\) provides the average treatment effect (on the treated) at period \(t=1\) (i.e. we get the effect for those with \(D_i=1\) at \(t=1\)). It is the average impact of the treatment on those workers who actually received the treatment. \(\beta\) states by how much the average earnings of treated individuals would have changed if they had not received the treatment.
Let’s see how we can estimate this linear diff-in-diff model!
Recall that we have information of workers in the years 1982-2012 and the training program (the treatment) was introduced in 2003. We’ll keep one year prior and one year after the program, to keep things consistent with the previous section. Specifically, we can think of year 2002 as \(t=0\) and year 2003 as \(t=1\).
keep if year==2002 | year==2003(120,535 observations deleted)Notice that the diff-in-diff linear model in Equation (1) can be seen as a specific case of a linear model with many fixed-effects. We can use the command reghdfe and the option absorb() to run this type of regression, which we saw in Module 13. We can also use the command areg alongside the option absorb() which has the same syntax. In either case, don’t forget to list the fixed-effects in absorb() to avoid seeing them in the regression output!
Recall that we can create fixed-effects with the i. operator and interactions with the # operator.
areg logearn treated#2003.year i.year, absorb(workerid)
Linear regression, absorbing indicators Number of obs = 17,603
Absorbed variable: workerid No. of categories = 15,576
F(2, 2025) = 7.82
Prob > F = 0.0004
R-squared = 0.9109
Adj R-squared = 0.2253
Root MSE = 0.9911
------------------------------------------------------------------------------
logearn | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
treated#year |
1 2003 | .1798409 .0709288 2.54 0.011 .0407399 .3189419
|
year |
2003 | .0476606 .0362003 1.32 0.188 -.0233332 .1186543
|
_cons | 10.68747 .0167741 637.14 0.000 10.65457 10.72037
------------------------------------------------------------------------------
F test of absorbed indicators: F(15575, 2025) = 1.282 Prob > F = 0.000This says that, on average, workers who entered the program received 18 percentage points more earnings relative to a counterfactual scenario where they never entered the program (which in this case is captured by the control units). How did we get this interpretation? Recall that OLS estimates are interpreted as a 1 unit increase in the independent variable: a 1 unit increase of \(D_i \mathbf{1}\{t=1\}\) corresponds to those who started receiving treatment at \(t=1\). Furthermore, the dependent variable is in log scale, so a 0.18 increase corresponds to a 18 percentage point increase in earnings.
16.3.1 Adding Covariates
The first thing to notice is that our regression specification in Equation (1) involves worker fixed-effects \(\alpha_i\). This means that every worker characteristic that is fixed over time (for example, sex at birth) will be absorbed by the fixed-effects \(\alpha_i\). Therefore, if we added characteristics such as sex and race as covariates, those would be omitted from the regression due to perfect collinearity.
This means that we can add covariates to the extent that they are time varying by nature (e.g. tenure, experience), or are trends based on fixed characteristics (e.g. time dummies interacted with sex). We refer to the latter as covariate-specific trends.
Algebraically, we obtain a specification that is very similar to Equation (1): \[ y_{it} = \beta D_i \mathbf{1}\{t=1\} + \gamma X_{it} + \lambda_t + \alpha_i + e_{it} \tag{2} \]
where \(X_{it}\) is a time-varying characteristic of worker \(i\) and time \(t\).
16.4 Multiple Time Periods
In keeping only the years 2002 and 2003, we have excluded substantial information from our analysis. We may want to keep our data set at its original state, with all its years.
A very natural approach to extending this to multiple time periods is to attempt to get the average effect across all post-treatment time periods. For example, it may be that the effects of the training program decay over time, but we are interested in the average effect. We may think of maintaining the parallel trends assumption in a model like this:
\[ y_{it} = \beta D_i \mathbf{1}\{t\geq 1\} + \lambda_t + \alpha_i + e_{it} \tag{3} \]
where the \(\beta\) corresponds now to all time periods after the year in which treatment was applied: \(t\geq 1\). Some people rename \(D_i \mathbf{1}\{t\geq 1\}\) to \(D_{it}\), where \(D_{it}\) is simply a variable that takes 0 before any treatment and 1 for those who are being treated at that particular time \(t\). This is known as the Two-Way Fixed-Effects (TWFE) Model . It receives this name because we are including unit fixed-effects, time fixed-effects, and our treatment status.
Let’s load our fake data set again and estimate a TWFE model step-by-step.
* Load data
clear*
use fake_data, clear
* Generate log-earnings
generate logearn = log(earnings)Remember that now we need to create \(\mathbf{1}\{t\geq 1\}\), a dummy equal to 1 for all years following the year in which the treatment was administered. In our example, we need to create a dummy variable taking value 1 for all years greater than or equal to 2003.
generate post2003 = year>=2003We can again use areg or reghdfe to estimate Equation (3), but remember to use the new post2003 dummy variable.
areg logearn 1.treated#1.post2003 i.year, absorb(workerid)
Linear regression, absorbing indicators Number of obs = 138,138
Absorbed variable: workerid No. of categories = 39,999
F(17, 98122) = 31.52
Prob > F = 0.0000
R-squared = 0.4520
Adj R-squared = 0.2285
Root MSE = 1.0019
------------------------------------------------------------------------------
logearn | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
treated#|
post2003 |
1 1 | .0686984 .0151408 4.54 0.000 .0390226 .0983742
|
year |
1996 | -.0162053 .0158475 -1.02 0.307 -.0472663 .0148557
1997 | .0269527 .0159317 1.69 0.091 -.0042732 .0581786
1998 | .0642001 .0160094 4.01 0.000 .0328218 .0955783
1999 | .086081 .016202 5.31 0.000 .0543254 .1178367
2000 | .0770715 .0162897 4.73 0.000 .0451438 .1089992
2001 | .1115958 .0164495 6.78 0.000 .0793551 .1438366
2002 | .1444224 .0166584 8.67 0.000 .1117722 .1770727
2003 | .1765399 .0175397 10.07 0.000 .1421624 .2109174
2004 | .1586781 .0178837 8.87 0.000 .1236262 .19373
2005 | .1238951 .0183022 6.77 0.000 .0880231 .1597671
2006 | .1430492 .0185189 7.72 0.000 .1067523 .179346
2007 | .1491766 .0190106 7.85 0.000 .1119161 .1864372
2008 | .1722747 .0194074 8.88 0.000 .1342365 .2103129
2009 | .1993529 .0201299 9.90 0.000 .1598986 .2388072
2010 | .2325934 .020442 11.38 0.000 .1925274 .2726594
2011 | .19393 .0209931 9.24 0.000 .1527839 .2350761
|
_cons | 10.56453 .0113703 929.13 0.000 10.54224 10.58682
------------------------------------------------------------------------------
F test of absorbed indicators: F(39998, 98122) = 1.827 Prob > F = 0.000The results say that a 1 unit increase in \(D_i \mathbf{1}\{t\geq 1\}\) corresponds to a 0.07 increase in log-earnings on average. That 1 unit increase only occurs for those who start receiving treatment in 2003. Given that the outcome is in a log scale, we interpret these results in percentage points. Therefore, the coefficient of interest says that those who started treatment in 2003 received, on average, a 7 percentage point increase in earnings.
In this fake data set, everyone either starts treatment at year 2003 or does not enter the program at all. However, when there is variation in the timing of the treatment (i.e. people entering the training program earlier than others), a regression using this model may fail to capture the true parameter of interest. For a reference, see this paper.
16.5 Event Studies
The natural extension of the previous section, which is the standard approach today, is to estimate different treatment effects depending on the time period.
It may be possible that the effect of the treatment fades over time: it was large right after the training program was received, but then decreased over time.
To capture the evolution of treatment effects over time, we may want to compute treatment effects at different lags after the program was received: 1 year after, 2 years after, etc.
Similarly, we may want to compute “treatment effects” at different years prior the program.
This is a very powerful tool because it allows us to more formally test whether the parallel trends assumption holds or not: if there are treatment effects prior to receiving the treatment, then the treatment and control groups were likely not having the same trend before receiving the treatment. This is often known as a pre-trends test.
A linear model where we test for different treatment effects in different years is usually called an event study.
Essentially, we extend the diff-in-diff linear model to the following equation:
\[ y_{it} = \sum_{k=-T,k\neq-1}^T \beta_k \mathbf{1}\{K_{it} = k\} + \lambda_t + \alpha_i + e_{it} \tag{4} \]
where \(K_{it}\) are event time dummies (i.e. whether person \(i\) is observed at event time \(k\) in time \(t\)). These are essentially dummies for each year until and each year since the event, or “time to” and “time from” dummies. For example, there will be a dummy indicating that a treated individual is one year away from being treated, two years away from being treated, etc. Notice that, for workers who never enter treatment, it is as if the event time is \(\infty\): they are an infinite amount of years away from receiving the treatment. Due to multicollinearity, we need to omit one category of event time dummies \(k\). The typical choice is \(k=-1\) (one year prior to treatment), which will serve as our reference group. This means that we are comparing changes relative to event time -1.
How do we estimate Equation (4) in practice?
We begin by constructing a variable that identifies the time relative to the event. For instance, if a person enters the training program in 2003, the observation corresponding to 2002 is time -1 relative to the event, the observation corresponding to 2003 is time 0 relative to the event, and so on. We call this variable event_time and we compute it as the difference between the current year and the year in which the treatment was received (stored in variable time_entering_treatment).
In this fake data set, everyone enters the program in 2003, so it is very easy to construct the event time. If this is not the case, we need to make sure that we have a variable which states the year in which each person receives their treatment.
* Load data
clear*
use fake_data, clear
* Generate log-earnings
generate logearn = log(earnings)
* Generate a variable for year in which treatment was received
capture drop time_entering_treatment
generate time_entering_treatment = 2003 if treated==1
replace time_entering_treatment = . if treated==0
* Generate a variable for time relative to the event
capture drop event_time
generate event_time = year - time_entering_treatment(97,482 missing values generated)
(0 real changes made)
(97,482 missing values generated)To make sure we have created event_time properly, let’s see which values it takes.
tabulate event_time , missing
event_time | Freq. Percent Cum.
------------+-----------------------------------
-8 | 3,154 2.28 2.28
-7 | 3,202 2.32 4.60
-6 | 3,134 2.27 6.87
-5 | 3,065 2.22 9.09
-4 | 2,969 2.15 11.24
-3 | 2,850 2.06 13.30
-2 | 2,737 1.98 15.28
-1 | 2,598 1.88 17.16
0 | 2,420 1.75 18.92
1 | 2,294 1.66 20.58
2 | 2,105 1.52 22.10
3 | 2,090 1.51 23.61
4 | 1,799 1.30 24.91
5 | 1,806 1.31 26.22
6 | 1,552 1.12 27.35
7 | 1,479 1.07 28.42
8 | 1,402 1.01 29.43
. | 97,482 70.57 100.00
------------+-----------------------------------
Total | 138,138 100.00Notice that all untreated workers have a missing value for the variable event_time. We want to include untreated workers in the reference category \(k=-1\). Recall that we are still trying to understand the effect of being treated compared to the reference group, those that are untreated. Therefore, we code untreated units as if they always belonged to event time -1.
replace event_time = -1 if treated==0(97,482 real changes made)We then decide which window of time around the treatment we want to focus on (the \(T\)’s in Equation (4)). For instance, we may want to focus on 2 years prior to the treatment and 2 years after the treatment, and estimate those treatment effects. Our choice should depend on the amount of information we have in each year. In this case, notice that the number of workers 8 years after treatment is substantially lower than the number of workers 8 years before treatment is started.
We could drop all observations before \(k=-2\) and after \(k=2\). This would once again reduce the amount of information we have in our dataset.
An alternative approach, called binning the window around treatment, is usually preferred. It works by pretending that treated workers who are observed before event_time -2 were actually observed in event_time -2 and treated workers who are observed after event_time 2 were actually observed in event_time 2.
replace event_time = -2 if event_time<-2 & treated==1
replace event_time = 2 if event_time>2 & treated==1(18,374 real changes made)
(10,128 real changes made)Notice how these steps have modified the values of variable event_time:
tabulate event_time
event_time | Freq. Percent Cum.
------------+-----------------------------------
-2 | 21,111 15.28 15.28
-1 | 100,080 72.45 87.73
0 | 2,420 1.75 89.48
1 | 2,294 1.66 91.14
2 | 12,233 8.86 100.00
------------+-----------------------------------
Total | 138,138 100.00The next step is to generate a dummy variable for each value of event_time.
tabulate event_time, gen(event_time_dummy)
event_time | Freq. Percent Cum.
------------+-----------------------------------
-2 | 21,111 15.28 15.28
-1 | 100,080 72.45 87.73
0 | 2,420 1.75 89.48
1 | 2,294 1.66 91.14
2 | 12,233 8.86 100.00
------------+-----------------------------------
Total | 138,138 100.00Notice that event_time_dummy2 is the one that corresponds to event_time -1.
Once again, Equation (4) is nothing but a linear model with many fixed-effects. We can again use either command areg or reghdfe.
This time, we must include dummy variables for the different values of event_time, with the exception of the dummy variable for the baseline event time \(k=-1\): event_time_dummy2.
areg logearn event_time_dummy1 event_time_dummy3 event_time_dummy4 event_time_dummy5 i.year , absorb(workerid) // do you recall how we included worker and year fixed-effects?
Linear regression, absorbing indicators Number of obs = 138,138
Absorbed variable: workerid No. of categories = 39,999
F(20, 98119) = 28.06
Prob > F = 0.0000
R-squared = 0.4521
Adj R-squared = 0.2287
Root MSE = 1.0018
------------------------------------------------------------------------------
logearn | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
event_time~1 | .037974 .0285633 1.33 0.184 -.0180098 .0939578
event_time~3 | .2015815 .0383469 5.26 0.000 .1264221 .2767409
event_time~4 | .1522862 .0390394 3.90 0.000 .0757695 .2288029
event_time~5 | .0664033 .0299573 2.22 0.027 .0076873 .1251193
|
year |
1996 | -.0163025 .0158458 -1.03 0.304 -.0473601 .014755
1997 | .0269746 .0159299 1.69 0.090 -.0042479 .0581971
1998 | .0642057 .0160076 4.01 0.000 .0328309 .0955805
1999 | .0860578 .0162002 5.31 0.000 .0543056 .11781
2000 | .0772236 .016288 4.74 0.000 .0452993 .1091479
2001 | .1115752 .0164476 6.78 0.000 .079338 .1438123
2002 | .1549811 .0184733 8.39 0.000 .1187737 .1911886
2003 | .1499054 .0189292 7.92 0.000 .1128043 .1870065
2004 | .1455681 .0194297 7.49 0.000 .1074862 .18365
2005 | .1353666 .0184554 7.33 0.000 .0991943 .1715388
2006 | .1546189 .0186729 8.28 0.000 .1180203 .1912175
2007 | .1602752 .0191496 8.37 0.000 .1227421 .1978082
2008 | .1840454 .0195587 9.41 0.000 .1457106 .2223801
2009 | .2106673 .0202651 10.40 0.000 .170948 .2503866
2010 | .2438228 .0205728 11.85 0.000 .2035004 .2841452
2011 | .2052274 .0211236 9.72 0.000 .1638253 .2466295
|
_cons | 10.55347 .0141317 746.80 0.000 10.52577 10.58117
------------------------------------------------------------------------------
F test of absorbed indicators: F(39998, 98119) = 1.591 Prob > F = 0.000Again, the interpretation is the same as before, only now we have dynamic effects. The coefficient on the event_time1 dummy says that 2 years prior to entering treatment, treated units experienced a 0.4 percentage point increase in earnings relative to control units.
Should we worry that we are finding a difference between treated and control units prior to the policy? Notice that the effect of the policy at event time -2 (event_time_dummy1, when there was no training program) is not statistically different than zero.
This confirms that our parallel trends assumption is supported by the data. In other words, there are no observable differences in trends prior to the enactment of the training program. Checking the p-value of those coefficients prior to the treatment is called the pre-trend test and does not require any fancy work. A mere look at the regression results suffices!
Furthermore, we can observe how the policy effect evolves over time. At the year of entering the training program, earnings are boosted by 20 percentage points. The next year the effect decreases to 15 percentage points, and 2+ years after the policy, the effect significantly decreases towards 6 percentage points and is less statistically significant.
16.5.1 Event Study Graph
The table output is a correct way to convey the results, but it’s efficacy is limited, especially when we want to use a large time window. In those cases, a graph does a better job of representing all coefficients of interest.
We can easily do that using the command coefplot, which we covered in Module 9. We keep all coefficients of interest by including all event_time dummies as inputs in keep(), and we rename them one-by-one in rename() to increase clarity of the graph.
coefplot, keep(event_time_*) vertical graphregion(color(white)) yline(0) ///
rename(event_time_dummy1="k=-2" event_time_dummy3="k=0" event_time_dummy4="k=+1" event_time_dummy5="k=+2")
graph export graph2.jpg, as(jpg) replacefile graph2.jpg written in JPEG format
In the graph, it is easy to see that the parallel trends assumption is satisfied: the difference between the treatment and the control group before the treatment is administered (the coefficient for \(k=-2\)) is not statistically different than zero.
16.6 Common Mistakes
The most common mistake when dealing with a diff-in-diff research design is to add covariates that are already captured by the fixed-effects.
Let’s see what happens if we try to estimate Equation (2) where \(X\) is gender at birth.
* Load the data
clear*
use fake_data, clear
* Set as panel data
xtset workerid year, yearly
* Generate log-earnings
generate logearn = log(earnings)
* Keep only two years
keep if year==2002 | year==2003
* Estimate incorrect specification
capture noisily areg logearn treated#2003.year i.year sex, absorb(workerid)
Panel variable: workerid (unbalanced)
Time variable: year, 1995 to 2011, but with gaps
Delta: 1 year
(120,535 observations deleted)
no observationsWe cannot estimate the specification above because sex does not change over time for the same individual. Remember: in diff-in-diff regressions, we can only add covariates that are time varying by nature (e.g. tenure, experience) or are trends based on fixed characteristics (e.g. time dummies interacted with sex).
Another common mistake when dealing with event studies is to forget to re-assign untreated workers to the reference group \(k=-1\). Let’s see what happens if we try to estimate Equation (4) without this adjustment.
* Load data
clear*
use fake_data, clear
* Generate log-earnings
generate logearn = log(earnings)
* Generate a variable for year in which treatment was received
capture drop time_entering_treatment
generate time_entering_treatment = 2003 if treated==1
replace time_entering_treatment = . if treated==0
* Generate a variable for time relative to the event
capture drop event_time
generate event_time = year - time_entering_treatment
* Binning
replace event_time = -2 if event_time<-2 & treated==1
replace event_time = 2 if event_time>2 & treated==1
* Create event_time dummies
tabulate event_time, gen(event_time_dummy)
* Run regression
areg logearn event_time_dummy1 event_time_dummy3 event_time_dummy4 event_time_dummy5 i.year , absorb(workerid)(97,482 missing values generated)
(0 real changes made)
(97,482 missing values generated)
(18,374 real changes made)
(10,128 real changes made)
event_time | Freq. Percent Cum.
------------+-----------------------------------
-2 | 21,111 51.93 51.93
-1 | 2,598 6.39 58.32
0 | 2,420 5.95 64.27
1 | 2,294 5.64 69.91
2 | 12,233 30.09 100.00
------------+-----------------------------------
Total | 40,656 100.00
note: 2002.year omitted because of collinearity.
note: 2003.year omitted because of collinearity.
note: 2004.year omitted because of collinearity.
note: 2011.year omitted because of collinearity.
Linear regression, absorbing indicators Number of obs = 40,656
Absorbed variable: workerid No. of categories = 12,330
F(16, 28310) = 15.86
Prob > F = 0.0000
R-squared = 0.4340
Adj R-squared = 0.1871
Root MSE = 0.9989
------------------------------------------------------------------------------
logearn | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
event_time~1 | -.1243259 .030969 -4.01 0.000 -.1850266 -.0636252
event_time~3 | .1965016 .0324545 6.05 0.000 .1328893 .2601138
event_time~4 | .1428233 .0329751 4.33 0.000 .0781904 .2074562
event_time~5 | .1316765 .0385399 3.42 0.001 .0561364 .2072166
|
year |
1996 | -.018105 .0286996 -0.63 0.528 -.0743577 .0381476
1997 | .0204409 .0290207 0.70 0.481 -.036441 .0773228
1998 | .0784235 .0293629 2.67 0.008 .0208709 .1359762
1999 | .0937216 .0297821 3.15 0.002 .0353474 .1520959
2000 | .089555 .0300803 2.98 0.003 .0305961 .148514
2001 | .134164 .0305146 4.40 0.000 .074354 .1939741
2002 | 0 (omitted)
2003 | 0 (omitted)
2004 | 0 (omitted)
2005 | -.0614212 .0396794 -1.55 0.122 -.1391948 .0163523
2006 | -.0812372 .0396164 -2.05 0.040 -.1588872 -.0035872
2007 | -.0576709 .0407776 -1.41 0.157 -.137597 .0222552
2008 | -.0258348 .0406242 -0.64 0.525 -.1054602 .0537907
2009 | -.0106226 .0418626 -0.25 0.800 -.0926753 .0714302
2010 | -.0215037 .0421509 -0.51 0.610 -.1041215 .0611142
2011 | 0 (omitted)
|
_cons | 10.10904 .0225122 449.05 0.000 10.06491 10.15316
------------------------------------------------------------------------------
F test of absorbed indicators: F(12329, 28310) = 1.736 Prob > F = 0.000There are no error messages from Stata, but do you notice anything different compared to our results in Section 16.5?
The number of observations has decreased dramatically: instead of 138,138 workers as in Section 16.5, we only have around 40,000 workers. We are estimating our linear model only on the treated workers. This is a conceptual mistake: we cannot uncover the effect of the treatment if we do not compare the earnings of treated workers with the earnings of untreated workers.
16.7 Wrap Up
In this module, we’ve seen how the difference-in-differences design relies on two components:
- Panel data, in which units are observed over time, and
- Time and unit fixed-effects.
These two components make regressions mathematically equivalent to taking time-differences that eliminate any time-invariant components of the error term creating endogeneity. Furthermore, when we have access to more than 2 time periods, we are able to construct dynamic treatment effects (run an event study) and test whether the parallel trends condition holds.
16.8 Wrap-up Table
| Command | Function |
|---|---|
areg depvar indepvar, absorb(fixed-effects)) |
It runs a linear regression with fixed-effects, while suppressing the coefficients on the fixed-effects. |