Empirical application

Encouragement, iron supplementation, and school performance

This application uses the AEJapp replication data included with sreg to estimate the effects of two encouragement treatments on school performance. It illustrates data preparation, stratified estimation, covariate adjustment, and reporting.

01 · Experimental setting

Chong, Cohen, Field, Nakasone, and Torero (2016) study an intervention encouraging iron supplementation among students in rural Peru. The two active treatments are videos featuring a soccer player or a physician; the placebo video concerns oral hygiene. Assignment was stratified by year in school. The package's analysis dataset contains 215 students in five strata.

The outcome, gradesq34, measures school performance in the final two academic quarters. The estimands are the average effects of assignment to each encouragement video, relative to placebo. They are not the causal effects of consuming iron supplements. This application also appears in Bugni, Canay, and Shaikh (2019).

02 · Load and prepare the data

Install the selected implementation using the installation guide. Recode placebo from 3 to 0, as required by sreg, while retaining the original assignment variable.

library(sreg)
data("AEJapp", package = "sreg")

# Strip imported Stata labels without changing numeric codes.
Y <- as.numeric(unclass(AEJapp$gradesq34))
D_original <- as.numeric(unclass(AEJapp$treatment))
D <- ifelse(D_original == 3, 0, D_original)
S <- as.numeric(unclass(AEJapp$class_level))
age <- as.numeric(unclass(AEJapp$age_months))

table(D, S)

use "/path/to/sreg-stata/data/sreg_aejapp.dta", clear
generate byte D = cond(treatment == 3, 0, treatment)
tabulate D class_level

from sreg import AEJapp, sreg

aej = AEJapp()
Y = aej["gradesq34"]
D = aej["treatment"].replace(3, 0)
S = aej["class_level"]
X_age = aej[["age_months"]]

print(aej.assign(D=D).groupby(["D", "class_level"]).size())

The treatment–stratum counts are:

03 · Estimate the unadjusted ATEs

Use individual-level assignment and the large-strata procedure. Without covariates, the point estimator aggregates treatment–control contrasts within strata using sample stratum shares. The standard errors use the stratified-design variance estimator, with HC1 enabled by default.

fit_unadjusted <- sreg(Y = Y, S = S, D = D)
print(fit_unadjusted)

sreg gradesq34, treatment(D) strata(class_level)
estimates store unadjusted

fit_unadjusted = sreg(Y=Y, S=S, D=D)
print(fit_unadjusted)

The physician-encouragement estimate is approximately 0.41 grade units. Its marginal 95% interval narrowly excludes zero; the soccer-player interval includes zero. These are separate asymptotic tests, without adjustment for multiple testing. The empirical results alone cannot establish the coverage properties of an inference procedure.

04 · Covariate adjusted inference

As a baseline-adjusted illustration, include age in months. The randomization strata remain class_level; adding an adjustment covariate does not change the original stratification design.

fit_age <- sreg(Y = Y, S = S, D = D,
                X = data.frame(age_months = age))
print(fit_age)

sreg gradesq34 age_months, treatment(D) strata(class_level)
estimates store age_adjusted

fit_age = sreg(Y=Y, S=S, D=D, X=X_age)
print(fit_age)

Both standard errors are smaller in this sample, and both intervals include zero. Adjustment choices should follow the analysis specification, rather than the statistical significance of a realized estimate.

X_repository <- data.frame(
  pills_taken = as.numeric(unclass(AEJapp$pills_taken)),
  age_months = age
)
fit_repository <- sreg(Y = Y, S = S, D = D, X = X_repository)
print(fit_repository)

sreg gradesq34 pills_taken age_months, treatment(D) strata(class_level)
estimates store repository_specification

fit_repository = sreg(
    Y=Y, S=S, D=D,
    X=aej[["pills_taken", "age_months"]],
)
print(fit_repository)

05 · Extract and plot results

The following commands report the unadjusted specification. Use the corresponding stored result to report the age-adjusted specification.

data.frame(
  estimate = fit_unadjusted$tau.hat,
  se = fit_unadjusted$se.rob,
  lower = fit_unadjusted$CI.left,
  upper = fit_unadjusted$CI.right
)
plot(fit_unadjusted,
     treatment_labels = c("Soccer-player encouragement",
                          "Physician encouragement"),
     title = NULL,
     x_axis_title = "Effect on grades relative to placebo")

estimates table unadjusted age_adjusted, b(%9.5f) se(%9.5f) stats(N)
estimates restore unadjusted
matrix list e(b)
matrix list e(V)
sregplot, treatmentlabels("Soccer player" "Physician")

print(fit_unadjusted["tau_hat"])
print(fit_unadjusted["se_rob"])
ax = fit_unadjusted.plot(
    treatment_labels=["Soccer-player encouragement", "Physician encouragement"],
    title="Encouragement effects on school grades",
    x_axis_title="Effect on grades relative to placebo",
)
ax.figure.savefig("peru-effects.png", dpi=150, bbox_inches="tight")

Sources and replication

The data and core examples come from the R repository, Stata repository, and Python practical guide. Numerical tables and the figure on this page were computed using R sreg 2.1.0. The age-only adjustment is an additional illustration; the two-covariate repository specification is reproduced separately above.

• Chong, A., Cohen, I., Field, E., Nakasone, E., and Torero, M. (2016). Iron Deficiency and Schooling Attainment in Peru. American Economic Journal: Applied Economics, 8(4), 222–255.
• Bugni, F. A., Canay, I. A., and Shaikh, A. M. (2019). Inference under Covariate-Adaptive Randomization with Multiple Treatments. Quantitative Economics, 10(4), 1747–1785.

See the design map for other assignment structures and the user guide for simulated examples and argument conventions.