This page provides links to resources and tools that you can use to implement methods for generalizability in conducting impact evaluations. These include R packages and code, Stata code, and The Generalizer—a free online tool for selecting samples and assessing their generalizability. All of these resources were designed to support research in education, but the code can be adapted for other applications.
Online tools
- The Generalizer, a free online application for selecting schools to participate in education studies. The Generalizer stratifies schools using k-means clustering, rank orders schools within each cluster based on their distance from the cluster mean and constructs an index that measures the generalizability of the sample (Tipton & Miller, 2002; Tipton, 2014a, 2014b).
R code
- generalizeR, an R package to design a sample recruitment plan and assess its generalizability to broader populations. (Ruel et al., 2022).
- sampling, a R package for selecting samples using the cube method (Tillé & Matai, 2021). This package can be used to select two-stage samples of districts and schools.
- sitepickR, a new R package that combines the functionality of ‘sampling’ for selecting a sample with the functionality of MatchIt for selecting matched replacements. This package can be used to select cube samples of districts and schools, and to pre-select lists of possible replacement districts.
SAS code
- Cluster_k, a new SAS macro for dividing school districts and schools into clusters for various purposes, including creating sampling strata.
- Select_sites, a new SAS macro for selecting random or balanced samples of districts and schools, and for selecting replacement districts and schools.
Disclaimer: We have not independently tested and verified the accuracy of the software that we cite but was produced by others. And others have not independently tested and verified the accuracy of the code that we developed and offer through this website. We encourage you to make your own assessment of the code offered and to notify us if you find errors.
Have questions about available software? Or knowledge about relevant software that we should add to this webpage? Please contact Rob Olsen (robolsen@gwu.edu) or Elizabeth Stuart (estuart@jhu.edu).