From Local to Global: The External Validity Challenge of Experiments
Images from the event.
©Goldman: Courtesy of NYU Photo Bureau
Flyer for the event.
Event Overview:
In recent decades, the use of experimental and quasi-experimental methods has become widespread across a range of fields in economics, such as labor, education, health, and especially development. The emphasis on experimental and quasi-experimental methods was driven by an attempt to generate internally valid results, i.e., accurate estimates of the impact of the policy of interest in the time and place the experiment was implemented. But the now global scale of experiments points to the central question of external validity: to what extent and how can we generalize the knowledge generated by experiments beyond the setting of the experiment to other contexts?
Conference Program:
8:30 - 9:00 AM: Registration, coffee, and pastries
9:00 - 9:05 AM: DRI welcome remarks by Rajeev Dehejia, New York University
9:05 - 9:10 AM: Introductory remarks by Yanoula Athanassakis, Associate Vice Provost, Academic Affairs and Special Projects; Director, Environmental Humanities Initiative, New York University
9:10 - 9:50 AM: Susan Athey, Stanford University. “Heterogeneous Treatment Effects.” View Presentation
9:50 - 10:30 AM: Cyrus Samii, New York University. “Evaluating Ex Ante Counterfactual Predictions Using Ex Post Causal Inference.” View Presentation
10:30 - 10:45 AM: Break with coffee and pastries
10:45 - 11:25 AM: Sylvain Chassang, New York University. “Designing RCTs with External Validity in Mind.” View Presentation
11:25 - 12:05 PM: Rohini Pande, Yale University. “Politics, Power and Proof: Asking the Right Questions”
12:05 - 1:00 PM: Lunch for audience and speakers
1:00 - 1:40 PM: Rajeev Dehejia, New York University. “A Thought Experiment: Seven Questions Regarding External Validity.” View Presentation
1:40 - 2:20 PM: Michael Kremer and Kevin Croke, Harvard University. “Incorporating Theory and Decision Analysis into Meta-Analysis: The Case of Deworming”. View Presentation
2:20 - 3:00 PM: Roundtable moderated by Timothy Ogden, Managing Director of the Financial Access Initiative, New York University.
Speaker Bios:
Susan Athey
Stanford University
Susan Athey is the Economics of Technology Professor at Stanford Graduate School of Business. She received her bachelor’s degree from Duke University and her Ph.D. from Stanford, and she holds an honorary doctorate from Duke University. She previously taught at the economics departments at MIT, Stanford, and Harvard. Her current research focuses on the economics of digitization, marketplace design, and the intersection of econometrics and machine learning. She has worked on several application areas, including timber auctions, internet search, online advertising, the news media, and the application of digital technology to social impact applications. As one of the first “tech economists,” she served as consulting chief economist for Microsoft Corporation for six years, and now serves on the boards of Expedia, Lending Club, Rover, Turo, and Ripple, as well as non-profit Innovations for Poverty Action. She also serves as a long-term advisor to the British Columbia Ministry of Forests, helping architect and implement their auction-based pricing system. She is the director of the Shared Prosperity and Innovation Initiative at Stanford GSB, and associate director of the Stanford Institute for Human-Centered Artificial Intelligence.
Sylvain Chassang
New York University
Sylvain Chassang is a Professor of Economics at New York University. His research focuses on microeconomic theory and game theory. Dr. Chassang received his Ph.D. from the Massachusetts Institute of Technology in 2007 and his M.A. in Mathematics and Economics in 2003 from The Ecole Normale Supérieure.
Rajeev Dehejia
New York University
Rajeev Dehejia is Professor of Economics and Public Service at NYU's Robert F. Wagner Graduate School of Public Service. His research interests include: econometric methods for program evaluation, external validity of experimental and non-experimental methods, financial incentives and fertility decisions, religion and consumption insurance, and the causes and consequences of child labor. He received his Ph.D. from Harvard University.
Michael Kremer
Harvard University
Michael Kremer is the Gates Professor of Developing Societies in the Department of Economics at Harvard University. He is a Fellow of the American Academy of Arts and Sciences, a recipient of a MacArthur Fellowship and a Presidential Faculty Fellowship, and was named a Young Global Leader by the World Economic Forum. Kremer’s recent research examines education, health, water, and agriculture in developing countries. He has been named as one of Scientific American’s 50 researchers of the year, and has won awards for his work on health economics, agricultural economics, and on Latin America. He helped develop the advance market commitment (AMC) for vaccines to stimulate private investment in vaccine research and the distribution of vaccines for diseases in the developing world. In the fall of 2010 he became the founding Scientific Director of Development Innovation Ventures (DIV) at USAID. Dr. Kremer received his Ph.D. in Economics from Harvard University. He is a member of the board of Precision Agriculture for Development.
Timothy Ogden
New York University
Timothy Ogden is Managing Director of the Financial Access Initiative, a research center focused on financial services for low-income households around the world, and an adjunct professor at NYU-Wagner. He is also chairman of GiveWell, a senior fellow of the Aspen Institute’s Economic Opportunities Program and Financial Security Program, and a partner of Sona Partners. Ogden is the editor of the faiV, a widely read email newsletter on financial inclusion, digital finance, evidence-based policy and economic development. His book, Experimental Conversations: Perspectives on the Use of Randomized Trials in Development Economics, collects interviews with 20 leading thinkers on the topic; he is currently at work on Financial Inclusion: What Everyone Needs to Know, co-authored with Jonathan Morduch.
Rohini Pande
Yale University
Rohini Pande is the Henry J. Heinz II Professor of Economics and Director of the Economic Growth Center, Yale University. Her research is largely focused on how formal and informal institutions shape power relationships and patterns of economic and political advantage in society, particularly in developing countries. She is interested the role of public policy in providing the poor and disadvantaged political and economic power, and how notions of economic justice and human rights can help justify and enable such change. In 2018, she received the Carolyn Bell Shaw Award from the American Economic Association for promoting the success of women in the economics profession. She is the co-chair of the Political Economy and Government Group at Jameel Poverty Action Lab (J-PAL), a Board member of Bureau of Research on Economic Development (BREAD), a former co-editor of The Review of Economics and Statistics, and a co-editor of American Economic Review: Insights. Dr. Pande received her Ph.D. in economics from the London School of Economics.
Cyrus Samii
New York University
Cyrus Samii is Associate Professor in the Wilf Family Department of Politics of New York University and Executive Director of the Evidence in Governance and Politics (EGAP) network. He writes and teaches on quantitative social science methodology, with an emphasis on causal inference. He is a prominent expert on the design of quantitative field research and field experiments. He also conducts applied research on governance in contexts where formal institutions are weak, the political economy of development, and social, economic, and psychological causes of violent conflict. Dr. Samii received his Ph.D. in political science from Columbia University, a master’s in international affairs from the Columbia University School of International and Public Affairs, and BA from Tufts University.