Publication Detail
Peer-Reviewed Research on Education Technology
Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning
Abstract
This paper presents the first experimental evidence on the effects of live versus internet media of instruction. Students in a large introductory microeconomics course at a major research university were randomly assigned to live lectures versus watching these same lectures in an internet setting, where all other factors (instructor, content, supplemental materials) were the same. Counter to the conclusions drawn by a 2009 U.S. Department of Education meta-analysis of non-experimental studies of internet instruction in higher education, we find modest evidence that live-only instruction dominates internet instruction. These results are particularly strong for Hispanic students, male students, and lower-achieving students.
Key Findings
- Live instruction modestly outperforms internet-only. Contrary to a 2009 U.S. Department of Education meta-analysis that favored online learning, this RCT finds live lectures produce modestly better learning outcomes when all other factors are held constant.
- First randomized experimental comparison. This is among the first studies to use a true RCT to compare live and internet instruction in higher education — a direct test that meta-analyses of observational studies could not provide.
- Heterogeneous effects for Hispanic, male, and lower-achieving students. The learning advantage of live instruction is particularly strong for these subgroups. Pushing them into online-only delivery could widen achievement gaps.
- Implications for equity in online expansion. Observational studies of online learning overstate its effectiveness because students who choose online differ from those who choose in-person. Experimental design is needed to isolate the media-of-instruction effect.
Live vs. Online Instruction Outcomes
Methods
Research Design
Randomized controlled trial (RCT) with students randomly assigned to live or online instruction sections of the same course taught by the same instructor.
Setting
Large introductory microeconomics course at a major research university. Online students watched recordings of the same lectures delivered to the live section.
Outcome Measures
Student performance on identical course examinations, with analysis of heterogeneous effects across student subgroups defined by prior academic preparation, demographics, and engagement patterns.
Policy Implications
- Observational studies overstate online learning effectiveness. A 2009 U.S. Department of Education meta-analysis of non-experimental comparisons had favored online learning. This paper’s experimental design reached the opposite conclusion when selection effects are removed.
- Universal online substitution risks widening equity gaps. The subgroups that benefit most from live instruction (Hispanic, male, and lower-achieving students) are precisely those for whom equity-driven policy attention is greatest.
- Hybrid and supplemental online models may differ. The experiment tests internet-only versus live-only; the relative performance of blended models is left as an open question for future experimentation.
- Randomized experiments should anchor online-learning evaluations. The paper lays out a template for replication in other courses, institutions, and modality comparisons.
Citation
Figlio, D., Rush, M., & Yin, L. (2013). Is it live or is it internet? Experimental estimates of the effects of online instruction on student learning. Journal of Labor Economics, 31(4), 763–784. https://doi.org/10.1086/669930