Can we *repeatably* reduce education gaps?

Results

Comparison of program metrics between our three pilot schools. The # Family Emails Sent counts how many personalized updates were sent to each family, while the Family Email Open % is the percentage of those emails that were opened. Competitions were typically 1-week long and points were based on how much each student practiced. Standard errors for the growth metrics are shown in parenthesis.

Projections

  1. Business as usual: this is what our student cohort’s trajectory would look like if they continue progressing at their historic growth rate of 0.7 grade levels per year. By the time they start middle school, they would be on average almost two grade levels behind in math.
  2. The Math Agency (academic year): this trajectory assumes that students continue forward at their current growth rate of 1.5 grade levels per academic year. In this case, they are almost on grade level by the time they start sixth grade.
  3. The Math Agency (academic year + summer): this trajectory assumes that students keep their current growth rate and enroll in a summer program similar to the one we piloted this year. This gives a total growth rate of 1.5 grade levels/year + 0.4 grade levels/year = 1.9 grade levels/year. In this scenario, they are materially above grade level, on average, by the time they start middle school.
Projections of student skill level as a function of grade for the students in our program at Northgate Elementary.

Detailed Discussion (including caveats)

  • We are making relatively short-term growth measurements (10 weeks) on a relatively small sample (~70 students). This means there is fair amount of noise in the growth estimates, as shown by our standard errors. Our confidence in our results will increase over time and as we work with more students.
  • For the sake of simplicity, our projections show student averages. This obscures much of the variability of student outcomes. It is possible to get the “average” student above grade level but still have some students below grade level. Of course, it is still a big win to improve the average!
  • As always, correlation does not prove causation. It is possible that the increases in growth we are seeing at our schools are due to something unrelated to our program, such as curriculum changes, other interventions, etc. This doesn’t seem very likely at this point, but is always a possibility we need to keep in mind.

Next Steps

  • Demonstrating that we can maintain our growth rates for an entire year, and matching those rates to growth measured via standardized tests.
  • Working with more students to increase our sample size, and thus the reliability of our measurements.
  • Seeing if we can improve growth rates by transferring practices (such as high-intensity parent engagement and student competitions) across schools.

--

--

--

PhD in Applied Physics from Stanford. Data scientist and entrepreneur. Working on eliminating education gaps in elementary school math.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Quarantine and the hidden ‘labour’ of care in higher education

How leaving academia changed my life

Campus Controversy: Protecting our Freedom in schools

Selection Reflection

READ/DOWNLOAD=@ PPI Mechanical Engineering Referen

Digital Was the Spark, but Educational Disruption Was the Fire — Dan Rosensweig, CEO Chegg

Education innovation in China

The best Learning Management Systems (LMS)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mike Preiner

Mike Preiner

PhD in Applied Physics from Stanford. Data scientist and entrepreneur. Working on eliminating education gaps in elementary school math.

More from Medium

Diversity, Equity, Leadership - Diversity Deb for the Corporate Playbook — Deborah Augustine Elam

The weaponization of Economics

Why Ransomware is Such a Problem?

What is CoinsPaid and why is it the top 1 at the moment?