Can after school programs help close educational gaps?
After-school and summer programs seem like obvious ways to help close educational gaps and improve student learning. They are especially relevant given the current post-pandemic challenges facing our K-12 educational system, which includes the largest drops in student learning ever documented in the United States.
The potential benefits of these supplemental academic programs seem clear. First, they are purely additive to school-day work: no in-school academic compromises are needed. They also often layer in enrichment: providing extra experiences in sports, arts, STEM, and career-connected learning. And finally, they can also help working families that would otherwise need to find additional child care.
Despite the potential benefits, these programs are an often ignored component of educational strategy. In more extreme cases, they are actively discouraged. For example, a recent report sponsored by a major philanthropic organization claimed tutoring needs to happen during the school day to be effective, and anything else would result in inequitable results for students.
So what are the arguments against after-school programs? They essentially boil down to access and attendance. In other words, it can be difficult to get the targeted students to reliably attend the program due to things like transportation and competing extracurricular activities. Finally, there are also sometimes arguments around potential student fatigue and long school days.
As with many educational debates, the strengths of the argument really depend on the details of any given program. There are surely many after-school programs that do have low attendance and where transportation is a major barrier for students. However, it also seems clear that those don’t have to be major issues. Fundamentally, the question boils to “how well does any particular program address the potential pitfalls?”
With that in mind, we’ll now take a look at some key metrics that we’d use to judge an after-school program’s ability to close educational gaps. We’ll break things down into access/attendance and effectiveness, and then look at some actual program data to provide context.
Access and Attendance. The question of access relates to our ability to enroll the desired students into the program. For example, if transportation isn’t provided by the program, families are forced to find their own solutions, which can result in high-need families not enrolling. Similarly, if the program isn’t fun for the students, families may simply decline to enroll, regardless of transportation! A simple metric for access is the fraction of targeted students that end up enrolled in the program.
Even after a student enrolls, there is still the question of attendance. If a student doesn’t attend they aren’t getting the benefits of the program, even if they are officially enrolled.
Effectiveness. Finally, even if the the students are enrolled and attending, there is the question of the program’s effectiveness in boosting student learning. This metric will encompass many of the miscellaneous questions around things like student fatigue.
So with these metrics in mind, let’s take a look at some actual data!
Case Study: Northgate Elementary’s Reads and Counts Program
The Math Agency began running a pilot program inside of Northgate’s Reads and Counts Program in 2021–22. It is one of six Community Learning Centers (CLC) run by Seattle Parks and Recreation. It is the only elementary school location; the rest are located at middle schools.
Access and Attendance
The Northgate CLC has several features that help directly with access and attendance. For example, they intertwine enrichment and academic activities. Fun enrichment activities (like swimming, woodworking, and role-playing games) make the program something that kids want to attend. The CLC also runs targeted training for all staff to make sure the students experience a welcoming and safe environment. Enrollment is free and transportation (busing) is provided, thus eliminating many of the barriers to enrollment and attendance. The CLC also works closely with the school to share data and identify the students that could benefit from the program the most.
To get parents engaged, the CLC runs end-of-session “Eagle Fairs” where families can come and see student projects; this includes live performances from some of the enrichment classes. The program also sets clear attendance expectations with families (their goal is 95% attendance). Does all of this work?
First, let’s look at access. To better understand the success of the CLC at Northgate at providing access, we show a detailed breakdown of the 4th and 5th grade classes below. Based on the 2021–22 OSPI results (and assuming last year’s 3rd class is similar to this year’s 4th grade class, etc.), we estimate that about 17 fifth graders and 22 fourth graders were not meeting standards in math at the start of this year, and thus in need of additional support. The CLC has enrolled 32 of these students, slightly over the headcount target of 30 students. This means that the CLC is likely serving a little over 80% of the “at risk” students in these grades.
In terms of raising the “Target % Enrolled” number higher, the biggest barriers right now are around childcare (targeted students sometimes have to take care of siblings at home) or other extracurricular conflicts. As Northgate grows, the limited amount of seats in the program (nominally 30 for the 4th/5th grade cohort) will also likely become a limiting factor.
Next, let’s look at attendance. During the 2021–22 academic year, the average attendance in The Math Agency’s component of the program was 83%. To put this in perspective, below we’ve plotted the in-school attendance of all elementary schools in Seattle with high poverty rates, which we define as more than 60% of students on free-and-reduced price lunch. We see that in-school attendance rates range from less than 60% to over 90%, and the attendance rate for the Northgate CLC is approximately the same as the median value. In other words, attendance at the Northgate CLC after-school program is not significantly lower than in-school attendance at similar schools in the district*.
*The official OSPI definition of “regular attendance” is somewhat complicated, so we can’t do an exact apples-to-apples comparison. However, it does seem likely that our results will be qualitatively similar to the the OSPI metrics.
Effectiveness
We spent a lot of time last year examining the effectiveness of our program. With just 3 schools, we had quite a few uncontrolled variables in our programs, but our data suggests that after-school programs can be just as effective as in-school. Students in our two after-school programs averaged about 2x their historical academic growth rates. Northgate in particular showed large increases in math proficiency for the 4th and 5th grades. We also haven’t observed any evidence that “learning fatigue” is a material issue. We suspect that more fundamental factors (like program dosage, student/coach ratios, and coach training) have a much larger impact on program effectiveness.
Similarly, we’ve seen that summer programs can have a major impact on academic growth: we’ve observed growth of 0.4 grade levels over the summer for students enrolled in our programs.
The takeaway
To summarize, despite the fact that after-school (and summer) academic programs sometimes get dismissed in the context of high-impact tutoring, all of our internal evidence indicates these types of programs can play a major role in closing educational gaps for students, and should be seriously considered by education leaders interested in improving student outcomes, especially as a method for addressing COVID-related loss.