Seattle Public Schools: A district case study

Over the last few months we’ve spent a lot of time looking at both statewide and school-level data. This week we’re going to do a case study of one of the biggest districts in Washington: Seattle Public Schools #1 (SPS). I’m not sure why it is called #1, since there isn’t a #2 as far as I can tell :) There are some interesting things that we’ll discover looking at this intermediate level of aggregation that we haven’t really seen before.

A natural starting place is to compare SPS to the rest of the districts in the state. We’ll do that via our standard plot of achievement vs. low-income status; shown below. We see that SPS is fairly standard district on average, performing right about where we’d expect given the income level of its families.

District averages: percent of 5th grade students passing the SBA test as function of low-income fraction for all school districts in Washington in 2018–19. Dot size is proportional to the number of 5th grade students enrolled, and SPS is highlighted in orange.

Of course, this district has a lot of schools (76 that serve 5th graders), and there is a lot of variety. I show data for all of the SPS 5th grade schools below. We see that SPS schools span the full spectrum of family income and school performance.

School averages: percent of 5th grade students passing the SBA as function of low-income fraction for all schools in Washington. Dot size is proportional to the number of 5th grade students enrolled, and SPS schools are highlighted in orange.

Of course, it is interesting to see if SPS as a whole is changing with time. The results are shown below, and seem to how an encouraging, consistent in increase in test results.

Percent of 5th grade students passing the SBA in Seattle Public Schools over time.

At this point, however, we need to remember that test scores are closely related to family-income! It turns out that over the last 5 years the fraction of low-income students in SPS has steadily decreased.

The fraction of low-income students in Seattle Public Schools has steadily decreased over the last 5 years.

Is the decrease in low-income students enough to explain the increase in test scores? It is, as can be seen in the plot below. SPS very neatly traces out part of the income-achievement trend line. Unfortunately, it looks like SPS’s increased performance over the last 5 years can be entirely explained by simple demographic changes.

Percent of 5th grade students passing the SBA as function of low-income fraction for SPS over a 5-year period. The blue dots show data for all Washington school districts in 2018–19, and the dashed line is a polynomial fit to the 2018–19 data.

Finally, let’s take a look at the achievement gaps in the 5th grade test scores. As a reminder, the “Effect in # of students” is the number of “missing students” who would have passed the test if they tested similarly to their peers. As we are used to seeing, the biggest effect by far is that from low-income students…614 more low-income 5th graders would pass the SBA if their scores were similar to their higher income peers.

Group effects for Seattle Public school sin 2018–19. The effects are calculated from our “SBA L1” metric, where 1.0 corresponds to a 100% chance of passing both the Math and English portions of the test.

What does it all mean?

Not surprisingly, Seattle Public Schools seems to be pretty representative of Washington as a whole. I think there are two interesting things that stand out from this district-level analysis:

1. The improvements in test scores in SPS over the last 5 years are completely in line with increases in family incomes. When looking at individual schools, the data is often too noisy to see such a clear relationship. Unfortunately, this means that SPS hasn’t really made academic improvements independent of family income recently.
2. As with Washington state as a whole, the largest achievement gaps are those for low-income students. It seems a bit odd to me that income-based gaps don’t seem to come up much in the communications that I tend to get from SPS.