Do better schools have smaller standardized testing gaps?

Last week we saw how graduation gaps (the difference between low-income and non-low income graduation rates) increased as school quality increased. This week we’ll run a similar analysis for standardized test scores; specifically the Washington SBAC test. Before looking at the data, it is helpful to remember that 1) standardized test scores often show many of the same patterns as graduation rates and 2) there is a much greater range in SBAC pass rates than graduation rates, so we often see a much stronger “signal” when analyzing SBAC results. So, what do we see?

Overall, as school quality increases, test scores of low-income students increase far slower than scores of their high-income peers. Interested in the details? Read on!

Our analysis follows a very similar path as last week. The main difference is that we are looking at combined math and english SBAC scores (I call it SBAC L1); normalized so that 1.0 means a 100% pass rate for both tests. The 2019, 8th grade results for all schools in Washington are shown below. What does the data tell us? Similar to graduation rates, we quickly see that:

a) both low-income and non-low income SBAC tend to increase together. I’ll call schools with higher SBAC scores for both groups “higher quality” schools

b) low-income scores almost always lag non-low income scores, and the gap gets significantly larger as school quality increases.

Figure 1. Plot of 8th grade low-income SBAC scores versus non-low income SBAC scores for all Washington schools in 2019. Dot size is proportional to the total number of test takers in each schools. The 1:1 line is shown in grey.

To highlight the gap, we plot the differences of the low-income and non-low income scores below.

Figure 2. The 2019 low-income SBAC gap as a function of non-low income SBAC scores. The dotted black line is a linear fit weighted by the total number of test takers.

If we plot a simple linear fit to the data, we see that low-income scores increase at only about 45% of the rate of the non-low income scores. This means that while low-income do get benefits from being in “higher quality” schools, low-income students get less than half the benefits of non-low income students.

Finally, I won’t go into the details for every demographic group, but as with graduation rates, this seems to be a general pattern for underperforming groups.

How many low-income students are in the “high quality” schools?

At this point, it is natural to wonder about the actual distribution of low-income students. If there are very few low-income students at high quality schools, then it may not be very useful to focus on the large achievement gaps at these schools. On the other hand, if there are a significant fraction of low-income students at these schools, it could be effective to focus efforts at these schools, since there is so much room to improve scores from the low-income students.

To highlight the distribution of low-income students, I’ve re-plotted the data from Figure 2 below, but with the dot size proportional to the number of low-income students in each school.

Figure 3. The same data as Figure 2, but with dot size proportional to the number of low-income test takers in each school.

We can see from this plot that many of the high quality schools often have relatively large numbers of low-income students. If it is easier to get low-income students at a school to perform at the level of their higher income peers than to raise the performance of the entire school, then this data suggests that it may make sense to focus efforts on low-income students at high performing schools, which is a pretty counter-intuitive strategy! We’ll look closer at which schools are the best targets for interventions in future posts.