Posted Date: 11/01/2022
Kansas student performance in reading and math declined in state and national results released last month, continuing a long-term trend.
The results will impact debate over educational policy from the State Legislature and Congress to local school boards and site councils.
President Joe Biden’s Secretary of Education Miguel Cardona said in a statement that newly released results “are appalling, unacceptable, and a reminder of the impact that this pandemic has had on our learners. The data also represent a call to action for the important work we must do now for our students—especially those who have suffered the most during the pandemic.” He said the administration will be releasing new resources to address math and literacy learning.
Kansas Commissioner of Education Randy Watson told the State Board of Education “We must move students out of Level I (the lowest level on state assessments),” while also stating that students scoring at Level 2 are not “failing,” although the goal is to move 75 percent of students to Level 3 or 4 as the best preparation for postsecondary. About one-third of Kansas students are at the lowest level in the latest results.
A decline in test scores was widely expected and follow trends in virtually every other state following the COVID pandemic. But Kansas tests scores have been declining for almost a decade, after rising for most of the previous ten years. There is wide disagreement on the reasons for the decline, the importance of test scores compared to other educational measures – some of which have been increasing even as test scores have declined – and what to do about it.
This report is an attempt to help school leaders and the public understand the tests results, the debate over the significance of these tests, and how schools might respond.
What new test results have been released?
The National Assessment of Education Progress (NAEP) reading and math report cards for 2022 were released Oct. 25, the first such reports since 2019, prior to the COVID pandemic. These reports show student performance at grades 4 and 8 for each state and certain other jurisdictions since at least 2003. Since 2019, scores declined sharply in both subjects and grades and declined for most states, including Kansas. In the 2019, on an average of the four tests, 73 percent of Kansas students scored at the “Basic” level or higher, and dropped to 66 in 2022. The percentage at “Proficient” or higher was 35 precent in 2019, dropping to 29 percent in 2022.
Experts say much of the decline since 2019 was caused by pandemic-related disruptions in education, social interactions and employment.
Also in October, the Kansas State Department of Education released 2022 results of state tests in English Language Arts and math, which were given to students last Spring. Formally called the Kansas Assessment Program (KAP), it is the second year of results since the COVID pandemic began in 2020, when schools were closed to in-person learning and no tests were given. Last year, math performance fell more than reading. This year, math performance slightly recovered, but reading dropped more.
(Earlier this fall, the “long-term” NAEP results were released, which also showed a significant decline. This report, which extends back to the 1970’s, provides longer trends for the nation but not for individual states.)
What are the differences between the NAEP tests and state assessments?
NAEP is a national test given to a small sample of students in each state, so there are no individual, school building or district results. NAEP math and reading tests are given only at fourth and eighth grade for state results, and typically only every two years. The scheduled 2021 test was delayed due to COVID. As a result, there is a three-year gap between 2019 and 2022 results. NAEP scores provide a way to compare state academic learning over time, since the test is basically unchanged since 2003, and to compare states with each other. However, the test is developed by a federal agency under the U.S. Department of Education and does not reflect individual state academic standards, so different results may be due to different state expectations.
Kansas state assessments in math and English Language Arts are given annually to virtually all students in grades 3-8 and 10, so individual student, school and district results are available, as well as state totals. Accredited private schools also participate. These tests are developed by the Kansas State Department of Education and testing experts to reflect Kansas academic standards in reading and math, which schools are encouraged but not required to teach.
The current tests have only been used since 2015, providing a more limited period of comparison over time. These tests may be used by educators and parents to assess the progress and knowledge of individual students. They are also used as one of several criteria for school accreditation and may be used to compare the performance of Kansas schools and districts. KSDE issues “star recognition” awards based on high achievement on these tests. Results of state assessments have also been used in school finance litigation.
How are the NAEP and state tests similar?
Federal law requires states to develop and administer state assessments and to participate in NAEP. (State assessments are also required by Kansas law.)
Both NAEP and Kansas assessments place student scores in one of four benchmark levels. NAEP “achievement levels” from low to high are designed Below Basic, Basic, Proficient and Advanced. Kansas “performance levels” are designed Levels 1 through 4.
Although NAEP and Kansas tests are developed individually, the results are quite similar. In 2022, 66 percent of Kansas students scored at the Basic level or higher on NAEP when all four tests are averaged, compared with 66 percent of students who scored at Level 2 or higher on state assessments. Likewise, 28 percent scored Proficient or higher on NAEP, compared to 31 percent at Level 3 or higher on Kansas tests. Results were also very close when looking separately at low-income students (eligible for free or reduced-price meals) and not low-income students.
Neither NAEP or Kansas assessments designate any level as “passing” or “grade level.”
What does a score at various levels really mean for students?
If NAEP and KAP levels are not meant to designate a passing score or grade level performance, what do they mean? After all, it is highly unlikely any student has ever been asked about their state assessments to attend college or get a job or promotion. Recent research by the Kansas State Department of Education indicates that higher performance levels mean a greater likelihood – but not a guarantee – of future educational success.
Looking at students who have previously taken the state assessments through 10th grade and are now at least two years past graduation, KSDE found a correlation between achievement levels and both graduation rates and postsecondary success, which the KSDE defines as having completed a postsecondary credential or being enrolled in a postsecondary programs two year after graduation. (Postsecondary means any technical certification, two-year or four-year degree program.)
For students scoring at Level 1, only about 80 percent graduated and less than 25 percent had postsecondary success after two years. For Level 2, the graduation rate increases to 90 percent and the postsecondary rate is just under 50 percent. For Levels 3 and 4, the graduation rate exceeds 95 percent, and the postsecondary rate is about 75 percent.
Because about 75 percent of Kansas jobs are expected to require some type of postsecondary education and because additional education after high school on average results in higher income, higher employment and less poverty, the Kansas State Board of Education adopted a goal of getting 75 percent of students to Level 3 or higher. (According to a national study comparing state assessment results to performance on NAEP, Kansas has among the highest standards in the nation for its state tests.) However, Kansas Education Commissioner Randy Watson has told the State Board of Education that the most important task was to move students out of Level 1 to at least Level 2.
How have Kansas scores on the NAEP changed over time and how do they compare to other states?
NAEP provides results separately for four tests: both math and reading in fourth and eighth grade. To simplify these results for trends and comparisons, KASB averaged the percent of students at the Proficient level or higher – similar to Kansas Level 3 and higher. The results were compared to the average of all public school students in the nation, and an average of states in the Plains regional and other states bordering Kansas (Minnesota, North and South Dakota, Nebraska, Iowa, Colorado, Missouri and Oklahoma).
From 2003 to 2013, the percentage of Kansas students scoring Proficient or higher rose from about 35 to 40 percent. Kansas performed better than the national and regional average, both of which were also rising. From 2013 to 2019, Kansas performance fell back to about 35 percent, similar to national and regional averages which were basically flat over that period. From 2019 to 2022, Kansas dropped to under 30 percent, slightly below the national and regional averages.
In Kansas – as in all states – low-income students (defined as those eligible for free or reduced prices meals) have significantly lower academic achievement, but the trends have been the same. Kansas low-income students scoring at proficient rose from just over 20 percent in 2003 to over 25 percent in 2013, higher than the national and regional average. Kansas then dropped back to about 20 percent from 2015 through 2019, similar to the national and regional averages, and fell to 15 percent in 2022, three points below the national average and one point below the regional average.
Finally, the same pattern held for non-low-income students, who do better than the average for all students. Kansas increased from almost 45 percent at proficient in 2003 to almost 55 percent in 2013, higher than the region and nation, then fell back to about 50 percent from 2015 through 2019; finally falling another ten points to 40 percent in 2022 – about four points below the national average and slightly below the regional average.
NAEP performance fell more sharply for non-low-income students than low-income students. Although poorer students continue to trail substantially, the gap between the two gaps actually narrowed. That’s somewhat surprising, given that many educational experts believe the impact of the pandemic will be much greater on low-income students. It should be noted that the federal government has provided billions of dollars in temporary aid to school districts to address learning loss and other issues, and the distribution of those funds are heavily weighted to schools based on poverty.
What major, measurable changes occurred in Kansas and nationally from 2003 to 2022 that might have impacted test results?
Rising NAEP scores in Kansas from 2003 to 2013 coincided with a decade of “real” (more than inflation) increases in Kansas per pupil funding, especially following the Montoy school finance case from 2006 to 2009.
Nationally and regionally, funding also increased more than inflation. This was also the decade of the No Child Left Behind Act, which placed a major – almost exclusive – focus on test scores, and national and regional test scores increased as well as those in Kansas.
Following the deep recession of 2008-09, almost all states had per pupil funding below inflation for several years, but Kansas cuts were deeper and lasted longer than the national and regional averages.
Total Kansas funding dropped from 98 percent of the U.S. average in 2009 to 88 percent in 2019. Over that period, Kansas NAEP scores fell from above the U.S. average to about the same as the U.S. average.
Kansas began increasing school funding more than inflation in 2018 under a six-year funding plan in response to Gannon school finance decision, but by 2020, the most recent year available, Kansas was still far below 2009 levels compared to the nation.
Also in 2015, the federal No Child Left Behind Act was replaced by the Every Student Succeeds Act, which de-emphasized test scores and placed more emphasis on high school graduation and college and career preparation. In 2016, the Kansas State Board of Education began the Kansans Can program, which increased emphasis on high school graduation and postsecondary attainment, as well as broader workplace, community and personal skills. From 2015 to 2019, prior to the COVID pandemic, NAEP scores in Kansas, the nation and region were generally flat, while other measures such as graduation rates and college participation, continued to increase.
Finally, the COVID pandemic caused school closures in every state that reduced or disrupted in-person learning and student support services, and increased social, emotional and economic stress on students and families, which many experts say is the major cause of steep drops in student test scores from 2019 to 2022. However, Kansas had a slightly deeper drop in NAEP results from 2019 to 2022 than the national or regional average.
While restrictions on school attendance due to COVID ended, many students continued to miss learning time. Commissioner Watson reported to the State Board of Education that chronic absenteeism almost doubled from about 14 percent in 2018 through 20, to 25.7 percent in 2022. (Chronic absentees is defined as missing 10 percent or greater of the total number of days enrolled during the school year, including both excused, unexcused, out-of-school suspensions and in-school suspensions that last more than one-half of the school day.)
Many other reasons have been proposed for declining educational performance, including more students dealing with mental health issues, the impact of teacher shortages, and more focus on social and emotional learning and social issues. However, there are no standardized sources of information on these issues to provide state comparisons.
How have Kansas scores on state assessments changed over time and how does performance differ by public and private schools and by student income and disability?
The current Kansas state assessment program began in 2015, so it provides a much shorter time frame than NAEP. Earlier versions of the state testing program were generally improving for all students and subgroups, just as reflected in NAEP scores, until the early 2010’s, when Kansas NAEP scores began to fall.
The charts below show the percent of all students (averaging math and English Language Arts tests) by the red line in the center for the percentage of students scoring at Level 2 or higher and at Level 3 or higher.
Because of the debate over expanding public funding for students attending private schools, the average results of the five private school system in the state (Lutheran schools and four Catholic diocese schools), are also shown, indicated by the blue line.
Private schools have very different student demographics than most public school districts. Since 2015, each of these five districts have averaged less than 33 percent of their enrollment qualifying for free or reduced-price meals plus students with disabilities. There are only 10 of 286 public school districts with fewer than 33 percent low-income students plus students with disabilities. The average of these districts is the orange line in both charts.
Finally, at the other end of the demographic range, there are 11 districts with more than 92 percent of students who are low income plus students with disabilities. The average of these districts is shown by the yellow lines.
In 2015, an average of over 75 percent of all Kansas students scored at least at Level 2 and about 37 percent of students scored at Level 3 or higher. Based on KSDE research just under half of students at Level 2 go on to postsecondary success two years after graduation and about 75 percent of Level 3 and 4 students go on to postsecondary success.
For students in private systems and public school districts with similar low percentages of low-income and disabled students, scores were higher. In 2015, about 90 percent were at Level 2 and higher and nearly 55 percent of private students and 53 percent of similar public districts at Level 3 and higher. Even with similar demographics, every private school system has fewer students with disabilities than any public school system, and private schools can adopt and enforce academic performance requirements that public schools cannot.
Districts with the highest percentages of low-income and disabled students were lower, as expected. In 2015, 67 percent at Level 2 and higher and 34 percent at Level 3 and higher.
File: State Assessment History 2015 to 2022 by Group
Source data: State Assessment Results 2015 to 2022 All Students, Demographic Table 2015 to 2021
Between 2015 and 2019 private school averages dropped about two points, similar public schools dropped about three points and all students dropped about five points. The districts with the highest poverty/disability districts declined by five points for Level 2 and above from 2015 to 2018, but actually increased by about three points in 2019, the first year after increased funding under Gannon began. (The largest share of Gannon funding went to districts with the highest percentage of low-income students.)
Similar to NAEP results, each group declined from the pre-pandemic year of 2019 to 2021, the first year tested after COVID. Lower poverty systems, public and private, declined about 1-2 percent; the state average dropped 3-4 percent, and the highest poverty districts declined 9 percent for Level 2 and higher and 6 percent for Level 3 and higher.
The larger declines for higher poverty districts in 2021 were expected; many of these districts remained in remote or hybrid learning longer; these families had less access to broadband for remote learning; these students need more school services that were disrupted by COVID and had fewer family resources to make up the loss.
In 2022, low poverty private and public systems and the state average continued to decline, but at a slower rate than the previous. However, the percentage of students in the highest poverty districts at Level 2 and higher and Level 3 or higher actually rose slightly. As noted under the discussion of NAEP results, one reason may be the impact of federal ESSER COVID aid to school districts, which is disproportionally targeted at high poverty schools.
What will these new test results mean for Kansas educational policy going forward?
For critics of public education, these results will be used as evidence the system isn’t working. Both nationally and in Kansas, many Republicans have argued that loss of learning under COVID has been a self-inflected wound, with schools using remote or hybrid leaning longer than necessary. Many also argue that public schools have placed too much emphasis on “non-academic” issues, such as social emotion learning, equity and diversity, gender and sexual orientation, which has distracted from academic learning.
In Kansas, these concerns have led to new laws that have restricted remote learning, added requirements for early literacy and funding for supplemental math instruction, new reporting heavily focused on test scores, and requiring school boards to document a review of test scores in adopting their annual budgets. Previous proposals – likely to considered again – would add requirements for access and challenges of school curriculum and materials, expanding funding for students to transfer to private schools using public tax dollars, and putting new requirements on how districts use funds to address at-risk students.
Because districts have now received the final funding scheduled under the six-year Gannon plan, and because many districts are receiving significant (but temporary) funding for federal COVID relief, school leaders may want to review how they are using those funds and the results they are seeing. The Legislature and others will certainly do so.
The Kansas State Board of Education, which will likely have least three new members after this fall’s election, has adopted an education vision called Kansans Can and is implementing a new accreditation system that focuses on academic preparation, measured by state assessments, plus high school graduations rates and postsecondary success. But it also looks at kindergarten readiness, social and emotional learning, civic engagement and individual plans of study, as ways to support academic preparation, graduation and postsecondary success, as well as provide students a broader set of skills for life.
These goals were adopted after extensive input from educators, parents and community leaders and employers in 2015. Similar input was received in public meetings last summer and fall.
This is a much broader set of educational goals than under the No Child Left Behind era, when test scores were rising but there was widespread belief the focus was too narrow. The challenge for local school districts is whether those goals can be balanced so all measures are rising together, rather than seeing those goals as competing.
As school leaders begin to share and discuss these results with school boards, site councils and parent groups, legislators and community leaders, they may wish to address the following: