NAEP 2015: Should we worry or celebrate?

dmbtestYes, the National Assessment of Educational Progress (NAEP) average scores are slightly down overall in 2015 from 2013.  The grade four reading scores did go up from 217 in 2002 to 221 in 2015.  This is a one point increase from 2007  Average eighth grade math and reading as well as fourth grade math scores declined.

  • Math grade four scores dropped one point and grade eight dropped 3 points.
  • Reading grade four was the same and grade eight dropped two points.

Do changes of a point or two have any significance?  Not really.  It is comforting when scores tend to rise, but they often fluctuate some.  It is more meaningful to look at scores over time.  This is a very revealing exercise.

For example, if you compare the percentage of students below the NAEP basic proficiency level between 2007 and 2015, you discover that some real improvement has occurred in reading but not in mathematics.

Between 2007 and 2015, the percentage of Florida students scoring below basic proficiency

  • increased in grade 8 math from 32% in 2007 to 36% in 2015.
  • was nearly the same in grade 4 math, 14% of the students were below proficiency in 2007 and 15% were below in 2015.
  • decreased in grade 8 reading from 29% in 2007 to 25% in 2015.
  • decreased in grade 4 reading from 30% in 2007 to 25% in 2015.

There was little change at the top of the scale.  The same percentage of students scored at the advanced level across years.  Thus, the improvement was in the scores of the lowest achieving students.  What is curious is the dramatic improvement in the grade four reading scores of lowest achieving students from 2002 when 40% scored below basic proficiency and 30% were below 2007.  This is attributed to the decision to invoke retention of third grade students who scored a ‘one’ on the state assessment test.  Retained students reading scores improved and elevated fourth grade scores.  Florida also put an emphasis on encouraging low income children to attend private schools, on tax rebate scholarships, most of which were small religious schools.  This policy may also have contributed to the increase in NAEP scores for younger children simply because there was a smaller percentage of low achieving students in public schools.

I did a comparison of the percentage of students scoring below NAEP basic proficiency across states from 2002-2015.  These data are a little misleading because only fourteen states followed Florida’s example and retained low scoring third graders.  Non retention states were likely to have lower fourth grade scores than other states.

While there are differences among the states, the striking result was that the percentage of these lowest scoring students changed very little within states over time.  Nineteen states did significantly reduce their percentage of lowest (NAEP below proficiency) scoring students between 2002 and 2015.  Only nine states, however, continued to significantly reduce their percentage of below basic proficiency scores after 2007.  Progress stopped for all but three states in 2013:  Louisiana, Mississippi, and South Carolina.

Given the diverse nature of state populations, I compared the five largest states in grade four reading. The state NAEP average (S AVG) is listed along with the percent below NAEP basic proficiency.  While there were minor changes in scores across years, progress stopped early.  Florida had the largest average gain in reading, but the asterisk indicates that changes after 2007 were not statistically significant.  Florida also had the largest drop in the percentage of students scoring in the lowest category (40-25%) .   Changes after 2007 in Florida did not reach statistical significance.

Table:  State Grade 4 Reading:  Average NAEP Score and Percent of Students Scoring Below NAEP Basic Proficiency

Year: 2002 2007 2015
S Avg   % below S Avg   % below S Avg   % below
State
California 206       50* 209      47* 213       41
Illinois 217       39* 219      35 222      32
New York 222       33 224      31 223      32
Texas 217       38 220      34 218      36
Florida 214       40* 224      30* 227      25

Some educators and legislators contend that increasing standards will motivate schools, teachers and students to raise achievement.  Standards in states across the country vary dramatically not only in the skills required but in their passing standards.  There is a NAEP comparison of state assessment standards that illustrates the difference in state assessment proficiency standards.  By mapping the state assessment scores to the NAEP assessments, it is possible to estimate the difference in NAEP equivalent scores in grade four and eight reading and mathematics.  In other words, which state assessments have higher standards for passing than others?  In 2013 there were large differences among states in what students were expected to learn.

  • Florida had the tenth highest standard in 4th grade reading and ninth highest in 8th grade reading.
  • In math grade four, Florida was 14th highest standard, and 13th highest in grade eight.

The standards ranking by state only tells part of the story.  Most state assessments, including Florida, set passing levels at the NAEP basic level of proficiency.  A few states, Wisconsin, New York, Texas, and Massachusetts, had higher state standards at least in one subject or grade level.  The impact of  the higher standards on achievement is difficult to discern.  In most states, the percentages of students scoring at the lowest level on NAEP changed very little, if at all over the past ten years.

The results of the experiment in school reform based on testing and accountability are not compelling.

 

 

 

 

Posted in California, Florida, New York, Public Education, Testing, Texas, Wisconsin.

One Comment

  1. National Assessment of Education Progress NAEP is the nation’s report card, Scores in math and reading were flat or in decline across the nation. Duncan says wait a decade, Bill Gates said “It will take at least 10 years to know whether this stuff works”. Race to The Top is a flop! Let it go. NCLB failed. It left many children behind. Stop funding failure. Stop making excuses.

    Randi Weingarten, president of the American Federation of Teachers, said that slipping NEAP scores are evidence that the nation’s focus on using standardized tests to judge teachers and schools failed.

    Students in the US public and private schools have taken the NAEP every 2 years since 1990’s. The exam in the country’s most consistent measure of K-12 progress, it can offer insight into the effects of demographic and policy change.

Leave a Reply