A lot of hoopla has been centered around third grade retention based on Florida assessment scores. It was credited for the large increase in Florida’s fourth grade reading scores. Something went wrong, however, in eighth grade. What went ‘wrong’ may have nothing to do with the quality of instruction. If so, what could explain the apparent decrease in eighth grade achievement? Some possibilities are suggested.
The case for retention. Retention became mandatory for third grade in 2002-03. Florida, however, retains students beginning in kindergarten as the above chart shows. Thirteen percent of third graders were retained. An additional nine percent of the lowest scoring students were promoted with a ‘good cause exemption’ such as a portfolio, an approved alternative test or consideration of a type of disability, and certain English language proficiency rules.
Retention helped (58%) some students according to a 2006 Office of Program Policy Analysis and Government Accountability (OPPAGA) study. It certainly boosted the image of the ‘Florida miracle’ of achievement increase. In spite of the publicity about school reform measures, however, the chart below shows the greatest improvement occurred between 1998 and 2002, before reforms and mandatory retention were implemented. Florida went from below the national average in fourth grade reading and math scores to above the national average. Some gains continued afterward.
Changes in retention policy. Over the years, retention has diminished because more exemptions were granted. (Some students were simply getting too old for the grade in which they were enrolled.) In 2014-15, the Florida Department of Education granted exemptions to 28,436 third grade students.
Note below the significant decrease in non promotions in 2014-15, in spite of enrollment increases, especially for third and ninth grade.
Retentions: 2002-03 2014-15
Kindergarten: 13,278 7,674
First Grade: 15,360 8,250
Third Grade: 27,713 9,458
Ninth Grade: 51,638 9,870
As of 2011-12, only fourteen states mandated third grade retention. Florida’s high retention rates compared to other states no doubt helped boost its fourth grade National Assessment of Education Progress (N.A.E.P.) scores in earlier years, but there may be more to that story.
Questions about the value of retention. When results are not clear, opposition grows. HB 131 Cortes (SB 1280 Rodriquez) have been filed to end third grade retention.
The big puzzle now perplexing the legislature, Senator Stargel SB 360 in particular, is why Florida’s fourth grade NAEP reading scores are six points higher than the national average, but eighth grade scores are just average. Fourth grade math scores are higher than average, but eighth grade math scores are consistently lower. Stargel proposes a study to find the answer. A look at the NAEP data gives some clues:
In fourth grade, 39% of students score at or above the NAEP reading proficiency level. Only thirty percent do so in eighth grade. These are the top scoring students.
One would expect that the percentage of high scoring students would be relatively the same for fourth and eighth grades unless, in some way, students’ characteristics change from elementary to middle school. What could account for the difference?
Do Florida students’ reading skills get worse over time? This seems unlikely. There was a relatively large increase in reading scores for eighth grade in 2009, and scores have only fluctuated slightly since then.
Other possible explanations. Perhaps eighth grade student characteristics are now different than in fourth grade? How could this be? There are several ways to explore this possibility:
- Florida’s retention policy is more extreme than most other states’ policies. Thus, Florida has more students who have been in school longer by fourth grade than do other states. One would expect their reading and math levels to be higher. This advantage may be lost by eighth grade when these skills are more complex.
- Does Florida’s school choice policy pull out more low scoring students in elementary grades, thereby elevating its fourth grade scores compared to other states? Do many of these students return to public schools in middle school and lower the state achievement scores?
We know from Florida DOE data that the Florida tax credit program enrollment drops more than one half between kindergarten and eighth grade. Which students leave the private schools and which remain? If the struggling students leave, as the DOE evaluations suggest, eighth grade scores in public schools would decline.
A similar examination of the achievement levels of students who return to public schools from charter schools between fourth and eighth grade may also shed some light on the changing student achievement
I welcome an evaluation of Florida’s school accountability approach to improving student learning.
“I welcome an evaluation of Florida’s school accountability approach to improving student learning.”
Actually, that “evaluation” has already been performed, back in 1997 by Noel Wilson. Any “reforms” (sic) that uses standardized testing as its basis is COMPLETELY INVALID as proven by Wilson. So that from the start the “reform”, actually educational malpractice, using student standardized test scores suffers all the foundational conceptual (onto-epistemological) errors and falsehoods that render any usage of the results, as Wilson puts it, “vain and illusory” (not to mention the unethical usage of using the results of a test for something other than what the test was designed.
For 15 years I’ve searched, begged and pleaded with testing supporters to show me where Wilson might have gone wrong with his analysis. Nothing, zilch, zero, more crickets than I hear from my nerve damage hearing loss and tinnitus. To understand his analysis please read is never refuted nor rebutted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
Arkansas, and the Little Rock School District in particular, has struggled for years with trying to close the gap between black and non-black (their terminology) students. The 8th grade slippage is present in scores here. Part is believed to be the general upheaval of black youths going through puberty in low socio-economic households. The U of Central Arkansas had done a lot of work tracking and studying this phenomena. I suggest any study look at these existing data.