The Urban Institute published a report showing a six point higher rate of college attendance for students with tax credit scholarships than for ‘comparable’ traditional public school students. There is a serious flaw in the comparison groups. The FTC group included younger and older student groups. The younger group had fifteen percent more advantaged students and the older FTC group had 23 percent more advantaged students than the public school ‘matched’ groups. Given the correlation between achievement and income, it is not surprising that the FTC group entered college at a higher rate. The fact that most of the increase was associated with private and out-of-state colleges is another clue that these students were more advantaged.
NEPC published an evaluation of the Urban Institute study. In a nutshell, the answer to which group attends college at a higher rate is…’who knows?’ The issue, as always, is the validity of the comparison groups. Here are the three concerns raised in the NEPC study.
- Selection bias: There were two-three times more public school students than there were private school students in the groups compared. The groups were matched on free and reduced lunch status. There is a known selection bias that can result in advantages for private school students. (This is particularly true in the FTC program because such a relatively small percentage of students remained in the FTC program for four years. The Urban Institute study did acknowledge that students who had the highest college attendance rate were Hispanic and born out of the U.S. )
- Higher college attendance rates could be associated with better college matching rates for FTC students.
- The higher college entrance rate for FTC students does not indicate college graduation. The low college completion rate for FTC students may signify that the effect of FTC participation may be null or even negative.
It is discouraging to find studies of program results being used as pawns to justify programs instead of improving student achievement.