Teach For America Research Paper

This was written by Philip Kovacs, assistant professor of education at the University of Alabama, Huntsville assistant professor Philip Kovacs. A version of this was first published on Anthony Cody’s blog, Living in Dialogue, on the Education Week Teacher site.

He starts the post by referring to Teach for America’s training system, which involves taking newly minted college graduates and putting them through five weeks of intensive summer training before sending them into classrooms in some of the country’s most troubled schools.

Kovacs has tried to get the Hunstville school board to re-examine its decision to spend $1.7 million to bring Teach For America teachers to its public schools after it laid off 300 teachers over the past two years.

By Philip Kovacs

Recently I exchanged emails with a Teach for America employee in my city. On my last exchange, I tried to press her to answer at least one of my questions.

"Given the choice, would you see a doctor with 5 weeks of training or a certified doctor? A lawyer? An actuary?"

Answering with a ‘yes’ would be absurd. Answering with a ‘no’ would indicate a blatant disrespect for teachers.

Unfortunately this disrespect is exactly what we have going on in our country at this time: a blame-the-teacher mentality that ignores real world issues and concerns.

The TFA employee directed me to the organization's "research" page where TFA claims this: "A large and growing body of independent research shows that Teach For America corps members make as much of an impact on student achievement as veteran teachers."

This claim, based on the "studies" supplied by TFA, is misleading at best and demonstrably false at worst. I read all of the 12 "studies" available on TFA's website, and here is what I found.

Four of the 12 "studies" are irrelevant to the argument re: "make as much of an impact on student achievement as veteran teachers." Of these four:

One,“Creating a Corps of Change Agents,” is a fluff piece from Education Next that discusses the high rate of entrepreneurs who come from TFA;

The second is a peer-reviewed piece, “The Price of Misassignment: The Role of Teaching Assignments in Teach For America Teachers' Exit from Low Income Schools and the Teaching Profession,” which discusses improving TFA retention ;

The third, “Teacher Characteristics and Student Achievement:
Evidence from Teach For America,” discusses predicting outcomes at the time of TFA hire. (This one could have gone under "problematic" as the front page contains the disclaimer "PRELIMINARY AND INCOMPLETE" in all caps.)

The fourth is another peer-reviewed article, “Assessing the Effects of Voluntary Youth Service: The Case of Teach for America” presents evidence against TFA's claim that TFAers go on to "pro social" jobs.

There are problems with seven of the 12 studies that include some some methodological flaws. In fact, two of the seven acknowledge such flaws and warn the reader against making judgments based on their data.

And some of the seven show mixed results, i.e. TFA recruits are better at math than some teachers in some cases but are not better in other subjects, or they are better than novice teachers but not better than those with experience, etc.

Importantly, all of the seven studies that show mixed or problematic results are based on the use of Value Added Measurement (VAM). Here is a link to one peer-reviewed research paper, “Teacher Effects and Teacher Effectiveness, a Validity Investigation of the Tennessee Value Added System,” which argues that there are "several logical and empirical weaknesses of the system" used to evaluate teachers in Tennessee, the system which found its way to TFA's "research" page.

VAM is flawed at best, as argued in this report from the Annenberg Institute, an institute that can hardly be called partisan or pro-status quo, though some readers will no doubt level the criticism. Education historian Diane Ravitch, discussing the Annenberg report, asks an important question: "[Dr. Corcoran] describes a margin of error so large that a teacher at the 43rd percentile (average) might actually be at the 15th percentile (below average) or the 71st percentile (above average). What is the value of such a measure? Why should it be used at all?"

One of the issues educator Anthony Cody rightly addresses, however, is that the more we talk about VAM, the more we reify it as an accurate tool for determining teacher effectiveness, which it simply isn't.

A teacher raising student scores from the 15th to 25th percentile is going to look, to bean counters, much more effective than a teacher who raises student scores from the 85th to the 90th.

Which teacher is more effective? That's debatable, but it is the type of debate that happens when people go to football games and stare at the scoreboard for two hours. Both teachers might be equally effective, given the students that they are teaching and the conditions under which the students live and the teachers work. The teacher with the smaller gain might be more effective, but to really know, you'd have to know something about the teams and you would have to watch the game.
Finally, one TFA cited “study” is overwhelmingly positive towards the organization, but that "study" is actually a one-page summary from a survey of principals. The questions and data are not available.

It turns out that TFA left some reports off of its website. They aren't very flattering though, so I understand.

See, for example, “The Effectiveness of ‘Teach for America’ and Other Under-certified Teachers,” by Ildiko Laczko-Kerr and David C. Berliner, and “Does teacher preparation matter? Evidence about teacher certification, Teach for America, and teacher effectiveness,” by Linda Darling-Hammond et al. Note that both of these research papers are from the Education Policy Analysis Archives, one of the two peer-reviewed journals included on TFA's website.

As to the importance — or non-importance — of peer review, scholars and scientists have the mechanism in place to make sure research is sound and people aren't simply making things up and convincing others that they have found the cure for cancer, created a miracle drug like Vioxx, cloned a sheep, or narrowed the achievement gap.

Two of the 12 studies on TFA's website are peer-reviewed. Both are, however, irrelevant to TFA's claim "that Teach For America corps members make as much of an impact on student achievement as veteran teachers."

What is troublesome here is that we now live in a world where foundations and organizations have millions of dollars to spend lobbying and at the same time can bypass peer review in order to make a case for whatever they are selling. If you have enough money, science no longer matters. For more on this ask the scientists trying to address global warming.

Here is what I can say with some certainty based on TFA's "reports:" In some cases, in some places, and in some grades, TFA might produce better results on math tests than traditionally certified, novice teachers.

The rest is very debatable. The "research" is certainly not worth Huntsville paying an extra $1.7 million for Teach for America recruits, and it is certainly not good enough for the children who need experienced teachers.

The data showing experience matters is overwhelming, something the Coalition for Teaching Quality recently brought to the attention of Sen. Bernie Sanders. Consider taking the time to watch the entire briefing. TFA does not come out so well.

But don't take the CTQ's word for it, as TFA acknowledges that experience matters on its "research" page. Check out the "Portal Report" (which is a pdf): "Teachers with 4 years or more experience out perform teachers with 1 year of experience on 9 out of 10 indicators."

-0-

Follow The Answer Sheet every day by bookmarking http://www.washingtonpost.com/blogs/answer-sheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page. Bookmark it!

September 7, 2015; Las Vegas Review-Journal

America has a love-hate relationship with Teach for America. What began as the dream of one idealistic undergraduate in the late 80s is now, some 26 years later, an internationally recognized behemoth in the education reform movement, with more than $200 million (yes, you read that correctly) in investments as of last year.

A recent book, edited by T. Jameson Brewer and Kathleen deMarrais, titled Teach for America Counter-Narratives is the latest to put the organization under scrutiny. In an article this week in the Las Vegas Review-Journal, Washington Post columnist Esther J. Cepeda writes about the “explosive and jaw-dropping” stories written by 20 of TFA’s alumni, which she says “eviscerate the myth of TFA’s unmitigated success.” Her takeaway is that the book should be a cautionary tale to those studying the education reform movement. The stories reveal the smoke and mirrors (“money and great marketing,” in her words) that TFA uses to recruit the best and brightest while convincing their donors and other partners that they are moving the needle on outcomes.

According to its most recent tax return, TFA has total assets of close to half a billion dollars and revenues of more than $330 million, of which about 90 percent comes from government grants and contributions from corporations, foundations and individuals. An organization of this size and stature has an obligation to its constituents to demonstrate its success, and TFA has accumulated years of research findings about its programming, expansion and scale-up efforts. Marty Levine and Ruth McCambridge asked on this site several weeks ago whether Teach for America’s results justify its pillar status.

In 2013, Mathematica Policy Research concluded a federally-funded controlled study of TFA. Comparing TFA secondary math teachers across eight states with a control group of math teachers in the same schools, the study found that, on average, students in TFA classrooms gained the equivalent of an additional 2.6 months of school, as evidenced by end-of-year math assessments. However, two years later, a subsequent Mathematica evaluation was unable to replicate those results.

While the later study concluded that TFA teachers in early primary grades produced roughly 1.3 months of extra reading gains, that good news was overshadowed by the more troubling evidence that an overwhelming majority of TFA staff (87 percent) reported that they did not plan to spend the rest of their career as a classroom teacher or, for that matter, in any education-related career.

TFA takes great pride in how well it prepares its corps members for the classroom (in a smidgen of the time it takes a teacher to train through traditional avenues). Studies published on its website find that TFA corps members are “as effective” as other teachers in the same schools and that they promote achievement in measures “equal or sometimes greater.” Two state-wide studies in North Carolina and Tennessee demonstrating TFA’s relatively higher effectiveness in teaching STEM content corroborate these findings, although the Tennessee Higher Education Commission report doesn’t go much further than stating that TFA teachers “tend to be” more effective than other beginning teachers.

As NPR correspondents Eric Westervelt and Anya Kamenetz point out, this can be read as either evidence of TFA’s superior pedagogy or an “indictment” of traditional teacher preparation programs. But it should also be noted that some question the reliability of the research itself or claim that even in those limited cases in which TFA shows a positive impact, it is consistently small, and other reform efforts, such universal pre-K, teacher mentoring programs, and smaller class sizes, may have more promise over the long run. The sticking point that returns again and again is that of teacher attrition.

While TFA claims that two-thirds of its alumni have gone on to pursue careers in the education sector, they do not have hard statistics on the number of alumni who have remained full-time classroom teachers for even a minimum period of time. That is a shame, as it’s an obvious question on people’s minds and would be compelling information. Moreover, it’s hard to align the two-thirds claim with the overall trend away from the education space cited in Mathematica’s most recent study.

While there is little else in terms of concrete student gains, much less long-term, systemic results, what’s perhaps most confounding of all is TFA’s almost willful refusal to acknowledge the role of state schools of education (which train the majority of public school teachers) or its very partner schools, as allies in reform. The more one reads from those who have gone through the experience, the more apparent this becomes, with non-TFA faculty regarded as the “problem” that TFA must come in and “fix.” While it’s clear the majority of the TFA teachers and staff are sincere, smart, and hard-working, a corporate reputation too often deemed exclusionary and imperious precedes them, a reputation that seems to have fossilized despite numerous attempts at rebranding over the years.

So, for instance, while TFA has (wisely) moved its narrative beyond a rehashing of its early history and founder, conspicuously missing from the stakeholders it chooses to represent in the present day—its core members, alumni, students, and the more elusive “community”—are the very faculty and administrators of the schools they serve. Apart from national surveys that TFA commissions to query principals about their satisfaction with TFA faculty (consistently good), there is remarkably little testimony available from veteran teachers, guidance counselors, parents, and other school stakeholders about TFA’s ultimate impact on whole school environments.

Similarly, a five-minute video on TFA’s website focusing on greater New Orleans is astonishingly self-referential, attributing a disproportionate share of credit to itself for the turnarounds in that city. (“While [TFA] was expanding its footprint, the community has seen rapid growth in student achievement.”) While its footprint has indeed grown six-fold, the growth filled a notable vacuum resulting from the systematic firing of 7,500 public school faculty, and TFA teachers now make up 20 percent of the city’s teaching force. No mention is made of the hundreds of civic leaders, educators and others—for better or worse—who stepped up to reinvent the system in the disaster’s aftermath. And it should be pointed out that many questions remain as to how successful the Recovery School District has been. As reported recently in the New York Times, “there is perhaps no topic of the last 10 years as polarizing.”

At the time of this reporting, Matthew Kramer, appointed co-CEO of TFA two years ago, announced he was stepping down and handing over the reins to his counterpart, Elisa Villanueva Beard. Like all other news coming out of Teach for America, it, too, will be closely examined.

In the meantime, the question of whether TFA is living up to its mission to enlist, develop, and mobilize future leaders to strengthen the movement for educational equity remains an unanswered one. With more than $75 million coming in from government at last count and another $220 million from the philanthropic community, we should be seeing more evidence of long-term student gains and far more alumni continuing their impassioned work in the classroom.

 

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *