Welcome to 2013, gentle readers. I’d planned on starting off the new year with a post on my name choice, but this document seemed a little more important.
A friend of the blog recently produced this paper and a number of other very enlightening publications and pieces of work I’d like to share with you over the coming weeks. Dr. Mercedes Schneider, a proud product of the St. Bernard Parish Public Schools (1972-85) system, holds three traditionally earned college degrees, all in the field of education: B.S.,secondary education, English and German (LSU, 1985-91); M.Ed., guidance and counseling (West Georgia, 1996-98), and Ph.D., applied statistics and research methods (Northern Colorado, 1998-2002). She also has 18 full-time years in the classroom teaching a wide variety of subjects including English, German, teacher prep, and statistics in regular ed classes, alternative school, and special ed settings. She has taught both inside Louisiana and outside in Georgia, Colorado, and Indiana. For the past six years she has taught sophomore English and a teacher prep course in St. Tammany Parish, at Slidell High School.
As for her research experience, what is especially pertinent is that Dr Schneider was a manuscript reviewer for the Journal of Mental Health Counseling, the official professional, blind-review research journal of the American Mental Health Counselors Association (AMHCA). She later became the associate editor of research for the same journal. For four years (reviewer, 2006-08; associate editor, 2008-10), she reviewed statistical research studies for their quality, clarity, pertinence, and completeness for a nationally recognized professional organization.
Dr. Schneider covers a wide range of issues with Louisiana’s VAM teacher evaluation system. Some of the key points I’d like you to pay attention to are s follows:
- The VAM study was not done impartially. It was designed to produce a specific outcome and does not even try to address the data from an impartial standpoint.
- The researchers of this study failed to impartially point out the limitations of its application to all teachers, and to emphasize that poor data quality was impacting their results.
- The researchers did not verify any of the data, but recognized there were large sections missing and manually “fixed” data on their own in ways that would lend support to their conclusions..
- The researchers vastly understated the correlation of their results, which in most cases are only about as accurate as flipping a coin twice and getting the same result.
- Proper controls and rigor for a basic scientific study were not met, let alone one with such dramatic implications for teachers and Louisiana.
- The study actually indicates that a large part of student performance was not quantifiable (revealed by the moderate instability), but this point was not pointed out by the researchers.
I’ve been notified this report has been sent to local school district superintendents, legislators, and many relevant folks at the Louisiana Department of Education, including John White. If you have a stake in the outcome of VAM (if you are a teacher or parent of a public school student for instance) you might consider reading this article and dropping a line to your local legislator or super and see what they think of VAM and this analysis.
Happy New Year!
(Let’s hope we can all make a commitment to achieving an out with “old” and in with the “new” in terms of Education leadership.)