CREDO is not credible, and never has been

Recently the CREDO institute based in Stanford has published national and state by state comparisons of charter schools that indicated charter schools may perform slightly better than traditional public schools in certain circumstances. Despite the lukewarm conclusions, privatizers and education reform advocates immediately touted these findings as proof positive that their draconian agenda was working. Following closely upon the national release was a local release evaluating charter schools in the New Orleans area. After Katrina, most of the city of New Orleans’ schools were turned into charter schools of one flavor or another. Anecdotal success stories have abounded in the wake of this charter subversion an invasion, and the New Orleans model/miracle has been touted far and wide as cure-all that should be adopted by anyone wishing to remake their education system to one that can ignore the systemic and generation impacts of poverty and instead focus pure “education choices.” Despite substantial evidence to the contrary, “school choice” has erroneously become synonymous with “school quality” and advocated as the elixir that has made dramatic improvement in New Orleans possible.

This is where CREDO comes in. Anecdotes make nice stories to tell at workshops, and great fodder for the people sections of newspapers, but Reformers were realizing they needed their own “evidence based research” reports to tout the miraculous claims they were making to lawmakers, legislators and civic organizations not already sold on the privatization movement and the dubious narrative that schools and student performance as declining (especially compared to international standards) and that “bad” teachers, uncaring teachers unions and inflexible traditional education venues were responsible for this decline.

CREDO received its first set of data from Louisiana January of 2012. I was responsible for helping pull all their data together, but I left the Louisiana Department of Education shortly after this data was sent to them. When I left, I took a lot of institutional knowledge with me – specifically much knowledge about flaws in this data, what this data contains, and what it doesn’t, and some of the limitations of using this data to evaluate different aspects of our student populations. I contacted the head of the research team, Devora Davis, not long after leaving DOE to offer my help and insight, but was rebuffed at the time.

From: devdavis@stanford.edu
Date: Mon, 27 Aug 2012 09:26:04 -0700
Subject: Re: Louisiana Education Data
To: jasonfrance@hotmail.com

Hi Jason,

Thank you for reaching out to us. Collaboration isn’t a possibility, since our agreement with the L-DOE does not permit us to share data with other researchers.

Best regards,
Dev

—
Dev Davis
CREDO at Stanford University

I assumed this was because CREDO was simply trying to observe the letter of the MOU they had signed with the LDOE as they claimed (however despite their assertions at the time, I discovered later that there were no exclusions to consulting with outside sources about the dataset.) When I contacted Devora, or “Dev”, as she told me she preferred to be called, this was not a cold call. Dev and I had discussed the possibility of working out a data sharing project for several years, but the logistics and legal framework was not there. Additionally quite a few resources would have to be committed for some time and at no small expense to satisfy their needs. It came as a quite a surprise then that John White, someone who has become notorious for refusing to release any data to anyone (except where there was money or free positive publicity involved) had agreed to such a large, resource intensive project before even officially signed on as the State Superintendent of Education. Nevertheless “Dev” confidently informed us that “arrangements” had been made and “John” was happy to share the state’s data with them as soon as we could get it. Devora called us weekly to see if we’d made any progress and a task force was set up of 5 or 6 folks to work on pulling all the data CREDO wanted from all of our different databases. I was consulted to link the data all together with a random identifier and to make sure the most sensitive FERPA protected elements like name and SSN were removed from the final dataset.

However what is even more interesting is this blurb I found on Devora’s boss, the director at CREDO and the “project director” of this study, is none other than Margaret E. Raymond:

“Margaret E. Raymond is a research fellow at the Hoover Institution. At Hoover, Raymond serves as director of the Center for Research on Education Outcomes (CREDO), which analyzes education reform efforts around the country. CREDO’s mission is to improve the quantity and quality of evidence about the impacts of education innovations on student achievement in public K–12 education. Raymond, who has done extensive work in public policy and education reform, is currently researching the development of competitive markets and the creation of reliable data on program performance.

In partnership with the Walton Family Foundation and Pearson Learning Systems, Raymond is leading a national study of the effectiveness of public charter schools. The public-academic-private partnership helps public charter schools adopt information technologies as a means to both support their operations and generate information required by the study design. More than 250 public charter schools have joined the study to date.”

And moreover Mrs. Raymond is married to another more famous (infamous) senior fellow at the Hoover institution:

Eric Alan Hanushek (born, 1943) is a Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University. He is an expert on educational policy, and the economics of education. His research spans both the economics of school policy and the impact education on individuals and on economies. Major lines of research have focused on controversial areas of education policy including class size reduction, high stakes accountability, and the importance of teacher quality. He is perhaps best known for the controversial assertion that “money doesn’t matter”—that is, he says that the amount of money spent in an American school district is not related to the amount of student learning in that district—and he is often called to testify in court about school funding schemes

Hanushek is famous for his bizarre claims that class size doesn’t matter and money doesn’t matter in terms of educational gains for children that has made him a favorite of the Walton Family Foundation, individual Waltons, and Pearson Learning systems and Michael Bloomberg who have donated funds (usually the maximum allowable by law) to pro-charter state and local school board candidates and organizations like New Schools for New Orleans.

And if that weren’t enough to question the credibility of CREDO on charter school evaluations there’s also the situation that CREDO runs a charter school leadership institute in conjunction with many major charter school associations.

The PMI was initially developed as part of Building Charter School Quality, a three-year National Leadership Activities Project funded by a grant from the U.S. Department of Education’s Charter Schools Program. Under this grant, CREDO, in partnership with the National Alliance for Public Charter Schools, the National Association of Charter School Authorizers, and the Colorado League of Charter Schools, identified and developed best practices in the measurement of student and school performance, the management of performance data, and the use of performance measures to increase school and student accountability. The PMI has attracted more than 100 participants to date, and an interactive online version of the Institute was launched in June 2009.

I think this explains why CREDO was able to convince John White to turn over Louisiana’s complete dataset for a secretly pro-charter school organization to do their best to draft a propaganda piece masquerading as a credible study. CREDO is a complete sham. These are the same folks that promote TFA, John White, elimination of public education, for profit charters, and that poverty is irrelevant to academic performance.

Would you like to see how this played out in this pathetic study? The sad part is with all the gerrymandering of the data, the best they could come up with is a modest endorsement of charter schools with a low confidence of reliability.

Even if concerns over the study’s analytic methods are set side, however, Maul and McClelland point out that the study itself shows only a tiny real impact on the part of charter schools: “less than one hundredth of one percent of the variation in test performance is explainable by charter school enrollment,” they write. Specifically, students in charter schools were estimated to score approximately 0.01 standard deviations higher on reading tests and 0.005 standard deviations lower on math tests than their peers in traditional public schools.

“With a very large sample size, nearly any effect will be statistically significant,” the reviewers conclude, “but in practical terms these effects are so small as to be regarded, without hyperbole, as trivial.”

CREDO came out with a report that said charter schools do better than traditional schools, but the schools they compared to are RSD schools, state run schools, and taken collectively the worst district in Louisiana filled with all the students charter schools rejected, and even so, many of the charters did no better than the worst schools in the state, and many did even worse than the average of the worst district.

CREDO did not disclose that many of the charter schools that did better have selective admissions processes, specifically related to performance and test scores like Benjamin Franklin or socio-economically favorable geography and an admissions test like Lusher.

Benjamin Franklin High School Admissions Policy

PROCEDURE

1. Complete applications with all required documents must be submitted to the Admissions Office. • You may print it from the website (www.benfranklinhighschool.org and click the Admissions tab), or pick one up from the school. • Applications are accepted during school hours beginning in October. • Applications may be submitted in person, by mail, email, fax, or online (when available). The timely application deadline will be in January and the date specified in the Admissions Calendar. We will continue to accept applications after that date as long as space is available.

2. If the student does not have Iowa test scores, they will be scheduled to test when the application is submitted.

3. Test scores are mailed to the applicant when they are available (approximately a month after testing).

4. Acceptance letters are sent beginning in February of the year of application when we determine that the applicant is qualified to enter Benjamin Franklin High School and will continue on a rolling basis.

5. The final admission letter is sent from the principal when all required documents have been submitted including final report cards and LEAP scores where applicable.

VISITS TO THE SCHOOL

• Our Admissions Open House is held in the fall every year; check the Admissions Calendar and/or the website for the date and time. Our students, teachers, and administrators will be here to provide tours of the school, explain our academic and extracurricular programs and to answer questions.

• Families may schedule tours of the school by calling or emailing the Admissions Office. We are happy to provide guided tours but we do not provide for student “shadow” days.

ADMISSIONS REQUIREMENTS

Applying for 9th grade:

1. Residency in Orleans Parish.

2. 88 points on the admissions matrix which come from the reading, language, and math portions of the Iowa Test of Basic Skills taken in 7th or 8th grade and the 1st trimester or mid-year report card from 8th grade.

3. Promotion by the current school to 9th grade with no failing grades in academic subjects.

4. Passing score on the LEAP exam if you reside in Louisiana.

Applying for 10th grade:

1. Residency in Orleans Parish.

2. 88 points on the admissions matrix from the reading, language, and math portions of the Iowa Test of Educational Development taken in 8th or 9th grade and the grades from the 1st trimester or mid-year report card from 9th grade.

3. Promotion by the current school to 10th grade with no failing grades in academic subjects.

4. One credit each in English, math, science, social studies, and foreign language from your 9th grade school.

Applying for 11th grade:

1. Residency in Orleans Parish.

2. 108 points on our matrix from the reading, language, and math portions of the Iowa Test of Educational Development taken in the 9th or 10th grade and the student’s transcript showing all high school grades.

3. Promotion by the current school to the 11th grade with no failing grades in academic subjects.

4. Two credits each in English, math, science, social studies, and foreign language.

Applying for 12th grade:

Applications are not accepted for senior year.

Lusher Admissions

In-District Admissions
The in-district (neighborhood) process is available for those families who live within the Lusher school district (neighborhood).  To view the In-district address list, please visit our website at http://www.lusherschool.org – forms and downloads page.  In-district applicants must complete the application packet, provide the required documentation, and provide proof of residency.  

The in-district (neighborhood) process runs from February 18^th to March 14^th from 9 a.m. to 3 p.m. and on March 15^th from 9 a.m. until noon. Lusher strongly encourages parents living in the neighborhood district to complete a community application as well.

Community Admissions

The Community process is open to all those living in Orleans Parish. All community applicants must complete an application packet, take an admissions test, attend a curriculum meeting (K-1 only), and submit other required documentation.  Community applicants, who also live within the Lusher district, may submit an in-district application in addition to the community application.  

Lusher Middle School (6-8) first admits current 5th grade Lusher students. Remaining seats are filled with candidates who have applied through the community application process.

Lusher is close to 50-60% White while RSD averages 98% poor and African American.

Of course many of the remaining schools that perform better have stricter “unenrollment” criteria which filter out the lower performing students. For instance, charters can decide that any disciplinary problem warrants expulsion, but they can offer the students a chance to leave the school voluntarily to avoid an explusion.

These factors were not considered by CREDO, nor were they even mentioned which is completely absurd. CREDO went into a big spiel about trying to find “matched pairs” of students based on identical demographics to mask this glaring deficiency. It sounds good, but one of the characteristics they did not match on is the fact that many of these students were segregated by test scores and performance. How could there not be a difference in achievement learning when charters are already pre-selecting and de-selecting based on the very metric CREDO is measuring?

Another laughable claim that CREDO makes relates to SPED achievement.

Special education students in New Orleans charter schools progress significantly more than their counterparts in New Orleans TPS in both reading and math. This amounts to 65 additional days of learning in reading and 43 more days in math for special education students in New Orleans charter schools. These results are slightly higher than were found statewide.

I would say that this study finding borders on the criminal and strongly caution parents not to pay attention to this finding. Charter schools do not take on the more significantly impaired students. Even though the CREDO folks has access to the severity of disabilities, the CREDO study relied on the most basic of Special Education indicator for their study. SPED = Y/N. This indicator also included gifted “Special Education” students in many years. Most of the disabled students charter schools do accept are the mild/moderate classification with speech and hearing impairments, not the severe profound students that may even be hospital bed bound that traditional schools must serve.

I sent multiple questions to CREDO for an detailed explanation of how they accounted for these issues, the charter schools that filter students based on high test scores, the SPED indicator, the disparity of severity in SPED enrollment, but they have refused to reply to me to date. Here is a copy of my most recent inquiry that has gone unanswered.

Date: Wed, 28 Aug 2013 01:55:35 -0500
Subject: CREDO and New Orleans Charters
From: jasonfrance@hotmail.com
To: devdavis@stanford.edu

Dev,

From the article I saw posted locally it appears you used a number of selective admission charter schools that select students based on high test scores. Were you aware of this and how did you account for this?

Did you do any studies on differences in severity of SPED diagnosis between charter schools and traditional schools. Our charters tend to turn away the more severe profound and serve more mild moderate disabled students.  Did you examine or account for this difference in student population or simply classify them all as SPED or not SPED?

Did you account for data being wildly inaccurate and incomplete for many charters and RSD schools in 2005-2006 and 2006-2007?

I’ve seen a rebuttal that whether simply being a charter school only accounted for one half of 1 % of differences on Colorado’s NEPC site.  Do you have a response?

The Louisiana Department of Education has likewise refused to release this data to any non-pro-charter front organizations to conduct true independent research (they have been fighting the release of this data in the courts for the last 18 months), but this has not prevented newspapers from pushing this propaganda paid for by the Walton’s as independent research. If I was a journalist I would be ashamed that I allowed myself to be fooled by such an easy thing to research and such a glaring conflict of interests.

CREDO is simply not credible, they are not a research institution, they are pro-charter propaganda churner and should be classified as such by anytime anything they produce is quoted in an newspaper or news program that claims to be unbiased and impartial. If you are a parent, please do not pay CREDO any more attention than you would a miscellaneous propaganda pamphlet handed out at neighborhood grocery store, or stuffed under you front door handle. You can see CREDO as a joke, a wolf in sheep’s clothing, a PR firm or a charter school pimp, but an independent research organization they are not.