There is a data crisis at LDOE. Almost all of the data collection systems are failing. The data, statistics and reports being generated are garbage. Data is being ferried back and forth between the department and school districts using Excel worksheets and through e-mail correspondence. This leaves many students at high risk to data theft and privacy violations. Because the systems impacted are numerous and core to much of the reporting and analysis performed by the Department, it is impossible for LDOE to claim they are reporting accurate or reliable numbers for dropouts, graduates, TOPS scholarship awards, school performance scores, test scores, student counts and breakdowns for MFP funding, program counts. . . the list goes on and on. The situation is really serious and probably just about hopeless at this point.
I will explain how this situation developed and give specific examples of systems, impacted and correspondence I’ve received from school districts trying to work with the department.
This crisis was created intentionally by John White and his second in command that he brought with him from New York, Kunjan Narechania. White did not really care what the data said, because he had already determined the outcome for many of his programs. (I don’t think he was also not planning to be here longer than 2 years when all the cut-backs and destruction he’d wrought really started to impact daily operations.) White undertook a slash and burn campaign on the department’s data and analysis folks and immediately implemented policies that guaranteed data would deteriorate immediately. White abandoned a 4 million dollar warehouse named LEDRS we were just finishing. . . as he arrived on the scene, but not before using it to transmit almost all of the data contained in the Warehouse to CREDO to produce reform friendly propaganda masquerading as true data analysis.
John White immediately set about driving off all of the Department’s programming staff and replaced those positions with 6 figure salaried recruits with only a few years of teaching to their name and pricy vendors that cost 10 to 20 times more than the employees they replaced, and which in many cases were completely ineffective. The fresh faced 6 figured recruits came in with diverse backgrounds that ranged from shoe sales, Exxon intern, and barista and often sported poli sci degrees from expensive private universities. These folks talked about data, claimed they loved data, but they did not understand it, and did not care what the actual data said – only what they wanted it to say. This approach, combined with scrubbing the department website clean of historical data and refusing to produce data requests for anyone but reform biased organizations and cheerleaders worked for years. The data systems languished, bugs developed that were not addressed and workarounds introduced that didn’t work all that well. Eventually institutional knowledge was completely obliterated as data coordinators shuffled out 2 or 3 times and programmers supporting the internal systems dwindled from retirements, furloughs, and better offers – which was any offer at this point.
White sought to contract with numerous vendors like inBloom to house our data in their own warehouse and was planning to contract with them for all of our reporting needs until word of this scheme leaked out and parents fought back. When this plan failed White tried to squeeze IT blood out of the savaged turnip of a former department, but it was too late.
Kunjan, White’s second, fired existing heads of IT and data collections and replaced them with a former Education Technology Specialist named Kim Nesmith. (Education Technology Specialists basically review Ipad learning applications and make sure electronic blackboards turn on and review vendor products that might go to teachers, but they have no supervisory duties and they do not have significant data analysis, programming or reporting backgrounds.)
Kim is now the director of Data Collections and responsible for FERPA compliance, application modifications, state and federal data reporting and data requests. Before becoming appointed as the director of the Data Collections department, she shrewdly added the title “Data Quality Director” in her e-mail tagline. This means she self-appointed herself as a director of data (but not people). To get catapulted to a real director position, Kim worked out a special deal with Kunjan. Kunjan created a special position for Kim that they posted on the Civil Service website for one day to meet the minimum statutory requirements. Kim was instructed to immediately apply by completing an SF10 that would qualify her for the position. Her years of experience updating her church website for events became 10 years of web development. Kim and Kunjan went through the motions of the interview process, and then Kunjan gave Kim the job as they had discussed beforehand. I don’t believe anyone else was even interviewed, not that they would have had much chance to see the posting or apply. Sadly this is not entirely uncommon practice. As in so many things in life, it’s not what you know; it’s who you know that will land you a job, even in Civil Service land.
Kim comes off as very knowledgeable for people that have no idea what she is talking about. She often makes up jargon and details about processes on the spot to make her sound more informed and important. I’m told this drives the school districts crazy (many of them have contacted me about this and provided specific examples) because they know she’s making things up that don’t make any sense and which are not helpful or often even correct.
I imagine Kim snowed Kunjan this way at first. Kim actually called me to tell me much of this back when we were friends, that was when I thought she actually cared about my friends and former coworkers and the children of our state – so I did agree to help her numerous times to keep the systems running once she obtained her job. We both knew very well it was over her head (she told me as much herself when she asked for help) but I didn’t want to let the systems I had built and nurtured all those years go to seed and I was concerned about school districts getting a fair shake from the department. A lot of that data can make or break a school district and significantly impact all that district’s children. Even though I did not agree with her morals or behavior I set that aside and did my best to help when called upon . . . for a while. At first I thought Kim really wanted to try and be useful and help school districts out (as well as herself) and I still missed my job and the friends and school districts I had left behind. . . so I helped.
After we had a falling out I was contacted by a number of other former co-workers on the DL from time to time and districts connected with me through my blog and Facebook so I was able to communicate messages through them to send back to the department to help things run . . . if not smoothly, at least run. All of my coworkers are gone now. The first batch of replacements is gone . . . in two years this happened. Kim hired a few of her personal friends to run things . . .basically the same way she had been hired by Kunjan. Even though I could, I won’t name them because they may be decent people. I try not to hate the players until I have to, even if I do hate the game . . . but they know who they are. There is actually a lot more backstory I could go into, but it’s really not important for this piece. What is important is that I cannot help the department anymore . . .they are beyond my abilities to help through school district personnel. I tried before writing this piece and have now given up and I am imploring anyone who will listen to please try and secure the data and to provide the personnel LDOE needs to fix this area. I have learned 2 of the third round of replacements is leaving for reasons I won’t go into, but things are very bleak and this data can impact our schools. This data will impact our kids present, and their futures. State and federal funds are being jeopardized, which may not be recoverable, scholarship awards may be late or incorrect, SPS scores will be even more garbagy than usual. It’s a mess; a complete and utter cluster with no light in sight. That’s the sugar coating.
Here is an excerpt of an exasperated e-mail that was sent to LDOE about numerous system failures and problems from one of the district data coordinators.
- STS—-late start to open and broken at the moment (error report are not visible; processing limits, outside contractor hired to make up for the exodus of talent and yet no results) and have prevented LEA’s from relying on STS/OTS to verify graduation requirements and, thus, jeopardizing the integrity of graduates’ data at the LEA level;
- Dropout reports are chaotic and do not function properly—LDE has had conference calls this week with key, experienced LEA SIS coordinators whom I collaborate with regularly to pick their brains about what is wrong. Shouldn’t they know already??
- CVR report tab not operational making it very challenging to manage this task for the state. Furthermore, CVR and CIS was rumored to merge into a consolidated system. They remain separate. Why?
- Alternative Program reporting—LDE is due a report to BESE on alternative programs. LDE made LEA’s adopt program codes (900), but, yet, LDE still wanted LEA’s to report data already reported to the state—-Barry Landry pulled this requirement from INSIGHT last week after concerned was expressed. Why are others having to advise LDE of proper data management?
- Reports for Audit Schedules—Although I’m confident the LDE staff is managing time wisely and as efficiently as possible, request reports (ex: schedule 6 class size characteristics report) have not been processed timely or by estimates provided by the department.
- Monthly webinars—lack actionable details and in-depth experienced is lacking to advise on the intricacies of complex requirements.
To summarize what these comments mean.
- STS is the student Transcript System that defines which students meet the necessary graduate requirements and which ones qualify for TOPS scholarships. The report does not work and cannot calculate who should get a scholarship and what type of scholarship they qualify for. STS is one of the most complicated systems the department has. If this is broken beyond repair and a new vendor trying to rewrite it. . . that’s just messed up.
- Dropouts have not been working completely correctly for years resulting in lower than actual dropout percentages and higher than actual graduation rates. Now districts cannot fix incorrectly flagged dropouts at all. Left unfixed this will result in ginormous dropout increases. Spreadsheets are being sent back and forth to try and rig the results behind the scenes. This is dangerous and unlikely to be accurate and could open up possibility for a lot of abuse. This will impact SPS scores significantly if not fixed, which are used to calculate letter grades, which are used to define which schools are eligible for takeover by the state or charter operators.
- CVR is used to calculate highly effective and ineffective teachers. This translates to raises and firings. This was being done with spreadsheets earlier in the year and it is still fraught with problems. Using spreadsheets to transmit millions of teacher and student records and SSNs is bad. (If none of these transmissions resulted in some of this data being sent in an unsecured format and/or intercepted by criminals I would call that a miracle.) It also means the data is really, really crappy. . . and this is data that is used to rank teachers. Even though the results of VAM are postponed, VAM requires historical data to work. VAM requires accurate data to work. This data is probably complete garbage. Postponing the impacts of VAM is probably less attributable to being kind and listening to stakeholders, and more attributable to the fact they can’t actually create those reports or run that system anymore. If they do run a report off of this data the results will be random in many cases.
- The alternative programming collection is the one I wrote about last month and helped get cancelled – in theory. It was entirely based on school districts sending data the LDOE already had via spreadsheets. Apparently LDOE lacks the ability to access or run reports against their own data, even when they can collect it, because this data collection required districts to calculate the reports for the Department. LDOE also asked school districts to provide dropout numbers, which is a data element that comes from the department, not the other way around. That request shows LDOE has little confidence in their ability to ever fix their dropout calculation – which is used to calculate SPS scores which are used to shutter schools.
- The class size report is part of list of required documentation legislatively mandated auditors are obligated to ask school districts to produce every year. This report is supposed to come from LDOE and be compared to the numbers the district reflects in their systems at the end of the school year. This item indicates LDOE can’t produce this report for lack of resources or because they never collected the supporting data from the Student Information System in the first place. Class schedule data is used to pre-populate CVR which is used to rank teachers for VAM. Since the department was also requesting Excel files listing information for CVR earlier in the year, it is highly likely the Curriculum and Class Schedule data is complete crap.
I get lots of texts and e-mails mocking after every webinar Kim produces. LDOE folks that report to her are unable to answer any questions and they just read off of scripts for 30 minutes to an hour. The webinars are worse than useless since communication is completely one-sided and of poor quality. I’m assuming most data coordinators could read the script themselves. Unfortunately the content will still be weak and unhelpful in any event because of the lack of anyone with significant knowledge of how the various data systems work. Sadly this may be the only way LDOE data coordinators can communicate at all since John White passed a decree on the first day he took over LDOE that department staff cannot e-mail more than 10 people at a time, even in the event of emergencies. On the plus side they can submit a request to have info included in a newsletter every two weeks that only goes to the district superintendent. Brilliant.
Here are some additional e-mails I was included on so you can get a flavor of what kind of situation we have going on between LDOE and school districts.
Subject: Re: Revised Dropout Spreadsheet
YES — but how many of US will have to tell them that — Kim left me a voice mail today — that ended with I can send you a spread sheet of your STS data which will show you what you need to see???? come one people.
FIX the routines.
Yes. Apparently they are sending spreadsheets of STS data back and forth now.
Sent: Thursday, May 22, 2014 12:05:01 PM
Subject: RE: Revised Dropout Spreadsheet
Jessica [Baghian] and Kim,
Thank you for the call yesterday. It was very helpful to confirm that the Potential Drop out spreadsheet with the four students was a list of only those students who left my district and were never picked up by any school district in any processing period throughout the year. The three elementary students on the list will not affect dropout rates, but you will continue to research two of these children to try to locate them after leaving my district since the name link and social security links were not effective. You are going to research using every possible data link available. You will let me know the status of that search. I did mention the possibility of a search by the GUID.
The one high school student on the spreadsheet is a true potential dropout for my district. He is the only potential dropout for us.
As for the SISR22, Potential Dropout roster on SIS, the students that are included on this list will remain until the other districts file SIS data. At an undetermined time, you will work with these districts to clean up this data to get these students off my Potential Dropout roster.
As for my question regarding what dropout cleanup list will be used during the later Dropout cleanup time period, and how these children will be determined, I understand this list will be forthcoming from the SISR22 Potential Dropout roster on SIS. Any student who remains on the list at the close of the EOY processing period will be on the Dropout cleanup list. Therefore, it is imperative I have the SISR22 clean and the excess students from the other districts who are not potential dropouts removed.
I appreciate the information and will continue to monitor the SISR22. Please let me know when you have information concerning the extra children on the report.
Thank you very much for your help.
Wow. Districts are telling Kim and LDOE step by step what they need to do, like they are children. Sadly even this approach is not working.
Sent: Thursday, May 22, 2014 12:28:09 PM
Subject: RE: Revised Dropout Spreadsheet
ha ha ha ha! another spreadsheet – that’s funny!
I just got a call from Kim – all of 14 min. after sending my email – guess what – Duplicate Student report doesn’t work!! Gasp! Shock! Duh!
Yup, another spreadsheet. Very impressive data system they have there, eh?
Date: Fri, 23 May 2014 08:10:05 -0500
Subject: Fwd: Revised Dropout Spreadsheet
Apparently not even the duplicate students report is working, Jason. That is SIS-101 stuff.
I know, I know. That report is 20 years old and used by auditors to verify multiple districts are not getting paid for the same student. I have to wonder why bother auditing anything when the data is such a mess.
Date: Thu, May 22, 2014 at 12:20 PM
Subject: Re: Revised Dropout Spreadsheet
Well — they are steadiiily tearing down the data warehouse that WE worked so hard to create for the past twelve years. I guess it is easier to loooooose data with excel spread sheets?
Farewell poor LEDRS. We hardly knew ye. On the plus side, Gates vendors did get to earn 4 million dollars of Uncle Sam’s money creating it.
Last, but not least, is a little cartoon explanation one of the districts created to describe the dropout problem that LDOE was denying existed. They have literally become a laughing stock; so absurd is the situation at LDOE now.
I made a simple visual explanation of what is wrong with dropouts at LDE. I hope you enjoy it. Perhaps I’ll turn it into a video with voice-over….who knows, it could become a you tube sensation!!
See attached (just scroll through the pdf images and imagine my most sarcastic voice accompanying the images).