The folks who think Charlotte-Mecklenburg Schools makes up its numbers might want to skip this one, because a couple of interesting, complex national reports on CMS and data have come out recently.
The Michael and Susan Dell Foundation spent 2010-11studying the district's efforts to create a "data-driven culture." Researchers saw the turmoil of school closings, the data overload some teachers face and the battle over testing and performance pay, but still came away impressed by much of what was going on. The 18-page report is worth a read for anyone trying to get a handle on how CMS can move forward.
About the same time, an article in EdWeek cited CMS as a district making sophisticated use of data to identify which students are at risk of dropping out, using risk factors that show up as early as elementary school.
Given CMS' recent data woes, particularly the erroneous public report that purported to track students on track for graduation, that may sound like a setup for saying EdWeek got duped. But I don't think that's the case. The irony of the simplistic graduation-track calculation is that CMS does use a complex set of factors to identify and work with at-risk kids.
What strikes me about both of these reports is the tangle of moving pieces that makes up the CMS data picture. For every spreadsheet I peruse, there are reams of internal reports. Some are almost certainly helpful to kids and teachers; other internal data systems are so flawed or cumbersome that they fuel employees' skepticism.
For better or worse, the CMS accountability department has lost so many key players in the past year that rebuilding it is going to be a front-burner task for the new superintendent. The CMS data machine needs some repairs, but as these articles make clear, "scrap the whole thing" isn't a realistic solution.
Sunday, February 26, 2012
CMS gets kudos on data
Subscribe to:
Post Comments (Atom)
Posts
32 comments:
Has CMS submitted the school reports with corrected projected graduation numbers yet? Did I miss it or did they discover additional issues and don't want to risk another embarassment? If it is wrong a second time who will be forced to resign (and make positive comments about CMS to the press afterwards)?
NO DATA!
NO PEACE!
Sounds like a good read. Are the reports reviewing the process by which CMS numbers are derived or the fact that the numbers are derived to begin with. What happenned to the data driven Cochrane success story? Contrived or Derived?
The graduation data debacle, no data to support Bright Beginnings, suspect planning and data to push through bonds, building schools in the wrong place and schools overcrowded before the doors open and yes, the really big fish CMS has no data or clue about that drives nearly all education funding - the school lunch program - all lead to "stakeholders" not believing anything coming out of the mouth of CMS.
Simplify and verify.
6:59, CMS has not yet resubmitted the school progress reports, and I do wonder what's taking so long. At the school board meeting two weeks ago, Hugh Hattabaugh reported that they're still working on a better quality control system.
Ann, Sounds like political speak for we do not know what we are doing and cannot publish the very most elementary of school accountability data without fear of community reprisal.
BOE+
Broad+
Gates+
bogus data+
beaten down workforce+
suspicious clientele+
Music Man educrats= Suspect/no QC
ISO standards aren't applicable to this organization
Well then, problem solved CMS is the best district in the country bar none. No more here to report, everyone just move on "nothing to see here folks!"
Ashley Smith, a third-grade teacher at Endhaven Elementary, a high-performing school
in southwest Charlotte. “I have children who can already work at grade level, so it is my job
then to push them even further and help them learn that they can do anything. And there
are kids who come to me and can barely read, and it is my job to help them. Using the data
has allowed me to really reach those kids who need it”
Does your data show how these kids got to the third grade with a first grade reading level?
After forcing my way through those 18 pages, I came to the conclusion that the word data was broadly used over a hundred times and that we have teachers and principals using different data tools to manage their students.
CMS should already have in place detailed data for EACH student and each teacher should have that data prior to the start of each school year so they can be prepared to help those students who need the extra attention.
I think the more immediate issue will be rebuilding the trust and confidence of the teaching corps, which found the details of the recent trial consistent with their feelings of how they are treated within CMS.
When the economy finally does improve (which within Public education probably will be another year or so), the data that will be of most interest is how many people abandon a district which views them at best as interchangeable and at most as an impediment to reform.
No data in the world will be able to compute the endless changes in policies teachers have had to endure these past five years.Teachers do not trust the data because the first thing they know is that the system will turn around and change the entire system every few years.Administration is a joke and the policies that keep them in office are a joke.
The data on students has been available for years. The only change, and not a bad one, is that teachers can now have the data easily at hand. The data is compiled in a way that shows the entire testing, behavior and attendance history of student. It is used to assess where a student is. That is great as a diagnostic tool. "Johnny had trouble with reading beginning in the 3rd grade." Fine. Now what? This is the real issue. We can diagnose the problem but where we have difficulty in CMS (and in education in general) is in the way we apply the solution. We can't even begin to agree what the solution is. All the sellers of different programs will tell you that their solution is the best. Diagnosis is easy, treatment is the issue where we are still trying to figure things out.
If ISO doesn't apply to CMS why does CMS post banners claiming ISO certification? ISO is good about requiring documentation in industry, but much less useful in gauging the success of the educational process in schools. Most school districts that went with ISO certification opted out after a few short years. After investing thousands of staff hours, certification fees and paying for Broad program leadership CMS will no doubt opt out of ISO certification as well. CMS needs to prove itself to the community and stop looking to outsiders to be the proof.
I have read the 18 page document, and here are my comments (broken into 2 parts).
1) "Schools that performed well won almost charterlike autonomy, while less successful schools remained subject to stringent oversight and control."
Do tell – please name the schools that were given "almost charterlike autonomy..." as those are the school(s) I want to visit.
2) "Under the decentralized “freedom with flexibility and accountability” management model..."
When did CMS move to a "decentralized" model?
3) "In 2005, the academic performance of CMS’ minority and low-income students was significantly below that of their peers both statewide and in neighboring Wake County. By spring 2011, roles had reversed, with the same population of CMS students outperforming similar students across the state."
Since when did Wake County become our 'neighbor?’
4) During the course of our research, we saw both frustration and renewed inspiration among
educators. Overall, CMS has managed to create an educational culture in which teachers and administrators alike embrace data in the richest sense."
I'd love to know who they spoke to that praised all this data, especially when all the data from last year's 52 tests got LOST.
5) “We had so many initiatives going on at the same time that a lot of our teachers and principals got overwhelmed,” says Dr. Lynne Tingle, executive director of performance management for the district. “
Agreed.
6) “For instance, I will make a test that is directly correlated to the end-of-grade test and give it to my students..."
Great -- teaching to the test.
7) “Teachers at each grade level set a goal,” she explains. “And we discovered that we had set our goals way too low. That just blows you away as the principal that you can set your goals too low. "
I'm speechless. Don't even know what to say about this.
(Part 2 of 2)
8) "Even so, some teachers inevitably felt that the intense focus on assessing, reporting and analyzing data could, if not carefully channeled, be a distraction from the core business of teaching: “I think everybody in the building at times feel overwhelmed with data right now...”
Yep.
9) "Mint Hill Middle School teacher Jessica Shoup paints a vivid picture of the dizzying array of tools she uses to gather data. Her usage mirrors that of every other teacher and administrator we interviewed. “My data is scattered,” says Shoup. “Common assessment data comes in through Microsoft Excel, so there’re different spreadsheets that I can look at. NC WISE is a program that was brought into the whole state of North Carolina, and it’s a system that teachers use to keep track of attendance for each class period. It keeps track of the individual grades for each class. We can look up emergency information about the students, even though 90 percent of the time it’s wrong, and we can look at their class schedule so that we know where they are at all different times. And then the teacher portal also has information about the students.”
Remind me again why this article shows the authors still coming “…away impressed by much of what was going on.”
10) "And come May, we won’t get much teaching done between the end-of-grade assessments we’ve created with our data team, and the tests CMS has coming down the pike.”
This is one of the biggest understatements in the entire article.
11) "Moreover, training on using data-driven insights to inform more effective classroom practice has been inconsistent at best and nonexistent at worst."
Again, why is this 18-page report showing that they came “… away impressed by much of what was going on.”
12) "And then, they say, ‘Okay, now take this and go re-loop your students.’ But they never really give us any ideas on successful ways to go about re-teaching or re-looping the students.”
Wow.
13) All of page 16 is something that should be published at a Board meeting. Are they utilizing ANY of these suggestions?
14) All of page 18 should be given to the Board, as well.
15) "The proliferation of data tools used within and among CMS schools contributed to an environment that one 2010 report (which, in general, was highly laudatory of CMS reforms) characterized as “intermittent in its use of quality-control metrics.”
This says it all. CMS can't control its data or trust its data. And remember, CMS still hasn’t reissued their school progress reports.
Not only has CMS not yet released the "corrected data", but they have never, to my recollection, responded with a clear explaination of what the incorrect numbers actually represented!
They claimed that they were "copied from the wrong cells in the spreadsheet..." but they cannot tell us what valid statistic those numbers represented... that makes a reasonable person suspect that the numbers were intentionally falsified!
Ann, Ann, Ann. This is just the classic case of circling the wagons and everyone speaking from the same erroneous piece of paper. The more people you get and the more important people you get to say something is true, will in the end make it true. Don't you know that! It is worse than the self fulfilling philosophy. It is worse because it can not become true. It is simply looking at this through rose colored glasses.
It was a human error - a human (hmmm, who is the CIO??) authorized the purchase of a system that was wholly and completely incompatible with the existing database (data warehouse). (SAS vs. SQL for you cpuheads) No surprise CMS put out erroneous info.
8:16, isn't one of the future teacher evaluation systems running through SAS? I heard someone say it would essentially be a black box so whatever is spit out at the end is unverifiable in the process.
Only scoundrels and thieves blame the data systems.
John...
You'll have to explain that one.
I curently deal with and use one of the largest data collection/management companies in the world and right now I have issues with a certain part of their data reporting in one measure.
Their initial look and response back to me was they agreed the data looked suspect and I'm waiting for their fix to the problem.
In your mind, I suppose I'm either a thief or a scoundrel.
Data out, is only as good as data in.
I've come to only appreciate these kinds of reports with many grains of salt.
Sorta reminds me of an excellence in quality organization that a number of Texas companies formed and were members of, that met once or twice a year to award two of their members quality excellence awards, and it appeared to simply rotate within the membership who got the award until everyone eventually had won....
Ann Clark administrated over CMS schools during a period of time when a number of schools were shorted arts teachers, but recently gets an award from the NCAEA.
Vicky Hamilton administrated over CMS during a period of time when serious issue surfaced over eligibility of athletes at various schools but was inducted into the NCADA Hall of Fame.
Things are not always as they seem, in my opinion.
Chief Academic Officer Ann Clark received the Friends of the Arts Award for Administration from the North Carolina Art Education Association
http://www.cms.k12.nc.us/News/Pages/CMSeducatorshonoredforsupportingartseducation.aspx
Hamilton, who was the first female athletic director of a North Carolina public school district, has been inducted into the North Carolina Athletic Directors Association Hall of Fame for her innovative work at CMS.
http://www.cms.k12.nc.us/News/Pages/CMSathleticdirectorisconvocationspeaker.aspx
The Dell report is collection of anecdotes rather than a study. Ironically, the two schools prominently featured are now closed – Wilson Middle and Irwin Ave. Elem.
More disturbing is that this report is being touted in other media as a “data-driven” success story: http://www.educationnews.org/k-12-schools/charlotte-nc-schools-success-with-performance-driven-model/
Not surprisingly, the Dell Foundation has a vested interest in proving the effectiveness of the “data drive” approach: “Dell Foundation Launches Tool to Connect Student Data”: http://blogs.edweek.org/edweek/inside-school-research/2011/07/dell_foundation_launches_tool.html
That report wasn't so much a study as a commentary.
As a former IT guy with an MBA concentration in Information Management, their systems sound like an uncoordinated mess, typical of unplanned growth in technology without any overall plan.
It doesn't sound like the CIO has a real handle on their data needs.
Ideally, they would be working out of a single database with all their data or at least with a tool that could consolidate data from several sources.
But I'd be willing to bet they have duplicated (and most likely incorrect) data all over the place.
This report's description of their processes (basically uncoordinated and unrepeatable from school to school, see the reference to "superman"...) sounds like they really do not have solid processes in place for using their data, either.
Altogether it still sounds like a mess to me.
Csawyer:
Actually, that tool that Dell talks about sounds like something they might want to try.
You're probably right about this "study" being a bit self-serving because it looks like it was written with that "solution" in mind.
At least, that is pretty much what the study drove me to conclude.
And the folks at Dell are anything but stupid when it comes to making money.
It makes me wonder what Gates has in mind.
I always thought Gates was looking for the teacher "certification" and training piece of the pie.
Funny that both of these guys were college dropouts, too.
Not saying they weren't smart, mind you, but I do wonder why the sudden interest in education.
Here's a link to the Ed-Fi "standard".
It sounds to me like the Dell "study" was a sales brochure for this.
See if y'all agree after reading that report knowing that this is Dell's baby.
Ed -Fi:
http://www.ed-fi.org/
(Whoops, left out the link in prior message)
Gates's company is creating these extra tests like the ones CMS started to do last year. He is no dummy. He sees where the taxpayer troughs are getting loaded up.
In response to Wiley's comment regarding the comment from Ashley Smith:
Did Ms Smith actually need data to tell her which kids were at, above or below grade level?
Fact: Good teachers know within a week of school starting where their kids are, without ANY data from the district.
Bad teachers need the district to feed them reports that tell them what their kids can and can't do.
Fact: You don't need "data" to figure that out. You do need to understand education and we do need start producing teachers that are actually ready to teach.
FACT: 1st year teachers, understandably, are BAD for the business of education. We'd do our kids a great service by NOT allowing ANY first year teacher the ability to have their own classroom. They simply do not graduate from college ready to handle a modern classroom. EVERY first year teacher should be required to spend a full year learning from an experienced, high quality teacher. 2 6 week or 2 8 week blocks of student teaching are not enough for a new teacher, especially in elementary, to be ready to take over their own class. And our kids should not be guinea pigs for 1st year teachers, no matter what part of the county they live in.
Dell, Broad, Gates all have ulterior motives; to make more money by manipulating incompetent school boards into buying their pitches hook line and sinker.
We don't need more data, we need leaders who have a clue. I dare say we have a couple....but 2 out of 9 on the BOE and a lower percentage in central admin ain't getting it done I'm afraid.
Anon 1:07...
You missed the point of my question...
It doesn't matter what the data was or how many years she has been teaching.
I kid enters the third grade with a first grade reading level.
Why?
How did someone not know the same child went all the way through the second grade without advancing at all?
1:07 AM, I have tried ot run the math to see what that cost impact would be. As with most school districts the last few years, there have been few and far between new teachers hired. CMS and some other districts are big into TFA's and other such (short term contractors) nonsense. In the eyes of many professionals, they are worse than the worse because they did not even go through a typical education curriculum in the college. Yet, they all come with high flying accolades when we all know most of them ar ejust padding their resumes until the job market opens back up.
I have experience with one NC university that has its education majors in classrooms starting the freshman year. These teaching candidates do have a clue. They may only be doing tutoring at first and helping the teacher. But as they take a class like math education, they are required to create a certain number of lessons and actually do into the classroom and teach it. They may not be assigned the "master" teacher at that time but they are working closely with th elocal school system to get these candidates into various challenging classrooms.
And in their "practice teaching semester", they are actually in that classroom the semester before at least 3 days a week getting to know the kids and seeing the teacher work first hand.
Post a Comment