Friday, August 9, 2013

Lake Woebegon and grad rates

Astute readers looking at the graduation rates for Charlotte-Mecklenburg's 28 high schools may notice something odd:  24 of them list rates above the district average of 81 percent, including 13 above 90 percent.

That may call to mind Garrison Keillor's mythical Lake Woebegon, where all the children are above average.  Worse, it may raise questions about whether the numbers are valid.

In this case,  the seeming impossibility is due to the way school and district rates are calculated.  It's not a CMS thing;  I'm willing to bet you'll find the same pattern in virtually every district.

North Carolina,  like other states,  tracks the students who start high school and calculates the on-time graduation rate by the number who get a diploma four years later  (Superintendent Heath Morrison says our state is unusual in not including summer-school grads).  But students who switch schools after falling behind don't count toward school calculations.

Take,  for instance,  a student who starts ninth grade at School A but doesn't get enough credits to be promoted.  Sometime in his second year of high school he moves to School B.  School A doesn't get blame for his failure to graduate  (or credit if he does)  because he left.  But School B also isn't held responsible because he was behind when he got there.  As long as that student stays within one district,  though,  he still counts toward the district rate.

As you can imagine,  students who fall behind and move around are at special risk for failing to graduate.  So you see those students bringing down the district and state numbers without dragging down schools.

Morrison and his crew are well aware that plenty of people,  including me,  scrutinize CMS numbers closely  --  and with good reason.  The district has gotten black eyes for bad data,  including flawed graduation rates reported in 2006,  when the current tracking system debuted,  and error-filled school progress reports posted last year.  The skepticism started flowing as soon as we posted a story on this year's gains.

John King of Harrisburg quickly emailed me questioning why CMS couldn't quantify the impact of a change in graduation requirements,  from 28 credits in previous years to 24 starting with the Class of 2013.

"Given any level of competent statistical tracking,  it should require very little time or effort to produce dual graduation numbers, one assuming 28 required credits and one assuming 24,"  King wrote.  "I do something similar almost every day in my job!  It’s a key step in evaluating the effectiveness of the decision to make the change.  The failure to do so simply proves that there is more interest in managing the perception than the result and that there is no more transparency under this Superintendent than under the last!"

I requested exactly that calculation Tuesday afternoon,  when I heard that graduation rates would be released Thursday.  I agree with King that it's valuable information,  and I'm disappointed it wasn't available right away.  But Morrison and Chief Accountability Officer Frank Barnes have repeatedly said they're going to check and recheck anything they release,  rather than take another credibility hit by giving out something they have to retract.  They say they'll give me the numbers when they're confident of them,  and I'll report on that information when I get it. 

30 comments:

Anonymous said...

Sure Ann,
They'll give you great numbers they have no idea of accuracy and worry about follow up later when they're proven bogus. As you suggested, same channel, different faces, same modus operandi. No credibility and what about the dropped rates at some of the high schools?

Wiley Coyote said...

....and Project LIFT will be taking full credit for a 15 percentage point increase at West Charlotte.

Disgusting.

Let's have a breakout of the effect of going to a 24 credit threshold and badgering kids to come back to school and "graduate".

Let's also have the actual numbers by high school as to how many students were eligible to graduate broken out by how many did and did not graduate.

Old accounting joke:

What is the answer to 2+2?...

...whatever you want it to be.

Anonymous said...

Lies

Damn Lies

Statistics


Another year of smoke and mirrors and moving around the deck chairs on the Titantic. Constant changes with the learning zones and on and on. It is only to justify administration jobs. Nothing they do impacts the student on a day to day basis. I hope you fee good about your MARKET ADJUSTMENT on the backs of all these frontline teachers who make less than $40,000

Anonymous said...

Pay the Teacher and bring back their benefits!


COMMON GOAL for a COMMON CORE

Anonymous said...

It's interesting that, as noted in the article about grad rates, CMS now tops Wake in graduation rates for most demographics, including whites, African Americans, Hispanics, and most especially high poverty kids (CMS rate is 10 percentage points higher than Wake's). But wait--remember all the touting of Wake County's socio-economic based assignment plan (which the school board tried to ditch a few years ago but a new board reinstated). Activists and the press here loved that plan and were oh so sure that these particular demographic groups were doomed in CMS, which they felt was catering to the "selfish" suburbanites. The Wake Ed blog from the News Observer today points out that CMS has a much higher number of black and economically disadvantaged students than does Wake, and a much smaller number of white students. Anyone care to explain why the nirvana of school assignment has failed those it was supposed to help?

Anonymous said...

As 10:15 points out and as we have ssen the history of poor CMS performance when it comes to data, I believe are part of the reason the NC Legilature did a lot of what it did this last ssession. Combine that with the horrible economic outlook for NC and the ever increasing burden by the federal government for unemployment payback, Race to the top strings, Medicaid reimbursement, and Obamacare, you can make sense of its actions. While I do not like a lot of them, at least they were willing to look at reality rather than what the democrats had done in past years and just arbitrarily raise the tax revenue projections.

At least there is a raise for teachers in the following year's budget and relief by lower income tax rates.

As for the Wake County race/socioeconomic pupil assignment plan and how CMS has passed them, Raleigh is also tired of the race hustlers taking over public education. They fully understand public education has lost the battle to this special interest group.

Anonymous said...

They have the freaking data they must. Heath and Barnes don't want you to call their bluff for it would be a knockout to CMS image again. What they have done is shorten the list of possible students to track then spew good numbers from the data. It's crap and nobody believes their hypothetical data.

Anonymous said...

Either they have the data or they are too stupid to understand & comprehend that knowing the effect of lowered credits is critical. Either way they are incompetent.
On the other hand Charlotte Observer gave them the (false) front page headline they wanted. Shouldn't CO have all the facts before putting that headline?
Sad, very sad.

Pamela Grundy said...

Actually, calculating the difference in graduation rate based on a 24-credit graduation requirement vs. a 28-credit requirement is complicated. As with most statistics, it's important to look at human as well as statistical factors.

Since only 24 credits were required to graduate, a fair number of students who graduated with fewer than 28 credits this year might well have taken more classes if the requirement had remained at 28. Similarly, for students with 18-20 or so credits, 24 might have seemed a attainable goal, while 28 may have seemed out of reach.

It was definitely misleading of CMS to have a press conference touting the graduation rate increase without making it clear that requirements had dropped. However, producing numbers that can be directly compared with previous years would be pretty difficult.

It would be equally problematic for the state to pretend that it's possible to calculate student "growth" from the 2013 EOGs and EOCs, which were completely different tests from previous years. We'll see what happens with that in October.

Shamash said...

The honest way to do this is to post results using both methods for a period of a few years when the methods change.

That way anyone can see what differences are due to actual progress vs. changing the scales.

Pamela Grundy said...

Interesting to see those folks so eager to compare Charlotte to Wake don't seem to have felt the need to mention that the Wake graduation requirements, as best I can tell, held steady at 26 credits, while Charlotte's dropped from 28 to 24.

If hands-on work was successful with Charlotte's struggling students, as some CMS folks suggested, then Wake could perhaps learn from those techniques, adding those benefits to the well-documented benefits that come from economically diverse schools.

Wiley Coyote said...

Pam,

You hit the nail on the head.

We cannot afford anymore "misleading" information from CMS or the state....

We all want the same thing. It is in everyone's interest to have all kids graduate, but there is and has been too much gray area, too much deflection, too much misdirection and bad data.

I know I incessantly harp on the NSLP, but it drives much of the funding and decisions made in public education. If we can't fix that and get honest assessments based on verifiable data (verifiable data on dropout rates, test scores, grasduation, etc.), CMS will continue to have a cloud over its head and people will continue to dismiss what they are feeding us, while true success stories get buried.

Pamela Grundy said...

Wiley,

Indeed we cannot afford more misleading information. The national obsession with what I think we would both call "pseudodata" (or names unfit for Ann's family-friendly blog) has done an enormous disservice to children, to teachers and to the country as a whole.

As you know, I don't agree with you on the school lunch program. Part of the problem with data, as you know, is that getting high quality data is often time-consuming and expensive, so the payoff needs to be big. In my mind, the very considerable time and expense it would take to verify and continue to monitor incomes, family size, etc., would outweigh the benefits of having that data. But this is a point on which reasonable people can disagree.

Anonymous said...

Economically diverse schools?

Good luck getting the rich back into public schools.

At best, you will mix the poor with the middle-class.

And even then, only a portion of the middle-class.

Wiley Coyote said...

Pam,

I'm sure if you keep slighting the IRS they will find you.

Verification can be done at the state level.

As with education data, we can no longer afford not to have accurate immigration data, poverty data, etc.

There is too much at stake on both sides of these issues.

Anonymous said...

Since the credits were dropped froom 28 to 24, some students graduated early. It's doable to complete high school in 3 years vs 4.

Anonymous said...

IT IS ONLY DUE TO THE EVALUATION and BONUS MONEY THE PRINCIPALS RECEIVE.

Principals will do and say anything to their teachers to NOT have a student fail. From changing grades to harassment of teachers. This has been seen, heard and displayed by various principals within CMS.

BofE and MOrrison encourage this because of one thing : RACE TO THE TOP FEDERAL FUNDING DOLLARS.

IT IS ALL A SHAM to teachers , students and taxpayers.

Anonymous said...

Is the goal to have all kids graduate? If so, lower the credits required to 4 or 5.
I feel the goal is to have as high a percentage as possible learn so they can be productive members of society for themselves and the community. Sometimes the goal of having all graduate can impede the greater majority from learning as much as they should.
For some kids leaving school early and getting their GED may be more beneficial (they could start work, go to junior college, etc). Not ideal, but what is the benefit of all who do graduate being perceived as having a watered down HS diploma.

Ann Doss Helms said...

Pamela, I'm chuckling at your "best I can tell" on Wake's credit requirement. You must have found the same policy I did, which says it's 20 or 26, with 22 for occupational course of study. WHAT? I called the Wake PIO late yesterday to get clarity and just now got an answer: It's 26 for most high schools, but varies at some magnets: http://www.wcpss.net/what-we-teach/curriculum/pdfs/2013-14hs_planning_guide_final2.pdf

Wiley Coyote said...

. <------ Photo of a grain of salt I saved from lunch to send to the CMS Data Department.

Shamash said...

Anon 2:41pm.

A HS diploma is simply a "credential".

In that way, it is much like a drivers license.

Few really care what it actually represents in terms of knowledge.

This attitude even exists in the work world.

As a "registered rep" at a huge bank responsible for selling stocks to the public, I was required to pass a Series 7 and 63 test.

I made a 92 on that test.

The managers said that anyone who made over a 70 on the test (the score necessary to pass it) probably wasted too much time studying the material.

Nevermind that we were all FORCED to attend classes to study the material (and couldn't take the test early).

But only 10% of the class passed the test the first time and much of the the class didn't even pass a few months later.

(And yet the managers said that those of us who did too well "wasted" OUR time...)

That is just one sign of how pervasive the bias against real knowledge and achievement is in our society.

So kids, don't worry.

No one outside academia will care one whit what you actually learned in school or what your diploma actually represents.

Just get that credential and move on.

Wiley Coyote said...

CMS mirrors the real world...

Last year, my wife and I bought a new Hyundai Elantra for her. Beautiful car, lots of stuff on it and my wife loves it.

Rated at 40MPG on the road, this would save us a lot on fuel costs.

In order to get that 40MPG, here's what Hyundai (and other car manufacturers as well) did to achieve the fuel rate.

First, there is no spare tire. Yes, you get a kit, basically a can of Fix-a-Flat and a pump. This cuts down on the weight of the car (there is a place for a spare if you decide to buy one).
----Like reducing the number of credits to graduate-----

Secondly, they cut the fuel tank capacity down to 12.8 GALLONS, which means you're not hauling around the weight of an extra 5 to 7 gallons of fuel in the tank.
----Like reducing the number of credits to graduate-----

But here's the real kicker; Hyundai overstated the MPG!
-----Like CMS has overstated data in the past-----

So to compensate those of us who bought one of these cars, Hyundai will pay us for every mile we drive for as long as we own the car in the form of a debit card. This comes out to about $100 per year.
----Like CMS giving a 50 just to put your name on an assignment----

Shamash said...

Wiley,

Well, at least SOMEONE in the real world held Hyundai responsible for their deception.

And banks do get sued for mal-practice of their under-achieving (and dishonest) "licensed" representatives.

But no one will ever get compensated for an over-inflated HS grade.

Even if their diploma proves rather worthless in the real world.

Can you imagine if EVERYONE got a debit card each year compensating them for their poor public education...



Wiley Coyote said...
This comment has been removed by the author.
Wiley Coyote said...

Shamash,

Over 77,000 of the 141,000 students already get free and reduced stuff.

Add to that multiple chances in school.

Awhile back, one middle school student was suspended 13 times because "he was angry"... still in school.

Adding up the days missed - unless being suspended is an excused absence - they would have to repeat the grade.

Maybe they did, who knows.....

CMS is strating to sound like a used car salesman:

Bring in ANY car; running, not running, drag it, tow it, drive it or anyway you can get it here and we'll give you $5,000 towards a new car.

Pleaaaaase come back to school. Pleeeaseee... We need you to walk across the stage. We'll give you a 50 and reduce the amount of credits plus other perks so you can "graduate".

Anonymous said...

So we give out 15-20% meaningless diplomas. How does that help the many who actually did the work? It just makes employers question the validity of all diplomas, hurting those who really earned it. It devalues all diplomas.

Anonymous said...

Affirmative action does the same thing. Which minorities you work with got the education and the job not because they were the best, but because of affirmative action. How would you like to be in one of the preferred groups, have gotten the job because you were best qualified but the rest of your career everyone assumes you got it at the expense of someone more qualified.

Anonymous said...

4:26,
They will in CMS. It's called free breakfast. It's also called virtually a nutritional throwaway. Check the volume

Jim said...

Here comes September and a new school year -- follow this story up with one on the % of entering college freshmen from CMS who require remedial work before they can take freshman level classes. Down? There's your proof.

Anonymous said...

It amazes me how uninformed some of the posters who claim to be teachers are.
Check the salary database - see where the money is going. Ask simple questions like why were 5 assistant superintendents added when we had no money for teachers? Use the critical thinking skills you are supposed to have learned in college. My guess is most have them - unfortunately many who are posting here show a lack. The problem in NC did not occur overnite - to put all the blame on a group that has been in office less than a year does not make sense. Let's get the state back on good financial ground after all the debt, etc than let's see what the priorities are. Just once I'd like to hear from a teacher that the Perdue education governor was a sham and really hurt.