Saturday, March 8, 2014

CMS high school teachers hosed on ratings?

The irony was obvious last spring:  Charlotte-Mecklenburg Schools didn't trust the state's new final exams enough to count them toward high school students' grades.  But those tests were created to rate teachers,  so teachers not only had to give the exams but spend hours grading new open-ended questions.

Now the results of the teacher effectiveness ratings are in,  and they indicate something went awry in rating CMS high school teachers.

Across the state and in CMS,  more than three-quarters of all teachers met or exceeded the goal for student gains.  But when I broke that out by grade level,  more than 80 percent of teachers in CMS elementary,  middle and K-8 schools met or exceeded the goal,  compared with just over 60 percent of CMS high school teachers.

Erlene Lyde at West Charlotte


I also ran the numbers for more than 12,700 non-CMS high school teachers around the state,  and 78 percent of them met or exceeded the target.

It's possible that these numbers reveal a real shortcoming unique to CMS high school teachers.  But a handful of  teachers and principals I spoke with questioned the results on two grounds:  The validity of the tests and the fact that CMS teens knew they had no stake in scoring high.  Erlene Lyde, a West Charlotte High teacher and vice president of the Charlotte-Mecklenburg Association of Educators,  put it most bluntly:  "Flawed data generated from flawed tests administered in flawed conditions and graded using a flawed scoring mechanism."

Thus, the perpetual challenge:  I think it's important to analyze and report on education data.  But at the same time,  you have to question what the numbers really mean.

I'm still not sure how well the EVAAS formulas from SAS Institute turn student test scores into meaningful measures of school growth and teacher value. But the EVAAS site for looking up school growth ratings is one of the best public data presentations I've seen.  It's a simple matter to look up schools and make comparisons in a number of different ways  (fellow geeks, check out the scatterplot option under comparison reports).

School growth and teacher effectiveness are both based on students'  year-to-year progress on state exams.  As you'd expect,  schools that score well on one measure are likely to look good on the other.  But they're not identical for a number of reasons.  One of them is that school growth is based only on End-of-Grade and End-of-Course exams,  while the teacher ratings include more tests.

Some of you asked excellent detail questions when those ratings first came out. I asked Jennifer Preston of the N.C. Department of Public Instruction to clarify the tests used and the way student results were assigned to teachers.  Here's her report,  for those who are interested in diving deep:

  • The Department of Public Instruction and the SAS Institute were able to provide teacher-level value-added data for a pretty expansive list of grades/subjects and courses.  They are: Reading/ELA in Grades 4 – 8, Mathematics in Grades 4 – 8, Science in Grades 5 – 8, Social Studies in Grades 5 – 8, Biology, Earth/Environmental Science, Chemistry, Physics, English I, English II, English III, Algebra I/Math I, Geometry, Algebra II/Integrated Math III, World History, Civics and Economics, United States History, American History I, and American History II.  These estimates are all based on the administration of End-of-Grade assessments, End-of-Course assessments, and NC Final Exams.  North Carolina has also had a well-established Career and Technical Education assessment program for many years; teachers of more than twenty-five Career and Technical Education courses received individual value-added scores.  
  • In order to ensure that all value-added estimates are fair and valid, we do have some safeguards in place around minimum student counts.  For End-of-Grade Assessments in Science, End-of-Course Assessments. NC Final Exams, and the CTE State Assessments, teacher must be connected at least ten students and the equivalent of six "full students," defined as students with 100% instructional responsibility claimed by one teacher.  This point is most easily explained with examples.  Let's say that an Exceptional Children's teacher has claimed 20 students at 10% instructional responsibility for each one.  While the teacher is connected to ten students, he is only connected to the equivalent of two "full students" (20 students X 10% each = 2 full students). The teacher will not have a value-added score because he is connected to fewer than six "full students."  A different Exceptional Children's teacher has claimed 20 students at 50% instructional responsibility for each one.  This teacher is connected to at least ten students, and is connected to the equivalent of 10 "full students."  He will have a value-added score.  Each of the students must have at least three prior test scores (in any grade/subject or course) in order to be used in the analysis.  For End-of-Grade Assessments in English Language Arts and Mathematics (Grades 4 – 8), a teacher must be connected to six "full students," using the same terminology as described above.  These business rules are to ensure the quality of the value-added data – if a value-added estimate is calculated using a very small number of students, it's simply not valid.  While a bit complicated, these rules simply reflect the reality of teaching today – there are lots of cases in which teachers share instructional responsibility for students and work as a team to provide them with the services they need.


40 comments:

Anonymous said...

Once again Helms falls short on her analysis of the multitude of factors that hamper student success on the limited amount of factors that go into "rating" a teacher. To what extent are some teachers assigned with students who do not even possess the capacity to read the exams at the grade level which they are written? Helms proves once again that there are certain topics, schools, and CMS personnel that are "beyond reproach" by the only daily newspaper in the largest city in North Carolina.

Anonymous said...

As one of the teachers who got hosed, I will tell you that the NC tests were a farce! We "teachers" were the very ones who graded the tests! I was honest along with lots of other teachers and graded my students low, if they scored low and high if they scored high. I tended to grade them low because I believe in maintaining high standards. I got penalized for my belief in standards. This state is absolutely decimating its teaching ranks. I have never seen so many young teachers and teachers with experience quit their jobs. Before it is over with, NC will have to pay bonuses to anyone willing to enter a public school. The algorithms used to determine the test scores are a joke, the tests are a joke, the scoring was a joke and North Carolina's approach to public education is the culmination of one big joke!

Wiley Coyote said...

But the test data has to be correct.

All of the school lunch data is correct.

Interesting how data is either right or wrong, depending on what you need it to be.

Anonymous said...

Wiley,

So I guess you are saying that teachers are responsible for school lunch data as well? Great rationale, let's job the teachers because of school lunch data.

Unknown said...

.
THIS IS NO WAY TO TREAT PEOPLE.

HISTORY
Last year three men spoke about this testing fiasco before the full School Board at its regular meeting. Everything Ms. Helms has written in this article they covered completely and passionately. Their case was beyond compelling. It demanded a response. And they got it.

Within seconds of the adjournment of the Board meeting, Dr. Morrison gathered the men around him and explained his hands were tied by the state. He explained he had no other options than make them grade the tests without pay. After that get-together I spoke with Dr. Morrison. He said that they needed to hear directly from him, right then.

PARALLEL
The only parallel I can see to the way NCDPI is running education is the terrible disregard of the O-ring problem in the Challenger disaster. Raleigh knows it has problems but sees the mission is more important than the people.

BACK TO THAT DR. MORRISON THING
How he treated those three men should be an example to Raleigh, NCDPI and the General Assembly about how important is the State’s relationship with teachers. Well, maybe with everyone.

The superintendent has faith in what is called Cultural Competency. There are many variations of it but it boils down to trying to understand people and put that to use for the betterment of everyone. None of us succeed at that all the time. But I’m of the feeling that NCDPI has chosen, in these hard economic times, to be less human than they could be. Shame!


Bolyn McClung
Pineville
.

Wiley Coyote said...

6:53

I didn't say teachers were responsible for either data.

Ann states "somethign went awry" and "you have to question what the numbers really mean".

My comment was sarcastic in that there is never the same outrage over flawed school lunch data, which drives much of the funding and other things - which these same teachers have to deal with.

Wiley Coyote said...

Bolyn,

What Morrison really meant to say was, "I'm not going to rock the boat and be a leader and demand change."

He's just going to say "his hands are tied."

The status quo will rule every time.



Shamash said...
This comment has been removed by the author.
Shamash said...



Anon 6:03pm

"I was honest along with lots of other teachers and graded my students low, if they scored low and high if they scored high."

Punishing teachers for having higher standards than other teachers sounds about right to me.

Let that be a lesson to them.

Lower that bar. Higher standards only hurt the ratings. Give all the kids an A and graduate them all for seat time.

Anonymous said...

If A student is truant for 20 or more days in a semester the teacher is still responsible for 100% of that students' score on the NC ready exam. How is that fair?

Anonymous said...

Teaching in NC is bad business. 2/3 of teachers are going another year without a raise. This testing crap. How do they plan for the future? They are loosing money every year with inflation. I would not want to be a financial planner for a teacher.

jon golden said...

Wiley is a one-trick pony. All he ever seems to write about is FRL numbers. Snooze....

Wiley Coyote said...

10:48...

yep...since many of the decisions and funding made in public education are predicated upon those FRL numbers.

The last data from CMS shows 54% of students in "poverty". The waste and fraud in my opinion is more egregious than the fraud in testing and the same testing used to determine teacher pay and competence.

I realize there are many people incapable of drawing the connection between bogus FRL numers and what teachers are asked to deal with in regards to teaching, accountability, testing and compensation.

Anonymous said...

Who cares anymore? Get out of NC or dye trying :)

Anonymous said...

Die:)

John said...

You don't have to leave the state, just the county. I had a much better experience with Cabarrus County schools! Just like my realtor told me to expect 20 years ago!

Pamela Grundy said...

Given that so many of us are such fools, Mr. Coyote, why do you waste so much time with us?

Anonymous said...

Wiley gets his panties in a wad when there is any discussion about the fraud in the FRL program, but doesn't say a damn thing about the fraud in corporate America, even when it happens under our noses. When is the last time Bank of America paid federal taxes?

Regarding the op-ed, teacher ratings tied to test scores are just plain stupid, especially when the high school students known the tests don't count. You are all scratching your heads like we're trying to solve Divinci's Code. Young kids will still try their best because that's what they've been told to do. Doesn't work that way with high school kids. If they know it won't count, it becomes an exercise in bubble test patterns (Christmas tree?). Perhaps this explains the drop-off.

Wiley Coyote said...

Pam,

To shine the light of hypocrisy on outrage of other issues that money is being wasted on that have roots in the bogus school lunch program.

When you have no clue as to who you need to be targeting with those extra dollars, all of the whining over teacher compensation, testing and accoutabilty becomes a hollow argument.

Until that is fixed, public education will still be going down the same road it's been on for years.

Pam, you whine your way and I'll whine mine.

I just do it without carrying a big pencil.

Wiley Coyote said...

9:08

Because whether corporations pay taxes or not is not the subject here and has no bearing on whether little Johnny can read.

For the record, I totally agree with you that corporations should be paying their "fair share" of taxes.

What's ironic though is that Obama used that term ("fair share")hundreds of times trying to get re-elected, yet those same corporations helped fund his re-election.

They still aren't paying a dime.

Anonymous said...

10:44

You are correct !

How about 5 years of LOSING MONEY !!!

Loss of Dental Benefits
Loss of Vision Benefits
Loss of 80/20 Health Benefits
Loss of COLA Salary Adjustment
Loss of Pension

Add 2 to 3% Inflation Cost Loss =

THOUSANDS OF LOST SALARY AND BENEFITS

All the while Teachers in most areas of Charlotte (non-Westside) have the workload of at least 1.5 teachers. Doing more for THOUSANDS less.

Why do you wonder that CMeS spends TENS OF THOUSANDS going out of this state to try and attract the "Best and Brightest" ?

PLEASE

Anonymous said...

The MSL tests are a joke. Most students finished in about an hour, tests are scored by teachers, and results are curved. Proctors were not even required. MSLs are in no way comparable to a three-hour EOG math or reading test. Maybe that's why most of the teachers who administered MSL's exceeded growth expectations.

Anonymous said...

7:00 AM What BS. What school in CMS? How can anybody teach with a GED? Janitor dont count. Lies.

1) End the mandatory public school requirement. Freedom of choice for all. If you want to go to school then its a choice but not a requirement.

We support freedom of choice for abortion so why not attending school? Legalize drugs with 50% taxation to abolish the IRS and end drug dealers plus paying the 47% moocher welfare bums. Regulation is the key with a 21 age limit like booze. Legalize opposite sex only prostitution and casino gambling. Regulate it.


2) Reverse the law on child labor and regulate the industry espec with the 3rd worlding of Amerika as this way 3rd world nations who make everything now can be inside US borders. Regulation is the key. Why go 12k miles to Asia and pay tariffs for exporting. China is notorious where 1000s of corps relocated to get cheap labor for 1 and allow cheap prices like at Kohls, Macys, Walmart, Target,
etc.


Doing all this will allow IRS to be abolished and save trillions in taxes.
It will also free up problems with troublemakers in the public schools for teachers.

Stop blaming teachers instead of rogue students. Freedom of choice will get Amerika back to prosperity and success. 50% of students today are not fit or cut out to be in public schools anyway.

X Real Degreed Certified Teacher.


ps Hypocrites need to clean out their homes and closets of cheap foreign made property, clothing,etc and stop shopping at hese stores.
It would be impossible to do this anyway. Foreign made goods are in the fabric of Amerika in the 3 lower, middle and upper classes.

Stop being a hippo, hippo ...

Anonymous said...

Mantra of the Independent,
Vote the other way next time. Against Tillis in November and Mcory when his times up. I thought Mcory would bring practical solutions to help our teachers. 6 years without a pay plan. Stuck in some weird limbo. Health care cost was not frozen, inflation stops for was no man. Teachers all over the country are on pay plans. Many with back pay or pay increase for the lean years. Our teachers are given nothing. 2/3 of our most experienced teachers will spend another year "frozen". A 6% pay increase still puts them behind. I am for the end of tenure but was this summer really the best time? They have already admitted to the blunder over masters pay. This testing debacle, is just another slap in the face. I am for charters but not for the abandonment of the other. The rotating teachers are hard on our schools and children. We love our children's teachers. 2 out of 3 will not be back next year. One has already left. How can Mcory talk about the importance of teachers and do nothing to help them. He says that we have to much testing but nothing has been done. Tillis was smiling when teacher tenure was taken away. That's not professional. That's not how you lead. My children's teachers work hard. They truly care for our kids. This whole thing is ridiculous.

Anonymous said...

HOW ABOUT YOU LOOK INTO THE STATE BOE's recent vote? Guess what they just CHANGED the rules AGAIN and it's MARCH. NOW there are 5 achievement levels and it's MARCH and the new rules takes effect for THIS YEAR even though some tests were given at MID-TERM. We just got the memo this weekend. And did I mention, it's MARCH?

So, how much validity do these tests REALLY have? It's a CLUSTER of EPIC PROPORTIONS. It's an EMBARASSMENT. It's DISGUSTING. And the fact that MY proficiency is measured by this MOVING scale is a JOKE.

Anonymous said...

Another slap...

Anonymous said...

Being a teacher in NC is bad business. Why is this state so bad at this?

Anonymous said...

Poor leadership.. Excuses, blame the last guy and on and on and on
.........

Anonymous said...

Well...at least the School Board is not SUING the County like Union.

I feel bad about what's happening to the Parents and students.

If you live and PAY GOOD MONEY to live in an area. YOUR child should be in that school, NOT the 8-10 away from your home.

COUNTRYBOY JJ...

Anonymous said...

NC at it's finest.. What a mess, but I have heard Union county is a much better place to teach.

Anonymous said...

When will the "TEACHER OF THE YEAR"
ever leave and go work at BURGER KING?

It is a disservice to his family and to himself to continue to work as a teacher in CMeS.

Moving the deck chairs around and now even moving the TOWER.

Anonymous said...

Teachers teachers. Com is a great place to start. You can create a resume and get it out to different state recruiters all over the country. Many states are preparing for the retirement of baby boomers. They are looking for certified teachers in all areas. Some states offer bonuses for Special Education, Math and Science teachers. It is a great resource. The recruiters help you with licensing and relocation. Many states have reciprocity with NC. This site has been a blessing. Good luck NC teachers.

Jeff Wise said...

While most of you seem distracted by the usual litany of comment content, did anyone actually read through the 2 bullet points Ann provided from DPI?

That 2nd point is a doozy. And yet, as usual, there's not a single word in there about how SAS or anyone else controls for external factors.

What about elective teachers that incorporate math, English and history concepts in their lessons, how is that controlled for? Don't see anything in the formula that even attempts that.

Which in turn underscores the point that not a single VAM formula, to my knowledge, has withstood scrutiny to be considered absolutely valid by any noted researcher.

Everyone is still grasping at straws trying to find the magic bullet that ties test scores to teacher ability.

The bottom line is no computer can yet mimic how the brain works and likewise there is formula that currently exists that can successfully and consistently correlate specific student learning to a specific teacher.

Anonymous said...

Well said, now what do we do? complain, find another career, leave the state.

Anonymous said...

" Each of the students must have at least three prior test scores (in any grade/subject or course) in order to be used in the analysis."

So, if the students must have three prior test scores....how can elementary teachers have scores?

Confused?

Anonymous said...

How are the different test corolatred? Civics is not the same as American history. Algebra is not the same as geometry. How much did this formula cost? Why don't we give students a test in the beginning and end of year. Check for growth and move on. If a student misses x days, they do not count. Mabey I am missing something.

Anonymous said...

One of the biggest factors that hampers a high school student's success is the 7:15am start time. no joke. It's kind of hard to teach a roomful of sleeping zombies.

Anonymous said...

Teachers are overrated, technology is the way to go (at least in CMS).

Anonymous said...

Well, here's a thought that I've never heard one person mention... but I'll mention it... In high school, the students have to earn 28 credits to graduate. This means, technically, they can fail 4 courses and still meet state requirements. Additionally, there are only certain courses students HAVE to pass in order to graduate (they keep modifying the requirement, but basically all 4 years of English, Algebra, and probably a science and/or history course - again, they keep changing the requirements. A few years ago they were known as "the big 5" in that you had to pass these courses to graduate).

So, there are those on the outside (read: non-teachers) who want to tie test scores to teacher pay. So my question is...what about the teachers who teach a "non-essential" course for graduation (the "electives," if you will)? There are percentages of students in every high school who have the mindset that if it is not "required," then it's not important. So now you are going to base a teacher's pay on courses with students who have no carrot dangling over their heads. BAD IDEA.

Anonymous said...

I am sure you will find those who would disagree, however, I have heard very good things about the Cabarrus County school district. My children attend Kannpolis City Schools ( A.l. Brown starts at 8:40), and we have been very pleased, KCS has exceeded our expectations. I am sure there are many parents who live in Mooresville who would say the same thing.