Measure of Effective Teaching
Have you walked on over to the Bill and Melinda Gates sponsored Measure of Effective Teaching Project? This is where Gates tries to answer the question: "How can effective teaching be identified and developed?"
Some would argue that this should have been the first and main thrust of his education efforts. For without the answer to the first half of this question school reform is doomed to failure. Others might say that the answer is and always will be "it depends".
Some of my highlights and comments about the MET Project Preliminary Findings Policy Brief.
Our goal is to help build fair and reliable systems for teacher observation and feedback to help teachers improve and administrators make better personnel decisions.
For this report, we have studied student achievement gains on the state test and the supplemental tests in grades 4 through 8 for five MET districts. (The comment I have is how effective are gains in standardized tests in measuring teacher effectiveness? Not good as far as I remember)
we measure student achievement gains using two different tests in each subject, the state standardized test and an additional, more cognitively demanding test (It is nice to know they are using more than one measure of improvement)
we anticipate expanding these outcomes beyond traditional tests to include noncognitive measures (When?)
Each student’s performance at the end of the year is then compared to that of similar students elsewhere (just when you thought it was straight value added measurements they throw in a curve, but is it a true measure of teacher effectiveness)
a teacher’s past success in raising student achievement on state tests (that is, his or her value-added) is one of the strongest predictors of his or her ability to do so again (Except that we are using value added measurement to measure the ability to add value so of course this is consistent)
the teachers with the highest value-added scores on state tests also tend to help students understand math concepts or demonstrate reading comprehension through writing.
In many classrooms students reported that “We spend a lot of time in this class practicing for the state test,” or “Getting ready for the state test takes a lot of time in our class.” However, the teachers in such classrooms rarely show the highest value-added on state tests. On the contrary, the type of teaching that leads to gains on the state tests corresponds with better performance on cognitively challenging tasks and tasks that require deeper conceptual understanding, such as writing. (Shouldn't this suggest that we need to put more emphasis on teaching higher level thinking skills and less on classroom management?)
students report positive classroom experiences, those classrooms tend to achieve greater learning gains
valid feedback need not be limited to test scores (for students and teachers I think)
First we sorted teachers based on student perception surveys and value-added on the state math assessment. (The question I was looking for but didn't see was something to the effect of: My teacher refused to give me the answer but made me figure it out for myself?)
The difference in learning associated with being assigned a top quartile teacher rather than a bottom quartile teacher was more than seven months— roughly two-thirds of a school year! (This whole notion of putting student learning into grade level broken down by month is really a poor measurement of education. It losses effectiveness with age. Also when we ask students to identify and use specific skills they seem to be less knowledgeable than if we just ask students to solve problems, but that's just my opinion.)
[project time line for Winter 2011] Preparing systems for multiple measures of teacher evaluation: using digital video, training observers, and meeting data requirements. (I have found using video to observe is wholly ineffective. You can't switch focus from teacher, to student, to board, to whole class as fast or often enough. I also can't zero in on a conversation or student when I want to. Finally, how do you ask the students for their feedback?)
CONCLUSIONS
Reinventing the way we develop and evaluate teachers will require a thorough culture change in our schools. No longer should teachers expect to close the door to their classrooms and “go it alone.” (Good teachers will agree with this and have been pushing for collaboration for a long time)
retraining those who do classroom observations to provide more meaningful feedback
we need to be humble about what we know and do not know
In the end I don't think this commission is doing the right research to answer the first half of their original question, "How can effective teaching be identified?" The assumption is that effective teaching can be identified by some sort of value-added measure with one or two standard tests. However, the question is how to you identify effective teaching so we should start with: Is this value-added idea working? How do we effectively observe teacher effectiveness in the classroom? What is an effective measure of student growth? The answers to these questions are being assumed and they shouldn't be.
Some would argue that this should have been the first and main thrust of his education efforts. For without the answer to the first half of this question school reform is doomed to failure. Others might say that the answer is and always will be "it depends".
Some of my highlights and comments about the MET Project Preliminary Findings Policy Brief.
Our goal is to help build fair and reliable systems for teacher observation and feedback to help teachers improve and administrators make better personnel decisions.
 |
| From the MET Project Preliminary Findings Policy Brief |
For this report, we have studied student achievement gains on the state test and the supplemental tests in grades 4 through 8 for five MET districts. (The comment I have is how effective are gains in standardized tests in measuring teacher effectiveness? Not good as far as I remember)
we measure student achievement gains using two different tests in each subject, the state standardized test and an additional, more cognitively demanding test (It is nice to know they are using more than one measure of improvement)
we anticipate expanding these outcomes beyond traditional tests to include noncognitive measures (When?)
Each student’s performance at the end of the year is then compared to that of similar students elsewhere (just when you thought it was straight value added measurements they throw in a curve, but is it a true measure of teacher effectiveness)
a teacher’s past success in raising student achievement on state tests (that is, his or her value-added) is one of the strongest predictors of his or her ability to do so again (Except that we are using value added measurement to measure the ability to add value so of course this is consistent)
the teachers with the highest value-added scores on state tests also tend to help students understand math concepts or demonstrate reading comprehension through writing.
In many classrooms students reported that “We spend a lot of time in this class practicing for the state test,” or “Getting ready for the state test takes a lot of time in our class.” However, the teachers in such classrooms rarely show the highest value-added on state tests. On the contrary, the type of teaching that leads to gains on the state tests corresponds with better performance on cognitively challenging tasks and tasks that require deeper conceptual understanding, such as writing. (Shouldn't this suggest that we need to put more emphasis on teaching higher level thinking skills and less on classroom management?)
students report positive classroom experiences, those classrooms tend to achieve greater learning gains
valid feedback need not be limited to test scores (for students and teachers I think)
First we sorted teachers based on student perception surveys and value-added on the state math assessment. (The question I was looking for but didn't see was something to the effect of: My teacher refused to give me the answer but made me figure it out for myself?)
The difference in learning associated with being assigned a top quartile teacher rather than a bottom quartile teacher was more than seven months— roughly two-thirds of a school year! (This whole notion of putting student learning into grade level broken down by month is really a poor measurement of education. It losses effectiveness with age. Also when we ask students to identify and use specific skills they seem to be less knowledgeable than if we just ask students to solve problems, but that's just my opinion.)
[project time line for Winter 2011] Preparing systems for multiple measures of teacher evaluation: using digital video, training observers, and meeting data requirements. (I have found using video to observe is wholly ineffective. You can't switch focus from teacher, to student, to board, to whole class as fast or often enough. I also can't zero in on a conversation or student when I want to. Finally, how do you ask the students for their feedback?)
CONCLUSIONS
Reinventing the way we develop and evaluate teachers will require a thorough culture change in our schools. No longer should teachers expect to close the door to their classrooms and “go it alone.” (Good teachers will agree with this and have been pushing for collaboration for a long time)
retraining those who do classroom observations to provide more meaningful feedback
we need to be humble about what we know and do not know
In the end I don't think this commission is doing the right research to answer the first half of their original question, "How can effective teaching be identified?" The assumption is that effective teaching can be identified by some sort of value-added measure with one or two standard tests. However, the question is how to you identify effective teaching so we should start with: Is this value-added idea working? How do we effectively observe teacher effectiveness in the classroom? What is an effective measure of student growth? The answers to these questions are being assumed and they shouldn't be.
Comments