• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Advertising
  • Write for Us
  • Job Board
  • Privacy Policy
  • Contact
    • Consulting
    • Advertising
  • Shop
    • Books
    • Shirts

The Educators Room logo

  • Start Here
    • Impact Statements: Teacher Expertise
    • Newsletter
  • Browse Topics
    • Content Strategies
      • Literacy
      • Mathematics
      • Social Studies
      • Educational Technology
      • ELL & ESOL
      • Fine Arts
      • Special Education
      • Popular Topics
        • Teacher Self-Care
        • Instructional Coach Files
        • Common Core
        • The Traveling Teacher
        • The Unemployed Teacher
        • The New Teacher Chronicles
        • Book Review
        • Grade Levels
          • Elementary (K-5)
          • Middle (6-8)
          • Adult
          • New Teacher Bootcamp
          • Hot Button Topics
            • Menu Item
              • Principals' Corner
              • Charter Schools
              • Confessions of a Teacher
              • Interviews
              • The State of Education
              • Stellar Educator of the Week
            • Menu
              • How to Fix Education
              • Featured
              • Ask a Teacher
              • Teacher Branding
              • Current Events
  • Podcasts
  • Courses
    • Practicing Self-Care to Avoid Teacher Burnout- An 8 Week Course
    • Becoming An Educational Consultant
    • Teacher Branding 101:Teachers are The Experts
    • The Learning Academy
    • Books
    • Shirts
  • Education in Atlanta
  • Teacher Self-Care
  • The Coach's Academy
menu icon
go to homepage
subscribe
search icon
Homepage link
  • Advertising
  • Write for Us
  • Job Board
  • Privacy Policy
  • Contact
    • Consulting
    • Advertising
  • Shop
    • Books
    • Shirts
×

January 14, 2014 Featured

Disproportionate Evaluative Rigor and The Three Laws of Data

  • About the Author
  • Latest Posts

About John Kuhn

John Kuhn is a public school administrator in Texas and a vocal advocate for public education. His ''Alamo Letter'' and YouTube videos of his 2011 speech at a Save Texas Schools rally went viral, as did his 2012 essay ''The Exhaustion of the American Teacher.'' He has written two education-related books, 2013's Test-and-Punish (Park Place Publications) and 2014's Fear and Learning in America (Teachers College Press).
  • Mismatched: Your Brain Under Stress is a Must-Watch Documentary for Educators - May 7, 2021
  • The Experiential Illiterates - February 13, 2014
  • Fordham and Hess Temporarily Acknowledge that Reformers Can't Have it Both Ways - January 23, 2014
  • Disproportionate Evaluative Rigor and The Three Laws of Data - January 14, 2014
  • Teaching: The Card Game - January 10, 2014
  • The Tyranny of the Datum - January 6, 2014
  • Ed Reform's Atari Problem - January 4, 2014
  • Five New Years Resolutions for Public Education Supporters - December 31, 2013
  • The Wizards of Ed- The Conundrum of Education - December 30, 2013
  • The Exhaustion of the American Teacher - December 26, 2013

[fusion_builder_container hundred_percent="yes" overflow="visible"][fusion_builder_row][fusion_builder_column type="1_1" background_position="left top" background_color="" border_size="" border_color="" border_style="solid" spacing="yes" background_image="" background_repeat="no-repeat" padding="" margin_top="0px" margin_bottom="0px" class="" id="" animation_type="" animation_speed="0.3" animation_direction="left" hide_on_mobile="no" center_content="no" min_height="none"]

courtesy educationnext.org

courtesy educationnext.org

I promised in a recent post called The Tyranny of the Datum to write about some guiding standards for appropriate data usage, in the spirit of Isaac Asimov's Three Laws of Robotics, and I will do that here today. Before I get to that, though, I'd like to briefly discuss--in a general sense--what I see as a fairly blatant disproportionality in evaluative rigor as applied to various facets of K-12 education.

A frequent complaint about the educational "status quo" has been that there is too little evaluative rigor. This complaint has been raised by those who note that high percentages of teachers are rated highly proficient by their appraisers even when comparatively lower percentages of students pass their required state tests. The suggestion is that the evaluation mechanism is broken and isn't therefore adequately identifying strong versus weak teachers. The blame for this is generally placed at the feet of teacher unions, which are blamed by reformers on the left and right for protecting bad teachers and, as a consequence, compromising the teaching profession and harming students. The proposed remedy is increased evaluative rigor in the form of appraisals tied--in some percentage or another--to student test scores. This cause has been embraced by the federal department of education, and such appraisal requirements are embedded in waivers from No Child Left Behind. My state of Texas--a recent recipient of a federal NCLB waiver--will begin piloting a new teacher appraisal system that ties student test scores to their teachers' job performance appraisals soon.

Another area where we hear calls for more rigorous evaluative protocols is in teacher preparation. Many of the people who have tasked themselves with improving the American K-12 public education system (deemed failing as a result of its students' performance on international standardized tests primarily, and, secondarily, due to the high percentage of graduates requiring remediation when they enter post-secondary institutions) have identified teacher training as an area of weakness. Individuals and organizations have devised sometimes controversial rubrics for judging these programs and have released their results to the public. Others have suggested tying student test scores not only to their teachers but also to the teacher prep programs that trained their teachers. This is, to steal a quote from my book Test-and-Punish, like the Six Degrees of Kevin Bacon game, except that it's Six Degrees of Student Test Scores.

These are only two examples of the increased evaluative rigor that has become, ahem, de rigueur. I could also point to increased fiscal scrutiny facing public schools--in Texas, for example, we have the half-decade-old FAST reporting system from the Comptroller's office. It rates schools based on a comparison of their spending to their test scores. (The test scores appear to be the linchpin for all comparisons and quality conclusions.)

I'm not complaining about evaluative rigor. As a person who values public education as our nation's greatest equalizer--the most effective and broadly-implemented social structure we have that mixes our classes and races and let's them not only get to know one another but also, in theory at least, let's them struggle together against a common foe of ignorance, as well as against one another for preeminence in the classroom--I want public education to be great. I wish the system that I've given my adult life to were above reproach, but it isn't.  Adequate evaluation is vital. I have concerns that we are over-extending and misusing student test scores fairly pathologically, but that doesn't mean that I oppose improvements to our procedures for judging schools, administrators, teachers, and teacher prep programs. Certainly improvements are possible and necessary.

My concern, then, isn't that I think we should evaluate less in K-12 ed. My concern is that many prominent voices appear to be extremely disproportionate in their calls for evaluative rigor. A few brief examples are in order.

1. Louisiana officials have resisted subjecting voucher schools to the same test-based accountability systems that traditional schools in that state are held to. The rationale for this distinction was that the free market would be the accountability, that parents would choose to send their kids to the better schools and the worse schools would close for lack of business. But, of course, in a system of choice, the public schools are also a choice, so you would think the free-market-in-lieu-of-test-based-accountability argument would extend to those schools as well. It did not. Rigorous accountability for purposes of evaluation remained for the traditional public schools. The great wrong here is not the accountability system (although it most likely had its flaws). The big problem was disproportionate evaluative rigor.

2. Charter school application processes have in many states been rubber stamp parties. Ohio--where the school choice juggernaut known as the Fordham Foundation resides--has some of the nation's most lax charter authorizing practices, and also some of the nation's most underperforming charter schools. In other states--my own included--it has come to light that many charter school applications are literally copied from other charter schools' applications. This floors me: schools that are introducing themselves to the public as potential academic caretakers of our children commit the capital academic offense of copying right out of the gate,  on their very first assignment. And yet, for all the hue and cry over a lack of rigorous evaluation in our teacher prep programs, there is very little noise when it comes to evaluative rigor in charter authorizing. The disproportionality is the thing.

3. Charter policing is also often lax. This is perhaps to be expected, as charters were initially devised as vehicles for innovation (as opposed to profitability, which today sometimes takes precedence), and were therefore freed from many of the regulations that apply to traditional schools. However, the deregulation that was intended to create instructional advantages for children has been repurposed to provide competitive advantages for businesspeople and investors. As a result, the number of charter schools with major financial and academic performance problems that actually get closed down is surprisingly small. Choice as an inherent good has overshadowed any calls for quality control in the school choice movement, while the same people advocating for school choice have been relentless in their quest for ever-higher bars that traditional schools must clear.

There are certainly many more examples of disproportionality in evaluative rigor as it is applied to aspects of traditional K-12 education versus aspects of reform K-12 priorities, but I don't have time to cover them all this morning.

Somewhat relatedly, there appear to me to be fairly low standards for determining how data should be used in education. Here are my suggested Three Laws of Data:

1. Data belongs to the human who generated it.

2. Humans should always be informed when their data is being collected and analyzed.

3. If data is being aggregated for the purposes of creating a value meant to reflect positively or negatively on a human, that human should have ongoing access to the aggregated data and the formula that will result in the final value, so that the human has an informed opportunity to rectify any performance-related issues that might compromise his or her final score.

I'm sure there are other considerations. This is my first swipe at data rules. I'm also sure there are complex documents on this topic from the AERA and the testing companies, but what I'm advocating is a brief and easy-to-understand set of ironclad rules for regular people to be able to cling to in our data-informed future. Also, I'm not strictly thinking about educational data. I'm also thinking about credit scores, Google advertising data collection, and so forth.

Whether we speak of traditional education's totems or the icons of nouveau reform, the rigor of our evaluations should be the same. We should be particularly mindful to carefully vet standards for our use of data, given how central data will be (and already is) in this field.

 [/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]

Print Friendly, PDF & Email

Related posts:

Default ThumbnailData-Driven Politics Default ThumbnailThe Bandaid of Efficiency Default ThumbnailCharters: They're not all KIPP Default ThumbnailZero Tolerance For Zero Tolerance
« Crazy New Math
The Dog Won’t Eat Your Homework Now: Rethinking Homework in Elementary Classrooms »

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

The Educator's Room was launched in 2012 to amplify the voice of educators. To date, we have over 45+ writers from around the world and boast over twelve million page views. Through articles, events, and social media we will advocate for honest dialogue with teachers about how to improve public education. This mission is especially important when reporting on education in our community; therefore, we commit our readers to integrity, accuracy, and independence in education reporting. To join our mailing list, click here.

What we do

At The Educator's Room, we focus on amplifying and honoring the voice of educators as experts in education. To date, we have over 40 staff writers/teachers from around the world.

Popular Posts

  • My Union Showed Up for Me, and I'll Never Forget It
  • Your Students Deserve a Diverse Classroom Library. Here's How to Set It Up.
  • You Don't Have to Watch the Tyre Nichols Video, But Be Ready to Talk About It
  • "Let's Make This Happen": Following Student Interests to Interest-Based Mentorships

Featured On

Buy Our Books/Courses

How to Leave Your Job in Education

Practicing Self-Care to Avoid Teacher Burnout

Using Your Teacher Expertise to Become an Educational Consultant

Check out our books on teaching and learning!

The Learning Academy

Footer

↑ back to top

About

  • Privacy Policy
  • Disclaimer
  • Terms & Conditions
  • Accessibility Policy

Newsletter

  • Sign Up! for emails and updates

Contact

  • Contact
  • Services
  • Media Kit
  • FAQ

 

Copyright © 2021 The Educator's Room.