Archive Hosted by the AFL-CIO

The Data We Deserve

January 04, 2012 By Michael J. Diedrich, Policy Associate

As Minnesota waits to hear if Washington will give it a waiver from the failed No Child Left Behind initiative, and while congressional conservatives vie with the President over what will replace NCLB, one thing is certain: we are going to keep talking about test results. As these discussions—laced with terms like “accountability” and “data-driven instruction”—go on, it behooves us to consider how we're using the various assessments available to us.

In particular, we need to figure out what, exactly, it is we want all this data for. Today we'll look at a few different reasons for collecting data, as well as the right kind of assessment to collect that data.

Reason 1: Knowing How Our Education System Is Working

What We Want to Do:
It makes sense to have a way of showing the general public what their tax dollars are producing by way of educational results. As such, it makes sense to gather data on the overall system. The tricky part is getting that sense of performance without corrupting the measurements we're using to gauge it.

The Data We Need:
The Minnesota Comprehensive Assessments (MCAs) could partially serve this purpose if they weren't linked to school-specific consequences. Since those consequences have been established, however, the MCAs now measure how well schools do at trying to beat the test.

More helpful is the National Assessment of Educational Progress (NAEP), which is strictly about data collection, not consequences. Even better would be multiple assessments using different mechanisms, as there are significant limitations on data from multiple choice exams.

Reason 2: Identifying Schools in Need of Improvement

What We Want to Do:
Schools that aren't getting the job done ought to be identified for appropriate support in improving. Our problem is that we've confused a couple of tests with “getting the job done,” when the real job of schools is preparing students for long-term success.

The Data We Need:
Unfortunately we don't have a lot of good systems for gathering this data right now. We can look to some degree at graduation or college admission rates, but even these are scattershot in their ability to indicate long-term student success.

Ideally, we'd be able to look at long-term outcomes for students: college completion (for both 2-year and 4-year degrees), employment status, etc. Once such data was gathered, we would still need to consider broad trends from schools and look at interactions between elementary, middle, and high schools as well as with students' initial socioeconomic status to get a full picture of a school's impact on its students.

This is much more ambitious and complex than most people want to take on, so we're left with a dependence on multiple choice tests of math and reading, with token attention given to writing and science. The effects of this speak for themselves.

Reason 3: Improving the Quality of Teachers/Instruction

What We Want to Do:
One of the chief obsessions of education reform efforts these days is improving teacher quality. Research has shown over and over again that teacher quality is the most important in-school factor affecting student performance (though socioeconomic status and other non-school factors add up to a larger overall effect). As a result of this focus, we've seen continued interest in so-called “performance pay” systems that use standardized tests (again) to determine teacher pay and retention. What we're really trying to get at, however, is making sure teachers are focused on student performance and using student data to get better.

The Data We Need:
This is an area where the government just plain can't do it. For data to actually be useful in helping teachers improve, it needs to be regular and tied to specific teacher actions. An average MCA score delivered to a teacher in August or September after the relevant school year is meaningless for this. It gives them no specifics on which particular techniques or lessons did the trick, and it's coming too late to do any good.

Strong in-class assessment happens regularly (multiple times a week if not every day) and is directly tied to what was taught. This allows teachers to figure out how well their instruction worked, as well as giving them a starting point for professional development conversations with peers or administrators. There is simply no way for a state-level assessment scheme to provide data with the nuance and precision necessary to guide teacher improvement; it must come from the teachers (and be accessible to administrators and instructional coaches).

So where does this leave us? Clearly, the assessments we have are inadequate for meeting all the different goals we have. What's worse, a lot of public policy is being made with little attention to the kind of data necessary to make it work. We're simply using the tests we have for everything we think we need, and the results are not good. It's time for a little more sophistication in our approach to data and reform.

Thanks for participating! Commenting on this conversation is now closed.

1 Comments:

  • Aaron Grimm says:

    January 9, 2012 at 9:55 am

    Michael,
    I appreciate your comments on the last post I made. I apologize for being a bit too blunt in my last post, but education is currently in a frustrating state. Ed reformers have lost much of their credibility because they haven’t really changed anything. We know that you cannot standardize curriculum to students by their age using a “one size fits all” approach, but we keep doing it.  People continue to use conventional thinking inside of the traditional education frame work of what learning is, which will get us nowhere.  I guess I implore MN2020 to look beyond “the way it has always been.” Looking at more progressive models like Montessori in elementary grades or individualized project based learning at the secondary level can help to begin to think outside the box. School assessments must include long term growth models, emotional health of the child (environmental/climate measures) and more of a focus on getting kids to find their passion. Developing kids who can build their own dreams, invent their own future jobs and contribute to their community is a much better scenario than dreaming of becoming rich working for corporate America. Working for the man is deeply entrenched in the traditional system, for both students and teachers. Thanks for the post and I know I still have lots to learn wink—Aaron