Describe a rainbow in seven words to someone blind since birth.

This is the fundamental problem with PIRS or the current craze towards non-dashing dashboards (if the data change only once a year, it is only a dashboard for a glacier). Institutions are complex with  many things going on that simply don’t reduce to seven metrics, let alone four or five as in the White House College Scorecard. Spending yesterday and this morning at an NGA-hosted meeting on Higher Education Effectiveness and Efficiency Metrics Learning Lab reinforces my (probably curmudgeonly) belief that knowledge sometimes has to be earned through effort and study – not a 60 second review of a web-page or Powerpoint slide.

I want to believe in the power data to transform systems, to transform lives. I worry though that over-simplification of the presentation of performance data leads to under-recognition of the lives affected.

Speaking of over-simplification, I was part of an expert panel on Tuesday about PIRS and community colleges. Deputy Under-secretary Jamienne Studley was present. She was clear that PIRS is going forward based on existing data. Data that are completely inadequate to the task, in my considered opinion. However, she does seem open to some ideas that others in the Department  and the White House find anathema. I won’t share those at this time but I was kind of, umm, vocal in my suggestions. I know she heard me.

A timely example of over-simplification is this. “Starbuck’s Offers Free College to Employees.” Robert Kelchen provides a more in-depth understanding here. Matt Reed does a mea culpa from his original position and acknowledges  the efforts of others who read the fine print and went beyond the metric of “Free College.”

While I am not sure that either of these books were covered in Reading Rainbow, if one compares two of my favorites, The Sun Also Rises and The Stand, one can easily see a difference in the prose styles. Hemingway is much tighter and sparse than King (I suspect) ever dreamt of being. Despite that, neither can be meaningfully reduced to seven words or other metric. Any critical rating is meaningless to people that eschew one genre over another. And books are static. They don’t change over time. Our interpretations may change, their placements in a rating system may change, but the books themselves don’t change (except when King added 100-plus previously cut pages to a revised edition).

Institutions change. Measurement can cause change in institutions. Bad measurement, bad incentive structures are likely to cause bad changes. Let’s really be clear what we are doing and why, while recognizing that not everything can be as simple as we might like.