IPEDS is not GRS

Say it with me, “IPEDS is not GRS. GRS is a part, a small part, of IPEDS.”

Matt Reed (@DeanDad) set me off a bit this morning with his Confessions piece over at InsideHigherEd and I kind of piled on with my long-time colleague Vic Borden that IPEDS and GRS are not simply one and the same with a focus (or fetish, if you prefer)  on first-time, full-time undergraduates. It really ticks me off when I read something like this since it takes my mind of the very good points he was trying to make. I could have written thousands of words of comments about how what we are doing in Virginia is so different, and so much better.

Every time someone says he or she wouldn’t be counted in IPEDS because they transferred, or took eight years (like yours truly), I cringe. It is just not true. It is false. It is wrong.

Yes, that person would not be in the GRS metric. However, she certainly would show up in the Completions survey if shefinished a degree or certificate, whether it took one year or 20. Likewise, she would show up in the fall enrollment survey anytime she was enrolled in a fall term.

As important as a graduation rate is, there is not much more important than the degree conferral, the completions, themselves. That is something that folks should keep in mind.

Now I could brag about some of the things we are doing at research.schev.edu, but instead I will simply highlight this tweet from ACSFA PIRS Hearing:

I think Matt has the right ideas, and I would support them in a Technical Review Panel, although I would probably offer supportive amendments. The problem is getting to a TRP. The type of collection required to support these measures, if not Student Unit Record (or IPEDS-UR, although being a Thomas Covenant fan, I want to lean towards ur-IPEDS), would be so burdensome, the collection would never happen without Congressional action. And that’s the rub. USED only controls the details. Congress makes the ultimate determination and that is where AACC and ACCT (and probably a bunch of groups representing four-year colleges) need to get involved.

The easiest thing at this point is to pile on to support the Student Right-to-Know Before Go Act.

 

 

 

Accreditation is not what it used to be

Accreditation is not what it used to be, at least in terms as being recognized as a standard of academic quality. Last week, Belle S. Wheelan and Mark A. Elgart published an essay at InsideHigherEd arguing that we should “Say No to ‘Checklist’ Accountability” as an argument against the White House Scorecard, PIRS, and other initiatives to differentiate institutions based on performance. In a nutshell they argue that the peer-review process accomplishes so much more than any single measurement could, that we should trust accreditation.

In many ways I agree with their arguments. However, at this point, I am a bit tired of explaining of explaining to state legislators that we should trust SACS and other accreditors, regional and national, when the only time an institution is shut down is based on financial issues, not academic issues. If academic quality issues are involved, we never hear about them.

Attacked at times by policy makers as an irrelevant anachronism and by institutions as a series of bureaucratic hoops through which they must jump, the regional accreditors’ approach to quality control has rather become increasingly more cost-effective, transparent, and data- and outcomes-oriented.

If this is true, where exactly is the transparency? I look at the SACS Commission on Colleges website and I see no data tools about outcomes. Under “Accreditation Actions & Disclosure Statements,” I see only two documents. Apparently nothing happened prior to 2013. Where can I download or review the site visit reports?

I can’t.

All I can do is trust the accreditation works even though I have no evidence that a college has have ever lost standing for academic quality reasons. Colleges rarely share their site visit reports, if ever. In Virginia, we frequently have to beat back legislation that requires those be published or at least shared with the General Assembly. I’ve seen enough site visits and assisted in enough decennial self-study to recognize that there is a great deal of value in the process, but to suggest it is transparent or outcomes-oriented goes too far in my opinion.

I don’t think ratings or scorecards solve the problem of differentiating between institutions based on quality, but clearly accreditation currently does not, or else the proposals would not be out there. These things occur when a need is not being met. Legislators and policymakers want a better answer than “trust us.”  So do a lot of other people. The accreditors ultimately determine if an institution is eligible to participate in Title IV aid programs. If the accreditors do not become much more transparent and provide explicit data and information on institutional quality, these proposals will become more than proposals and will make accreditors ultimately irrelevant.

After all, if a single score, like a Cohort Default Rate, can end eligibility, accreditors are only relevant for initial participation. At that point maybe we don’t need them as a gatekeeper.

trainings and reflections

Kilimanjaro is a snow-covered mountain 19,710 feet high, and is said to be the highest mountain in Africa. Its western summit is called the Masai “Ngaje Ngai,” the House of God. Close to the western summit there is the dried and frozen carcass of a leopard. No one has explained what the leopard was seeking at that altitude. – Hemingway, “The Snows of Kilimanjaro.”

So, I spent the week in the first half of an executive leadership institute. (Yes, I realize I should not start with “So,” but, well, so what? If you have read any earlier entries, you will realize I am not exactly rigidly bound by convention.) I have been through a number of leadership exercises, but this is my first at the executive level. First, I will say that it was distinctly non-military in its approach and thus quite different from my earliest experiences.

I have a problem in that I am cynical, sarcastic, and curmudgeonly in a world that does not value these qualities nearly as highly as I do. This also causes me to not look forward to being the target of well-meaning people wishing to help me fit better into their worldview, particularly when I feel that I have a lot to do and a sense of lacking productivity at a given moment is not helpful for my sense of well-being. Despite this, I made the best of the week. Even going so far as tossing my cell phone under the table and retrieving it only during breaks. Further, I sacrificed evenings of meaningful productivity to stay in the moment, with a sense of pausing my life.

It was a good week. The individual sessions were helpful, even when some of the information was not new to me, but thinking through things in a different model is almost always helpful. Honestly, I could probably spend most of a week practicing the art of handling tough interviews. A useful skill and fun. It is always useful to practice public speaking as well. Mostly though, I spent the week in reflection about purposes, drive, vision, and shortcomings.

I’m looking forward to week two next month.

I’ve thought about leadership a lot over the last decades. I kind of really hate being able to say “decades” in this context, but it is true. It has been over 32 years since I joined the Army, and just about two years more since getting involved in ROTC and obtaining a full scholarship (that I tossed away when I dropped out and enlisted). One of the first things I have noticed, is that in higher ed, leadership seems to be little more than doing something first. That may seem unfair, but when I see things like the “adjustments” made by USED this week (I wrote about those here), I wonder about moral/ethical leadership in higher education. It wasn’t that long ago one could count on USED to be boringly ethical and consistent. Apparently not so much now.

One of the implicit lessons this week, and I mean it was so implicit that perhaps it only reflects my imagination or my own deep-hidden demons, is that the appearance of impropriety is every bit as bad as real impropriety for a public agency. When I reflect upon the sample scenarios we used for the “gotcha” interviews and the examples of social network messages that went badly wrong, it seems clear that there is very little difference between impropriety and the mere of appearance of such in terms of the damage that can be done to a public organization, its reputation, and the people involved. With large enough entities, this can create a ripple effect of across the industry. Higher education has enough problems, and the sub-industry of higher ed data is now just a little weaker.

I hate that.

So, I will pause a bit. Reflect some more. Move on to what I can affect. More importantly, I will keep to my principles and values and let those drive the work I do.

Like the leopard though, there is no explanation. We’ll see.

what if your forecast is wrong?

Let’s pretend you did some research and wrote a paper that got published. Say also that predicted a stunning shortfall of an expensively and highly educated population of employees. The paper was received well and embraced by most everyone. It became a driver for change.

A mere two or three years later you realized you were wrong. Badly wrong.

Shouldn’t you have told someone?

Read this story about the forecast shortfall of faculty jobs in the humanities. And then read anything by Rebecca Schumann here or at Slate.

I hope that I would have the courage and wherewithal to retract my original work early on. And the apologize. Profusely.

 

Prediction is not destiny, and joy is in the trip, not the destination

Really don’t mind if you sit this one out.
My words but a whisper your deafness a SHOUT.

I was part of a conversation this week with someone important. Someone who believes in the power of Big Data. Someone who believes in this more than I do. I believe in the power of data and the power of predictive analytics, but power is a two-edged sword that cuts as it is allowed. Power of this type is also dumb, it needs to be guided by intellect for good purpose. And an over-arching intent to first, and always, do no harm.

I was not the only one present in the meeting with this important person. I raised the question, “…while we are talking about training teachers on how to use these tools, will the training include knowing when to ignore the predictions?” I don’t now how my colleagues behind me reacted. I don’t really care.

I may make you feel but I can’t make you think.
Your sperm’s in the gutter your love’s in the sink.
So you ride yourselves over the fields and
you make all your animal deals and
your wise men don’t know how it feels to be thick as a brick.

I really want to live in a world where everyone has an equivalent opportunity to be successful. Not the same opportunity, but equivalent opportunity. Opportunity that recognizes disadvantages, provides an additional boost where needed, and allows for happy accidents.

And the sand-castle virtues are all swept away
in the tidal destruction the moral melee.
The elastic retreat rings the close of play
as the last wave uncovers the newfangled way.

If predictive analytics had been around years ago when I was young, I have no idea what they would have predicted for me. I’m not sure I really want to know as I was not good student. I was not academic. In fact, my original high school plan at Loudoun Valley was vo-tech – I was going to be a printer/press operator. With four-years of vocational agriculture along the way. (I actually only did two years in vo-ag.) The change to a college track midway in high school when I moved to Joplin was a good thing. It was also a good thing a few years later when I became an art major after my sabbatical in the Army.

I’d rather have taken the route I did than another, even if it might have been more efficient or led to greater success.

But your new shoes are worn at the heels
and your suntan does rapidly peel
and your wise men don’t know how it feels
to be thick as a brick.

I would be a lot more comfortable if the advocates of Big Data/Predictive Analytics (BDPA) would limit their talking points and their dreams to just modestly improving student outcomes. Outcomes like increased learning and reduced behavioral problems. They don’t need to promise to eliminate problems, just improve them a bit. I tend to trust teachers enough that I don’t think they need to be told what to do nearly as much as they need to be given the resources and freedom to follow their own intellect and knowledge of their students.

Certainly we can improve on the last bit. But how much do we really need to do? BDPA taken too far, which is not all that far, simply removes the professional and replaces her with a content-delivering automaton. Gee, let’s just go to Disney World!

Spin me back down the years and the days of my youth.
Draw the lace and black curtains and shut out the whole truth.
Spin me down the long ages: let them sing the song.
See there! A son is born and we pronounce him fit to fight.
There are black-heads on his shoulders, and he pees himself in the night.
We’ll make a man of him, put him to trade
teach him to play Monopoly and how to sing in the rain.

Yeah, I know that Pink Floyd would probably be a better understood piece of background music than Jethro Tull, but the day of this conversation I heard “Thick as Brick” at some point and it all came together. The only thing I thought might be better was Harry Chapin’s “Story of a Life” but I was sure it would be misunderstood.

The Poet and the Painter casting shadows on the water
as the sun plays on the infantry returning from the sea.
The do-er and the thinker: no allowance for the other
as the failing light illuminates the mercenary’s creed.
The home fire burning: the kettle almost boiling
but the master of the house is far away.
The horses stamping, their warm breath clouding
in the sharp and frosty morning of the day.
And the poet lifts his pen while the soldier sheaths his sword.
And the youngest of the family is moving with authority.
Building castles by the sea, he dares the tardy tide to wash them all aside.

I spent 10 years as a scout leader. During a span of those years I was involved in totem pole carving and overseeing scouts as they added their own bit of carving. Despite the maxim that a sharp knife (or chisel, or gouge) is safest, invariably a boy would cut himself before I could intervene. Including my son, who managed to cut through thick leather gloves before cutting his leg.

Even the sharpest tools can turn on the user. Especially if the user is neither strong enough or skilled enough to control the tool. Data are like that. They also can turn on the user because they don’t always represent what we think we see.

And sometimes predictions are wrong. The use of algorithms does not eliminate the human element, it merely puts it further away – writing the algorithms.

The cattle quietly grazing at the grass down by the river
where the swelling mountain water moves onward to the sea:
the builder of the castles renews the age-old purpose
and contemplates the milking girl whose offer is his need.
The young men of the household have all gone into service
and are not to be expected for a year.
The innocent young master – thoughts moving ever faster –
has formed the plan to change the man he seems.
And the poet sheaths his pen while the soldier lifts his sword.
And the oldest of the family is moving with authority.
Coming from across the sea, he challenges the son who puts him to the run.

In our search for efficiency and effectiveness in education, do we always need to kill the cat? Maybe he can find his own way out of the box before the timer goes off. I’m uncertain either way.

I’ve got two grandsons in public school. I want them to do well, but not at the risk that everything everyday is mapped out to the smallest degree. Let’s leave room for a little wrongness, a few more mistakes. I learned more from my mistakes than had I not made them. Including failing seventh grade math was I recalled the other day when talking to grandelf #1 about his classes (seventh grade). My mother was not happy as I recall – especially since it had far more to do with laziness and being obstinate than anything else.

So!
Come on ye childhood heroes!
Won’t you rise up from the pages of your comic-books
your super crooks
and show us all the way.
Well! Make your will and testament.
Won’t you? Join your local government.
We’ll have Superman for president
let Robin save the day.

I think I understand both sides of the BDPA debate, at least to some degree. As usual, I think the better answers are somewhere in the middle…probably more towards the traditional model of teaching. Let’s just accept the fact that education is expensive (since that it is at the heart of a lot of these innovations) and understand the most efforts to make education cheaper do little more than cheapen it.

You put your bet on number one and it comes up every time.
The other kids have all backed down and they put you first in line.
And so you finally ask yourself just how big you are
and take your place in a wiser world of bigger motor cars.
And you wonder who to call on.
So! Where the hell was Biggles when you needed him last Saturday?
And where were all the sportsmen who always pulled you though?
They’re all resting down in Cornwall
writing up their memoirs for a paper-back edition
of the Boy Scout Manual.

Remember, the joy is in the trip, not the destination. So also is the learning.

Just quit whining

Yesterday, at the Summer Hearing of the Advisory Committee of Student Financial Aid about PIRS (the proposed rating system), there was plenty of talk about some kind of adjustment for inputs or weighting based on types of students enrolled. As I heard things, there are four positions on this topic as they relate to issues of PIRS for use as a consumer information product and as an accountability tool.

1) Everything must be input-adjusted for fairness (both for consumer information and accountability).

2) Input-adjustments are only appropriate for accountability.

3) Consumers need to see non-adjusted numbers, particularly graduation rates, to know their likelihood of finishing.

4) Institutions that serve predominantly low-income, under-prepared students (or a disproportionate share of such – I guess they all feel they are entitled to a righteous share of smart, rich students) are doomed to fail with a significant number of these students.

The fourth point just makes my teeth ache. Part of me wants to scream out in public, “If you don’t feel you can be successful with these students, quit taking their money and giving them false hope. Get out of this business.” I know that is somewhat unfair. Also, I believe that a certain amount of failure should be allowed and expected, especially in the name of providing opportunity. Further, each student does have to do the work and make an effort – but I believe that most want to do so. To publicly state that at some point, your institution just won’t be able to do any better (especially if that is short of 100%) just strikes me as conceding battle before fully engaging.

There is so much ongoing effort and research focused on improving student outcomes, it is hard for me to believe that someday every student that wants to succeed will be able to do so.

As you might surmise, I disagree with point one. I can live with the concept of input-adjustment for accountability, especially given differences in public support and student/family wealth. But to provide input-adjusted scores to students that attempt to level the comparisons between VSU and UVa, doesn’t make sense to me. They are radically different institutions with different mixes of students, faculty, and programs. And costs.

I’m also not a big fan of comparisons in general. They are overly simple for big decisions and so easily misleading. At SCHEV, our institution profiles are designed to avoid the comparison trap, and ignore the concepts of input-adjustment. We do provide the graduation rate data (a variety of measures on the “Grad Rates” tab) on a scale anchored with sector’s lowest and highest value in the state.

pirs6

Likewise, when we released the mid-career wage reports this week, we created these only at the state-level. While there might have been more interest in comparing institutions, we think policy discussions deserve something more.

However, the US News & World Reports Best College rankings get 2500 (or more!) page views for every page view these reports get*. The PayScale Mid-Career Rankings have also gotten far more coverage. I think this is pretty strong values statement of the higher ed community, that despite what the faculty/faculty-researchers say and teach, the great bulk of the community want rankings and comparisons.

*What, you think I don’t know that non-highered people look at the rankings? Of course they do, given the number of colleges and universities ranked, the number of administrators at each, and numbers of journalists writing stories about rankings, it doesn’t take long to get to a half-million page views in a day.

So, quit whining about input-adjustments and focus on becoming exceptional in teaching and graduating students. Quit whining about government ratings if you are going to keep feeding the economic engine that saved Us News & World Report.

We are going to fail with some students. We don’t have to fail with most, which some institutions manage to do.

 

Imagine a world without rankings

Imagine there’r no rankings
It’s easy if you try
No more US News
Above us just IPEDS
Imagine all the people
Living for today…

Libby Nelson at Vox has written elegantly about the US News & World Report rankings of America’s Best Colleges. She points to the fact that the rankings lead colleges to lie and they cause institutional leaders to focus on the wrong things. Finally, Libby calls for a boycott of US News surveys.

I just don’t see it happening.

I know I am middle-aged and bitter, and have occasionally been accused of a cynicism that is matched only by my tendency towards rank empiricism, but I’ve known Bob Morse a long time and I’ve known a lot of college presidents. What I don’t know is the necessary difference between a parasitic relationship and a symbiotic relationship. Especially when the potential parasite is not easily identifiable as such.

Imagine there’s no rankings
It isn’t hard to do
No reputation surveys
And no climbing walls too
Imagine all the people
Living life in peace…

More importantly, the desire to compare is taught to us as children. “See the apple, it is different from the orange.” Comparison is second nature and when there are many things to compare, we try to place them in an ordered list for simplicity.

But you know, US News didn’t exactly dream all this up. Colleges and presidents were using many of these metrics already. “Come to our college, we have a lower student to faculty ratio than other colleges.” All USNWR did was organize and systematize the metrics, and then put them into a numeric order. If they hadn’t done this, somebody else would have. Further, if USNWR America’s Best Colleges goes away tomorrow, somebody else will step in.

You may say I’m a dreamer
But I’m not the only one
I hope someday you’ll join us
And the world will be as one

Imagine no student debt
I wonder if you can
No need for PLUS or Stafford
A brotherhood of man
Imagine all the people
Sharing all the world…

Just about every college has one or more peer groups for internal rankings. In Virginia, the public institutions have formal peer groups for faculty salary recommendations and targets (60th percentile of the peer group). Sooner or later, someone would attempt to the monetize these peer groups and create rankings. The specter of a national rating simply adds fuel to this…especially as a way to validate the rankings or the ratings.

You may say I’m a dreamer
But I’m not the only one
I hope someday you’ll join us
And the world will live as one

Finally, Libby makes this excellent point:

Even if most colleges won’t disarm unilaterally, as Reed did, higher education has proved it can organize effectively for collective action when necessary. Colleges have worked together many times to defeat government proposals they don’t like. The logical conclusion is that their umbrella organizations have decided the power of a single ranking system is more fearsome and difficult to confront than that of the president and Congress combined.

Or perhaps college presidents are happy to have their cake and eat it too, denouncing the rankings in public and frantically trying to climb them in private. We’ll know in a few days, when the cycle begins again.

Public college presidents like having the rankings to use with/against state policymakers. They are just another tool they can use to claim external validation of either how wonderful they are, or how under-resourced they are. Of course, the same arguments work with governing boards, whether public or private.

I’m afraid that rankings are here to stay. After all, Imagine rose to #3 on the Billboard Hot 100 and is also ranked as one of the 365 greatest songs of the century. A Guinness survey has ranked it as the second best song of all time. And these are not all the rankings by any means – and I am pretty sure that rankings were not John Lennon’s goals.