Why Ratings Seem to be Necessary to Outsiders

Right here.

Rebecca Schuman reacts to the MLA Report of the Task Force on Doctoral Study in Modern Language and Literature. I am not going to try summarizing it or even highlighting it, as such contempt and wrath needs to be read in its original language.

If a higher education insider can react this strongly to what is clearly a well-intentioned and highly focused effort to improve an alleged profession, then what are the normies outside academe to think?

From the InsideHigherEd story “We are faced with an unsustainable reality: a median time to degree of around nine years for language and literature doctoral recipients and a long-term academic job market that provides tenure-track employment for only around [60] percent of doctorate recipients.”

How ’bout that? This is from the same large group of faculty-types that objects when we in government start talking about credits-to-degree, time-to-degree, placement rates, and job market outcomes for undergraduates. Schuman says this:

So as I talk about this report, please keep in mind that my issue isn’t with the MLA’s leadership—it’s with the MLA’s membership, which consists almost entirely of people who can both afford to pay the dues, and haven’t been so traumatized by the convention that they drop out for their psychological health (I am in the second group).

So she is holding the large group of faculty responsible. Many of whom, at least in Virginia have bemoaned my work with wage and debt outcomes. I guess when it comes to just a continuing stream of sacrificial lambs to fund one’s salary, it is a different story.

In all fairness, this is a healthy debate to have for an academic community to have, especially given the apparent over-production of PhDs compared to the full-time, tenure-track jobs available – which may be the result of people like me (and above) pushing for cost-constraints. It is certainly a result of decreased funding (which I have not advocated). The problem is that the path the MLA suggests, doesn’t seem to make a lot of sense. It seems contradictory and under-informed. To me it looks like they are suggesting producing more of what cannot be currently consumed, without a complete re-funding or restructuring of higher education. (But probably not the restructuring suggested here.)

Reports and debates like this suggest to people like President Obama that higher education has no clue. The proposal of a rating system is a way to enforce a simple message, “Get a clue!” Unfortunately, as currently proposed, and because USED’s focus has historically been on undergraduate access, measurement of graduate and professional programs has not been talked about – save by me. I was the lone voice at the Technical Symposium making that argument. I don’t know that a rating system will help the academe get a clue in a changing world, but I don’t know that it won’t. I do know, as I have said before, that the current data available to the Department are inadequate.

I really enjoyed reading Schuman’s post. As I read it, I wondered, “Is this really much different than current accreditation practices in terms of the resultant nonsensical solution?”

 

 

A Response to Schuman and Warner

I love all the coverage that the proposed Postsecondary Institutional Ratings System (PIRS, #PIRS) is getting these days. Rebecca Schuman over at Slate has written a nuanced support of the plan here. John Warner, over at Inside Higher Ed, has written an opposing viewpoint to Schuman’s. Warner wrote the previous day about Jamienne Studley’s unfortunate comparison of colleges and blenders.

Both are well worth the read.

I have neither the following nor the writing skills of either Schuman and Warner, but that has never stopped me from expressing my opinion. Nor will it now.

Both authors are right and wrong.

First off, while I am glad everyone is having such fun with the blender comment, where were you months ago when the comment was made and reported in Politico and elsewhere? Those of us in the higher ed data world have been shuddering for months about her use of Cooks Illustrated as a model for PIRS. The resurgence off the comment and the announcement of the first delay in the ratings have been amusing to watch.

Warner thinks the ratings will empower the already powerful on campus by giving presidents even greater leverage for their policies. Absolutely. With the data currently available to USED, any thought of nuanced, targeted approaches to improving student outcomes will go right out the window. There will be more sledgehammer approaches to institutional policies, especially as institutions try to ensure they are in the same rating as their peers.

Warner also suggests that new deanlets will be created to collect and manage all the new data required. Maybe eventually, but that depends on what happens reauthorization of the Higher Education Act (HEA). If the unit record ban is lifted, and something like the Student Right-to-Know  Before You Go Act is passed, most institutions could experience a reduction in burden. In the near term, USED still has to get OMB clearance to expand collections, which is subject to burden review. Unfortunately, reporting burden is going to increase anyway, with or without the ratings system, because  well, just because. There is always more data to collect, and lots of organizations asking USED to collect more, and at some point, with enough increases, the institutions will demand to report student-level data because it will be easier and less burdensome. (Something like 45 states have unit record collections, with about 90 different collectors. SC public institutions report student level data to the state. Sending a similar file to USED would cost less than the current IPEDS submissions.)

Schuman suggests that it “is time to come at higher education with a sledgehammer.” I tend to agree, but it depends on who is swinging the sledgehammer. The problem with the way higher education and USED have traditionally approached ratings, rankings, benchmarking, and the like, is through the use of direct institutional comparisons and peer comparisons. This is the kind of madness that has led us to today. Driving the bus by looking at the other buses is just silly. As I argued in my presentation at the PIRS Technical Symposium the proper comparisons are intra-institutional. Rather than worry about institution A compares to B on graduation rates let’s focus instead on the difference in graduation rates between Pell recipients and non-recipients encourage policies to bring those numbers in line with each other, thus increasing graduation rates across the board.

As for Schuman’s suggestion about ratings including values such as percentage of courses taught by full-time, tenure track instructors, fine. Just be warned that using the existing data for community colleges, there have been a number of research projects that have found no direct correlation between community college graduation rates and either numbers or ratios of full-time TT faculty. So, depending on the biases of the folks in the department, such a rating component might do more to support the status quo. Further, state law-makers may well push back against ratings that use such components because of the inherent cost-drivers to funding higher education.

Which is also part of the reason we are here. Not everyone wants to pay what it costs to support higher education.

I am glad that people are talking a lot more about PIRS. I think a good ratings system can be built, just not with the existing data nor the traditional mindset towards evaluation of higher education. We in Virginia  know far more about outcomes of students in Title IV aid programs than USED does – and that is only an off-shoot of our other work. If PIRS is done badly, it will empower presidents to have more of their way on campus and perhaps further damage the concept of shared governance.

The most important thing to keep in mind is that off this is taking place under the umbrella of reauthorization of the HEA with the added context of Gainful Employment. Whatever happens will be with for years and, if historical trends are true, the federal control on campus will be more intrusive. This may not be bad, but it will not be easy.

 

 

 

 

The Gainful Employment Ratings System?

On my solo road-trip to Orlando today for AIR Forum 2014, it occurred to me that people are still not thinking about the ratings system (PIRS) and gainful employment (GE) the way I am.

I am not convinced that PIRS is a real thing. Certainly the Department has spent a lot of time and effort to create something, including the illusion that PIRS is real.

Why? Because both GE and PIRS attempt to do the same thing – eliminate bad actors and low performing programs/institutions from Title  IV eligibility. Since we don’t have details on PIRS, the only difference we can really point to is that GE is about disqualifying programs and PIRS is about disqualifying institutions.

So, if you are a member of Congress or congressional staff working on the reauthorization of the higher education act, are you going to pick one or both? Given the nature of the lobbying and letters of support and opposition you might receive, would you consider combing the two? Especially in light of the argument that GE should apply to all programs.

If you were a lobbyist and became convinced that something would be done, would you grudgingly support program-level effects v. institution-level? Especially if student preparation is factored in to the mix with other measures than earnings and debt repayment?

Do policy-makers and law-makers want two different consumer ratings system that accomplish essentially the same thing? To me, two ratings seems to add more confusion than helping students and family.

The cynic in me wonders if PIRS is simply a way to justify GE for all programs as a reasonable compromise.

 

The damage of relying on federal data

The IPEDS graduation rates are based on the following formula:

Number of Graduates from CohortA with x years/(Number in CohortA-Approved exclusions [Peace Corps, AmeriCorps, military service, religious mission, death, etc.]) 

Where CohortA are all First-time students in the fall of a given year, including those whose enrollment also began in the summer, initially enrolled as full-time.

The calculation is performed where “x” is four, five, and six years post-entry for students pursuing four-year degrees, two and three years for students at two-year colleges.

There is no magic here, simply a set of assumptions of normal expectations that full-time students are likely to remain full-time – assuming their initial enrollment indicates their academic plans should thus complete in four years, but additional years are “allowed” for incidental delays along the way.

Institutions complain frequently that this measure is imperfect, especially community colleges.

It seems that most fail to realize that just because the law requires Title IV-participating institutions to report these figures to USED each year, THEY DON’T HAVE TO STOP THERE.

If an institution wants to report graduation rates for part-time students, or transfer students, star-bellied Sneetches and Sneetches with out stars, they are completely free to do so.

And they should.

Most don’t, I guess because they can’t compare to someone else, such as their self-defined peer groups. Isn’t this insane? Does it really matter how one institution performs against another when an increase in the metric over time is clearly the desired change? Institutions should be focused on improving student success and understanding who doesn’t graduate on time, and why. From there they can work to better support students and address structures that  improve their success.

It simply isn’t that difficult. Comparisons and benchmarking have stopped institutional development, or at least slowed it, for institutions that rely on federal data. The only way to change that is to change the data available – with something like the Student Right to Know Before You Go Act. States can also get involved, like we do in Virginia.

 

 

 

 

 

Rating the Ratings Game

What a big week for PIRS – the President’s proposed Postsecondary Institution Ratings System!

Well, at least in my little world.

On Monday, I was back at Ferrum College for the first time since my son’s graduation a year ago for SCHEV’s meeting with the Private College Advisory Board (the private college presidents) and our regular May meeting of Council. At the very end of the meeting I was asked to give an update of my activities with the wage and debt reports. During this briefest of updates I also volunteered supportive responses to issues raised during the meeting.  (Of course, “supportive responses” were really along the lines of “If you would look at the damn website you would see that these things you are asking for already exist in great detail.”)  When I asked for questions, I received one, “Can you tell us about your involvement with the ratings system?”

It was kind of a set-up, in that I had spoken with that president just prior to the meeting and so he knew something of my involvement. I gave about a four sentence response summarizing my presentation at the symposium, which was greeted with the only applause of the day. The reception following the meeting contained a number of side conversations about the topic and requests for materials.

Parallel to all this, due to the marvels of mobile email, there is an exchange on this topic with my boss and a public college president, and the sharing of my presentation with that president.

Tuesday night was the time for pair of separate email conversation with a public and private president, both of whom have become highly involved in the topic and have been offering alternatives to USED and members of Congress. I’m really glad to see that they have been engaged and are offering some thoughtful, and good, suggestions.

Wednesday we saw the blog post from Jamienne Studley that announced that draft ratings system release would be delayed until fall. Many of us were not surprised by this news. This is a big project with high stakes – the initial release sets the tone as to whether Congress might actually tie Title IV eligibility to the ratings.

One of Tuesday night’s conversation led to a call from a congressional staffer. During the course of the discussion I learned there was a proposed budget amendment to prevent the department from spending any money on PIRS. This is a reaction to Duncan’s commitment last month to continue the project, even if Congress does not provide the $10 million requested for PIRS.

So, now we have a bit of horse race to watch.

Will PIRS make it out the door before October 1? (The federal fiscal year ends September 30.)

Will Congress pass a budget before PIRS is released?

Will the budget have an amendment killing PIRS?

If PIRS hits the street before next year’s budget is passed, and is a good product, then it has a chance. If delayed too much in the fall, it is quite likely dead. (One might wonder how much intention is in this delay….)

Today’s letter opposing Gainful Employment from 34 members from both parties might be an indicator of where this might go. PIRS is merely GE at the institution level. If the initial draft places large numbers of for-profits in the lowest ratings, I suspect we will see a very similar letter from members.

So, all told, I give PIRS three stars out of 10. It had a better chance if the Department had been able to keep to its announced schedule. It seems to me that an August release of a good product is necessary for its survival. I understand the need for a delay, it is a big project and I have delayed quite a few myself. Unfortunately, political realities can get in the way.

Non-urgent update, basically a late, “See? I told you!”

InsideHigherEd confirmed the idea of the amendment last week with a copy of the email.

 

College Affordability

It seems to me that most people define “affordable college” as little or no out-of-pocket cost.

Some define it as low/reasonable debt.

Both positions focus, reasonably, on the student/family perspective of paying for college. College affordability is more than that though, it is also a question of what is affordable for the institution, the local/state government, and the federal government. For government, the affordability question is rarely addressed directly. Most often it is addressed as “what do we have to spend on higher education this year,” often ignoring any funding models in place.

One of the fundamental problems we have is that we simply have not sat down and decided who shall pay how much.

These are the questions that need to be asked and answered:

  1. Should students and families have a role in paying for public higher education? If so, to what degree?
  2. Should the federal government have a role in financial support of public higher education? If so, to what degree?
  3. Should the federal government have a role in financial support of private higher education (nonprofit or for-profit)? If so, to what degree?
  4. Should the state governments have a role in financial support of private higher education (nonprofit or for-profit)? If so, to what degree?
  5. Should publicly-financed debt have a role in supporting students in higher education? If so, to what degree?

Some will argue we have already answered the “should” questions, which is true enough, but we certainly have not effectively answered the questions of degree. In the states, despite all manner of good intentions, public higher education funding is quick to be cut because institutions can increase tuition. The same has been true for grant support to students attending private institutions. So I say we should address these questions as if they are new.

One thing beyond doubt is that any institution is unaffordable for any student if the student can’t graduate. The obstacles to graduation go well beyond student preparation and include what it costs to live and the fact that many students are not 18-22 year-olds without family responsibilities.

There is also the aspect of the student’s responsibility to do the work. This is an issue that allows policy-makers to treat postsecondary education as something different. For the very large part, college students are legal adults. This allows policy-makers to assume that students should pay to go to college. However, policy-makers conveniently ignore the fact the students are adults and make their own choices when they choose to hold institutions accountable for graduation rates.

We really need to think about these issues. Higher education is no longer only for the second sons of the elite, or even their first sons. It is for everyone. That is why so many options for study and credentials exist and thus we need to rethink what we are doing, why we are doing it, and how we should pay for it.

The current hodgepodge of funding streams and policies do little more than disguise the cost of education to the student and their family. It also seems to do little to support student success.

We can do better.

 

 

 

VLDS Insights 2014 Conference

We are coming up on the second annual Virginia Longitudinal Data System Insights Conference. This is kind of a big deal since it is our second conference and we have a little more to talk about in terms of research outcomes. This is still very much a work in progress, but VLDS is here to stay.

I will be speaking a number of times. First, as part of an opening panel on VLDS. Second, the impact of VLDS on legislation leading to HB886 that requires high schools, school divisions, colleges and universities to link to the outcomes and student performance data on our website. Third, I will be discussing the upcoming release of our mid-career wage outcomes data and other data products.

There are lots of other sessions to attend if you are not interested in hearing me. College and Career Readiness, Teacher Preparation, Career and Technical Education, Workforce Education, and other topics. It will be another great event.

 

Virginia Longitudinal Data System Logo

 

 

 

 

 

The second annual Insights Conference will be held on Tuesday, June 24, 2014 at the Hilton Richmond Hotel & Conference Center. The conference theme, “Bridging the Data Divide,” will emphasize how the Virginia Longitudinal Data System (VLDS) is bridging the data divide among citizens, researchers, and policymakers by providing a single point of access for educational and workforce information. The conference will open at 8:00 AM with registration, a networking breakfast and a demonstration of VLDS. Sessions will run from 9:00 AM until 5:00 PM with a break for lunch. For more information and registration, please see: http://www.cit.org/insights-conference/

The objectives of the Insights Conference:

  • Link communities of data providers and data consumers
  • Celebrate and build the future of VLDS
  • Inspire stakeholders to provide insights to citizens