One thing is not like the others

Juliet (Romeo’s love) spaketh thus: ‘Tis but thy name that is my enemy; Thou art thyself, though not a Montague. What’s Montague? it is nor hand, nor foot, Nor arm, nor face, nor any other part belonging to a man. O, be some other name!  What’s in a name? that which we call a rose By any other name would smell as sweet; So Romeo would, were he not Romeo call’d, Retain that dear perfection which he owes without that title. Romeo, doff thy name, And for that name which is no part of thee take all myself.

The Naming Of Cats
by T. S. Eliot
The Naming of Cats is a difficult matter,
It isn’t just one of your holiday games;
You may think at first I’m as mad as a hatter
When I tell you, a cat must have THREE DIFFERENT NAMES.
First of all, there’s the name that the family use daily,
Such as Peter, Augustus, Alonzo or James,
Such as Victor or Jonathan, George or Bill Bailey–
All of them sensible everyday names.
There are fancier names if you think they sound sweeter,
Some for the gentlemen, some for the dames:
Such as Plato, Admetus, Electra, Demeter–
But all of them sensible everyday names.
But I tell you, a cat needs a name that’s particular,
A name that’s peculiar, and more dignified,
Else how can he keep up his tail perpendicular,
Or spread out his whiskers, or cherish his pride?
Of names of this kind, I can give you a quorum,
Such as Munkustrap, Quaxo, or Coricopat,
Such as Bombalurina, or else Jellylorum-
Names that never belong to more than one cat.
But above and beyond there’s still one name left over,
And that is the name that you never will guess;
The name that no human research can discover–
But THE CAT HIMSELF KNOWS, and will never confess.
When you notice a cat in profound meditation,
The reason, I tell you, is always the same:
His mind is engaged in a rapt contemplation
Of the thought, of the thought, of the thought of his name:
His ineffable effable
Deep and inscrutable singular Name.
I have little interest in college ratings.  This was even true when I as at an institution. Perhaps it is because I don’t like it when someone tells me what to think (I really don’t). I suspect though it is because I little interest in assigning rankings based on assumed commonality. Certainly many colleges and universities have a lot in common in structure and approaches to education. Accreditation, federal, and state involvement enforce additional commonalities. But. These are complexx organizations with lots of moving parts.
For example, I have been thinking three songs of the rock era and how they compare. They are all songs with similar purposes of expression and entertainment. They all share a common word in the title: Stand.
The three songs are quite different. Different styles of music, tempo, and subject. Does it make sense to rank them on anything other than preference or popularity?  Clearly those choices are personal, subjective.
Taking the (what some might consider ridiculous) comparison further, what about two songs with the same names ?
Turn to Stone. ELO &Joe Walsh
You might think I can’t find two examples even more different from each other.
You’d be wrong.
You and Me. Alice Cooper & The Book of Mormon.
I think rating and ranking colleges and universities is pretty much a pointless exercise, right up there with peer group analysis, other than to make money. There’s nothing wrong with making money. Quite possibly our next American President will have made a lot of money while establishing the Trump brand. He will certainly be rated highly on that score compared to other presidents. I guess that matters.
The problem of course is that rating is based on one dimension – wealth. Which is kind of true about most, if not all, college rankings. Despite all the blather otherwise, all the measures are proxies for wealth – institutional wealth, and student (family) wealth. Sometimes the proxy measures are simply the wealth of the surrounding communities – the employers that higher the graduates. In the end, the rankings all come back to money.
It’s summer and the college rankings season has begun. And I just can’t take it seriously.


The percent of graduates who complete a degree in four years from start to finish is not the same thing as a four-year graduation rate.

The former measure is a backward-looking measure. It looks at all the graduates in a given year and then checks back to see what percentage started at the institution within the previous four years.

The latter measure, a four-year graduation rate, is a forward-looking measure in that it starts with a group or cohort of similar students (in this case, first-time in college, full-time at first enrollment) and checks to see what percentage of these students completed their degree within four years.

Why does this matter?

Four years ago,  a Small Liberal Arts College (a private, nonprofit SLAC) announced a new commitment to (certain) students: The college announced this week it will waive tuition costs for any additional courses needed to complete a degree if a student isn’t able to graduate in four years, provided certain requirements are met. The article goes on to report, based on the announcement on the college’s website, that 95% of graduates complete their degree within four years and that this is a much higher rate than the four-year completion rate for all graduates at private colleges (almost 80%) and for public institutions (below 50%). The source of the data is the National Association of Independent Colleges and Universities (NAICU).

In fact, one of the financial aid pages on the College’s website goes so far as to say this:

Many consider state colleges as having a low sticker price. In reality, once we factor in aid, the final cost isn’t significantly different than a state college. It is also important to note that 90% of our students graduate in four years saving the cost of a 5th year.

The College and NAICU should know better. The comparisons they are making are not appropriate. They are, in fact, deceitful. Note: We should probably be fair to NAICU as we do not know they were complicit in this deceit.

So, what is the real story here?

First, we assume that nearly all students graduating from the College  probably do graduate within four-years. It is simply too expensive not do so.

The National Center for Education Statistics (NCES) maintains an excellent website,College Navigator, that is quite useful for finding and comparing colleges. It’s data is based on annual surveys, the Integrated Postsecondary Education Data System (IPEDS), required of all Title IV (federal student aid programs) participating institutions. It reveals that only 49% of all the students at the College starting in the Fall of 2008 graduated in four-years or less. Only 62% graduated within six years.

This is a far cry from 95%.
If the statements on the College’s website are representative of the critical thinking and analysis skills taught at the college, we have a serious problem. I kind of hope these are cynical, deceitful statements of an institution trying to stay afloat. That can be dealt with greater ease than an inability to engage in critical thinking because I just don’t think they understand the problem.

The College seems to be intent on putting the slack into SLAC.

Just as a reminder, here is the Principle of Integrity from Southern Association of Colleges and Schools Commission on Colleges:

Integrity, essential to the purpose of higher education, functions as the basic contract defining the relationship between the Commission and each of its member and candidate institutions. It is a relationship in which all parties agree to deal honestly and openly with their constituencies and with one another. Without this commitment, no relationship can exist or be sustained between the Commission and its accredited and candidate institutions.

Integrity in the accreditation process is best understood in the context of peer review, professional judgment by peers of commonly accepted sound academic practice, and the conscientious application of the Principles of Accreditation as mutually agreed upon standards for accreditation. The Commission’s requirements, policies, processes, procedures, and decisions are predicated on integrity.

The Commission on Colleges expects integrity to govern the operation of institutions and for institutions to make reasonable and responsible decisions consistent with the spirit of integrity in all matters. Therefore, evidence of withholding information, providing inaccurate information to the public, failing to provide timely and accurate information to the Commission, or failing to conduct a candid self-assessment of compliance with the Principles of Accreditation and to submit this assessment to the Commission, and other similar practices will be seen as the lack of a full commitment to integrity. The Commission’s policy statement “Integrity and Accuracy in Institutional Representation” gives examples of the application of the principle of integrity in accreditation activities. The policy is not all-encompassing nor does it address all possible situations. (See Commission policy “Integrity and Accuracy in Institutional Representation.”) Failure of an institution to adhere to the integrity principle may result in a loss of accreditation or candidacy.

1.1 The institution operates with integrity in all matters. (Integrity)

(Note: This principle is not addressed by the institution in its Compliance Certification.)

Just Don’t Do It

Last night, my wife stabbed be in the chest with a pair of scissors. I spent the rest of the night in the TMC (Troop Medical Clinic). Don’t piss me off today. Just don’t.

This was Drill Sergeant Stringer, in June of 1982 at Ft. Benning, GA. It was a Sunday morning near the very end of Basic Training and Advanced Infantry Training. Somewhere along the way, one of the 30-plus young men of the second platoon did something to piss off Sergeant Stringer. In short order, he had us in formation in front of the office window, where he sat and directed us in grass drills (a form of physical training designed to wear you out).

Drill sergeants are supposed to be tough. Part of their job is to instill discipline and weed out the weakest. And those least able to show discipline. I spent a brief time as an acting drill sergeant while in the Army Reserves. It was an honor and a challenge. And not a career for me.

But, “Don’t piss me off,” still holds. Call me a hard-ass all you want (and apparently some have recently), but do it to my face. Quit making excuses about how the actions of others affect you and deal with your mission. I am as much of a hard-ass as I need to be fulfill my mission.

And I am not even talking about what I consider ethical lapses in providing bad data or making misleading statements on your website. That’s actually a whole other story.

It’s quite simple though. If your institution has a 62% graduation rate, 90% of students do not graduate in four years. Saying they do is wrong. The correct phrase is, “of our students who graduate, 90% do so in four years.” If your liberal arts college cannot be honest about that, then perhaps there are other problems that are deeper, more disturbing.

A colleague asked me about this on Friday. Is it likely that the best early indicators of failing college, one on the brink of closure, is the intentional misuse of data? Intentionally misleading prospective students about the quality and cost of an institution is something I find to be offensive, at the very least. So I wonder if we can develop an index of such things on the top pages of an institution’s web site and tie them to what we know about admissions, retention, and discount rates. I’ve written before that any private institution of fewer than 2000 students without a large graduate program (master’s program are often profit centers) and a first-year retention of less than 60% is one that should be on deathwatch unless it has a lot of unrestricted endowment.

Likewise, all institutions participating in Title IV financial aid programs, and all Virginia institutions requesting/receiving general fund support (including the tuition assistance grant)  must make certain disclosures on their website. The ability to identify and locate those disclosures would possibly be another good metric.

In other words, just don’t.

maybe I should stop paying attention

one day I was listening to a college president drone on.

that, in and of itself, is not unusual. I spend lots of time with college presidents, probably more than I would like.

he began talking about the role of a college education. “Sure, for some students it is about a job or career. But traditionally, college is really about preparing citizens.”

Oh, shit.

maybe it was because the night before I had let Starship Troopers run in the background and the discussion of the difference between citizens and civilians.

and then thinking about the traditions of college education that seemed to exclude an awful lot of non-white, non-male people

and I started thinking about the really, super privileged background from which he spoke

and I wasn’t happy. he thought he was making good, classical argument about the value of higher education.

I started thinking about Henry Gibson in the Blues Brothers.

I was laughing in sorrow. and anger. I’ve been fortunate. I am fortunate. and I know it.

not sure I would have done as well myself coming from a less-privileged background.

Accessible, and home

The long saga that began (kinda) in February is done (it actually began a year ago with the first temporary ramp, and then replaced with the huge sprawling ramp). All the bathrooms in the house are fully accessible by my wife. That’s not to say they are all wheelchair accessible as that would have been major reconstruction of the house to make the master bath wheelchair accessible. We could have taken the door to 36″ instead of a mere 32″, but since the bedroom door is only 32″ and widening it is simply not feasible, we simply made sure that any of her three walkers could get through the door. Making the first floor bath accessible meant giving up the closet in the entryway and the utility closet under the stairs. It also meant re-positioning the door to the front of the house instead of from the kitchen (which is an improvement). This bathroom is wheelchair accessible. Technically the shared bathroom upstairs is wheelchair accessible…as long as the wheelchair is already upstairs.

Remember those horrid water lines made of Flowguard Gold cpvc that would break without effort? All replaced. All throughout the house. One less thing to worry about for awhile.

The stair-lift is also installed. A day late…actually a week late. After a series of errors for which no one seemed willing to take ownership, the lift was installed the day AFTER my wife was sent home from the nursing home after two weeks of rehab and very minimal care following her first total knee replacement. Somehow she found the strength within herself to make it up the stairs to the bedroom. Of course, three weeks in hospital and nursing home can inspire a tremendous will to sleep in one’s own bed. I know this from my own experience.

But the dust is settling and her recovery from surgery continues.  I still worry about making the backyard more easily accessible, but that can wait. It is simply a longish way around. That unfortunately is the nature of accessibility. Sometimes I wonder how much the ADA as it is currently written, really helps. Frequently the ramps at movie theaters and restaurants seem to be designed as an afterthought. Worse yet, they seem designed only to meet the requirement of one inch of rise per linear foot. Little thought goes into the fact that these long, gradual paths, that tend to be traversed slowly by the people they are intended to accommodate are fully exposed to the weather. Just as importantly, it is worth remembering that not all those with disabilities needing this pathways are in wheelchairs and not all wheels are large. Take a look at some of the mobility devices people are using these days, with both permanent and temporary disabilities.

Think also about your home. Is it accessible in case you or a family member has a surgery affecting mobility? Can your home be visited by anyone or only the able?

I see our home in a whole new light these days. (Truth to tell, I have also replaced most of those.) There are design choices that should never have been made. They simply represent trying to combine function with lowest cost to meet code requirements. Probably this is not a good way to build.


A work in progress

(A presentation at VLDS Insights 2015.)

Work (and thoughts) in progress: When is research and information enough to warrant change in policy?


This is really a working paper that came about from a response I gave to a reporter a few months ago.

“What policy changes has the state or institutions made since you published the wage outcomes of graduates?”

“Well, none, I hope. It is far too early in our understanding of the quality and value of these data to make any kind of sweeping policy changes based on them.”

The great promise of VLDS (Virginia’s Longitudinal Data System) is the ability use research based on individual data from administrative datasets to make policy recommendations that improve citizen outcomes.  For too long, education policy was based on series of one-off studies using data from single-use collections, national surveys, or work done in other states. I’m not suggesting these studies have no value, only that they may have less relevance than data on Virginia students. Good research, using test and control groups, and enough randomly selected students, creates important models for understanding interactions and relationships. By itself though, it is not enough.

First, even an exceptionally good study done in another state may not really be relevant to Virginia students. State policies and funding, school and institutional policies and funding, and specific curricula may simply add too many confounding factors. However, even at worst, these studies can guide us in our own research.

Second, these studies are expensive. They require skilled researchers with advanced training. The data collection itself is expensive and has to be repeated for each new study. And our constituents for the results, policymakers and their staff, have little interest in paying again and again for such studies. They also have little interest in waiting.

VLDS provides opportunities to create datasets for researchers that can look across years of students’ experience in schools, divisions, education levels, and even into the workforce with relatively little cost. Even more importantly, VLDS offers the ability to not only recreate the data for a specific study a year or five years later, it also allows us to develop regular, annualized reporting from the same data elements allowing our constituents and ourselves to track progress.

So we can do research and make policy recommendations we couldn’t before. But how quickly should we do the latter?

In 1959, Charles E. Lindblom wrote about “The Science of ‘Muddling Through’” in Public Administration Review where he posited two views of policymaking: Rational-Comprehensive and Successive Limited Comparisons. Lindblom also describes these models, respectively, as the “root” method which starts from fundamentals of the problem and grounded in theory, and the “branch” method that always builds out from the current situation.  I won’t go into two much detail about these other than enumerate their steps and attempt to draw a comparison that I think adds value.

Rational-Comprehensive (Root)

  1. Clarification of values or objectives distinct from and usually prerequisite to empirical analysis of alternative policies.
  2. Policy-formulation is therefore approached through means-end analysis: First the ends are isolated, then the means to achieve are sought.
  3. The test of a “good” policy is that it can be shown to be the most appropriate means to the desired ends.
  4. Analysis is comprehensive; every important relevant factor is taken into account.
  5. Theory is often heavily relied upon.

Successive Limited Comparisons (Branch)

  1. Selection of value goals and empirical analysis of the needed action are not distinct from one another but are closely intertwined.
  2. Since means and ends are not distinct, means-end analysis is often inappropriate or limited.
  3. The test of a “good” policy is typically that various analysts find themselves directly agreeing on a policy (without their agreeing that is the most appropriate means to an agreed objective.)
  4. Analysis is drastically limited:
    1. Important possible outcomes are neglected.
    2. Important alternative potential policies are neglected.
    3. Important affected values are neglected.
  5. A succession of comparisons greatly reduces or eliminates reliance on theory.

Lindblom discusses each step of the Branch method in detail, for those that wish to do the reading. The points I wish to make begin by drawing a comparison of the Root method with the research support that VLDS provides. It is without a doubt that very few agencies of Virginia state government are staffed and funded to perform the in-depth, long-term, theory-based research that university faculty perform each year. It is also unfortunately true that few agency staff have the time to stay current in all the published research related to the data for which they are stewards.

The way Lindblom describes the Root method, it is an impossible method for all but the simplest problems, “It assumes intellectual capacities and sources of information that men simply do not possess, and it is even more absurd as an approach to policy when time and money is limited, as is always the case.” Of course, he bases this conclusion on his initial premise that the analyst in question would perform incredible amounts of due diligence in values identification, data collection, and comparison of potentially relevant policies. I think we can put that aside and take a more reasoned view, a doable view, of the Root method that assumes an appropriately thorough conduct of due diligence grounded in theory from prior research. It’s possible, just expensive. Even completely thorough research has to make some assumptions.

The Branch method is what we do every day. “This is what we know now. This is where we want to be and these are the resources we have to make our analysis.” We find agreement without necessarily debate if a policy is the best, only if it as Herbert Simon put it, “satisfices.” We use data trends and comparable measures to confirm our agreement on policy.

Relying on either method alone is sub-optimal. The Root method is too expensive and it takes too long. Sometimes policy turns on a dime (a 20-minute phone call while one or two queries are performed “live”) and a “researched” decision has to be reached very quickly. The Branch method alone is shallow and may ignore a likely history that exists on similar data studying the same or closely similar question.

Uniting the two methods makes far more sense.

A message that I frequently push, perhaps to the annoyance of some of my colleagues, is that elected officials and their staff have little patience with the in-depth reports from the Root method. They are typically too long, too nuanced, and too detailed for their needs. Further, and this is the most important thing, if the findings of the research are adopted, they want to know that in each succeeding year data will be available to judge the results. Paying for another in-depth study is rarely a considered option.  Thus, the reporting that supports and justifies the Branch method plays a critically important role.

If you accept this model, our next question is the title of the session – When is research and information enough to warrant a change in policy?

I participate occasionally in a forum for people with a certain kind of brain tumor. New members facing treatment decisions frequently struggle with understanding how to decide what to do. The challenge is particularly acute for smaller tumors where there are more options – watch and wait, micro-surgery, and radio-surgery (radiation). This compounded further by the fact that each doctor tends to favor his or her own specialty and thus a patient doing due diligence and seeking a second, or multiple, opinions may become confused. In fact, one of their first posts following, “Oh my God, I have been diagnosed with a brain tumor!” is “How do I decide what to do? My surgeon says surgery, but the radiation oncologist says radiation.  I don’t have serious symptoms, do I have to do anything at all?”

Even after that point once a decision is made about surgery or radiation, there may be questions about what surgical approach or what form of radiation therapy. To some degree, the answers to these questions are about the experience and preferences of the selected surgeon or the availability of specific forms of radio-treatment at the selected facility. Selecting the facility and treatment team is another decision tree once a course of action is chosen.

In my own experience, I had a large tumor with very limited time to decide. I spoke to two surgical teams and both said very similar things. That made the decision very easy for me, especially within the framework for decision-making I had already made for myself. For example, it was important to me, if possible, to have the surgery done at a university hospital. Closer to home was better for my family than clear across country. Further, all the research I had done about how to make the decision to fully understand the context of my situation made it possible to recognize that hearing the same messages from surgical teams 3,000 miles apart gave a true clarity to the situation.

In other words, when you get the same response multiple times, you are probably on to something. Assuming that you have also done the research to ensure you understand both the question asked and the answer received.

Consistency of results from multiple tests seems a very good place to start. Of course, this implies multiple tests, multiple research projects. It implies, I hope, good research that is supported by well-defined theory should be a required feature of these tests. It seems to me that policy recommendations made based on one study, on one result set, are a poor thing on which to risk the lives of citizens. Our goal should always represent some form of improving lives of Virginians. One set of results seems counter-intuitive to this goal. To my mind the stakes are too high. And this is why agency staff tend towards the conservative and that the “best” policy is found in the agreement of multiple analysts, perhaps using the mantra, “yeah, we can live with this.”

Lindblom states: “If agreement directly on policy as a test for “best” policy seems a poor substitute for testing the policy against its objectives, it out to be remembered that objectives themselves have no ultimate validity other than they are agreed upon. Hence agreement is the test of “best” policy in both methods.”

So, when multiple analysts agree, we have another marker as to when to make a policy change.

Distilling these thoughts into a simple list, I see the following to be key indicators as to when make policy recommendations:

  • When replicable/replicated research confirms theory.
  • When measures developed from research are reproducible and readily from administrative datasets, such as those exposed to VLDS.
  • When multiple analysts agree.

This makes sense to me. It is reasonable and allows for time to consideration of the theory, data, and alternatives. I think also this is how our constituents wish us to make policy recommendations. Unfortunately, a lot of policy is not made this way. Sometimes we are given a matter of weeks to formulate a response to a policy question. Clearly this is not much time. Worse, there are calls that come during the legislative session giving us 20 minutes to develop a query against student-level data over multiple years and provide an answer that sets policy, or rather law. It’s not pretty, but that is the nature of law and sausage.

The purpose of VLDS is to support an environment that allows the three indicators above to take place. A mix of sound research, readily produced data and information, and analytic concurrence.