January 4, 2018

MEGAN MCARDLE: Metrics and Their Unintended Consequences: The best intentions combine with imprecise data for perverse effects in health care and education.

In December, doctors at a VA hospital in Oregon decided to admit an 81-year-old patient. He was dehydrated, malnourished, plagued by skin ulcers and broken ribs — in the medical professionals’ opinion, he was unable to care for himself at home. Administrators, however, overruled them.

Was there no bed for this poor man? No, the facility had plenty of beds; in fact, on an average day, more than half of the beds are empty, awaiting patients. Was there no money or medicine to care for him? No, and no. Reporting by the New York Times suggests that Walter Savage was, perversely, turned away because he was too sick. Very sick patients tend to worsen the performance measures by which VA hospitals are judged.

If this had happened in isolation, we could simply gape at the monstrosity that bureaucracies are occasionally capable of.

But such examples abound in health care. For example, in the 1990s, New York and Pennsylvania started publishing mortality data on hospitals and surgeons who did coronary bypasses. The idea was that more informed consumers would steer themselves toward the teams with the better statistics — theoretically good for patients, bad for slacking providers. The reality was less ideal: In those states, surgeons seem to have started doing more operations on healthier patients, while turning away the sickest ones who might otherwise have benefited.

From this we can take a few lessons. The first is one that has been well-known to other sorts of businesses: What you measure is what you get, not necessarily what you want. In fact, if your measurement is badly designed, you may get a great deal of something you don’t want.

Well, one way to deal with this is to have less management. If VA beneficiaries got a card that they could use anywhere to purchase medical services, this wouldn’t be such an issue. But it’s always true, as Donald Rumsfeld said, that what you measure will improve, but mostly in terms of the measurement.