Jay Wieriman (Courtesy Photo)
The scandal at the Veterans Administration hospitals is serious. After reading the VA’s yearly performance overview, it became clear that the VA is a big believer in targets. I became aware of the dangers of arbitrary targets through the work of W. Edwards Deming and statistical process control.
Suppose the safety director of a large company set a target to reduce accidents by 20 percent. He didn’t choose zero accidents because he felt it was unrealistic, but he also did not receive an increased budget, acquire more staff or develop a new way of preventing accidents. He created the target to give people an incentive to do well. How could the target hurt?
First, the safety director’s full-time job is to reduce accidents. If he needs additional incentives to perform, replace him tomorrow. People with good ideas should be rewarded as part of the normal business process.
Second, targets give people an incentive to cheat. If the safety director is able to reduce accidents only 18 percent, he will probably check to see if a few of those accidents were “misclassified.” Most students — 70 to 95 percent — admit to cheating, often because they are paid for every A and B they get. People trying to meet targets often miss the point.
A third problem with targets is that often one target is achieved at the expense of another. Suppose the VA sets a target of seeing 50 percent of its clients within 30 days, and a cancellation occurs. The hospital can either move up a client who is scheduled to be seen in 32 days from his first contact or a client who has been waiting six months to see a doctor. Moving up the first client helps achieve the target of seeing clients within 30 days. Once a client has passed the 30-day target, it does not matter when he is seen.
The fourth reason targets are bad is that achieving one does not guarantee things have improved. The number of murders that take place in a large city can vary widely from year to year. Measurements have normal variation. It is unlikely that a drop or increase in murders in any one year can reliably indicate anything. Statisticians doing quality control use a technique called statistical process control to determine if the variation observed, such as the length of a widget, is normal or if the process is “out of control” based on observations over time. Statistical process control helps identify real change.
For the above reasons, the VA should not set arbitrary targets. Its goal will always be to see people as soon as possible while providing the best of services. I recommend the following:
■ Give the data to a statistician who can create performance distributions for the combined VA hospitals on each of the measures. Those hospitals that fall at the bottom of charts will have plenty of incentive to improve.
■ Graph the means and/or medians of the performance measures over a sufficient period of time to see if any real change is occurring.
■ Do a select number of studies to show if any new procedures appear to be effective.
■ Implement correlation studies to both reduce the number of performance measures and to look for promising relationships.
VA hospitals will need to make many significant changes to improve. Getting rid of arbitrary targets will be the easiest.
Jay Wieriman retired from Waterborne Commerce Statistics Center, where he worked as a statistician for the federal government. As a student, he worked one summer at a VA hospital.