Excesses

Measurement-Based Management -- And Its Excesses

Henry Mintzberg is a widely-respected professor of management at McGill University in Montreal. He wrote an article called "Managing Government, Governing Management" for Harvard Business Review in May-June, 1996. In it, he said:

"Next, consider the myth of measurement, an ideology embraced with almost religious fervor by the Management movement. What is its effect in government? Things have to be measured, to be sure, especially costs. But how many of the real benefits of government activities lends themselves to such measurement? Some rather simple and directly delivered ones do -- especially at the municipal level -- such as garbage collection. But what about the rest? Robert McNamara's famous planning, programming, and budgeting systems in the U.S. federal government failed for this reason: Measurement often missed the point, sometimes causing awful distortions. (Remember the body counts of Vietnam?) How many times do we have to come back to this one until we finally give up? Many activities are in the public sector precisely because of measurement problems: if everything was so crystal clear and every benefit so easily attributable, those activities would have been in the private sector long ago."

As we begin to embark on a comprehensive measurement-based management program, what are we to make of this critique? Certainly the excesses of the "Whiz Kids" of the 1960's present a notorious example of the failings and unintended consequences of previous measurement systems. This critique has to be confronted directly lest we be doomed to repeat history.

In the context above, it is clear that Mintzberg is talking about outcome as opposed to output measurements. This is an important distinction. Briefly, outputs are productivity or efficiency metrics such as the number of reports written per month, the number of widgets manufactured per day, etc. They also might include service metrics such as the number of customers visited per month, the number of help desk calls resolved, etc.

Outcomes, on the other hand, deal with long-term effectiveness or results of a mission or function. One generic example of an outcome metric is "customer satisfaction". In private companies, this is a leading indicator of future profitability, growth and increased market share: all outcomes indicating success. However, in the case of government, it is difficult to define outcomes adequately. Since government is a not-for-profit organization, financial results are not relevant as outcome metrics. Growth and market share are also not applicable. "Customer satisfaction" is somewhat relevant, but who is the customer? The government user of a process? The civilian user? The taxpayer? The sponsor or administrator? Congress? The OMB? It could be all the above.

Probably the only generally applicable outcome metric for government is "mission effectiveness". The missions of government are undertaken for the public good, not for gain. But that 'good' is defined differently depending on the agency's mission. Sometimes it is feasible to measure. For example, a police department's outcome would be the overall crime rate in its district. But how would you define the outcome for a military base? Number of wars won? What about a basic research lab, whose inventions might revolutionize technology 20 years from now? It becomes infeasible to measure outcomes in many such cases, and attempts to do so may turn missions astray, as Mintzberg says they did in Vietnam.

So I will grant that in the context in which he writes, Mintzberg is correct. We should be very careful about establishing metrics for government mission outcomes, and in some cases not even try.

But for output measurements, it seems that the situation is quite the opposite. Government offices, warehouses, bases and facilities have much the same support services and user needs as any private company's. So we should expect to find degrees of efficiency of operating and support processes that can be measured, benchmarked and improved. This is the appropriate function of a measurement program, and its goal is simply to improve productivity and efficiency, i.e. reduce the cost and cycle time of internal processes. In all conceivable cases, such reductions will please customers, whoever they are, and probably result in improved outcomes.

Therefore, I believe there is an appropriate place for a measurement system in government agencies, although its emphasis should be on measuring outputs (such as process cycle times) and only the outcomes that are short-term and relatively easy to measure, such as self-reported customer satisfaction from several classes of customers. Such a system will, I believe, meet the letter and the spirit of the Government Performance and Results Act while avoiding excessive agonizing over metrics definitions.