Best practice recommendations photo source behealthtips
Testing is the crucible of science. Which ideas are right and which are wrong? Experiment tells us by putting them to the test
By Robert Gerst : “All life is an experiment. The more experiments you make the better.” wrote Ralph Waldo Emerson. That’s because learning only happens through experiment, through trial and error and the testing of ideas and theories in the real world.
This testing is the crucible of science. Everyone has ideas about the way the world works. Which ideas are right and which are wrong? Experiment tells us by putting ideas to the test.
Liberating ourselves from opinion
Does drug A outperform drug B? Is it safer? There is no way to tell without experiment. The only other option is opinion, the world’s cheapest commodity. Everyone has an infinite amount and will give you any amount you want for free. Liberating ourselves from opinion requires that drugs be tested, experiments run. Only then can reliable conclusions deserving of public confidence be drawn.
In our society, we expect those tests have been done and done right, minimizing the corrupting influences of politics or bias. We want medicines that work and are safe. The same goes for so much affecting our everyday lives, from the products we buy, to the food we eat, to the way we choose to engage in sex. We assume someone, somewhere, did the experiment. Condoms, it turns out, really do reduce the transfer of HIV/AIDS. Millions in South Africa died believing otherwise, relying on the state-sanctioned opinions of the former Health Minister, instead of the results of experiment.
This brings us to the role of experiment in public policy and decision making. Having spent 30 years applying the scientific method to problems in the public and private sectors, I am still surprised by how much can be accomplished through a simple experiment, and how little such experiments are used. The potential for improved social well-being as well as increased effectiveness and efficiency in industry, is mindboggling. Yet, these are a wasteland of pseudo-scientific nonsense that costs us jobs, money and lives. Why?
One reason is a basic mistrust of science. I can recall making a presentation before a committee explaining why health care system capacity was constrained by Little’s Law — the fundamental law governing system and process capacity – when the committee leader proclaimed; ” I just don’t believe in that science stuff.” No word on what ‘stuff’ will be used instead but such cases are, thankfully, rare.
More common is confusion concerning what constitutes experiment or evidence. For example, industry is currently obsessed with pursuing ‘best practice’, examining how others do things and copying what seems to work. This assumes the experiment has been done by someone else, but in reality, it hasn’t been done at all. This makes best practice in education, health services, purchasing, project management, human resource management, and operations, exercises in implementing the opinions and mistakes of others.
Best practice recommendations litter the Interim Report on Restructuring of the Niagara Health System. One calls for closing a number of older, smaller hospitals and replacing these with fewer, newer, larger hospitals because in part, “All evidence points to consolidation of expertise as a key determinant of quality health care.” Nonsense. What evidence? Those benefitting from the building of new hospitals may be of the opinion that consolidation is a key determinant of quality, but, that’s not evidence, its marketing, and it will make conditions in Niagara worse.
Testing to see if something works or not inevitably yields things that don’t, making experimentation anathema to some. When was the last time you heard a Minister stand up and say; “We screwed up!”? This produces a reluctance to experiment, killing innovation while producing some strange definitions of success.
Some people are taking notice
This month, Alberta Health Services (AHS) is leading a conference trumpeting the many successes attributable to their new ‘improvement model’, including reducing emergency department wait times. Albertan’s will be excused for failing to notice. Not surprising perhaps, as AHS’s improvement model, and conference sponsorship, comes courtesy of those charged with the ‘cultural transformation’ at BP, after the Texas City Refinery explosion in 2005. Success in transforming BP’s culture, to one concerned with safety and the environment, was proclaimed in 2009, a year before the Deepwater Horizon disaster. It’s still being proclaimed today.
There is a difference between BS and a B.Sc. Some people are taking notice. Evidenced-based management initiatives are springing up in North America and Europe, taking evidenced-based medicine as their model. The recently published Geek Manifesto by Mark Henderson outlines an aggressive agenda of demanding evidence, based on experiment, for public policy and business decisions. Perhaps this is the result of a new generation refusing to accept the opinion of their elders.
Let’s hope so.
Robert Gerst is a Partner in Charge of Operational Excellence and Research and Statistical Methods at Converge Consulting Group Inc. He is author of The Performance Improvement Toolkit: The Guide to Knowledge-Based Improvement and numerous articles in peer-reviewed journals.
