Who was at fault here? UK Woman left to die after computer decision support blunder

Who was at fault here? Those who modified the fall height parameters, those who designed the decision support system such that it could override life threatening problems based on a single parameter, endusers, their managers, or all?

Or was the problem the syndrome of inappropriate confidence in computers (SICC syndrome)?

I opine all of the above, in this cautionary tale:

Woman left to die after 999 ambulance blunder
By Laura Donnelly, Health Correspondent
Telegraph.co.uk
Published: 9:00AM GMT 21 Mar 2010

An investigation into a woman�s death has exposed a catastrophic decision by ambulance chiefs which may have cost hundreds of lives.

The blunder arose when call centre staff were not warned of flaws with a computer system that prioritises emergencies before dispatching ambulances.

Bonnie Mason, 58, fell down the stairs and died from a head injury after 999 controllers in Suffolk failed to identify her situation as �life-threatening�.

An investigation by The Sunday Telegraph has uncovered a critical danger placed in the software used by most ambulance services. For years, 999 calls in life-threatening situations like Mrs Masons�s were accidentally �downgraded�, with call handlers told not to send the most urgent response.

While some services spotted the risk, ordering operatives to override the computer�s orders manually, five of England�s 12 ambulance trusts did not allow call handlers to upgrade such calls [A belief that "the computer is omniscient" seems to be the only basis for such an exclusion of human judgment - ed.] They include the East of England ambulance service, which covers Suffolk and which only identified the risk after Mrs Mason�s death.

The danger in the system was created by the country�s most senior ambulance officials as they altered the program used by most control centres in an attempt to manage demand for 999 services.

Most ambulance services use an international computerised system designed in America. In the US version, a fall of more than 6ft receives the maximum priority response. However, the government committee which governs its use in this country decided that such cases should be deemed less urgent [what were they thinking? - ed], and excluded from an eight minute category A target response time.

In doing so, they created a potentially lethal flaw in the system. It meant that if a call involved a fall of more than 6ft it was designated a lower priority � a category B response � despite the presence of life-threatening conditions which were supposed to receive the most urgent category A response.

[NOTE: If the other life threatening conditions were ignored by the computer system after its "first-pass" look at the height of the fall, then who actually created the most severe flaw is unclear to me, those who altered the parameter or those who designed the overall decision support logic - ed.]

As a result, Mrs Mason lay unconscious for more than 38 minutes. The first ambulance sent to her home in the village of Eye, Suffolk, was diverted to attend to a drunk woman who had fallen on a pavement 22 miles away in Thetford, Norfolk. Because the inebriated woman had fallen at ground level, her situation was prioritised over that of Mrs Mason [perhaps because their was no entry of a "height of fall" - ed.], who was close to death by the time paramedics arrived. The East of England ambulance service, which also covers Bedfordshire, Cambridgeshire, Essex, Hertfordshire and Norfolk, said its operatives were instructed never to �override� the advice of the automated system.

Ambulance dispatchers instructed to "never override the advice of the automated system?" Simply stunning if true.

Read the whole story at the link above.

On another note: government committees have rarely worked well in domains where critical thinking is essential. (I can't wait for the comparative effectiveness committees using flawed data from flawed EMR's to start their work. I'd written about that issue here.)

Finally, "Our policy is to always trust the computer" is not a way to run life-critical healthcare services. Ever.

-- SS

Read More

The Syndrome of Useless Information

I occasionally elevate comments and replies to the level of full posts if I feel they better illustrate and clarify significant points I raise.

In my Jan. 9, 2010 post "Does the CEO of Google Use Google? - And: Platform, Platform, Who's Got The Platform?" I lamented the intrusion of ill-informed, reductionistic, "database platform"-centric views of non-clinical IT personnel into healthcare.

I received the following feedback from an anonymous commenter:

Anonymous wrote:

Or then again, there's this:

"Non-physicians may reach correct diagnoses by using Google: a pilot study."

http://www.ncbi.nlm.nih.gov/pubmed/19130327

"Researchers found that almost six-in-10 difficult cases can be solved by using the world wide web as a diagnostic aid."

http://www.dailymail.co.uk/news/article-415562/Doctors-using-Google-diagnose-illnesses.html

I think this post [i.e., my HC Renewal post about Google's CEO and "platforms" - ed.] misses the larger point that Gawande and Schmidt were addressing (and to some extent agree on) - the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.

The future will no doubt involve ever more sophisticated and useful clinical DSS. Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.

January 16, 2010 10:03:00 AM EST

Anonymous apparently meant these article references to exemplify the coming Age of Cybernetic Miracles in medicine.

My response to this anonymous feedback was placed in the comment thread of the "Does the CEO use Google" post and covered a number of bases.

I've reproduced my response, and added additional explanatory notes not in my original response in bold red italics below:

MedInformaticsMD wrote:

Re: Anonymous Jan 16 @ 10:03 AM-

The article [the first, at nlm- ed.] you cite reports the following:

Non-physicians may reach correct diagnoses by using Google: a pilot study.

OBJECTIVE: We endeavoured to determine whether individuals who are not physicians are likely to arrive at correct diagnoses [note the stated objective carefully -- "are likely to arrive at correct diagnoses" - ed.] by using Internet resources.

METHODS: In this prospective study four non-physicians used Google to search for diagnoses. They reviewed the 26 diagnostic cases presented in the case records of the New England Journal of Medicine during 2005; they were blind to the correct diagnoses. The main measurement was the percentage of correct diagnoses arrived at by non-physicians by using Google. The diagnostic success of the four non-physicians was compared to that of four young physicians.

RESULTS: The average diagnostic success of non-physicians was 22.1% (95% confidence interval [CI] 4.5-39.7%). There was no statistically significant difference between the non-physicians regarding this outcome (p = 0.11). They took 8.9 +/- 6.7 (mean +/- standard deviation) minutes for case record reading and 17.4 +/- 7.9 minutes for Google searching per case. Non-physicians performed worse than physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).

CONCLUSION: Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search. Doctors should realise that patients may assume a more active role in their health decision-making process and take this development into consideration in physician-patient interaction.

--------------------------

This article suffers from what can be termed "the syndrome of useless information."

[It is, in fact, a failure or negative finding with regard to its stated objective - ed.]

Let's see:

The average diagnostic success of non-physicians was 22.1%

They omit the opposite semantic: that the google-armed nonphysicians got more than three quarters of the diagnoses WRONG. Will you trust your grandmother to them?

Non-physicians performed worse than [young, a.k.a. inexperienced] physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).

Young, inexperienced physicians (trainees or residents?) arriving at 51% of the diagnoses correctly using ONLY google (not, for example, Harrison's, and other standard medical texts that would not be highly usable to most non-physicians lacking background to fully understand it) is not unexpected.

What is surprising is that the article omitted an essential control group: *experienced* physicians. It's not that they're hard to find.

The article concludes:

Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search.

Occasionally reach correct diagnoses? How about "most often reach incorrect diagnoses?" Cf. a broken clock is still occasionally correct.

[I repeat, the article is, in fact, a failure or negative finding with regard to its stated objective of "determining whether individuals who are not physicians are likely to arrive at correct diagnoses." They are in fact unlikely to get it right, with only a 22% hit rate, although the authors appear to have de-emphasized that fact. This will only make physicians' work harder as they "take this development into consideration", i.e., deal with patients armed with search engine-gleaned misdiagnoses - ed.]

I do not view this article as revealing anything of practical value other than perhaps the dangers of allowing non-physicians armed with search engines to think they can perform medical diagnosis to any meaningful extent.

Anonymous wrote: the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.

Reductionist views of non-medical IT personnel spouting off about 'database platforms' will not advance the science or the art of medicine.

Also, yes, IT is a tool of science; however, it and its designers and implementers facilitate science; the enablers of science are: scientists, using their insight, creativity, ingenuity, experience and expertise.

Concerning the second article [at dailymail.co.uk - ed.], I note:

"But they [the authors] stress the efficiency of the search and the usefulness of the retrieved information depend on the searchers' knowledge base." [The 'searchers' in this case were apparently - experienced physicians - ed.]

In other words, search engines can facilitate experts. That is not exactly new knowledge.

The 'art' of medicine, i.e., judgment, likewise is both indispensible, and irreproducible via "database platforms." Perhaps one day with advances in cognitive computing we will get to that point, but at present we can't even do as well as a cat. Note the statement from the IBM P.I. that "there are no computers that can even remotely approach the remarkable feats the mind perform."

Until we get there, I think it's not unreasonable to hold off on non-clinicians touting reductionist information retrieval-centric views.

Anonymous wrote: Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.

You must not have read my original post. It is advances in information science (i.e., in informatics) that will provide those advances, not advances in [information] technology. IT is a tool; information science is an activity of the mind.

Your statements clearly demonstrate a conflation of information technology and information science.

Computers facilitate information science research, but they are certainly not its sine qua non.

I should also add that the second article referenced by "Anonymous" refers to "obscure conditions such as Cushing's syndrome." I'm not exactly sure to whom Cushing's disease is an obscure disease. A non-medical IT person, perhaps?

Now, back to reality. Computers serve as aids to clinicians, when the IT is "done well." However, there is no substitute -- except in sci fi -- for expertise.

Medical expertise can only come from ~4 years of premed and then 4+ years of hard medical study in a wide variety of preclinical and clinical sciences leading to the M.D. degree, 3-4 years of postgraduate residency, and often 2+ additional years of postdoctoral fellowship training beyond residency for specialists (it should be noted, unfortunately, that there are no additional medical degrees beyond M.D., although residency and fellowship training often makes pursuing a Ph.D. in most fields seem like a cakewalk). Add to all that years of additional clinical experience in actual practice.

I am quite disturbed that this study and experience has become so casually regarded outside of medicine, for example in the IT sector. I think it reflects poorly on the IT culture specifically, and on our current culture as a whole.

Medical judgment is borne of that study and experience, not of healthcare computer dabbling typical of the business IT/MIS world in hospitals, and the all-too-often technical bachelor's degrees held by leaders in that domain. Google CEO Schmidt at least is a true computer scientist with a doctorate in CS; that does not, however, qualify him to comment on medical and medical informatics-related issues as in my prior post.

Stated frankly: he and many others with similar views are venturing far outside their competencies and as a result are talking nonsensically.

Incidentally, I believe it's time to move on from continually responding to the non-medical, IT-personnel proffered platformorrhea on how IT databases, decision support, artificial intelligence etc. will transform medicine from an art to a science, revolutionize medicine, and similar positions trafficked for at least several decades now.

They are largely manifestations of the Syndrome of Inappropriate Overconfidence in Computing (SICC syndrome), a term I coined in the 1990's and described is some detail here in a post entitled "A 21st Century Plague? The Syndrome of Inappropriate Over-Confidence in Computing." At that post I stated that:

I, for one, would welcome a cessation of claims that IT will "revolutionize" any field that depends primarily on cognition, such as biomedicine, and a return to more temperate attitudes instead of the almost bellicose grandiosity about HIT we see today. That is to say, that HIT - with proper contributions from the aforementioned specialties [e.g., social science, social informatics, biomedical informatics, HCI, etc.] will facilitate better health care, not "revolutionize" it.

In the future, similar comments as those from "Anonymous" will simply get referred back to the post you are now reading and the bolded one on SICC above.

Finally, I will not hold my breath for Emergency Medical Hologram Mark I, or R2D2 and C3PO to appear anytime soon.

-- SS

Read More