What about Watson?

Watson, IBM’s supercomputer for artificial intelligence tasks, has in the past given advice deemed as dangerous in the treatment of cancer patients. The question that then arises is whether this is due to Watson, or due to the use of Watson. 

“This product is a piece of s–” is the characterization that one of the doctors at Jupiter Hospital gave to Watson. The comments are part of IBM internal documents reported by Stat News. In 2012, IBM partnered with the Memorial Sloan Kettering Cancer Center in New York to launch a project to use Watson in diagnosing and suggesting treatments for cancer patients.

Expectations do not come true

However, the high expectations with regard to the help that Watson could provide, have so far been rather disappointing. The supercomputer regularly gave wrong and even dangerous advice. For example, in a patient with severe bleeding Watson had prescribed a medication that would have significantly worsened the bleeding.

A presentation by Andrew Norden, who was responsible for Watson’s deployment in healthcare at IBM, shows that there was not only e a lot of dissatisfaction with the performance of Watson for Oncology, but that there was also a lot to complain about with regard to Watson’s training. By the way, Norden left IBM not long after giving this presentation.

Wrong training as a possible cause

Watson was initially planneed to be trained for Oncology, the system would be fed with large amounts of data. The idea was that with artificial intelligence new insights would emerge from analysis of the data. According to Norden’s presentation, however, the system was not fed with real patient data, but with hypothetical data provided by only a handful of doctors. As a result, the system became dependent on the treatment preferences of those specific doctors.

IBM has no intention of stop its development of Watson for Oncology. The system is still learning and has now provided advice on the treatment of 84,000 patients according to an IBM spokesperson against Gizmodo.