A variant of the language model behind Google’s AI chatbot Bard, was designed and launched for the purpose of answering health and medical questions. An improved version of the model, named Med-PaLM 2, is currently being tested at medical hospitals, including the Mayo Clinic research hospital.
Med-PaLM became the first AI system to exceed the US medical licensing exam’s pass mark of 60%. While Med-PaLM has not been capable of generating comprehensive answers as compared to clinicians, the technology has been found to maintain accuracy and safety in its responses. “We are still learning,” Google quoted about the system’s efficiency.
The company has now released Med-PaLM 2, a new and more improved version of Med-PaLM. Google further revealed performance results of the model, that showed 85% accuracy on the medical exam benchmark in research. “This performance is on par with expert test takers,” Google confirmed. This still does not suggest perfect accuracy of the model, of course - there’s plenty of room for errors, as is the case with other language models.
Curated with medical expert demonstrations, Google believes Med-PaLM 2 to be a useful technology especially for countries that have limited access to medical experts. The company plans to continue working with researchers and experts from the healthcare world to advance its work.