Counterpart Health, MEDITECH, and Suki are using Vertex AI Search for healthcare to improve clinicians’ access to information and gain a more comprehensive view on patient health
Google Cloud announced new generative AI (gen AI) capabilities in Vertex AI Search for healthcare, including a new feature called Visual Q&A that searches tables, charts, and diagrams, and the availability of Gemini 2.0 as one of the models now within the offering. Vertex AI Search for healthcare’s new, multimodal search capabilities can help organizations give doctors, nurses, and others a more comprehensive view of patient health.
“We’re helping clinicians to work more efficiently with new multimodal capabilities embedded in Vertex AI Search for healthcare,” said Aashima Gupta, global director of Healthcare Strategy & Solutions, Google Cloud. “Multimodal analysis processes diverse sources of patient data, like medical images and genetic information, for a more comprehensive understanding and improved decision-making.”
Multimodal AI: A comprehensive view of patient health
Multimodal AI refers to technology that is capable of processing and integrating information from diverse sources – such as images, videos, and text. This capability is especially important in healthcare, where nearly 90% of healthcare data is in image form – such as x-rays, scans, or photos. To give healthcare professionals a comprehensive understanding of a patient’s health, AI models can combine data from medical images, patient history, genetic information, and more. With these richer data inputs, multimodal AI can help providers in making more accurate diagnoses, designing more personalized treatment plans, and ultimately improving patient outcomes.
Also Read: GenBio AI Appoints Ziv Bar-Joseph as Co-Founder and Chief Scientific Officer
Vertex AI Search for healthcare goes multimodal
Vertex AI Search for healthcare is a specialized tool within Google Cloud‘s Vertex AI platform that helps developers build better assistive technology for searching and retrieving information from complex medical data within health records, medical documents, and other healthcare data sources. Now with Visual Q&A and Gemini 2.0, the search technology has even greater capabilities:
- Visual Q&A enables Vertex AI Search for healthcare to receive images such as tables, charts, or diagrams directly as an input – instead of taking the image and first converting it into text. For example, it could receive a diagram of a patient’s feet indicating where they have an injury, alongside a checkbox from a form the patient filled out indicating if they are wearing therapeutic shoes, and any other symptoms indicated on the form. Visual Q&A can also analyze medical forms where information is entered into tables – and when this information is a relevant part of a search, it will include it in the findings for the clinician.
- Gemini 2.0 helps Vertex AI Search for healthcare deliver faster, more accurate answers. For example, Gemini 2.0 Flash outperforms 1.5 Pro on key benchmarks, at twice the speed. This performance boost accelerates the retrieval and processing of multimodal data, including visual medical content like charts, tables, and images, enabling faster, more informed decision-making.
Source: PRNewswire
