Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More<\/em><\/p>\n\n\n\n Even as large language models (LLMs) become ever more sophisticated and capable, they continue to suffer from hallucinations: offering up inaccurate information, or, to put it more harshly, lying.\u00a0<\/p>\n\n\n\n This can be particularly harmful in areas like healthcare, where wrong information can have dire results.\u00a0<\/p>\n\n\n\n Mayo Clinic, one of the top-ranked hospitals in the U.S., has adopted a novel technique to address this challenge. To succeed, the medical facility must overcome the limitations of retrieval-augmented generation (RAG). That\u2019s the process by which large language models (LLMs) pull information from specific, relevant data sources. The hospital has employed what is essentially backwards RAG, where the model extracts relevant information, then links every data point back to its original source content.\u00a0<\/p>\n\n\n\n Remarkably, this has eliminated nearly all data-retrieval-based hallucinations in non-diagnostic use cases \u2014 allowing Mayo to push the model out across its clinical practice.<\/p>\n\n\n\n \u201cWith this approach of referencing source information through links, extraction of this data is no longer a problem,\u201d Matthew Callstrom, Mayo\u2019s medical director for strategy and chair of radiology, told VentureBeat. <\/p>\n\n\n\n Dealing with healthcare data is a complex challenge \u2014 and it can be a time sink. Although vast amounts of data are collected in electronic health records (EHRs), data can be extremely difficult to find and parse out.\u00a0<\/p>\n\n\n\n Mayo\u2019s first use case for AI in wrangling all this data was discharge summaries (visit wrap-ups with post-care tips), with its models using traditional RAG. As Callstrom explained, that was a natural place to start because it is simple extraction and summarization, which is what LLMs generally excel at.\u00a0<\/p>\n\n\n\n \u201cIn the first phase, we\u2019re not trying to come up with a diagnosis, where you might be asking a model, \u2018What\u2019s the next best step for this patient right now?\u2019,\u201d he said.\u00a0<\/p>\n\n\n\n The danger of hallucinations was also not nearly as significant as it would be in doctor-assist scenarios; not to say that the data-retrieval mistakes weren\u2019t head-scratching.\u00a0<\/p>\n\n\n\n \u201cIn our first couple of iterations, we had some funny hallucinations that you clearly wouldn\u2019t tolerate \u2014 the wrong age of the patient, for example,\u201d said Callstrom. \u201cSo you have to build it carefully.\u201d\u00a0<\/p>\n\n\n\n While RAG has been a critical component of grounding LLMs (improving their capabilities), the technique has its limitations. Models may retrieve irrelevant, inaccurate or low-quality data; fail to determine if information is relevant to the human ask; or create outputs that don\u2019t match requested formats (like bringing back simple text rather than a detailed table).\u00a0<\/p>\n\n\n\n While there are some workarounds to these problems \u2014 like graph RAG, which sources knowledge graphs to provide context, or corrective RAG (CRAG), where an evaluation mechanism assesses the quality of retrieved documents \u2014 hallucinations haven\u2019t gone away.\u00a0<\/p>\n\n\n\n This is where the backwards RAG process comes in. Specifically, Mayo paired what\u2019s known as the clustering using representatives (CURE) algorithm with LLMs and vector databases to double-check data retrieval.\u00a0<\/p>\n\n\n\n Clustering is critical to machine learning (ML) because it organizes, classifies and groups data points based on their similarities or patterns. This essentially helps models \u201cmake sense\u201d of data. CURE goes beyond typical clustering with a hierarchical technique, using distance measures to group data based on proximity (think: data closer to one another are more related than those further apart). The algorithm has the ability to detect \u201coutliers,\u201d or data points that don\u2019t match the others.\u00a0<\/p>\n\n\n\n Combining CURE with a reverse RAG approach, Mayo\u2019s LLM split the summaries it generated into individual facts, then matched those back to source documents. A second LLM then scored how well the facts aligned with those sources, specifically if there was a causal relationship between the two.\u00a0<\/p>\n\n\n\n \u201cAny data point is referenced back to the original laboratory source data or imaging report,\u201d said Callstrom. \u201cThe system ensures that references are real and accurately retrieved, effectively solving most retrieval-related hallucinations.\u201d\u00a0<\/p>\n\n\n\n Callstrom\u2019s team used vector databases to first ingest patient records so that the model could quickly retrieve information. They initially used a local database for the proof of concept (POC); the production version is a generic database with logic in the CURE algorithm itself.<\/p>\n\n\n\n \u201cPhysicians are very skeptical, and they want to make sure that they\u2019re not being fed information that isn\u2019t trustworthy,\u201d Callstrom explained. \u201cSo trust for us means verification of anything that might be surfaced as content.\u201d\u00a0<\/p>\n\n\n\n The CURE technique has proven useful for synthesizing new patient records too. Outside records detailing patients\u2019 complex problems can have \u201creams\u201d of data content in different formats, Callstrom explained. This needs to be reviewed and summarized so that clinicians can familiarize themselves before they see the patient for the first time.\u00a0<\/p>\n\n\n\n \u201cI always describe outside medical records as a little bit like a spreadsheet: You have no idea what\u2019s in each cell, you have to look at each one to pull content,\u201d he said.\u00a0<\/p>\n\n\n\n But now, the LLM does the extraction, categorizes the material and creates a patient overview. Typically, that task could take 90 or so minutes out of a practitioner\u2019s day \u2014 but AI can do it in about 10, Callstrom said. \u00a0<\/p>\n\n\n\n He described \u201cincredible interest\u201d in expanding the capability across Mayo\u2019s practice to help reduce administrative burden and frustration.\u00a0<\/p>\n\n\n\n \u201cOur goal is to simplify the processing of content \u2014 how can I augment the abilities and simplify the work of the physician?\u201d he said.\u00a0<\/p>\n\n\n\n Of course, Callstrom and his team see great potential for AI in more advanced areas. For instance, they have teamed with Cerebras Systems to build a genomic model that predicts what will be the best arthritis treatment for a patient, and are also working with Microsoft on an image encoder and an imaging foundation model.\u00a0<\/p>\n\n\n\n Their first imaging project with Microsoft is chest X-rays. They have so far converted 1.5 million X-rays and plan to do another 11 million in the next round. Callstrom explained that it\u2019s not extraordinarily difficult to build an image encoder; the complexity lies in making the resultant images actually useful.\u00a0<\/p>\n\n\n\n Ideally, the goals are to simplify the way Mayo physicians review chest X-rays and augment their analyses. AI might, for example, identify where they should insert an endotracheal tube or a central line to help patients breathe.\u00a0\u201cBut that can be much broader,\u201d said Callstrom. For instance, physicians can unlock other content and data, such as a simple prediction of ejection fraction \u2014 or the amount of blood pumping out of the heart \u2014 from a chest X ray.<\/p>\n\n\n\n \u201cNow you can start to think about prediction response to therapy on a broader scale,\u201d he said.\u00a0<\/p>\n\n\n\n Mayo also sees \u201cincredible opportunity\u201d in genomics (the study of DNA), as well as other \u201comic\u201d areas, such as proteomics (the study of proteins). AI could support gene transcription, or the process of copying a DNA sequence, to create reference points to other patients and help build a risk profile or therapy paths for complex diseases.\u00a0<\/p>\n\n\n\n \u201cSo you basically are mapping patients against other patients, building each patient around a cohort,\u201d Callstrom explained. \u201cThat\u2019s what personalized medicine will really provide: \u2018You look like these other patients, this is the way we should treat you to see expected outcomes.\u2019 The goal is really returning humanity to healthcare as we use these tools.\u201d\u00a0<\/p>\n\n\n\n But Callstrom emphasized that everything on the diagnosis side requires a lot more work. It\u2019s one thing to demonstrate that a foundation model for genomics works for rheumatoid arthritis; it\u2019s another to actually validate that in a clinical environment. Researchers have to start by testing small datasets, then gradually expand test groups and compare against conventional or standard therapy.\u00a0<\/p>\n\n\n\n \u201cYou don\u2019t immediately go to, \u2018Hey, let\u2019s skip Methotrexate\u201d [a popular rheumatoid arthritis medication], he noted.\u00a0<\/p>\n\n\n\n Ultimately: \u201cWe recognize the incredible capability of these [models] to actually transform how we care for patients and diagnose in a meaningful way, to have more patient-centric or patient-specific care versus standard therapy,\u201d said Callstrom. \u201cThe complex data that we deal with in patient care is where we\u2019re focused.\u201d<\/p>\n Daily insights on business use cases with VB Daily<\/strong><\/p>\n If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n \n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here.\n\t\t\t\t<\/p>\n An error occured.<\/p>\n<\/p><\/div>\n
\n<\/div>Accounting for every single data point<\/h2>\n\n\n\n
Referencing every data point<\/h2>\n\n\n\n
\u2018Incredible interest\u2019 across Mayo\u2019s practice<\/h2>\n\n\n\n
Tackling more complex problems with AI<\/h2>\n\n\n\n
![\"\"\/](\"https:\/\/venturebeat.com\/wp-content\/themes\/vb-news\/brand\/img\/vb-daily-phone.png\")
\n\t\t\t\t<\/div>\n<\/p><\/div>\n<\/div>\t\t\t<\/div>\r\n