Dr. Ted Grover’s research shows how AI, with proper oversight, can efficiently integrate patient voices into health care decision-making.
Understanding patient experiences and preferences is essential for shaping health care that truly meets people's needs. Researchers have typically gathered this information through interviews or focus groups, listening carefully to what patients say about the risks and benefits of new health products, services, and technologies. This approach provides deep insight into the perceived benefits and risks of health care innovations, but analyzing these conversations takes considerable time and effort.
A new study, published in PLOS Digital Health, shows that artificial intelligence (AI) can help. The Regulatory Science Lab, led by Dr. Dean A. Regier, a senior scientist at BC Cancer, associate professor in the School of Population and Public Health at the University of British Columbia (UBC), and the director of the Academy of Translational Medicine in UBC's Faculty of Medicine, and lead author, Dr. Ted Grover, a research methodologist at BC Cancer alongside fellow BC Cancer researchers Emanuel Krebs, Deirdre Weymann, and Morgan Ehman, applied an open-source large language model called Hermes-3-Llama-3.1-70B to perform thematic analysis of patient focus group transcripts collected from a prior published study examining patient concerns and expectations for secured health data-sharing platforms. Using optimized prompts, the model identified patterns and themes in patient preferences, producing results similar in content and style to human analysis.
"Patients have important things to say about their care, and we have an obligation to listen carefully. This study shows that AI can help us do that at a scale not previously practical. With the right guidance and oversight, AI can accelerate how we bring patient voices into health decisions," said Dr. Grover.
The study compared themes generated by AI to those produced by human researchers, using advanced methods to measure semantic similarity, lexical diversity, and reading grade levels. The best-performing model framework showed moderate-to-high agreement with human-analyzed themes and even exceeded previously published benchmarks for similarity. While some prompts produced less accurate or irrelevant findings, the results show that AI can reliably support the interpretation of patient experiences when used thoughtfully and with proper oversight.
These findings have important implications for patient-centred health care. Faster thematic analysis means health care teams can more quickly integrate patient perspectives into product development, service design and clinical decision-making. The study also highlights the importance of combining AI tools and human expertise to ensure that patient voices are accurately interpreted and applied.
This work, funded by the UBC AI and Health Network, demonstrates how UBC and BC Cancer researchers are thoughtfully exploring artificial intelligence to improve evidence generation and strengthen patient‑centred health care.