An online assessment framework for reliable generative AI-driven recommender apps in chronic disease management

Chronic conditions are becoming a serious global and national health problem. Recommendation systems play an important role in supporting patients in managing their long-term health issues. They generally rely on expert rules or machine learning models to provide health advice. Recently, generative AI tools, such as ChatGPT, have become a popular focus of research. In healthcare, they show strong potential to facilitate the process of generating health-related advice without the need for predefined rules or training data. Yet, their reliability remains a serious concern. 

This project aims to first explore current techniques such as fine-tuning, model alignment, prompt engineering and Retrieval Augmented Generation (RAG) to improve reliability of generated recommendations for two cases of chronic conditions (low back pain and diabetes). It will then co-create a theory-driven, evidence-based assessment rubric that will form the foundation of an online framework for assessing the reliability of recommendations produced by generative AI tools for chronic disease management.