The Care2Report research program: Engineering Generative AI for Automated Medical Reporting, and beyond.

Speaker: Sjaak Brinkkemper, Utrecht UniversityAbstract Since

November 2022, we have been witnessing the emergence of large language models (LLMs) and generative pre-trained transformers (GPT) as easy-to-use artificial intelligence (AI) technologies, that have produced an abundance of smart applications using AI components.The administrative burdens in various social domains (healthcare, public administration, customer service, law enforcement) motivated the Care2Report research program at Utrecht University to use a GPT to generate reports of professional conversations. The innovative scientific domain of AI engineering requires experimenting with architectures of an end-to-end solution for structured summarization of human activities.We will present and demonstrate the Care2Report system that allows summarizing medical consultations in real time based on audiovisual input in healthcare. We discuss the audio transcription, prompt engineering, action recognition, and report generation that are part of automated medical reporting. We end with an outlook on the research questions and the social impact of conversational applications.