Ebook: Context Sensitive Health Informatics: AI for Social Good

Healthcare delivery is undergoing a profound transformation driven by two interconnected trends. The first is the digitization of the healthcare system, which has expanded opportunities for citizens and patients to access and contribute their own data – including patient-reported outcomes – and use digital technologies to manage their health and interact with healthcare professionals. This shift is often referred to as the healthcare system’s new digital front door, and redefines how healthcare services are accessed and delivered.

Simultaneously, the rise of artificial intelligence (AI) in healthcare is being fuelled by the vast amount of data generated by this digitization. AI is increasingly being used to support patients in managing their own health, as well as to assist healthcare professionals in the making of complex decisions. But as healthcare moves beyond physical locations and into digital spaces, critical questions emerge.

∙ How can we understand and design for a healthcare system no longer confined to hospitals and clinics?

∙ How can AI be harnessed to drive meaningful and lasting social impact, ensuring equal access and improved outcomes?

∙ What are the ethical considerations, risks, and challenges associated with AI-driven solutions, and how can they be responsibly addressed?

∙ What insights can we draw from recent advancements, and how can they inform the future of AI in health and beyond?

The theme of CSHI 2025, ‘AI for Social Good’, reflects these pressing questions, focusing on the role of AI and digital healthcare in promoting health, well-being, and equality. While some of the papers in this volume engage directly with this theme, others contribute to broader discussions on the design, implementation, and impact of socio-technical systems in healthcare.

This volume brings together cutting-edge research across the following themes:

∙ AI-driven decision support and usability in healthcare

∙ EHR implementation, usability, and evolution

∙ Human-centred AI and health informatics

∙ Clinician experiences, health-data interpretation, and medication safety

∙ AI in simulation, screening, and clinical trials.

The discussions and findings presented in these proceedings reflect the depth and diversity of research in this rapidly evolving field. We hope they will serve as a valuable resource for researchers, practitioners, and policymakers, fostering further dialogue and collaboration on the future of AI in healthcare.

Retrieval Augmented Generation has been shown to improve the output of large language models (LLMs) by providing context to the question or scenario posed to the model. We have tried a series of experiments to understand how best to improve the performance of the native models. We present the results of each of several experiments. These can serve as lessons learned for scientists looking to improve the performance of large language models for medical question answering tasks.

This paper presents findings from a formative process evaluation of the implementation of a commercial AI application in a large Norwegian hospital trust. The study identified three key areas critical for successful AI adoption: (1) Implementation strategies, (2) Implementation methods, and (3) Effects and values. We examine how the hospital developed a bottom-up implementation strategy, emphasizing a “learning by doing” approach. The iterative implementation method, refined across multiple hospitals, prioritized extensive stakeholder involvement and rigorous quality assurance. This approach facilitated key outcomes, including enhanced digital maturity, early benefits realization, and strengthened user trust and engagement—essential factors for long-term AI adoption.

The integration of generative AI in digital health tools introduces opportunities and challenges to assist healthcare practices and improve health outcomes. A rapid review was performed to investigate the user experience of generative AI in digital health after the introduction of ChatGPT in 2022. Ten papers were included in the review. Findings show three types of generative AI systems (ChatGPT-specific, tailored, and custom) that were developed and studied on user experiences, measurements and outcomes differed between studies, and novel future research directives are needed to ensure good user experiences, trust, and effectiveness of generative AI in digital health.

Explainable Artificial Intelligence (XAI) offers promising advancements in enhancing transparency and usability of AI-based Clinical Decision Support Systems (CDSS) in healthcare settings. These tools aim to improve clinical outcomes by assisting with diagnosis, treatment planning, and risk prediction. However, integrating XAI into clinical workflows requires effective involvement of healthcare professionals to ensure that the explanations provided by these tools are comprehensible, relevant, and actionable. This scoping review aimed to investigate how (potential) end users were involved in the design and development of XAI-based CDSS for hospitals. A systematic search of Medline, Embase, and Web of Science identified 11 studies meeting the inclusion criteria. Interviews and focus groups, mainly with physicians, were common, while some included nurses and developers. Four of the 11 studies engaged users across multiple stages, from pre-design to prototype testing, and specifically tested different explanation techniques with end-users. A quality assessment of papers found some studies had unclear recruitment strategies and insufficiently detailed analyses. Future work should engage end-users early in the design process, include health professionals with diverse experiences and backgrounds, and test explanation techniques to ensure appropriate methods that align with cognitive processes are chosen.

We developed a doctor-facing chatbot named SCAI. We aimed to evaluate response speed and user satisfaction. Ten questions were used to test the response speed of the SCAI chatbot. The response time was long, but overall the participants were satisfied with SCAI.

We evaluated the performance of Semantic Clinical Artificial Intelligence (SCAI, pronounced Sky), a large language model (LLM) knowledge resource through usability testing. This pretest-intervention-posttest mixed-methods user interface (UI) design study investigates usability to determine whether the LLM provides a more comprehensive, efficient, and enhanced user-friendly means of delivering end user information as compared to using primary sources of information from the Internet (Web). Our analysis focused on assessing the LLM’s efficiency and usability in helping users arrive at accurate and reliable outcomes, to ultimately determine its value as an innovative tool for packaging and presenting information. Usability test sessions were conducted using the cognitive walkthrough approach, via Zoom. Participants were asked to respond to scenarios using only the LLM, and then only the web, and vice versa. These sessions were followed by user feedback sessions where participants rated their experiences and responded to open-ended questions related to the overall usability and satisfaction with SCAI.

The implementation of electronic health record (EHR) suites is a complex process that involves configuring the EHR suite for local healthcare practices. In this study, we investigate the process of configuring Epic’s EHR suite for Norwegian healthcare practices. From the point of view of the interviewed subject-matter experts (SMEs) and super users, this EHR implementation has struggled with managing and processing the input from the highly prioritized user participation in the configuration process. In addition, the SMEs have been in a difficult in-between role. These struggles and difficulties are evident in three main issues. First, SMEs and Epic’s developers tended to negotiate rather than collaborate. Second, SMEs tended to contribute their personal expertise rather than represent their peer groups. Third, the specialty-specific groups for doing the configuration work tended to optimize the EHR for each specialty rather than for teamwork.

Electronic Health Record (EHR) systems continually evolve to meet the demands of user organizations, national authorities, and end users. Finland, a digital health pioneer, encountered challenges like system fragmentation and interoperability despite early EHR adoption. The rollout of national health information exchange (Kanta) services sought to address these issues but also introduced new usability concerns. In national cross-sectional usability surveys conducted among physicians in 2010, 2014, 2017, and 2021, we asked respondents to select five most pertinent development targets from a predefined list. This study analyzed specialist physicians’ responses by study year and specialty group: surgical, nonoperative medical, psychiatry, general practice (GP), occupational healthcare (OH), and anesthesiology and intensive care. Compared to earlier years, unexpected EHR downtime and slowness became less urgent across all specialties in 2021. In 2010, summary views were a high priority (36–61% in other groups; 18% in OH). After EHR development, this priority decreased in 2014–17 (18–26%), but rose again, particularly among GPs and OH physicians (60–65%; 37–43% for others) in 2021, likely due to increased awareness of its potential. In 2021, the usability development of Kanta Services became a top priority across all specialties. Monitoring the evolution of physicians’ priorities remains crucial for understanding the impact of EHR-system and working environment changes.

A new system for handling electronic patient reported outcomes (ePROs) is being implemented in the Danish municipal health centers. The system, called Kommunal PRO (KPRO), is being implemented to provide dialogue support for healthcare professionals (HCPs) in their conversations with citizens, streamline PRO questionnaires nationwide, and enable the sharing of citizens’ responses between municipalities and hospitals. This study examines HPCs’ approaches to and use of KPRO through observations and interviews. Based on the results, we find that the implementation causes changes in work tasks for both HCPs and secretaries. Furthermore, the system may not be applicable to all disease areas within the municipal context. Lastly, the system is incompatible with municipal record systems, leading to duplicated work. To improve the implementation and use of KPRO, these issues must be examined further and considered in future deliberations regarding the system.

Usability and user satisfaction are critical for Electronic Health Records. This study evaluates DIPS Arena, a hospital-based EHR, and compares the System Usability Scale (SUS) and the National Usability-focused HIS-Scale (NuHISS) in predicting satisfaction. A cross-sectional survey of 127 users across three hospitals collected the SUS, the NuHISS, and satisfaction data. The SUS was a stronger predictor of satisfaction explaining 45% of variance. The SUS scores were higher and more consistent, while the NuHISS provided task-specific insights. The SUS effectively predicts satisfaction, but a combined usability approach is recommended. Interoperability challenges in DIPS Arena impact usability and Valkyrie’s Virtual Health Record could enhance data integration. Future research should refine usability frameworks with subjective and objective measures.

There is an increased interest in real-world usability data, especially various global regulatory bodies for Medical Devices emphasize the importance of clinical evaluation of the usability of the devices. As promising as the combination of clinical investigations in a real-world setting and usability evaluation can be, it is challenging in practice to combine these two well-established methodologies. This paper presents an exploratory study which asked why, when and how to conduct this type of combined study.

Medical educators frequently use simulations with standardised patients in their curriculum to expose learners to high-stakes scenarios in a safe, monitored context. It can be challenging to ensure a standardised experience for learners and provide consistent opportunities for faculty to measure competencies before piloting with target learners. We, therefore, designed a mixed-methods evaluation instrument based on our work conducting usability tests with health information technologies. We gathered quantitative data on task completion rates, competency assessment rates, and user perceptions of the task. We also gathered qualitative information on usability issues. Half of the testers did not complete the telehealth safety checks, and one tester did not complete an audio/visual cross-check. These issues interfered with the faculty assessment of three competencies: clinical data collection, proper equipment use, and meeting professional standards. We used testers’ qualitative feedback to identify easy improvements that we plan to test another round of testers. We believe the method illustrated here is an easily reproducible approach that clinician educators can adapt for various medical education simulations.

While robot acceptance in different populations is well-studied, little is known about how individuals with dementia perceive and respond to humanoid assistive robots. This paper explores how individuals affected by dementia react to and engage with such robots, focusing on interactions with Pepper, a humanoid robot. Conducted in an all-dementia nursing home with residents experiencing varying stages of dementia, the study has collected direct observations and participant feedback. A common concern among clinicians, family members, and caregivers is that individuals with dementia may find robots frightening or unsettling, raising questions about their suitability for caregiving roles. However, the findings of this study suggest otherwise. Residents consistently identified the robots as “cute” and “child-like,” with many expressing comfort and interest in interacting with them. These results highlight the potential for humanoid robots like Pepper to serve as non-threatening, engaging companions for individuals with dementia, addressing caregiving needs while enhancing their well-being. This study provides a foundation for further exploration into the acceptance and application of assistive robotics in dementia care settings.

The Kübler-Ross Five Stages of Grief Model (KR model) is examined in the context of human-centered health informatics. The KR model is positioned as a non-linear set of heuristics which can assist in contextualizing the complexities of human cognitive-emotional processing of distressing events (e.g., organizational change, technology implementation, trauma). Thus, unique perspectives can be provided in each conceptual stage: 1) Denial and Isolation, 2) Anger, 3) Bargaining, 4) Depression, 5) Acceptance. This scoping review sought to: 1) Examine the application of the KR model in the design, evaluation or implementation of healthcare technology, 2) Determine if the KR model has been used to guide context specific user-experience (UX) activities (i.e., persona creation, journey mapping) to understand the patient, physician, nurse or caregiver experiences. The findings underscored that the KR model has various healthcare applications and use, however it remains underutilized as a valuable tool to contextualize human experiences.

The care of osteoporosis is being revolutionized by developments in AI-enabled exoskeletal robotics, which can improve mobility and facilitate the recovery from fragility fractures. The literature on current advancements in exoskeletal robotics and their therapeutic implications for patients with osteoporosis with particular attention paid to investigations on bone regeneration, functional rehabilitation, and gait analysis, is reviewed in this study, The review focuses on exoskeletons used for mobility aid and polymer-coated synthetic bone grafts, highlighting these technologies’ safety, viability, and efficacy.

External validity is the extent to which the findings of an experimental study are applicable or can be generalized to other people and contexts. External validity includes both population validity (i.e., the representativeness of the sample) and ecological validity (i.e., the representativeness of other contextual variables such as the stimuli, workflow, and environment). There is no consensus on the number and names of the ecological validity dimensions researchers should consider when designing or evaluating usability evaluations in health care. Therefore, in this paper, we integrated concepts from 3 ecological validity frameworks into a unified external validity. We used this new framework to describe the dimensions of external validity of a previous usability study. This framework can inform the design and description of usability evaluations to enhance reproducibility and comparison of findings and ultimately better understand the impact of different dimensions and external validity holistically.

Cognitive Reserve (CR) refers to the brain’s ability to compensate for brain damage or age-related changes, which can explain why some individuals show greater cognitive resilience to brain pathology despite damage or age-related changes. Understanding CR is crucial for identifying factors that contribute to cognitive decline among individuals. Currently, there is no direct, valid and widely accepted method for quantifying CR. To address this gap, we conducted a systematic review regarding approaches used by researchers and identified that Machine Learning (ML) based approaches offer promising potential for developing reliable, data-driven, and accessible methods to quantify CR. However, ML models have been known for their black-box nature due to their lack of transparency and interpretability which makes it difficult for clinicians to trust the decision making processes of these models. To address this limitation, a literature review was conducted using Google Scholar and 21 relevant papers were included in the final systematic review. Our review highlights that while ML-based approaches enhance CR mea-surement, the lack of standardized proxies variables and model transparency limits clinical adoption. Our reviewed approach will bring transparency and interpretability in measuring CR.

Electronic Health Records (EHRs) are vital in healthcare, but their usability often varies, affecting user satisfaction and adoption. This study explores the relationship between Work-related Quality of Life (WrQoL) and EHR usability during transitions to updated systems. A comparative analysis of clinical users over two time points found that WrQoL significantly influenced perceptions of usability, highlighting the importance of emotional and contextual factors in technology adoption highlighting the socio-technical aspect of EHR implementation and use. The findings support models emphasizing the role of well-being and organizational context in shaping user experiences. Practical implications include fostering WrQoL through well-being programs, context-aware system design, and user feedback. Future research should investigate causal pathways to enhance EHR usability and healthcare efficiency. This study emphasizes the critical role of user well-being in successful health information system implementation.

Patient-accessible electronic health records (PAEHRs) are regarded as a means to empower patients, especially those with chronic conditions, to take greater responsibility for their own health. However, clinicians often express concerns that PAEHRs may negatively impact their work and patient care, particularly if patients misinterpret their data. We investigated how clinicians working in home dialysis care for patients with chronic kidney disease experienced PAEHRs. Eleven clinicians participated in semi-structured interviews. The effects of PAEHRs varied depending on the patient and their circumstances. When the patient was knowledgeable and motivated, PAEHRs were seen to reduce clinicians’ workload and improve communication. Conversely, the impacts were negative with patients who were already anxious or otherwise had challenges in understanding their data. Although home dialysis patients are generally considered more capable of taking responsibility for their own health than the average patient, the challenges reported by clinicians were similar to those observed in previous studies involving other patient groups. PAEHRs should be examined from a socio-technical systems perspective, taking into account patients’ situations, capabilities, and experiences, and the communication between patients and clinicians. Further research is needed to explore the socio-technical aspects that influence the impact of PAEHRs on clinicians’ work with individuals who have chronic conditions requiring significant patient involvement, such as home dialysis.

Health information technology implementations are challenging and costly projects that face numerous barriers to success. One such barrier is the negative attitudes of users towards the specific technology. Understanding the various categories of negative attitudes could help develop implementation strategies. The negative attitudes of paramedics toward speech recognition technology were identified in a survey and semi-structured interviews. Five themes emerged from the analysis: previous poor implementation/adaptions, technology being a crutch, too much technology, personal technology exposure, and non-beneficial technology. Introducing technology to paramedics during education and demonstrating usefulness can alleviate some negative attitudes. Future research opportunities are discussed.