National use of artificial intelligence for eye screening in Singapore

Publication Type

Journal Article

Publication Date

11-2024

Abstract

Diabetes is a major health care challenge, affecting 10% of the global population. One third of patients with diabetes have an ocular complication known as diabetic retinopathy (DR). DR progression to manifestations such as vision-threatening diabetic retinopathy (VTDR) remains the leading cause of blindness in working-aged adults. Yearly DR screening is a universally recommended practice in primary care settings for patients with diabetes, but it is often difficult to implement due to a lack of staffing and screening capacity in primary care. This case study highlights our experience with developing a medical artificial intelligence (AI) software-as-a-medical-device (SaMD) solution for DR screening and implementing it at a national level to provide the capacity needed for DR screening in Singapore. Our approach involved two broad phases. First, we established a national telemedicine screening program, Singapore Integrated Diabetic Retinopathy Program (SiDRP), for population screening of DR in primary care run by trained, nonclinician human graders. Second, we deployed a deep learning–based AI solution, Singapore Eye Lesion Analyzer (SELENA+), into the SiDRP to scale-up the DR screening process by the human graders. We demonstrated the cost-effectiveness of this solution, and obtained medical device regulatory approval for clinical use in health care settings. We report the prospective evaluation of SELENA+ in SiDRP using real-world pilot data from the first 1712 patients consecutively recruited. Sensitivity and specificity of SELENA+ in detection of referable DR cases were 94.7% (95% confidence interval [CI] 88.0% to 98.3%) and 82.2% (95% CI 80.8% to 83.5%), respectively. In comparison, sensitivity and specificity of human graders were 98.9% (95% CI 94.0% to 99.9%) and 97.2% (95% CI 96.6–97.8%), respectively. For patients with VTDR, SELENA+ demonstrated a substantial advantage of higher sensitivity compared with human performance, reflecting the benefit of the fine-tuning of SELENA+ that we performed to enhance the AI solution’s ability to detect VTDR. We outline the clinical, technical, operational, regulatory, and governance challenges encountered as well as the lessons learnt in this AI algorithm implementation journey. We also present a conceptual framework with considerations and strategies for the broader adoption of medical AI SaMD solutions in the field of ophthalmology and beyond.

Discipline

Analytical, Diagnostic and Therapeutic Techniques and Equipment | Artificial Intelligence and Robotics | Health Information Technology

Research Areas

Information Systems and Management

Publication

NEJM AI

First Page

1

Last Page

8

ISSN

2836-9386

Identifier

10.1056/AIcs2400404

Publisher

Massachusetts Medical Society

Embargo Period

11-19-2024

Additional URL

https://doi.org/10.1056/AIcs2400404

Link to Full Text

Share

COinS