The New York Eye and Ear Infirmary of Mount Sinai (NYEE) has demonstrated how GPT-4 can match or surpass human ophthalmologists in diagnosing and treating eye diseases.
The results, detailed in a study published in JAMA Ophthalmology, we discuss how AI may also help ophthalmologists of their decision-making processes.
The research team at Mount Sinai included 12 treating specialists and three senior trainees from the Department of Ophthalmology on the Icahn School of Medicine.
They compared the responses of each AI and human specialists to a series of questions and patient cases related to glaucoma and retinal diseases. The responses of GPT-4 were evaluated and located to be equal to or higher than those of human specialists, particularly in glaucoma.
Here's a bit of more detail on the way it worked:
- Study structure: The research was conducted with a team of 15 ophthalmologists from the Department of Ophthalmology at Mount Sinai, consisting of attending physicians and senior trainees who focus on glaucoma and retinal diseases.
- Data compilation: For evaluation purposes, the team used a comprehensive set of 20 ophthalmology questions (evenly split between glaucoma and retinal topics) and 20 patient cases, which were anonymized to take care of privacy. These were chosen to reflect a spread of cases from Mount Sinai clinics.
- Use of AI: OpenAI's GPT-4 was asked to reply the questions and analyze the patient cases. The aim of the AI was to reply like a practicing ophthalmologist, using clinical shorthand where appropriate to reflect the concise style typical of clinical notes.
- Evaluation: The study used a scoring system to guage each the accuracy and completeness of the responses from GPT-4 and the human specialists. This enabled a direct comparison of the performance of AI with the performance of trained professionals.
- Glaucoma results: In glaucoma, GPT-4 provided very accurate and more comprehensive answers than human specialists. This demonstrates GPT-4's strong ability to grasp and advise glaucoma cases, potentially providing precious support to ophthalmologists on this specialty.
- Retina results: For retina-related queries and patient scenarios, GPT-4 has brought together human specialists, demonstrating its ability to properly diagnose retinal diseases and recommend treatments. In addition, GPT-4 often provided more detailed answers, suggesting a radical evaluation and understanding of cases, which could possibly be particularly helpful when managing more complex or nuanced patient situations.
Dr. Andy Huang, an ophthalmology resident at NYEE and lead writer of the study, shared his findings and stated, “The performance of GPT-4 in our study was quite revealing. We recognized the big potential of this AI system from the moment we began testing and were fascinated to watch that GPT-4 couldn’t only support, but in some cases surpass or exceed the expertise of experienced eye care professionals. “
Dr. Louis R. Pasquale, vice chair of ophthalmology research and senior writer of the study, was also impressed by the outcomes, stating, “AI was particularly surprising in its proficiency in treating glaucoma and retinal patient cases, which was consistent with accuracy.” Completeness of diagnoses and treatment suggestions made by human physicians in a clinical note format.”
“Just because the AI application Grammarly can teach us to grow to be higher writers, GPT-4 can provide us precious guidance on methods to grow to be higher clinicians, especially when it comes to how we document the outcomes of patient exams.”
Dr. Huang sees a future for AI in ophthalmology, noting its potential to offer diagnostic support to ophthalmologists and reduce their workload, especially with complex cases.
AI has performed well in eye health applications, reminiscent of in 2018 to 2020 when DeepMind used machine learning to accurately analyze high-resolution three-dimensional optical coherence tomography (OCT) and diagnose quite a few medical conditions.
In 2023, researchers developed an AI model, to be precise Recognize early signs of Parkinson's disease from eye scans.
Another was trained to trace Parkinson's, heart disease and other illnesses, again from eye scans and one other one at that detect certain diseases in infantsagain from retinal scans.
