Skip to main content

Image-based Phenotype Detector

Primary supervisor

Ehsan Shareghi


  • Dr Bhavna Antony

In Ophthalmology, patients are routinely scanned with multiple retinal imaging systems that provide complementary information to the clinicians. However, unlike other specialties, the images are not analysed by a radiologist and the treating ophthalmologist or optometrist is expected to analyse this data on their own. This is extremely time consuming, and difficult to achieve in clinical settings. Thus, AI models for disease detection have been extremely popular. 

Current AI models designed for computer-aided diagnosis of diseases attempt to directly identify the underlying disease. While this is extremely effective in most cases and models have been built to differentiate between multiple conditions, there are some underlying assumptions that hamper its translation into clinical practice. First, the presence of a disease is assumed to be mutually exclusive, i.e., each image is either healthy or shows features of one disease. This is typically not how patients present in clinics as they may have multiple underlying conditions. One example of this is diabetes, the occurrence of which in the ageing population is extremely high (1 in 11 adults are estimated to suffer from diabetes). Thus, it is very likely that an adult patient presenting with symptoms of another condition, will also likely suffer from diabetes and may already show some features of diabetic retinopathy (a retinal complication associated with diabetes). 

We propose to build a phenotype detector that will identify the presence of specific pathological features in the images instead of identifying diseases. This will allow for 1) the images to be “captioned”, and 2) for the specific image features to be grounded in a knowledge base in order to detect the concurrent presence of multiple diseases. In particular, we are seeking to build a multi-labelling pathology detector in multimodal retinal images.

Student cohort

Single Semester
Double Semester

Required knowledge

  • Proficiency in Python is required
  • Working knowledge of Image Processing (ideally medical image processing) is required
  • Familiarity with the pytorch libraries is desired
  • Familiarity with Image-based Transformer models and HuggingFace is desired
  • Very good verbal and written communication skill is required