Machine Learning Prediction of Response towards Anti-VEGF Injections in Patients with DME: Prediction of Post-Injection CST

Diabetic macular edema (DME) has become one of the most potential complications that results in loss of vision in patients with diabetic retinopathy. Treatment outcomes that have been predicted directly with advent of machine learning (ML) methods after the initial anti-vascular endothelial growth factor (anti-VEGF) injection, has become extremely vital in the management of DME

Purpose:The aim of this study is to analyze the efficiency of the ML regression models which were developed and validated to predict the possible post-injection central subfield thickness (CST) value and distant vision best corrected visual acuity (DV BCVA) in eyes with DME before the anti-VEGF injection is administered at either treatment initiation or during treatment monitoring.

Methods:This retrospective study was conducted in Medical Research Foundation, Chennai, India from January 2010 to December 2020. The model development emphasized on an ensemble ML system consisting of four ML models that were developed and trained independently using the clinical parameters to predict the post injection CST value. The dataset consisting of 906 patients with total of 1874 samples [Optical coherence tomography (OCT) images and clinical parameters] were divided into trained and test set, and the model was validated on test dataset. The predicted CST values was then compared against the respective sample’s post injection actual CST value. The comparative results were measured in terms of Correlation Coefficient and Mean Absolute Relative Error (MARE).

Results:On evaluation, we found that Support Vector Regression (SVR) with linear kernel performed best among the other models with four different scenarios in term of both CST and DVBCVA prediction with correlation coefficient of 0.65, 0.73, 0.75, 0.85 and 0.83, 0.87, 0.89 and 0.92 respectively.

Conclusion:Our ML system did predict the post-treatment CST, and DVBCVA in patients with DME following anti-VEGF injections and may be used systematically to determine the efficacy of anti-VEGF therapy in DME management.

Keywords: Machine Learning, Artificial Intelligence, Automated Detection, Visual Outcomes, Lucentis, Ranibizumab, anti-VEGF treatment options, Diabetic Macular Edema, Diabetic Population, Regression, SVM, RF, DL, CNN, DME Detection, CST Reduction, Central Sub-Field Thickness Prediction

Diabetes mellitus (DM) is a chronic disease that affects 537 million people worldwide, and the number of individuals with diabetes is expected to rise to an estimated 783 million by the year 2040 [1]. Diabetic Retinopathy (DR) is a commonly associated complication and leading cause of vision loss in the working age population in developed countries [2]. DME is a frequent manifestation of DR which is characterized by hard exudates in the macula region of the retina, and it can occur at any stage of DR, prolonged macular edema can lead to irreversible damage to the retina [3].

Among the treatment options available for DME, intravitreal injections of anti-VEGFs have been the most practiced option. Most widely used anti-VEGF drugs at present are ranibizumab (Lucentis) and bevacizumab (Avastin) [4]. Both drugs are monoclonal antibodies that bind to all three forms of VEGF. They are very similar drugs, but ranibizumab is a smaller molecule and is believed to bind VEGF in the eye with greater affinity. Ranibizumab is intended purely for intraocular injection and is currently approved for management of age-related macular degeneration (AMD) and DME. Many studies have proven the safety and effectiveness of ranibizumab in improving the visual acuity and reduction of CST in the treatment of DME [5]. However, arriving at a decision to adapt an effective treatment regimen for a given patient requires substantial expertise.

Most trials have shown good treatment effectiveness of regimens available for administering the anti-VEGF drugs. However, in either regimens patient review is essential since the treatment outcome may vary [6,7]. Even among those patients who respond well and have improvement with no further requirement of injections, many may relapse and require further injections in the future. Another challenge is that ophthalmologists, with exception of Europe, at least half of the time, switch the anti-VEGF molecule after three injections. Whereas in Europe, ophthalmologists wait until after at least four injections to switch treatments three-quarters of the time, and often they wait until after six or more [8]. This clearly reveals the complexity in determining an optimal treatment regimen and to differentiate patients who respond to anti-VEGF. Since it is also expensive, a significant financial and injection burden is created for both the patient and the health care providers if the patient is not responding. Hence, any clinically acceptable methods to predict the treatment outcomes right after the first anti-VEGF injection holds huge value in efficient management of DME.

Optical coherence tomography (OCT) technology is being used widely as imaging modality in diagnosis and follow up in DME [9]. Clinical markers for determining response to anti-VEGF therapy in DME management include measurable post-injection OCT parameter like CST and BCVA values (Distance and near BCVA) [10].

Advances in computing and the availability of large data sets of retinal fundus images and OCT scans have spurred the development of autonomous artificial intelligence (AI) system for detecting various retinal abnormalities like macular edema. Deep neural networks, an automated algorithm, have also been deployed in the paradigm of grading DME stage or severity [11]. Further AI algorithms based on Neural Networks have also been introduced to predict the visual outcomes in DME management with ranibizumab therapy [12]. Algorithms developed previously for automated analysis of retinal OCT cans include biomarker detection [13,14], classification [15-23] and retinal layer segmentation [24,25]. Some studies [26-30] have identified that biomarkers such as integrity of outer plexiform layer (OPL) and hyperreflective dots (HRD) on OCT, could be indicative of anatomical outcomes post the administration of Anti-VEGF injections. However, these models take into consideration the anatomical outcomes which can vary for each patient and multiple clinical variables like age, gender, duration of diabetes etc could influence these outcomes. Most of these models predict the response towards Anti-VEGF injections only after the three Anti-VEGF injections have been administered. In clinical diagnosis, the rationale and other clinical parameters considered behind a treatment decision also plays a crucial role apart from diagnostic accuracy.

In the present study, we aim to develop and validate ML regression model to predict the possible post-injection CST value and DV BCVA in eye with DME either before the anti-VEGF injection is administered at either treatment initiation or during treatment monitoring. These kind of AI systems which can predict treatment outcomes could improve the speed and accuracy of implementing appropriate treatment regimens and bring about efficiency both in treatment expenditure and treatment outcomes. This is the only AI based prediction model which can predict the response towards Anti-VEGF injection before the administration of three injections. Hence, incorporation of such a model in clinical practice can assist the clinician in adjusting the therapeutic schedule with precise injection frequencies and injection doses as per different kinds of biological changes of retinal structures detected by AI. This ML regression model would be useful in predicting the post-injection CST and DV BCVA value based on pre-injection OCT data and clinical parameters. Since the post-injection outcomes can be associated with many clinical variables, OCT images and clinical features like CST, the Early Treatment Diabetic Retinopathy Study grid (ETDRS) subfield and subfield average volume along with clinical variables like near/DV BCVA, number of anti-VEGF injections administered previously, if any, and number of days after administration of previous injection will be used as input data. Such a model can provide clinicians with referable information to help them design precise and effective therapeutic schedule. This AI model was presented as a poster at the ARVO 2022 conference[31]. AI based detection would propel precision decision making by identifying the molecular subtype and individualized drug response hence shifting the goal treatment from sight preservation to sight improvement.

For the development of the AI algorithm, the Cirrus HD OCT scans, clinical parameters and longitudinal patient records from January 2010 to December 2020, were collected retrospectively by the optometrist from ophthalmology institute, Medical Research Foundation, Chennai. The OCT images were collected from the Zeiss CIRRUS 4000/5000 SD-OCT device and the electronic medical records (EMR) of the institute. A total of 1874 datasets (OCT data + clinical parameters) from 906 patients were extracted from the eye clinic’s EMR system along with the demographic data like age and gender and clinical parameters like DV BCVA, Near BCVA, number of days after injection and injection count for each visit of the patients. The macular OCT scans, with scan pattern macular 512x128, were extracted using FORUM review software. These OCT scans were then anonymized using the open-source anonymization tool [32]. As part of the anonymization, the patient’s first name and last name were replaced by anonymous names. The OCT Analysis software was executed on the macular OCT scans and macular thickness measurement parameters were extracted. These measurement parameters included CST, other eight ETDRS grids (Subfield Inner Nasal, Subfield Inner Superior, Subfield Inner Temporal, Subfield Inner Inferior, Subfield Outer Nasal, Subfield Outer Superior, Subfield Outer Temporal, Subfield Outer Inferior, Subfield Average Volume, and Subfield Average Thickness). The pre-injection CST value and post-injection CST value were then mapped according to patient’s injection visits.

All eligible eyes with confirmed diagnosis of center-involving DME and being treated with anti-VEGF injections were included. The anti-VEGF medications used included Lucentis, Accentrix, Eyelea, Razumab and Avastin. Patients with a history of vitrectomy, and any other ocular diseases that might affect ocular circulation (e.g, AMD or retinal artery/vein occlusion), severe cataracts, or DME previously treated with intravitreal or periocular injections or pan retinal photocoagulation (PRP) were excluded. Figure 1: Data Collection Strategy summarizes the data collection strategy.

The samples with the poor OCT image quality, the signal strength less than 0.4, were excluded from the database. The samples with more than 15 days gap between the pre-injection OCT visit and the injection visit date were also excluded from the dataset. The dataset of clinical variables from the medical records and OCT device were retained and saved into a Microsoft Excel spreadsheet (Microsoft Corporation, Redmond, WA, USA, version 2017). The final dataset was then divided into train and test set to train and evaluate the AI model respectively. This study was conducted according to the Declaration of Helsinki (as revised in 2013) and was approved by the Ethics Committee of Medical Research Foundation. Individual consent for this retrospective analysis was waived.

The response of the anti-VEGF injection was predicted in terms of either CST value or the BCVA values. To predict the possible post-injection CST and BCVA before the anti-VEGF injection is administered, four benchmark models including Linear Regression [33], SVR [34] with Radial Basis Function (RBF) kernel [35], SVR with linear kernel and Random Forest (RF) [36] were trained using the clinical parameters, such as near BCVA value, DV BCVA value, number of previous injections, number of days after the injection, and pre-injection macular thickness measurement parameters, such as CST, Sub-field Average Thickness, Sub-field Inner Nasal Thickness, Sub-field Inner Superior Thickness, Sub-field Inner Temporal Thickness, Sub-field Inner Inferior Thickness, Subfield Outer Nasal Thickness, Sub-field Outer Superior Thickness, Sub-field Outer Temporal Thickness and Sub-field Outer Inferior Thickness. These parameters were considered as the feature vector and were provided as an input to the ML regression models for training. The post injection CST value or post injection BCVA value were considered as the target output for the training. All the four ML regression models were trained independently using the input parameters to predict the post injection CST value. The predicted post-injection CST and BCVA were compared with the true values obtained from the medical records. Figure 2 summarized the workflow of the model development.

Evaluation Metrics (To Assess the Quality of the Prediction Per Model)CST reduction was evaluated at each consecutive anti-VEGF injection visit. The predicted CST values were then compared against the respective sample’s post injection actual CST value. The comparative results were measured in terms of Correlation Coefficient and MARE.

(1) Correlation Coefficient [37], to determine the strength of AI model predicting post CST value compared to the actual CST value

(2) MARE, at each injection visit to validate the accuracy of the predicted CST value by the AI model

n = total number of data points

The higher the correlation coefficient and the lower the MARE, the closer the predicted CST and BCVA values were to the true value. The ML regression models were validated against the test dataset and compared against the actual post injection CST value and the correlation coefficient and MARE were recorded.

Of the 906 patients included in the study, 644 were males and 262 were female patients. However, the inclusion criteria for the study were independent of the gender of the patient. Age of the patients were recorded at their first visit. All the patients for whom the OCT data and clinical parameters have been collected were at least 18 years of age. The age distribution has been represented graphically in Figure 1. Most of the patients (n=667) were aged between 50-75 years.

The patient demographics have been summarized in Table 1:

For each patient, data points from multiple visits were collected. Each data sample consists of information from the pair of pre and post injection visits. As part of pre injection visit information, the parameters collected are macular OCT scan, patient’s age, gender, Near BCVA, DV BCVA values, CST value, other eight ETDRS grid values, number of previous anti-VEGF injections, and number of days after injection. While as part of post injection visit information, the parameters collected were macular OCT scan, post injection CST value, post injection Near BCVA and DV BCVA value. The samples that did not have both pre and post injection OCT scans acquired on the CIRRUS 4000/5000 device were excluded during the data collection. The samples having pre injection OCT signal strength or post injection OCT signal strength less than 0.4 were excluded from the dataset.

Duration of Diabetes:An average duration of diabetes for a set of 891 patients was found to be 13.8 years approximately. Diabetes duration was not available in the clinic’s record for a total of 15 patients. However, it was ensured that all the patients included in the study were the cases of diabetes.

Anti-VEGF Injection Type:The data set included all three anti-VEGF injection types: Ranibizumab (Accentrix, Lucentis, Razumab), Bevacizumab (Avastin) and Aflibercept (Eyelea). A total of 466 patients, out of 906 patients, were being managed with Bevacizumab for DME, 383 patients were being administered Ranibizumab and nearly 56 patients were being managed with multiple anti-VEGF molecules in their treatment regimen in order to improve the vision outcomes.

Eye Follow-up and Visit Period Distribution:In the total data set from 906 patients, 681 patients were followed up for treatment for one eye while 225 patients were being managed for both the eyes.

The visit period distribution for patients’ data was also evaluated and analyzed in order to determine the average follow up for the patients. The visit period was defined as the difference between the first visit and last visit of the patient. Maximum follow up period was 78 months while a total of 726 patients were followed for 0-6 months, and 103 patients were followed up for 6 to 12 months. The average follow-up period was approximately 4 months.

The final dataset consisted of 906 patients with total of 1874 samples. The train set includes 1461 samples from 721 DME patients while the test set includes 413 samples from 185 DME patients. The predictive model was validated on the test data set. The performance of the model was further evaluated for four different scenarios 1) Number of injections administered are more than or equal to one, 2) Number of injections administered are more than or equal to two, 3) Number of injections administered are more than or equal to three and 4) Number of injections administered are more than three. Table 2 shows how the performance of the CST prediction models improves from scenario one to scenario four.

SVR with linear kernel model achieved correlation coefficient of 0.65 for scenario with number of anti-VEGF injections administered are greater than or equal to one. The model achieved correlation coefficient of 0.73 for number of injections administered are greater than or equal to two and 0.75 for number of injections administered are greater than or equal to three and 0.85 for number of injections administered are greater than three. The model achieved best performance in terms of correlation coefficient for the number of injections administered are greater than three. Linear regression model achieved slightly lesser performance than SVR with linear kernel. It achieved correlation coefficients of 0.64, 0.71, 0.72 and 0.82 for scenarios one, two, three and four respectively. RF regression model achieved correlation coefficient of 0.63, 0.66, 0.68 and 0.79 for scenarios one two, three and four respectively while SVR with RBF kernel achieved correlation coefficient of 0.50, 0.53, 0.53 and 0.68 scenarios one two, three and four respectively

The performance of these four models was also evaluated in terms of MARE. Table 3 shows the MARE of the four ML regression models for four different scenarios. Similar to correlation coefficient, SVR with linear kernel model achieved the best performance of 0.25, 0.22, 0.22 and 0.18 for the scenarios one, two, three and four respectively. While the linear regression model, RF and SVR with RBF kernel achieved MARE of 0.29, 0.28, 0.31 respectively for greater than or equal to one anti-VEGF injections administered scenario, 0.27, 0.25 and 0.30 respectively for scenario two, 0.26, 0.25 and 0.31 respectively for scenario three and 0.23, 0.21 and 0.29 respectively for scenario four with number of injections administered are greater than three.

Figures 3, 4, 5 and 6 show the scatter plots of predicted CST value using linear regression model, SVR with RBF kernel model, SVR with linear kernel model and RF model respectively versus actual CST value for all four different scenarios.

Like CST value prediction models, for the BCVA value prediction, models were evaluated using the test set. The performance of the models was evaluated in terms of correlation coefficient. Table 4 shows the correlation coefficient for the BCVA prediction models for four different scenarios. SVR with linear kernel achieved correlation coefficient of 0.83, 0.87, 0.89 and 0.92 for the scenarios one, two, three and four respectively. However, with linear regression model, SVR with RBF kernel and RF models achieved correlation coefficient of 0.83, 0.82 and 0.83 for scenario one-number of injection administered greater than or equal to one, 0.87, 0.86 and 0.86 respectively for scenario two-number of injection administered greater than or equal to two, 0.89, 0.88 and 0.87 for scenario three-number of injection administered greater than or equal to three, and 0.92, 0.89 and 0.90 for scenario four-number of injection administered greater than three.

Figure 7, Figure 8, Figure 9 and Figure 10 show the scatter plots of predicted BCVA value using linear regression model, SVR with RBF kernel model, SVR with linear kernel model and RF model respectively vs actual BCVA value for all four different scenarios.

The advent of ML methods and their rapid advancement has fostered interest in new ways of predicting response to anti-VEGF from baseline OCT among diabetic patients with DME. To our best understanding, four previous works have used deep learning for response prediction to anti-VEGF in patients with DME. The first is the work by Rasti et al. where they developed a deep learning model using CADNet as a baseline model, followed by adding certain convolutional layers to improve model accuracy [38]. They performed their experiment using baseline OCT images of 127 patients and concluded that using relatively small sample size, the model was able to predict and classify patients with high accuracy. The other similar work is by Jing Cao et al. where separate deep learning models were used and trained on different features, including hyper-reflective foci, intra-retinal, and subretinal fluid, which might lead to higher computational demand than feature extraction using one model [39]. Their research developed a ML model to predict treatment outcomes, an approach that is considered inferior and less sophisticated compared to deep learning [40] and also predicts the response only after administration of the third injection. Dan Cao et al, constructed a computer-aided system to predict anti-VEGF response in patients with DME [41]. The system used an RF classifier based on features extracted by deep learning algorithms and revealed the importance of related features. However, the major limitation was that the inconsistencies in the features on which the model was trained were corrected and labelled manually by seven ophthalmologists and hence the variability in the labelling was unavoidable [41]. The latest model development is by Saif Aldeen et al. where they used the approach of having two deep learning models [42]. With the cascading model, the segmentation of input image is performed followed by classification. Their model was developed on a small sample size from a specific ethnicity and from one single OCT device type. To understand model accuracy, it is difficult to compare models that were not tested on similar data, which is the case for the mentioned studies and ours. In the current study we evaluated four benchmark models including Linear Regression, SVR with RBF kernel, SVR with linear kernel and RF. The previous studies either classify or predict the CST value only after third injection during the course of the DME treatment by anti-VEGF injection while our study shows significant performance predicting CST value at any time of the treatment. Our model can predict the response for every injection. The predicted post injection CST and BCVA were compared with the true values and the compression results were measured in terms of correlation coefficient and MARE for four different scenarios. Combining the results in Table 2, Table 3, we concluded that the among four ML regression models, SVR with linear kernel performed best for all the four scenarios in both the prediction models.

ML algorithms are designed to forecast how well a patient will have a high or low treatment demand and a good or bad response. This research implies that progress is being made in this intriguing subject. To our knowledge, this is the first study in India to develop an ML model which predicts the response of the DME patient towards anti-VEGF injection before the next injection is administered. Unlike, conventional regimen in which the response towards anti-VEGF is determined after administration of at least three antiVEGF injections, the current model attempts to give an insight on possible CST value right after the first anti-VEGF injection is administered. This can prompt the clinician to adjust the therapeutic schedule with precise injection frequencies much earlier in the treatment regimen of the patient. In contrast to prior studies, the cohort size is large enough to derive significant findings using the ML technique we used. Likewise, in our study, which included patients who were on a long-term anti-VEGF medication, yielded a remarkable result henceforth, with the application of this model, it will be possible to estimate an individual's long-term anti-VEGF therapy demand. ML models attempt to predict whether or not a patient will thrive. Although this study has some optimistic results, it does have some limitations. First, because this is a single-center study, the algorithms established here must be externally validated and tested on different cohorts. Second, the cohort's patients are mostly elderly, which precludes us from drawing conclusions about the algorithm's relevance to younger patients. Another limitation in this study is the noisy treatment intervals which is due to inconsistencies between the prescribed and actual treatment intervals, which can occur due to visit rescheduling, patient’s convenience or other unknown variables. This in turn can have an influence on treatment demand. Inclusion of clinical information associated with treatment outcomes, such as DR stage, blood glucose, and baseline HbA1c (glycosylated haemoglobin) , did not improve the prediction accuracy of model hence more diverse dataset is required to train the model with other risk factors. Likewise, DME eyes treated with different anti-VEGF medications may have provided varying degrees of results however eyes treated with intravitreal steroidal injections were excluded.

The recent Coronavirus pandemic, which led to enormous pressure on the healthcare systems globally, has triggered the application of AI models in triaging patients towards limited number of available resources. The need for developing a prediction model that can predict response to anti VEGF before starting the treatment has increased many-folds. It is estimated that patients need 9 -11 times of repeat injections in the first year of treatment [43] up to 17 times in 5 years on average [44]. The cost of injections and OCT required greatly increase the healthcare burden and reduce the quality of life of patients due to more time spent on the road and a feeling of anxiety caused by injections. Such AI models which can predict the response towards the anti-VEGF injection before the injection is administered, can improve treatment efficiency, reduce the socioeconomic and humanistic burden and reallocate medical resources more efficiently at the same time.

An ML model was developed to predict the possible post-injection CST value in patients with DME before the next injection is administered. The model was trained on three OCT image features (ETDRS grid subfields, CST and subfield average volume) and three clinical variables (BCVA, Previous injection, and days after injection). To the best of our knowledge this is the first ML model that can predict potential post injection CST value, before the next anti-VEGF injection is administered, based solely on pre-treatment OCT data. OCT being a non-invasive and quick diagnostic tool with reproducible investigation has revolutionized the imaging of posterior segment lesions leading to early diagnosis of macular diseases like DME. Various combinations of findings on OCT and comparison with known OCT-based biomarkers have made disease identification and prognostication much simpler. These granular OCT features along with such ML based prediction models can accelerate the decision-making process for DME management with anti-VEGF injections as the first-line of treatment. Thereby shifting the goal of the treatment from sight preservation to sight improvement. Nevertheless, the implementation of such ML models in real-world setting requires extensive training and re-training of the model to improve its prediction accuracy. Developments in this field would include choosing different patient population in clinical studies, where non-responders might be further investigated for better understanding.

KP, MG and Dr.GB are employees of Carl Zeiss India (Bangalore) Pvt. Ltd. AV is an employee of Harman Connected Services, and declares contractual work for Carl Zeiss India (Bangalore) Pvt. Ltd. CDS and Dr. RR declare consulting work for Carl Zeiss India (Bangalore) Pvt. Ltd. All authors approved the final version of the manuscript. All authors had full access to all the data in this study.

The sponsor was involved in the design and conduct of the study, collection, management, analysis, interpretation of the data, development of the ML model, preparation and review of the manuscript, and the decision to publish.

This study was supported by Carl Zeiss India (Bangalore) Pvt. Ltd. The sponsor also supported the journal’s article processing charges (APC).