An Innovative Study to Evaluate the Treatment Efficiency of Percutaneous Nephrolithotomy by DEA

The aim of this study was to measure the treatment efficiency of percutaneous nephrolithotomy (PNL) patients by Data Envelopment Analysis (DEA), as a novel method with an output minimizing model. Urinary stone disease is a frequent problem with increasing prevalence in recent years. Depending on the location and size, the treatment of renal stone 2 cm or more is surgical and PNL. In this context, a retrospective study in which 154 patients with kidney stones over 2 cm was conducted. Some biochemical perioperative and postoperative characteristics of patients (Sodium (NA), potassium (K), creatinine (CRE), and hemoglobin (HB) and stone area) were analyzed. Contrary to classical DEA theory, a novel output minimization model with uncontrollable output was proposed, then Malmquist analysis was used to find out the difference between patients according to gender and age. The decomposition of the index revealed that no remarkable difference within decision-making units clearly, however between age and gender groups the results were quite different. This is a new research concept, and it can be suggested to develop alternative treatment modalities for PNL by gender, age, or various groups.

Keywords: Data Envelopment Analysis; Malmquist analysis; Percutaneous Nephrolithotomy; Kidney Stone

List of abbreviations: PNL: Percutaneous Nephrolithotomy; DEA: Data Envelopment Analysis; NA: Sodium; K: Potassium; CRE: Creatinine; HB: Hemoglobin; ESWL: Extracorporeal Shockwave Lithotripsy; RIRS: Retrograde Intrarenal Surgery; CIRF: Clinically Meaningless Stone Fragments; UTI: Urinary Tract Infection; ANOVA: Analysis of Variance

Urolithiasis could be a painful disease. It can cause colic-like pain, nausea, vomiting, hematuria, and frequent urination, which can negatively affect patients’ quality of life. Regardless of the stone size, its prevalence has been steadily increasing in recent years and ranges from 5 to 15% worldwide [1]. Urinary system stones can be seen in any location of the urinary system. The location and dimensions of the stone are the first step in the selection of surgical treatment to be applied. According to EAU guidelines in kidney stones, the first choice for stones 2 cm or more is percutaneous nephrolithotomy (PNL). For stones smaller than 1 cm, preferences are Extra corporeal shockwave lithotripsy (ESWL) or retrograde intrarenal surgery (RIRS) [2].

PNL is the accepted gold standard treatment in all regions of the world in terms of stone-free rates in large kidney stones [2,3]. Success in PNL is evaluated by stone-free rates. Stones smaller than 4 mm remaining after surgery are called clinically insignificant stone fragments (CIRF). In the follow-ups CIRFs can grow and may cause symptoms due to obstruction in urinary system. Furthermore, it may require additional surgery [4].

Consequently, such minimizations of some measurements like stone area as mentioned in PNL treatments has special importance with its contribution to the patients’ health as well as the success of the physicians. In this regard, the measurement of treatment efficiency could be achieved by statistical analysis of the available treatment data.

The general statistical method used to measure the effectiveness of a treatment, i.e. success, is parametric methods based on paired Student t-test and Variance Analysis (ANOVA) after the procedure. If the necessary assumptions are not provided, methods with their non-parametric equivalents are proposed. Although these methods do not contain preliminary assumptions, they are useful. However, they are not as sensitive as parametric methods.

At this point, Data Envelopment Analysis (DEA), which has been used for measuring efficiency/success for many years and has many applications in medicine, will be recommended. In the literature, it is not mentioned any treatment efficiency study with Data Envelopment Analysis using pre- and post-treatment values as input-output. Although there are a few studies with related methods in the field of health, there is no similar study sample.

If the pre-treatment values are the input of DEA and the post-treatment values are accepted as the output of the method, it is an effective method that can be used to determine the patients with the most successful treatment and to ensure that they are taken as exemplary application and provide a precise measurement. The aim of this study was to measure the treatment efficiency of PNL patients by DEA, as a novel method with a minimized output model.

This is a retrospective study in which 154 patients with kidney stones over 2 cm were enrolled. The universe of the study was composed of patients who underwent PNL between 2011-2018 in the Urology Department of Süleyman Demirel University Research Hospital.

This study included male and female patients who underwent a percutaneous nephrolithotomy operation. The biochemistry values of the patients were measured in the laboratory with blood samples taken in the pre-op and post-op periods. There is no criterion for exclusion from the study. This operation is not performed on patients with any pregnancy, substance abuse, or psychiatric pathology.

The stone surface area was measured preoperatively and postoperatively according to the formula of “maximum diameter × width × π × 0.25” [5,5] to calculate the stone-free rate. Preoperative and post-operative Xray’s are used for measurement. The non-contrast computerized tomography results were not available for all patients.

Some biochemical values of patients will be analyzed, and 5 main variables of patients are Sodium (NA), Potassium (K), Creatinine (CRE), Hemoglobin (HB), and area of the stones recognized in Kidneys (AREA) values will be measured before and after treatment, and treatment performance will be evaluated over 10 variables.

A total of 154 operative patients, 82 Men, and 72 Women, were included in the study, and the efficiency of the treatments was analyzed by a well-known performance measurement tool Data DEA. Moreover, Malmquist analysis was performed to reveal whether the treatment performance varies according to gender and age [7-9].

DEA is a classical efficiency measurement tool used to measure the performance of individual Decision-Making Units (DMUs), which have input-output structures. Keeping the initial study of Charnes, Cooper, and Rhodes [10] in mind, Emrouznejad & Yang summarized the development of DEA in its first 40 years [11].

The method essentially calculates performance by revealing the proportional relationship between the outputs obtained, because of processing the inputs considered as sources in the economic sense with a formula based on linear programming. Although their applications in the medical field are quite common, they are generally used in hospitals/health facilities or doctors, nurses, etc. It is aimed to measure the performance of healthcare professionals through several economic variables [12-14]. Apart from these studies, there exist studies in which patients are taken as decision-making units (DMUs) and the efficiency of some treatments is measured as well, but it wasn’t aimed at output minimization [15,16].

The initial model of DEA, namely CCR, is given in (1), which is also used in this study as a base.

The Malmquist index is based on the early works of Sten Malmquist [17] and proposed by Caves, Christensen, and Diewert [18]. It is commonly used to compare the productivity growth or technical progress changes of a process at two different periods. A brief explanation of the Malmquist Index can be found in Färe, Grosskopf, and Russell [19]. Moreover, Camanho and Dyson [7] used the Malmquist Index for group comparison.

Consequently, as defined above, the matrix model of the Malmquist index for group comparison can be given as [7-9].

A detailed recent development about DEA under Malmquist context can be found in [20].

Due to the nature of the data used in this study, this model is not intended to maximize outcomes, but rather to minimize it. Because the purpose of the treatment is to decrease the measured post-treatment values of the patient compared to the pre-treatment. In this case, the values before the treatment are input, the values after the treatment are output and it is aimed to decrease these later values.

The DEA was performed by EMS (Efficiency Measurement System) with version 1.3 to solve proposed (3) model. This software can be downloaded free from the internet [21].

Basic DEA results are summarized in Table 1 & Table 2. As seen in Table 1, it is obvious from the DEA mentality that the maximum (highest) efficiency score is 1.00 and these scores represent the best practice treatments.

On the other hand, the efficiency of the worst treatment is 0.5069 and this shows that the worst practice is 50.69% of the best. The table also includes the quartiles and the efficiency values of 0.5496 for 25%, 0.6809 for 50% and 0.9999 for 75% indicate that the treatment is phenomenally successful. This is remarkably a high rate in the field of health.

Table 2 reveals the big picture. The treatments are divided into 6 classes between best and the worst cases and it is seen that 49 treatments are found efficient and get an efficiency score of 1. On the other hand, 14 treatments have an efficiency score between 0.90 and 1, 15 treatment gets an efficiency score between 0.80 and 0.90 and 33 treatment gets an efficiency score between 0.70 and 0.80. These are also remarkably high scores for the field of health care. This means that 72.08% of the total treatments have 70% or more treatment success.

In this study, the treatments which have lower efficiency scores could be benchmarked by corresponding efficient treatments. These efficient treatments are selected by the model not cursory. In this way, the doctors can see what and how much can be improved in their patients. As an example, Table 3 depicts the situation for the patient who has the least efficient treatment value of 0.5069 which means his/her treatment is %50.69 of its efficient peers/targets.

Another feature or perhaps the most important feature of DEA is the ability to benchmark which gives inefficient DMUs efficient targets. In this way, an inefficient DMU is benchmarked by its efficient peers. These efficient DMUs and their achieved data values are the observed targets for inefficient DMUs. Table 3 includes the data values of the least efficient patient and its target efficient benchmarks who have the better (efficient) treatment and the values inside the parenthesis give the contribution of the benchmarking DMU to the virtual target DMU.

By applying these multipliers to the data of corresponding DMUs, the virtual target DMUs are achieved. This are the achievable targets for the inefficient DMU. Table 3 demonstrates the target value determination for the least efficient treatment in this study. For the patient who has the least efficient treatment, it is obvious and impossible to reduce the pretreatment values. Moreover, sodium (Na), Potassium (K), and Hemoglobin (HB) values are taken as uncontrollable. Consequently, for this inefficient treatment, if the Creatinine (CRE) and Stone Area (AREA) values of the patient could be reduced from 1.11 mg/dL and 47 mm2 to 0.56 mg/dL and 0.00 mm2 respectively after treatment as indicated in Table 3, the treatment becomes efficient. This is the main strength of our proposal.

In the next stage of our study, the treatment success of patients varies according to gender and age was examined according to the Malmquist index. The decomposition of index revealed that the efficiency scores of males are a little bit more separated than those of females (within groups IE Male-Female =1.036).

On the other hand, the efficiency scores of patients who are over 40 are more consistent and unseparated then below 40 (IE <40 – 40+ =0.8990), however, the results are quite different between age and gender groups.

The Malmquist results showed that the females’ response to treatment was extremely different than those of males’, and similarly the patients’ (over 40 years) response was also quite different than the patients’ (under 40 years) (between groups IF Male-Female = 1670.357 and IF <40 – 40+ =0.000696). On the other hand, these results also coincide with the literature [22].

This is a new research concept, and it can be suggested to develop alternative treatment modalities for PNL by gender, age, or various groups.

As well as the obtained statistical results, clinical facts are also determinative over the efficiency of treatments. Therefore, as an example, the worst-case and reference efficient benchmarking DMU’s were evaluated clinically. Fundamentally, PNL aims to achieve maximal stone-free rates with minimal complications, but the definition of post operation success is not still clear. While the upper limit for clinically insignificant stones is generally taken as 4 mm in studies, the stone-free rates in PNL vary between 40% and 90% in the literature [4,6,23-26].

In this study, 113 patients were completely stone-free. However, unlike these cut-off values, the success of the treatment is evaluated with the DEA method, which should be emphasized. As was discussed above and shown in Tables 1, 2, and 3, the worst treatment has an efficiency score of %50.69 which has Post Treatment Stone Area value of 47 mm2. In the follow-up examinations, it has been revealed that the patient dropped this stone residue naturally. This is an indication that the treatment is successful or unsuccessful. The result obtained in DEA is completely directed towards the comfort of the patient and could be discussed as follows.

Adang, Gerritsma, Nouwens, van Lieshout, and Wensing [27] focused on primary care practices and applied a two-stage DEA approach to a data set of 106 GPs and 1671 patients. They found that technical efficiency increased depending on the status of the practice experience, but they could not obtain meaningful results regarding the inclusion of nurses' participation and practice size. Azadeh, Tohidi, Zarrin, Pashapour, and Moghaddam [28] aimed to seize the main factors which have negatively affected the patients’ satisfaction and explore their optimized levels. For this purpose, they used 51 DMUs and 10 indicators with a simulation-based DEA model to investigate the effects of changes in the input parameters on the quality measurement metrics. And proposed future work areas. Consequently, the authors have mentioned the application of the proposed simulation model for the estimation of ICU equipment for designing a new intensive care unit as a future study. Friesner et al [15] applied DEA to the patients of knee replacing surgery for the fiscal year 2002 and noticed that the rehabilitation process was quite effective in terms of providing consistent, quality care, as more than half of the patients in the study accepted as technically efficient and for rest of the patients that did not achieve maximum results, most could obtain increases in the degree of flexion gain and reductions in the degree of knee extension. PuigJunoy [29] intended to obtain empirical measures of performance in the management of critical patients treated in intensive care units. Therefore, to evaluate the factors associated with performance, a two-stage approach was applied to 993 critical care patients in intensive care units in Catalonia (Spain) in 1991–92. An extended version of DEA was used to obtain measures of technical efficiency in the treatment of in the first stage. In the second stage, a loglinear regression model is applied to test several hypotheses about the role of different environmental factors which include ownership, market structure, dimension, internal organization, diagnostic, mortality risk, etc. to explain differences in the efficiency scores. Consequently, the study obtained three main findings, DEA weights, distributions of efficiency scores, and factors relative to the efficient management of clinically ill patients. Nathanson, Higgins, Giglio, Munshi, and Steingrub [30], studied patients with severe head trauma retrospectively to determine the success of DEA if it fully models patients early in their stay in an intensive care unit. Variables examined were cerebral perfusion pressure, body temperature, mean arterial pressure, serum osmolarity and pCO2. DEA is applied to evaluate each patient individually calculating an “efficiency” score based on a patient’s ability to maximize output for a given set of physiologic inputs. Patients with high efficiency scores were found to have a better chance of making a full recovery than similarly injured patients that were inefficient. Nevertheless, they proposed a prospective study. Kamranrad, Amiri, and Niaki [31] performed a case study in health care and, categorized the kidney patient according to sex and age also, but it was rather different than DEA. They developed two approaches which are characterized by log-linear models to monitor multivariate categorical processes based on the contingency tables in Phase-I.

Furthermore, under Malmquist Analysis DEA context; Mohammedi et al [32] applied Malqmuist analysis with DEA for a macro level health performance comparison between Iran and upper-middle income countries, Bağcı and Konca [33] studied on the technical efficiency of hospitals which provides tertiary health care in Turkey, also Şahin and İlgün [34] aimed to reflect the efficiency of hospitals under the Ministry of Health in Turkey before and after the implementation of Public Hospital Associations. Sing et al [35] concerned health care system efficiency of ASEAN countries, Mitropoulos has used two stage DEA with Malmquist analysis to evaluate healthcare services during the economic crisis in EU countries and measured the production and quality performance. On the other hand, Zang et al [36] explored resource allocation in primary health care of mainland China. Sajadi et al [37] implemented an extended data envelopment analysis (EDEA) and Malmquist productivity index respectively to assess the efficiency of Iran health system with a selected set of counties. Similarly, Mısri and Asbu [38] evaluated the productivity change of national health systems in the WHO Eastern Mediterranean region.

However, in most of the studies in the literature, the issue is mostly dealt with financially as mentioned above, and although the patients are DMUs [5,13-16,27,28,30], the aim of the analyzes in these applications are to reveal whether the treatments are worth the resources spent. In this study, as stated above, unlike [16] the pre-treatment values of the patient were inputs, the post-treatment values were outputs, and the effect of the treatment itself on the patients will be evaluated.

As a result of the analysis, the treatments that are evaluated as efficient are shown as references that constitute the efficient frontier, the other treatments applied to the patients was considered as inefficient and directed to this frontier. Thus, by revealing the deficits of efficiency suggestions could be provided to further improve the treatment.

As was depicted in statistical results, despite the clinical evaluation the statistics reveal best practices against the rest of the data set and compares the data with each other. It is not important whether the treatment is clinically efficient, the performance of the treatment is relatively under its benchmarks it is characterized as inefficient and targeted to its benchmarks.

The decision here is in the physician, if the treating physician finds the treatment successful and the patient is in good health, or if a more advanced treatment will make the patient’s health worse than preoperatively, it would not be correct to talk about the inefficiency of the treatment, on the other hand, it is a guide in terms of treatments for future attempts which has similar results.

Although the study size of 154 treatments is relatively small, especially in treatments under the age of 40, Malmquist results confirmed the literature. However, for a more comprehensive assessment e.g. cross comparison between gender and age, further study would be needed.

In classical economic theory, it is normally expected to reduce inputs and/or increase output. However, in this study, while the inputs should be taken as the values that cause the decision of surgery before the operation, i.e. constant data, the success of the operation is obtained by providing a decrease in the postoperative values compared to the preoperative values. That is, the process runs counter to classical economic theory. For this reason, the DEA model has been modified to explain the process, that is, to provide output minimization. In this respect, it is quite groundbreaking in the literature.

Although DEA is widely used in medicine, it is rare to measure the success of the surgery applied directly to the patient as suggested in this study. Hence, for the future, with the development of the study, it will be able to create a guideline for the results to be obtained after the surgery with the evaluation and database connection with an efficient benchmarking frontier formed by the most efficient practices. Since these efficient applications will show how much improvement can be achieved in a new patient in which area, they will also have the ability to guide the treatment to be applied.

The authors declare that there is no conflict of interest to disclose.

The authors declare that there is no private or governmental financial aid, funding or any grant used in this study.

This research study was carried out under the Helsinki Declaration of Principles. The Scientific Research and Publication Ethics Committee approval was obtained dated 28th November 2019 with the record number 275 of approval.

Clinical Evaluation: SAÖ
Statistical Analyses: AG
Literature review and arrangement of the manuscript: AK