• Editorial Board +
• For Contributors +
• Journal Search +
Journal Search Engine
ISSN : 1598-7248 (Print)
ISSN : 2234-6473 (Online)
Industrial Engineering & Management Systems Vol.17 No.4 pp.840-849
DOI : https://doi.org/10.7232/iems.2018.17.4.840

# The Use of Bootstrap for Weight Control in Data Envelopment Analysis

Akbar Amiri*, Saber Saati*, Alireza Amirteimoor
Department of Mathematics, Lahijan branch, Islamic Azad University, Lahijan, Iran
Department of Mathematics, North Tehran branch, Islamic Azad University, Tehran, Iran
Department of Mathematics, Rasht branch, Islamic Azad University, Rasht, Iran
* Corresponding Author, E-mail: s_saatim@iau-tnb.ac.ir
February 14, 2018 August 31, 2018 August 31, 2018

## ABSTRACT

Data Envelopment Analysis includes a wide range of mathematical models and is used for assessing the relative efficiency of a set of homogenous Decision Making Units (DMUs). Multiplicative DEA models achieve a set of weights for input and output variables for each DMU. Based on these weights, relative efficiency of each DMU is evaluated. Determining different weights for same factor in a set of homogenous DMUs is not realistic. To solve this problem, Common Set of Weights (CSW) is used. To improve the results of DEA models and reduce the weight flexibility, bootstrap method is used to determine a CSW. The aim of this article is to determine the CSW by finding a confidence interval for weights with bootstrap. By evaluating these intervals, a Probabilistic Common Set of Weights (PCSW) is obtained, which provides flexibility for weights.

## 1. INTRODUCTION

Data envelopment analysis (DEA) was proposed based on the research work by Farrell (1957) to evaluate the efficiency of a set of homogenous decision making units (DMUs). Charnes et al. (1978) were interested in Farrell (1957) and proposed CCR

model in 1978. The model was used for measuring and comparing the relative efficiency of organizational DMUs with multiple inputs and outputs. This method in known as a useful tool for evaluating the performance of organizations. Liu et al. (2013) surveyed the data envelopment analysis (DEA) literature by applying a citation- based approach. and With growth curve analysis hints, the DEA literature’s size eventually grow to at least double the size of the existing literature. In traditional DEA models, each DMU is in the best situation compared to other DMUs. The model is solved to achieve a set of best input and output weights. In these models, the obtained weights in the evaluation of different DMUs are not usually the same. In the other words, the same inputs and outputs have different weights in evaluating different DMUs and this is not realistic. On the other hand, because of the compensatory property of existing models, being with the assign a weight of zero may have some input and output factors are removed, that DMU under evaluation where there is weakness. To fix this problem, a common set of weights (CSW) is used. The idea of a CSW was first introduced by Roll et al. (1991) for using DEA in evaluating DMUs in highway maintenance. Roll et al. (1991) and Roll and Golany (1993) suggested several alternative approaches to generating the common weights, including running a general unbounded DEA model to obtain different sets of weights and then taking their average or weighted average with DEA efficiencies as the weights, maximizing the average efficiency of DMUs, maximizing the number of DEA efficient units, and ranking various factors by some order of importance and then assigning low weights to less important factors and maximal feasible weights to important ones. Sinuany-Stern et al. (1994) proposed a two-step linear discriminant analysis to obtain common weights. Friedman and Sinuany-Stern (1997) used a canonical correlation analysis to provide a single weight vector for inputs and outputs, respectively, common to all DMUs. Sinuany-Stern and Friedman (1998) provided a non-linear discriminant analysis to determine the common weights of DMUs. Liu and Peng (2008) developed a common weight analysis method to find common weights of DMUs. Hashimoto and Wu (2004) proposed a compromise programming model to find the common weights of DMUs by combining DEA and compromise programming model. Kao and Hung (2005) proposed a similar compromise solution approach for generating common weights the DEA framework. Wang et al. (2011) used regression analysis to find a CSW. In this method, the common weights are determined in views of efficiency fitting the DEA efficiencies computed with the most favorable weights to DMUs as their target efficiencies to be achieved, which are best fitted with the efficiencies computed from the common weights. By minimizing their fitting errors, the common weights are optimally estimated. Jahanshahloo et al. (2005) used a non-linear multi-objective fractional programming problem for creating the common weights. Saati et al. (2012) proposed a set of common fuzzy weights using a computationally-efficient virtual ideal units. Omrani (2013) combined uncertainty in the DEA model for the CSW proposed by Zohrehbandian et al. (2010). Omrani (2013) first developed a robust DEA based on optimal robust method proposed by Bertsimas and Sim (2004) to calculate the efficiency of each DMU as the ideal solution and then used ideal planning approximation to find the common weights by minimizing deviation from the ideal solution. Saati (2008) applied bounds on the factors to limit the flexibility of the DEA model in assigning weights to each DMU. Then, by limiting the flexibility, a CSW was proposed for the assessment of all DMUs. Payan et al. (2014) defined ideal DMUs as a criterion for efficient DMUs and obtained a model for finding a CSW.

Bootstrap was first introduced by Efron (1979) in 1979 for confidence intervals of a statistics. It was developed by Efron and Tibshirani (1993). Simar (1992) used bootstrap to estimate the efficiency of panel data in nonparametric models. In the case of unavailability of the analytic sampling distribution of efficiency, they provided a bootstrap method within this framework. Using this method, the statistical significance of estimation was evaluated. Ferrier and Hirschberg (1997) used bootstrap to assess the linear programming efficiency scores. Bootstrap was applied on efficiencies obtained from the DEA model. Using this method, a confidence interval was obtained. The use of bootstrap in nonparametric boundary models backs to a research work by Simar and Wilson (1998). They used bootstrap for the first time for sensitivity analysis of DEA efficiency. They also offered another bootstrap method to remove disadvantages of bootstrap introduced by Efron (1979). Simar and Wilson (2000) developed the algorithm introduced in Simar and Wilson (1998) and proposed a general framework for estimation of the data production process in non-parametric boundary models. Sadjadi and Omrani (2010) used bootstrap to estimate robust efficiency in DEA. Alexander et al. (2010) conducted a two-stage (DEA and regression) analysis of the efficiency of New Zealand secondary schools using bootstrap procedure. For the first time, Song et al. (2013) used the Super-SBM model to measure and calculate the energy efficiency of the BRICS, and then analyzed its development process. The Bootstrap was applied to modify the values based on DEA derived from small sample data, and finally the relationship between energy efficiency and carbon emissions were measured. Wanke et al. (2016) proposed new Fuzzy-DEAα-level models to assess underlying uncertainty. Furthermore, bootstrap truncated regressions with fixed factors are used to measure the impact of each model on the efficiency scores and identify the most relevant contextual variables on efficiency. The proposed models demonstrated using an application on Mozambican banks to handle the underlying uncertainty. Kim et al. (2016) used bootstrap and DEA models to study the efficiency of management, quality of services and quality of market network in the wireless industry. They concluded that distribution of wireless carriers with high management productivity is same as distribution of wireless carriers with high-efficiency network quality.

The above approaches are unable to separate efficient DMUs easily. In this article, a new bootstrap method is proposed to find a CSW. The use of common weights allows comparison DMUs on uniform basis. The new bootstrap method specifies a 95% confidence interval for input and output weights. A Probabilistic Common Set of Weights (PCSW) is obtained using this confidence interval. Using this interval, efficient DMUs are separated from each other.

This paper is organized as follows. The standard bounded CCR model is introduced in Section 2. Section 3 discusses the bootstrap method and confidence interval algorithm to obtain a set of probabilistic common weights. A numerical example is discussed in Section 4. Conclusions are presented in Section 5.

## 2. THE STANDARD BOUNDED CCR MODEL

Consider n DMUs, each with m inputs and s outputs. Suppose that $x i j ( i = 1 , ... , m )$ and $y r j ( r = 1 , ... , s )$ are the inputs and outputs of the $DMU j ( j = 1 , ... , n )$ respectively. Then, the efficiency of $DMU j ( j = 1 , ... , n )$ can be represented as follows:(1)

$θ j = ∑ r = 1 s u r y r j ∑ i = 1 m v i x i j ∀ j .$
(1)

where $v i ( i = 1 , ... , m )$ and $u r ( r = 1 , ... , s )$ are the input and output weights assigned to m inputs and s outputs respectively. Charnes et al. (1978) proposed the CCR model to determine the optimal set of input and output weights as follows:(2)

$M a x θ o = ∑ r = 1 s u r y r o ∑ i = 1 m v i x i o s . t : θ j = ∑ r = 1 s u r y r j ∑ i = 1 m v i x i j ≤ 1 ∀ j , u r , v i ≥ 0 ∀ r , i .$
(2)

Using Charnes and Cooper (1962) conversion, the fractional programming model can be converted to a linear programming problem as follows:

$M a x θ o = ∑ r = 1 s u r y r o s . t : ∑ i = 1 m v i x i o = 1 , ∑ r = 1 s u r y r j − ∑ i = 1 m v i x i j ≤ 0 ∀ j , u r l ≤ u r ≤ u r u ∀ r , v i l ≤ v i ≤ v i u ∀ i .$
(3)

The input and output weights in the DEA models are variable and calculated for each DMU by the model (3). Sometimes a very small or zero weight is assigned to an input or output. This reduces the effect of that factor in the efficiency of DMU under consideration. By applying bounded weights, the model flexibility in assigning weights to each DMU can be limited. The bounded model (4) can be used to control weight flexibility:

$M a x θ o = ∑ r = 1 s u r y r o s . t : ∑ i = 1 m v i x i o = 1 , ∑ r = 1 s u r y r j − ∑ i = 1 m v i x i j ≤ 0 ∀ j , ∑ r = 1 s u r y r j − ∑ i = 1 m v i x i j ≤ 0 ∀ j , u r l ≤ u r ≤ u r u ∀ r , v i l ≤ v i ≤ v i u ∀ i .$
(4)

where $u r l ( r = 1 , ... , s ) , u r u ( r = 1 , ... , s ) , v i l ( 1 , ... , m ) and v i u ( 1 , ... , m )$ are the lower and upper bounds on output and input weights, respectively. When this flexibility is limited to the extent that only one value is obtained for each weight, a common set of weights is obtained for evaluating all DMUs. Researchers have proposed several methods to obtain a common set of weights. In the next section, the bootstrap method is discussed to obtain a probabilistic common set of weights.

## 3. BOOTSTRAP AND COMMON WEIGHT

Bootstrap is a method to approximate sampling distribution and estimator accuracy sizes using existing data resampling. This method was first introduced by Efron (1979) in 1979 and widely used in many fields of science. Bootstrap is a simulation method that the actual data are used in the bootstrap method to obtain an empirical approximation. For this purpose, numerous samples are used to estimate the sampling distribution of a statistics by insertion from an existing sample. By generating many samples, the sample conditions become close to the conditions of population for calculating the average, standard deviation, confidence interval and skewness in statistical inference. Bootstrap in non-parametric boundary models was first introduced by Simar (1992). Simar and Wilson (1998) used bootstrap for sensitivity analysis and confidence intervals of efficiency. In bootstrap method, samples are taken from an existing sample B times with replacement. Then, the parameter is obtained for each sample and their average is considered as a bootstrap estimation.

If ${ v 1 , ... , v m }$ and ${ u 1 , ... , u s }$ are respectively the input and output weights from model (3), the aim is to gain a confidence interval for the input and output weights using bootstrap method. To achieve a common set of weights, bounded weights provides a way to find a common set of weights. In this method, upper and lower bounds are considered for input and output weights. In the present article, these bounds are obtained using the bootstrap method. The bounds are used as probabilistic bounds of input and output weights to achieve a set of probabilistic common weights. Suppose B is the number of samples in the bootstrap, then the bootstrap algorithm for calculating bounds is as follows:

Step 1: The weights and efficiency of all DMUs are obtained by model (3). Then these weights are examined to obtain the following column matrices:

$[ V i j ] n × 1 = 1 ∑ j = 1 n θ j [ θ j v i j ] n × 1 ( i = 1 , ... , m ) , and [ U r j ] n × 1 = 1 ∑ j = 1 n θ j [ θ j u r j ] n × 1 ( r = 1 , ... , s )$

where $θ j ( j = 1 , ... , n )$ is the efficiency, vij the weight of ith input and urj the weight of rth output of DMUj, respectively.

Step 2: By the observed samples $( V 1 j , ... , V m j )$ and $( U 1 j , ... , U s j ) ,$, a bootstrap sample $( V i j * , ... , V m j * )$ and $( U 1 j * , ... , U s j * )$ is obtained by simple random sampling with replacement.

Step 3: The average value of the sample $( V i j * , ... , V m j * )$ and $( U 1 j * , ... , U s j * )$ is calculated.

Step 4: Repeat the steps 2 and 3 B times to obtain B bootstrap statistics ${ V ¯ b * ; b = 1 , ... , B }$ and ${ U ¯ b * ; b = 1 , ... , B } .$ The standard error and estimator distribution are calculated by the following formula:

$U ¯ = 1 B ∑ b = 1 B U ¯ b * , S e = { 1 B − 1 ∑ b = 1 B ( U ¯ b * − U ¯ ) 2 } 1 2 V ¯ = 1 B ∑ b = 1 B V ¯ b * , S e = { 1 B − 1 ∑ b = 1 B ( V ¯ b * − V ¯ ) 2 } 1 2$

Step 5: The empirical cumulative distribution function for B bootstraps, $V 1 j * , ... , V m j *$ and $U 1 j * , ... , U s j * ,$ is respectively calculated as:

$G v ( t ) = P ( V ¯ * ≤ t )$ and $G u ( t ) = P ( U ¯ * ≤ t )$ Hence, $V ¯ b o o t ( t ) = G v − 1 ( α )$ and $U ¯ b o o t ( t ) = G u − 1 ( α ) .$

Step 6: A confidence interval of 100(1−α) % of bootstrap for the mean of input and output weights is obtained from the following equations:

$( V ¯ b o o t ( α 2 ) , V ¯ b o o t ( 1 − α 2 ) ) , ( u ¯ b o o t ( α 2 ) , U ¯ b o o t ( 1 − α 2 ) ) .$

Implementing the above steps and considering α = 0.05, a confidence interval of 95% is obtained as probabilistic bounds for input and output weights. Using the above algorithm, the bounded model (4) can be written as follows:

$M a x θ o = ∑ r = 1 s u r y r o s . t : ∑ i = 1 m v i x i o = 1 , ∑ r = 1 s u r y r j − ∑ i = 1 m v i x i j ≤ 0 ∀ j , ∑ r = 1 s u r y r j − ∑ i = 1 m v i x i j ≤ 0 ∀ j , u r l ≤ u r ≤ u r u ∀ r , v i l ≤ v i ≤ v i u ∀ i . U ¯ r ( b o o t ) ( α 2 ) ≤ u r ≤ U ¯ r ( b o o t ) ( 1 − α 2 ) ∀ r , V ¯ i ( b o o t ) ( α 2 ) ≤ v i ≤ V ¯ i ( b o o t ) ( 1 − α 2 ) ∀ i .$
(5)

where $U ¯ r ( b o o t ) ( α 2 ) ( r = 1 , ... , s ) ,$ $U ¯ r ( b o o t ) ( 1 − α 2 ) ( r = 1 , ... , s ) ,$ $V ¯ i ( b o o t ) ( α 2 ) ( i = 1 , ... , m )$ and $V ¯ i ( b o o t ) ( 1 − α 2 ) ( i = 1 , ... , m )$ are the lower and upper bounds on output and input weights obtained from the bootstrap algorithm respectively.

By the proposed algorithm we be able to design the bounded CCR problem with probabilistic bounds.

Theorem 1. Model (5) is feasible.

Proof. It is evident that the lower bounds of input and output weights obtained from the bootstrap algorithm are greater than or equal to zero.

Now, we have to show that the upper bounds of input and output weights obtained from the bootstrap algorithm are less than or equal to the upper bounds of variables in CCR model.

The upper bound of input and output weights are given by Saati (2008) as follows:(6)

$v i ' = 1 M a x 1 ≤ j ≤ n { x i j } i = 1 , ... , m u r ' = 1 M a x 1 ≤ j ≤ n { y r j } r = 1 , ... , s$
(6)

Since the bounds of the weights obtained by average of the achieved weights by CCR model, the weight intervals are a subset of the $[ 0 , v i ′ ]$ and $[ 0 , u r ′ ]$. That is, Model (5) is always feasible. ■

Bounded weights restrict the flexibility the DEA model in assigning a set of weights to DMUs. If flexibility in the DEA model is not possible, CSW is used to evaluate DMUs. Starting from the bounded model (5), the model (7) is provided to achieve a CSW as follows:

$M a x ϕ s . t : ∑ r = 1 s u r y r j − ∑ i = 1 m v i x i j ≤ 0 ∀ j , U ¯ r ( b o o t ) ( α 2 ) + ϕ ( U ¯ r ( b o o t ) ( 1 − α 2 ) − U ¯ r ( b o o t ) ( α 2 ) ) ≤ u r ≤ U ¯ r ( b o o t ) ( 1 − α 2 ) + ϕ ( U ¯ r ( b o o t ) ( 1 − α 2 ) − U ¯ r ( b o o t ) ( α 2 ) ) ∀ r , V ¯ i ( b o o t ) ( α 2 ) + ϕ ( V ¯ i ( b o o t ) ( 1 − α 2 ) − V ¯ i ( b o o t ) ( α 2 ) ) ≤ v i ≤ V ¯ i ( b o o t ) ( 1 − α 2 ) + ϕ ( V ¯ i ( b o o t ) ( 1 − α 2 ) − V ¯ i ( b o o t ) ( α 2 ) ) ∀ i$
(7)

The role of the variable ϕ in the model (7), minimize the range between the weights variable so that we reach a common weight. It can be easily show that the problem (7) is feasible and its optimal value is bounded and positive.

A PCSW is obtained by solving (7), and the efficiency of each DMU can be evaluated as follows:

$e j = ∑ r = 1 s u r * y r j ∑ i = 1 m v i * x i j , ∀ j$
(8)

where $u r * ( r = 1 , ... , s )$ and $v i * ( i = 1 , ... , m )$ are optimal values of (7).

## 4. NUMERICAL EXAMPLE

In this section two numerical examples are to be examined using the proposed model to test their discrimination capabilities.

Example 1. Consider the data of Kao and Hung (2005) as Table 1.

The efficiency values of these DMUs by CCR model are shown it the second column of Table 2. As seen, According to the results of the CCR model, nine units of the seventeen decision units are efficient. The weights of inputs and outputs are presented in column 3 to 9.These weights are very different for DMUs. For example, the weights of the first input may vary from 0 to 0.036898. The flexibility in selecting weights can cover many shortcomings in the DMUs, falsely displaying inefficient units as efficient. Fourteen items in the third output are associated with zero weight, making this output insignificant in the calculation of efficiency. In other words, the third output is ignored in the calculation of efficiency, showing another problem with the DEA models. In the following, it is attempted to reduce the number of 1 values by using the bounded CCR model. Using the results of Table 2 for weights of inputs and outputs and applying the proposed method, the bounds of factors are calculated as Table 3.

The columns of vi′ and ur′ are the upper bounds of input and output weights, respectively, in the standard CCR model. As seen the obtained probabilistic intervals are a subset of CCR weight intervals. The results of the model (5) are shown in Table 4.

It is evident that the number of efficient units was reduced from nine to five. It means that the efficient DMUs in Table 2, are week efficient. But, this model is also not capable of discriminating the efficient units from one another. Moreover, the weights of the same factors are dissimilar in different DMUs. For example, the weights of the first input may vary from 0 to 0.004856. Despite improvement compared to the Model (3), there is still flexibility in allocating weights in the Model (5). Also, the number of zero weights in the third output is reduced from fourteen to nine. In order to overcome these problem, a confidence interval was obtained for the weights of every input and output using the bootstrap algorithm (Efron and Tibshirani (1993), Davison and Hinkley (1997) and Hall (1986) B = 1000) (Table 3). Applying these constraints to the CCR model, a bounded CCR model is obtained. Then, using the Model (7), a set of Probabilistic Common Set of Weights (PCSW) was obtained and presented in Table 5. The efficiency of the DMUs in the Model (8) was calculated using the PCSW and presented in the seventh column of Table 6.

It is evident from Table 6 that by using the proposed method, all DMUs were discriminated with only one having a value of 1. The results of the CCR efficiency method, the Kao and Hung (2005) methods, and the bounded CCR are also presented in the second through sixth columns, none of which was able to discriminate all DMUs. Meanwhile, the proposed method (seventh column) managed to discriminate all DMUs without allocating a zero weight to any of the inputs or outputs. Moreover, the weights of the same factors are to resemble in different DMUs and the flexibility of the weights has been controlled.

Example 2. Consider the data of (Shang and Sueyoshi, 1995) as Table 7.

The efficiency values of these DMUs by CCR model are shown it the second column of Table 8. As seen, the traditional CCR model evaluates seven of the 12 DMUs as DEA efficient. The weights of inputs and outputs are presented in column 3 to 8.These weights are very different for DMUs. For example, the weights of the first input may vary from 0 to 0.27248.

The flexibility in selecting weights can cover many shortcomings in the DMUs, falsely displaying inefficient units as efficient. In the following, it is attempted to reduce the number of 1 values by using the bounded CCR model. Using the results of Table 8 for weights of inputs and outputs and applying the proposed method, the bounds of factors are calculated as Table 9.

The columns of vi′ and ur′ are the upper bounds of input and output weights, respectively, in the standard CCR model. As seen the obtained probabilistic intervals are a subset of CCR weight intervals. The results of the model (5) are shown in Table 10.

It is evident that the number of efficient units was reduced from nine to four. It means that the efficient DMUs in Table 8, are week efficient. But, this model is also not capable of discriminating the efficient units from one another. Moreover, the weights of the same factors are dissimilar in different DMUs. For example, the weights of the first input may vary from 0.026580 to 0.056370. Despite improvement compared to the Model (3), there is still flexibility in allocating weights in the Model (5). In order to overcome these problem, a confidence interval was obtained for the weights of every input and output using the bootstrap algorithm (B = 1000) (Table 9). Applying these constraints to the CCR model, a bounded CCR model is obtained. Then, using the Model (7), a set of Probabilistic Common Set of Weights (PCSW) was obtained and presented in Table 11. The efficiency of the DMUs in the Model (8) was calculated using the PCSW and presented in the seventh column of Table 12.

It is evident from Table 12 that by using the proposed method, all DMUs were discriminated with only one having a value of 1. The results of the CCR efficiency method, the Kao and Hung (2005) methods, and the bounded CCR are also presented in the second through sixth columns, none of which was able to discriminate all DMUs. Meanwhile, the proposed method (seventh column) managed to discriminate all DMUs without allocating a zero weight to any of the inputs or outputs. Moreover, the weights of the same factors are to resemble in different DMUs and the flexibility of the weights has been controlled.

## 5. CONCLUSION

Inability to separate efficient DMUs is one of the disadvantages of DEA. This issue has received much attention in DEA literature. In this article, bounded models were first used for weight control. By applying bounded weights, the model flexibility in assigning weights to DMUs was limited. Because finding all of the optimal answers is not easy considering the multiple answers of the CCR model, for this reason, we go to the bootstrap algorithm to estimate the probability of intervals. A confidence interval for the input and output weights with one of the standard DEA models was presented using bootstrap model. Using this confidence interval, a probabilistic common set of weights (PCSW) was obtained. This PCSW in the confidence interval removed some drawbacks of standard DEA model in weight control. If at least one of the input and output weights are non-zero, efficient DMUs can be differentiated using the PCSW obtained from the bootstrap method. The efficiencies obtained from the proposed method are probable and the units are efficient or inefficient with a probability of 95%.

## ACKNOWLEDGEMENT

The authors are grateful to the suggestions and comments made by anonymous referees on an earlier version of the paper.

## Table

Input and output data of the 17 DMUs

The results of the model (3) in Table 1

Confidence interval of input and output weights from the bootstrap method

The results of the model (5)

The PCSW obtained from the bootstrap method (B = 1000)

Efficiency and ranking scores from the Kao and Hung (2005) model, bounded CCR and the bootstrap method (proposed method)

Input and output data of the 12 DMUs

The results of the model (3) in Table 7

Confidence interval of input and output weights from the bootstrap method

The results of the model (5)

The PCSW obtained from the bootstrap method (B = 1000)

Efficiency and ranking scores from the Kao and Hung (2005) model, bounded CCR and the Bootstrap method (proposed method)

## REFERENCES

1. Alexander, W. R. J. , Haug, A. A. , and Jaforullah, M. (2010), A two-stage double-bootstrap data envelopment analysis of efficiency differences of New Zealand secondary schools, Journal of Productivity Analysis, 34(2), 99-110.
2. Bertsimas, D. and Sim, M. (2004), The price of robustness, Operations Research, 52(1), 35-53.
3. Charnes, A. and Cooper, W. W. (1962), Programming with linear fractional functionals, Naval Research Logistics (NRL), 9(3-4), 181-186.
4. Charnes, A. , Cooper, W. W. , and Rhodes, E. (1978), Measuring the efficiency of decision making units, European Journal of Operational Research, 2(6), 429-444.
5. Davison, A. C. and Hinkley, D. V. (1997), Bootstrap Methods and Their Application, Cambridge University Press.
6. Efron, B. (1979), Computers and the theory of statistics: thinking the unthinkable, SIAM Review, 21(4), 460-480.
7. Efron, B. and Tibshirani, R. J. (1993), An introduction to the bootstrap, Number 57 in Monographs on statistics and applied probability. Chapman & Hall, New York.
8. Farrell, M. J. (1957), The measurement of productive efficiency, Journal of the Royal Statistical Society. Series A (General), 120(3), 253-290.
9. Ferrier, G. D. and Hirschberg, J. G. (1997), Bootstrapping confidence intervals for linear programming efficiency scores: With an illustration using Italian banking data, Journal of Productivity Analysis, 8(1), 19-33.
10. Friedman, L. and Sinuany-Stern, Z. (1997), Scaling units via the canonical correlation analysis in the DEA context, European Journal of Operational Research, 100(1), 629-637.
11. Hall, P. (1986), On the number of bootstrap simulations required to construct a confidence interval, The Annals of Statistics, 14(4), 1453-1462.
12. Hashimoto, A. and Wu, D. A. (2004), A DEA-compromise programming model for comprehensive ranking, Journal of the Operations Research Society of Japan,47(2), 73-81.
13. Jahanshahloo, G. R. , Memariani, A. , Lotfi, F. H. , and Rezal, H. Z. (2005), A note on some of DEA models and finding efficiency and complete ranking using common set of weights, Applied Mathematics and Computation, 166(2), 265-281.
14. Kao, C. and Hung, H. T. (2005), Data envelopment analysis with common weights: The compromise solution approach, Journal of the Operational Research Society, 56(10), 1196-1203.
15. Kim, C. , Kim, S. W. , and Kang, H. J. (2016), Driving sustainable competitive advantage in the mobile industry: Evidence from US wireless carriers, Sustainability, 8(7), 659-672.
16. Liu, F. H. F. and Peng, H. H. (2008), Ranking of units on the DEA frontier with common weights, Computers & Operations Research, 35(5), 1624-1637.
17. Liu, J. S. , Lu, L. Y. , Lu, W. M. , and Lin, B. J. Y. (2013), Data envelopment analysis 1978-2010: A citationbased literature survey, Omega, 41(1), 3-15.
18. Omrani, H. (2013), Common weights data envelopment analysis with uncertain data: A robust optimization approach, Computers & Industrial Engineering, 66(4), 1163-1170.
19. Payan, A. , Noora, A. A. , and Lotfi, F. H. (2014), A ranking method based on common weights and benchmarkpoint, Applications & Applied Mathematics, 9(1), 318-329.
20. Roll, Y. , Cook, W. D. , and Golany, B. (1991), Controlling factor weights in data envelopment analysis, IIEtransactions, 23(1), 2-9.
21. Roll, Y. and Golany, B. (1993), Alternate methods of treating factor weights in DEA, Omega, 21(1), 99-109.
22. Saati, S. (2008), Determining a common set of weights in DEA by solving a linear programming, Journal of Industrial Engineering International, 4(6), 51-56.
23. Saati, S. , Hatami-Marbini, A. , Agrell, P. J. and Tavana, M. (2012), A common set of weight approach using an ideal decision making unit in data envelopment analysis, Journal of Industrial and Management Optimization, 8(3), 623-637.
24. Sadjadi, S. and Omrani, H. (2010), A bootstrapped robust data envelopment analysis model for efficiency estimating of telecommunication companies in Iran, Telecommunications Policy, 34(4), 221-232.
25. Shang, J. and Sueyoshi, T. (1995), A unified framework for the selection of a Flexible Manufacturing System. European Journal of Operational Research, 85, 297-315.
26. Simar, L. (1992), Estimating efficiencies from frontier models with panel data: a comparison of parametric, non-parametric and semi-parametric methods withbootstrapping, International Applications of Productivity and Efficiency Analysis, Springer, 167-199.
27. Simar, L. and Wilson, P. W. (1998), Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models, Management Science, 44(1), 49-61.
28. Simar, L. and Wilson, P. W. (2000), A general methodology for bootstrapping in non-parametric frontiermodels, Journal of Applied Statistics, 27(6), 779-802.
29. Sinuany-Stern, Z. and Friedman, L. (1998), DEA and the discriminant analysis of ratios for ranking units, European Journal of Operational Research, 111(3), 470-478.
30. Sinuany-Stern, Z. , Mehrez, A. , and Barboy, A. (1994), Academic departments efficiency via DEA, Computers & Operations Research, 21(5), 543-556.
31. Song, M. L. , Zhang, L. L. , Liu, W. , and Fisher, R. (2013), Bootstrap-DEA analysis of BRICS’ energy efficiency ased on small sample data, Applied Energy, 112, 1049-1055.
32. Wang, Y. M. , Luo, Y. , and Lan, Y. X. (2011), Common weights for fully ranking decision making units by regression analysis, Expert Systems with Applications, 38(8), 9122-9128.
33. Wanke, P. , Barros, C. P. , and Emrouznejad, A. (2016), Assessing productive efficiency of banks using integrated Fuzzy-DEA and bootstrapping: A case of Mozambican banks, European journal of operational research, 249(1), 378-389.
34. Zohrehbandian, M. , Makui, A. , and Alinezhad, A. (2010), A compromise solution approach for finding common weights in DEA: An improvement to Kao and Hung’s approach, Journal of the Operational Research Society, 61(4), 604-610.