Journal Search Engine
Search Advanced Search Adode Reader(link)
Download PDF Export Citaion korean bibliography PMC previewer
ISSN : 1598-7248 (Print)
ISSN : 2234-6473 (Online)
Industrial Engineering & Management Systems Vol.18 No.1 pp.143-153
DOI : https://doi.org/10.7232/iems.2019.18.1.143

Developing a Performance-Based Budgeting Maturity Model and Constructing a DEA-Based Composite Indicator to Measure It’s Score

Mohammadreza Amini*, Adel Azar, Mahmoud Dehghan Nayeri, Karim Bayat
Faculty of Management and Economics, Department of industrial management, Tarbiat Modares University, Tehran, Iran
Center of Management Studies and Technology Development, Tarbiat Modares University, Tehran, Iran
Corresponding Author, E-mail: Azara@modares.ac.ir
August 8, 2019 December 2, 2018 January 3, 2019

ABSTRACT


Performance-based budgeting has been a controversial topic over the past decades among budget professionals. Since the 1990s, a new wave of eagerness on performance-based budgeting has emerged among governments. In recent years, although utilization of performance-based budgeting has begun and various organizations have claimed the establishment of this system, its progress is not satisfactory. To evaluate the progress in establishing a performance based budgeting system, two budgeting maturity models have been presented. However, these two models cannot fully report the maturity of a performance-based budgeting system. By reviewing maturity reference models as well as two performance-based budgeting maturity models presented in budgeting literature, it has been attempted to highlight weaknesses of the recent models in order to present the developed performance-based budgeting maturity model. In addition to emphasizing the existence of necessary subsystems in a performance-based budgeting system, the model also emphasizes the results of this system. After developing a conceptual performance-based budgeting maturity model, it is now essential to develop an instrument for measuring this maturity. Obviously, this instrument should be able to aggregate criteria and sub-criteria of the conceptual maturity model in the form of a composite indicator and report the budgeting maturity indicator in the form of a score. Therefore, the second objective of this study is to develop a multi-layer data envelopment analysis model to measure PBB maturity index using optimization approach in constructing the composite indicator. By calculating the maturity score of each decision unit, the weight of sub-criteria in the macro indicator will be calculated.



초록


    1. INTRODUCTION

    Performance based budgeting has been the most popular reform effort in the public sector since the 1990s and perhaps in the present century (Melkers and Willoughby, 1988). During the 1990s, performance-based budgeting has been synonym with outcome-based budgeting and results-based budgeting (Herzog, 2006). Budgeting problem in the organization can be considered as a variety of organizational decision-making problem of which process has a semi-structured approach. Despite the existence of mathematical models proposed by various scholars for budget allocation problem, budget allocation problem is influenced by qualitative factors such as variable budgeting structures in different organizations as well as various quality factors and experiences of managers and decision makers (Russell and Norvig, 2016).

    Performance-based budgeting (PBB) is a budgeting system which provides the resources needed to achieve short-term and long-term goals, the cost of related programs and activities to achieve those goals and outcomes, or services that each program must produce or deliver (Andersson et al., 1996). In other words, PBB is a budgeting system which ideally links programs to results. One of the most important elements of this budgeting method is strategic planning, cost estimation and performance evaluation (Shah and Shen, 2007). PBB is presented in two phases. In the first phase, it is not successful; hence, plan budgeting, target-based management, zero-based budgeting have been used. Due to deficiencies that each of these methods brought with them, finally, return to PBB has occurred (Lu, 1996). According to researchers (Curristine, 2006;Kong, 2005), this budgeting method has many advantages over other budgeting methods, the most important of which are increased accountability, optimal performance management, improved resource allocation, simultaneous inclusion of cost and performance, ability to compare cost of departments and support of informed decisions about organizational resources.

    In Iran, budgeting system suffers from several problems. The main problems include the lack of a clear definition of the relationship between annual budget targets and Development Plan targets, implementation of incremental costing method and retrospective budget regardless of duties of organizations, personal tastes in determining maximum credits of executive agencies, unrealistic estimates of project costs, political allocation of resources, public unawareness of economic and social outcomes and goals of budget and effect of political tastes on budget (Nili and Rastad, 2007). It is anticipated that in the current situation if budgeting is based on performance, there would be 5-8% savings annually from the deployment of this system [Azar, 2009]. Therefore, implementation of this budgeting method is very important in Iran.

    PBB has been considered in Iran over a decade ago and its application has been emphasized in various laws, including the Fourth Development Plan Law, the State Service Management Law, the Fifth Development Plan, and general policies of the system. In addition to legal requirements, according to the Supreme Leader’s orders, the best economic model for Iran is a “resistive economy” model, which highlights efficiency, effectiveness, and cost savings. Considering that PBB is one of the main instruments for achieving efficiency and cost-effectiveness, consideration of its implementation in the current situation is more imperative.

    An idea beyond performance-based budgeting is that if policymakers make financial decisions objectively based on efficiency and effectiveness, then both they and people can have a clearer judgment about the performance of the government. In fact, performance-based budgeting enhances government accountability to legislators and people by linking budget decisions and government performance. In general, performance-based budgeting process seeks to answer these questions: where is our current position? Where do we want to be? How should we achieve these goals? How should we measure our progress?

    Performance-based budgeting has long been a reform proposal in developed and developing countries. In Iran, the idea of implementation of performance-based budgeting in recent years has been introduced first in paragraph (b) of the 23th note of the 2002 Budget Low. According to this paragraph: “to reform the budgeting system, the Iran Management and Planning Organization (is required) to implementing PBB, reform the revenue and expenditure estimating system for 2003 for all executive agencies and companies and organizations which are subjected to general laws and regulations and distribute cost-related credits based on the needs of agencies and activities taken.” This was also repeated in paragraph (t) of Note 1 of the 2003 Budget Low and paragraph (g) of Note 1 of the 2004 Budget Low (Azar et al., 2015).

    Each of the ministries, organizations, companies and public institutions, private or public, has done activities on design, implementation, and deployment of a PBB system. The Iran Management and Planning Organization has also developed a PBB plan which has been notified to all agencies in the annex to the 2004 Budget Low and has required some agencies to provide budget based on the system. Although the move towards the use of PBB system has begun in the budgeting process of ministries, organizations, companies and private and public institutions, its speed is not satisfactory and is proportional to the predicted times. Given this, organizations need to make their best efforts to change the budgeting system. In order to achieve this, it is necessary to identify, introduce and encourage the ministries, organizations, companies and private and public institutions which have taken actions in using PBB system. In addition to encouraging these organizations, other organizations can use their experiences to implement a PBB system.

    Accordingly, this study tends to, at the first, develop a PBB maturity model by review on the main models in the field of maturity, as well as special attention to current PBB maturity models and then present an optimization approach to calculating maturity score. So in section two a short review on PBB model and PBB maturity model are presented. The first contribution of our work is provided in section 3. So that by a critical reviewing on PBB maturity model, a developed PBB maturity model is suggested and the second contribution of this paper is provided on section four so that adopted by conceptual PBB maturity model presented in section 3, an optimization based CI model is suggested to calculating PBB maturity score. At the last section, in order to indicating DEA based-CI model’s capabilities, a numerical example is provided.

    2. LITERATURE REVIEW

    This section a short review on PBB models and PBB maturity models will be presented.

    2.1 PBB Models

    Different conceptual models and frameworks are presented for performance-based budgeting. Azar (2009) presented a comprehensive PBB model. In 2002, Robinson proposed a PBB cascade model (Robinson, 2002). Diamond (2005) also proposed a performance management framework for allocating budget (Diamond, 2005). In 2006, McGill introduced the rational PBB model (McGill, 2006). Furthermore, other researchers also examined various PBB models in various areas. For example, India’s banks, as well as the United Arab Emirates, have developed certain models of PBB in line with their bank structures (Azar et al., 2013). A short review on features of PBB models is presented in the Table 1.

    Definitely, a budgeting system is made up of subsystems; proper functioning of these subsystems can lead to desirable results. In the following, subsystems of each of the proposed budgeting models are compared. The criteria that these models are evaluated on and, in other words, are PBB subsystems included: planning, costing, performance evaluation, control, and monitoring, model universality, systemic approach to the budget.

    2.2 Maturity Model

    Maturity can be defined as a measure of ability, capability or capacity of an organization in relation to a certain field. Maturity models are conceptual models which represent the path of desired, rational, natural and predictable transformation towards maturity in a given context or field. Organizations often use maturity models to improve organizational performance. In other words, maturity models are designed to help organizations improve. Organizations also use maturity models as the basis for evaluating and comparing improvements in order to adopt a conscious approach to enhance the capability of a certain area in the organization. Maturity models are designed to measure maturity (complexity, competitiveness, and capability) of a selected domain based on a number of certain criteria.

    Maturity models are based on the assumption that there are predictable patterns in the evolution of organizations. They express this concept in different levels of transformation. These distinct levels illustrate a roadmap for improving organizations, in which each level surpasses the previous level in term of improvement and progress. Therefore, progress in this path of transformation means step by step development of organizational capabilities. For this reason, a set of traits or variables is required for modelling to describe the features and characteristics of each level based on them. Finally, these models provide the criteria and attributes which must be provided to reach a certain level of maturity. In other words, they provide a basis for evaluating maturity and give a picture of the organization and its status using the determined criteria. Based on the evaluation of the existing status of organizations, suggestions can be made to improve the organization in the desired field and prioritize them to achieve higher levels of maturity. So far, many maturity models have been presented in different domains and fields. Table 2 presents some of the most important models and brief explanations (Azar et al., 2015).

    These models are just a selection of dozens of maturity models in various fields; currently, organizations and enterprises in each field of activity have a set of best practices. Selecting a model or models which are close to businesses and their visions help to promote that organization. According to the subject, in the literature two PBB models are presented as below:

    2.3 PBB Maturity Models

    In addition to generic maturity models presented in the previous section, two PBB maturity models have been developed. The first model was developed by Neu-Brain Institute in 2012 (NeuBrain, 2012).

    2.3.1 Neu-Brain’s PBB Maturity Model

    Organizations have a number of PBB best practices and approaches, but rarely will they achieve a mature level all at once because implementing PBB is a process rather than an event. This process can be measured on a maturity level scale of 0-3, along which organizations will achieve differing PBB capabilities. Depending on how quickly an organization advances through the maturity levels, it will either institute stringent, light, or no disciplines (NeuBrain, 2012).

    2.3.2 Azar et al.’s PBB Maturity Model

    The second model was developed by Azar (2009). Process areas of the model are determined by reviewing the theoretical literature of budgeting and selective PBB model (Azar et al., 2013). Maturity levels of process areas of the model are defined based on (service) integration capability maturity model. Taking into account the definition of each level, the realization of twelve indicators of each domain, and the experiences gained in developing and implementing PBB in the organization, maturity levels of each process area are determined in development and deployment of PBB in the organization and the role of each process area is determined at maturity levels. For example, some indicators of information management are realized at the first level, some are realized at the second level, and the rest are realized at the third level. PBB maturity level is measured in each organization based on the realization of key indicators of each process area and aggregation of scores of maturity levels. In other words, each process area will be measured by twelve indicators. Based on these indicators, a Likert based-questionnaire is developed in each area as an instrument for measuring PBB maturity in each organization.

    Apart from these two models, two other studies have also been conducted on the budgeting maturity. However, none of these two studies provided a maturity budget model. Lu and Willoughby (2015) pointed to factors influencing the strength of the performance-based budgeting system and Popesko et al. (2017), investigated the relationship between maturity of the budgeting system and the company's performance. Popesko et al. (2017) defined the four factors, that provided by Libby and Lindsay (2010), as criteria for measuring the maturity of the budget. In the next section, a critical review on above study and models, will be leaded to designing a developed PBB Maturity model.

    3. A CRITICAL REVIEW ON CURRENT PBB MATURITY MODELS AND PRESENTING A DEVELOPED PBB MATURITY MODEL

    The key feature of PBB system is to integrate the goals of budget management system with accountability. This means that a certain amount of expenditures within the framework of each plan should meet a certain set of goals (Azar et al., 2013).

    Since the PBB maturity model uses CMMI integrated capability maturity model as a reference model, it should be noted that the main focus of current maturity models is on capabilities; although it emphasizes the establishment of some performance evaluation sub-systems, but there is no any emphasis on results and performance of PBB system and also realization of PBB system goals. In other words, the main focus of both current PBB maturity models of Azar et al. (2013) and NeuBrain (2012) is on capabilities of a PBB system and the role of results and outcomes is undermined.

    Also, Lu and Willoughby (2015) explains the factors influencing the strength of the performance-based budgeting system. While the present paper focuses on the capabilities and results of a mature budgeting system. Considering Popesko et al. (2017) model, shows that there is a weakness point on maturity definition. In other word, maturity reference models (like CMM) need to be considered in order to define maturity criteria. What was not seen in model of Popesko et al. (2017) is the lack of attention to the concepts presented in maturity reference models.

    So two main challenges of current models are mere “focus on capabilities of PBB system, not necessarily the results of PBB system,” and “application of statistical techniques and analyses to calculate maturity score.” Accordingly, our two main contributions on this study tends to dealing with these two challenges.

    3.1 Focusing Simultaneously on the Results and Capabilities of a PBB System

    The first challenge is to answer this question: “If an organization succeeds in fully deploying PBB sub-systems (as presented in Azar et al. (2013) and Neu-Brain (2013) models), can it be guaranteed that it necessarily has a mature budgeting system?”

    The answer to this question is to focus on the definition of maturity in these models. According to a definition given in Azar et al. (2013), maturity can be defined as a measure for measuring and evaluating the ability, capability or capacity of an organization in relation to a certain field or context (2013). According to this definition, the model focuses on capabilities and capacities of the organization rather than the organizational ability to the implementation of the PBB system.

    A review of the most important maturity models including capability maturity model (CMM), organizational project management maturity model (OPM3), personnel capability maturity model (P-CMM), knowledge management maturity model (KM3), process maturity framework (PMF), service-oriented architecture maturity model (SOAMM), service integration maturity model (SIMM) reveals that this kind of look (i.e. emphasize on capabilities) at maturity models also exists in other maturity models (Azar et al., 2015).

    Hence, the lost loop of the PBB maturity model is focused on results or outputs of the system (not necessarily existing a system for evaluation of result) as the most important indicator of the effectiveness of a successful deployment of a PBB system. Hence, the developed PBB maturity model by maintaining the features of the previous model will also include a second layer which emphasizes the results.

    As noted earlier, the developed PBB maturity model will have two main components. The first section will assess capabilities of the PBB sub-systems. This section, like models of Azar et al. (2013) evaluates six main subsystems of the budgeting system including “planning, information management, process management and documentation, costing system, performance management, and control and monitoring.”

    Obviously, the mere deployment and equipping of an organization with these sub-systems is not indicative of full maturity of PBB system. But the interaction of subsystems and real utilization of the PBB system in the budgeting process can be a more complete criterion for assessing maturity level of budgeting system.

    The second section of developed PBB maturity model will assess the results of successful deployment of a PBB system. In addition to evaluating all capabilities of a budgeting system, the extent to which the goals of that system are achieved will be also considered. In defining the indicators of results, what has been explained is focus on expectations that each organization will benefit from in the event of a full and correct implementation of PBB systems. Accordingly, through interviewing the budgeting experts, four main indicators including “transparency and accountability, budget discipline, application of costing information in budgeting process and application of performance indicators in budget formulation” are defined as results indicators of the developed PBB maturity model so that it will focus on efficiency and effectiveness of this system.

    It is necessary to point out that the indicators of results can be defined and analyzed on two levels. The first level involves indicators related to organizational goals (indicators such as improving employment status, etc.) and the second level involves goals of PBB system (indicators such as transparency and accountability, budget discipline and etc.). This study will be addressed on the second level indicators. Thus, our developed PBB maturity model is presented in Figure 1.

    To measure each of the six criteria of PBB system capabilities, including information management, planning, process management and documentation, costing system, performance management, and control and monitoring, Azar et al. (2013) presented 12 questions on the Likert scale (totally 72 questions). In order to measure PBB system results, several quantitative and qualitative criteria and sub-criteria are considered as follows:

    Transparency and accountability

    Frequency of budget reports, latency in report of previous year’s expenditures, transparency of the relationship between resources and expenditures (Likert based questionnaire)

    Budget discipline

    Ratio of the budget of various rows to total budget in million Rials, the number of amendments to the budget, the ratio of the amount of budget amendments to total budget, the ratio of the shift of cost headings to total cost in million Rials, the ratio of non-included credits to total credits

    Applying costing information in budgeting process

    Using cost information of activities (rate per unit activity), using the deviation about cost information of all activity, using the cost information for each cost center (cost per unit of an organization.

    Applying performance information in budgeting formulation

    Using the percentage to which goal of each plan or activity is realized, using a norm or average for the quantity of each plan, using the coefficient of plan quantity change, using the coefficient of change in cost per plan. It is noteworthy that costing and performance information will be scored by evaluating the budget support documents on a 5-point Likert scale. The second challenge of current maturity models, particularly the budgeting maturity models, is how to calculate the maturity score, as detailed in the next section.

    4. MULTILAYER DEA BASED-CI MODEL TO CALCULATE MATURITY SCORE

    Currently, most maturity models use qualitative questionnaires for assessment and evaluation so that using statistical techniques and analyses, maturity score is calculated. This is a remarkable point in Azar et al.’s budgeting maturity model (2009). While some mathematical techniques have been developed in evaluating and measuring performance. One of the main problems and challenges of current methods of measuring PBB maturity is neglect of the weight of sub-systems, as well as the lack of a measurement mechanism based on the comparison of the performance among the units.

    In order to overcome the challenge discussed, review of the literature on performance evaluation indicates that mathematical model of Data Envelopment Analysis (DEA) can be effectively used as an optimization approach for measuring budgeting maturity score. By reviewing different DEA models and considering the developed PBB maturity model, multilayer DEA model can be used as an approach for measuring PBB maturity score.

    Basic DEA models use linear programming techniques to measure relative efficiency of different decision units with multiple inputs and outputs (Nazari-Shirkouhi and Keramati, 2017;Shen, 2012;Yazdi et al., 2018). DEA has a self-assessment logic considered in various domains. In recent years, various studies have been done on the application of these models in constructing composite indicators (CI) to integrate a set of individual criteria into a general indicator (Shen et al., 2011a).

    DEA-based CI model was developed first in 1991 by Melyn and Moesen (1991) and used to assess macroeconomic performance. Mathematically, this model is the same as CCR model with constant input:

    C I O = max r = 1 S u r Y r o s . t . r = 1 S u r Y r j 1 j = 1 , , n u r ε r = 1 , , s
    (1)

    After presenting this approach, extensive studies have been carried out on construction of composite indicators based on DEA model; some of the most important studies are listed in Table 3.

    Basic DEA model will have weaknesses. Therefore, the question is whether the basic DEA model is capable of considering the hierarchical structure of criteria and sub-criteria or is it necessary to change the structure of basic model? By reviewing the literature of DEA models, there are different models for considering the hierarchical structure of criteria and sub-criteria. Various models have been proposed regarding the hierarchical, or multilayer, structure. In 2008, Meng et al. (2008) presented a twolayered structure for criteria and sub-criteria by presenting a two-level DEA approach in the evaluation of DMUs. However, the model of Meng et al. (2008) is a nonlinear model which does not compute the weight of sub-criteria. Their strategy for linear modeling is to use soft decision techniques such as AHP to calculate weighs of the subcriteria. Then, Kao (2008) presented a linear formulation of two-level DEA model by making the transformation and defining new variables to develop a linear form of the two-layer Kao’s model (Shen et al., 2011b). However, this model still contains only two layers of criteria and sub-criteria and does not offer a structural consideration with more than two layers. Shen et al. (2011b) consider the model of Meng et al. (2008) and Kao (2008) and presented a linear form of multilayer DEA model. The advantage of the model of Shen et al. (2011a) is to calculate the weight of criteria and sub-criteria by the model directly (Shen et al., 2011b). The multilayer DEA model is presented by Shen et al. (2011b) as follows:

    θ c = max f 1 = 1 S u ^ f 1 Y f 1 c s.t. g 1 = 1 M v ^ g 1 X g 1 c = 1 f 1 = 1 S u ^ f 1 Y f 1 j g 1 = 1 M v ^ g 1 X g 1 j   0 , j = 1 , ,   n f 1 A f k ( k ) S u ^ f 1 = u f k , f 1 = 1 ,   , s ; f k = 1 , , s ( k ) g 1 B g L ( L ) S v ^ g 1 = v g L , g 1 = 1 , , m ;   g L = 1 , , m ( L ) f 1 A f k ( k ) S u ^ f 1 f 1 A f k + 1 ( k + 1 ) S u ^ f 1 = P f k   f k   A f k + 1 ( k + 1 ) ( k ) f k = 1 , , s k , k = 1 , , K 1 g 1 B g L ( L ) m v ^ g 1 g 1 B g L + 1 ( L + 1 ) m v ^ g 1 = q g l   g l   B g l + 1 ( L + 1 ) ( l ) g L = 1 , , m L , l = 1 , , L 1 u f k , v g L ε f k = 1 , , s k , g L = 1 ,   , m ( L ) P f k   f k   A f k + 1 ( k + 1 ) ( k ) ξ f k = 1 , , s k , k = 1 , , K 1 q g l   g l   B g l + 1 ( L + 1 ) ( l ) ξ g L = 1 , , m L , l = 1 , , L 1 u ^ f 1 , v ^ g 1 ξ ( k 1 ) ε f 1 = 1 , , s ;   g 1 = 1 , , m
    (2)

    In the model (2), Y f k j represents the sub-criteria of the jth decision-making unit under the f criterion on the k level. In the research model, the values of Y f k j represent the normalized values of each sub-criterion on the lowest level of the criteria hierarchy. In addition, u ^ f k stands for the weight of the f criterion on the level. After calculating the weight of each sub-criterion in the last layer, the u f k variable is calculated as the weight of each criterion in the first layer based on the total weight f 1 A f k ( k ) S u ^ f 1 , of the sub-criteria of a criterion. Variable P f ( k ) also represents the weight of each sub-criterion under the f criterion. The relative frequency of the weights of the sub-criteria of each criterion is obtained to calculate the share. Based on previous section, the structure of criteria and sub-criteria in the budgeting maturity model is as follows:

    According to the structure presented in Figure 2, multilayer DEA based-CI model for measuring of PBB Maturity score is defined as follows:

    M a x   C I 0 = f 1 = 1 S U ^ f 1 Y f 1 0 s.t. f 1 = 1 S U ^ f 1 Y f 1 j 1 f 1 A f k k S U ^ f 1 = U f k = f 1 = 1 , , S , f k = 1 , S k f 1 A f k ( k ) S u ^ f 1 f 1 A f k + 1 ( k + 1 ) S u ^ f 1 = P f k   f k   A f k + 1 ( k + 1 ) ( k ) U f k ε P f k   f k   A f k + 1 ( k + 1 ) ( k ) ξ U ^ f 1 ξ ( k 1 ) ε
    (3)

    To improve discriminative power of model between decision units, some restrictions on the weight of criteria and sub-criteria which can be imposed to the model according to the expert judgments. It should be noted that the model will report the weight of sub-criteria for each decision-making unit, in addition to PBB maturity score; so that, the decision can be made for improving each criterion by focusing on the role of each sub-criteria in realizing the maturity.

    5. Numerical example

    After presenting mathematical formulation of our presented model, in this section its application will be presented. In order to indicate DEA based-CI model capabilities, suppose that there are 11 governmental organization in Iran that should be evaluated based on an Indicators hierarchy (that describe in fourth section).

    Our results will be presented in three types: CI score of maturity for each DMU, Ranking of all DMU based on CI score, weights of indicators and sub-indicators for each DMU. So, CI score of PBB maturity, ranking of all DMU and weights of indicators in first layer are presented on Table 4 and Figure 3.

    According to MLDEA-CI model, all normalized indicator values are ready to combine into a composite index for each DMU by selecting the best possible indicator weight. Distribution of scores and number of DMU with score 1 are two indicators for evaluating the discriminatory power. As it is shown in Table 4, the discriminatory power of model is farely proper.

    By applying multiple layer DEA-based CI model, the allocated weights of each indicator in each layer per DMU can be calculated, while in the basic DEA-based CI model, all indicators are simply treated to be in a one indicator and all weights will be allocated to indicators regardless of the indicator’s position in the hierarchical structure. While MLDEA-CI model not only seeks to attain the optimal score in the hierarchical form of the indicators but also to increase the discriminatory power of the model, the weight for each indicator in each layer can be restricted by expert opinion. As it is shown in MLDEA model, the weight of each indicator in each layer can be calculated by its formulation.

    In Table 4, the allocated weights (the values in brackets are shares) are presented based on the MLDEA-CI model for all DMUs so that, the allocated weights (shares) of first layer are provided in Table 4. The weights of second and third layers also are calculated by model which because of the paper limitations is being avoided.

    In MLDEA based-CI model, the weight assigned to each indicator can be interpreted as the importance share of that indicator (Shen et al., 2011a). For a better understanding, again look at First layer indicators in DMU04 from the aspect of share importance.

    For first layer, it is shown that the weight of the first criterion or I (PBB system capabilities) is 0.8098, while the weight of the second criterion or V is 0.1901. It implies that performance of I (PBB system capabilities) is better than V (PBB system results).

    6. DISCUSSION AND CONCLUSION

    Resource allocation is one of the major decision problems arising in higher education. Resources must be allocated optimally in such a way that the performance of universities can be improved (Melkers and Willoughby, 1988). Allocation of resources in government agencies and private companies is one of the main concerns of decision-makers. Although utilization of performance based budgeting has begun as one of optimal resource management methods, its progress is not satisfactory. In order to monitor the progress made in establishing a PBB system in different organizations and companies, various models have been measured PBB maturity.

    In literature, two PBB maturity model is presented (NeuBrain, 2012;Azar, 2009). The lost loop of the PBB maturity model is focused on results or outputs of the system (not necessarily existing a system for evaluation of result) as the most important indicator of the effectiveness of a successful deployment of a PBB system. Hence, the developed PBB maturity model by maintaining the features of the previous model will also include a second layer which emphasizes the results.

    As noted earlier, the developed PBB maturity model will have two main components. The first section will assess capabilities of the PBB sub-systems. This section, like models of Azar et al. (2013) evaluates six main subsystems of the budgeting system including “planning, information management, process management and documentation, costing system, performance management, and control and monitoring.”

    Obviously, the mere deployment and equipping of an organization with these sub-systems is not indicative of full maturity of PBB system. But the interaction of subsystems and real utilization of the PBB system in the budgeting process can be a more complete criterion for assessing maturity level of budgeting system.

    The second section of developed PBB maturity model will assess the results of successful deployment of a PBB system. In addition to evaluating all capabilities of a budgeting system, the extent to which the goals of that system are achieved will be also considered. In defining the indicators of results, what has been explained is focus on expectations that each organization will benefit from in the event of a full and correct implementation of PBB systems. Accordingly, through interviewing the budgeting experts, four main indicators including “transparency and accountability, budget discipline, application of costing information in budgeting process and application of performance indicators in budget formulation” are defined as results indicators of the developed PBB maturity model so that it will focus on efficiency and effectiveness of this system.

    It is necessary to point out that the indicators of results can be defined and analyzed on two levels. The first level involves indicators related to organizational goals (indicators such as improving employment status, etc.) and the second level involves goals of PBB system (indicators such as transparency and accountability, budget discipline and etc.). This study is addressed on the second level indicators.

    Although various studies have been carried out on the construction of DEA-based-CI in different domains, there is no any optimization study on optimization approach in the field of PBB maturity. Another method for providing maturity index is based on the mean of subcriteria, so that ultimate maturity score of each organization in most of these methods is calculated through the weighted average of related sub-indicators and statistical analyses. In other word, in previous studies, the maturity score was calculated and analyzed based on descriptive statistics (such as mean, standard deviation, and variance). While the model presented in this study considered the hierarchical structure of criteria and sub criteria and also aggregating of sub-criteria score will be calculated by the DEA based CI model directly. Taking advantage of the proposed approach of this study, in addition to calculating maturity score, it provides other useful information such as the weight of criteria and sub-criteria, benchmarks, and so on.

    Figure

    IEMS-18-1-143_F1.gif

    The developed PBB maturity model (authors’ conceptual model).

    IEMS-18-1-143_F2.gif

    Multilayer structure of the PBB maturity model.

    IEMS-18-1-143_F3.gif

    MLDEA based-CI scores of DMUs.

    Table

    Investigating and comparing conceptual PBB models (Azar et al., 2015)

    Types of maturity models in organization

    Some studies conducted on construction of composite indicators based on DEA

    CI score, Rank and Assigned weights (and shares) in the first layer based on the MLDEA based-CI model

    REFERENCES

    1. Andersson, E. A. , Oddsson, L. I. E. , Grundström, H., Nilsson, J., and Thorstensson, A.(1996), EMG activities of the quadratus lumborum and erector spinae muscles during flexion-relaxation and other motortasks, Clinical Biomechanics, 11(7), 392-400.
    2. Azar, A. (2009), PBB and performance auditing, Proceeding of audit conference on Performance Auditing, Accountability and Improved Productivity, (First Edition), Publication of the Center for Human Resources Development and Training, Tehran, Iran.
    3. Azar, A , Bayat, K. , Khadivar, A., and Amirkhani, T.(2013), PBB reward model, technical report (3rd ed), Performance-Based Budgeting Conference Secretariat, Tehran, Iran.
    4. Azar, A. , Dolatkhahi K. , and Goudarzi, G.(2015), Provide a conceptual model of performance-based budgeting, Public Management Vision, 24, 15-33.
    5. Cherchye, L. , Lovell, C. A. K. , Moesen, W., and Van Puyenbroeck, T.(2007), One market, one number? A composite indicator assessment of EU internal market dynamics, European Economic Review, 51(3), 749-779.
    6. Cherchye, L. , Moesen, W. , Rogge, N., Van Puyenbroeck, T., Saisana, M., Saltelli, A., Liska, R, and Tarantola, S.(2008), Creating composite indicators with DEA and robustness analysis: The case of the technology achievement index, Journal of the Operational Research Society, 59(2), 239-251.
    7. Curristine, T. (2006), Performance information in the budget process, OECD Journal on Budgeting, 5(2), 87-131.
    8. Despotis, D. K. (2005), Measuring human development via data envelopment analysis: The case of Asia and the Pacific, Omega, 33(5), 385-390.
    9. Diamond, M. J. (2005), Establishing a performance management framework for government, Working Paper No. 05/50, International Monetary Fund.
    10. Färe, R. , Grosskopf, S. , and Hernandez-Sancho, F.(2004), Environmental performance: An index number approach, Resource and Energy Economics, 26(4), 343-352.
    11. Hatefi, S. M. and Torabi, S. A. (2018), A slack analysis framework for improving composite indicators with applications to human development and sustainable energy indices, Econometric Reviews, 37(3), 247-259.
    12. Hermans, E. (2009), A methodology for developing a composite road safety performance index for crosscoountry comparison, Ph.D. dissertation, Hasselt University, Belgium.
    13. Herzog, R. J. (2006), Performance budgeting: Descriptive, allegorical, mythical, and idealistic, International Journal of Organization Theory & Behavior, 9(1), 72-91.
    14. Kao, C. (2008), A linear formulation of the two-level DEA model, Omega, 36(6), 958-962.
    15. Kong, D. (2005), Performance-based budgeting: The US experience, Public Organization Review, 5(2), 91-107.
    16. Libby, T. and Lindsay, R. M. (2010), Beyond budgeting or budgeting reconsidered? A survey of North- American budgeting practice, Management Accounting Research, 21(1), 56-57.
    17. Lu, E. Y. and Willoughby, K. (2015), Performance budgeting in American states: A framework of integrating performance with budgeting, International Journal of Public Administration, 38(8), 562-572.
    18. Lu, Lu , H. (1996), Performance budgeting resuscitated: Why is it still in viable?, Journal of Public Budgeting, Accounting & Financial Management, 10(2), 151-172.
    19. McGill, R. (2006), Achieving results: Performance budgeting in the least developed countries, Chapters 7 and 8, Available from: http://www.uncdf.org/english/local_development/uploads/thematic/UNCDF_pbbJuly2006.pdf.
    20. Melkers, J. and Willoughby, K. (1998), The state of the states: Performance-based budgeting requirements in 47 out of 50, Public Administration Review, 58(1), 66-73.
    21. Melyn, W. and Moesen, W. (1991), Towards a Synthetic Indicator of Macroeconomic Performance: Unequal Weighting When Limited Information is Available, Public Economics Research Papers, Katholieke Universiteit Leuven, Leuven.
    22. Meng, W. , Zhang, D. , Qi, L., and Liu, W.(2008), Twolevel DEA approaches in research evaluation, Omega, 36(6), 950-957.
    23. Nazari-Shirkouhi, S. and Keramati, A. (2017), Modeling customer satisfaction with new product design using a flexible fuzzy regression-data envelopment analysis algorithm, Applied Mathematical Modelling, 50, 755-771.
    24. NeuBrain(2012), Performance-Based Budgeting Methodology and Tools, White paper, Available from:https://icma.org/sites/default/files/305288_Neubrain_PBB_WhitePaper.pdf.
    25. Nili, M. and Rastad, M. (2007), Addressing the growth failure of the oil economies: The role of financial development, The Quarterly Review of Economics and Finance, 46(5), 726-740.
    26. Popesko, B. , Novák, P. , Dvorský, J., and Papadaki, Š.(2017), The maturity of a budgeting system and its influence on corporate performance, Acta Polytechnica Hungarica, 14(7), 91-104.
    27. Ramanathan, R. (2006), Evaluating the comparative performance of countries of the Middle East and North Africa: A DEA application, Socio-Economic Planning Sciences, 40(2), 156-167.
    28. Robinson, M. (2002), Best practice in performance budgetting, Discussion Paper, No.124, School of Economics and Finance, Queensland University of Technology.
    29. Russell, S. J. and Norvig, P. (2016), Artificial intelligence: A modern approach, Pearson Education Limited, Malaysia.
    30. Shah, A. and C. , Shen,(2007), A primer on performance budgeting. Budgeting and budgetary institutions, ed. I.A.A. (Ed), The World Bank, 137-178.
    31. Shen, Y. (2012), Inter-national Benchmarking of Road Safety Performance and Development using Indicators and Indexes: Data Envelopment Analysis based Approaches, Ph.D. dissertation, Hasselt University, Hasselt.
    32. Shen, Y. , Hermans, E. , Brijs, T., and Wets, G.(2014), Fuzzy data envelopment analysis in composite indicator construction, Performance Measurement with Fuzzy Data Envelopment Analysis, Springer, Berlin, Heidelberg, 89-100.
    33. Shen, Y. , Hermans, E. , Ruan, D., Wets, G., Brijs, T., and Vanhoof, K.(2011a), A generalized multiple layer data envelopment analysis model for hierarchical structure assessment: A case study in road safety performance evaluation, Expert Systems with Applications, 38(12), 15262-15272.
    34. Shen, Y. , Ruan, D. , Hermans, E., Brijs, T., Wets, G., and Vanhoof, K.(2011b), Modeling qualitative data in data envelopment analysis for composite indicators, International Journal of System Assurance Engineering and Management, 2(1), 21-30.
    35. Shwartz, M. , Burgess Jr, J. F. , and Zhu, J.(2016), A DEA based composite measure of quality and its associated data uncertainty interval for health care provider profiling and pay-for-performance, European Journal of Operational Research, 253(2), 489-502.
    36. Yazdi, M. R. T. , Mozaffari, M. M. , Nazari-Shirkouhi, S., and Asadzadeh, S. M.(2018), Integrated fuzzy DEA-ANFIS to measure the success effect of human resource spirituality, Cybernetics and Systems, 49(3), 151-169.
    37. Zhou, P. , Ang, B. W. , and Poh, K. L.(2007), A mathematical programming approach to constructing composite indicators, Ecological Economics, 62(2), 291-297.