Journal Search Engine
Search Advanced Search Adode Reader(link)
Download PDF Export Citaion korean bibliography PMC previewer
ISSN : 1598-7248 (Print)
ISSN : 2234-6473 (Online)
Industrial Engineering & Management Systems Vol.18 No.4 pp.872-883
DOI : https://doi.org/10.7232/iems.2019.18.4.872

New Optimization Model for Multi-Period Multi-Product Production Planning System with Uncertainty

Asmaa A. Mahmoud*, Mohamed F. Aly, Ahmed M. Mohib, Islam H. Afefy
Industrial Engineering Department, Faculty of Engineering, Fayoum University, Egypt
Corresponding Author, E-mail: sweetjolut_2003@hotmail.com
July 12, 2019 August 19, 2019 October 16, 2019

ABSTRACT


The focus of this study is to develop a multi-period multi-product (MPMP) production planning system with uncertainty, and products demand (seasonal demand) uncertainty. Mainly, the problem aims reach the production levels of each product according to the uncertain demand for various periods, which depend on constraints of capacity, inventory, and resources. An analytical model proposed for this problem that can be categorized into two classes: non-linear and stochastic. The objective is to minimize the summation of variable production costs. As uncertain demand is a dynamic stochastic data process in the planning horizon, it is considered as a tree model. Each stage in the demand tree model is related to a cluster of a period time. Hence, depending on the tree model for the fluctuation demand; Two-Stage Stochastic Programming (TSP) model is presented as an alternative for all demand scenarios. In some of the reviewed articles validation of the analytical model were missing, while other studies were missing either manufacturing set up costs or assumptions of seasonal demand. Therefore, this study proposes TSP model using Sampling Average Approximation method (SAA) that is suitable for a production planning system in any manufacturing environment considering seasonal demand using optimization program (Lingo 16) to solve the mathematical model. Further, investigation of seasonal demand is performed using the multiplicative seasonal method, and the model validation was checked using Mathworks Matlab R2015a (64-Bit) considering manufacturing set up costs. Finally, some recommendations for future research are suggested.



초록


    1. INTRODUCTION

    Production planning is an important task that requires collaboration between different organization departments. To solve the problem of production planning, different optimization models have been introduced.

    In a competitive market, products demand may fluctuate significantly due to seasonal/periodical reasons. In such an environment, the existence of a reliable and stable production planning system is needed to adjust production rates with available resources and market demand. The objective of MPMP is to match production rates of each product with uncertain market demand overlong planning horizon. The target of this study is to define the MPMP production planning system suitable for the manufacturing environment, where several processes generate more than one product using several resources. The current study deals with seasonal product demand.

    The problem of production planning in the current study includes deciding the quantity of each product family that should be produced through each process in each slot of the planning phase using the TSP model. The study aims to minimize products inventory/backorder, over the cost of time and setup, according to product demands and machine capacities. The standard MPMP problem in the production planning system is introduced to determine individual products production rates in each production slot. Uncertainty of MPMP is analyzed with probabilistic modeling approach.

    In addition to determining the production volumes, appropriate production planning system should also propose the proper strategies for absorbing demand fluctuations. General strategies for absorbing demand fluctuations were summarized by (Fogarty et al., 1989) as follows: changing rate of production, uneven production rates by the introduction of outside subcontracting, accumulating seasonal inventories, and allowing backorder planning.

    MPMP models can generally be divided into deterministic and probabilistic (uncertain) as introduced by (Khor et al., 2008). Deterministic models are usually optimized using linear or non-linear programming approach, as shown in Figure 1.

    This paper is structured into five sections. In section 1, the MPMP approach was described shortly. Section 2, is a literature review. A methodology for PMS solving is addressed in section 3. Section 4, the results, where the outcomes for various cases under uncertainty using proposed model are presented, and finally conclusion and recommendation are summarized in section 5.

    2. LITERATURE REVIEW

    The main goal of many researchers was to obtain an optimized solution for a system consuming available resources to tolerate variability in products demand. Some of MPMP problem in different cases are summarized hereafter.

    MPMP was first introduced by (Masud and Hwang, 1980). In their study, Masud and Hwang structured multiple objective formulation of the MPMP whole production planning problem. Their suggested model considered the conflicting multiple objectives without regarding prior trade-off decisions through subjective cost estimation.

    Mula et al. (2006) provided a review article about production planning with uncertainty. Escudero et al. (1993) used multi-stage stochastic programming procedure for introducing the MPMP random demand model. They formalized implementable policies by a two casebased models. Bakir and Byrne (1998) introduced a stochastic linear programming model that is dependent on two-stage deterministic problem and took into account demand variability in MPMP production planning model. The production and capacity planning with uncertainty models based upon the Multi-stage Stochastic Programming (MSP) has been suggested by (Huang and Shabbir, 2010).

    The application of multi-stage stochastic models in MPMP and capacity planning in the manufacturing systems was also reviewed by Alfieri and Brandimarte (2005), the authors had stressed on the importance of proper model formulation from two points of view: first building strong mixed-integer formulations; while the second one was representing uncertainty in cases trees with customize size. Alfieri and Brandimarte (2005) had also compared the method of stochastic programming with traditional dynamic programming and to robust optimization. Kazemi et al. (2007) discussed two- stage stochastic models to study MPMP with uncertain output.

    Two robust optimization models with different optional cost variability measures for MPMP with uncertain output have been introduced by (Kazemi et al., 2008b). While, Byrne and Hossain (2005), presented an improved hybrid model that incorporated just-in-time (JIT) concepts. Kazemi et al. (2007) and Kazemi et al. (2008a) developed a comparative analysis of the model that had been developed by (Byrne and Hossain, 2005). Byrne and Hossain (2005) proved that their model had advantages over the other models due to the incorporated JIT considerations.

    Ramezanian and Saidi-Mehrabad (2013) introduced the flow shop system sizing and scheduling problem considering capacity constraints, sequenced setups, uncertain processing times, and MPMP demand uncertainty. Uncertain parameters evolution was modeled by the probability distributions and the theory of Chance- Constrained Programming (CCP). Computational results of random variable generated instances show the hybrid meta-heuristic efficiency versus exact solution algorithm and heuristics.

    A multi-objective, multi-period, multi-product, multi- site Aggregate Production Planning (APP) model was proposed by Entezaminia et al. (2016) in a green supply chain considering a reverse logistics network. The model showed how green standards could be integrated with an APP model. A numerical case study was used to examine the model validity. Al-Ashhab (2016) proposed enhancement model to solve the partner choice, and production planning problem in the manufacturing chains working under a multi-product, multi-echelon, multi-period, and multi-target manufacturing environment. The results verified the developed model performance and accuracy.

    Genetic Algorithm Optimization (GAO) and Big M approach had been used by (Hossain et al., 2016) to solve a real-time aggregate production planning decision problem. The suggested model was applied on an industrial case study. According to cost minimization objective of APP, GAO results were found to be better than the Big M method. Khalili-Damghani et al. (2017) proposed APP model with some parameters that were assumed to be uncertain and handled through a two-stage stochastic programming (TSSP) approach. The proposed TSSP is solved using three multi-objective solution procedures and was applied on a real case study of an automotive resin & oil supply chain.

    2.1 Research Gap

    Kazemi Zanjani et al. (2010) addressed MPMP production planning system under demands and processes output uncertainty. They have recommended further extensions of their work considering seasonal demand and different trends at each case. Therefore, the current work proposes TSP model that can be applied on production planning in any manufacturing environment considering seasonal demand. Nonlinear model is converted to a linear model to simplify the solution, using an optimization software program (Lingo program) to solve mathematical model. The general validation model is solved using Matlab software. TSP model could be accustomed to minimize total manufacturing costs of products for any manufacturing system. Table1 shows a comparison of the features of the proposed with the existing Kazemi Zanjani et al. (2010) model.

    3. METHODOLOGY

    In this research, the PMS problem is solved using Sampling Average Approximation method (SAA) using Lingo program, which is a simple and efficient comprehensive tool for constructing and solving optimization models. Additionally, it includes fast built-in functions that efficiently solve most optimization models cases. Lingo's PMS outputs are; cost reduction, time-saving, increasing market demand fulfillment, and achieving the target of the system under design constraints. An MPMP optimization model with stochastic data along the planning horizon was proposed. In TSP model, the decision should be emphasized to be announced today, given present resources, future uncertainties, and feasible future recourse actions. Scenario tree and objective function- based uncertainty are chosen to show the decision sequences risk, then the problem is solved as a large scale linear program. In the following, scenario trees characteristics are discussed, and then a general twostage stochastic programming for PMS formulation is provided.

    3.1 Mathematical Modeling

    The problem is developed as the analytic model (Non-linear, two-Stage stochastic programming as appropriate) which minimizes total variable production cost, inventory holding subject capacity restrictions, in addition to capacity and inventory balance constraints. This section will present three parts of the MPMP problem solution. The first part illustrates the objective function, the second defines the mathematical model (a deterministic mathematical model) where two-stage stochastic formulation of the problem (according to products demands uncertainty) is proposed, then in the third, indexing and notation as in appendix A are considered.

    3.1.1 Objective Function

    Variable production cost (i.e., costs of production, department setup, inventory holding, and regular capacity) minimization is our objective function. These variables can be clarified as

    M i n C o s t = t = 1 T j = 1 J ( i = F S ( j ) C i j t X i j t + i P j ( B P i j / Q i j t ) X i j t + i S j ( B S i j / Q i j t ) X i j t + o j t O j t + r j t R j t ) + t = 1 T i = 1 N h i t I F i t + t = 1 T i = 1 N b i t B i t + t = 1 T i = 1 N W i t w i t

    3.1.2 Mathematical Formulation

    Table 2 introduces a deterministic proposed mathematical formulation model for the system. The suggested model constraints are “families and items,” production and inventory constraints. Production capacity constraints and resources capacity constraints through the planning phase. The constraints are tabulated rely on the solution sequence. The constraints (2) and (6) are shown in Table 2 are families and items inventory constraints, both the amount of inventory left in the stock at the end of each period and the amount of inventory of the previous period. Backorder is not allowed. Constraint (7) shows the link between the item and family inventory. Constraints (3) and (8) are capacity feasibility constraints for group technology department and resources.

    3.2 TSP Development

    In this section, the proposed model for seasonal demand is first described, and then the production planning using the SAA method is formulated.

    3.2.1 Modeling Seasonal Demand

    It is supposed that the seasonal demand evolves as a discrete-time stochastic process along with the planning phase. This information structure may be planned as scenario trees, as shown in Figure 2. Nodes at stage t of the tree show the demand states (scenarios) that could be achieved by the information available up to stage t.

    For each stage, a restricted number of demand scenarios are considered (e.g., high, average, low). To define the scenarios for each stage, either the scenarios introduced by the experts or the traditional approach of making distributional assumptions suggesting the parameters from previous data could be used. To keep the size of the resulting multi-stage stochastic model acceptable, the planning phase is grouped into N stages, where each stage consists of number of periods. Stationary seasonal demand is assumed during each stage. For example, if the demand scenario of the first period at stage n is high, it remains the same (high) for the rest of the periods at stage n. However, the demand scenario might change (e.g., to low) for the first period in the next stage (n+1). It should be taken into account that the number of periods that may be planned at each stage depends on the demand behavior in the industry.

    3.2.2 TSP model for MPMP Production Planning with Seasonal Demand

    Assembling the problem as TSP model rely upon the scenario tree for the seasonal demand, the decision (control) variables of deterministic model equations (1:14) are the considered production plans. The inventory and backorder variables are considered the main variables of the plan. In this problem, we suppose that the decision-maker can adjust the production plan for different demand scenarios at each stage of the demand scenario tree. Otherwise stated, it is supposed at the start of the first period of each stage, enough data is available to the decision-maker to know which demand scenario is effective for that stage. Thus, it can be selected from the plans proposed by the TSP model for different scenarios. The compact formulation is applied to illustrate the problem, the decision variables are known for all node of the demand scenario tree. As stated by the last arguments, the following assumptions, also, are used in the Two-stage stochastic model.

    Assumptions Indices

    Scenario tree:

    • Scenario tree:

    • n, m Node of Scenario tree.

    • a(n) Definition of node n in the scenario tree.

    • tn Set of periods corresponding to node n in the scenario tree.

    Parameters

    • dit(n) Demand for family i in period t at node n of the scenario tree.

    • dktw(n) Demand for item k in sub-period w of period t at node n of the scenario tree.

    • P(n) Probability of node n of the scenario tree

    Decision Variables

    • IFit(n) Inventory of family i at the end of period t at node n of the scenario tree

    • Ikitw(n) Inventory of item k at the end sub-period w of period t at node n of the scenario tree

    • Bit(n) Backorder of family i at the end of period t at node n of the scenario tree

    • Rjt(n) Regular time used by department j in period t at node n of the scenario tree.

    • Ojt(n) Overtime used by department j in period t at node n of the scenario tree.

    3.2.3 TSP Model (Compact Formulation)

    The first term of the objective function determines the expected production cost for demand nodes of the scenario tree. The second term is the expected inventory, backorder and labor costs (regular time and overtime) for the demand nodes scenarios as developed by Kazemi Zanjani et al. (2010). Table 3 exhibits the model equations (15- 28), the decision variables are indexed for each node and each period time, since the stages do not correspond to periods. As it was mentioned in the previous section, each node; at a stage, contains a set of periods which is denoted by tn.

    In this model, there are coupling variables between different stages and these are the ending inventory and backorder variables at the end of each stage for family and items. Two different node indices (n, m) are used for inventory/backorder variables in the inventory balance constraints (16), (20) and (21). Specifically, at the first period of each stage, the inventory or backorder is computed by considering the inventory or backorder of the last period according to its instant predecessor node, while for the rest of the periods at that stage, the inventory/backorder size of the earlier period corresponding to the same node is taken into consideration.

    4. CASE STUDY

    This contextual analysis breaks down the effect of the progressions in parameters of a few objectives that must be accomplished. To start with, a genuine contextual analysis of General Manufacturing Company (GMC); in Egypt, is characterized. The company manufactures three items: fully-automatic washing machine, gas cooker and electrical water heater (EWH) with volumes of (30, 40, 50, 80,100 Liters).In this study, the results of applying the TSP model on EWH particularly with volume of 50 L, 80L and 100L; as an example, is presented. Thus, the proposed optimization model is solved using data collected from the manufacturer

    4.1 Problem Statement

    In this part, the problem formulation of this application is described. Initially the problem articulation is examined, then the arrangement technique is presented. The principal goal of the PMS is to reach the ideal technique that can handle the demand uncertainty (seasonal demand).The objective of the proposed contextual analysis is to limit the aggregate item cost and increase profit. Using SAA method introduced by Verweij et al. (2003) to achieve this objective it is important to manage and control the relationship between the restricted resources and fluctuation in items demand.

    4.2 Solution Method

    The SAA method is applied to manage uncertainty and solve the stochastic model which provides an efficient framework for identifying and statistically testing a variety of candidate production plans. The resulting SAA problem is then solved using deterministic optimization techniques

    In the SAA plan, a random sample of n scenarios of the random vector ζ and the expectation is as follows:

    n T r e e p ( n ) ( t = 1 t n i = 1 N h i t I F i t ( n ) + t = 1 t n i = 1 N b i t B i t ( n ) )

    This is approximated by the following sample average function:

    1 n [ n T r e e p ( n ) ( t = 1 t n i = 1 N h i t I F i t ( n ) + t = 1 t n i = 1 N b i t B i t ( n ) ) ]

    Using SAA, the “objective function” is given by equation (29):

    M i n Z ^ = n T r e e p ( n ) ( t = 1 t n j = 1 J ( i = F S ( j ) c i j t X i j t + i P j ( ( B P i j / Q i j t ) ) X i j t ) + i S j ( ( B S i j / Q i j t ) ) X i j t + o j t O j t ( n ) + r j t R j t ( n ) + 1 n [ n T r e e p ( n ) ( t = 1 t n i = 1 N h i t I F i t ( n ) + t = 1 t n i = 1 N b i t B i t ( n ) ) ] t = 1 t n i = 1 N W i t w i t
    (29)

    Using SAA plans, the solution of the mathematical model should be repeated with independent samples. Solution quality depends on the statistical confidence interval. In SAA there is an optimality gap; which is defined as the difference between true objective value and true optimal solution. In this study, normal distribution is used to obtain the optimality gap confidence interval which provides the sample mean and variance. This approach is presented as follows:

    Step (1): Create ng as independent identically distributed of samples batches of size n from the normal distribution of ζ i.e. {ζ1y, ζ2y, ζ3yζny}, for y=1……,ng. For each sample, solve the SAA equation (29). Let Z ^ n y Optimal objective value

    Step (2): Calculate

    Z ¯ n , n g = 1 n g y = 1 n g Z ^ n y
    (30)

    S Z ¯ n , n g 2 = 1 n g ( n g - 1 ) y = 1 n g ( Z ^ n y Z ¯ n , n g )
    (31)

    In this approach, it should be known that the value of Z ^ n is less than or equal to the optimal value Z * of the problem as E [ Z ¯ n , n g ]   Z * . Therefore, Z ¯ n , n g presents a lower statistical bound value for the optimal value Z * of the problem, while S Z ¯ n , n g 2 is the variance of the estimator E [ Z ¯ n , n g ] .

    4.3 Results

    In this section, the outcomes for various cases are demonstrated.

    4.3.1 Demand Scenario Tree

    In this investigation, three distinctive typical presentations for the demand of each item (per period) with different probability distribution (0.5, 0.3, and 0.2) are presented. Therefore, three interest trees (DT1, DT2, and DT3) and an aggregate of three test problems are considered. Figures (3, 4, and 5) show expected total cost for different probability distribution for the considered EWH (50, 80,100L). The uncertain demand was considered as a dynamic stochastic process and displayed as a scenario tree.

    In Figure 3 it could be seen that for EWH (50L), the total product costs are: 8.7E+09 L.E/Month, 1.08E+10 L.E/Month, and 1.47E+10 L.E/Month for probability distributions (0.2, 0.3, and 0.5), respectively.

    Similarly, the total product costs for EWH (80L) are 1.1E+10 L.E/Month,1.98E+09 L.E/Month, and 3.40E+09 L.E/Month at the probability distributions 0.2, 0.3, and 0.5 respectively which can be seen in Figure 4.

    In Figure 5, the total production cost of EWH (100L) is; 3.09E+09 L.E/Month, 3.25E+09 L.E/Month, and 3.52E+09L.E/Month for probability distributions (0.2, 0.3, and 0.5), respectively. For the three considered EWH volumes (50, 80, and 100 L) the uncertain (seasonal) demand is introduced as a dynamic stochastic procedure that is exhibited as a scenario tree.

    4.3.2 Estimation of Future Seasonal Demand

    Multiplicative seasonal method is used to estimate the seasonal demand. In this method seasonal indices are multiplied by an estimate of the average demand to calculate the seasonal demand forecast. Figure 6 exhibits the seasonal demand forecast of EWH (50 L). In this case the forecast is for the winter season, i.e. December, January, and February.

    5. CONCLUSIONS

    Various MPMP models applied to manufacturing and industrial processes have been reviewed. These models have been applied in a various production environment. Some of the reviewed models were deficient in:

    • - Validation of the mathematical model.

    • - The assumption of manufacturing set up costs ignored in the MPMP models

    • The assumption of seasonal demand and different trends at each stage of the demand scenario tree.

    In the current study, TSP model is proposed. The proposed approach can be used to represent production planning problem in different manufacturing environments, considering seasonal demand. Nonlinear models are converted to linear to simplify the solution process. (Lingo 16) optimization software program is used to solve mathematical model, while (Mathworks Matlab R2015a (64-Bit)) is used for solving intensive models validation considering set up costs. Future seasonal demand is estimated using multiplicative seasonal method, Applying TSP model could be used to minimize products’ manufacturing total costs which is required at any manufacturing system to:

    • - Reduce expenses to be competitive in the market.

    • - Achieve the system targets under uncertain criteria and constraints.

    • - Meeting market seasonal demand whilst saving time.

    As a result of the TSP model for the PMS, the recommendations for future study could be outlined as:

    • - Development of the model under more uncertain (random) constraints, for example, resources, and process yield.

    • - TSP model for the PMS model with other assumptions such as layout problems.

    ACKNOWLEDGMENT

    The authors would like to express their gratitude to the staff individuals of General Manufacturing Company (GMC) Egypt, for their help during conduct this research.

    Figure

    IEMS-18-4-872_F1.gif

    A mathematical model for MPMP.

    IEMS-18-4-872_F2.gif

    Scenario tree for TSP model.

    IEMS-18-4-872_F3.gif

    Expected total product cost for different probability distribution for EWH (50L).

    IEMS-18-4-872_F4.gif

    Expected total product cost with different probability distribution for EWH (80L).

    IEMS-18-4-872_F5.gif

    Expected total product cost with a different probability distribution for EWH (100L).

    IEMS-18-4-872_F6.gif

    Forecast seasonal demand for EWH (50L).

    Table

    Comparison of the features of the proposed model with the existing Kazemi Zanjani et al. (2010) model

    Deterministic proposed mathematical formulation model

    TSP Model (compact formulation)

    REFERENCES

    1. Al-Ashhab, M. (2016), An optimization model for multiperiod multi-product multi-objective production planning, International Journal of Engineering & Technology, 16(1), 43-56.
    2. Alfieri, A. and Brandimarte, P. (2005), Stochastic programming models for manufacturing applications: A tutorial introduction, design of advanced manufacturing systems, models for capacity planning in advanced manufacturing systems, Dordrecht, Netherlands, Springer, 73-124.
    3. Bakir, M. and Byrne, D. (1998), Stochastic linear optimization of an MPMP production planning model, In-ternational Journal of Production Economics, 55(1), 87-96
    4. Byrne, D. and Hossain, M. (2005), Production planning: An improved hybrid approach, International Journal of Production Economics, 94-94, 225-229.
    5. Entezaminia, A. , Heydaria, M. , and Rahmani, D. (2016), A multi-objective model for multi-product multi-site aggregate production planning in a green supply chain: Considering collection and recycling centers, Journal of Manufacturing Systems, 40(1), 63-75.
    6. Escudero, L. , Kamesam, P. V. , King, A. J. , and Wets, R. J. B. (1993), Production planning via scenario modeling, Annals of Operations Research, 43(6), 309 -335.
    7. Fogarty, D. , Hoffmann, T. , and Stonebraker, P. (1989), Production and Operations Management, South- Western Publishing Co, Cincinnati.
    8. Hossain, M. M. , Nahar, K. , Reza, S. , and Shaifullah, K. (2016), Multi-period, multi-product, aggregate production planning under demand uncertainty by considering wastage cost and incentives, World Review of Business Research, 6, 170-185.
    9. Huang, K. and Shabbir, A. (2009), the value of multistage stochastic programming in capacity planning under uncertainty, Journal of Operations Research, 57, 893-904.
    10. Kazemi Zanjani, M. , Nourelfath, M. , and Ait-Kadi, D. (2010), A multi-stage stochastic programming approach for production planning with uncertainty in the quality of raw materials and demand, International Journal of Production Research, 48(6), 4701-4723.
    11. Kazemi, M. , Nourelfath, M. , and Ait-Kadi, D. (2007), A stochastic programming approach for production planning in a manufacturing environment with random Yield, CIRRE LT, 58 Working Document, Canada.
    12. Kazemi, M. , Nourelfath, M. , and Ait-Kadi, D. (2013), A stochastic programming approach for sawmill production planning, International Journal of Mathematics in Operational Research, 5(1), 1-18.
    13. Kazemi, M. , Nourelfath, M. , and Ait-Kadi, D. (2008b), Robust production planning in a manufacturing environment with random yield: A case in sawmill production planning, CIRRELT, 52 Working Document, Canada.
    14. Khalili-Damghani, K. , Shahrokhb, A. , and Pakgoharc, A. (2017), Stochastic multi-period multi-product multiobjective aggregate production planning model in multi-echelon supply chain, International Journal of Production Management and Engineering, 5(2), 85-106.
    15. Khor, C. , Elkamel, A. , Ponnambalam, K. , and Douglas, P. (2008), Two-stage stochastic programming with fixed recourse via scenario planning with economic and operational risk management for petroleum refinery planning under uncertainty, Chemical Engineering and Processing, 47(9-10), 1744-1764.
    16. Masud, A. S. M. and Hwang, C. L. (1980), An aggregate production planning model and application of three multiple objectives decision methods, International Journal of Production Research, 18(6), 742-752.
    17. Mula, J. , Poler, R. , Sabater J. P. , and Lario, F. C. (2006), Models for production planning under uncertainty: A review, International Journal of Production Economics, 103(1), 271-285.
    18. Ramezanian, R. and Saidi-Mehrabad, M. (2013), Hybrid simulated annealing and MIP-based heuristics for stochastic lot-sizing and scheduling problem incapacitated multi-stage production system, Applied Mathematical Modeling, 37(1), 5134-5147.
    19. Verweij, B. , Ahmed, S. , Kleywegt, A. , Nemhauser, G. , and Shapiro, A. (2003), the sample average approximation method applied to stochastic routing problems, Computational Optimization and Applications, 24(2-3), 289-333.