Journal Search Engine
Search Advanced Search Adode Reader(link)
Download PDF Export Citaion korean bibliography PMC previewer
ISSN : 1598-7248 (Print)
ISSN : 2234-6473 (Online)
Industrial Engineering & Management Systems Vol.14 No.4 pp.325-334
DOI : https://doi.org/10.7232/iems.2015.14.4.325

A Quantitative Study of Influencing Factors on Crowd Participation in a Crowdsourcing Project for Consumer Product Design

Tuananh Tran, Joon Young Park*
Department of Industrial and Systems Engineering, Dongguk University, Seoul, Korea
Corresponding Author, jypark@dgu.edu
March 30, 2015 September 14, 2015 October 10, 2015

ABSTRACT

Nowadays, crowdsourcing has become a popular concept and is widely used in many social, economical and technological areas. For consumer product design, crowdsourcing is implemented extensively with many cases. Although there has been a lot of research on the application of crowdsourcing for product design, the big picture of how factors influence the participartion of individuals from the crowd in a crowdsourcing project for product design has not yet been understood. This paper aims to investigate the relationships of crowd participation and influencing factors including: process, product, and reward. To do this, we conducted a survey on a crowd of engineering individuals and analyzed the collected data with data mining techniques. Main findings include the relationships of crowd participation versus process, product, and reward factors as well as regression models to predict crowd participation.


초록


    Ministry of Educationhttp://dx.doi.org/10.13039/501100002701$$NO.013R1A1A2013649

    1.INTRODUCTION

    1.1.Application of Crowdsourcing to Consumer Product Design

    According to Ulrich (2012), (consumer) product design is defined generally as “the set of activities beginning with the perception of a market opportunity and ending in the production, sale and delivery of a product.” Market opportunity may come from market pull or from technology push. The “technology push” describes a situation where an emerging technology or a new combination of existing technologies provide the driving force for an innovative product and problem solution in the marketplace (Herstatt and Lettl, 2000). Conversely, the term “market pull” implies that the product or process innovation has its origins in latent, unsatisfied customer needs in the marketplace. The identification of these needs occurs first and is then followed by the required development activities (Chidamber and Kon, 1994). There are two parallel paths involved in the product design process: one involves the generation of ideas (idea generation), development of concepts (concept development) and detailed engineering calculations (detailed engineering); the other involves market research and marketing analysis (market pull) or internal R&D (technology push). Engineering oriented researchers are interested in the first path which is common in both technology push and market pull strategy; it consists of idea generation, concept development and detailed engineering. Product design plays an important role among the activities of a company. Conventionally, product design is taken place inside the company within the design team. The external resources are limited to close partners and/or vendors only.

    The development of web 2.0 offers a lot of chances for companies to receive benefits from user involvement. There have been many cases of projects whose development is based mostly on users’ contribution, such as: iStockphoto (Howe, 2006), Wikipedia (Kleemann, 2008), Threadless (Howe, 2009), etc. In these cases, the company or organization, who owns the project, makes an “open call” through the open Internet environment for people’s participation in their projects by contributing users’ work, knowledge, designs, comments, votes, etc. The action of giving an open call to the “crowd” on the Internet and receiving feedback in the form of user participation with solutions, suggestions, designs, comments, votes, etc. is termed “crowdsourcing” by Howe (2006). “Crowdsourcing” is the combination of “crowd” and “outsourcing” which means “outsource to the crowd.” A popular definition of crowdsourcing nowadays is “the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of individuals in the form of an open call” (Howe, 2009).

    Using the Internet for supporting product design activities such as customer research has been proposed previously (Shekar, 2012) but further, many companies nowadays run product design projects implementing of crowdsourcing concept. Those include Local Motors (Redlich, 2008), Dell Computers (Kleemann, 2008), Fiat Mio CC (Brondoni, 2010), etc. The basic concept of crowdsourcing is simple: anyone has the potential to plug in valuable information (Greengard, 2011). Implementation of crowdsourcing can help companies to grasp solutions in short time and at low cost (Brabham, 2008; Blohm et al., 2011). The design activities where crowdsourcing concept can be potentially applied are: Idea Generation, Concept Development, Detail Engineering Design, Physical Prototyping and Design Evaluation.

    1.2.Foundation of Crowdsourcing

    Users’ motivations and information technology infrastructure development are the foundation for the rise of crowdsourcing nowadays. Major motivations for individuals from the crowd to join a certain crowdsourcing project can be summarized in Table 1.

    “Web 2.0” is another factor that contributes to the foundation of crowdsourcing. Web 2.0 allows individuals from the crowd to participate easily in crowdsourcing projects by submitting contents (ideas, suggestions, files, etc.) and leaving comments, votes, etc. Without these features of Web 2.0 (or later web generation), crowdsourcing projects cannot be executed on the Internet.

    1.3.Neccessity of a Quantitative Study of Influencing Factors on Crowd Participation

    The total performance of a crowdsourcing project depends on two conditions: crowd participation (neccessary condition) and quality of user-submitted solutions (sufficient condition). Therefore, crowd participation is very important to a crowdsourced product design project and it must be analyzed in details for effective implementation of crowdsourcing.

    So far, most researchers considered reward as the only factor leading to participation of the crowd in a crowdsourcing project (Albors et al., 2008; Horton, 2010) and ignored other factors (i.e. “process” factor, “product” factor). How factors (reward, process, and product) altogether influence crowd participation is still a research question.

    2.LITERATURE REVIEW

    2.1.Crowdsourcing for Consumer Product Design

    Recently, much research has been done on application of crowdsourcing for consumer product design. Howe (2006) wrote the article in which the term crowdsourcing is coined for the first time. Kleemann (2008) reviewed many cases of applying crowdsourcing concept for product design and suggested two types of crowdsourcing that can be applied to consumer product design, which are ‘Participation of consumers in product design and configuration’ and ‘Product design.’

    Hinchcliffe (2007) claimed that “leveraging crowdsourcing effectively” is a critical factor for “product development 2.0” in terms of competitive advantage. In other work, Hinchcliffe (2009) suggested five functional business areas that are suitable for applying crowdsourcing. These are problem solving, design, collaborative work, testing and support. Furthermore, Bertoni et al. (2011) suggested that crowdsourcing, through Web 2.0, can be an intuitive way to leverage bottom-up tools for the benefit of product design where customers and the crowd play in the innovation process. Snow et al. (2010) also agreed that crowdsourcing can be used as a problem solving model for collaboration issues in organizing continuous product design and commercialization.

    One of the reasons that crowdsourcing can be a source of product innovation is that there might be someone outside the company who knows the solution to the problems faced inside the company (Panchal et al., 2008). Poetz and Schreier (2012) claimed that crowdsourcing initiatives among users can actually outperform professional in-house activities for the generation of product ideas, at least under certain conditions. Taha et al. (2011) studied user’s involvement in product design and identified factors governing user involvement and designer practices. Crowdsourcing for consumer product design can also benefit firms by using consumers’ expertise, and the innovation provided by consumers can also be used for marketing purposes (Kleemann, 2008; Whitla, 2009).

    There are debates on the benefits of opening the product design process, such as the one which was discussed in the work of Knudsen and Mortensen (2011). Similarly, Cooper and Edgett (2008) commented that crowdsourcing results are weak (in some cases), and recommend to implement crowdsourcing in certain types of firms. In addition, there is a lack of practical guideline for firms to decide what kind of task should be outsourced to the crowd in order to gain high rate of participation.

    2.2.Previous Works on Crowd Participation

    To obtain good results from crowdsourcing implementation for product design projects, crowd participation is an important issue. The high crowd participation rate might lead to high chance of bringing innovation (Redlich, 2008).

    The participation of the crowd in a certain crowdsourced product design project depends on many factors, such as: reward factor, process factor, product factor and crowd factor. So far, most of research considered reward as the major factor that leads to the participation of the crowd to a certain crowdsourcing project. Reward factor is mentioned and considered as the main motivation for crowd participation by many authors (Albors, 2008; Brabham, 2008; Malone, 2010; Blohm, 2011).

    Mason et al. (2009) claimed that increased payments increases the quantity of work performed, but not its quality, and particular design of the compensation scheme can have a significant effect on quality in a crowdsourcing project. Horton (2010) presented a model of workers supplying labor to paid crowdsourcing projects. In this work, a method of estimating a worker’s reservation wage (the lowest wage a worker is willing to accept for a task) has been introduced. Borst (2010) explored the effects of motivations and rewards on participation as well as performance in voluntary online activities.

    However, effects of other factors (rather than reward) on crowd participation have not been investigated extensively. Only a few noticeable papers mentioned the effect of task complexity which includes the characteristics of product design phases on the continuance of the sustained participation (Sun, 2012).

    2.3.Research Objectives

    For a product design project where the crowdsourcing concept is implemented, there are four main factors affecting the crowd participation: reward factor, process factor (i.e. product design steps or activities), product factor and crowd factor (Mason et al., 2009; Malone, 2010; Horton, 2010; Tran et al., 2012). Details of these four factors are shown in Table 2.

    In terms of mathematical function, we can write:

    CP = f(reward, process, product, crowd)
    (1)

    where CP stands for Crowd Participation

    This paper focuses on quantitatively figuring out the variation of crowd participation, measured by crowd participation rate, for different combinations of process, product and reward variables with a fixed crowd of engineering individuals, which is one kind of qualified crowd (Adepetu, 2012), through a survey study. The reason why we chose qualified crowd to perform the survey is that qualified crowd can bring higher crowd participation (i.e. quantity) and better crowdsourcing performance (i.e. quality) and thus, this kind of crowd is important to product design. Data mining techniques are used to analyze the collected data from the survey and the quantitative and visualized results help to bring a big picture of how factors affect crowd participation. The regression models generated in this paper can help companies to make strategic decisions when implementing crowdsourcing for their product design projects.

    3.RESEARCH METHODOLOGY

    The authors conducted a survey on an online community of engineering individuals. This community had more than 60,000 members in total and is actively available on the internet in a form of a discussion forum with more than 26,600 engineering topics (as of March, 2015).

    We explained the issues related to the survey (i.e. products, process, rewards, and the crowdsourcing project itself) and asked each individual whether they would agree to participate the crowdsourcing project for a certain combination of {reward, process, product}. In this survey, we considered four kinds of products: Plastic Cup, (Fruit) Blender, Washing Machine and CNC Machining Center. We chose those four kinds of products due to their levels of complexity. In this paper, it is assumed that, in terms of product complexity, those products can be classified as follows (Table 3):

    Details of each product design step and their complexity levels, according to Tran et al. (2012), are shown in Table 4. The reward for this survey ranges from 0 (i.e. no reward) to 0.9X (with increment of 0.1X) where X equals to the amount of money the individual would receive if he or she works as a permanent employee in a company (i.e. average wage).

    4.RESULTS AND DISCUSSIONS

    4.1.Minimum Reward amount for Various Crowdsourcing Tasks

    We obtained 120 responses from the individuals for the survey. Table 5 shows average amount of minimum reward (i.e. minimum required amount for one individual to participate) for all products and Figure 1 and Figure 2 represent those data graphically.

    From Figure 1, for Detail Engineering Design and Physical Prototyping, the minimum amount of reward to attract individuals to participate is higher than that for steps of Idea Generation, Concept Development and Design Evaluation. From Figure 2, for Plastic Cup, the minimum amount of reward to attract individuals from the crowd to participate is quite low and this amount increases for Blender, Washing Machine and CNC Machining Center. This result comes from the difference of the complexity level of products and steps.

    4.2.Crowd Participation for Various Crowdsourcing Tasks

    From the survey responses, the authors calculated for each combination of certain reward, step and product, how many percent of individuals from the crowd would accept to participate in the crowdsourcing project. Crowd participation rate (CP) is the ratio between the number of individuals who would participate and the total surveyed crowd population. Figures from 3 to 6 illustrate crowd participation rates for Plastic Cup, Blender, Washing Machine and CNC Machining Center, respectively, for different combinations of reward amounts and product design steps.

    From Figure 3 to Figure 6, we can see that crowd participation rate tends to be linearly proportional to reward amount for simple products (i.e. Plastic Cup, Blender) and the slopes of the “curves” tends to decrease for complicated products (i.e. CNC Machining Center). Figure .4, 5

    4.3.Regression Models for Predicting Crowd Participation

    Regression models are used to numerically predict output values. From the set of collected data, the authors built regression models and used the models to predict output values for certain combinations of input variables. Input variables include reward, product, and process factors. Output variable is crowd participation.

    The authors calculated crowd participation (CP) rate for each combination of {reward, product, and process} by counting the number of individuals who agreed to participate for each combination of {reward, product, and process} and divided it to 120 (i.e. total number of individuals who responded to the survey questions). We obtained the data which are partly shown in Table 6.

    In this paper, the authors consider 2 kinds of regretssion: Regression Tree and Multiple Linear Regression. Both of these 2 regression algorithms are carried out by the Data mining software named XLMiner (Shmueli, 2007). The reason why we choose these 2 kinds of regression is that we concern more on the overall participation than on behavior of individuals. These 2 kinds of regression produce outputs that can be used to estimate the amount of users in a given population would possibly participate in the crowdscourcing project and thus, these regression models can help decision makers to evaluate chances of success of a crowdsourcing project. In this sense, models such as: Classification Tree, Logistic Regression, etc. or the C4.5 classifier cannot be used.

    Regression Tree result with XLMiner is shown in Figure 7. This Regression Tree model shows us, for a certain given combination of {reward, product, and process}, how much crowd participation we can obtain. A real world example can be found in Section 5.

    The authors also conduct a Multiple Linear Regression model to predict precise value of crowd participation.

    Multiple Linear Regression result with XLMiner is shown in Table 7. From Table 7, we can write:

    CP = 0.78(Reward)-0.0174(Step Complexity) -0.05(Product Complexity)+0.242
    (2)

    Using formula (2), for any combination of {reward, product, process}, we can calculate crowd participation rate. A demonstration of how formula (2) works can be found in Section 5.

    4.4.Validation of Regression Models

    We validate the models with randomly selected surveyed results for products of Plastic Chair, Phone Case, Portable Gas Stove and Refrigerator (which were surveyed in the same manner with the previous products). The predicted and actual results are shown in Table 8. The numbers in the parentheses in Table 8 indicate absolute values of the differences between predicted participation and actual participation.

    As mentioned in Section 4.3, Multiple Linear Regression model is used because we want to predict amount of participation as a whole instead of to know behavior of each individual. In this work, we limit the research within crowdsourcing tasks with step complexity is smaller than 5 and product complexity is smaller than 4. For products which are not too complicated, i.e. and product complexity is smaller than or equal to 3; even when reward equals to 0, the minimum CP is 0.005, from formula (2). When the product is at the highest level of complexity, i.e. product complexity is equal to 4, we assume that, in order to make the crowdsourcing project to be feasible, minimum reward needs to be 0.1. This assumption matches real world situations of applying crowdsourcing to product design: If the task is too complicated, there would be zero participation without adequate amount of reward. With the assumption of minimum reward is 0.1, for the most complicated product and step, minimum CP is 0.0033. This assumption ensures CP to be positive in all cases.

    We perform a Chi-square test to verify the independence of Reward (R), Product complexity (P) and Step complexity (S) variables. P and S are categorical variables and R can be considered as a categorical variable by assigning levels of reward from 0 to 10: Reward level 0 (i.e. R = 0); Reward level 1 (i.e. R = 0.1), Reward level 2 (i.e. R = 0.2), etc. Hence, Chi-square test is applicable to test the indepence of R, P and S.

    In order to select a sample for testing the independence of R and S, we randomly select records which show the number of participants for P = 2 (Blender). For testing the independence R and P, we randomly select records which show the number of participants for S = 4 (Detail Engineering Design). And for testing the independence of R and P, we randomly select records which show the number of participants for R = 0.4 (Reward level 4). The result of Chi-square test for the independence of R, S and P is shown in the Table 9 below. The result verifies that R, S and P are statistically independent.

    5.INDUSTRIAL APPLICATIONS

    The results of this paper can support companies with tools and guidelines to enhance the performance of their crowdsourcing project for consumer product design in terms of participation. Our Regression Tree provides companies a quick tool for assessment of how much participation can be obtained for a given set of input variables (i.e. reward, process and product). For instance, Mulenserv is a company who is preparing for a crowdsourcing project for designing their new book cover. In this project, they would ask users to generate design concepts for the book. The product complexity equals to 1 and step complexity equals to 2 in this case. using Regression Tree model in Figure 7, if Mulenserv offers a reward amount of 0.4, the amount of crowd participation they can obtain is about 0.31 (i.e. 31%). If this company increases reward amount to 0.5, they can obtain crowd participation rate of 0.61 (i.e. 61%) which is far different from 0.4 reward amount. In this case, small change in the input (reward amount increases 20%) can lead to large change in the output (CP increases 96,7%).

    Our Multiple Linear Regression model allows Mulenserv to calculate crowd participation rate more precisely. For the above project, if this company decides reward level of 0.6, crowd participation can be calculated as follows:

    CP= 0.78 × 0.6 0.0174 × 2 0.05 × 1 +0.242=0.63
    (3)

    The regression models in this paper can be applied to crowdsourcing projects in other areas including cell phone industry. In this sense, a cell phone manufacturer can open a crowdsourcing project which calls for innovative ideas from potential users. These ideas may include suggesting new features, enhancing UI/UX, customizing exterior design, etc. Given a certain condition of crowdsourcing task and amount of reward, using our regression models, the manufacturer can predict amount of participation.

    Our results also provide companies useful suggestions for their implementation of crowdsourcing concept for product design. Those are:

    Suggestion 1: Try to simplify the crowdsourcing task. Simplify the product and choose easy design steps. This is because products and steps with low complexity level attracts higher participation. If the crowsourcing task is complicated, devide it into sub-tasks.

    Suggestion 2: Formula (2) suggests that reward is the most influencing factor on crowd participation rate. The coefficient of reward factor in formula (2) is much larger than that of product and process factor. Increasing reward amount is the fastest way to grow crowd participation.

    Suggestion 3: Increasing the reward amount would lead to the increasing of crowd participation rate. But as shown in Figure 3 to Figure 6, for complicated products, the influence of increasing reward amount on increasing crowd participation tends to decrease with the increasing of product complexity (the slopes of the “curves” tends to decrease for complicated products).

    Suggestion 4: Figure 7 suggests that, even for the zero amount of reward, there are individuals from the crowd willing to participate in the crowdsourcing project. Companies can boost this participation by appropriate design of the crowdsourcing task (i.e. simplify the task).

    The above examples of how our Regression Tree and Multiple Linear Regression work and the suggestions from our results might help companies to implement crowdsourcing projects for consumer product design more effectively.

    6.CONCLUSIONS

    Through conducting a survey to a crowd of engineering individuals and analyzing the responses, the big picture of how factors influence crowd participation in a crowdsourced product design project has been investigated. The relationship between crowd participation rate and other factors has been figured out quantitatively. Regression models for predicting crowd participation have been constructed and validated with surveyed data. These models have high accuracy levels and can be used as direct guidelines as well as decision making supporting tools for companies who want to implement crowdsourcing for their product design projects. The models can help a company to quickly estimate the participation rate for a given qualified crowd and a set of conditions from the company (i.e. combinations of {product, process, reward}).

    There are also limitations to this paper. First, as the crowd is fixed to a qualified crowd, the results cannot be applied for a random crowd. Secondly, as we build the models based on a limited number of products, the results might not be accurate when being applied for products that have complexity levels higher than 4 or lower than 1 or not equivalent to those which are used to build the models. Moreover, in this paper, we use a quite rough estimation to assign numeric values complexity levels of process and product factor. Future work may include the analysis of actual industrial cases with wider range of products as well as a more precise numeric assignment method of product and process complexity levels.

    Figure

    IEMS-14-325_F1.gif

    Minimum reward amount for different product design steps.

    IEMS-14-325_F2.gif

    Minimum reward amount for different products.

    IEMS-14-325_F3.gif

    Crowd participation for different reward amounts (Plastic Cup).

    IEMS-14-325_F4.gif

    Crowd participation for different reward amounts (Blender).

    IEMS-14-325_F5.gif

    Crowd participation for different reward amounts (Washing Machine).

    IEMS-14-325_F6.gif

    Crowd participation for different reward amounts (CNC Machining Center).

    IEMS-14-325_F7.gif

    Regression tree from XLMiner.

    Table

    Major motivations for participating a crowdsourcing project

    Four factors affecting crowd participation in a crowdsourcing project for consumer product design

    Assignment of product complexity with numerical values

    Details of product design steps and assignment of step complexity with numerical values

    Minimum amount of reward required for participation for different steps and products

    A partial extraction of the regression datasheet

    Multi Linear Regression results with XLMiner

    Differences between predicted participation and actual participation

    Result of Chi-square test for the independence of R, S and P

    REFERENCES

    1. Adepatu A , Ahmed KA , Zaabi Y , Abd A , Svetinovic D (2012) CrowdREquire: A Requirements Engineering Crowdsourcing Platform , AAAI Technical Report SS-12-06,
    2. Albors J , Ramos JC , Hervas JL (2008) New learning network paradigms: Communities of objectives, crowdsourcing, wikis and open source , International Journal of Information Management, Vol.28 ; pp.194-202
    3. Bertoni M , Chirumalla K (2011) Leveraging Web 2.0 in New Product Development: Lessons Learned from a Cross-company Study , Journal of Universal Computer Science, Vol.17 (4) ; pp.548-564
    4. Blohm I , Faehling J , Krcmar H , Leimeister M , Fischer J (2011) Accelerating customer integration into innovation processes using Pico Jobs , International Journal of Technology Marketing, Vol.6 (2) ; pp.130-147
    5. Borst WAM (2010) Understanding Crowdsourcing: Effects of Motivation and Rewards on Participation and Performance in Voluntary Online Activities , Doctoral Thesis, EPS-2010-221-LIS, ISBN: 978-90-5892-262-5, http://hdl.handle.net/1765/1
    6. Brabham D (2008) Crowdsourcing as a Model for Problem Solving , Convergence, Vol.14 ; pp.75-90
    7. Brondoni S (2010) Intangibles, Global Networks and Corporate Social Responsibility , Symphonya Emerging Issues in Management, Vol.2
    8. Chidamber SR , Kon HB (1994) A research retrospective of innovation inception and success: the technology-push, demand-pull question , International Journal of Technology Management, Vol.9 (1) ; pp.94-112
    9. Cooper RG , Edgett SJ (2008) Ideation for product innovation: What are the best methods? , Visions Magazine, Vol.32 (1) ; pp.12-17
    10. Greengard S (2011) Following the crowd , Communications of the ACM, Vol.54 (2) ; pp.-20
    11. Herstatt C , Lettl C (2000) Management of technology push development projects , http://www.tuharburg.de/tim/downloads/arbeitspapiere/Working_Paper_5.pdf,
    12. Hinchcliffe D (2007) Product Development 2.0 , Communications and Strategies, Vol.65 ; pp.105-111
    13. Hinchcliffe D (2009) Crowdsourcing: 5 reasons it’s not just for startups any more , Ebizq, URL: http://www.ebizq.net/blogs/enterprise/2009/09/crowdsourcing_5_reasons_its_no.php,
    14. Horton J , Chilton LB (2010) The labor economics of paid crowdsourcing , Proceedings of the 11th ACM conference on Electronic commerce,
    15. Howe J (2006) The rise of crowdsourcing , Wired Magazine, Vol.14 (6)
    16. Howe J (2009) Crowdsourcing-Why the Power of the Crowd is Driving the Future of Business , Three Rivers Press,
    17. Huberman BA , Romero DM , Wu F (2009) Crowdsourcing, Attention and Productivity , Journal of Information Science, Vol.35 (6) ; pp.758-765
    18. Kleemann F (2008) Un(der) paid Innovators: The Commercial Utilization of Consumer Work through Crowdsourcing , Science, Technology and Innovation Studies, Vol.4 ; pp.5-26
    19. Knudsen MP , Mortensen TB (2011) Some immediate- but negative-effects of openness on product development performance , Technovation, Vol.31 ; pp.55-64
    20. Malone TW , Laubacher R , Dellarocas C (2010) Harnessing Crowds: Mapping the Genome of Collective Intelligence , MIT Sloan School Working Paper, ; pp.4732-09
    21. Mason WA , Watts DJ (2009) Financial incentives and the ‘performance of crowds’ , Association for Computing Machinery Explorations Newsletter, Vol.11 (2) ; pp.100-108
    22. Panchal JH , Fathianathan M (2008) Product Realization in the Age of Mass Collaboration, Proceedings of IDETC/CIE ASME , International Design Engineering Technical Conferences and Computers and Information in Engineering Conference,
    23. Piller FT Anne S Huff (2010) Interactive value creation with users and customers , Leading Open Innovation, Peter-Pribilla Foundation, ; pp.16-24
    24. Poetz M , Schreier M (2012) The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas? , Journal of Product Innovation Management, Vol.29 (2) ; pp.245-256
    25. Redlich T , Wulfsberg JP , Bruhns FL (2008) Virtual Factory for Customized Open Production , URL: http://www.mikromaschinenbau.com/redlich/redlich_open_production_290608.pdf,
    26. Shekar A (2012) Internet User Research in Product Development: rapid and low cost initial research for Information , International Journal of Industrial Engineering: Theory, Applications and Practice, Vol.19 (1)
    27. Snow CC , Fjeldstad OD , Lettl C , Miles RE (2011) Organizing Continuous Product Development and Commercialization: The Collaborative Community of Firms Model , Journal of Product Innovation Management, Vol.28 ; pp.3-16
    28. Sun Y , Fang Y , Lim KH (2012) Understanding sustained participation in transactional virtual communities , Journal of Decision Support Systems, Vol.53 ; pp.12-22
    29. Taha Z , Alli H , Rashid SHA (2011) Users involvement in product development: A designers’ perspectives, , Industrial Engineering and Management Systems, Vol.10 ; pp.191-196
    30. Tran AT , Hasan S , Park JY (2012) Crowd Participation Pattern in the Phases of a Product Development Process that Utilizes Crowdsourcing , Industrial Engineering and Management Systems, Vol.11 (3) ; pp.266-275
    31. Ulrich KT , Eppinger SD (2012) Product Design and Development , . Irwin, Mc Graw-Hill,
    32. Wightman D (2010) Crowdsourcing human-based computation , Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries,
    33. Whitla P (2009) Crowdsourcing and Its Application in Marketing Activities , Contemporary Management Research, Vol.5 (1) ; pp.15-28
    34. Zimmermann HD (2010) From Ecommerce to Ecommerce 2.0: The Changing Role of Customer , Proceedings of the Liberec Informatics Forum, ; pp.171-179
    Do not open for a day Close