Microtasks |
Microtasking (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Microtasks (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
) Volunteer microtask, casual game microtask, microtasks market, microtask forced labor, educational microtask (Good & Su, 2013Good, B. M., & Su, A. I. (2013). Crowdsourcing for bioinformatics. Bioinformatics, 29(16), 1925-1933. https://doi.org/10.1093/bioinformatics/btt333 https://doi.org/10.1093/bioinformatics/b...
) Crowdsourcing of simple tasks (Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
) Virtual labor marketplaces (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
) Web-based crowdsourcing (Majchrzak & Malhotra, 2013Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. The Journal of Strategic Information Systems, 22(4), 257-268. https://doi.org/10.1016/j.jsis.2013.07.004 https://doi.org/10.1016/j.jsis.2013.07.0...
) Crowd processing systems (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
) Integrative sourcing with fixed remuneration (Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
) Support (Zogaj, Bretschneider, & Leimeister, 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
) Collaborative science project model (Saxton, Oh, & Kishore, 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) Microtasker (Poblet et al., 2018Poblet, M., García-Cuesta, E., & Casanovas, P. (2018). Crowdsourcing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers, 20(6), 1363-1379. https://doi.org/10.1007/s10796-017-9734-6 https://doi.org/10.1007/s10796-017-9734-...
) Micro-task (Bhatti, Gao, & Chen, 2020Bhatti, S. S., Gao, X., & Chen, G. (2020). General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey. Journal of Systems and Software, 167(3), 110611. https://doi.org/10.1016/j.jss.2020.110611 https://doi.org/10.1016/j.jss.2020.11061...
) |
- Paid or unpaid work (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
; Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Low remuneration, when available (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
; Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
; Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
); - Low involvement (Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
); - Large volume of collaborating individuals (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
; Good & Su, 2013Good, B. M., & Su, A. I. (2013). Crowdsourcing for bioinformatics. Bioinformatics, 29(16), 1925-1933. https://doi.org/10.1093/bioinformatics/btt333 https://doi.org/10.1093/bioinformatics/b...
; Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
; Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Low complexity tasks (Bhatti et al., 2020Bhatti, S. S., Gao, X., & Chen, G. (2020). General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey. Journal of Systems and Software, 167(3), 110611. https://doi.org/10.1016/j.jss.2020.110611 https://doi.org/10.1016/j.jss.2020.11061...
; Majchrzak & Malhotra, 2013Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. The Journal of Strategic Information Systems, 22(4), 257-268. https://doi.org/10.1016/j.jsis.2013.07.004 https://doi.org/10.1016/j.jsis.2013.07.0...
; Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
; Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
; Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Fast, easy, specific, and independent tasks (Bhatti et al., 2020; Poblet, García-Cuesta, & Casanovas, 2018Poblet, M., García-Cuesta, E., & Casanovas, P. (2018). Crowdsourcing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers, 20(6), 1363-1379. https://doi.org/10.1007/s10796-017-9734-6 https://doi.org/10.1007/s10796-017-9734-...
). |
Competition |
Tournament crowdsourcing (TC) or ideas competition (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
) Mega innovation competition (Good & Su, 2013Good, B. M., & Su, A. I. (2013). Crowdsourcing for bioinformatics. Bioinformatics, 29(16), 1925-1933. https://doi.org/10.1093/bioinformatics/btt333 https://doi.org/10.1093/bioinformatics/b...
) Contests (Nakatsu, Grossman, & Iacovou, 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
) Co-creation boundary management (Majchrzak & Malhotra, 2013Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. The Journal of Strategic Information Systems, 22(4), 257-268. https://doi.org/10.1016/j.jsis.2013.07.004 https://doi.org/10.1016/j.jsis.2013.07.0...
) Selective sourcing crowd without assessment, selective sourcing with crowd assessment (Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
) Innovation development (Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
) |
- Work rewarded with awards, monetary or not (Good & Su, 2013Good, B. M., & Su, A. I. (2013). Crowdsourcing for bioinformatics. Bioinformatics, 29(16), 1925-1933. https://doi.org/10.1093/bioinformatics/btt333 https://doi.org/10.1093/bioinformatics/b...
; Majchrzak & Malhotra, 2013Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. The Journal of Strategic Information Systems, 22(4), 257-268. https://doi.org/10.1016/j.jsis.2013.07.004 https://doi.org/10.1016/j.jsis.2013.07.0...
; Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
; Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
; Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
); - Volume of individuals varied, depending on the complexity of the activity, which can be groups or individuals (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
; Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
); - Need for minimally specialized knowledge in the field of activity (Nakatsu, Grossman, & Iacovou, 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
; Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
; Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
); - Possibility of peer assessment throughout the competition (Schuurman et al., 2012). |
Evaluations |
Crowdevaluation (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Consumer report model (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) Customer-driven innovation (Nakatsu, Grossman, & Iacovou, 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
) |
- Work rewarded monetarily or not (Nakatsu et al., 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
; Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
; Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Feedback from individuals regarding a particular product or service offered by an organization (Nakatsu et al., 2014; Sivula & Kantola, 2016). |
Complex tasks |
Macrotasking (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Crowdsourcing for innovation challenges (Majchrzak & Malhotra, 2013Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. The Journal of Strategic Information Systems, 22(4), 257-268. https://doi.org/10.1016/j.jsis.2013.07.004 https://doi.org/10.1016/j.jsis.2013.07.0...
) Crowd solving systems (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
) Crowdsourcing of complex tasks (Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
) Mega hard game (Good & Su, 2013Good, B. M., & Su, A. I. (2013). Crowdsourcing for bioinformatics. Bioinformatics, 29(16), 1925-1933. https://doi.org/10.1093/bioinformatics/btt333 https://doi.org/10.1093/bioinformatics/b...
) Complex-task (Bhatti et al., 2020Bhatti, S. S., Gao, X., & Chen, G. (2020). General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey. Journal of Systems and Software, 167(3), 110611. https://doi.org/10.1016/j.jss.2020.110611 https://doi.org/10.1016/j.jss.2020.11061...
) Macro-task (Bhatti et al., 2020) |
- Jobs usually rewarded monetarily (Bhatti et al., 2020Bhatti, S. S., Gao, X., & Chen, G. (2020). General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey. Journal of Systems and Software, 167(3), 110611. https://doi.org/10.1016/j.jss.2020.110611 https://doi.org/10.1016/j.jss.2020.11061...
; Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
); - Solution of difficult problems and complex tasks (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
; Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
); - Selection of a working group with specific characteristics and knowledge (Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
; Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Greater involvement of individuals; - Collaboration through calls on platforms (Majchrzak & Malhotra, 2013Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. The Journal of Strategic Information Systems, 22(4), 257-268. https://doi.org/10.1016/j.jsis.2013.07.004 https://doi.org/10.1016/j.jsis.2013.07.0...
); - Possible decomposition of complex tasks to smaller sub-tasks (Bhatti et al., 2020). |
Classification |
Terms used in the literature and reference authors |
Characteristics |
Financing |
Crowdfunding for a project, crowdfunding in an organization, crowdfunding as a loan (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Crowdfunding platforms (Nakatsu et al., 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
) Funding (Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
) Peer-to-peer social financing model (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) |
- Participation can generate a financial return for those who participate (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Crowdfunding by the community or by individuals involved in a particular project (Nakatsu et al., 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
; Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
; Sivula & Kantola, 2015; Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
); - Financing can take place either without consideration, or in the form of a loan (Sivula & Kantola, 2016). |
Design |
Crowdcreation (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Integrative sourcing with success-based remuneration (Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
) Product design model, citizen media production model, digital goods sales model (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) Design (Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
) Idea generation (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
) Crowdsourcing of creative tasks (Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
) Creative task (Bhatti et al., 2020Bhatti, S. S., Gao, X., & Chen, G. (2020). General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey. Journal of Systems and Software, 167(3), 110611. https://doi.org/10.1016/j.jss.2020.110611 https://doi.org/10.1016/j.jss.2020.11061...
) |
- Jobs usually financially rewarded (Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
; Schenk & Guittard, 2011Schenk, E., & Guittard, C. (2011). Towards a characterization of crowdsourcing practices. Journal of Innovation Economics & Management, 7(1), 93-107. https://doi.org/10.3917/jie.007.0093 https://doi.org/10.3917/jie.007.0093...
); - Individuals develop products and can offer them on platforms (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
; Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
; Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
); - Need of specific knowledge for the development of items (Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
); - May rely on the validation by experts (Bhatti et al., 2020Bhatti, S. S., Gao, X., & Chen, G. (2020). General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey. Journal of Systems and Software, 167(3), 110611. https://doi.org/10.1016/j.jss.2020.110611 https://doi.org/10.1016/j.jss.2020.11061...
). |
Content development |
Integrative sourcing without remuneration (Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
) Crowd creation systems (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
) Knowledge base building model (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) Wikies, citizen journalism (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
) Social computing (Poblet et al., 2018Poblet, M., García-Cuesta, E., & Casanovas, P. (2018). Crowdsourcing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers, 20(6), 1363-1379. https://doi.org/10.1007/s10796-017-9734-6 https://doi.org/10.1007/s10796-017-9734-...
) |
- Jobs usually not monetarily rewarded (Schuurman et al., 2012Schuurman, D., Baccarne, B., Marez, L., & Mechant, P. (2012). Smart ideas for smart cities: Investigating crowdsourcing for generating and selecting ideas for ICT innovation in a city context. Journal of Theoretical and Applied Electronic Commerce Research, 7(3), 49-62. https://doi.org/10.4067/S0718-18762012000300006 https://doi.org/10.4067/S0718-1876201200...
); - Result from collective work (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
; Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
; Poblet et al., 2018Poblet, M., García-Cuesta, E., & Casanovas, P. (2018). Crowdsourcing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers, 20(6), 1363-1379. https://doi.org/10.1007/s10796-017-9734-6 https://doi.org/10.1007/s10796-017-9734-...
); - Any individual may contribute (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Individuals can add, delete, and edit someone else’s content (Hossain & Kauranen, 2015). |
Software development |
Collaborative software development model (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) Open-source software (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
) |
- Jobs may be rewarded monetarily or not (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Collaborative software development and improvement (Saxton et al.,); - Need of specific knowledge (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Individuals ‘motivated’ by their involvement in the field of computational development (Hossain & Kauranen, 2015). |
Voting |
Crowdvoting (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Crowd rating systems (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
) |
- Work usually not financially rewarded, used internally to improve services provided by an organization (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Wisdom of the crowd used to vote on topics on a platform (Faber & Matthes, 2016Faber, A., & Matthes, F. (2016). Crowdsourcing and crowd innovation. In A. Faber, F. Matthes, & F. Michel (Eds.), Digital mobility platforms and ecosystems: State of the art report (Report, pp. 36-48). Munich: Technische Universität München. https://doi.org/10.14459/2016md1324021 https://doi.org/10.14459/2016md1324021...
). |
Knowledge diffusion |
Crowdwisdom (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
) Reporter (Poblet et al., 2018Poblet, M., García-Cuesta, E., & Casanovas, P. (2018). Crowdsourcing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers, 20(6), 1363-1379. https://doi.org/10.1007/s10796-017-9734-6 https://doi.org/10.1007/s10796-017-9734-...
) |
- Monetary or unpaid work (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - Organization employs the crowd to increase its knowledge, through innovative ideas or the dissemination of knowledge (Sivula & Kantola, 2016Sivula, A, & Kantola, J. (2016). Integrating crowdsourcing with holistic innovation management. International Journal of Advanced Logistics, 5(3-4), 153-164. https://doi.org/10.1080/2287108X.2016.1221590 https://doi.org/10.1080/2287108X.2016.12...
); - People can offer firsthand information (Poblet et al., 2018Poblet, M., García-Cuesta, E., & Casanovas, P. (2018). Crowdsourcing roles, methods and tools for data-intensive disaster management. Information Systems Frontiers, 20(6), 1363-1379. https://doi.org/10.1007/s10796-017-9734-6 https://doi.org/10.1007/s10796-017-9734-...
). |
Open collaboration |
Open collaboration (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
) |
- Work that is usually not paid with money (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
); - Use of social networks and intermediary platforms or wikies in the search for solvers (Prpic et al., 2015Prpic, J., Taeihagh, A., & Melton, J. (2015). The fundamentals of policy crowdsourcing. Policy & Internet, 7(3), 340-361. https://doi.org/10.1002/poi3.102 https://doi.org/10.1002/poi3.102...
); - Problems or opportunities posted in information technology media (Prpic et al., 2015). |
Sales |
Marketing and sales (Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
) |
- Work usually paid for in the form of fees or participation (Zogaj et al., 2014Zogaj, S., Bretschneider, U., & Leimeister, J. M. (2014). Managing crowdsourced software testing: A case study based insight on the challenges of a crowdsourcing intermediary. Journal of Business Economics, 84(3), 375-405. https://doi.org/10.1007/s11573-014-0721-9 https://doi.org/10.1007/s11573-014-0721-...
); - Platforms that help organizations define markets and use the crowd as a sales amplifier (Zogaj et al., 2014). |
Classification |
Terms used in the literature and reference authors |
Characteristics |
Collaboration intermediaries |
Model with intermediary (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
) |
- Work usually paid with money (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
); - Platform aims to serve as middleman between organizations and individuals willing to collaborate with each other (Saxton et al., 2013Saxton, G. D., Oh, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2-20. https://doi.org/10.1080/10580530.2013.739883 https://doi.org/10.1080/10580530.2013.73...
); - Medium or low levels of voluntary collaboration, but with a guaranteed high return (Saxton et al., 2013). |
Public projects |
Public participation (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
) |
- Usually unpaid or low-cost work (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Public participation in issues related to public planning and public projects (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Citizen involvement in the development of public policies, ensuring government commitment (Hossain & Kauranen, 2015). |
Citizen science |
Citizen science (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
) |
- Usually unpaid, voluntary work (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Involvement in basic or highly sophisticated and complex activities (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Usually, the participation happens due to personal interest or curiosity, or even because of the individual’s sense of responsibility (Hossain & Kauranen, 2015Hossain, M., & Kauranen, I. (2015). Crowdsourcing: A comprehensive literature review. Strategic Outsourcing: An International Journal, 8(1), 2-22. https://doi.org/10.1108/SO-12-2014-0029 https://doi.org/10.1108/SO-12-2014-0029...
); - Use of platforms such as social networks to engage participants (Hossain & Kauranen, 2015). |
Sharing |
Crowdsharing applications (Nakatsu et al., 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
) |
- Provision of the service may be paid with money or not (Nakatsu et al., 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
); - Sharing spaces, services, and knowledge (Nakatsu et al., 2014Nakatsu, R. T., Grossman E. B., & Iacovou C. L. (2014). A taxonomy of crowdsourcing based on task complexity. Journal of Information Science, 40(6), 823-834. https://doi.org/10.1177/0165551514550140 https://doi.org/10.1177/0165551514550140...
); - Classification covers the sharing of both goods and knowledge (Nakatsu et al., 2014). |