A MODEL FOR THE INDIVIDUAL EMPOWERMENT USING INTRINSIC PERSONALIZATION OF CROWDSOURCING TASKS

Dennis Paulino, INESC TEC and University of Trás-os-Montes e Alto Douro, Portugal dpaulino@utad.pt

Abstract

Crowdsourcing is a paradigm of outsourcing work that is done using human capabilities through the Internet. Given the various possibilities of overcoming cultural and social barriers, crowdsourcing provides an opportunity for people with disabilities to have a financial compensation and help them feel realised. In crowdsourcing, people with disabilities face problems related with the lack of task description or usability. This article it is presented the main threads for my PhD thesis which main goal is to prove, that it is possible to map crowdsourcing tasks effectively to each individual, focusing particularly on the cognitive abilities.

Introduction

Crowdsourcing is defined as an online activity where a person or organization proposes the performance of a task to a group of volunteer individuals, always having mutual benefit and where the volunteers must receive a reward [8]. There are currently crowdsourcing platforms that reward volunteers (or crowd workers) monetarily1,2,3. Tasks may consist of translating texts, classifying images, or filling out questionnaires. Crowdsourcing is considered an emerging and viable technology, being able to delegate tasks simultaneously to many crowd workers [15].

Crowdsourcing is used by a wide spectrum of people, including those who have physical or mental limitations, however there are platforms that do not consider accessibility for this group of people. Problems related to design on a crowdsourcing platform can cause its users to never use it again [3]. A study identified problems that people with disabilities (including autistic people) face when using crowdsourcing platforms such as Amazon Mechanical Turk conducted [21]. The problems identified on the platforms are related to the usability and the lack of description of the tasks related to the capabilities of the crowd workers. It was proposed some solutions to improve the accessibility of these platforms:

People with disabilities who participate in crowdsourcing include people with autism. Autism is defined by the American Psychiatric Association as "a disturbance in a person’s development, characterized by difficulties in social interaction, communication and restrictive and repetitive behaviours" [2]. It is estimated that there are around 25 million people with autism worldwide [20].

Autistic individuals have several limitations in their social interactions, including the actions they perform in their daily lives [10, 17]. Social limitations result in increased isolation, which in turn is related to deteriorating quality of life [14]. Working helps to combat the effects of isolation [6]. A study identified the social difficulties on the perspective of individuals with autism, reveals that most participants liked to be able to contribute to improving the environment around them [18]. Given the various possibilities of overcoming cultural and social barriers, crowdsourcing provides an opportunity for people with autism to have financial compensation and help them feel fulfilled.

The main goal of this thesis is to empower each crowdsourcing crowd worker, by matching the capabilities of each individual to the most suitable crowdsourcing tasks. It will include a case study with people with autism, to prove the goal of the thesis that even people that face social barriers, can perform crowdsourcing tasks with the suitable matching of systems assumptions and users’ capabilities. This article will present the roadmap of the proposed thesis.

1 https://www.mturk.com/
2 https://rapidworkers.com/
3 https://www.clickworker.com/

Objectives and Research Questions

The first objective of this thesis is to explore the empowerment of crowdsourcing tasks in the capabilities of crowd workers. The purpose is to map the crowdsourcing tasks to crowd workers whose skills are adequate to perform this task, whether the crowd worker may have limitations or not. The mapping must follow the General Data Protection Regulation (GDPR) to ensure the privacy of data from crowd workers [5]. The second objective is to analyse whether it is possible for people with disabilities to participate and remain motivated in the proposed tasks of crowdsourcing, including improving their skills. The two objectives are related to each other since the mapping of tasks allows improving the skills of crowd workers. The skills can be considered as the necessary abilities for accomplish a crowdsourcing task with success, which subsequently can improve when the crowd worker is motivated [11]. This allows anyone to participate in a crowdsourcing platform, regardless of their limitations and even leveraging their capabilities.

In order to assess these objectives, a case study will be carried out in autistic people, who despite having limitations in social interactions, they have the ability to consistently perform repetitive tasks [2]. If the objectives are achieved, it is possible that people with limitations (including autistic people), can use crowdsourcing in a motivated way, through the mapping of tasks that are appropriate to their abilities. The real-world impact of the accomplishment of these measures will also benefit the employers in the crowdsourcing market. With the emergence of Artificial Intelligence systems, it was created a need for labelling datasets which subsequently was found in crowdsourcing a way for labelling efficiently [4]. A study conducted by Hara and Bigham [12] indicates that is feasible for people with autism to perform image transcription tasks by using an adaptive tool.

This thesis has the following research questions:

To answer the research questions, the tasks to be performed will be defined in the detailed description.

Workplan

Crowdsourcing allows many crowd workers to be delegated tasks simultaneously (Kucherbaev et al., 2016). These crowd workers are composed of people with limitations, who while using crowdsourcing platforms, face problems related to the lack of job description or the usability of these platforms (Zyskowski et al., 2015). The thesis presented in this article wants to prove that it is possible to map crowdsourcing tasks effectively to people’s capacities. To show that it is possible to map tasks with people with disabilities, a case study involving the participation of people with autism will be conducted. In order to be able to more precisely delineate the execution of each aforementioned objective, a set of tasks are proposed below to be carried out:

  1. Study of personalization cognitive and physical abilities on the crowdsourcing tasks.
    1. 1.1 Make a exploratory review on this theme.
  2. State of the art on current personalization techniques that are applied on crowdsourcing and collaboration tools.
    1. 2.1 Construction of a model for conducting a literature review with the theme related to existing personalization techniques of crowdsourcing or collaboration platforms.
    2. 2.2 Conducting a systematic review of the literature guided by the model mentioned in subtask 2.1. After the review, problems and opportunities about crowdsourcing technologies applied to people with autism should be presented. The personalization will help technology to adapt for each individuals’ capabilities.
  3. Specification of a conceptual model that allows for the intrinsic customization of crowdsourcing tasks.
    1. 3.1 The conceptual model should present an architecture that does the intrinsic personalization of crowdsourcing tasks, considering people’s abilities (the case study for people with autism will be carried out as described in task 1 as well as the current crowdsourcing technologies (completion of task 2).
    2. 3.2 Specify the model to be able to assess individuals’ abilities to perform tasks. In crowdsourcing qualitative tests can be used periodically to assess which tasks are appropriate for each crowd worker [9].
    3. 3.3 Specification of tasks that can motivate people with autism to use the crowdsourcing platform.
  4. Implementation of the conceptual model for the development of a computer system.
    1. 4.1 The implementation must follow the model proposed in task 3, and must use the online services available from the crowdsourcing platforms (e.g. Amazon Mechanical Turk) to be able to obtain the available tasks and allow their execution. The proposed implementation will be a computer system, representing a layer that allows obtaining the tasks available from the crowdsourcing platforms and mapping them to users with the appropriate capabilities.
    2. 4.2 The proposed computer system must follow the user-centered design, thus allowing the user not to spend much effort to learn how to use it [1].
    3. 4.3 During the construction of the computer system, it must be validated for people with autism, using usability tests in order to find out if the platform is usable.
    4. 4.4 After building the computer system, tests should be carried out to find out whether people with autism can perform tasks more efficiently than people without autism.
  5. Writing of the thesis on the work developed in the tasks mentioned above.

Next, the methodology for conducting the proposed work of this application is defined. The subject of this application is in the field of computer systems. For this domain, an approved research methodology is design science [16]. The design science is the use of scientific knowledge for the creation of artifacts [7]. The artifacts created using the design science methodology are intended to solve a problem with greater efficiency or to propose a solution to a problem that has not yet been solved [13].

The works proposed in this application must follow the design science methodology. Next, the work to be carried out relating to the six activities proposed by [19]:

This research methodology results in the production of several artifacts, in the form of prototypes of computer systems or scientific publications.

Research Stage

I will be a second year PhD student in the University of Trás-os-Montes e Alto Douro, Portugal. The current state of the work done is the elaboration of an exploratory review on the cognitive and physical abilities personalization of web technologies. Subsequently, it was found that the thematic of the personalization of physical abilities was already well explored, while in contrast it was necessary to research in more depth the personalization of cognitive abilities. This lead to produce a systematic literature protocol, tailored to research about the cognitive personalization techniques applied on crowdsourcing or similar platforms. The next step on the current work is to conduct the systematic literature review, which subsequently will produce important considerations to guide further work.

Expected Contributions to the Accessibility Field

The expected contributions from this research to the accessibility field will be: (i) increase knowledge about the cognitive personalization in work tasks; (ii) demonstrate that cognitive personalization can improve the performance on crowdsourcing tasks; (iii) people with disabilities can be realized using crowdsourcing if it is performed the proper matching of crowdsourcing tasks assumptions and the users’ abilities.

Acknowledgments

This work is financed by the FCT – Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) with research grant SFRH/BD/148991/2019.

References

  1. Chadia Abras, Diane Maloney-Krichmar, and Jenny Preece. 2004. User-centered design. Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications 37, 4 (2004), 445–456.
  2. American Psychiatric Association et al. 2013. Autism spectrum disorder. Diagnostic and statistical manual of mental disorders (2013), 50–59.
  3. Avinoam Baruch, Andrew May, and Dapeng Yu. 2016. The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform. Computers in Human Behavior 64 (2016), 923–931. https://doi.org/10.1016/j.chb.2016.07.039
  4. Joseph Chee Chang, Saleema Amershi, and Ece Kamar. 2017. Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets. (2017), 2334–2346. https://doi.org/10.1145/3025453.3026044
  5. Abhik Chaudhuri. 2016. Internet of things data protection and privacy in the era of the General Data Protection Regulation. Journal of Data Protection Privacy 1, 1 (2016), 64–75.
  6. Caitlin E. Coyle and Elizabeth Dugan. 2012. Social Isolation, Loneliness and Health Among Older Adults. Journal of Aging and Health 24, 8 (2012), 1346–1363. https://doi.org/10.1177/0898264312460275
  7. Nigel Cross. 2001. Designerly ways of knowing: Design discipline versus design science. Design issues 17, 3 (2001), 49–55.
  8. Enrique Estellés-Arolas and Fernando González-Ladrón-de Guevara. 2012. Towards an integrated crowdsourcing definition. Journal of Information Science 38, 2 (2012), 189–200. https://doi.org/10.1177/0165551512437638
  9. Ju Fan, Guoliang Li, Beng Chin Ooi, Kian-lee Tan, and Jianhua Feng. 2015. icrowd: An adaptive crowdsourcing framework. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. 1015–1030.
  10. Centers for Disease Control Prevention. 2016. Prevalence and characteristics of autism spectrum disorder among children aged 8 years–Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2012. Morbidity and mortality weekly report. Surveillance summaries (Washington, DC: 2002) 65, 3 (2016), 1–23.
  11. Ujwal Gadiraju and Stefan Dietze. 2017. Improving Learning through Achievement Priming in Crowdsourced Information Finding Microtasks. (2017), 105–114. https://doi.org/10.1145/3027385.3027402
  12. Kotaro Hara and Jeffrey P. Bigham. 2017. Introducing People with ASD to Crowd Work. (2017), 42–51. https://doi.org/10.1145/3132525.3132544
  13. Alan R Hevner. 2007. A three cycle view of design science research. Scandinavian journal of information systems 19, 2 (2007), 4.
  14. Julianne Holt-Lunstad, Timothy B. Smith, Mark Baker, Tyler Harris, and David Stephenson. 2015. Loneliness and Social Isolation as Risk Factors for Mortality:A Meta-Analytic Review. Perspectives on Psychological Science 10, 2 (2015), 227–237. https://doi.org/10.1177/1745691614568352
  15. P. Kucherbaev, F. Daniel, S. Tranquillini, and M. Marchese. 2016. Crowdsourcing Processes: A Survey of Approaches and Opportunities. IEEE Internet Computing 20, 2 (2016), 50–56. https://doi.org/10.1109/MIC.2015.96
  16. Allen S. Lee, Manoj Thomas, and Richard L. Baskerville. 2015. Going back to basics in design science: from the information technology artifact to the information systems artifact. Information Systems Journal 25, 1 (2015), 5–21. https://doi.org/10.1111/isj.12054
  17. Brenda Smith Myles and Richard L Simpson. 2001. Understanding the hidden curriculum: An essential social skill for children and youth with Asperger syndrome. Intervention in school and clinic 36, 5 (2001), 279–286.
  18. Eve Müller, Adriana Schuler, and Gregory B. Yates. 2008. Social challenges and supports from the perspective of individuals with Asperger syndrome and other autism spectrum disabilities. Autism 12, 2 (2008), 173–190. https://doi.org/10.1177/1362361307086664
  19. Ken Peffers, Tuure Tuunanen, Marcus A. Rothenberger, and Samir Chatterjee. 2007. A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems 24, 3 (2007), 45–77. https://doi.org/10.2753/MIS0742-1222240302
  20. Theo Vos, Christine Allen, Megha Arora, Ryan M Barber, Zulfiqar A Bhutta, Alexandria Brown, Austin Carter, Daniel C Casey, Fiona J Charlson, Alan Z Chen, et al. 2016. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. The lancet 388, 10053 (2016), 1545–1602.
  21. Kathryn Zyskowski, Meredith Ringel Morris, Jeffrey P Bigham, Mary L Gray, and Shaun K Kane. 2015. Accessible crowdwork? Understanding the value in and challenge of microtask employment for people with disabilities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 1682–1693.

About the Authors

Dennis has received the Master´s Degree in Informatic Engineer, at UTAD in December 2018. He participates since September 2016 until December 2018 in the project NanoSTIMA RL2 - Passus Mobile, responsible for developing a system that makes exercise supervision of people with peripheral arterial disease. From January 2019 until November 2019, he participated in the project Ecsaap, responsible for the construction of an informatic system to help in the visualization and detection of meteorological phenomena. In Dezember 2019 he ingress in PhD in Informatics at UTAD, with a scholarship financed by FCT.