USING SOCIAL ROBOTS TO CREATE INCLUSIVE CLASSROOM EXPERIENCES FOR CHILDREN WITH MIXED VISUAL ABILITIES

Isabel Neto, University of Lisbon, isabel.neto@tecnico.ulisboa.pt

Abstract

Visually impaired children are increasingly educated in mainstream schools rather than special schools. However, even though being included with their sighted peers, previous research reveals concerns with classroom participation, lack of collaborative learning, reduced social engagement, and potential for isolation. My dissertation explores designing and building interactive social robots to promote inclusive classroom experiences and social engagement between children with mixed visual abilities. Social robots have the potential to foster engagement, participation, and collaboration due to their unique multi-sensory capabilities, actuators, and agency. Situated in a local mainstream school, I conducted extensive field work that involved engaging with the school community, including children, teachers, therapists, and parents through ethnographic observations, contextual inquiry, group interviews, and design activities. The identified challenges and opportunities will guide the design of social robots capable of supporting inclusive classroom experiences and sustaining long-term social engagement between mixed visual abilities' pupils. I will explore opportunities for off-the-shelf and custom-built robots to address existing challenges by following a user-centred methodology to design prototype, and evaluate solution with the school community.

Introduction

All children regardless of gender, age, disability or socio-economic status have the right to a quality and equitable education [12]. Prior research shows that pupils with disabilities, namely visually impaired children, are increasingly educated in mainstream schools [9, 11], demanding new types of support, learning experiences, and social activities.

Schools need to ensure that children fully participate in all activities, while feeling comfortable and secure [8, 18].However, previous studies suggests that visually impaired pupils have a risk of exclusion from classroom dynamics due to the individual learning approach to their needs [19]. Moreover, current assistive technologies (e.g., braille typewriters, screen readers and enlargers) are specifically designed to be used by children with visually impairments alone, which can lead to learning in isolation and reduce opportunities for classroom participation, peer interaction, and engagement. Current technologies prioritize accessibility over inclusion, focusing on children's disabilities rather than on wide range of abilities present in mainstream classrooms [17, 20].

My research explores a complimentary approach that aims to augment classrooms with technologies that account for the mixture of visual abilities within group and individual learning activities. Particularly, I propose using social robots due to their inherently engaging nature, physical attributes, multi-sensory capabilities, and intelligent behaviours. Social robots have the potential to play a role in building social engagement and improving classroom participation, learning, and inclusion. The CREANOVA project [10] showed that inclusion and innovation could be driven from creative and collaborative activities that value the differences, openness to fail, co-creation, and joint knowledge production.

Recently, Human-Robot Interaction (HRI) studies showed that social robots have the ability to influence group dynamics [7, 15], and foster inclusion and engagement [23]. In schools, educational robots can play different roles such as social agents in learning activities [1, 3, 5, 6, 13, 16] , facilitators in assisted therapy [22, 24], or tools for learning [2, 14, 21]. Nevertheless, previous research on children-robot interaction are largely limited to single user activities, focusing on individual needs rather than group dynamics. Moreover, these studies focus on short-term evaluations, typically during one test session, preventing an accurate evaluation of robot influence in children behaviours.

In the Human-Computer Interaction (HCI) and accessible computing research fields, researchers have started to explore how to design for groups of children with mixed-visual abilities [17, 20] and investigate the potential of robots to drive inclusive play experiences [18] and learning activities [21]. However, to the best of our knowledge, the potential for inclusion of using social robots in mixed visual abilities classrooms deployments remain unexplored.

Dissertation Research

This research will explore the use of cost-effective robots to tackle inclusive long-term experiences in classrooms, enriching the robot with multiple-sensory interactions to allow access, manipulation and engagement within a group with sighted and visually impaired children. Our motivation is to extend educational robots to promote inclusion. This novel approach will help children have access to group activities, be more inclusive in their peers' relations, and overcome their challenges and differences. Our proposed research program aims to push forward the current state of the art in inclusive classrooms. It will explore different ways to promote inclusion between children with mixed visual abilities using social robots.

This work is at the crossroads of multiple research fields: accessible computing, human-robot interaction, and human-computer interaction. Therefore, the research goals and technical duties focus on the following : 1) Identify the needs of each stakeholder (children, teachers, therapists and parents) through community engagement activities in a mainstream school environment; 2) Design social robots for mixed visual abilities classrooms using participatory design practices to explore appropriate robot characteristics, behaviours, interactions, and roles; 3) Create interactive modalities that are socially acceptable for the specific contexts, including verbal and non-verbal behaviours of the robot, allowing multiple types of interactions to adapt to each child needs; 4) Build robot experiences, using a user-centred design approach, based on cost-effective solutions (using off-the-shelf robots or DIY prototypes); 5) Long-term evaluation of the robot's influence on children inclusion at school;

To better understand inclusion challenges and opportunities, I conducted a six-month-long community engagement effort in a local school (from September 2019 to February 2020) using a multi-methods approach including ethnographic observations, contextual inquiry, group interviews, and design activities. By the end of February 2020, I recorded more than 40 hours of interviews and observation of classroom and therapy activities. In total, I engaged with 10 schoolteachers, therapists, and psychologists (two of them were visually impaired), and six parents. I observed multiple classes with children from different ages and school levels (N=91, Mean Age =10.6, SD=2.70). In this community, we had four children that experienced autism spectrum disorders and seven visually impaired children (blind N=2, low vision N=5).

From the qualitative data collected during the field work, I conducted a thematic analysis [4] to examine the stakeholders' needs, experienced challenges, and expectations. Interestingly, inclusion and social engagement were common themes that come up from children's', parents', and teachers' views, emphasizing the importance of social activities accessible to all pupils. Based on previous results, I developed five scenarios that explored the use of social robots in classroom activities: dance, friends, student helper, teacher assistant and storyteller.

In March 2020, I performed participatory design sessions with 54 children. These activities were embedded in school curricula and technology education classes, where the goal was to design robots for inclusion in school settings. Children were grouped based on their school level: one group from primary school (N=18, Mean Age =8.55, SD=1.29, Low vision N = 3); other group from the 5th grade (N=19, Mean Age = 10.78, SD=0.41, Low vision N= 1); and one group from the 8th grade (N=17, Mean Age = 13.23, SD=0.43, Blind N = 1). In each classroom, children organized in teams of three to four children, and shuffle scenarios between them. Due to COVID pandemic, the primary school group only had one session, while the other two had remotely three other PD sessions.

I am currently conducting a thematic analysis based on the transcripts and observations of the community engagement and participatory design sessions. Based on this fieldwork, I will continue to follow a user-centred approach by involving the school's stakeholders in all stages of the design process. In this iterative process, I intend to prototype and regularly test robot designs with children and other school stakeholders, exploring different robot roles and behaviors, multiple scenarios, and multi-sensory interfaces. Using the lessons learned from field studies and the refined prototypes, I aim to run an empirical study, during a six-month-long period to measure the impact of the robot on the social engagement and bonding of children with mixed visual abilities.

Preliminary Findings

The initial findings from observations and interviews highlighted that the fears, aims, and perspectives for inclusion are different between stakeholders. However, there is a common concern in creating healthy environments that value the differences and promote peer-engagement.

In the participatory design sessions, children worked in teams (each of them with one of the five scenarios) to build an inclusive robot that promoted group activities to all children. These sessions took place within classroom activities with the teachers' support. We followed a design thinking process approach with several phases : (1) Problem statement: they had to discuss how to use robots in group activities; (2) Problem specification: discuss challenges and opportunities in using a robot; (3) Research phase: we showcased several robots (illustrated in Figure 1), for children to explore and be inspired to the next phases; (4) Ideation: explore group ideas; (5) Team presentation to the class, which allowed classmates to critique initial designs and further refine ideas (6) Detailed robot design: each team had to prototype their robot using recycled materials and define several features; (7) Enacting: children role-played with their robot in a situated classroom context simulating how the robot and other users would behave within a group activity. These activities took place throughout 4 weeks and 4 sessions with 54 children (5 visually impaired).

These activities resulted in 30 robots' designs. For each of robots, children had to define its features: (1) Physical components, (2) Appearance, (3) Communication channels, (4) Exclusion factor being addressed, (5) Role, (6) Accessibility features, (7) Senses, (8) Context, (9) Locomotion features and (10) Personality. Figure 2 illustrates examples of the robots; one common characteristic was the use of sound as a main communication modality, added by visual information (writing, colours, and lights). The use of accessibility features was frequent in all scenarios, and different exclusion reasons appear shyness, knowledge, loneliness and disability. The roles and traits of each robot were aligned with the scenario, (1) helping and calm (2) dance in groups, outgoing (3) repeat information, print in braille and engaged (4) Cheer-up, collect objects and playful and (5) read and write the story in groups and intelligent.

Contributions

This research explores the use of social robots as agents in fostering inclusion and social engagement between children with mixed visual abilities. The long-term evaluation of different scenarios, robot roles, personality and interactive modalities will enrich classroom dynamics and advance knowledge in designing robots for inclusive experiences. On a broader level, I expect my dissertation research to contribute to finding novel approaches to integrating visually impaired children in society, exploring group dynamics within HRI, improving accessibility, and valuing individual abilities to create inclusive experiences. These contributions can inform research in HRI with adults or children with other inclusion challenges (e.g., gender, race, religion, bullying). I am in the second year of my PhD and looking forward to receive feedback from other accessible computing researchers about the fieldwork conducted in the past months to refine initial robot designs and group activities for children with mixed visual abilities as well as suggestions for the evaluation process.

Acknowledgments

I want to thank the staff and children of Centro Helen Keller for their participation, support, and understanding through these studies. National funds have supported this work through FCT, Fundação para a Ciência e a Tecnologia, under project UIDB/50021/2020.

References

1. Patrícia Alves-Oliveira, Patrícia Arriaga, Ana Paiva, and Guy Hoffman. 2017. Yolo, a robot for creativity: A co-design study with children. In Proceedings of the 2017 Conference on Interaction Design and Children. 423–429.
2. Renata Pitta Barros, Aquiles Medeiros Filgueira Burlamaqui, Samuel Oliveira de Azevedo, Sarah Thomaz de Lima Sa, Luiz Marcos Garcia Goncalves, Akynara Aglae R S da Silva, et al. 2017. Cardbot-assistive technology for visually impaired in educational robotics: Experiments and results. IEEE Latin America Transactions 15, 3 (2017), 517–527.
3. Tony Belpaeme, James Kennedy, Aditi Ramachandran, Brian Scassellati, and Fumihide Tanaka. 2018. Social robots for education: A review. Science robotics 3, 21 (2018).
4. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
5. Shruti Chandra, Pierre Dillenbourg, and Ana Paiva. 2019. Children teach handwriting to a social robot with different learning competencies. International Journal of Social Robotics (2019), 1–28.
6. Shruti Chandra, Raul Paradeda, Hang Yin, Pierre Dillenbourg, Rui Prada, and Ana Paiva. 2018. Do Children Perceive Whether a Robotic Peer is Learning or Not?. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 41–49.
7. Filipa Correia, Samuel Mascarenhas, Rui Prada, Francisco S Melo, and Ana Paiva. 2018. Group-based emotions in teams of humans and robots. In Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction. 261–269.
8. Clare Cullen and Oussama Metatla. 2019. Co-designing Inclusive Multisensory Story Mapping with Children with Mixed Visual Abilities. In Proceedings of the 18th ACM International Conference on Interaction Design and Children. 361–373.
9. Gareth Davies. 2019. Support for pupils with special educational needs and disabilities in England: report by the Comptroller and Auditor General. (2019).
10. John M Davis. 2013. Supporting creativity, inclusion and collaborative multi-professional learning. Improving Schools 16, 1 (2013), 5–20.
11. DGEEC Direção Geral de Estatística da Educação e Ciência. 2019. Students with special needs in Portuguese Schools. https://www.dgeec.mec.pt/np4/224.
12. Lani Florian. 2008. Inclusion: special or inclusive education: future trends. British Journal of Special Education 35, 4 (2008), 202–208.
13. Wafa Johal. 2020. Research Trends in Social Robots for Learning. Current Robotics Reports (2020), 1–9.
14. Wafa Johal, Sonia Andersen, Morgane Chevalier, Ayberk Ozgur, Francesco Mondada, and Pierre Dillenbourg. 2019. Learning symmetry with tangible robots. In International Conference on Robotics and Education RiE 2017. Springer, 270–283.
15. Malte F Jung, Selma Šabanović, Friederike Eyssel, and Marlena Fraune. 2017. Robots in groups and teams. In Companion of the 2017 ACM conference on computer supported cooperative work and social computing. 401–407.
16. Takayuki Kanda, Takayuki Hirano, Daniel Eaton, and Hiroshi Ishiguro. 2004. Interactive robots as social partners and peer tutors for children: A field trial. Human–Computer Interaction 19, 1-2 (2004), 61–84.
17. Oussama Metatla. 2017. Uncovering challenges and opportunities of including children with visual impairments in mainstream schools. In Proceedings of the 31st British Computer Society Human Computer Interaction Conference. BCS Learning & Development Ltd., 102.
18. Oussama Metatla, Sandra Bardot, Clare Cullen, Marcos Serrano, and Christophe Jouffrais. 2020. Robots for Inclusive Play: Co-designing an Educational Game With Visually Impaired and sighted Children. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
19. Oussama Metatla and Clare Cullen. 2018. “Bursting the Assistance Bubble” Designing Inclusive Technology with Children with Mixed Visual Abilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–14.
20. Oussama Metatla, Marcos Serrano, Christophe Jouffrais, Anja Thieme, Shaun Kane, Stacy Branham, Émeline Brulé, and Cynthia L Bennett. 2018. Inclusive education technologies: Emerging opportunities for people with visual impairments. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1–8.
21. Isabel Neto, Wafa Johal, Marta Couto, Hugo Nicolau, Ana Paiva, and Arzu Guneysu. 2020. Using Tabletop Robots to Promote Inclusive Classroom Experiences. In Proceedings of the Interaction Design and Children Conference (London, United Kingdom) (IDC ’20). Association for Computing Machinery, New York, NY, USA, 281–292. https://doi.org/10.1145/3392063.3394439
22. Brian Scassellati, Laura Boccanfuso, Chien-Ming Huang, Marilena Mademtzi, Meiying Qin, Nicole Salomons, Pamela Ventola, and Frederick Shic. 2018. Improving social skills in children with ASD using a long-term, in-home social robot. Science Robotics 3, 21 (2018).
23. Sarah Strohkorb Sebo, Ling Liang Dong, Nicholas Chang, and Brian Scassellati. 2020. Strategies for the Inclusion of Human Members within Human-Robot Teams. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction. 309–317.
24. Serge Thill, Cristina A Pop, Tony Belpaeme, Tom Ziemke, and Bram Vanderborght. 2012. Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook. Paladyn 3, 4 (2012), 209–217.

About the Authors