LEARNING MUSIC BLIND: UNDERSTANDING THE APPLICATION OF TECHNOLOGY TO SUPPORT BLV MUSIC LEARNING

Leon Lu, Carleton University, Ottawa, Canada, leonlu@cmail.carleton.ca

Abstract

Learning to play a musical instrument and engaging in musical activities have enabled blind and/or low vision people to develop self-identity, find community and pursue music as a career. However, blind and/or low vision music learners face complex obstacles to learn music. They are highly reliant on their learning environment and music teachers for accommodations and flexibility. Prior research has identified the challenges faced by blind and/or low vision musicians and recognized the importance of touch for music reading and physical guidance. However, limited research has addressed these challenges through the development of assistive technology. The development of music computer technologies with haptics and the affordances of wearable technologies provides encouraging opportunities to develop haptic wearable devices to support blind and/or low vision music learning. I identify three unexplored research questions: (1) what design considerations must be addressed in future assistive technologies for BLV music learning, (2) how can wearable technologies with vibrotactile feedback support BLV student-teacher interactions, and (3) what are the long-term benefits and limitations of the use of assistive technologies for BLV music learning? I outline my research to date and highlight my findings. 

Introduction

The World Health Organization estimated that there are currently 284 million people in the world who are low vision and 39 million people who are blind [34]. Blind and Low Vision (BLV) people actively use other senses (especially hearing) to compensate for vision and find music as a source of personal enjoyment [25, 36]. Furthermore, studies indicated that BLV people exhibit higher levels of musical aptitude than sighted individuals [14, 15]. BLV musicians such as pianist Alec Templeton, Nicholas Constantinidis and Michael Arnowitt; opera singer Andrea Bocelli; violinist Takayoshi Wanami and popular singers like Jose Feliciano, Ray Charles and Stevie Wonder have been celebrated for their contribution to music [25]. However, learning to play music as a BLV person remains a complex and challenging endeavour [1, 3, 11, 12]. BLV music learners have limited access to learning resources and new music [11]. They are reliant on their music teachers and peers for accommodations and support [22]. However, music teachers are, at times, underprepared and ill-equipped to meet the individual needs of BLV music learners [8, 9, 16]. Learning resources such as braille music scores are expensive and time consuming to acquire [26]. Furthermore, commercially available assistive technologies (ATs) for BLV music learning are limited to programs for music reading and composition only [13, 19, 32]. Single line braille readers are not suited for braille music due to the limited number of dots and slow navigation [26].

The sense of touch is vitally important for BLV music learners to develop an understanding of musical instruments [3], interpret instructions through physical guidance [23] and read musical information [27]. Wearable devices, or wearables, are smart electronic devices worn on the body [35]. Wearables have the potential to transform the music learning experience of BLV learners. Functionally, wearables can allow BLV musicians free use of their fingers and move their bodies to play musical instruments. Socially, wearables can be integrated into clothing or hidden. This can allow BLV musicians to play music for an audience without hindrance to their musicianship or performance. Studies have explored the use of wearable technologies with vibration motors to communicate musical information [4, 33] however limited research has considered BLV music learners in developing wearable technologies for music.

My research will explore that application of wearable technologies (on the body or the instrument) with vibrotactile feedback to support BLV music learning. This will enable BLV music learners to have free use of their fingers, hands and body while playing and performing while also being aware of musical cues, nonverbal information, and music notation. This research area is important to the ASSETS community as I explore novel, innovative solutions to support people of an underserved community.

Literature Review

Development of ATs such as screen readers [6, 17] and refreshable braille displays [2, 29] have enhanced or replaced vision to empower BLV people to perform tasks, actions and activities. However, BLV music learners face several complex barriers to learn music and engage in musical activities [1, 3, 22]. Studies suggest that these barriers are a result of rigid music learning traditions, ill-equipped music teachers, and limited and inaccessible commercially available tools to support BLV music learning [1, 3, 11]. Baker and Green reported that music teachers in public schools were not equipped to teach BLV music learners as they required special training and knowledge of sight conditions, learning processes and accessibility technologies [3]. Furthermore, Ernits and Kutsar found that some music teachers perceived BLV music learners to be difficult to teach [11]. Unfortunately, however, Moss found that accommodations, flexibility and experience of music teachers greatly influenced the learning experience of BLV music learners [22]. BLV music learners also described limited access to braille music scores and further described braille music cumbersome to learn [1, 26]. Also, studies found that BLV music learners found nonverbal cues and gestures such as pointing, nodding and facial expressions challenging to decipher, especially while playing music [3, 18].

McCord and Watts conducted a survey of 1,416 music educators including 400 music educators in mainstream K-12 schools [21]. They found that music teachers had very little knowledge on using ATs for music learning and limited knowledge of ATs in general. Ernits and Kutsar [11] pointed out that ATs are expensive which makes them inaccessible to BLV musicians and music teachers teaching BLV students. In addition to limited knowledge and access to ATs, commercially available ATs for BLV musicians have been limited to programs for music reading and composition only [13, 19, 32]. Additionally, single line braille readers are not suited for braille music due to the limited number of dots and slow navigation [26].

Studies also found that the sense of touch plays an essential role for BLV music learners. Touch is used to develop an understanding of the instrument [3], interpret instructions through physical guidance [23] and read musical information [27]. Additionally, BLV musicians described self-developed strategies such as asking peers to tap on their shoulder to inform them of musical cues [22]. Modhrain and Gillespie found that the vibrations from an instrument help musicians develop virtuosity and higher levels of skill with an instrument [24]. They stated that haptic receptors in the skin, muscles and joints can feel the tactile response from an instrument and relay that information of force and motion to the player. Furthermore, studies found that vibration patterns can be used to convey musical information such as pitch, tempo, timbre, articulation, dynamics and rhythm [28, 30, 31]. These findings indicate that ATs must be developed considering the sense of touch and vibrotactile. However further research is required to understand the application of vibrotactile feedback to support BLV music learning.

Problem Statement

The long-term goal of this research is to understand, design, develop and evaluate technological solutions that can make music learning more accessible, enjoyable, and equitable for BLV people. I have identified three broad research questions that relate to BLV music learning:

RQ1: What are the design challenges need to be addressed in future assistive technologies for BLV music learning?

RQ2: How can wearable technologies with vibrotactile feedback support BLV student-teacher interactions?

RQ3: What are the long-term benefits and limitations of wearable vibrotactile devices for BLV music learning?

Methods and Progress to Date

Music can be a catalyst for self-development, creative expression and community building for BLV people [25]. It can also be a source of livelihood for gifted individuals. My research focuses on designing and fabricating wearable devices with the BLV music community using a “Nothing About Us, Without Us” approach [7]. This approach recognizes the tacit knowledge and expertise of people with disability and acknowledges that no policy should be decided by any representatives without the full and direct participation of members of the group affected by that policy. Co-design is a research method that enables participants to act as co-researchers and have an active role in data collection, analysis, prototype generation and final action. Using co-design as my overarching research method, BLV music experts will actively participate in the design process and generate design solutions through tactile making and tangible rapid prototyping. BLV music experts take the role of co-researcher and designer to identify and solve problems in collaboration with myself.

This research is organized into three studies. The insights and learning from each study will inform the design, development, and testing of new prototypes to support BLV music learning.

Study one – Understanding design challenges of BLV music learning (completed, manuscript in preparation): To answer RQ1, I conducted 40 one-hour-long interviews with professional and amateur BLV musicians and music teachers (with experience teaching BLV students). I built on previous research [1, 3, 22] to expand on the challenges faced by BLV musicians and identified design challenges that need to be addressed to support individual practice, nonverbal communication, and music memorization. I contacted relevant institutions including the Canadian Council for the Blind (Canada), the Canadian National Institute for the Blind (Canada), the Royal National Institute of Blind People (UK), the American Foundation for the Blind (US) and the World Access for the Blind (US). Simultaneously, we recruited participants through social media platforms (Twitter, Facebook, and Reddit).

Before each interview, participants gave oral or written consent based on the information I provided about the study and the procedure of the interview. I informed participants that I would record audio of the interviews. Every interview began with an introduction of the study followed by an open-ended discussion based on predetermined topics of discussion related to music experience, challenges of music learning, personal adaptations made to learn music and current and future use of technology to support BLV music learning. Next, with the assistance of the second author, I conducted a reflexive thematic analysis of the data using MAXQDA [20] to organize the data into themes. The second author and I began the analysis by agreeing upon a set of initial codes based on individual analysis of one dataset. Next, we divided the remaining 39 datasets and applied the agreed upon codes. We produced 59 codes with 2033 data points. Later, I further organized the 59 codes into meaningful themes.

Through this research I found three broad themes that lead to a discussion of design challenges that future technologies must address to support BLV music learning.

• Importance of adaptations, flexibility, and personal support from music teachers.
• Challenges of music memorization and the advantages and shortcomings of braille music and auditory learning.
• Limited use of existing technology and shortcomings of single line braille displays.

These findings informed subsequent research questions for study two and three. I later discussed potential design challenges that need to be addressed to support BLV music learning through technology.

1. How can technology support BLV student-teacher interaction and communication?
2. How can technology assist music memorization and make braille music more user-friendly?
3. How can future technologies use vibrotactile feedback to convey nonverbal communication and musical instruction?

Study two – Co-designing ATs for BLV music learning (In progress): To respond to the findings from study one and to answer RQ2, I will conduct six virtual co-design sessions with expert BLV musicians and music teachers to prototype design ideas for wearable technologies that support interaction between students and teachers. I will send participants a box of deformable material, a swatch of textures and a modular set of remotely controlled vibration motors. I will join the participants over a video call and facilitate a co-design session where participants will envision and describe wearable design ideas. Furthermore, I will ask participants to envision the application of vibrotactile feedback to convey musical information with the help of modular vibration motors that can be attached to different parts of the body and can be remotely activated. I will ask family members or friends of expert participants to document the codesign process and take photographs of the design ideas presented by expert participants. I am currently designing and pilot testing this study in collaboration with a BLV music expert who will be a co-researcher for this study. Recruitment of expert participants will be done from contacts made through study one. The design and pilot testing phase of this study is scheduled for the Summer 2022 (July and August), followed by co-design sessions and data collection which will take place in early Fall 2022 (September and October) and finally, analysis and writing will be completed by the end of the year (December).

Study three – Longitudinal observation of AT for BLV music learning (future work): To answer RQ3, this study will test the long-term application and usefulness of the wearable prototype designed and developed through study two. The prototype will be in a real-world setting and used by BLV music learners for three months. For this study, I will apply an autoethnographic approach [5]. This approach is method of research that involves self-observation and reflexive investigation. Prior work defines it as ‘research, writing, story and a method that connects the autobiographical and personal to the cultural, social and political’ [10] which is opposed to other theory driven, hypothesis testing methods. An autoethnographic method will enable BLV music learners to be co-authors of this study and reflect upon the real-world application of the prototype. This research will uncover the benefits and limitations of the wearable prototype for BLV music learning over a three-month period. I will ask three participants to journal and reflect on their ongoing music learning experience with the prototype. Additionally, I will conduct open-ended interviews with the participants every month to discuss their ongoing experience with the prototype. This study is scheduled for the Spring and Summer 2023 (March to July).

References

Potential Contribution

This research can considerably advance the area of accessible and wearable computing by providing novel, embodied and wearable user interactions to support BLV people in creative expression and learning. Furthermore, the findings from this research could potentially inform future developments in BLV learning, music therapy and identify applications of vibrotactile feedback and haptics to improve accessibility.

The findings and discussion from Study one provides an empirical research contribution. Through a thematic analysis of the interview data, I identify design challenges that can inform the development of future ATs for BLV music learning. Study two will apply a novel method of co-design that will enable BLV people to participate remotely and engage in the design process. The methodology of the study can potentially inform how future research is remotely conducted with BLV participants. Furthermore, the artifacts created will enable researchers to envision new possibilities for wearable technologies and facilitate new insights and knowledge into the development of ATs. Finally, the development and testing of the prototype in Study three will inform the development of a future AT for BLV music learning that will be open source and shared with industry for wider development and distribution.

1. Abramo and Pierce 2013. An Ethnographic Case Study of Music Learning at a School for the Blind. Bulletin of the Council for Research in Music Education. 195 (2013), 9. DOI:https://doi.org/10.5406/bulcouresmusedu.195.0009.
2. Baker, D. and Green, L. 2016. Perceptions of schooling, pedagogy and notation in the lives of visually-impaired musicians. Research Studies in Music Education. 38, 2 (Dec. 2016), 193–219. DOI:https://doi.org/10.1177/1321103X16656990.
3. Brewster, S.A. and Brown, L.M. 2004. Non-visual information display using tactons. Extended abstracts of the 2004 conferenceonHumanfactorsandcomputingsystems -CHI’04(Vienna,Austria,2004),787.
4. Chang, H. 2016. Autoethnography as Method. Routledge. (2016), 18.
5. Charlton, J. Nothing about us without us. University of California Press.
6. Cooper, N. 1999. A Survey of Current Music Inclusion Practices and Issues in New Jersey. Ohio Music Education Association. 26, 2 (1999), 30.
7. Darrow, A.-A. 1999. Music Educators’ Perceptions Regarding the Inclusion of Students with Severe Disabilities in Music Classrooms. Journal of Music Therapy. 36, 4 (Dec. 1999), 254–273. DOI:https://doi.org/10.1093/jmt/36.4.254.
8. Ellingson, L.L. and Ellis, C. Autoethnography as Constructionist Project. 23.
9. Ernits, T. and Kutsar, K. 2017. Problems of Music Education for Blind and Visually Impaired People in Estonia. Problems in Music Pedagogy. 16, (2017).
10. Goldstein, D. 2000. Music pedagogy for the blind. International Journal of Music Education. os-35, 1 (May 2000), 35– 39. DOI:https://doi.org/10.1177/025576140003500112.
11. GOODFEEL Braille Music Translator: Braille Music Converter - Dancing Dots: https://dancingdots.com/main/goodfeel.htm. Accessed: 2022-04-14.
12. Gougoux, F. et al. 2004. Pitch discrimination in the early blind. Nature. 430, 6997 (Jul. 2004), 309–309. DOI:https://doi.org/10.1038/430309a.
13. Hamilton, R.H. et al. 2004. Absolute pitch in blind musicians: NeuroReport. 15, 5 (Apr. 2004), 803–806. DOI:https://doi.org/10.1097/00001756-200404090-00012.
14. Hammel, A.M. 2001. Special Learners in Elementary Music Classrooms: A Study of Essential Teacher Competencies. Update: Applications of Research in Music Education. 20, 1 (Nov. 2001), 9–13. DOI:https://doi.org/10.1177/875512330102000103.
15. Lime Lighter: Music-Reading Solution for Low Vision Performers - Dancing Dots: https://dancingdots.com/limelighter/limelightermain.htm. Accessed: 2022-04-14.
16. MAXQDA | All-In-One Qualitative & Mixed Methods Data Analysis Tool: https://www.maxqda.com/. Accessed: 2022-07-08.
17. Moss, F.W. Quality of Experience in Mainstreaming and Full Inclusion of Blind and Visually Impaired High School Instrumental Music Students.
18. O’Connell, M. et al. 2006. The use of Tactile Modeling and Physical Guidance as Instructional Strategies in Physical Activity for Children who are Blind. Journal of Visual Impairment & Blindness. 100, 8 (Aug. 2006), 471–477. DOI:https://doi.org/10.1177/0145482X0610000804.
19. Park, H.Y. 2017. Finding meaning through musical growth: Life histories of visually impaired musicians. Musicae Scientiae. 21, 4 (Dec. 2017), 405–417. DOI:https://doi.org/10.1177/1029864917722385.
20. Park, H.-Y. 2015. How Useful is Braille Music?: A Critical Review. International Journal of Disability, Development and Education. 62, 3 (May 2015), 303–318. DOI:https://doi.org/10.1080/1034912X.2015.1020921.
21. Pino, A. and Viladot, L. 2019. Teaching–learning resources and supports in the music classroom: Key aspects for the inclusion of visually impaired students. British Journal of Visual Impairment. 37, 1 (Jan. 2019), 17–28. DOI:https://doi.org/10.1177/0264619618795199.
22. SharpEye Music Scanning Software: https://dancingdots.com/prodesc/sharpeye.htm. Accessed: 2022-04-15.
23. West, T.J. et al. 2019. The Design of the Body:Suit:Score, a Full-Body Vibrotactile Musical Score. Human Interface and the Management of Information. Information in Intelligent Systems. S. Yamamoto and H. Mori, eds. Springer International Publishing. 70–89.
24. WHO releases new global estimates on visual impairment: http://www.emro.who.int/control-and-preventions-of- blindness-and-deafness/announcements/global-estimates-on-visual-impairment.html. Accessed: 2022-07-08.
25. Williams, M.A. et al. 2015. What not to wearable: using participatory workshops to explore wearable device form factors for blind users. Proceedings of the 12th International Web for All Conference (Florence Italy, May 2015), 1–4.
26. Wolffe, K. and Sacks, S.Z. 1997. The Lifestyles of Blind, Low Vision, and Sighted Youths: A Quantitative Comparison. Journal of Visual Impairment & Blindness. 91, 3 (May 1997), 245–257. DOI:https://doi.org/10.1177/0145482X9709100310.

About the Authors