Approaches for Creating Therapy Games

Jared Duval, University of California Santa Cruz,
From left to right: 1) The first image depicts a screenshot of the SpaceTrash minigame in SpokeIt where players charge a crane with their speech magic to clear the moon of debris 2) The second image depicts a player in VR casting a healing spell by gesturing their wand in an 'infinity symbol' motion 3) The last image depicts children on gym mats using a wearable device attached to their head that measures breathing.
Figure 1. From left to right: SpokeIt (Space Trash Mini-game), Spell Casters (Casting a spell in VR), and Super Trouper [16] (Activity with wearable).


Therapy games have the potential to offer people with disabilities a cost-effective, personalized, data-driven, connected, and motivating context for otherwise tedious and repetitive therapy. The paramount challenge in creating therapy games is creating a motivating experience with mechanics that translate into improved health outcomes—a wicked problem. To this end, I use research through design to explore multiple approaches to the co-creation of therapy games for various populations, including children with speech impairments, adults with developmental disabilities, children with Sensory-Based Motor Disorder (SBMD), and stroke survivors. I have collaborated on 3 therapy games, which serve as case studies where I explore identifying best practices, unique insights, and suggestions for future therapy game creators. Specifically, I discuss game-first versus therapy-first approaches, closed-game systems versus more open-ended playful systems, and potential future research directions.


Games are no longer used solely for entertainment. With the insurgence of approaches like gamification [6] and playification [18], games are increasingly used as a motivator in education [5], in the workplace [27], and in healthcare [15,25]. While many ethical concerns have been raised about the use of games in the pursuit of capitalistic agendas [3, 21], there is an opportunity for improved therapy experiences for various populations of people with disabilities (e.g. [1, 12, 23–25]). Therapy games, also referred to as Serious Games for Health [28], offer many potential benefits including performance tracking [13], increased access to healthcare for those from lower socioeconomic statuses [22], reporting features for richer interactions with medical professionals [14], and personalized curricula [20] that dynamically adjust challenge based on performance. These benefits are only available to players who play (because the game is fun) and benefit from playing (because playing improves health).

Approaches and strategies for creating motivating experiences that translate into improved health outcomes is a wicked problem [4] and therefore can be engaged with via research through design (RtD). [29]. The following 3 RtD case studies illustrate various approaches to co-creating therapy games for populations of people with disabilities in the pursuit of creating generalizable knowledge for future therapy game creators. In each case study, I describe the populations, motivations, methods, and current state of development. Then, I discuss a preliminary set of lessons learned including game-first versus therapy-first approaches and closed game system versus open-ended play approaches. Finally, I conclude with envisioned contributions to the ASSETS community and possible future research directions.

Case Study 1: SpokeIt

SpokeIt [8–11] (, shown in the left-most position of Figure 1, is a speech therapy game for children born with orofacial cleft. SpokeIt’s gameplay is entirely driven by speech via two independent speech recognition systems that are capable of distinguishing correct speech from common speech errors [10]. One speech recognition system operates offline for universal access while the second recognition system can be used in tandem with the first when internet is available for improved accuracy and a larger corpus. A game controller can be used in lieu of the speech recognition systems for open-ended play, giving facilitators access to the benefits of Wizard of Oz [7]. SpokeIt was co-designed with medical professionals, developmental psychologists, children with cleft speech, and adults with developmental dis-abilities co-occurring with speech impairments [10] using Participatory Design [2, 26], Wizard of Oz [7], tangible design probes [11,17,19], and rapid medium-fidelity prototyping [11]. SpokeIt is gearing up for release in English (with more languages coming shortly after) and has been my primary project for the past 3 years. An in-the-wild evaluation using telemetry and longitudinal studies will follow release.

Case Study 2: Spell Casters

Spell Casters, shown in the middle image of Figure 1, is a social VR game previously developed purely for entertainment where teams of 5 wizards battle in a magical duel by drawing gestures with their "wands" to cast spells. Before the duel, players select a wizard hat which corresponds to their role and the set of available spells: attacker, tank, or support. Each team receives a pool of "lives" and the last team standing wins. By outfitting the "wand" in a Stroke Survivor’s weaker hand and changing the spell gestures to exercises that are beneficial to rehabilitation, Spell Casters has the potential to demonstrate how games developed primarily for entertainment can be adapted for therapy purposes, reducing development time and costs with designs that have already proven to be fun. We are currently implementing a "training ground" for medical professionals to train the gesture recognition system and for players to practice casting the spells before battling against other players. After medical professionals validate the game is appropriate for use, we will begin evaluating and iterating the game with Stroke Survivors.

Case Study 3: Super Trouper

The collaboration, Super Trouper, shown in the right-most image in Figure 1, began with the development of 6 wearable training devices called Physical Training Technology Probes (TTPs) [16] that can sense and support SMBD therapies: including precise motor skills, gross movements, overall co-ordination, breathing, muscle strength, focus, balance, and posture [16]. The TTPs were proven effective with trainers during authentic circus-themed training situations [16]. In future work, we plan to stream data from the TTPs to an open-ended game to be played at home that will track and guide activities while employing identified motivators—a challenge because children in the training sessions relied on the instructor for engagement, motivation, and instruction.


Game-First or Therapy-First?

Spell Caster takes a game-first, romantic (or designer-centric) approach while SpokeIt and Super Trouper each take a therapy-first approach where user-centered, participatory methods were favored. Game-first approaches should only be considered when source code is available, and the alterations needed to adapt the game for therapy use would be minimal. In these rare situations, the cost of creating the therapy game can be greatly reduced, but risk that the alterations may ruin the very qualities that made the game enjoyable in the first place. In user-centered approaches, many of these risks are mitigated by the iterative process and stakeholder involvement which ensure the game is (hopefully) made right in the first place but result in a much higher costs for time and development.

Closed Game System or Open-Ended Play?

Spell Casters is a closed game system whereas Super Trouper has been designed thus far with open-ended multiplicity in mind. SpokeIt is designed as a closed game system but attaching a game controller allows expansion of the magic circle. Therapy games that follow the closed system typically follow the patient-care-model and take the role of the medical professional by facilitating every aspect of the experience. The benefits of this approach are finely tuned user experiences, validated metrics of efficacy, and controlled magic circles, but runs the risk of frustrating players who are not accurately sensed or who feel they are being "fixed by technology". The less common, open-ended play approach allows players and facilitators to use the technology for their own contexts, in-creasing flexibility and adoption, capitalizing on surrounding expertise, and reaping the benefits of social play, but running the risk of unpredictable usage—and therefore a much more challenging design space


Understanding approaches for co-creating effective and motivating therapy games is a wicked problem that I engage with through research through design. I have involved a diverse set of populations using a variety of methods including focus groups, rapid medium fidelity prototyping, wizard of oz, bodystorming, and tangible design probes. The potential contributions of my work are 2-fold: 1) The games I make have the potential to improve some people’s therapy experiences (though their in-the-wild effectiveness needs evaluation), and 2) The breadth of approaches and applications I have worked on may provide unique insights and strategies. While I have some preliminary insights, such as game-first versus therapy-first approaches and closed game system versus open ended play, there are many more lenses that can be explored such as single player versus multiplayer experiences, symmetrical versus asymmetrical gameplay, and collaborative versus competitive gameplay.


Thank you, Sri Kurniawan, Katherine Isbister, Elena Márquez Segura, and my collaborators. This work is supported by NSF Grant #1617253. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.


  1. Gazihan Alankus, Amanda Lazar, Matt May, and Caitlin Kelleher. 2010. Towards customizable games for stroke rehabilitation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2113–2122.
  2. Ditte Amund Basballe, Kim Halskov, and
  3. Nicolai Brodersen Hansen. 2016. The early shaping of participatory design at PDC. In Proceedings of the 14th Participatory Design Conference: Short Papers, Interactive Exhibitions, Workshops-Volume 2. ACM, 21–24.
  4. Ian Bogost. 2015. Why gamification is bullshit. The gameful world: Approaches, issues, applications 65
  5. (2015).
  6. Richard Buchanan. 1992. Wicked problems in design thinking. Design issues 8, 2 (1992), 5–21.
  7. Alessandro De Gloria, Francesco Bellotti, and Riccardo Berta. 2014. Serious Games for education and training. International Journal of Serious Games 1, 1 (2014).
  8. Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. ACM, 9–15.
  9. Steven Dow, Blair MacIntyre, Jaemin Lee, Christopher Oezbek, Jay David Bolter, and Maribeth Gandy. 2005. Wizard of Oz support throughout an iterative design process. IEEE Pervasive Computing 4, 4 (2005), 18–26.
  10. Jared Duval. 2017. A mobile game system for improving the speech therapy experience. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 72.
  11. Jared Duval, Zachary Rubin, Elizabeth Goldman, Nick Antrilli, Yu Zhang, Su-Hua Wang, and Sri Kurniawan. 2017. Designing Towards Maximum Motivation and Engagement in an Interactive Speech Therapy Game. In Proceedings of the 2017 Conference on Interaction Design and Children. ACM, 589–594.
  12. Jared Duval, Zachary Rubin, Elena Márquez Segura, Natalie Friedman, Milla Zlatanov, Louise Yang, and Sri Kurniawan. 2018. SpokeIt: building a mobile speech therapy experience. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 50.
  13. Jared Scott Duval, Elena Márquez Segura, and Sri Kurniawan. 2018. SpokeIt: A Co-Created Speech Therapy Experience. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, D501.
  14. Kathrin Maria Gerling, Regan L Mandryk, Max Valentin Birk, Matthew Miller, and Rita Orji. 2014. The effects of embodied persuasive games on player attitudes toward people using wheelchairs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3413–3422.
  15. Stefan Göbel, Sandro Hardy, Viktor Wendel, Florian Mehm, and Ralf Steinmetz. 2010. Serious games for health: personalized exergames. In Proceedings of the 18th ACM international conference on Multimedia. ACM, 1663–1666.
  16. Maurits Graafland, Mary Dankbaar, Agali Mert, Joep Lagro, Laura De Wit-Zuurendonk, Stephanie Schuit, Alma Schaafstal, and Marlies Schijven. 2014. How to systematically assess serious games applied to health care. JMIR serious games 2, 2 (2014), e11.
  17. Pamela M Kato. 2010. Video games in health care: Closing the gap. Review of general psychology 14, 2, (2010), 113–121.
  18. Elena Márquez Segura, Laia Turmo Vidal, Luis Parrilla Bel, and Annika Waern. 2019. Circus, Play and Technology Probes: Training Body Awareness and Control with Children. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS ’19). ACM, New York, NY, USA, 1223–1236. DOI:
  19. Elena Márquez Segura, Laia Turmo Vidal, and Asreen Rostami. 2016. Bodystorming for movement-based interaction design. Human Technology 12 (2016).
  20. Elena Márquez Segura, Annika Waern, Luis Márquez Segura, and David López Recio. 2016. Playification: The PhySeEar case. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play. ACM, 376–388.
  21. Tuuli Mattelmäki and others. 2006. Design probes. Aalto University.
  22. Simon McCallum. 2012. Gamification and serious games for personalized health.. In pHealth. 85–96.
  23. Jane McGonigal. 2011. We don’t need no stinkin’ badges: How to re-invent reality without gamification. Retrieved December 18 (2011), 2015.
  24. S Anne Moorhead, Diane E Hazlett, Laura Harrison, Jennifer K Carroll, Anthea Irwin, and Ciska Hoving. 2013. A new dimension of health care: systematic review of the uses, benefits, and limitations of social media for health communication. Journal of medical Internet research 15, 4 (2013), e85.
  25. Tony Morelli, Lauren Lieberman, John Foley, and Eelke Folmer. 2014. An exergame to improve balance in children who are blind.. In FDG.
  26. Heidi Parisod, Anni Pakarinen, Anna Axelin, Riitta Danielsson-Ojala, Jouni Smed, and Sanna Salanterä. 2017. Designing a health-game intervention supporting health literacy and a tobacco-free life in early adolescence. Games for health journal 6, 4 (2017), 187–199.

About the Authors

Jared is a therapy game creator, a play chaser, and an advocate for a healthier tomorrow.  His broad range of skills and experiences—from the technical, aesthetic, playful, and medical—has culminated in a passion for chasing play in therapy contexts to co-create therapy games that are fun, more productive, and engaging for people with disabilities. He is the creator of SpokeIt, a speech therapy game for children with cleft speech, Cirkus, a movement-based game for children with Sensory Based Motor Disorder, and Spellcasters, a virtual reality physical rehabilitation game for stroke survivors.