Developing Inclusive e-Learning and e-Entertainment to Effectively Accommodate Learning Difficulties

Anthony Savidis and Constantine Stephanidis

Institution:

Human Computer Interaction Laboratory, Institute of Computer Science, FORTH, Heraklion, Crete, Greece

Email:

as@ics.forth.gr

Introduction

Inclusive e-Learning

The requirement for e-inclusion of software applications and services relates to the employment of appropriate development strategies so as to accomplish accessibility and increased interaction quality at deployment time. Effectively, inclusive e-learning is the outcome from the application of e-inclusion design and implementation methods in the context of e-learning systems.

The development of e-learning systems is targeted towards supporting the overall learning process with software instruments enabling: (a) learners to easily and effectively assimilate the learning material; (b) tutors to carry out more productive and effective learning processes; and (c) supervisors to organize, execute, monitor and evaluate the on-line learning process.

In this context, e-learning systems are basically “information systems”, however, not always structured and developed following the particular software engineering methods of the information-systems field. For instance, edutainment software employs multimedia interaction methods to deliver cartoon-like, mostly drill-and-practice, sessions in a “learning by playing” fashion. In such systems, the learning content is mostly fused with the multimedia presentation logic, usually not being modeled and stored as structured content in a database. Additionally, training systems targeted to putting trainees very close to the real working tasks and activities put primary emphasis in the implementation of “learning by doing” methods.

Inclusive e-Entertainment

The role of entertainment for pedagogical purposes, supporting or even complementing the learning process, is well known in the education domain, being universally acknowledged. Today, electronic entertainment is very popular, attributing to a huge number of commercial game titles, while only in US having an annual turnover far exceeding the American Box Office. Additionally, there is a wide spread total of specialized game consoles, with a total number being larger than the amount of home personal computers.

While there has been a lot of research and development towards universal access, it is only recently there the need for accessible entertainment has been widely recognized, with the formation of the Game Accessibility Special Interest Group (GA-SIG, 2004) of Independent Game Developers Association (IGDA), defining game accessibility as “the ability to play a game even when functioning under limiting conditions. Limiting conditions can be functional limitations, or disabilities - such as blindness, deafness, or mobility limitations”. More recently, there is a serious trend for introducing games for training and learning (known as game-base learning, Prensky, 2000) in order to take advantage of the unparalleled motivation and engagement that computer games can offer to learners of all ages.

While being primarily driven by our objective to provide e-learning systems for people with mild or sever cognitive disabilities, during early development and experimentation it quickly became obvious that playing is an added-value instrument that can be deployed not only to accomplish increased motivation, but also as a genuine learning or training tool to sharpen particular skills and capabilities.

Deploying systematically both domains

In this context of we have concentrated in developing an accessible application suite consisting of inclusive e-learning and e-entertainment applications, accommodating learning difficulties concurrently combined with severe sensory and / or motor impairments. This objective was put forward in our attempt to pursue integrated inclusive e-learning technologies, that would effectively deliver to learners an educational experience accompanied with thoroughly accessible entertainment instruments. This article briefly reports the design, implementation, evaluation, and deployment experience regarding specific inclusive e-learning and e-entertainment systems. The reported systems addressed the need of vocational training and entertaining for people with learning difficulties combined with sensory and / or motor impairments. In this context, two educational applications were developed, adopting a didactic style borrowing elements from both “learning by doing” and “learning by studying” approaches, and two accessible entertaining applications with an originally training deployment purpose:

• Educational applications. (a) An accessible “canteen manager” application, training hand-motor and people with cognitive disabilities for the cashier management of a typical “canteen”. In this case the training application was also the real-life application system, while the training process has been carried out with a supervisor either present in the field or indirectly monitoring through a camera. (b) A multimedia “sew tutorial” application, training people with cognitive disabilities in typical sewing tasks. In this case, the trainees are put in front of the real sewing devices, while the training process is based on two strategies: (i) giving each student a personal computer, requiring that they manually operate the multimedia tutorial application; and (ii) putting a single large projected display and loud speakers, as a coordinated training session guided and operated by a tutor-supervisor.
• Entertaining applications. Two desktop accessible and highly configurable action games, for combined disabilities like cognitive, hearing, motor ad vision impairments. The critical importance of universally accessible games for disabled people has been acknowledged in the recent establishment of the SIG on Game Accessibility of the Independent Game Developers Association - the HCI Laboratory is a member of this SIG. In particular, the two action games are accessible and configurable remakes of the Access Invaders and the Pong games, which from early experiments proved to play a significant role in challenging and sharpening basic kinesthetic skills, orientation capabilities, short-term strategic thinking, and decision making, while highly enhancing the esteem of involved disabled players / learners. Their effective deployment concurrently with the typical educational applications has had an amplifying effect and supported faster learning cycles.

The conclusions drawn from this particular research work relate both to the design aspects as well as to the deployment approach, for both the accessible educational and entertaining applications. Overall, this work has resulted in the definition of a systematic development and deployment strategy to fuse both approaches, so as to more effectively accommodate learning difficulties, especially when the latter are combined with sensory or motor impairments as well.

Inclusive training applications

The two purposefully developed inclusive training applications developed are: (a) a cashier application (for canteen management); and (b) a sew-trainer application.

Cashier application

The cashier application has been developed with a twofold role: (a) to support accessibility for people with cognitive difficulties combined with severe disabilities on the upper limbs; and (b) to serve both as the initial training instrument and as the target application to be eventually deployed in real-life. The foreseen real-life set-up has been as follows:

• The cashier application operator (the disabled user) interacts with the customer to receive the orders and then issues the receipt that is given to the customer;
• Another member of the staff (an able user) gets the printed receipt from the customer, together with the payment, deposits the cash and then supplies the corresponding products.

Following the key design requirements, the training session for the deployment phase has been eventually organized in two different ways (see Figure 1): (a) in direct supervised mode (initially, upon the early training sessions); and (b) in indirect supervised mode (after the user has gained adequate experience of use). The User Interface had to be easy to use, comprehensible and accessible. In this context, it has been decided to minimise the use of verbal explanations (text) as much as possible, by providing iconic representations, in most cases exact photographs of the canteen products, together with digitised speech-audio.

This design style is different from the one deployed in common cashier applications used in restaurants, cafes or canteens, all of which largely employ textual descriptions. The initial screen of the cashier training application is provided in Figure 2 (top left). As shown, product categories are provided with iconic representations (not photographs), which have been carefully chosen among alternatives after an initial interview and evaluation round with end-users, scoring the appropriateness of each alternative icon regarding comprehensibility. After a specific product category is chosen, a menu consisting of the product photographs is displayed (e.g., see Figure 2 - middle left). The trainee may either select a product, in which case the focus moves automatically on the dialogue box to provide the number of purchased items or the trainee “returns back” to the product categories menu.

The design decision to emphasize the use of iconic menus through exact photographic representations of products on the application menus has been proved to be suitable for the specific type of training topic and categories of trainees, as it allowed us to quickly shift to unsupervised sessions (actually indirectly supervised as the sessions were always monitored through a camera). It has been observed that trainees were capable to easily handle the sessions without any external support, a fact that allowed them to quickly increase their knowledge regarding the cashier system, while in the mean time developing their confidence and sense of independence in being able to effectively accomplish the assigned task.

The design of the User Interface to support accessibility by hand-motor impaired users has employed switch-based hierarchical scanning techniques, in particular the method developed for augmenting the basic windows object library, originally reported in (Savidis et al., 1997).

Sew-trainer application

This application had a similar purpose with the cashier trainer application, in the sense that, one the one hand it is focused on a very specific job, while on the other hand the training application is intended for direct deployment during the field trials. The only difference was related to the fact that while the cashier trainer had a dual role, being both the training application and the real application in the field, the “sew trainer” played primarily the role of a multimedia tutor. However, the deployment approach for the sew trainer application for real-life training sessions had to also support different tutoring scenarios, as it was derived from the requirements of the particular target user group.

During the initial design-discussion sessions with the experts in special education, coming mainly from the specific organization, where the sew trainer would be eventually deployed, it came out that it was critical to support both (see also Figure 3): (a) individualized and group training (i.e. each trainee or group of trainees sitting in front of a PC running the sew trainer application); and (b) tutoring classroom sessions, were the system is in use by a trainer, projecting into a large screen, while all trainees sit in front of the actual sewing machines.

The User Interface design had to be particularly simple, delivering minimal functionality while emphasizing iconic illustrations and pictures. Initially, as the target user group consisted of language literate adults, we assumed a satisfactory reading capability. However, it has been quickly understood that this was not necessarily the general case, as we were faced with users of varying reading capabilities or deficiencies. Hence, even for textual / verbal illustrations it was mandated to introduce digitised speech of a relatively slow speech rate (it was proved very quickly that the use of speech synthesis in the Greek language was far less than satisfactory).

As shown in Figure 4 (bottom) trainees may select the parts of the sewing machine for which they need explanation / training material. In this case, typical active areas over the graphical image were defined, providing audio feedback regarding the title of the particular associated machine accessory and / or sewing procedure. Selection can be carried out by either clicking on the active area or by pressing the associated push button (with the photographic or video camera icon). It is important to note that users had to be trained a little to accommodate the relationship between a line, its associated machine part and the iconic illustration denoting that an explanation is available.

Inclusive entertaining applications

The development of the entertaining applications was carried out from scratch, as we needed to introduce: (a) a number of purposeful game-play characteristics; (b) accessibility support to accommodate sensory and motor disabilities in accordance to learning difficulties; and (c) to encapsulate extensive configuration facilities both at the level of the User Interface as well as at the basic game logic. The role of the entertaining applications was very critical in our approach: to support implicit highly motivated and stimulating training for specific skills. In particular, we have chosen computer game genres through which we could sharpen some key basic skills, being particularly important for the type of the training applications we have developed. It is well-known that play is a fundamental activity of children that allows them to acquire the foundations of self-reflection and abstract thinking, develop complex communication and meta-communication skills, learn to manage their emotions and explore the roles and rules of functioning in adult society (Verenikina, et al., 2003).

This systematic concurrent deployment of purposefully developed accessible games, together with the targeted mission-specific training applications, has been proved fin the course of real practice that is more effectively accommodates learning difficulties. More specifically, some of the representative basic and non-basic skills that were exercised and sharpened with the two accessible video games developed are:

• Orientation
  • Recognizing direction of a moving object
  • Predicting direction of a moving object
  • Predicting collision of moving objects
  • Controlling motion with hand (mouse, keyboard, switches, joystick)
  • Moving objects in different directions, with different granularities of speed and displacement
  • Following a predetermined path
  • Reaching a desirable target position
  • Following the movement of another object
• Cause and effect
  • Assimilating cause and effect
  • Reproducing cause to accomplish and effect
  • Recognizing effect and understanding its cause
• Counting (with visual / auditory feedback)
  • Recognizing the total number objects in a set
  • Calculating when a desirable number of objects in a set is reached
  • Recognizing increment / decrement effect
  • Associating numeric values with explicitly enumerated objects
  • Numeric values (higher / lower)
  • Enumerated objects (more / less)
  • Volume and size (higher - bigger / lower - smaller)
  • Distances (farther / closer)
• Elementary arithmetic
  • Comparison (as above)
  • Counting (as above)
  • The notion of zero (nothing / empty)
  • Addition with objects and numeric values
  • Subtraction with objects and numeric values

Pong game

The Pong game is a classical remake of the original Pong game, one of the first computer-games ever. It has been selected due to its simplicity, as well as because it offers a platform to sharpen various kinesthetic and intellectual capabilities in a straightforward manner, enabling us to add different scenarios and conditional progress within the very simple game plot. The basic game has been developed to be accessible by both hand-motor impaired users, as well as blind users. This has been accomplished with the developed of a an auditory media space (approximately 2W x 1.5H meters), supporting spatial audio, consisting in the final version of 24 loudspeakers (4 rows, 6 loudspeakers each, with 24 separate audio channels). The prototype of the auditory lattice is illustrated in Figure 5 - a special purpose wood-made set-up is under construction. Game snapshots are provided in Figure 6.

As a training instrument, the Pong game was always deployed in direct supervised mode, putting specific task to users with cognitive difficulties. The configuration gives sufficient flexibility to therapists / tutors in fine-tuning game and interaction aspects to individual end-user requirements. For instance, ball speed, force-feedback support and degree of automatic player-paddle positioning, scoring increase / decrease factors, paddle speed, computer opponent accuracy, etc., can be effectively configured. Additionally, the configurability of bitmaps allows virtually new scenes to be delivered, as shown in Figure 7.

The tasks put by the supervisor are given either before start-up while the user is playing; representative categories are defined below (some of those are reusable across different types of game):

• Reach score N, then lose;
• Play until you hit the ball at your paddle M times;
• Make the ball hit the left / right / up / down wall;
• Play until the opponent score becomes K;
• Pause the game;
• Where is the direction of the ball after we restart?
• Lose now;
• Produce an explosion (happens after certain score increment);
• Make the computer “laugh” (happens in a certain profile when loosing with 0 score);
• Try to make a score K less / more than M;

Access Invaders game

This is another remake of a classical game, which delivers a radically different class of action and plot. Kinesthetically, it involves varying categories of skills, while perceptually it requires recognition of more complicated phenomena, relevant causes and associated effects. Due to its more rich graphical appearance, it usually becomes more attractive and appealing to end-users than the Pong game itself. In the meantime, due to the fact that it offers a larger repertoire of game-oriented events and user activities, it enables the definition of numerous alternative types of training tasks.

The availability of different profiles of the Access Invaders game is illustrated in Figure 8. While reflecting the numerous configuration capabilities of the game, some of the representative training tasks is:

• Destroy N aliens;
• Destroy the leftmost / rightmost platform.
• Destroy the Nth platform from the left / right;
• Set your ship at a position where there is no alien ship above;
• Set your ship at a position that there is a platform on top;
• Fire N bullets;
• Move left, right, and left again to return to the original position;
• How many aliens are there at the bottom row?
• Destroy a <specify color> alien.

Conclusions

We have concentrated on the effective accommodation of learning difficulties, when combined with sensory and motor impairments, through the development of accessible training systems. Additionally, being originally focused on the combination of entertaining sessions within a primarily learning process, we have resulted in the delivery of accessible games, with scenario-based targeted deployment. We did not prefer the delivery of integrated systems in the form of typical edutainment software, since for such job-targeted training systems, any sever deviation from the original tasks to be carried out by end-users in the real field becomes inappropriate. Instead, we have formulated a fused approach, emphasizing entertainment as a metaphoric training process, supported through supervised sessions, while heavily relying on metaphoric scenarios and tasks, and intensive trainee-trainer interaction. This integrated view of the training process, however, with distinct and separate roles of the deployed software instruments, is illustrated in Figure 9. During the playing / training sessions, it quickly became evident that the intensity of putting explicit tasks to end-users had to be always compromised by setting-up an overall atmosphere of a genuine playful experience. In other words, users did not appreciate the inexcusable transformation of a primarily joyful task to an exercising activity. So, we had to improvise and produce scenarios and trainer-trainee dialogue sessions that would facilitate the smooth integration and delivery of the training tasks as purely encapsulated events within the social gaming activity.

Acknowledgements

The Pong and Access Invaders entertaining applications have been mainly developed in the context of the MICOLE project (IST-2003-511592 STP), partly funded by the Information Society Technologies Programme of the European Commission DG Information Society. The partners in the MICOLE consortium are: University of Tampere (Finland); University of Glasgow (U.K.); University of Metz (France); Uppsala University (Sweden), Lund University (Sweden); Royal Institute of Technology KTH (Sweden); Siauliai University (Lithuania); Institute of Computer Science FORTH (Greece); University of Linz (Austria); University of Pierre and Marie Curie (France); France Telecom (France); Reachin Technologies AB (Sweden).

The cashier and sew-trainer training applications have been developed and are deployed under a sub-contract (2001) from the “Centre of Special Training”, of the “Therapeutic Clinic of Lifelong Diseases”, located at the city of Agios Nikolaos, region of Crete.

References

1. Game Accessibility Special Interest Group (GA-SIG), 2004. “Accessibility in Games: Motivations and Approaches”, electronically available at: http://www.igda.org/accessibility/IGDA_Accessibility_WhitePaper.pdf
2. Prensky, M. (2000). Digital Game-Based Learning. McGraw-Hill.
3. Savidis, A., Vernardos, G., Stephanidis, A. (1997). Embedding scanning techniques accessible to motor-impaired users in the WINDOWS object library. In, Design of Computing Systems: Cognitive Considerations (21A). Salvendy, G., Smith, M., Koubek, R. (Eds). Elsevier, 1997, 429-432.
4. Verenikina, I., Harris, P. and Lysaght, P. (2003). Child’s Play: Computer Games, Theories of Play and Children’s Development. In Proc. Young Children and Learning Technologies. Selected papers from the International Federation for Information Processing Working Group 3.5 Open Conference, Melbourne, Australia. Conferences in Research and Practice in Information Technology, 34. Wright, J., McDougall, A., Murnane, J. and Lowe, J., Eds., ACS. 99.