Simon Baron-Cohen, Jacqueline Hill, Ofer Golan, and Sally Wheelwright

Autism Research Centre
University of Cambridge
Departments of Experimental Psychology
Downing St, Cambridge, CB2 3EB, UK

Acknowledgments: The authors were supported during the period of this work by the Shirley Foundation, the Corob Foundation, the Three Guineas Trust, and the Medical Research Council (UK). With special thanks to Max Whitby and the Human Emotions team at Red Green and Blue Co, and the late Ted Barnes at Cambridge Learning. This paper is reproduced from Cambridge Medicine, with kind permission.

A Cambridge University team of psychologists have just completed a two-year project working closely with a London multi-media production company, Red Green and Blue Co, to produce the world's first electronic encyclopaedia of emotions. Produced on DVD-ROM, the product is entitled Mind Reading: the interactive guide to emotions. The Cambridge University team are all based in the Autism Research Centre, and were motivated to attempt this undertaking because of the lack of any tailor-made educational software for people on the autistic spectrum, many of who have difficulties in recognizing emotions.

The first step for the Cambridge team was to decide how many emotions there were. Psychologists have for decades worked with a standard set of 'Ekman' faces: these are photographs of the six 'basic' or universal emotions, developed by Californian psychologist Paul Ekman. The basic emotions are happy, sad, angry, afraid, surprise and disgust. These six emotions are universally recognized and universally expressed through the same facial features. The Cambridge team however decided to take a comprehensive approach, and used a thesaurus to identify every word in the English language that describes an emotion. They discovered that there were 412 human emotions (excluding synonyms). These 412 are therefore distinct emotions, in terms of their dictionary definition.

They then took the 412 emotion words into mainstream schools in the Cambridge area, to determine at what age typically developing children and teenagers know the meaning of each word. From this they were able to assign a level to each word, from 1 to 6 (1 being primary school age, 6 being adult level). Lastly, they decided set out to taxonomise all of the emotions, since 412 is too large a number to work with easily. They found that virtually all emotions could be assigned to one of 24 different groups. The 24 groups are shown in Table 1. Examples of emotions from just one of these groups (the Afraid Group) are shown in Table 2.

Armed with the first comprehensive encyclopaedia of emotion, they presented the multimedia company with the challenge of developing some software that would be suitable for people of all ages and abilities, who wanted to learn more about emotion recognition. They limited the brief by focusing on the expression of emotion through the face and the voice. The result is the DVD Mind Reading.

On this DVD, 6 actors portray each of the 412 emotions by acting the emotion, captured using video from their facial expression, and using audio from their vocalisation. Each of the 412 emotions is also explained through 6 six stories to give a flavour of the kinds of contexts that give rise to that particular emotion. The advantage of DVD-ROM format is that it can hold this many "assets": it runs into over 5000 separate video clips, audio clips, and text files.

To make the DVD useful as a browsable, searchable database of emotion, the authors have included an Emotions Library. To make it useful as a teaching tool, for parents or teachers or therapists, or users directly, the authors have included a Learning Centre, where there are guided tutorials to take you through the top 20 emotions, or the top 100 emotions. Here, you test yourself on matching emotion words to faces, emotions in intonation with emotion in the face, or emotion words to voices. Advanced users can go further, but these 100 emotions are more than enough to provide the fundamentals of emotion recognition. And to make the learning experience more attractive, you can choose your own type of reward to win, through the quizzes. (The rewards include flag or bird collecting, viewing the insides of precise mechanisms, watching trains, among others).

Finally, to hook the reluctant student into learning about emotions, the authors have included a Games Zone, in which learning about emotions happens indirectly, implicitly, whilst playing card games or puzzles. An example is Hidden Face, where you have to guess the emotion in the face as quickly as you can, and before too many clues are given. A very novel game is Famous Face, where the actor Daniel Ratcliffe (who played Harry Potter in the film of the book) poses different emotions, and you are invited to control his face via a slider, to move him from angry to happy, along an emotion spectrum of intensities.

The publisher of this attractive DVD, Human Emotions Ltd, describes it as suitable for anyone interested in emotions. Whilst they kept in mind people with autism as one important group of potential users (and indeed the charitable funding to develop the product came from the Shirley Foundation, devoted to helping autism), they recognize that there are many reasons why someone might need help to learn about emotions. Emotion recognition problems affect other clinical conditions, such as difficult-to-manage children or people with learning difficulties, for example. Emotion recognition is also an important area of study for people working in the dramatic arts. The world of emotions is also a key area in people-centred professions. Social-skills training is also part of management training, and an important part of the national curriculum in mainstream schools, through 'Personal and Social Education' (PSE). The DVD provides not only a rich collection of emotions as a collection, but a focus for discussions in class settings on the nature of emotions, and on the importance of empathy.

Will it make a difference to people with autism? The Cambridge team are now returning to the world of science, to evaluate the benefits of using the DVD through a controlled treatment trial. There are some obvious advantages to studying emotions on computer, for people with autism. First, emotions in the real world happen very fast, and are transient. You can't replay them if you didn't quite catch them in real time. On the computer, you can play them over and over again, until you've really cracked them. Secondly, putting emotions into the computer might solve the problem that some people with autism have, of not particularly wanting to socialize, yet needing to learn about people. In divorcing emotions from people, and in using computers as the learning tool (one which many people with autism actually enjoy), this approach may lead to better learning. Emotions without the anxiety that may accompany real social interaction.

This naturally prompts the question as to whether anything learnt from the DVD will generalise to the real world. The DVD has been designed to foster generalisation, in having a range of actors perform each emotion. That way, the user gets away from the idea that every emotion has just one format, and is forced to appreciate that different people show the same emotion in different ways. But it also allows them to extract that there may be some common features for any given emotional expression.

The Cambridge University team have been involved in the study of mind reading skills for many years. The background theory and experimental evidence was summarised in Mindblindness (Baron-Cohen, 1995, MIT Press) and the first training study was summarised in Teaching children with autism to mindread (Howlin, Baron-Cohen, and Hadwin, 1999, Wiley). The latter demonstrated that even if people with autism do not pick up mind-reading skills in the natural way, they can learn to do so when these are broken down into the specifics. But whereas the earlier work focused on teaching just a small number of mental states and emotions (happy, sad, angry, and afraid), the new DVD focuses on a comprehensive approach to emotions. The authors are clear that they are not presenting this as a cure, but see it as one component of social skills teaching that might be helpful.
 
For more information on Mindreading: The Interactive Guide to Emotions, including how to obtain it, visit www.human-emotions.com