[01] Barbra Zupan, Department of Applied Linguistics, Brock University, Ontario, Canada. [02] Duncan R. Babbage, Auckland University of Technology, Centre for Person Centered Research, Auckland, New Zealand. [03] Joan E. Sussman, Department of Communicative Disorders and Sciences, University at Buffalo, Buffalo, New York, USA.

We report a preliminary investigation of how perception of congruent and incongruent visual and auditory cues of emotion changes from early childhood to adulthood. Research has shown that for speech perception, children prefer auditory cues whereas adults prefer visual cues. Forty listeners participated, ten in each of the following age groups: preschool, school-aged, early adolescents, and adults. Participants were exposed to a total of 184 semantically neutral sentences (60 congruent visual-auditory) portrayed by one male and female speaker with happy, sad, angry, or fearful emotional expressions. As expected, increased accuracy in identifying the emotion portrayed was seen for congruent stimuli in comparison to incongruent stimuli. Although a clear auditory dominance by preschool children was not indicated, they produced significantly fewer visually-based responses for incongruent visual-auditory signals compared to older children and adults. Explanation for this may include a difference in the salience of visual versus auditory emotion cues for this age group, the limited number of emotion categories included in the current study, and small participant groups.

Emotion Processing, Bimodal Processing, Modality Dominance, Dynamic Cues, Incongruent, Children

[01] Ambady, N., & Gray, H.M. (2002). On being sad and mistaken: Mood effects on the accuracy of thin-sliced judgments. Journal of Personality and Social Psychology, 83(4), 947-961. [02] Banse, R., & Scherer, K. R. (1996). Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology, 70(3), 614–36. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/8851745 [03] Bastian, V.A., Burns, N.R., & Nettlebeck, T. (2005). Emotional intelligence predicts life skills, but not as well as personality and cognitive abilities. Personality and Individual Differences, 39, 1135-1145. [04] Batty, M., & Taylor, M. J. (2006). The development of emotional face processing during childhood. Developmental Science, 9(2), 207–20. doi:10.1111/j.1467-7687.2006.00480.x [05] Boyatzis, C. J., Chazan, E., & Ting, C. Z. (1993). Preschool children’s decodign of facial emotions. The Journal of Genetic Psychology: Research and Theory on Human Development, 154, 375–382. [06] Brackett, M.A., Mayer, J.D., & Warner, R.M. (2004). Emotional intelligence and its relation to everyday behavior. Personality and Individual Differences. 36, 1387-1402. [07] Byron, K., Terranova, S., & Nowicki, Jr., S. (2007). Nonverbal emotion recognition and salespersons: Linking ability to perceived and actual success. Journal of Applied Social Psychology, 37(11), 2600-2619. [08] Carrow-Woolfolk, E. (1995). OWLS: Oral and Written Language Scales. Minneapolis, MN: Pearson Assessments. [09] Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F. (2008). Audio-visual integration of emotion expression. Brain Research, 1242, 126–35. doi:10.1016/j.brainres.2008.04.023 [10] Copeland, W.E., Woke, D., Angold, A., & Costello, E.J. (2013). Adult psychiatric outcomes of bullying and being bullied by peers in chidlhood and adolesence. JAMA Psychiatry, 70(4), 419-426. [11] De Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye. Cognition & Emotion, 14(3), 289–311. doi:10.1080/026999300378824 [12] Dolan, R. J., Morris, J. S., & de Gelder, B. (2001). Crossmodal binding of fear in voice and face. Proceedings of the National Academy of Sciences of the United States of America, 98(17), 10006–10. doi:10.1073/pnas.171288598 [13] Ekman, P., & Friesen, W. V. (1978). Facial affect coding system: A technique for the measurement of facial movment. Palo Alto, CA: Consulting Psychologists Press. [14] Ekman, P., Friesen, W. V, & Ancoli, S. (1980). Facial signs of emotional experience. Journal of Personality and Social Psychology, 39(6), 1125–1134. [15] Ekman, P., Friesen, W. V, Sullivan, M. O., Diacoyanni-tarlatzis, I., Scherer, K., Chan, A., … Krause, R. (1987). Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion. Personality Processes and Individual Differences, 5(4), 712–717. [16] Elfenbein, H.A., Marsh, A.A., & Ambady, N. (2002). Emotional intelligence and the recognition of emotion from facial expressions. In L.F. Barrett & P. Salovey (Eds.), The Wisdom in Feeling: Psychological Processes in Emotional Intelligence (pp. 37-59). New York: Guilford Press. [17] Friend, M. (2000). Developmental changes in sensitivity to vocal paralanguage. Developmental Science, 3(2), 148–162. doi:10.1111/1467-7687.00108 [18] Friend, M., & Bryant, J. B. (2000). A development lexical bias in the interpretation of discrepant messages. Merrill-Palmer Quarterly, 46, 342–369. [19] Friesen, W. V, & Ekman, P. (1983). EMFACS-7: Emotional Facial Affect Coding System (Unpublishe.). University of California, CA. [20] Ganesalingam, K., Sanson, A., Aderson, V., & Yeates, K.O. (2007). Self-regulation as a mediator of the effects of childhood traumatic brain injury on social and behavioral functioning. Journal of the International Neuropsychological Society, 13(2), 298-311. [21] Goldman, R., & Fristoe, M. (2000). Goldman Fristoe Test of Articulation-2. Circle Pines, MN: American Guidance Service (AGS). [22] Goldwave Digital Audio Editor. (2008). St. John’s, NFLD: Goldwave, Inc. [23] Herba, C. M., Landau, S., Russell, T., Ecker, C., & Phillips, M. L. (2006). The development of emotion-processing in children: effects of age, emotion, and intensity. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 47(11), 1098–106. doi:10.1111/j.1469-7610.2006.01652.x [24] Hess, U., Adams, R. B., & Kleck, R. E. (2004). Facial appearance, gender, and emotion expression. Emotion (Washington, D.C.), 4(4), 378–88. doi:10.1037/1528-3542.4.4.378 [25] Johnstone, T., & Scherer, K. R. (2000). Vocal communication of emotion. In The Handbook of Emotion (pp. 22–235). [26] Keltner, D., & Ekman, P. (2000). Facial expression of emotion. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of Emotions. New York: Guilford Press. [27] Laukka, P., Juslin, P., & Bresin, R. (2005). A dimensional approach to vocal expressions of emotion. Cognition & Emotion, 19(5), 633-653. [28] Leppänen, J. M., & Hietanen, J. K. (2004). Positive facial expressions are recognized faster than negative facial expressions, but why? Psychological Research, 69(1-2), 22–9. doi:10.1007/s00426-003-0157-2 [29] Lopes, P.N., Grewa, D., Kadis, J., Gall, M., & Salovey, P. (2006). Evidence that emotional intelligence is related to job performance and effect and attitudes at work. Psichotherma, 18, 132-138. [30] Magix Entertainer. (2007). Movie Edit Pro 12. [31] Massaro, D. W., & Cohen, M. M. (1996). Perceiving speech from inverted faces. Perception and Psychophysics, 58(7), 1047–1065. [32] Massaro, D. W., & Egan, P. B. (1996). Perceiving affect from the voice and the face. Psychonomic Bulletin & Review, 3(2), 215–221. [33] McGurk, H., & MacDonald, J. (1976). Hearing Lips and Seeing Voices. Nature, 264. [34] Morton, J. B., & Trehub, S. E. (2001). Children’s Understanding of Emotion in Speech. Child Development, 72(3), 834–843. doi:10.1111/1467-8624.00318 [35] Morton, J. B., Trehub, S. E., & Zelazo, P. D. (2003). Sources of inflexibility in 6-year-olds’ understanding of emotion in speech. Child Development, 74(6), 1857–68. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/14669900 [36] Napolitano, A. C., & Sloutsky, V. M. (2004). Is a picture worth a thousand words? The flexible nature of modality dominance in young children. Child Development, 75(6), 1850–1870. [37] Nava, E., & Pavani, F. (2012). Changes in Sensory Dominance During Childhood: Converging Evidence From the Colavita Effect and the Sound-Induced Flash Illusion. Child Development, 00(0), 1–13. doi:10.1111/j.1467-8624.2012.01856.x [38] Nelson, N. L., & Russell, J. a. (2011). Preschoolers’ use of dynamic facial, bodily, and vocal cues to emotion. Journal of Experimental Child Psychology, 110(1), 52–61. doi:10.1016/j.jecp.2011.03.014 [39] Nowicki Jr., S., & Carton, E. (1997). The relation of nonverbal processing ability of faces and voices and children’s feelings of depression and competence. The Journal of Genetic Psychology, 158(3), 357–363. [40] Nowicki Jr., S., & Duke, M. P. (1994). Individual Differences in the Nonverbal Communication of Affect: The Diagnostic Analysis of Nonverbal Accuracy Scale. Journal of Nonverbal Behavior, 18(I), 9–35. [41] Nowicki, S. (2008). The Manual for the Receptive Tests of the Diagnostic Analysis of Nonverbal Accuracy 2 (DANVA2). Atlanta, GA: Department of Psychology, Emory University. [42] Nowicki, S. J., & Mitchell, J. (1998). Accuracy in Identifying Affect in Child and Adult Faces and Voices and Social Competence in Preschool Children. Genetic, Social & General Psychology, 124(1). [43] Pell, M. D. (2005). Nonverbal Emotion Priming: Evidence from the Facial Affect Decision Task. Journal of Nonverbal Behavior, 29(1), 45–73. doi:10.1007/s10919-004-0889-8 [44] Pickett, C.L, Gardner, W.L., & Knowles, M. (2004). Getting a cue: The need to belong and enhanced sensitivity to social cues. Personality and Social Psychology Bulletin, 30, 1095-1107. [45] Quam, C., & Swingley, D. (2012). Development in children’s interpretation of pitch cues to emotion. Child Development, 83(1), 236–250. [46] Riggio, R.E. (2006). Nonverbal skills and abilities. In V. Manusov & M. Patterson (Eds.). Handbook of Nonverbal Communication. Thousand Oaks, CA: Sage Press. [47] Robinson, C. W., & Sloutsky, V. M. (2004). Auditory Dominance and Its Change in the Course of Development. Child Development, 75(5), 1387–1401. [48] Robinson, C. W., & Sloutsky, V. M. (2007). Visual processing speed: effects of auditory input on visual processing. Developmental Science, 10(6), 734–40. doi:10.1111/j.1467-7687.2007.00627.x [49] Russell, J. a, Bachorowski, J.-A., & Fernandez-Dols, J.-M. (2003). Facial and vocal expressions of emotion. Annual Review of Psychology, 54, 329–49. doi:10.1146/annurev.psych.54.101601.145102 [50] Saldana, H. M., & Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgment. Perception & Psychophysics, 54, 406–416. [51] Scherer, K. (2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40(1-2), 227–256. doi:10.1016/S0167-6393(02)00084-5 [52] Scherer, K. R., Banse, R., Wallbott, H. G., & Goldbeck, T. (1991). Vocal cues in emotion encoding and decoding. Motivation and Emotion, 15(2), 123–148. doi:10.1007/BF00995674 [53] Sloutsky, V. M., & Napolitano, A. C. (2003). Is a Picture Worth a Thousand Words? Preference for Auditory Modality in Young Children, 74(3), 822–833. [54] Sobin, C., & Alpert, M. (1999). Emotion in Speech : The Acoustic Attributes of Fear, Anger, Sadness, and Joy. Journal of Psycholinguistic Research, 28(4), 347–365. [55] SuperLab. (2012). SuperLab Version 5. San Pedro, CA: Cedrus Corporation. [56] Thompson, L. A., & Massaro, D. W. (1994). Children’s integration of speech and pointing gestures in comprehension. Journal of Experimental Child Psychology, 57(327-354). [57] Vicari, S., Reilly, J. S., Pasqualetti, P., Vizzotto, a, & Caltagirone, C. (2000). Recognition of facial expressions of emotions in school-age children: the intersection of perceptual and semantic categories. Acta Paediatrica (Oslo, Norway : 1992), 89(7), 836–45. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10943968 [58] Warnes, E.D., Sheridan, S.M., Geske, J., & Warnes, W.A. (2005). A contextual approach to the assessment of social skills: Identifying meaningful behaviors for social competence. Psychology in the Schools, 42(2), 173-187. [59] Waxer, M., & Morton, J. B. (2011). Children’s judgments of emotion from conflicting cues in speech: Why 6-year-olds are so inflexible. Child Development, 82(5), 1648–1660. [60] Widen, S. C., & Russell, J. a. (2008). Children acquire emotion categories gradually. Cognitive Development, 23(2), 291–312. doi:10.1016/j.cogdev.2008.01.002 [61] Wocadlo, C., & Rieger, I. (2006). Social skills and nonverbal decoding of emotions in very preterm children at early school age. European Journal of Developmental Psychology, 3(1), 48–70. [62] Woods, S., Woke, D., Nowicki, S., & Hall, L. (2009). Emotion recognition abilities and empathy of victims of bullying. Child Abuse & Neglect, 33, 307-311. [63] Zupan, B., Neumann, D., Babbage, D. R., & Willer, B. (2009). The importance of vocal affect to bimodal processing of emotion: implications for individuals with traumatic brain injury. Journal of Communication Disorders, 42(1), 1–17. doi:10.1016/j.jcomdis.2008.06.001 [64] Zupan, B., & Sussman, J. E. (2009). Auditory preferences of young children with and without hearing loss for meaningful auditory-visual compound stimuli. Journal of Communication Disorders, 42(6), 381–96. doi:10.1016/j.jcomdis.2009.04.002