Abstract
A model of the development of coordinated processing of auditory and visual information is proposed and a distinction made between two types of cross-modal relationship. A structural relationship exists when the information in the first modality has a one-to-one correspondence with the information in the second, as for example between the visual and tactual features of an object's shape. An associative relationship on the other hand exists where there is only a probabilistic correspondence between the information from each modality. For example, with respect to voice pitch and facial features, high pitch is normally characteristic of female faces and low pitch of male faces. It is hypothesised that an infant's initial perceptual organisation is one where vision and audition are processed independently, but concurrently. However both modalities by and large operate on information from the same objects and events. It is argued that infants are predisposed to finding links and commonalities in sensory information from different sources and that they quickly move to a position of representing the common or amodal features. That is the infant differentiates the structural relationships between the modalities. As a result of increasing multi-modal representation of the environment the infant subsequently comes to incorporate the associative relationships that also hold for object properties. Four experiments are described, involving infants from three to twelve months of age, which address different aspects of this model. Experiments 1 and 2 investigate the processing of spatially related auditory and visual information in a bistimulus environment. The results show quite clearly that processing of the auditory information is affected by simultaneously presented visual information. However, there was no evidence of associative integration. These two experiments also investigated selective auditory attention and demonstrated that from at least six months, human infants can attend to two simultaneously presented auditory stimuli. Experiment 3 aIso dealt with associative dependence, but used naturally occurring relationships, that is those of age and gender in faces and voices. Once more there was no evidence for selective visual attention dependent upon the auditory stimuli, that is no associative integration. A subsidiary experiment carried out with a small group of infants exposed to parental faces and voices, again showed no evidence of integration. Experiments 4 and 5 manipulated structural relationships between vision and audition - that of lip movements and speech. There was no doubt that infants from three months of age were responsive to the co-occurrence of facial movements and voice production. The results supported the hypothesis that bimodal information was being utilised in an integrated fashion. Overall results from all four experiments provided partial support for a structural-associative hierarchical model of the development of sight-sound coordination.