Decoding Children’s Social Behavior

We introduce a new problem domain for activity recog- nition: the analysis of children’s social and communica- tive behaviors based on video and audio data. We specif- ically target interactions between children aged 1–2 years and an adult. Such interactions arise naturally in the di- agnosis and treatment of developmental disorders such as autism. We introduce a new publicly-available dataset con- taining over 160 sessions of a 3–5 minute child-adult inter- action. In each session, the adult examiner followed a semi- structured play interaction protocol which was designed to elicit a broad range of social behaviors. We identify the key technical challenges in analyzing these behaviors, and de- scribe methods for decoding the interactions. We present experimental results that demonstrate the potential of the dataset to drive interesting research questions, and show preliminary results for multi-modal activity recognition.