Stretching IoT Devices to Understand Human Interactions and Relations

Abstract: In this talk, we discuss the potential of stretching smart devices toward understand and helping human interactions and their relations. We first propose Socio-Phone, a novel initiative to build a mobile platform for face-to-face interaction monitoring. Face-to-face interaction, especially conversation, is a fundamental part of everyday life. Useful contexts to capture and support face-to-face interactions need to be explored more deeply. More important, recognizing delicate conversational contexts requires solving a number of technical challenges. As a first step to address such challenges, we identify useful meta-linguistic contexts of conversation, such as turn-takings, prosodic features, a dominant participant, and pace. These serve as cornerstones for building a variety of interaction-aware applications. The platform efficiently monitors registered contexts during in-progress conversations and notifies applications on-the-fly. Importantly, we have noticed that online turn monitoring is the basic building block for extracting diverse meta-linguistic contexts.

We deepen the study toward various real-world cases. A specific one is to reinforce everyday parent-child conversation with therapeutic implications for children with language delays. Language delay is a developmental problem of children who do not acquire language as expected for their chronological ages. Without timely intervention, language delay can act as a lifelong risk factor. We propose TalkBetter, an in-situ intervention service to help parents of children with language delays in daily parent-child conversation. Through extensive field studies with speech-language pathologists and parents, we report the multilateral motivations and implications of TalkBetter. We present our development of TalkBetter prototype and report its performance evaluation.

Bio: Junehwa Song is a Professor/KAIST-Chair Professor at the Department of Computer Science, KAIST, Korea. Before joining KAIST, he worked as a RSM at IBM T.J. Watson Research, Hawthorne, NY. He received his Ph.D in Computer Science from University of Maryland, College Park, NY from Professor Raymond Miller, also a former RSM at IBM Research. Junehwa has been working on mobile and ubiquitous systems, Internet and distributed systems, and social computing as well as multi-media systems. He served ACM SenSys 2015 as the general chair, and UbiComp 2014 as a TPC co-chair. He has also been on the editorial board of IEEE Transactions on Mobile Computing and IEEE Pervasive Computing.