Sensing and Handling Engagement Dynamics in Human-Robot Interaction Involving Peripheral Computing Devices


Mingfei Sun      Zhenjie Zhao       Xiaojuan Ma


Abstract
When human partners attend to peripheral computing devices while interacting with conversational robots, the inability of the robots to determine the actual engagement level of the human partners after gaze shift may cause communication breakdown. In this paper, we propose a real-time perception model for robots to estimate human partners' engagement dynamics, and investigate different robot behavior strategies to handle ambiguities in humans' status and ensure the flow of the conversation. In particular, we define four novel types of engagement status and propose a real-time engagement inference model that weighs humans' social signals dynamically according to the involvement of the computing devices. We further design two robot behavior strategies (\emph{explicit} and \emph{implicit}) to help resolve uncertainties in engagement inference and mitigate the impact of uncoupling, based on an annotated human-human interaction video corpus. We conducted a within-subject experiment to assess the efficacy and usefulness of the proposed engagement inference model and behavior strategies. Results show that robots with our engagement model can deliver better service and smoother conversations as an assistant, and people find the implicit strategy more polite and appropriate.
Video
Supplementary materials
Citation
@INPROCEEDINGS{Sun2017,
title = {{Sensing and Handling Engagement Dynamics in Human-Robot Interaction Involving Peripheral Computing Devices}},
year = {2017},
journal = {Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '17},
author = {Sun, Mingfei and Zhao, Zhenjie and Ma, Xiaojuan},
isbn = {978-1-4503-4655-9/17/05},
doi = {http://dx.doi.org/10.1145/3025453.3025469},
keywords = {Human-Robot Interaction; Engagement Awareness; Peripheral Computing Devices; Robot Behaviors.}
}
Thanks

We thank the WeChat-HKUST Joint Laboratory on Artificial Intelligence Technology (WHAT LAB) grant#1516144-0, and NSF CIFellows grant#1019343, for sponsoring this research.