I am currently a researcher for the ERATO Ishiguro Symbiotic Human-Robot Interaction project working in Kyoto University. In this project I create engaging conversations with ERICA through her dialog using gaze behavior, speech and gesture.

ERICA uses a spherical microphone array and a Kinect sensor to track users, recognize their dialog and evaluate their attention, eye gaze and other non-verbal behaviors. The user can converse without a hand-held microphone, leading to more natural non-verbal behavior during conversation. We have perfomed several demonstrations of ERICA's conversational abilities, both in a live setting and remotely.

Virtual Basketball

For my doctoral studies I created a virtual basketball game to analyze interactions between human users and agent players. My system uses a Kinect sensor, immersive displays and a foot pressure sensor to allow the user to play basketball without the need for a keyboard or mouse. I created a gesture recognition system and integrated speech recognition for multi-modal play.

I used virtual basketball to examine various aspects of human-agent interaction such as:

  • Application of real-world theories to virtual world interactions
  • Analysis of joint action behaviors between people in the virtual world
  • Evaluation of agent body communication
  • Multimodal analysis, particularly speech + body movement

My major research deals with theory, design, implementation and evaluation of virtual agents and robots. I have been a part of several other projects including designing cultural characters, virtual and robot telepresence and eye gaze tracking. Please contact me if you are interested in collaboration.