Grounded Action Segmentation and Association using Constrained Motif Discovery (2009)
Sponsor: Global Center of Excellence (GCOE) Young Researcher Fostering Program
Responsibility: Main Researcher
Duration: The projects started November 2008 and finished March 2009
Goals
To build and evaluate an unsupervised algorithm for learning interactive behavior both in implicit and explicit nonverbal communication settings using a novel technique that combines constrained motif discovery and probabilistic networks.
Scenario
The scenario used to evaluate the algorithm is a dyadic interaction in which an operator is commanding an actor to achieve some task that requires unknown number of actions to be done. The operator is using free hand gestures for communicating with the actor.
Approach
The proposed system can learn interactive behavior in three stages:
- Discovery Phase: Unsupervised discovery of the gestures and their associated actions using unsupervised multidimensional constrained motif discovery.
- Association Phase: Association of the discovered gestures and the discovered actions using Baysian Networks.
- Controller Generation Phase: The learned model is automatically converted into a controller program that can control an agent or a robot to behave according to it.
A proof of concept experiment was already conducted and the proposed system was able to discover six out of seven gestures, all seven actions, 83.5% of gesture occurrences and 87.5% of action occurrences. The final system was able to predict the actor's behavior with accuracy 95.2% on a separate test set.
Exhibitions
Kyoto University ICT Innovation Fair 2009. Kyoto University, Clock Tower, February 19th 2009
Publications
- Yasser Mohammad, Toyoaki Nishida, and Okada Shogo, Unsupervised Simultaneous Learning of Gestures, Actions and their Associations for Human-Robot Interaction, accepted for IEEE IROS 2009 (October 11 to 15, St. Louis, MO, USA), 2009
- Yasser Mohammad and Toyoaki Nishida, Mining Causal Relationships in Multidimensional Time Series, Smart Information and Knowledge Management: Advances, Challenges and Critical Issues, Studies in Computational Intelligence, Springer, edited by Edward Szczerbicki and Ngoc Thanh Nguyen, 2010
Natural Listening for a Humanoid Robot (2008)
Sponsor: Global Center of Excellence (GCOE) Young Researcher Fostering Program
Responsibility: Main Researcher
Duration: The projects started December 2007 and Finished March 2008
Goals
1. Building a human-human corpus of interactions fully annotated with motion tracking data that can serve as a training set for the robot (and can be used for other training purposes as well).
2. Finding an objective measure of how "natural" and interaction is based on processing motion and speech signals. This objective measure will be used in online training of the robot as well as serving as an objective measure of future research in the subject
3. Designing and implementing a learning system that can help the robot achieve natural listening behavior in an automatic way.
Experiments
A controlled experiment was conducted in order to achieve the goals of this project in which 32 untrained humans participated. A full corpus of audio/video data as well as motion captured data and physiological data is currently under construction using the information collected in this experiment.
Exhibitions
Kyoto University ICT Innovation Fair 2008. Kyoto University, Clock Tower, February 19th 2008
Publications
- Yasser Mohammad and Toyoaki Nishida, Reactive Gaze Control for a Humanoid Robot, In Proceedings of IEEE International Conference on Robotics and Mechatronics (RAM 2008), September 2008, China, pp 47:54.
- Yasser Mohammad and Toyoaki Nishida, Natural Listening Robot for AAL Applications, In Proceedings of the first International Workshop on IUI for Ambient Assisted Living (IUI4AAL 2008) in conjunction with the IUI 2008 conference, Canari Islands, Spain.
- Yasser Mohammad and Toyoaki Nishida, Natural Listening for a Humanoid Robot, In Proceedings of the International Conference on Informatics Education and Research for Knowledge-Circulating Society 2008 (ICKS 2008), IEEE.