Personalization in Long-Term Human-Robot Interaction

Today I attended Bahar Irfan‘s webinar “Personalization in Long-Term Human-Robot Interaction: Challenges and Suggestions”. She is the latest speaker of Softbank Robotics’ HEART Tech Talks. The talks are free and open to anyone who registers, and I commend Softbank for doing so.

Bahar covered four robot personalization projects she worked on during her PhD program.

Multi-modal Incremental Bayesian Network With Online Learning

Two papers were mentioned but only the 2018 paper is available, the latest still under review. This project explores the use of a multi-modal Bayesian network to improve person recognition accuracy by fusing evidence from multiple sources: face recognition (as the primary biometric) and soft biometrics (user’s gender, age, height, time of interaction). [Nitpick: I would treat “time of interaction” as contextual information instead of biometrics.] Here’s how it works:

When a user comes into the robot’s view, the robot identifies who the user is based on current sensor information combined with the Bayesian node weights. It choses the identity with the highest probability.

P( I=i | F=f, G=g, A=a, ...)

  where i = { "John", "Maria", ... }
Robot predicts the person standing on the left is John based on Face similarity, Gender and Age sensor data.

Next, it learns from that encounter by asking the person to confirm his/her identity. It uses the response along with the current sensor information to update the BN’s weights. Here is an example of the update for the Face similarity parameter:

When tested on a dataset consisting of 14 users in the Face Recognition database, they obtained a 4.4% improvement in accuracy vs. using only face recognition. However using the multi-modal BN fusion came with a cost: it performed better in identifying new users, but decreased in performance for known users.

I liked the idea of using Bayesian networks to combine evidence, but I was surprised that this technique yielded only a small gain in recognition accuracy. Perhaps it because the Face Recognition on its own already achieves high accuracy (82-93%, though I couldn’t find this in the paper) and the soft biometrics are adding very little new information.

Slide (title) User Recognition in the Real World
Slide 7 on presentation

Other Projects

Bahar described a project using a Pepper robot as a barista to take customers’ orders in a cafe. The robot identifies the customers and streamlines the conversation by asking if they want to order their “favorite” (inferred for past orders). Cool in concept but users were frustrated by the robot’s poor speech recognition accuracy. She also described her intership project at Disney Imagineering R&D adapting a robots interaction to users’ emotions, and another on using assistive robots to motivate cardiac patients to improve their rehab exercise on a threadmill.

It was a very enjoyable talk and I learned a lot. I plan to watch more future HEART Tech Talks.

Leave a Reply

Your email address will not be published. Required fields are marked *