Expressive Robots

Problem

During interactions with other people, speech is one part of the way we communicate, but a rich multidimensional space of nonverbal expression is also essential. Nuanced nonverbal expression is also important in interactions with robots, but the research community is yet to fully understand this space and harness full potential of nonverbal communication in human-robot interaction (HRI).

Solution

In studies involving robot light signals, sound, and facial expressions, we have begun to investigate human responses to and perceptions of different nonverbal robot signaling. One approach to unlocking the potential of nonverbal robot signaling is to quantify human responses to these nonverbal behaviors and share open-source resources from the explored space. Another approach is to create and validate models that can extend to future HRI work.

Upcoming Events

To help the research community on robot sound (and related areas) come together, we have proposed workshop on Understanding and Harnessing Sound in Robotic Systems, to take place at the ICRA 2022 conference. More information is available here.

People

Publications

• Nnamdi Nwagwu, Adeline Schneider, Ibrahim Syed, Brian J. Zhang, and Naomi T. Fitter, "The Benefits of Sound Resound: An In-Person Replication of the Ability of Character-Like Robot Sound to Improve Perceived Social Warmth," Proceedings of the Robotics Science and Systems (RSS) Conference, Delft, Netherlands, 2024. [BibTeX] [PDF]
• Nnamdi Nwagwu and Naomi T. Fitter, "Pilot Investigation of the Perception of Expressive Motion for the Stretch Robot," Proceedings of the Workshop YOUR Study Design! Participatory Critique and Refinement of Participants Studies Workshop, held in conjunction with HRI, Boulder, CO, USA, 2024. [BibTeX] [PDF]
• Stayce Mockel, Rafael Morales Mayoral, Sean Buchmeier, and Naomi T. Fitter, "An Evaluation of Eye Expression and Gaze Directionality for the Stretch Robot," Proceedings of the Workshop YOUR Study Design! Participatory Critique and Refinement of Participants Studies Workshop, held in conjunction with HRI, Boulder, CO, USA, 2024. [BibTeX] [PDF]
• Stayce Mockel, Rhian C. Preston, Jessica Herring, and Naomi T. Fitter, "A Pilot Investigation of Human Preference for Robot Arm Visual Form," Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boulder, CO, USA, 2024. [BibTeX] [PDF]
• Triniti Armstrong, Nnamdi Nwagwu and Naomi T. Fitter, "Should Assistive Feeding with the Stretch Robot Use Expressive Motion?," Proceedings of the Assistive Robotics for Citizens Workshop, held in conjunction with the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA, 2023. [BibTeX] [PDF]
• Brian J. Zhang, Bastian Orthmann, Ilaria Torre, Roberto Breslin, Jason Fick, Iolanda Leite, and Naomi T. Fitter, "Hearing it Out: Guiding Robot Sound Design through Design Thinking," Proceedings of the IEEE International Conference on Human and Robot Interactive Communication (RO-MAN), Busan, Korea, 2023. [BibTeX] [PDF]
• Brian J. Zhang and Naomi T. Fitter, "Nonverbal Sound in Human-Robot Interaction: A Systematic Review," ACM Transactions on Human-Robot Interaction, 2023. [BibTeX] [PDF]
• Rhian C. Preston, Nisha Raghunath, Christopher A. Sanchez, and Naomi T. Fitter, "“Armed” and Dangerous: How Visual Form Influences Perceptions of Robot Arms," Proceedings of the International Conference on Social Robotics (ICSR), Florence, Italy, 2022. [BibTeX] [PDF]
• Brian J. Zhang, Christopher A. Sanchez, and Naomi T. Fitter, "Using the Price Sensitivity Meter to Measure the Value of Transformative Robot Sound," Proceedings of the IEEE International Conference on Human and Robot Interactive Communication (RO-MAN), Naples, Italy, 2022. [BibTeX] [PDF]
• Brian J. Zhang, Noel Sigafoos, Rabecka Moffit, Ibrahim Syed, Lili Adams, Jason Fick, and Naomi T. Fitter, "SonifyIt: Towards Transformative Sound for All Robots," Robotics and Automation Letters, 2022. [BibTeX] [PDF]
• Ibrahim Syed, Jason Fick, Brian J. Zhang, and Naomi T. Fitter, "Toward Generative Sound Cues for Robots Using Emotive Musification," Proceedings of the International Conference on Auditory Display (ICAD), Virtual, 2022. [BibTeX] [PDF]
• Ibrahim Syed, Brian J. Zhang, Naomi T. Fitter, and Jason Fick, "Towards Generative Musical Cues for Emotive and Responsive Robot Sonfication," Proceedings of the Sound for Robots Workshop at the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022. [BibTeX] [PDF]
• Rhian C. Preston, Nisha Raghunath, and Naomi T. Fitter, "Going out on a Limb: Examining How Form Impacts Human Perception of Robot Arms," Proceedings of the Workshop YOUR Study Design! Participatory Critique and Refinement of Participants Studies Workshop, held in conjunction with HRI, Sapporo, Hokkaido, Japan (remote), 2022. [BibTeX] [PDF]
• Brian J. Zhang, Knut Peterson, Christopher A. Sanchez, and Naomi T. Fitter, "Exploring Consequential Robot Sound: Should We Make Robots Quiet and Kawaii-et?," Paper accepted to the IEEE International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic (remote), 2021. [BibTeX] [PDF]
• Lilian Chan, Brian J. Zhang, and Naomi T. Fitter, "Designing and validating expressive Cozmo behaviors for accurately conveying emotions," Paper accepted to the IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Vancouver, Canada (remote), 2021. [BibTeX] [PDF]
• Brian J. Zhang, Nick Stargu, Samuel Brimhall, Lilian Chan, Jason Fick, and Naomi T. Fitter, "Bringing WALL-E out of the Silver Screen: Understanding How Transformative Robot Sound Affects Human Perception," Paper accepted to the IEEE International Conference on Robotics and Automation (ICRA), Xi'an, China (remote), 2021. [BibTeX] [PDF]
• Brian J. Zhang, Christopher A. Sanchez, and Naomi T. Fitter, "Consequential robot sound: Should robots be quiet and high-pitched?," Proceedings of the Sound in Human-Robot Interaction Workshop, ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boulder, CO, USA (remote), 2021. [BibTeX] [PDF]
• Caitlin Frazee, Brian J. Zhang, and Naomi T. Fitter, "Enabling intentional sound for construction cobots," Proceedings of the 2020 IROS Workshop on Building Construction and Architecture Robotics, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA (remote), 2020. [BibTeX] [PDF]
• Elizabeth Cha, Naomi T. Fitter, Yunkyung Kim, Terry Fong, and Maja Matarić, "Generating expressive light signals for appearance-constrained robots," Proceedings of the International Symposium on Experimental Robotics, Buenos Aires, Argentina, 2018. [BibTeX] [PDF]
• Elizabeth Cha, Naomi T. Fitter, Yunkyung Kim, Terry Fong, and Maja Matarić, "Effects of robot sound on auditory localization in human-robot collaboration," Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), Chicago, IL, USA, 2018. [BibTeX] [PDF]
• Naomi T. Fitter and Katherine J. Kuchenbecker, "Designing and assessing expressive open-source faces for the Baxter robot," Proceedings of the International Conference on Social Robotics (ICSR), Kansas City, MO, USA, 2016. [BibTeX] [PDF]