Research

I my research, I have developed models and metrics for characterizing and assessing human informational needs in human-autonomy interaction settings. These approaches are grounded in concepts from human factors and cognitive psychology. Some of the main contributions of this work are:

• We developed the Situation Awareness Framework for Explainable AI (SAFE-AI), which is a three-level framework for the design and evaluation of explainable AI. A preliminary version of this work was published and presented at EXTRAAMAS 2020, and an extended version was recently published in the International Journal of Human-Computer Interaction.

• We proposed and validated a suite of assessment techniques and metrics for human understanding of reward functions, which are grounded in algorithmic reward learning techniques. We performed this validation by leveraging a Structural Equation Modeling approach from cognitive psychology, and our findings suggest two primary components of reward understanding: feature- and policy-based understanding. This work was published in the Computers in Human Behavior journal in May 2023. 

• Our current work is exploring the connections between human factors constructs and information-theoretic concepts that will enable the automatic generation of explanations that effectively balance human-centered design tradeoffs. 

I have also worked towards developing and evaluating algorithmic approaches that enable bi-directional communication between humans and agents. In this work, we have focused in particular on settings where humans aim to align the objectives of autonomous agents, encoded as reward functions, with their own objectives, which is called value alignment. Some of the main contributions of this work are:

• We performed an extensive human study comparing the performance of a broad range of reward explanation techniques in scenarios of differing reward function complexity and found that providing humans with reward abstractions most effectively balanced support of reward understanding and reasonable mental workload. The initial experiment outline was published and presented at the AI-HRI Fall Symposium in 2021, and the full experimental results were recently published in the Robotics and Automation Letters (RA-L) journal. 

• We are currently exploring explanations of reward models for reinforcement learning from human feedback (RLHF) in the large language model setting (LLM). To this end, we are developing techniques for human-AI co-design of abstract explanations to enable effective explainability, but also more efficient algorithmic generalization.

Since 2018, I have been a member of MIT's Work of the Future Task Force. As part of this work, I have visited over 50 factories across the United States and worldwide with interdisciplinary research teams in order to study how and if robots are being adopted in manufacturing and the impacts of new technologies on work. Some highlights from this work include:

• We published findings related to emerging technologies, challenges and barriers to robot adoption, and key robotics research directions which were curated from our interviews of robot manufacturers, integrators, and OEMs throughout Europe in the Foundations and Trends in Robotics journal. This work was also featured as one of the Task Force's research briefs.

• The findings of our highly-interdisciplinary "Ohio Research Team" from the dozens of interviews we performed throughout Ohio, Arizona, and Pennsylvania were featured in another Task Force research brief. We also presented these findings in two panels at the 2021 Industry Studies Association conference.

• Based on our findings from this work, we organized a workshop on the Accessibility of Robot Programming and the Work of the Future at the Robotics: Science and Systems conference in 2021. This workshop featured interdisciplinary perspectives, and paper contributors also piloted the Ethical Computing Protocol developed at MIT.

• I discussed the findings from our work on a global policy dialogue panel on the Future of Work, which was hosted by the UN Department of Economic and Social Affairs.

• Together with our colleagues from the Fraunhofer Institute in Germany, we published our findings from a study of small- and medium-sized manufacturers in Germany at the Conference on Production Systems and Logistics.

• The work of the Ohio Team was featured in Wired, and the findings from the European manufacturing study were highlighted in The Robot Report.