Hi! I am a PhD candidate in Computer Science and Engineering at the University of Michigan, where I work with Dr. Joyce Chai as a member of the Situated Language and Embodied Dialogue (SLED) Lab. My research focuses on language understanding, cognitive development, and embodied agents, specifically, how robots can take advantage of their ability to interact with the world around them to learn language during spoken interaction with others.

My recent work evaluates whether multimodal LLM agents can recognize and mitigate safety hazards while completing household tasks (SafetyALFRED, Findings of ACL 2026), and builds conversational robots that help people living with cognitive decline reflect on their resilience (under review at HRI 2027). I am currently studying how safety information travels from vision-language planners to vision-language-action controllers, using mechanistic interpretability to find where the safety signal lives and how to amplify it.

Prior to joining the University of Michigan, I graduated from Boise State University with a B.S. in Computer Science (Magna Cum Laude, Minor in Applied Mathematics), where during my last two years I performed research advised by Dr. Casey Kennington as a member of the Speech Language and Interactive Machines (SLIM) Lab. I am supported by the NSF Graduate Research Fellowship and the Gemini Academic Program.

🔥 News

📝 Publications

Robot to Elicit Resilience Reflection for Individuals Living with Cognitive Decline
Long-Jing Hsu*, Josue Torres-Fonseca*, Janice Bays, Joyce Chai, and Patricia Alves-Oliveira
Under review at the ACM/IEEE International Conference on Human-Robot Interaction (HRI) 2027 · *Equal contribution
ACL 2026 Findings
SafetyALFRED
SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
Josue Torres-Fonseca, Naihao Deng, Yinpei Dai, Shane Storks, Yichi Zhang, Rada Mihalcea, Casey Kennington, and Joyce Chai
Findings of the Association for Computational Linguistics: ACL 2026 · San Diego, CA
ICML 2026
The Mechanistic Emergence of Symbol Grounding in Language Models
The Mechanistic Emergence of Symbol Grounding in Language Models
Shuyu Wu, Ziqiao Ma, Xiaoxi Luo, Yidong Huang, Josue Torres-Fonseca, Freda Shi, and Joyce Chai
International Conference on Machine Learning (ICML) 2026 · Seoul, South Korea
SIGdial 2025
rrSDS 2.0
rrSDS 2.0: Incremental, Modular, Distributed, Multimodal Spoken Dialogue with Robotic Platforms
Anna Manaseryan, Porter Rigby, Brooke Matthews, Catherine Henry, Josue Torres-Fonseca, Ryan Whetten, Enoch Levandovsky, and Casey Kennington
Special Interest Group on Discourse and Dialogue (SIGdial) 2025 · Avignon, France
SIGdial 2022
Symbol and Communicative Grounding through Object Permanence with a Mobile Robot
Symbol and Communicative Grounding through Object Permanence with a Mobile Robot
Josue Torres-Fonseca, Catherine Henry, and Casey Kennington
Special Interest Group on Discourse and Dialogue (SIGdial) 2022 · Edinburgh, UK
LREC 2022
HADREB
HADREB: Human Appraisals and (English) Descriptions of Robot Emotional Behaviors
Josue Torres-Fonseca and Casey Kennington
Language Resources and Evaluation Conference (LREC) 2022 · Marseille, France

📖 Education

  • Aug 2023 – May 2028 (expected), Ph.D. in Computer Science and Engineering, University of Michigan · GPA 4.0
  • Aug 2023 – May 2025, M.S. in Computer Science and Engineering, University of Michigan · GPA 4.0
  • Fall 2018 – May 2023, B.S. in Computer Science, Minor in Applied Mathematics, Boise State University · Magna Cum Laude

💻 Research Experience

Ph.D. Candidate, SLED Lab, University of Michigan  ·  Aug 2023 – Present
Advised by Dr. Joyce Chai

  • Safety signals in VLM → VLA robot pipelines (ongoing): Building SafetyBehavior, an OmniGibson-based benchmark where robots must recognize and mitigate hazards from real injury data (NEISS) while completing BEHAVIOR-1K tasks, evaluating high-level planning (e.g., “wash the dishes”) and low-level control together in hierarchical and unified VLM-VLA systems. Using mechanistic interpretability to trace where the safety signal is lost and how to amplify it.
  • ReMI (HRI 2027, under review): A conversational robot, co-designed with dementia experts, that helps people living with cognitive decline reflect on their resilience. Now being extended into a multi-week in-home study.
  • SafetyALFRED (Findings of ACL 2026): A benchmark exposing a gap between recognizing and acting: multimodal LLMs identify kitchen hazards in question answering up to 92% of the time, but fix them less than 60% of the time as embodied agents.

Undergraduate Researcher, SLIM Lab, Boise State University  ·  Aug 2021 – Aug 2023
Advised by Dr. Casey Kennington

  • Object permanence for grounded language learning (SIGdial 2022): Studied how object permanence helps a robot learn language, giving a Cozmo robot memory of the objects it has seen and where. In a 24-participant study, this raised its accuracy answering questions about learned objects from 70.0% to 82.7%.
  • HADREB (LREC 2022): Built a dataset of over 1,000 robot emotional behaviors described and rated by 32 participants.

Undergraduate Researcher, Language & Intelligence Group, MIT  ·  May 2022 – Nov 2022
Advised by Dr. Jacob Andreas · MIT Summer Research Program

  • Built an interpretable model that annotates neurons without data.

Undergraduate Researcher, Intelligent Security Group, Texas State University  ·  Jun 2020 – Aug 2020
Advised by Dr. Mina Guirguis · NSF REU

  • Used game theory to secure a simulated water tank.

🏢 Industry Experience

Software Engineer Intern, Paylocity  ·  May 2021 – Aug 2021

  • Built and shipped full-stack features (React/Redux, C#/.NET, SQL) on a 13-person Agile team.

💬 Presentations

  • 2026.07, SafetyALFRED, poster at ACL 2026, San Diego Poster
  • 2022.09, Symbol and Communicative Grounding through Object Permanence, talk and poster at SIGdial 2022, Edinburgh Video Slides Poster
  • 2022.08, Interpretable Neuron Annotations Using No Data, video and poster at the MIT Summer Research Program Video Poster
  • 2022.06, HADREB, video presentation at LREC 2022, Marseille Slides

🎖 Honors and Awards

  • Spring 2026 – Spring 2027, Gemini Academic Program Award ($10,000)
  • Fall 2023 – Spring 2027, NSF Graduate Research Fellowship ($159,000)
  • Spring 2023, Boise State University Top Ten Scholar
  • Fall 2022 – Spring 2023, Barry M. Goldwater Scholarship ($7,500)
  • Fall 2018 – Spring 2023, Dean’s List, Boise State University
  • Fall 2018, Federal Employee Education & Assistance Fund Scholarship ($1,000)

🧑‍🏫 Teaching

  • Jan 2023 – May 2023, Tutor, Operating Systems, Boise State University
  • Jan 2021 – May 2021, Tutor, Introduction to Theory of Computation, Boise State University
  • Aug 2020 – Dec 2020, Tutor, Data Structures, Boise State University
  • Aug 2020 – Dec 2020, Lab Assistant, Microprocessors, Boise State University

👔 Service & Leadership

  • Fall 2024 – Spring 2025, Treasurer, Computer Science and Engineering Graduate Students, University of Michigan
  • Fall 2024 – Spring 2025, President, Artificial Intelligence Club
  • Fall 2019 – Spring 2022, President, Competitive Cybersecurity and Cyber-Physical Club, Boise State University