Minds live in bodies, and bodies move through a changing world. The goal of embodied artificial intelligence is to create agents, such as robots, which learn to creatively solve challenging tasks requiring interaction with the environment. While this is a tall order, fantastic advances in deep learning and the increasing availability of large datasets like ImageNet have enabled superhuman performance on a variety of AI tasks previously thought intractable. Computer vision, speech recognition and natural language processing have experienced transformative revolutions at passive input-output tasks like language translation and image processing, and reinforcement learning has similarly achieved world-class performance at interactive tasks like games. These advances have supercharged embodied AI, enabling a growing collection of researchers to make rapid progress towards intelligent agents which can:

• See: perceive their environment through vision or other senses.
• Talk: hold a natural language dialog grounded in their environment.
• Listen: understand and react to audio input anywhere in a scene.
• Act: navigate and interact with their environment to accomplish goals.
• Reason: consider and plan for the long-term consequences of their actions.

The goal of the Embodied AI workshop is to bring together researchers from computer vision, language, graphics, and robotics to share and discuss the latest advances in embodied intelligent agents. The overarching theme of this year's workshop is Open World Embodied AI: Being an embodied agent in a world that contains objects and concepts unseen during training. This theme applies the “open set” problem of many individual tasks to embodied AI as a whole. We feel that truly effective embodied AI agents should be able to deal with tasks, objects, and situations markedly different from those that they have been trained on. This umbrella theme is divided into three topics:

• Embodied Mobile Manipulation We go places to do things, and to do things we have to go places. Many interesting embodied tasks combine manipulation and navigation to solve problems that cannot be done with either manipulation or navigation alone. This builds on embodied navigation and manipulation topics from previous years and makes them more challenging.
• Generative AI for Embodied AI Generative AI isn't just a hot topic, it's an important tool researchers are using to support embodied artificial intelligence research. Topics such as generative AI for simulation, generative AI for data generation, and generative AI for policies (e.g., diffusion policies and world models) are of great interest.
• Language Model Planning When we go somewhere to do something we do it for a purpose. Language model planning uses large language models (LLMs), vision-language models (VLMs), and multimodal foundation models to turn arbitrary language commands into plans and sequences for action - a key feature needed to make embodied artificial intelligence systems useful for performing the tasks in open worlds.

• Workshop Talks: 8:50AM-5:30PM PT - TBD
• Poster Session: 1:00PM-2:00PM PT - TBD

Ask questions on Slack

In association with the Embodied AI Workshop, our partners and sponsors will present demos, date and times TBD.

The Embodied AI 2024 workshop is hosting many exciting challenges covering a wide range of topics such as rearrangement, visual navigation, vision-and-language, and audio-visual navigation. More details regarding data, submission instructions, and timelines can be found on the individual challenge websites.

The workshop organizers will award each first-prize challenge winner a cash prize, sponsored by Logical Robotics and our other sponsors.

Challenge winners may be given the opportunity to present during their challenge's presentation at the the workshop. Since many challenges can be grouped into similar tasks, we encourage participants to submit models to more than 1 challenge. The table below describes, compares, and links each challenge.

We invite high-quality 2-page extended abstracts on embodied AI, especially in areas relevant to the themes of this year's workshop:

• Open-World AI for Embodied AI
• Generative AI for Embodied AI
• Embodied Mobile Manipulation
• Language Model Planning

• Simulation Environments
• Visual Navigation
• Rearrangement
• Embodied Question Answering
• Embodied Vision & Language

The submission deadline is May 4th (Anywhere on Earth). Papers should be no longer than 2 pages (excluding references) and styled in the CVPR format.

• Paper submissions are now OPEN through May 4th, 2024

Note. The order of the papers is randomized each time the page is refreshed.

The Embodied AI 2024 Workshop is sponsored by the following organizations: