#

Overview

Within the last decade, advances in deep learning, coupled with the creation of large, freely available datasets (e.g., ImageNet), have resulted in remarkable progress in the computer vision, NLP, and broader AI communities. This progress has enabled models to begin to obtain superhuman performance on a wide variety of passive tasks. However, this progress has also enabled a paradigm shift that a growing collection of researchers take aim at: the creation of an embodied agent (e.g., a robot) which learns, through interaction and exploration, to creatively solve challenging tasks within its environment.

The goal of this workshop is to bring together researchers from the fields of computer vision, language, graphics, and robotics to share and discuss the current state of intelligent agents that can:

See: perceive their environment through vision or other senses.
Talk: hold a natural language dialog grounded in their environment.
Listen: understand and react to audio input anywhere in a scene.
Act: navigate and interact with their environment to accomplish goals.
Reason: consider and plan for the long-term consequences of their actions.

The Embodied AI 2022 workshop will be held in conjunction with CVPR 2022. It will feature a host of invited talks covering a variety of topics in Embodied AI, many exciting challenges, a poster session, and panel discussions.

#

Timeline

CVPR Workshop

Room 224, New Orleans Ernest M. Morial Conventinon Center
June 19, 2022
9:00 AM - 5:30 PM CT
Tentative Schedule:

Workshop Introduction
9:00 AM CT
Navigation & Understanding Challenge Presentations
(MultiON, SoundSpaces, RxR-Habitat, RVSU)
9:10 AM CT
Navigation & Understanding Challenge Q&A Panel
(MultiON, SoundSpaces, RxR-Habitat, RVSU)
10:00 AM CT
Ask questions on Slack
Invited Talk
Carolina Parada
Google AI
10:30 AM CT
Ask questions on Slack
Invited Talk
Roozbeh Mottaghi
Allen Institute for AI
11:00 AM CT
Ask questions on Slack
Invited Talk
Dhruv Batra
GaTech
FAIR
11:30 AM CT
Ask questions on Slack
Accepted Papers Poster Session
12:00 PM CT
Invited Talk
Katerina Fragkiadaki
Carnegie Mellon
1:30 PM CT
Ask questions on Slack
Invited Talk
Fei-Fei Li
Stanford
2:00 PM CT
Ask questions on Slack
Invited Talk
Jitendra Malik
Berkeley
2:30 PM CT
Ask questions on Slack
Interaction Challenge Presentations
AI2-Rearrangement, ALFRED, TEACh
3:00 PM CT
Interaction Challenge Q&A Panel
4:00 PM CT
Ask questions on Slack
Invited Speaker Panel
4:30 PM CT
Ask questions on Slack
Workshop Concludes
5:30 PM CT

Challenge Submission Deadlines

May 2022. Check each challenge for the specific date.

Paper Submission Deadline

May 16, 2022 (Anywhere on Earth)

Workshop Announced

Feb 14, 2022

#

Challenges

The Embodied AI 2022 workshop is hosting many exciting challenges covering a wide range of topics such as rearrangement, visual navigation, vision-and-language, and audio-visual navigation. More details regarding data, submission instructions, and timelines can be found on the individual challenge websites.

Challenge winners will be given the opportunity to present a talk at the workshop. Since many challenges can be grouped into similar tasks, we encourage participants to submit models to more than 1 challenge. The table below describes, compares, and links each challenge.

Challenge	Task	Interactive Actions?	Simulation Platform	Scene Dataset	Observations	Stochastic Acuation?	Action Space


AI2-THOR Rearrangement	Rearrangement	✓	AI2-THOR	iTHOR	RGB-D, Localization		Discrete
ALFRED	Vision-and-Language Interaction	✓	AI2-THOR	iTHOR	RGB		Discrete
Habitat	ObjectNav		Habitat	Matterport3D	RGB-D, Localization		Discrete
iGibson	Interactive Navigation	✓	iGibson	iGibson	RGB-D	✓	Continuous
iGibson	Social Navigation	✓	iGibson	iGibson	RGB-D	✓	Continuous
MultiON	Multi-Object Navigation		Habitat	Matterport3D	RGB-D, Localization		Discrete
Robotic Vision Scene Understanding	Rearrangement (SCD)		Isaac Sim	Active Scene Understanding	RGB-D, Pose Data, Flatscan Laser	✓	Discrete
Robotic Vision Scene Understanding	Semantic SLAM		Isaac Sim	Active Scene Understanding	RGB-D, Pose Data, Flatscan Laser	Partially	Discrete
RxR-Habitat	Vision-and-Language Navigation		Habitat	Matterport3D	RGB-D		Discrete
SoundSpaces	Audio Visual Navigation		Habitat	Matterport3D	RGB-D, Audio Waveform		Discrete
TDW-Transport	Rearrangement	✓	TDW	TDW	RGB-D, Metadata	✓	Discrete
TEACh	Vision-and-Dialogue Interaction	✓	AI2-THOR	iTHOR	RGB		Discrete, Text Generation

#

Call for Papers

We invite high-quality 2-page extended abstracts in relevant areas, such as:

Simulation Environments
Visual Navigation
Rearrangement
Embodied Question Answering
Simulation-to-Real Transfer
Embodied Vision & Language

Accepted papers will be presented as posters or spotlight talks at the workshop. These papers will be made publicly available in a non-archival format, allowing future submission to archival journals or conferences. Paper submissions do not have to be anononymized. Per CVPR rules regarding workshop papers, at least one author must register for CVPR using an in-person registration.

Submission

The submission deadline is May 16th (Anywhere on Earth). Papers should be no longer than 2 pages (excluding references) and styled in the CVPR format. Paper submissions are now closed.

Accepted Papers

Note. The order of the papers is randomized each time the page is refreshed.

FedVLN: Privacy-preserving Federated Vision-and-Language Navigation

Kaiwen Zhou, Xin Eric Wang

Data privacy is a central problem for embodied agents that can perceive the environment, communicate with humans, and act in the real world. [Expand]

#

Overview

#

Timeline

#

Challenges

#

Call for Papers

#

Organizers