Workshop on Visual Perception for Navigation in Human Environments

The JackRabbot Social Robotics Dataset and Benchmark

Workshop Goals: Dataset and Challenges

This workshop focuses on the unique perceptual problems related to autonomous navigation in indoor and outdoor human environments. These problems include human detection and tracking from 2D and/or 3D data, human posture detection and prediction, object detection, segmentation, trajectory forecasting and any other perceptual task that, when solved, provides valuable information to autonomous agents and robots that need to navigate safely in human crowded environments.

Last years the community has paid increasing attention to these perceptual problems and has provided annotated datasets to learn to solve them and evaluate them, yet restricted to the point of view of autonomous vehicles. Robots and other navigating agents, however, view the world from a different perspective, can go indoors, get closer to humans and need to perceive other types of information. In this workshop, we present the JackRabbot social navigation dataset , a novel annotated dataset with the signals from our mobile manipulator JackRabbot. For the workshop we launch the first set of visual challenges, to benchmark perceptual algorithms in this new type of data.

The dataset contains 67 minutes of ground truth annotated sensor data acquired from the JackRabbot mobile manipulator and includes more than 50 indoor and outdoor sequences in a university campus environment. Sensor data includes a stereo RGB 360° cylindrical video stream, 3D point clouds from two LiDAR sensors, audio and GPS positions.

In the workshop we will present the winners of the first challenge. We have also as invited speakers world renowned expers in perception for autonomous navigation. We aim to foster discussion between the attendants to find useful synergies and applications of the solutions of perceptual tasks for navigation in human environments.

Dataset

In this first phase, the annotations of the JackRobot Dataset include:

  • 2D bounding box annotations around pedestrians, cyclist and vehicles in five RGB streams and in their cylindrical composition.
  • 3D oriented cuboid annotations around pedestrians, cyclist, vehicles in two Velodyne-16 LiDAR streams
  • 2D-3D associations between bounding boxes and cuboids
  • Time consistent trajectories (tracks) for all annotated objects in both 2D and 3D
  • Open Challenge

    We launch in this workshop a first challenge on our annotated data in the following categories:
  • 2D-3D person detection
  • 2D-3D person tracking
  • The winner of each challenge will be given an opportunity to present their work as oral peresentation during the workshop.

    Call for Papers

    We invite researchers to submit their papers addressing topics related to autonomous (robot) navigation in human environments. Relevant topics include:

  • 2D or 3D object detection and pose estimation, tracking and forecasting
  • 2D or 3D pedestrian and vehicle detection, tracking and forecasting
  • 2D or 3D human skeleton pose estimation, tracking and forecasting
  • 2D or 3D human face detection
  • Human emotion recognition
  • Human walking behaviour analysis
  • 2D and 3D semantic, instance or panoptic segmentation
  • Motion prediction and social models
  • Individual, group and social activity recognition
  • Abnormal activity recognition
  • Visual and social navigation in crowded scenes
  • Traversability estimation
  • Dataset proposals and bias analysis
  • New metrics and performance measure for different visual perception problems related to autonomous navigation
  • Submisison format should match the ICCV format, e.g. 8 pages, double column format. By submitting to this workshop, authors agree to the review process and understand that we will do our best to match papers to the best possible reviewers. The reviewing process is double blind. Submission to the challenge is independent from the paper submission, but we encourage paper authors to submit to one of the challenges.

    Submissions can be made here. If you have any questions about submitting, please contact us here.

    Program

    Start Time
    End Time
    Description
    8:30 AM
    8:45 AM
    Introduction
    8:45 AM
    9:15 AM
    Invited Talk: Konrad Schindler, ETH Zurich - A Tale of Two Benchmarks
    9:15 AM
    9:45 AM
    Invited Talk: Raquel Urtasun, University of Toronto & Uber ATG - TBA
    9:45 AM
    10:15 AM
    The JackRabbot Dataset and Challenge
    10:15 AM
    10:45 AM
    Coffee Break
    10:45 AM
    11:15 AM
    Invited Talk: Michael Black, Max Planck Institute for Intelligent Systems & Amazon - Estimating 3D people interacting with 3D scenes
    11:15 AM
    11:45 AM
    Invited Talk: Bernt Schiele, Max Planck Institute for Informatics - 3D Human Pose Estimation and Model Stealing
    11:45 AM
    12:15 AM
    Invited Talk: Alexandre Alahi, EPFL - The 3P's of Mobility: Perceiving, Predicting, and Planning
    12:15 PM
    12:30 PM
    Discussion and Closing Remarks

    Invited Speakers

    Konrad Schindler

    Konrad Schindler

    Raquel Urtasun

    Raquel Urtasun

    Michael Black

    Michael Black

    Bernt Schiele

    Bernt Schiele

    Alexandre Ahi

    Alexandre Alahi

    Program Committee

    Name
    Organization
    Afshin Dehghan
    Apple, Inc.
    Alexandre Alahi
    EPFL
    Amir A. Sadeghian
    AiBee, Inc.
    Anthony Dick
    University of Adelaide
    Chunzhao Guo
    Toyota Research Institute
    Ehsan Adeli
    Stanford University
    Fatemeh Sadat
    Australian National University
    Jingwei Ji
    Stanford University
    Juan Carlos Niebles
    Stanford University
    JunYoung Gwak
    Stanford University
    Kiyosumi Kidono
    Toyota Central R&D Labs, Inc.
    Laura Leal-Taixé
    Technical University of Munich
    Niko Suenderhauf
    Queensland University of Technology
    Noriaki Hirose
    Toyota Central R&D Labs, Inc.
    Qinfeng Shi
    University of Adelaide
    Shun Taguchi
    Toyota Central R&D Labs, Inc.
    Shyamal Buch
    Stanford University
    Noriaki Hirose
    Toyota Central
    Stephen Gould
    Australian National University
    Wongun Choi
    AiBee, Inc.

    Organizers

    Hamid Rezatofighi

    Hamid Rezatofighi

    Roberto Martín-Martín

    Roberto Martín-Martín

    Ian Reid

    Ian Reid

    Silvio Savarese

    Silvio Savarese

    Sponsors