INTRODUCING
Aria Digital Twin Dataset
A real-world dataset, with hyper-accurate digital counterpart & comprehensive ground-truth annotation
WHAT IS IT?
An egocentric dataset with extensive and accurate ground-truth
Aria Digital Twin is an egocentric dataset captured using Aria glasses, with extensive simulated ground truth for devices, objects and environment.
This dataset sets a new standard for egocentric machine perception research, and accelerates research into a number of challenges including 3D object detection and tracking, scene reconstruction and understanding, sim-to-real learning, human pose prediction, while also inspiring new machine perception tasks for augmented reality (AR) applications.
DOWNLOAD THE DATASET BELOW
Dataset Content
- 200 sequences (~400 mins)
- 398 objects (324 stationary, 74 dynamic)
- 2 real indoor scenes
- Single + multi-user activities
Sensor Data per device
- 2 x outward-facing monochrome camera streams
- 1 x outward-facing RGB camera stream
- 2 x IMU streams
- 2 x Internal-facing eye tracking cameras
- Complete sensor calibrations
Annotations
- 6DoF device trajectory
- 3D object pose
- 3D human skeleton
- 3D eye gaze
- 2D Photo-realistic synthetic rendering
- 2D bounding box
- 2D instance segmentation
- 2D depth map
HOW WAS IT CREATED?
A digital twin for a physical world
The Aria Digital Twin Dataset was captured in 2 different locations within Meta offices in North America, each with extensive ground-truth survey.
Photo-realistic object reconstruction
Each object within the Aria Digital Twin are laser scanned to reconstruct highly precise geometry. Object material are modeled using a photogrammetry pipeline and fine-tuned to ensure the images rendered from the models accurately match real images of the object.
Hyper-accurate scene digitization
The Aria Digital Twin Dataset was captured in 2 different locations within Meta offices in North America. Each room was laser scanned & modelled to ensure a high quality ground truth for each environment.
HOW IS THE DATASET ANNOTATED?
Comprehensive ground-truth of the real-world environment
For every frame of motion in the real-world footage, the Aria Digital Twin Dataset has a complete set of ground-truth data at the human, object, and scene level.
High-quality device and object 6DoF poses
Camera and object trajectories are provided for every sequence, aligned to the same reference-frame as the scene geometry, allowing annotations to be understood within the same context.
High quality depth-maps and object segmentation
Aria Digital Twin derives depth maps and object segmentation by leveraging the complete scene reconstruction and dynamic object tracking. This data provides researchers with additional knowledge of objects and scene.
3D human poses
In addition to camera poses, each Aria wearer is outfitted with a full body motion capture suit, to estimate the joint positions of the wearer. This allows dataset users to explore methods for full body pose estimation.
Faithfully-simulated synthetic sensor data
Each real-world sequence is accompanied by a synthetic sequence matching the sensor characteristics of the RGB and monochrome sensors on Aria glasses, at photo-realistic quality.
3D eye gaze vectors
Using data from Project Aria’s eye-tracking cameras, Aria Digital Twin includes an estimate of the wearer's eye-gaze as a 3D vector with depth information. This introduces additional user-object interaction besides hands.
ARIA DIGITAL TWIN DATASET TOOLS
Comprehensive tools to load and visualize data easily
Tools for working with Aria Digital Twin allow researchers to access, interact with, and visualize all raw data and annotations available in the dataset.
We provide both C++ and python interfaces to load data, so that researchers can access data in the way best suited to their needs. We also provide tools for querying dataset contents, so that specific types of data can be surfaced.
VIEW ARIA DIGITAL TWIN TOOLS ON GITHUB
HOW WILL THE DATASET BE USED?
Use Aria Digital Twin to participate in Object Detection Challenges
Aria Digital Twin is designed to catalyze research related to object detection and spatialization.
Use the dataset to participate in open benchmark challenges and accelerate progress with the open community.
VIEW THE OBJECT DETECTION ARIA CHALLENGE
Enabling innovation, responsibly
All sequences within the Aria Digital Twin Dataset have been captured using fully consented researchers in controlled environments in Meta offices.
RESPONSIBLE INNOVATION PRINCIPLES
Read the accompanying ADT Research Paper
For more information about the Aria Digital Twin Dataset, read our paper on here.
ARIA DIGITAL TWIN RESEARCH PAPER
Access Aria Digital Twin Dataset
If you are a researcher in AI or ML research, access the Aria Digital Twin Dataset and accompanying tools here.
By submitting your email and accessing the Aria Digital Twin Dataset, you agree to abide by the dataset license agreement and to receive emails in relation to the dataset.
Subscribe to Project Aria Updates
Stay in the loop with the latest datasets and challenges from Project Aria.
