INTRODUCING
Project Aria Pilot Dataset
The Aria Pilot Dataset is a collection of 159 sequences captured using Project Aria, to accelerate the state of machine perception and AI.
WHAT IS IT?
A multi-purpose egocentric dataset created using Project Aria
The Aria Pilot Dataset and accompanying tools provides researchers in computer vision access to anonymized Aria sequences, captured in a variety of scenarios, such as cooking, playing games, or exercising. In addition to ‘Everyday Activities’, the dataset also includes ‘Desktop Activities’ captured with a multi-view motion capture system, helping to accelerate research into human-object interactions.
We believe this dataset will provide a baseline for external researchers to build and foster reproducible research on egocentric computer vision and AI/ML algorithms for scene perception, reconstruction and understanding.
LEARN MORE ABOUT PROJECT ARIA
Sensor Data
- 1 x 110 degree FOV Rolling Shutter RGB camera
- 2 x 150 degree FOV Global Shutter mono cameras for SLAM and hand tracking
- 2 x 80 degree FOV Global Shutter mono cameras for eye-tracking with IR illumination
- 2 x 1KHz IMU + barometer & magnetometer environmental sensors
- 7 x 48 KHz spatial microphones
- GPS 1 Hz
HOW IS IT ANNOTATED?
Automatic and manual annotations
In addition to providing sensor data from Project Aria, the Pilot Dataset also contains derived results from machine perception services which provide additional context to the spatial-temporal reference frames.
Multi-user poses in shared reference frame
In addition to providing per-frame trajectory for every recording, sequences captured within the same environment have been aligned to the same reference-frame, allowing those sequences to be understood within the same context.
Camera calibration
For a high-quality egocentric dataset, it is essential to understand how cameras perceive the world. The Project Aria Pilot Dataset provides full camera calibration parameters, including both intrinsics and extrinsics of every sensor.
Multi-view motion capture
To facilitate research into human-object interactions, the Aria Pilot Dataset includes a subset of “Desktop Activities” captured using a multi-view motion capture system.
Multi-device time sync
In addition to aligning the trajectories of sequences captured within the same environment, the Project Aria Pilot Dataset also provides precise time-alignment between sequences captured simultaneously.
Speech-to-text
For sequences where actors speak, we provide speech-to-text annotation. This supports egocentric communications research, such as predicting turn-taking in conversations and multi-speaker transcription.
Calibrated eye-gaze
Using data from Project Aria’s eye-tracking cameras, the Pilot dataset includes an estimate of the wearer’s eye-gaze. This can be used to accelerate research into user-object interactions.
HOW WILL THE DATASET BE USED?
Accelerating the state of machine perception and artificial intelligence
The Project Aria Pilot dataset consists of 159 sequences, which can be used to unlock several areas of research for progressing the state of machine perception and AI, including camera relocalization, and scene reconstruction.
Studying these research areas is crucial for researchers to engage with the challenges associated with AR devices.
PROJECT ARIA TOOLS
Powerful tools to harness Aria datasets
Tools for working with Project Aria Pilot Dataset allow researchers to access, interact with, and visualize all raw data and annotations available in the dataset.
Both C++ and python interfaces are provided to load data, so that researchers can access data in the way best suited to their needs.
VIEW ARIA DATA TOOLS ON GITHUB
Locations
To demonstrate a few representative scenarios in all-day-long activities with always-on sensing, multiple sequences were recorded using actors in five locations across USA.
Enabling innovation, responsibly
All sequences within the Aria Pilot Dataset have been captured using fully consented actors in controlled environments.
Additionally, faces and license plates have been manually blurred using human annotation prior to public release.
RESPONSIBLE INNOVATION PRINCIPLES
Access Project Aria Pilot Dataset and accompanying Tools
If you are a researcher in AI or ML research, access the Project Aria Pilot Dataset and accompanying tools from Meta.com.
By submitting your email and accessing the Aria Pilot Dataset, you agree to abide by the dataset license agreement and to receive emails in relation to the dataset.
Subscribe to Project Aria Updates
Stay in the loop with the latest news from Project Aria.
By providing your email, you agree to receive marketing related electronic communications from Meta, including news, events, updates, and promotional emails related to Project Aria. You may withdraw your consent and unsubscribe from these at any time, for example, by clicking the unsubscribe link included on our emails. For more information about how Meta handles your data please read our Data Policy.
