INTRODUCING

Reading in the Wild Dataset

100 hours of egocentric RGB video, eye gaze, and head pose data of reading and non-reading activities collected in diverse and realistic scenarios.

WHAT IS IT?

100 hours of reading in-the-wild

A first-of-its-kind dataset Reading in the Wild is designed to help solve the task of reading recognition by wearable devices in diverse environments. Reading in the Wild is a large-scale multimodal dataset comprising 100 hours of reading and non-reading videos captured in diverse and realistic scenarios using Project Aria. The dataset features video, eye gaze, and head pose sensor outputs, created to help solve the task of reading recognition from wearable devices.

EXPLORE THE DATA IN ARIA DATASET EXPLORER
ACCESS READING IN THE WILD GITHUB REPO
DOWNLOAD THE DATASET

Dataset Contents

  • 100 hours of video data
  • 1716 sequences
  • 111 participants
  • 60 Hz high-frequency eye tracking
  • 19 high-level scenarios of reading and non-reading activities
  • 150+ Reading materials
  • Includes non-reading activities containing both daily activities and hard negatives (where text is present but is not being read).

Sensor Data

  • 1 x 110 degree FOV + 30 Hz Rolling Shutter RGB camera
  • 2 x 80 degree FOV + 60 Hz, Global Shutter mono cameras for eye-tracking with IR illumination
  • 2 x 150 degree FOV + 30 Hz Global Shutter mono cameras for SLAM and hand tracking
  • 1 kHz + 800 Hz IMUs + barometer & magnetometer environmental sensors
  • 7 x 48 KHz spatial microphones

Annotations

  • Start and end time of reading activities in each sequences
  • Type of Reading material used
  • Type of reading
  • The participant’s description of the reading material
  • 3D eye gaze
  • 6DoF device trajectory

Highlights
WHAT MAKES IT SPECIAL?

Real-world multimodal egocentric data for reading recognition in the wild

Bird-eye view of one of the scenes contained within the Aria Synthetic Environments dataset.

Diverse scenes and scenarios

  • 19 high-level scenarios of reading and non-reading activities*
  • Various indoor and outdoor locations: Offices, Libraries, Lounges, Homes, Stores, Balconies, Patios, Roads, Trails, In the Woods.
Ground-levell view of one of the scenes contained within the Aria Synthetic Environments dataset.

Various Reading Modes

Our dataset includes multiple reading modes including:

  • Engaged reading
  • Skimming
  • Scanning
  • Reading out loud
  • Multi-tasking (While walking, Writing/typing)
Screenshot showing the CAD-like language used by the Aria Synthetic Environments dataset to describe doors, walls, and windows.

Comprehensive Reading Materials

150+ Reading materials in different mediums and text lengths.

Visualization of one of the scenes contained within the Aria Synthetic Environments dataset. The simulated trajectory of an Aria device is included within the scene.

Annotations

The recordings were processed by Aria Machine Perception Service to obtain accurate 6 DoF device trajectory, semi-dense point clouds, and 3D eye gaze ray estimation with depth.

In addition, the annotation contains

  • Start and end time of reading activities in each sequences
  • Type of reading material used
  • Type of reading mode
  • The participant’s description of the reading material

Seattle and Columbus subsets


The Reading in the Wild dataset contains two distinct subsets, one captured in Columbus and the other captured in Seattle. The Seattle subset focuses on diversity, while the Columbus subset aims to test our model’s ability to generalize in unseen settings, as well as identify edge cases where the model fails.

Seattle subset

The Seattle subset emphasizes diversity, comprising 80 hours of data from 80 participants engaged in various reading and non-reading activities across multiple indoor and outdoor settings.

DOWNLOAD THE DATASET

Columbus subset

The Columbus subset is designed to evaluate edge cases, comprising approximately 20 hours of data from 31 subjects containing reading and non-reading activities in indoor settings. It also features 4 different languages.

ACCESS COLUMBUS SUBSET ON GITHUB REPO
Screenshot from Project Aria Dataset Explorer

Discover the datasets you need, with Aria Dataset Explorer

Aria Dataset Explorer makes it easy to discover, visualize, filter, and download Project Aria public datasets from the Project Aria team, all in one place.

DISCOVER DATASETS
A screenshot from the Reading Recognition in the Wild research paper.

Read the Reading Recognition in the Wild Research Paper

For more information about our motivations, methods, and dataset details read our research paper on arXiv.

READ THE RESEARCH PAPER

Access Reading in the Wild Dataset

If you are a researcher interested in recreating our method of scene reconstruction, access the Aria Scene Reconstruction Dataset to get started.

By submitting your email and accessing the Reading Recognition dataset, you agree to abide by the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY_NC 4.0) and Apache 2.0 License, and to receive emails in relation to the dataset.

Subscribe to Project Aria Updates

Stay in the loop with the latest news from Project Aria.

By providing your email, you agree to receive marketing related electronic communications from Meta, including news, events, updates, and promotional emails related to Project Aria. You may withdraw your consent and unsubscribe from these at any time, for example, by clicking the unsubscribe link included on our emails. For more information about how Meta handles your data please read our Data Policy.

Meta logo, homepage link

Follow Us

facebook icon
YouTube icon
X icon
LinkedIn icon

Sign up for our newsletter

By providing your email, you agree to receive marketing related electronic communications from Meta, including news, events, updates, and promotional emails related to Project Aria. You may withdraw your consent and unsubscribe from these at any time, for example, by clicking the unsubscribe link included on our emails.