OPEN SCIENCE
EgoBlur: a set of AI models designed
to preserve privacy by detecting and
blurring PII from images
INTRODUCING EGOBLUR GEN 2
EgoBlur Gen 2 protects user privacy by detecting and blurring PII from images on egocentric devices.
EgoBlur Gen 2 is Meta’s next-generation open-source solution designed to preserve privacy in egocentric video and imagery. Building on the success of EgoBlur Gen 1, this update suite provides researchers and developers with the tools to automatically detect and obscure personally identifiable information (PII), such as faces and license plates, captured by Project Aria Gen 2 devices and other head mounted cameras.
How is EgoBlur Gen 2 Evaluated?
To evaluate performance and reduce bias, EgoBlur Gen 2 is benchmarked against the Aria Gen 2 Pilot Dataset and CCV2 Dataset.
Self-reported ‘responsible AI labels’ from the CCV2 dataset are used to evaluate EgoBlur Gen 2 against a number of attributes such as skin tone, self identified gender, age, and country. This helps to ensure EgoBlur works consistently for everybody.
Key Enhancements with EgoBlur Gen 2
EgoBlur Gen 2 is tuned for egocentric data, and trained using data from Aria Gen 2 hardware. Plus native support for high-resolution VRS, PNG, JPEG, and MP4 files.
Unlike EgoBlur Gen 1, the Gen 2 command-line tool automatically preserves the input video’s original FPS, ensuring the blurred output remains perfectly synced with the source material.
Read the EgoBlur Gen 1 Research Paper
For more information about the EgoBlur Gen 1 model, read our paper on arXiv.
READ THE PAPER
BibTex Citation
If you use the EgoBlur Model in your research, please cite the following:
@misc{raina2023aria,
title ={EgoBlur Model},
author ={Nikhil Raina and Guruprasad Somasundaram and Kang Zheng and Sagar Miglani and Steve Saarinen and Jeff Meissner and Mark Schwesinger and Luis Pesqueira and Ishita Prasad and Edward Miller and Prince Gupta and Mingfei Yan and Richard Newcombe and Carl Ren and Omkar Parkhi},
year ={2023},
eprint ={2308.13093},
archivePrefix ={arXiv},
primaryClass ={cs.CV}
}If you use the dataset or tools on GitHub, please consider starring the repo.
By submitting your email and accessing the EgoBlur model, you agree to abide by the model license agreement and to receive emails in relation to the model.
Download EgoBlur Gen 1 and Gen 2
If you are a researcher in AI or ML research, access the EgoBlur models and accompanying tools here.
Frequently Asked Questions
Yes. Both the Gen 2 face and license plate models are licensed under Apache 2.0, meaning they are available for both research and commercial applications.
Both the Gen 2 face and license plate models are approximately 400 MB each and have ~104 million parameters.
The Gen 2 face model is fine-tuned on the Gen 1 face model. The Gen 2 license plate model is trained from scratch. Both models are trained using Meta's publicly available Detectron2 and Detectron2go libraries.
No. EgoBlur models are only trained to locate the position of faces and license plates of vehicles within color or grayscale images. The models are not used to track or identify individual faces or license plates.
Yes. The Gen 2 models are trained on both egocentric Aria data and non-egocentric data. Although we have only quantitatively evaluated on Aria data which is the main use case of the models.
EgoBlur Gen 2 is based on the Faster R-CNN model with a ResNeXt backbone. See EgoBlur paper for details.
Based on a 2MP RGB image, both the EgoBlur Gen 2 face detection and license plate models run in approximately 21ms on a GPU and 1.5 seconds on a CPU. Based on a 0.3MP gray scale image, both models run in approximately 25ms on a GPU and 1.6 seconds on a CPU. The inference script resizes images to similar size for optimal results hence the similar inference speed.
No. The Gen 2 face and license plate models are for detection only. The models output rectangular bounding boxes, not masks or labels.
Yes. EgoBlur Gen 2 tooling is available to apply the models to PNG, JPEG, MP4, and VRS files. The egoblur-vrs-blur tool provides end-to-end VRS blur — reading all camera streams, running detection, applying blur, and writing back into a single output VRS with non-camera streams (IMU, VIO, barometer, eye tracking, etc.) copied verbatim.
The Gen 2 models are benchmarked against an internal Aria Gen 2 Evaluation Dataset.
All five Aria Gen 2 camera streams are supported with tuned default detection thresholds: camera-rgb, slam-front-left, slam-front-right, slam-side-left, and slam-side-right.
See Egoblur GitHub repo for details.
Use the model generation that matches your capture device. The EgoBlur Gen 2 face model is fine-tuned from EgoBlur Gen 1 for Aria Gen 2 data. EgoBlur Gen 2 adds camera-specific detection thresholds, batched GPU inference, and end-to-end VRS blur support via egoblur-vrs-blur.
Please email projectaria@meta.com to report any bugs, or if you have any further queries about the EgoBlur models.
Subscribe to Project Aria Updates
Stay in the loop with the latest news from Project Aria.
By providing your email, you agree to receive marketing related electronic communications from Meta, including news, events, updates, and promotional emails related to Project Aria. You may withdraw your consent and unsubscribe from these at any time, for example, by clicking the unsubscribe link included on our emails. For more information about how Meta handles your data please read our Data Policy.
Follow Us
Sign up for our newsletter
By providing your email, you agree to receive marketing related electronic communications from Meta, including news, events, updates, and promotional emails related to Project Aria. You may withdraw your consent and unsubscribe from these at any time, for example, by clicking the unsubscribe link included on our emails.