A new approach called LDFA (Latent Diffusion Face Anonymization) was recently developed by a research team from Karlsruhe Institute of Technology and FZI Research Center for Information Technology.
LDFA aims to anonymize faces in the datasets used to train ITS (Intelligent Transportation Systems). The novelty of this research consists in the use of the latent diffusion models to replace faces of people with in-paintings faces generated by the model.
LDFA pipeline
Image legend:
a) Face detection & context padding
b) Latent diffusion model
c) Synthesized faces & anonymized image
Privacy issues
Privacy has been a growing concern for many people. There is a significant amount of monitoring and data-collecting happening all the time, everywhere.
Social media platforms, search engines, and other tech companies collect vast amounts of data from their users. There are laws and regulations regarding the access to certain information and one of them is the General Data Protection Regulation (GDPR) sets of laws issued by the European Union (EU).
When it comes to the autonomous driving, protecting people’s privacy becomes a very complicated issue. ITS have to accurately identify and protect the pedestrians, cyclists and other VRUs (vulnerable road users).
Therefore, the perception models used by ITS must be trained with big datasets of a significant number of VRUs. However, privacy regulations require that faces and other personal information be anonymized in all the datasets.
LDFA – a new approach to anonymizing faces
Faces can be anonymized by using naïve techniques like blurring or cropping, but in this case the datasets may not accurately reflect real-world scenarios. If the images are not realistic, they reduce the accuracy of the trained model.
To address this problem, the research team proposed LDFA for self-driving applications that replace the real face with an AI generated one.
In the picture below there are depicted six methods of anonymization used in various contexts. The applied methods (from left to right) are: none (original image), CROP, GAUSS, PIXEL, LDFA, DEEPPRIVACY1, and DEEPPRIVACY2.
The research evaluation tests reveal that LDFA achieves higher mAP (mean average precision) scores than DEEPPRIVACY1 and DEEPPRIVACY2. This suggests that the LDM-based pipeline like LDFA perform better than other anonymization methods.
Moreover, LDFA was able to detect nearly twice the number of anonymized faces, as compared to the default settings of DEEPPRIVACY.
Conclusion
Generating realistic face in-paintings of VRUs in ITS improve the accuracy of video analytics systems that are used in ITS to detect and track VRUs such as pedestrians and cyclists.
By filling out missing information in the datasets (e.g. obscured or occluded faces of VRUs), the analysis will be more accurate, increasing the safety of road users.
In the future, LDFA on face anonymization may be extended to full-body anonymization.
Learn more:
- Research paper: “LDFA: Latent Diffusion Face Anonymization for Self-driving Applications” (on arXiv)
- Latent diffusion model (on Hugging Face)
