Anomaly Detection in AR Environments
Project Overview
This project leverages Unity3D, TensorFlow, and real-time webcam input to detect anomalies in physical environments — specifically designed to detect defects like rotten or discolored fruit. Using autoencoders and EfficientNetB0, it processes images in real time and provides live feedback.
Dataset Generation
We created a fully custom dataset from Unity by rendering synthetic apple and banana scenes under various lighting and rotation conditions. This allowed the model to learn generalized patterns without requiring massive real-world data collection.
Model Design
Our deep autoencoder was trained entirely in Google Colab using TensorFlow. The model compresses and reconstructs clean objects — allowing us to flag anything with high reconstruction error as anomalous. EfficientNetB0 was used as a backbone to extract efficient and meaningful features.
Live Application Results
The final application runs in real time, using a webcam feed to detect and highlight anomalies such as discolored or malformed bananas. Visual heatmaps and bounding boxes are displayed via a responsive UI built in Unity.
Evaluation & Detection Example
Here's an example of a detection in action, showing real-time feedback of the system catching an anomaly mid-frame. The autoencoder’s reconstruction error triggers the alert with minimal latency.
