This project implements a two-stage deep learning pipeline to clean raindrop- and glare-obstructed dashcam frames and restore clear license plates for downstream OCR.
The model is based on:
- A ConvLSTM-style attention module that predicts raindrop / artifact masks.
- A UNet-like encoder–decoder inpainting network that reconstructs a clean image from the masked input.
The project was built as part of my Computer Vision using Deep Learning (CVDL) coursework and extended with real dashcam footage from my own car.
- 🧠 Two-stage architecture: ConvLSTM-based mask prediction + UNet-style inpainting.
- 🌧️ Trained on DeRaindrop: Uses the public DeRaindrop dataset for supervised raindrop removal.
- 🚗 Real-world dashcam fine-tuning: Fine-tuned on custom car dashcam snapshots to generalize to real driving conditions.
- 🔍 License plate clarity: Significantly improves plate visibility for OCR / ANPR pipelines.
- 📊 Colab-ready demo: One-click notebook workflow for running inference on your own rainy frames.
The Generator model combines:
-
Detection / Attention Stage
- Several convolutional layers process the input image.
- A ConvLSTM-style recurrent block with gates (
i,f,g,o) produces a sequence of hidden states. - An attention mask is generated indicating raindrop / artifact regions.
-
Inpainting / Reconstruction Stage
- The input image is concatenated with the predicted mask.
- A UNet-like encoder–decoder network with skip connections reconstructs a clean image.
- The network outputs the restored frame and intermediate predictions.
- Trained and evaluated using the DeRaindrop dataset
- Model definition and demo pipeline implemented in PyTorch and inspired by raindrop removal literature.
Thank you