DepMGNN: Matrixial Graph Neural Network for Video-based Automatic Depression Assessment

• 📄 Table of Contents
• 🥳 🚀 What's New
• 📖 Introduction
• 🛠️ Installation
• 👨‍🏫 Get Started
• 👀 Models
• 🙌 Results
• 🖊️ Citation

👏👏👏Congratulations (2024.12.10): Our work DepMGNN: Matrixial Graph Neural Network for Video-based Automatic Depression Assessment has been accepted by AAAI-2025 and selected for oral presentation!

                

Existing vector-style graph (left) and Our matrixial-style graph (right)

                     

Our clip-level spatio-temporal matrixial graph (left) and the updated matrixial graph by our MGNN (right)

Pipeline of our DepMGNN

MGNN is built on top of mmaction2 and torch-geometric.

Please refer to their official tutorials for detailed installation instructions.

conda create -n MGNN python=3.9 -y
conda activate MGNN

pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

pip install -U openmim
mim install mmengine
mim install mmcv==2.1.0

pip install torch-geometric==2.4.0

pip install einops
pip install timm
pip install seaborn

git clone https://github.com/AffectAI/MGNN.git
cd MGNN
pip install -v -e .

1. Apply for and download the AVEC2013, AVEC2014, and First Impression datasets from their official websites.

2. Crop the face from original videos use face_detect.py

   Quick instructions

   pip install pyfacer
   
   python face_detect.py

3. Place the cropped face frames in the corresponding folder under ./datasets. The corresponding dataset labels have been uploaded to the directory.

4. Download the pretrained resnet-50 model on vggface2 and put it into ./pretrained_models.

# Training on AVEC 2014
bash ./tools/dist_train.sh configs/depression/mgnn_depression_avec2014_res50.py num_gpus --seed 0

# Training on AVEC 2013
bash ./tools/dist_train.sh configs/depression/mgnn_depression_avec2013_res50.py num_gpus --seed 0

# Training on First Impression dataset
bash ./tools/dist_train.sh configs/depression/mgnn_personality_first_impression_res50.py num_gpus --seed 0

# Testing on AVEC 2014 Northwind and Freeform
bash ./tools/dist_test.sh configs/depression/mgnn_depression_avec2014_res50_test_fusion.py your/model/path/your_model.pth 1

# Testing on AVEC 2013
bash ./tools/dist_test.sh configs/depression/mgnn_depression_avec2013_res50.py your/model/path/your_model.pth 1

# Testing on First Impression dataset
bash ./tools/dist_test.sh configs/depression/mgnn_personality_first_impression_res50.py your/model/path/your_model.pth 1

1. Pretrained models: vggface2 pretrained resnet-50 model

2. MGNN AVEC 2014: MGNN (resnet-50)

3. MGNN AVEC 2013: MGNN (resnet-50)

4. MGNN First Impression: MGNN (resnet-50)

If you find this project useful in your research, please consider cite:

@inproceedings{wu2025depmgnn,
  title={DepMGNN: Matrixial Graph Neural Network for Video-based Automatic Depression Assessment},
  author={Wu, Zijian and Zhou, Leijing and Li, Shuanglin and Fu, Changzeng and Lu, Jun and Han, Jing and Zhang, Yi and Zhao, Zhuang and Song, Siyang},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={39},
  number={2},
  pages={1610--1619},
  year={2025}
}