Introduction

I am a Computer Science major at Earlham College with a strong interest in Artificial Intelligence and ethical applications of machine learning. My capstone project focuses on developing a Facial Emotion Recognition (FER) system with bias mitigation techniques to ensure fairness across demographic groups.

Abstract

Facial emotion recognition (FER) models often exhibit uneven performance across demographic groups, reinforcing existing social biases. This capstone quantifies and mitigates those disparities by building a pipeline that spans data cleaning through advanced bias-aware training. Using three benchmark datasets—FER-2013, RAF-DB, and CK+—we first trained a 48 × 48-pixel custom CNN baseline that reached 59.6. I then fine-tuned an ImageNet-pre-trained ResNet-50 and applied three complementary mitigation strategies: inverse-frequency sample re-weighting, focal-loss augmentation (γ = 2), and adversarial debiasing with a gradient-reversal race classifier. The best reweighted ResNet-50 improved overall accuracy to 66.5. The results demonstrate that simple weighting and adversarial objectives can substantially improve cross-group fairness without sacrificing overall performance, offering a practical recipe for bias-aware FER in real-time applications.

Graphical Abstract

Data Architecture Diagram

Demo Video

Poster