Abstract
This project presents a Mood-Based Food Recommendation System that suggests food items according to a user’s emotional state. The system analyzes facial expressions, voice tone, and text input to detect mood using deep learning models. Facial emotions are recognized using a VGG16 based CNN, speech emotions through a Wav2Vec on raw audio, and text sentiments using a BERT model trained on sentimental datasets. The results from all three models are combined through a weighted fusion method to predict the user’s final mood. Based on this mood, suitable food recommendations are generated from a predefined mood–food dataset.
The system is implemented using Python (FASTAPI) for the backend and ReactJS for the frontend. Experimental results show improved accuracy and reliability compared to single-modality systems, making this approach effective for creating personalized and emotionally aware food recommendations.