PlatePal: A Comprehensive Image-Based Food Analysis and Dietary Assistant Mobile Application

This study presents a mobile food journaling application which offers functionalities to help users become aware of nutritional content and calories consumed throughout the day. We focus on composite cuisine, particularly Thai food, which comprises mixed ingredients, thus posing a challenge in recognizing and identifying combined components. Underlying techniques of the application are based on machine learning, aiming to perform three main tasks: (1) food classification, (2) volume estimation, and (3) chatbot functionality that offers dietary advice and consultations to improve the user experience. This study conducted extensive experiments to identify models that achieve the best performance for each task. The results indicate that Xception outperformed other models in both food classification and image segmentation. For volume estimation, convolutional neural networks (CNNs) demonstrated the ability to capture efficient depth perception, which was utilized by MiDas to generate a Point Cloud that approximates the volume of an input food image. For the chatbot, this study adopts OpenAI's GPT-3.5-turbo model, fine-tuned with nutritional data from the Department of Health of Thailand. The results show that the system is capable of delivering accurate responses. Finally, PlatePal received an overall evaluation of 8.53 out of 10, reflecting a high level of user satisfaction. This initiative is designed to promote healthier eating habits and contribute to the well-being of the Thai population across all age groups.