Recognizing the sweet and sour taste of pineapple fruits using residual networks and green-relative color transformation attached with Mask R-CNN

The aromas and tastes produced by different chemical compounds in fruits are important drivers of consumption,
but they are difficult to discern from visual appearance alone. This paper presents a novel three-stage deeplearning-
based model framework that first localizes a pineapple and then identifies its taste. The first stage
extracts the pixel-wise region of interest using a segmentation model. The segmented object is preprocessed and
transformed into green-relative color space (YCbCr) that highlights various pineapple elements, such as pineapple
buds and their surrounding areas regardless of lighting conditions. An experienced specialist characterizes
pineapple taste by a similar process. Finally, the segmented and processed image is fed into residual networks for
taste classification. After generalization via image augmentation with contrast adjustment, the model framework
was shown to be highly robust with an F1-score, precision, and recall of 0.9025, 0.9026, and 0.9025, respectively.
The proposed framework correlates the visual appearance of a fruit with its corresponding taste. Therefore,
it can be a valuable tool in fruit export or local consumption applications.