Authors: Xin Sun (North Dakota State University) , Jennifer Young (North Dakota State University) , Jeng Hung Liu (North Dakota State University) , Quansheng Chen (Jiangsu University) , David Newman (Arkansas State University)
The objective of this study was to investigate the ability of image color features to predict subjective pork color scores. Subjective and instrumental color were assessed on the bloomed, cross-sectional surface of pork longissimus thoracis et lumborum chops. Images of pork chop samples were acquired using a computer vision system, and 18 image color features (mean and standard deviation of R, G, B, H, S, I, L*, a*, b*) were extracted for inclusion in partial least squares (PLS) and support vector machine (SVM) regression models. For color scores 2, 3, 4, and 5, the accuracies were 50.4, 75.9, 72.4, and 47.3% classified correctly by PLS, respectively, and 70.7, 72.8, 76.7, and 69.7% by SVM, respectively. The overall prediction accuracies of 2 models for pork color scores were 68.3% for PLS and 73.4% for SVM. There was minimal major misclassification of samples (< 0.5%). Image color features isolated through the development of PLS and SVM models, particularly SVM, show potential as a method to predict pork color scores.
Keywords: computer vision, pork, color score
How to Cite: Sun, X. , Young, J. , Liu, J. , Chen, Q. & Newman, D. (2018) “Predicting Pork Color Scores Using Computer Vision and Support Vector Machine Technology”, Meat and Muscle Biology. 2(1). doi: https://doi.org/10.22175/mmb2018.06.0015
Pork quality influences consumers’ intent to purchase (Papanagiotou, Tzimitra-Kalogianni, and Melfou, 2013), with fresh color being one of the most important quality elements (Brewer, Lan, and McKeith, 1998). For pork producers, improving the quality of pork products they produce could have a positive impact on hog demand and market value. Methods to quantify the quality of pork products is a hot topic of both industry and researchers (Karamucki, Rybarczyk, Jakubowska, and Sulerzycka, 2017; Sun, Young, Liu, Newman, 2018; Caballero et al., 2017). Currently, the majority of pork-color grading is based on a subjective score assigned by a trained evaluator, which is time-consuming, and dependent on lighting source when evaluating (Mancini, 2009). A more objective method is to use a colorimeter, such as Minolta or HunterLab Miniscan; however, this method is limited in the measurement area (aperture size) and intramuscular fat within the measurement area may affect reliability (Kang, East, and Trujillo, 2008) and accuracy of this application.
Computer vision technology has been used for detecting meat quality for some time (O’Sullivan et al., 2003; Chandraratne, Samarasinghe, Kulasiri, and Bickerstaffe, 2006; Chen, Sun, Qin, and Tang, 2010; Ranasinghesagara et al., 2010; Chmiel, Słowiński, and Dasiewicz, 2011a,b; Sun et al., 2012; Girolami, Napolitano, Faraone, and Braghieri, 2013). Computer vision systems have the advantage of being able to evaluate the entire surface of samples compared to the limited area measurement of the portable colorimeters; therefore, computer vision can represent more accurately the true color features for the entire sample surface. Furthermore, through image processing, computer vision systems can segment the intramuscular fat and lean muscle tissues, allowing them to analyze only the color of the lean tissue; thus, computer vision technology may be more effective and accurate than traditional methods used currently.
Support vector machines (SVM) were first proposed by Cortes and Vapnik (1995), and the use of SVM is gaining popularity due to the ability to use polynomial, radial-based functions as a means to reach multilayer perception classifications. The SVM system fixes the classification decision function on the basis of the structural risk minimum mistake instead of the minimum mistake of misclassification based on the confines of the data presented through the training set. This analytical distinction is important because it allows the SVM to avoid over-fitting the problem. The greatest advantages of SVM technology were that it does not require a large number of training sets compared with other data prediction methods and dimensionality reduction analysis. Therefore, the objectives of this study were to: 1) investigate the usefulness of raw pork chop surface color characteristics in predicting pork quality color scores; and 2) establish traditional regression and SVM regression models to relate the pork color subjective standard score with image color features from the images obtained under consistent lighting.
Boneless center-cut pork longissimus thoracis et lumborum (LTL) chops were collected from a large retail pork study. One hundred forty-three retail supermarkets representing 29 cities from 23 states were chosen for sampling. Samples were collected throughout the United States from the top 3 major retailers in each city between January 2015 and April 2015 to eliminate any holiday or seasonal merchandizing variation. Retail supermarkets were visited between the hours of 09:00 and 17:00. Details of LTL chop collection can be found in Bachmeier (2016). Chops were vacuum packaged after purchase from the retail stores and shipped overnight (0 to 4°C) to North Dakota State University Meat Quality Laboratory for quality assessment. After arrival at NDSU, the packages were opened immediately and allowed to bloom for 10 to 15 min at room temperature before pork color assessment. Chops were then assessed on the cross-sectional surface for subjective color scores (Fig. 1; NPB, 2011), and instrumental color (CIE L*, a*, and b* color space values) was measured once per chop with a Minolta Colorimeter (CR-410, Minolta Co., Ramsey, NJ), equipped with a 50-mm aperture, 2° observer, and illuminant C. A total of 685 pork center-cut LTL chops were collected and used in this experiment (n = 75, 284, 240, and 86 categorized as color scores 2, 3, 4, and 5, respectively).
The image acquisition system used for this study consisted of three components: a CCD camera (MV-VS141FM/C, Micro-vision Ltd., Beijing, China) with 5-mm C-mount lens (aperture of f/1.4 to 16C, H0514-MP2 1.27-cm fixed Lens, CBC Americas Corp, Cary, NC), two adjustable white LED lighting systems (Lux = 401; 255mm × 45mm, model #: YX-BL25040, Yongxin Ltd., Yantai, China), and a personal computer. The camera was calibrated by adjusting the white balance value using a white standard board.
A black interior lighting box (Fig. 2) was designed for image acquisition to avoid backscattering effects from other lighting sources. Inside the box, there was a dome made of light-reflecting material at the top with a hole in the center for the camera lens. Additionally, there were 2 bars with white LED lights angled upward to reflect light from the dome to maintain consistent lighting on the meat surface. During the image acquisition step, each chop was manually placed on a black background surface platform located 45 cm under the camera lens. The pork image was captured using image acquisition software (MV1394, MicroVision Inc., Xian, Shanxi, China) in the computer linked to the camera. All the acquired images were 1,392 × 1,040 pixels and stored in BMP format for further image analysis.
Starting with the original pork chop image (Fig. 3a), background removal was accomplished based on the automatically-calculated optimum threshold value using Otsu’s thresholding method (Otsu, 1979), resulting in a binarized image of the pork chop (Fig. 3b). After transforming the image to a gray scale (Fig. 3c), segmentation of intramuscular fat (Fig. 3d) and lean tissue (Fig. 3e) was performed using Sun’s color threshold method (Sun, Gong, Zhang, and Chen, 2009). All algorithms used in this study for image processing and analysis were developed in MATLAB (Version 7; The Math-works, Natick, MA) by the authors.
For color image feature extraction, RGB (red, green and blue), HSI (hue, saturation, and intensity), and L*a*b* (lightness, redness, and yellowness) color spaces were used to extract the image color features. For each image, transformations from RGB color space to HSI and L*a*b* color spaces were performed according to Sun’s color threshold method (Sun et al., 2009). In this research, the mean (μ) and standard deviation (σ) of each color feature were calculated for each pork chop sample image.
For pork color prediction, partial least squares (PLS) regression and support vector machine regression models were created using unscrambler software (Camo Software, Woodbridge, NJ) based on the initial subsets.
For SVM model, the guassian kernel was used for the SVM to obtain classification of a 2-class model through the use of a separating hyperplane, and the method of using SVM modeling and choosing the kernel method (Sun et al., 2012) was utilized in this study.
To prevent overtraining of developed models, the original dataset was randomly divided into training (80%) and testing (20%) datasets. Using the training dataset, 10-fold cross-validation was employed to optimize both models. More specifically, 10 models were constructed by leaving out a different subset each time, with the remaining 9 subsets collectively representing the training set. Once a model was developed using the training dataset, the testing dataset was used as an independent set to calculate the final prediction accuracy. Minor and major misclassifications were calculated for each color score. An incorrect prediction was considered a minor misclassification if it was only one color score off the original color score (i.e., a 3 is predicted as a 4) and a major misclassification if it was more than one color score off (i.e., a 3 is predicted as a 5). This methodology was repeated 10 times using different training and testing datasets each time.
The large variation in the mean (μ) of color features agrees with the large variation in subjective color scores, whereas the standard deviation (σ) of color features had less variation (Table 1). Although the pork samples in this study did not cover all 6 possible color scores, the range represented in this study (color score 2 to 5) is representative of the U.S. retail pork market (Newman, 2015).
|Subjective color score||3.49||0.84||2||5||24.1|
Mean, μ, and standard deviation, σ, of R (red), G (green), B (blue), H (hue), S (saturation), I (intensity), L* (black to white), a* (green to red), and b* (blue to yellow).
Components variance analysis was first performed to determine the number of components (i.e., the latent factors) for the PLS model (Fig. 4). Principal component analysis (PCA) method was used as component decision algorithm for the PLS model. The 2 two components made significant contributions to the percentage of variance explained in the response up to nearly 67% while very little contribution was obtained when the remaining components were added. Therefore, the ideal number of the PLS components in the model was two.
The PLS and SVM model prediction results for color scores are shown in Table 2. The average, minimum, and maximum prediction accuracies and minor and major misclassifications are reported for each color score and regression model. For color scores 2, 3, 4, and 5, the average model prediction accuracies for PLS were 50.4, 75.9, 72.4, and 47.3%, respectively. The average overall prediction accuracy for the PLS model for all color scores was 68.3%. For color scores 2, 3, 4, and 5, the average model prediction accuracies for SVM were 70.7, 72.8, 76.7, and 69.7%, respectively. The average overall prediction accuracy for the SVM model for all color scores was 73.4%. The SVM model was better at predicting color scores 2 and 5 than the PLS model. Major misclassification (predicted color more than 1 color score off of original value) was minimal across all 10 repetitions. Four repetitions had 1 major misclassification for the SVM model. Since subjective color is being used to determine accuracy, the minor and major misclassifications may be due in part to error on the person calling subjective color or to error in the regression models.
|Color Score||Prediction accuracy, %||Minor misclassification, %||Major misclassification, %|
|Mean ± SE||Min||Max||Mean||Min||Max||Mean||Min||Max|
|PLS||2||50.4 ± 18.9||25.0||66.7||49.6||33.3||75.0||0.0||0.0||0.0|
|3||75.9 ± 3.8||67.2||86.7||24.1||13.3||32.8||0.0||0.0||0.0|
|4||72.4 ± 3||66.7||81.0||27.6||19.0||33.3||0.0||0.0||0.0|
|5||47.3 ± 27.1||23.1||72.2||52.7||27.8||76.9||0.0||0.0||0.0|
|Overall||68.3 ± 0.8||64.2||73.0||31.7||27.0||35.8||0.0||0.0||0.0|
|SVM||2||70.7 ± 9.2||56.3||91.7||29.3||8.3||43.8||0.0||0.0||0.0|
|3||72.8 ± 2.1||67.2||80.4||26.9||19.6||32.8||0.3||0.0||1.7|
|4||76.7 ± 4.7||66.0||89.4||22.8||10.6||31.9||0.5||0.0||2.5|
|5||69.7 ± 26.0||47.4||93.8||30.3||6.3||52.6||0.0||0.0||0.0|
|Overall||73.4 ± 1.2||69.3||79.6||26.4||20.4||29.9||0.3||0.0||0.7|
In this research, computer vision technology along with advanced image processing methods were investigated for the prediction of color scores of pork chops. Two regression models (PLSR and SVMR) were established to predict the different color scores of pork.
Partial least squares regression is a method for constructing predictive models when the factors are numerous and highly collinear. The advantage of PLS is that it combines principal component, canonical correlation, and multiple linear regression analyses in one analysis (Wold, Trygg, Berglund, & Antti, 2001). With PLS, matrices are included and excluded simultaneously to find the latent (or hidden) variables in the input that will best predict the latent variables in the output. Unlike the bilinear methods of PLS, support vector machine (SVM) regression uses kernels to transform nonlinear systems into linear systems before the application of regression. This is done by selecting an appropriate kernel and fine-tuning its parameters to achieve an acceptable result. The SVM regression is a method that handles linear as well as nonlinear situations in a regression context (Chang & Lin, 2011). The use of SVM was shown to have potential for effectively classifying the color or tenderness of beef in previous studies (Sun, Chen, Berg, & Magolski, 2011; Sun et al., 2012).
Eighteen color image features from three different color spaces (RGB, HIS, L*a*b*) were extracted from each pork chop image. Chmiel et al. (2011b) also used a computer vision method to detect pale, soft, and exudative (PSE) pork from three different color spaces, including RGB, HSV (hue, saturation, value), and HSL (hue, saturation, lightness), and observed a significant difference between normal meat and PSE meat when using the lightness (L) component as a predictor for the classification model. These results show that image color features have strong potential for accurately predicting meat surface attributes such as PSE pork. Additionally, both PLS and SVM models reached satisfactory prediction accuracies, which demonstrate the potential of establishing a nondestructive, rapid-detection system to evaluate pork color scores in the future.
Whether assessing pork color using subjective color scores or colorimeter device, marbling is an important element which can affect the “true” color evaluation. Since subjective color is being used to determine accuracy, the minor and major misclassifications may be due in part to error on the person calling subjective color or to error in the regression models. In this experiment, therefore, an image processing method to segment the marbling area out of the analysis area was used as a more advanced method of determining lean color compared to subjective color or colorimeter assessment. Compared to a colorimeter device, which is limited in the area analyzed and would also include any marbling present, the computer vision system considers the whole meat surface and segregates the lean tissue when determining color. While computer vision systems may be better at predicting the “true” color of lean tissue, subjective color scores and colorimeter devices may be influenced by intramuscular fat as are consumers when purchasing fresh pork from the retail store. Therefore, while assessing “true” color may be important for predicting quality, it may not be as important when considering consumer purchase intentions. Another issue with using colorimeter devices was evaluated by Girolami et al. (2013), who compared traditional colorimeter measurements and computer vision system to measure fresh color attributes of beef, pork and chicken. Girolami et al. (2013) showed that consumers thought that colors generated by Adobe Photoshop using the L*, a*, and b* from computer vision system was more similar to the actual color of meat than colors generated using the L*, a*, and b* from the colorimeter device. Girolami et al. (2013) concluded that the translucency of the meat influenced the L*, a*, and b* of the colorimeter device and resulted in color values that were not truly representative of the meat color. On the other hand, the computer vision method used in this research can produce valid measurements and also save images of the samples for further research and industry validation analyses.
Once novel methods of detecting pork color attributes have reached high-satisfaction levels, the next step will be to establish a system that can rapidly assess and segregate fresh pork cuts into quality grades. A number of researchers have employed hyperspectral and near infrared technology to predict pork quality components (Qiao, Ngadi, Wang, Gariépy, and Prasher, 2007a; Qiao et al., 2007b; Li et al., 2011; Zhou, Cai, Wang, Ji, and Chen, 2011; Barbin, Elmasry, Sun, and Allen, 2012a; Barbin, ElMasry, Sun, and Allen, 2012b; Liu and Ngadi, 2014; Pu, Sun, Ma, and Cheng, 2015; Tao and Peng, 2015; Tao, Peng, and Li, 2015); however, most of these studies were conducted in controlled laboratory settings. Although some of these studies reported significant accuracy values for the prediction of pork color, the systems were time consuming because of the heavy acquisition and processing procedure time. We believe our computer vison system will have the highest potential to approach industry online conveyor speed requirements.
The computer vision system used in the current study can accurately evaluate pork color, a major advantage over traditional subjective evaluation and/or colorimeter devices which have their own limitations. More importantly, the computer vision system has the potential to meet the speed requirements required of on-line applications in pork processing facilities.
Bachmeier, L. A. ( 2016). 2015 National Pork Retail Benchmarking Study (Master’s thesis). Retrieved from ProQuest Dissertations Publishing.
Barbin, D.Elmasry, G.Sun, D. W.Allen, P.. 2012a. Near-infrared hyperspectral imaging for grading and classification of pork. Meat Sci. 90(1):259–268.
Barbin, D. F.ElMasry, G.Sun, D. W.Allen, P.. 2012b. Predicting quality and sensory attributes of pork using near-infrared hyperspectral imaging. Anal. Chim. Acta 719:30–42.
Brewer, M. S.Lan, H.-Y.McKeith, F.. 1998. Consumer evaluation of pork appearance with differing physiological and packaging conditions. J. Muscle Foods 9(2):173–183.
Caballero, D.Pérez-Palacios, T.Caro, A.Amigo, J. M.Dahl, A. B.ErsbØll, B. K.Antequera, T. 2017. Prediction of pork quality parameters by applying fractals and data mining on MRI. Food Res. Int. 99:739–747.
Chandraratne, M. R.Samarasinghe, S.Kulasiri, D.Bickerstaffe, R.. 2006. Prediction of lamb tenderness using image surface texture features. J. Food Eng. 77(3):492–499.
Chang, C.-C.Lin, C.-J.. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2:27.
Chen, K.Sun, X.Qin, C.Tang, X.. 2010. Color grading of beef fat by using computer vision and support vector machine. Comput. Electron. Agric. 70(1):27–32.
Chmiel, M.Słowiński, M.Dasiewicz, K.. 2011a. Lightness of the color measured by computer image analysis as a factor for assessing the quality of pork meat. Meat Sci. 88(3):566–570.
Chmiel, M.Słowiński, M.Dasiewicz, K.. 2011b. Application of computer vision systems for estimation of fat content in poultry meat. Food Contr. 22(8):1424–1427.
Cortes, C.Vapnik, V.. 1995. Support-vector networks. Mach. Learn. 20(3):273–297.
Girolami, A.Napolitano, F.Faraone, D.Braghieri, A.. 2013. Measurement of meat color using a computer vision system. Meat Sci. 93(1):111–118.
Karamucki, T.Rybarczyk, A.Jakubowska, M.Sulerzycka, A.. 2017. A comparison of two methods of determining colour change in the assessment of the quality of pork. Acta Sci. Pol. Technol. Aliment. 16(3):321–329.
Kang, S. P.East, A. R.Trujillo, F. J.. 2008. Colour vision system evaluation of bicolour fruit: A case study with ‘B74’mango. Postharvest Biol. Technol. 49(1):77–85.
Li, Y.Zhang, L.Peng, Y.Tang, X.Chao, K.Dhakal, S.. 2011. Hyperspectral imaging technique for determination of pork freshness attributes. Sensing for Agriculture and Food Quality and Safety III 8027.
Liu, L.Ngadi, M. O.. 2014. Predicting intramuscular fat content of pork using hyperspectral imaging. J. Food Eng. 134:16–23.
Mancini, R. A. 2009. Meat color. Improving the sensory and nutritional quality of fresh meat, 89-110.
Newman, D. 2015. Pork Quality National Retail Benchmarking Study. World Pork Expo 2015, Des Moines, IA.
NPB (National Pork Board). 2011. Pork quality standard cards. Des Moines, IA.
O’Sullivan, M. G.Byrne, D. V.Martens, H.Gidskehaug, L. H.Andersen, H. J.Martens, M.. 2003. Evaluation of pork colour: Prediction of visual sensory quality of meat from instrumental and computer vision methods of colour analysis. Meat Sci. 65(2):909–918.
Otsu, N. 1979. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9:62–66.
Papanagiotou, P.Tzimitra-Kalogianni, I.Melfou, K.. 2013. Consumers’ expected quality and intention to purchase high quality pork meat. Meat Sci. 93(3):449–454.
Pu, H.Sun, D.-W.Ma, J.Cheng, J.-H.. 2015. Classification of fresh and frozen-thawed pork muscles using visible and near infrared hyperspectral imaging and textural analysis. Meat Sci. 99:81–88.
Qiao, J.Ngadi, M. O.Wang, N.Gariépy, C.Prasher, S. O.. 2007a. Pork quality and marbling level assessment using a hyperspectral imaging system. J. Food Eng. 83(1):10–16.
Qiao, J.Wang, N.Ngadi, M. O.Gunenc, A.Monroy, M.Gariépy, C.Prasher, S. O.. 2007b. Prediction of drip-loss, pH, and color for pork using a hyperspectral imaging technique. Meat Sci. 76(1):1–8.
Ranasinghesagara, J.Nath, T. M.Wells, S. J.Weaver, A. D.Gerrard, D. E.Yao, G.. 2010. Imaging optical diffuse reflectance in beef muscles for tenderness prediction. Meat Sci. 84(3):413–421.
Sun, X.Young, J.Liu, J. H.Newman, D.. 2018. Prediction of pork loin quality using online computer vision system and artificial intelligence model. Meat Sci. 140:72–77.
Sun, X.Chen, K.Berg, E.Magolski, J.. 2011. Predicting fresh beef color grade using machine vision imaging and support vector machine (SVM) analysis. J. Anim. Vet. Adv. 10(12):1504–1511.
Sun, X.Chen, K. J.Maddock-Carlin, K. R.Anderson, V. L.Lepper, A. N.Schwartz, C. A.Keller, W. L.Ilse, B. R.Magolski, J. D.Berg, E. P.. 2012. Predicting beef tenderness using color and multispectral image texture features. Meat Sci. 92(4):386–393.
Sun, X.Gong, H.Zhang, F.Chen, K. J.. ( 2009). A digital image method for measuring and analyzing color characteristics of various color scores of beef. Paper presented at the 2nd International Congress on Image and Signal Processing, 2009 (CISP’09).
Tao, F.Peng, Y.. 2015. A nondestructive method for prediction of total viable count in pork meat by hyperspectral scattering imaging. Food Bioproc. Techol. 8(1):17–30.
Tao, F.Peng, Y.Li, Y.. 2015. Feature extraction method of hyperspectral scattering images for prediction of total viable count in pork meat. International Journal of Agricultural and Biological Engineering 8(4):95–105.
Wold, S.Trygg, J.Berglund, A.Antti, H.. 2001. Some recent developments in PLS modeling. Chemom. Intell. Lab. Syst. 58(2):131–150.
Zhou, R.Cai, B.Wang, S.Ji, H.Chen, H.. ( 2011). Pork grade evaluation using hyperspectral imaging techniques. 2011 International Conference on Optical Instruments and Technology: Optoelectronic Imaging and Processing Technology, 82000P.