Estimation & Prediction

End-to-end Genomic Prediction: Predicting Images and Text from Genetic Markers

---

Abstract

Methods to predict breeding values from genetic markers, collectively known as genomic prediction, have become widely used in breeding. Currently, genomic prediction is limited to numeric phenotypes. In some cases, though, phenotypes are better understood through images and text rather than numbers. The current best practice for incorporating images and text in genomic prediction is to first extract scalar numeric phenotypes from images and text, and then to perform genomic prediction on the numeric phenotypes. While this approach is effective for some traits, it involves discarding most of the information in the image or text, including potentially useful information. Additionally, numeric phenotypes derived from images and text may not be as interpretable as the images and text themselves. For example, aesthetic preference and sensory traits such as taste are well represented in images and text, respectively, but are difficult to reduce into a few numeric features. We present a novel approach for predicting images and text from SNP markers, which we refer to as end-to-end genomic prediction, and validate this approach using genotypes, text, and image phenotypes. To demonstrate our method's utility, we have tested our end-to-end genomic prediction method in a strawberry breeding population to predict fruit shape and color traits, for which simple numeric features extracted from images contained insufficient information for selection. Our approach combines nonlinear latent space encoding with linear genomic prediction to generate accurate breeding values for a high-dimensional phenotype, for example, images or text. For both genome-to-image and genome-to-text prediction, we found that predicting images and text and then extracting numeric traits from them was in some cases as accurate as directly predicting extracted numeric phenotypes, demonstrating for the first time that genome-to-image prediction accuracy can be comparable to conventional genomic prediction accuracy. Based on our proof-of-concept predicting images and text in strawberry, we believe end-to-end could be of use in a wide range of visual and non-standardized phenotypes in animal breeding, although further work to improve the accuracy of the embedding and genomic prediction steps is needed.

Keywords: 2026

How to Cite:

Cheng, H., Feldmann, M., Watson, M. & Yu, H., (2026) “End-to-end Genomic Prediction: Predicting Images and Text from Genetic Markers”, World Congress on Genetics Applied to Livestock Production Digital Archive 2026(1): 2273588. doi: https://doi.org/10.31274/wcgalp.23416

Rights: 1