Phenomics

Data structure to facilitate behavior phenotyping of pigs using foundation vision-language models

---

Abstract

Accurate behavioral tracking and phenotyping of group-housed pigs are increasingly recognized as essential for genetic improvement and precision livestock management. Behavioral and interaction traits can serve as heritable indicators of production efficiency and animal welfare. However, generating such phenotypes from video data requires extensive storage, annotation, and computation, creating a major bottleneck for genetic studies of digital phenotypes. Foundation vision-language models (FVLMs) enable automated annotation and context-aware tracking. Recently, we developed a long-term pig tracking system using FVLMs applied directly to unseen datasets, without manual annotation or task-specific fine-tuning. The system can identify individual animals and segment their body contours frame by frame, but the direct output remains massive and inefficient for long-term storage or population-scale applications.To address this challenge, we designed a hierarchical data structure that progressively reduces the data footprint while retaining biologically relevant information. Starting with the segmentation masks generated by the tracking system, we applied principal component analysis (PCA) to each pig's mask to identify its centroid (mean pixel location), major axis (direction of maximal spatial variance), and elongation (ratio between the largest and smallest eigenvalues of the covariance matrix), which together capture the pig's overall body geometry. The head-tail orientation was inferred by combining shape asymmetry with short-term motion direction across consecutive frames. By projecting all mask pixels onto the head-tail axis, we divided each pig into three equal-length segments: front, middle, and end. We then quantified the pairwise distances between pigs at the body-part level. For each frame, the algorithm first used k-d tree search to rapidly filter out distant pigs and applied distance transforms to measure minimal Euclidean distances between the body parts of neighboring pigs. A final reduction was to keep only the records for the pigs that were in contact.We tested this data structure design on a two-hour video recorded from a pen containing ten nursery pigs. The tracking system demonstrated strong performance: among 1,306 manually annotated masks, 1,293 were correctly matched, yielding high overall accuracy (J = 0.83, F = 0.92, J&F = 0.87, MOTA = 99%, MOTP = 90.7%) with only 13 missed detections and no false positives or ID switches. The hierarchical data structure efficiently compressed the dataset, reducing raw RGB video (5865 MB) to segmentation masks (1063 MB), then to compact geometric feature files (368 MB) and finally to close-contact records (77 MB), achieving a 75à— reduction while preserving the capacity to reconstruct spatial relationships, postural states, and contact events.In conclusion, this data-efficient structure bridges foundation-model-based tracking with a hierarchical behavioral data representation, enabling scalable, automated analysis for large pig populations in genetics and welfare research.

Keywords: 2026

How to Cite:

Bi, Y., Shumaly, M., Acharya, B., Rosero, D. & Steibel, J., (2026) “Data structure to facilitate behavior phenotyping of pigs using foundation vision-language models”, World Congress on Genetics Applied to Livestock Production Digital Archive 2026(1): 2285850. doi: https://doi.org/10.31274/wcgalp.23808

Rights: 1