Heterosis is a phenomenon where the offspring of genetically distinct populations exhibit superior vitality, reproductive capacity, and adaptability compared with the average of their parent populations, which is an important genetic resource. Heterosis plays a significant role in modern agriculture, contributing to increase yields and quality of livestock and crops, rapidly improve traits, accelerate the breeding of new varieties, and enhance genetic diversity, thereby efficiently boosting the production of animal husbandry and agriculture while reducing costs. Despite the discovery of heterosis is over a century ago, the elucidation of its genetic basis lags far behind its application in agricultural production. The study of the complex formation mechanism of heterosis is a classic and an active topic in the field of genetics and breeding, but the clear conclusions remain limited. In response to the characteristics of heterosis, scientists have successively proposed various hypotheses for its formation, such as the dominance hypothesis, overdominance hypothesis, and epistasis hypothesis, revealing that the genetic basis of heterosis was non-additive genetic effects. However, these hypotheses are based on the effects of single genes, which are overly idealized and simplistic. Explorations at different levels, such as DNA, RNA, and proteins, have successively discovered the coexistence of multiple genetic effects. Particularly in hybrid crops like rice and corn, the related researches have been continuously identified the loci of heterosis effects, enriched the understanding of the formation mechanism for heterosis in crops, and promoted the transformation of crop breeding technologies, such as precise molecular design breeding. Heterosis is also widely applied in the breeding of livestock and poultry. In developed countries with advanced animal husbandry, over 80% of commercial pork, chicken, and eggs are obtained from hybrid breeds. To efficiently apply heterosis in production for animal husbandry, it is necessary to predict heterosis in advance. New methods, such as the inter- and intra-group phenotypic variance ratio prediction, hybrid heritability prediction, and molecular marker prediction, have been developed to solve the long experimental cycle, environmental sensitivity, and high human and financial costs associated with traditional hybridization experiments for predicting heterosis. However, the accuracy of these prediction methods is limited. Heterosis involves in interaction of multiple levels, and because of the complex genetic background and long breeding cycle, it is still a big challenge for the study of the heterosis formation mechanism and accurate prediction methods. In recent years, the gradual application of sequencing technology has provided a new perspective for understanding the molecular regulatory network of heterosis in livestock and poultry. QTL mapping and genome-wide association study reveal the molecular mechanism of heterosis at the genomic level, and the identified molecular makers are applied in selection and breeding. Combined with multi-omics researches, such as transcriptomics and metabolomics, the key functional genes, variations, and metabolites affecting heterosis can be more precisely located, which facilitate hybrid improvement. This review elaborated the research progress in the formation mechanism and prediction methods for heterosis in the field of livestock and poultry. For looking forward to future, the researches will gradually clarify the complex mechanism of heterosis by integrating multi-omics sequencing data and bioinformatics analysis, in order to identify genes and molecular markers related to heterosis, and innovate new prediction methods, which will provide a more accurate direction for the utilization of heterosis.