Current methods used in genome-wide association studies frequently lack power owing to their inability to detect heterogeneous associations and rare and multiallelic variants. To address these issues, quantile regression is integrated with a three (compressed) variance component multi-locus random-SNP-effect mixed linear model (3VmrMLM) to propose q3VmrMLM for detecting heterogeneous quantitative trait nucleotides (QTNs) and QTN-by-environment interactions (QEIs), and then design haplotype-based q3VmrMLM (q3VmrMLM-Hap) for identifying multiallelic haplotypes and rare variants. In Monte Carlo simulation studies, q3VmrMLM had higher power than 3VmrMLM, sequence kernel association test (SKAT), and integrated quantile rank test (iQRAT). In a re-analysis of 10 traits in 1439 rice hybrids, 261 known genes were identified only by q3VmrMLM and q3VmrMLM-Hap, whereas 175 known genes were detected by both the new and existing methods. Of all the significant QTNs with known genes, q3VmrMLM (179: 140 variance heterogeneity and 157 quantile effect heterogeneity) found more heterogeneous QTNs than 3VmrMLM (123), SKAT (27), and iQRAT (29)
q3VmrMLM-Hap (121) mapped more low-frequency (<
0.05) QTNs than q3VmrMLM (51), 3VmrMLM (43), SKAT (11), and iQRAT (12)
and q3VmrMLM-Hap (12), q3VmrMLM (16), and 3VmrMLM (12) had similar power in identifying gene-by-environment interactions. All significant and suggested QTNs achieved the highest predictive accuracy (r = 0.9045). In conclusion, this study describes a new and complementary approach to mining genes and unraveling the genetic architecture of complex traits in crops.