Genetic improvement has promoted wheat’s grain yield and nitrogen use efficiency (NUE) during the past decades. Therefore, the current wheat cultivars exhibit higher grain yield and NUE than previous cultivars in the Yangtze River Basin, China since the 2000s. However, the critical traits and mechanisms of the increased grain yield and NUE remain unknown. This study explores the mechanisms underlying these new cultivars’ increased grain yield and NUE by studying 21 local cultivars cultivated for three growing seasons from 2016 to 2019. Significantly positive correlations were observed between grain yield and NUE in the three years. The cultivars were grouped into high (HH), medium (MM), and low (LL) grain yield and NUE groups. The HH group exhibited significantly high grain yield and NUE. High grain yield was attributed to more effective ears by high tiller fertility and greater single-spike yield by increasing postanthesis single-stem biomass. Compared to other groups, the HH group demonstrated a longer leaf stay-green ability and a greater flag leaf photosynthetic rate after anthesis. It also showed higher N accumulation at pre-anthesis, which contributed to increasing N accumulation per stem, including stem and leaf sheath, leaf blade, and unit leaf area at preanthesis, and promoting N uptake efficiency, the main contribution of high NUE. Moreover, tiller fertility was positively related to N accumulation per stem, N accumulation per unit leaf area, leaf stay-green ability, and flag leaf photosynthetic rate, which indicates that improving tiller fertility promoted N uptake, leaf N accumulation, and photosynthetic ability, thereby achieving synchronous improvements in grain yield and NUE. Therefore, tiller fertility is proposed as an important kernel indicator that can be used in the breeding and management of cultivars to improve agricultural efficiency and sustainability.