When the dominant species in a plant community are palatable, many believe that large herbivores will reduce the dominant species and promote the proportion of previously suppressed species. However, this view may not always hold true. We conducted a 4-year yak grazing experiment on the Qinghai-Tibet Plateau and tracked the plant compositions of the rotational grazing (RG) and grazing exclusion (GE) grasslands during the four years. The results showed that in the absence of yaks under GE, the plant community was dominated by two palatable species, Kobresia pygmaea and Stipa capillata, due to their small leaf area and rapid growth strategy. The presence of yaks under RG significantly inhibited S. capillata and over half of the forbs, while the proportion of K. pygmaea increased and it became the absolute dominant species, contradicting the view that large herbivores inhibit palatable species. Interannually, the dominance of K. pygmaea under RG decreased in the dry year, leading to an increase in the dominance of the other eight species. Under GE, the dominance of K. pygmaea declined notably in the dry year, while S. capillata and seven other forbs increased substantially. Overall, these results suggest that K. pygmaea is grazing-tolerant but not drought-tolerant, whereas the other eight species are drought-tolerant but not grazing-tolerant. At the community level, community composition shifts resulting from succession after grazing exclusion exceeded those caused by drought, drought tends to induce community species turnover while grazing tends to induce species abundance variations. In summary, our conclusions remind ranch managers that when considering the impact of livestock on plant community composition, they should factor in local conditions and climate change rather than simply assuming that livestock will suppress the palatable species.