The double-rice cropping system is a very important intensive cropping system for food security in China. There have been few studies of the sustainability of yield and accumulation of soil organic carbon (SOC) in the double-rice cropping system following a partial substitution of chemical fertilizer by Chinese milk vetch (Mv). We conducted a 10-year (2008–2017) field experiment in Nan County, South-Central China, to examine the double-rice productivity and SOC accumulation in a paddy soil in response to different fertilization levels and Mv application (22.5 Mg ha–1). Fertilizer and Mv were applied both individually and in combination (sole chemical fertilizers, Mv plus 100, 80, 60, 40, and 0% of the recommended dose of chemical fertilizers, labeled as F100, MF100, MF80, MF60, MF40, and MF0, respectively). It was found that the grain yields of double-rice crop in treatments receiving Mv were reduced when the dose of chemical fertilizer was reduced, while the change in SOC stock displayed a double peak curve. The MF100 produced the highest double-rice yield and SOC stock, with the value higher by 13.5 and 26.8% than that in the F100. However, the grain yields increased in the MF80 (by 8.4% compared to the F100), while the SOC stock only increased by 8.4%. Analogous to the change of grain yield, the sustainable yield index (SYI) of double rice were improved significantly in the MF100 and MF80 compared to the F100, while there was a slight increase in the MF60 and MF40. After a certain amount of Mv input (22.5 Mg ha–1), the carbon sequestration rate was affected by the nutrient input due to the stimulation of microbial biomass. Compared with the MF0, the MF100 and MF40 resulted in a dramatically higher carbon sequestration rate (with the value higher by 71.6 and 70.1%), whereas the MF80 induced a lower carbon sequestration rate with the value lower by 70.1% compared to the MF0. Based on the above results we suggested that Mv could partially replace chemical fertilizers (e.g., 40–60%) to improve or maintain the productivity and sustainability of the double-rice cropping system in South-Central China.