© 2016, Springer Science+Business Media Dordrecht. The influences of five different fertilizer treatments on diversity of rhizobia in soybean nodule were investigated in a long-term experiment with with four replicates: (1) control (without fertilization), (2) balanced NPK fertilizer (NPK), and (3–5) unbalanced chemical fertilizers without one of the major elements (NP, PK, and NK) in Mollisol in Northeast China. The highest soybean yield was observed in the NPK treatment. Total of 200 isolates were isolated and grouped into four Bradyrhizobium genospecies corresponding to B. japonicum, B. diazoefficiens, B. ottawaense and Bradyrhizobium sp. I, based upon the multilocus sequence analysis of 6 housekeeping genes. The Bradyrhizobium sp. I was extensively distributed throughout the study site and was recorded as the dominant soybean rhizobia (82.5–87.5%). Except the NK treatment, the other fertilizer treatments had no effect on rhizobial species composition. Compared with the CK treatment, all the fertilizer treatments decreased species richness, diversity and evenness. The soil organic carbon contents, available N content and pH were the key soil factors to rhizobial community structure. Results suggest that long-term fertilization can decrease rhizobial species diversity, while balanced fertilization with NPK is the most suitable fertilization regime if taking both soybean yields and rhizobial diversity into account.