Abstract: In order to solve the problem of insufficient consideration of Duncan's nonlinear strength criterion for the biting force of rockfill under extremely low confining pressure, “three-parameter method” is used to improve the nonlinear strength index formula of the Duncan logarithmic relationship and combine with the classical Mohr-Coulomb model to form a nonlinear strength criterion. The improvement is also implemented in the finite difference software FLAC3D. The soil strength reduction method is also proposed, while the nonlinear strength criterion is adopted to analyze the slope stability. Two examples are presented. One is a homogeneous high fill slope filled with broken limestone and the other is a high fill slope on inclined rock ground. The difference between the slope stability safety factor and the most dangerous sliding surface calculated by the "three-parameter method", the "stress circle tangent method" and the linear strength criterion are studied by calculation and comparison. The results of the finite difference strength reduction method for a homogeneous rockfill slope are also compared with those of the simplified Bishop method. The results show that for homogeneous rockfill slope, the safety factor calculated by the finite difference strength reduction method based on FLAC3D is very close to that calculated by the simplified Bishop method, which indicates the feasibility to introduce the nonlinear strength index into finite difference strength reduction method. The three-parameter method can be applied to high fill slopes with large slope height ranges to obtain reasonable stability analysis results. When the slope height is not more than 20 m, it is not suitable to use the stress circle tangent method to analyze the stability of rockfill slopes. When the slope height range is relatively small, the linear strength criterion can be used to analyze the stability of the rockfill slope.