In this paper, a fretting damage model based on fractal theory is proposed. The Weierstrass–Mandelbrot function of fractal theory is used to represent the rough contact surface, and a corresponding contact parameter analysis method is also established. Based on neural network algorithm, the values of fractal parameters are fitted, and the fitting accuracy has been greatly improved compared with traditional methods. According to the fractal parameters of the actual surface, the fretting wear process of the rough contact surface is analyzed based on theory of adhesive and three-body abrasive wear. A generic program for the analysis of three-dimensional fretting wear problems is also proposed. Compared with material tests, the prediction error of the fretting wear simulation model is 13.4% for wear depth and 16.7% and 3.9% for width and length of wear scar in stable wear stage. The prediction results show that the model can be applied to the prediction of the actual three-dimensional fretting wear model.