The traditional one-directional neural network has some problems in the field of ultra-short-term power prediction in distribution networks, such as the out-of-shape curve prediction, the over-fitting phenomenon of the model, low prediction accuracy and slow convergence speed, etc. Thus, an improved bi-directional recurrent neural network model is proposed based on the wavelet transform and self-attention mechanism to overcome these problems. Firstly, the forward and reverse laws of the power data are studied by the bi-directional network to improve the prediction accuracy of the model. Afterward, the wavelet transform is employed to reduce the overall difficulty of power prediction. Consequently, the model overfitting is reduced, and the convergence speed is increased in the meantime. In the end, the self-attention mechanism is adopted to grasp the hidden layer dimensional relationship of the model to further improve the prediction accuracy. An example shows that the proposed improved model can eliminate the existing problems effectively. Compared with the traditional model, the MAE increased by 50.1%, MAPE increased by 43.3%, RMSE increased by 51.1%; in the reactive dataset, dataset MAE increased by 60.5%, MAPE increased by 63.8%, and RMSE increased by 60.1%.