Acoustic vector sensor (AVS) is a kind of sensor widely used in underwater detection. Traditional methods use the covariance matrix of the received signal to estimate the direction-of-arrival (DOA), which not only loses the timing structure of the signal but also has the problem of weak anti-noise ability. Therefore, this paper proposes two DOA estimation methods for underwater AVS arrays, one based on a long short-term memory network and attention mechanism (LSTM-ATT), and the other based on Transformer. These two methods can capture the contextual information of sequence signals and extract features with important semantic information. The simulation results show that the two proposed methods perform much better than the multiple signal classification (MUSIC) method, especially in the case of low signal-to-noise ratio (SNR), the DOA estimation accuracy has been greatly improved. The accuracy of the DOA estimation method based on Transformer is comparable to that of the DOA estimation method based on LSTM-ATT, but the computational efficiency is obviously better than that of the DOA estimation method based on LSTM-ATT. Therefore, the DOA estimation method based on Transformer proposed in this paper can provide a reference for fast and effective DOA estimation under low SNR.