The result is seen as a victory for President Mohamed Ould Abdel Aziz, who is accused by his rivals of trying to extend his mandate, which he denies.
He called the referendum after the Senate rejected his proposals to change the constitution.
Turnout was 53.73%, with 85% of voters supporting the change, officials said.
But members of the opposition denounced an "electoral farce which has given way to open-air fraud".
Mr Abdel Aziz, who described the Senate as "useless and too costly", said the move to abolish it would improve governance by introducing more local forms of lawmaking.
He is barred by the constitution from running a third term - he came to power in a coup in 2008, was elected president the following year and won a second term in 2014.
The president is a key ally of the West and neighbouring states in countering Islamist extremist groups.
In the same referendum, held on Saturday, voters also supported a change to the national flag.
The current green flag with yellow Islamic crescent and star will also feature red bands to honour the blood spilt by those who fought for freedom from France.
Mauritanians have voted to abolish the upper house of their parliament, the Senate, in a controversial referendum that was boycotted by the opposition.