Karl Oyston made the claim as he defended himself in a legal battle with Valeri Belokon, who has made counter claims against the chairman and his father Owen, the club's owner.
Mr Belokon claimed at London's High Court that they "improperly" extracted millions of pounds following promotion to the Premier League in 2010.
The pair have denied the allegations.
VB Football Assets, which is owned by Mr Belokon and is a minor shareholder in the club, is pursing a claim against the Oystons for "unfair prejudice" against shareholders.
Mr Oyston said neither the value nor the "success" of the club had been prejudiced by him or his father and all payments had been sanctioned by the board.
In a written statement to the court, he said: "It is a fact that the fortunes... have dwindled and will continue to do so, because of the fall of its fortunes on the pitch."
He said the trouble on the pitch had been added to "by the actions of a small, but highly vocal and active, group of supporters, who object to myself and my father".
"I believe that [Mr Belokon] has been instrumental in inciting those fans to cause trouble," he added.
Mr Oyston told Mr Justice Marcus Smith that a supporters group committee, "which was entertained by VB in Riga, published both my home and mobile numbers on the internet, causing me to receive over 4,000 abusive and threatening calls and text messages".
He added: "These... included threats to my wife and children and protests were and continually are held at our home, where criminal damage and trespass has occurred."
Andrew Green QC, appearing for Mr Belokon, suggested to Mr Oyston that he was the son of a wealthy and powerful man who did not take kindly to anyone standing up to him.
Mr Green asked: "Are you a bully?"
"Not in the slightest," Mr Oyston replied.
Mr Green also suggested there was no evidence to support Mr Oyston's allegations of incitement and asked him if he wished to withdraw his "grave allegations" in court.
"No, I don't," Mr Oyston said.
The case continues.
Blackpool's president has incited fans "to make trouble", the football club's chairman has told a court.