At least two members of the board were planning to travel but owner Vincent Tan has instructed them not to and they are said to support his position.
Wigan owner Dave Whelan was fined £50,000 for making racist comments about Jewish and Chinese people and served a six-week ban from football.
Wigan declined to comment on Tuesday.
The Football Association disciplinary commission that punished Latics owner Whelan said it was "satisfied" he "is not a racist" and that he had not intended to cause offence.
The 78-year-old's comments were made as he defended the appointment of Malky Mackay as the new Latics manager in November.
Mackay is currently under investigation by the FA regarding allegations he sent "sexist, racist and homophobic" text messages during his time in charge of Cardiff.
The Scot admitted sending offensive messages and apologised, adding that he is not racist, sexist, homophobic or anti-Semitic.
Mackay was sacked by the Bluebirds in December 2013.
Cardiff City's board members will boycott Tuesday's Championship game at Wigan in protest at what they allege to be a club owned and managed by racists.