Take the following as truth: Reading between the lines, it may be that the child's biological father and your husband's own offspring-situation are part of the equation.
Then the following statement: "Your child's birth father and your husband's children could be part of the problem." is true, false, or inconclusive?