Does 'Because my family is by no means rich, but we never had that much trouble, so--.' offer a contrasting position to 'We never had much trouble in our middle class family, alas--'
Because my family is by no means rich, but we never had that much trouble, so--.###We never had much trouble in our middle class family, alas--