Paul was speaking to Christians in general:
From a commentary: Allusion is made to the cruel spectacles in the Roman games. The thought that many eyes are upon believers, when struggling with difficulties or temptations, should encourage constancy and patience. Whatever the disciples of Christ suffer from men, they must follow the example, and fulfil the will and precepts of their Lord. They must be content, with him and for him, to be despised and abused. It is much better to be rejected, despised, and ill used, as Paul was, than to have the good opinion and favour of the world. Though cast off by the world as vile, yet we may be precious to God, gathered up with his own hand, and placed upon his throne. (My comment: Paul-like Christians, with God's Spirit within, are a spectacle to the universe; a peculiar spectacle ;)