garnet always maintained that it was a case of envy over progress : the port authority were the largest tenants of the world trade center and had a vested interest in maintaining their own building as the superlative structure on the waterfront .