the emperor of byzantium did all he could to maintain power within the walls of his kingdom , but his prosperity was due to mingling trade routes and the port city could scarcely be called an empire at all .