Around this time, family complications arose. In order to vest his grandson Ottavio Farnese with the dukedom of Camerino, Paul forcibly wrested the same from the duke of Urbino (1540). He also incurred virtual war with his own subjects and vassals by the imposition of burdensome taxes. Perugia, renouncing its obedience, was besieged by Paul's son, Pier Luigi, and forfeited its freedom entirely on its surrender. The burghers of Colonna were duly vanquished, and Ascanio was banished (1541). After this the time seemed ripe for annihilating heresy.