However, in the years following, the ownership of a variety of estates, including Norton Beauchamp, to which Steep Holm was attached, was disputed. At the heart of this was the dispute from how the Barony was passed on from Thomas Berkeley, the 5th Baron. These disputes had on one side James Berkeley, 1st Baron Berkeley, also known as 'James the Just' (not to be confused with the other 1st Baron Berkeley Thomas de Berkeley, 1st Baron Berkeley who pre-dated him by 150 years). Yet it was to James whom the Barony transferred to under a new creation by writ since his uncle, the aforementioned Thomas de Berkeley (5th Baron) had no male heirs eventhough he had named his only daughter (Elizabeth Berkeley) as his heir. This was to be the start of a long standing legal dispute that lasted for some time.