The name comes from Sir George Downing, who left money and land in his will to build a college when he died in 1749. However, not everyone in the family agreed with this. So, for many years until 1800 there were fights in courts until the judges decided that the will should be followed.