The unorganized territory, some of which had been surveyed into townships, together with parts of Victoria County and Nipissing District, was withdrawn in 1868-1869 to form the new District of Muskoka and District of Parry Sound. However, any municipalities established in them still formed part of the original counties for municipal purposes, and they were still responsible for the administration of justice, which proved to be problematic. The portions of the districts that were not part of Simcoe County were transferred to it in 1877. This lasted until 1888, when the territory contained within the districts was separated from Simcoe County.