Over the course of the next three seasons, Durham would endure successes but fall short of the ultimate goal, including the 1981 defeat at the hands of the Mitchell Red Devils, in the Intermediate "B" final. During the summer of 1981, the league underwent a major restructuring. The disbandment of two other local leagues (Central Grey-Bruce League, CGBHL and the Central Ontario Hockey League, COHL) increased the team numbers in The WOAA and required the creation of the Upper Tier Intermediate A Division and the Lower Tier Intermediate D Division.