"He had been the first Territorial Governor of Oregon (after Lincoln declined) "
No Mr. Walsh,   Lincoln did not "decline" the position of first Territorial Governor  of Oregon.   In 1848  Lane, a D,  was appointed Territorial Governor  by Democratic President Polk who would never have dreamed  of appointing Lincoln, a Whig.  Later Lincoln  campaigned heavily for the Whig  presidential candidate Gen. Zachary Taylor with the hope he'd be appointed  head  of the Government Land Office.
While Taylor won, he only proposed Lincoln as Oregon's Secretary of State, which he declined; Taylor then offered the  territorial governor's position but Lincoln declined that also.  Taylor then in 1850  appointed a fellow Whig, John Gaines, who proved to be rather unpopular in this largely Democratic state.
Why did Lincoln reject Taylor's offer  to be our second Territorial Governor?  Possibly because he'd have no political future in a largely D state, or  because Mary didn't want to "Go  way out West."