Moving to New York City, Fisher joined the Wall Street investment bank Brown Brothers, Harriman and Company, where he was assistant to former Undersecretary of the Treasury Robert V Roosa, specializing in fixed income and foreign exchange markets. From 1978 to 1979, he served as Special Assistant to Secretary W. Michael Blumenthal at the United States Department of the Treasury, where he worked on issues relating to the dollar crisis. Returning to Brown Brothers, he established and managed the bank's Dallas-based Texas operations.