Noriyoshi began as a servant in the Bansho shirabesho (a government-run Western studies education and research institute) and, by good fortune, became a jikisan (immediate retainer) of the bakufu (Japanese feudal government headed by a shogun) by buying a kabu (a right to become doshin) of doshin (a low-ranked official) in 1867.
則義は蕃書調所の小使いから1867年（慶応3年）には同心株を買い、運良く幕府直参となった。