Ms Yellen said that a rise in interest rates before the end of the year was "a live possibility".
In Japan, the benchmark Nikkei index closed up 1% at 19,116.41.
Takata shares dived 25% as both Mitsubishi Motors and Fuji Heavy Industries said they were considering stopping using its airbag inflators.
Investors will be watching shares of Toyota on Friday after the firm reported a rise in second-quarter profits but trimmed its annual sales target.
While shares in electronics firm Toshiba closed down by more than 3% after it confirmed a local media report that it would report its first operating loss in six years as a result of its accounting scandal.
The firm's six-month earnings report is expected on Saturday.
Elsewhere, Chinese shares were mixed, with the Shanghai Composite closing up 1.8% at 3,522.82, while Hong Kong's Hang Seng index closed flat at 23,051.04.
The benchmark indexes had rallied up to 4% on Wednesday on the back of positive comments about the economy's growth from President Xi Jinping.
Australia's S&P/ASX 200 closed down 0.9% at 5,193.00, while in South Korea the Kospi index ended 0.2% lower at 2,049.42.
Asian markets experienced mixed fortunes as investors digested comments from Federal Reserve chair Janet Yellen on the timing of a US rate rise.