Mr McCarthy, 49, will replace outgoing Republican Majority Leader Eric Cantor in the role.
Mr McCarthy is the current majority whip, the third-ranking member.
Mr Cantor, 51, unexpectedly lost a Republican Party primary election last week, throwing the party's House caucus into turmoil.
After his shock defeat in his Virginia constituency at the hands of David Brat, an economics professor, Mr Cantor said he would step down from his leadership position at the end of July.
Mr Brat will face a Democratic challenger in November's general election, and the winner of that race will assume Mr Cantor's seat in January.
Winning the leadership vote on Thursday places Mr McCarthy of California next in line to House Speaker John Boehner.
In the House of Representatives, the majority leader is tasked with scheduling legislation for consideration on the House floor, planning the legislative agenda, and consulting with party members to gauge sentiment.
The majority leader is also a highly visible representative of the party in negotiations with the White House and the Senate, both in Democratic hands.
In Thursday's party leadership vote, Mr McCarthy beat Congressman Raul Labrador for the role.
Republican Congressman Steve Scalise was subsequently elected to replace outgoing-Mr McCarthy as House majority whip.
The Republican Party is still reeling from Mr Cantor's surprise defeat on 11 June after nearly 14 years in the House.
Some Republicans have expressed concern over what Mr Cantor's defeat means for the party, which analysts say is struggling to broaden its appeal beyond its core base of conservative suburban and rural white voters.
US Congressman Kevin McCarthy has become the second-ranking member of the House of Representatives in a Republican caucus leadership election.