Murphy was born in Sydney, and attended Sydney Boys High School before going on to the University of Sydney. He initially graduated with a degree in chemistry, but then went on to Sydney Law School and eventually became a barrister. He specialised in labour and industrial law, and took silk in 1960. Murphy was elected to the Senate at the 1961 federal election, as a member of the Labor Party. He became Leader of the Opposition in the Senate in 1967.