During the late 1950s and early 1960s, Brown was one of a few African-Americans practicing law in San Francisco when he opened his own business. He practiced criminal defense law, representing pimps, prostitutes and other clients that more prominent attorneys would not represent. One early case was to defend Mario Savio on his first civil disobedience arrest. He quickly became involved in the Civil Rights Movement, leading a well-orchestrated sit-in to protest housing discrimination after a local real estate office refused to work with him because of his race. Brown helped organize the public protest and attract media coverage. His role in the protests gave him the notability to run for the State Assembly.