Levine was a member of the notorious Mafia gang, Murder, Inc., and is credited with being the trigger man, with Dutch Schultz lieutenant Abraham "Bo" Weinberg, in the 1931 murders of Joe "The Boss" Masseria and, along with Joe Adonis, Albert "Mad Hatter" Anastasia and Benjamin "Bugsy" Siegel, one of the three hitmen sent by Meyer Lansky to assassinate the Sicilian Mafia boss Salvatore Maranzano in his office. They managed to enter by posing as government agents. Once inside Maranzano's office on the 9th floor of The Helmsley Building, they disarmed the guards and shot and stabbed Maranzano to death.