When Fischer moved to Hollywood in 1958, he went to East L.A. to play and learn more about Latin-Jazz. He started in a charanga group with Modesto Duran as leader and played with many different groups. On November 2, 1959, he made a particularly fruitful connection within this genre when he appeared with the Hi-lo's at the First Annual Los Angeles Jazz Festival on the same bill with the already popular Latin jazz group led by vibraphonist Cal Tjader. Both his playing and arranging made an immediate impression on Tjader, who went on to employ Fischer in both capacities on several recordings over the next three years; the two would reunite in the mid-seventies, leading eventually to the formation of Fischer's own Latin jazz unit, Salsa Picante.