The cause of his death was not immediately clear. He was in a hospital being treated for exhaustion.
A rare artist to have won a Grammy in jazz, pop and R&B categories, Jarreau was famed for writing the theme to hit television show Moonlighting.
Earlier this month, he retired from touring after more than 50 years.
Born in Milwaukee, Wisconsin, in 1940, he started singing as a child, joining his family in church concerts and events.
But his career did not kick off until years later - first, he graduated in psychology and worked at a rehabilitation centre in San Francisco.
His first album, We Got By, came out in 1975 and earned him a German Grammy.
One year later, he launched Glow, with which he also won a German Grammy. In 1977, he released a live album, Look to the Rainbow, which earned him his first Grammy for best jazz singer.
He was awarded again one year later, this time for the album All Fly Home.
He then crossed over into pop, and in 1981, launched Breakin Away, which included the hit We're in This Love Together, which catapulted him to fame. Another Grammy came in, this time as best pop singer.
Finally, his 1992 Heaven and Earth album earned him a Grammy for best R&B vocal performance, meaning he had prizes in the three categories.
According to a post on the star's Twitter profile, his son had recently caught Jarreau singing Moonlighting to one of the nurses in hospital.
Jarreau had suffered health issues in recent years and was hospitalised in 2010 for respiratory problems when touring in France.
A statement on his website read: "His 2nd priority in life was music. There was no 3rd. His 1st priority, far ahead of the other, was healing or comforting anyone in need.
"Whether it was emotional pain, or physical discomfort, or any other cause of suffering, he needed to put our minds at ease and our hearts at rest."
Seven-time Grammy Award-winning jazz and pop singer Al Jarreau has died in Los Angeles at the age of 76, his publicist says.