Duvivier learned as a teenager violin and Broad composition teaching. He played from 1937 at the Royal Baron Orchestra. He then worked with Coleman Hawkins in 1941 and until 1942 at Lucky Millinder. From 1943 to 1945, he had to serve his military service and arranged in this time for Jimmy Lunceford, where he worked until he moved to Sy Oliver 1947. In the early 1950s he worked as an accompanist of singers such as Nellie Lutcher, Pearl Bailey, Billy Eckstine, and Lena Horne, but was with Terry Gibbs and until 1956 when Bud Powell and Herbie Nichols ( "Love, Gloom, Cash, Love" 1957) and Eubie Blake ( 1959) worked. In the early 1960s he played with Oliver Nelson, Eric Dolphy and Mal Waldron, but has also performed at various jazz festivals in the U.S. and Europe. The number of his LP recordings is unmanageable - adjacent to Ray Brown he was probably the meistbeschäftigste musician on his instrument at all. His sense of harmony and the essential jazz "time" was proverbial.