Jones studied piano in his youth with Daisy Sweeney, sister of Oscar Peterson, and spent much of his career working in pop and variety settings, including a period from 1964 to 1980 based in Puerto Rico with the singer Kenny Hamilton. Returning to Montréal, Jones turned to playing jazz exclusively and by 1987 had emerged internationally with the first of many concerts in Europe, Australia and the US. He drew critical notice for his technical dexterity and rollicking swing, often eliciting comparisons to Peterson.

Jones continued to travel extensively, on occasion under the aegis of the Canadian government, until he entered semiretirement in 1994; an African tour was documented by the National Film Board's Oliver Jones in Africa (1989). He was heard on 15 LPs and CDs recorded between 1982 and his semi-retirement from music in 1999, notably Lights of Burgundy, Cookin' at Sweet Basil (recorded at the New York club Sweet Basil), the solo piano Just 88 and From Lush to Lively (with big band and string orchestra 2006). Post-retirement recordings have included Just You, Just Me (with Ranee Lee, 2005), One More Time (with Dave Young and Jim Doxas, 2006) and Serenade (DVD, 2006).

Oliver Jones has received Félix Awards (presented by ADISQ, the L'Association du disque et de l'industrie du spectacle québécois) in 1989, 1994, 2007 and 2008 and Juno Awards in 1986, 1990, 2006 and 2009. His most recent album, Pleased to Meet You, was nominated for a Juno in 2010.

Oliver Jones was made an Officer of the Order of Canada in 1994 and in February 2013 his image was reproduced on a Canadian stamp, released as part of celebrations for Black History Month.