Plummer was born on December 13, 1929 in Toronto, Ontario. He is the only child of Isabella Mary (née Abbott), an artist who was secretary to the Dean of Sciences at McGill University, and John Orme Plummer, who sold stocks and securities. His great-grandfather on his mother's side was Canadian Prime Minister Sir John Abbott. His father's uncle was patent lawyer and agent F. B. Fetherstonhaugh. Plummer is a second cousin of British actor Nigel Bruce, known as Doctor Watson to Basil Rathbone's Sherlock Holmes. His parents divorced shortly after his birth, and he was brought up in the Abbott family home in Senneville, Quebec outside of Montreal. He speaks both English and French fluently.
Plummer began studying to be a concert pianist, but he developed a love for theatre at an early age, and he began acting while he was attending the High School of Montreal. He attended McGill where he also took up acting, after watching Laurence Olivier's film Henry V (1944). In 1946, he caught the attention of Montreal Gazette's theatre critic Herbert Whittaker with his performance as Mr. Darcy in the Montreal High School production of Pride and Prejudice. Whittaker was also amateur stage director of the Montreal Repertory theatre, and he cast Plummer at age 18 as Oedipus in Jean Cocteau's La Machine infernale.