Smythe was born on February 1, 1895, in Toronto to Albert Smythe, an Irish Protestant from County Antrim who immigrated to Canada in 1889, and Mary Adelaide Constantine, an English woman. Mary and Albert were married in the 1880s while immigrating to Canada, but their marriage was rocky and they did not live together for more than a few months at a time. Conn was the second of the couple's two children; he had a sister, Mary, five years older, who died due to illness in 1903. Smythe remembered his mother Mary, who was known as Polly, as pretty, a drinker, and troublemaker, while Albert was quiet, a vegetarian, and a devoted member of Madame Blavatsky's Theosophical movement. Albert Smythe was a charter member of the Theosophical Society of Canada in 1891, and edited its newsletter until the final years of his life.
Smythe's first home was 51 McMillan Street, now known as Mutual Street, not far from the future site of Maple Leaf Gardens. The family was poor and moved several times during Smythe's youth, the size of lodgings depending on Albert Smythe's wages at the time. At one point, Albert and Conn moved to a house in Scarborough while Polly and Mary stayed on North Street. Mary died in 1906, and Smythe attributed his lifelong teetotalism to his mother's drinking. At age eleven, Conn was christened, the occasion marking the first time that he insisted on the name "Conn" instead of his given name, Constantine. Albert and Conn became estranged after Albert began a new relationship with Jane Henderson. The two married in 1913 and had a daughter, Moira.
Smythe first attended high school at Upper Canada College, but disliked it and transferred to Jarvis Collegiate Institute after a year and a half. He developed his athleticism there, playing on the hockey, rugby football, and basketball teams, and playing on city championship teams in basketball and hockey in 1912. At the age of 16, Smythe met Irene Sands, his future wife, after a football game against Parkdale Collegiate Institute, which she attended. Albert Smythe wanted his son to attend university, but Conn defied his father, bolting at age 17 to become a homesteader on 150 acres (61 ha) in Clute Township, near Cochrane, Ontario. After one summer building a home on the property only to have it destroyed by a devastating fire, Smythe returned home and enrolled in engineering studies at the University of Toronto in the fall of 1912. There he played hockey as a centre, captaining the Varsity Blues men's ice hockey team to the finals of the 1914 Ontario Hockey Association (OHA) junior championships and to the OHA junior championship the following year. The coach of the losing team in 1915 was Frank J. Selke, who years later would work for Smythe at Maple Leaf Gardens. Smythe also played on the University of Toronto football team, although not as a starter.