Marcus "Mark" Laurence Elwin Oliphant was born on 8 October 1901 in Kent Town, a suburb of Adelaide. His father was Harold George "Baron" Oliphant, a civil servant with the South Australian Engineering and Water Supply Department and part-time lecturer in Economics with the Workers' Educational Association. His mother was Beatrice Edith Fanny Oliphant, née Tucker, an artist. He was named after Marcus Clarke, the Australian author, and Laurence Oliphant, the British traveller and mystic. Most people called him Mark; this became official when he was knighted in 1959. He had four younger brothers, Roland, Keith, Nigel and Donald. His parents were theosophists, and as such were opposed to eating meat. Marcus became a lifelong vegetarian while a boy, after witnessing the slaughter of pigs on a farm. He was found to be completely deaf in one ear and he needed glasses for severe astigmatism and short-sightedness.
Oliphant was first educated at primary schools in Goodwood and Mylor, after the family moved there in 1910. He attended Unley High School in Adelaide, and, for his final year in 1918, Adelaide High School. After graduation he failed to obtain a bursary to attend university, so he took up a job cleaning floors for a jewellery manufacturer. He then secured a cadetship with the State Library of South Australia, which allowed him to take courses at the University of Adelaide at night.
In 1919, Oliphant began studying at the University of Adelaide. At first he was interested in a career in medicine, but later in the year Kerr Grant, the physics professor, offered him a cadetship in the Physics Department. It paid 10 shillings a week (equivalent to AUD$34 in 2010), the same amount that Oliphant received for working at the State Library, but it allowed him to take any university course that did not conflict with his work for the department. He received his Bachelor of Science (BSc) degree in 1921 and then did honours the following year, supervised by Grant. Roy Burdon, who acted as head of the department when Grant went on sabbatical in 1925, worked with Oliphant to produce two papers in 1927 on the properties of mercury, "The Problem of the Surface Tension of Mercury and the Action of Aqueous Solutions on a Mercury Surface" and "Absorption of Gases on the Surface of Mercury". Oliphant later recalled that Burdon taught him "the extraordinary exhilaration there was in even minor discoveries in the field of physics".
Oliphant married Rosa Louise Wilbraham, who was also from Adelaide, on 23 May 1925. The two had known each other since they were teenagers. He made Rosa's wedding ring in the laboratory from a gold nugget (from the Coolgardie Goldfields) that his father had given him.