Jansons was born in Riga, Latvia—then under German occupation during the Second World War—to Iraida Jansons, the diva of the Riga Opera, and Arvīds Jansons, conductor of the opera orchestra. Iraida, who was Jewish, gave birth to her son in hiding after being smuggled out of the Riga Ghetto, where her father and brother were murdered by the Nazis. As a child, Jansons first studied violin with his father.
In 1956, Arvīds was appointed assistant conductor to Yevgeny Mravinsky at the Leningrad Philharmonic. Jansons joined his father in Leningrad, where he began to study conducting, and soon entered the Leningrad Conservatory. In 1968, Herbert von Karajan visited the Soviet Union, and he singled out Jansons and Dmitri Kitayenko from a group of young conductors. Karajan offered Jansons the opportunity to study with him in Berlin, but the Soviet authorities blocked the offer. In 1969, Jansons continued his training with Hans Swarowsky in Vienna, and then in Salzburg with Karajan. In 1971, Jansons won the second prize at the "Herbert von Karajan" International Conducting Competition. Karajan invited Jansons to be his assistant with the Berlin Philharmonic, but the Soviet authorities blocked Jansons from ever hearing about the offer.