Harsanyi was born on May 29, 1920 in Budapest, Hungary, the son of Alice Harsányi (née Gombos) and Károly Harsányi, a pharmacy owner. His parents converted from Judaism to Catholicism a year before he was born. He attended high school at the Lutheran Gymnasium in Budapest. In high school, he became one of the best problem solvers of the KöMaL, the Mathematical and Physical Monthly for Secondary Schools. Founded in 1893, this periodical is generally credited with a large share of Hungarian students' success in mathematics. He also won the first prize in the Eötvös mathematics competition for high school students.
Although he wanted to study mathematics and philosophy, his father sent him to France in 1939 to enroll in chemical engineering at the University of Lyon. However, because of the start of World War II, Harsanyi returned to Hungary to study pharmacology at the University of Budapest (today: Eötvös Loránd University), earning a diploma in 1944. As a pharmacology student, Harsanyi escaped conscription into the Hungarian Army which, as a person of Jewish descent, would have meant forced labor.
However, in 1944 (after the fall of the Horthy regime and the seizure of power by the Arrow Cross Party) his military deferment was cancelled and he was compelled to join a forced labor unit on the Eastern Front. After seven months of forced labor, when the German authorities decided to deport his unit to a concentration camp in Austria, John Harsanyi managed to escape and found sanctuary for the rest of the war in a Jesuit house.