Arp was born in Strasbourg, the son of a French mother and a German father, during the period following the Franco-Prussian War when the area was known as Alsace-Lorraine (Elsass-Lothringen in German) after France had ceded it to Germany in 1871. Following the return of Alsace to France at the end of World War I, French law determined that his name become Jean. Arp would continue referring to himself as "Hans" when he spoke German.
In 1904, after leaving the École des Arts et Métiers in Strasbourg, he went to Paris where he published his poetry for the first time. From 1905 to 1907, Arp studied at the Kunstschule in Weimar, Germany, and in 1908 went back to Paris, where he attended the Académie Julian. Arp was a founder-member of the Moderne Bund in Lucerne, participating in their exhibitions from 1911 to 1913.
In 1912, he went to Munich, called on Wassily Kandinsky, the influential Russian painter and art theorist, was encouraged by him in his researches and exhibited with the Der Blaue Reiter group. Later that year, he took part in a major exhibition in Zürich, along with Henri Matisse, Robert Delaunay and Kandinsky. In Berlin in 1913, he was taken up by Herwarth Walden, the dealer and magazine editor who was at that time one of the most powerful figures in the European avant-garde.
In 1915, he moved to Switzerland to take advantage of Swiss neutrality. Arp later told the story of how, when he was notified to report to the German consulate, he avoided being drafted into the German Army: he took the paperwork he had been given and, in the first blank, wrote the date. He then wrote the date in every other space as well, then drew a line beneath them and carefully added them up. He then took off all his clothes and went to hand in his paperwork.