Ponge was born into a Protestant family in Montpellier (South of France), the son of Armand Ponge, a banker, and his wife Juliette, née Saurel. He studied in Paris at the Sorbonne and the École de droit where he read law, In 1918–19 he served in the French army. In 1919 he joined the Socialist Party.
Ponge worked for the Parisian publishing companies Editions Gallimard (1923–31) and Hachette (1931–37), and before the outbreak of the Second World War he was briefly an insurance salesman. His earliest poems were published in 1923, and he established a reputation in French literary circles, principally for his contributions to the Nouvelle Revue Française. The editor of the publication, Jean Paulhan, became Ponge's mentor, and remained so for many years. Their correspondence continued until Paulhan's death in 1968. During the 1930s Ponge was for a short while associated with the Surrealist movement, influenced by which he joined the Communist Party in 1937.
During the Second World War, Ponge joined the French Resistance. He also worked for the National Committee of Journalists, 1942–44 and was literary and artistic director of the communist weekly L'Action 1944–46. He left the Communist Party in 1947. From 1952 to 1965 he held a professorship at the Alliance française in Paris. In 1966 and 1967 he was a visiting professor at Barnard College and Columbia University in the US.
In his later years Ponge was a recluse, living at his country house. He died in Le Bar-sur-Loup at the age of 89.
Awards made to Ponge included the Neustadt International Prize for Literature (1974), the Académie française's French National Poetry Prize (1981), and the Grand prix of the Société des gens de lettres (1985). He was a Commandeur of the Légion d'honneur (1983).