Axel Nielsen was born at Nykøbing Mors on the island of Mors in Denmark. His parents were Jørgen Nielsen (1859–1928) and Amalie Jacobsdatter (1861–1926). His father was a factory owner. He was the second youngest of nine children. He attended Staby vinterlærerskole 1915–1916. His mother was originally from Sandermosen at Maridalen in Aker, Norway. He changed his surname to Sandemose in 1921. 
Sandemose boarded a schooner for Norway at the age of seventeen. He was a sailor and lumberjack in Newfoundland.
He worked as a teacher at Nykøbing in 1916 and at Glyngøre in Skive during 1917. In 1930, Sandemose moved to Norway, and lived in Nesodden south of Oslo. After the Nazi German occupation of Norway during World War II, he fled to Sweden in 1941 due to his peripheral association with the Norwegian resistance movement. After the liberation of Norway, he moved back and settled in Søndeled. 
Sandemose published his first book in Denmark during 1923. His most notable work was En flyktning krysser sitt spor (1933). The novel was translated into English and published under the title A Fugitive Crosses His Tracks in 1936 by Alfred A. Knopf. In this novel, Sandemose introduced the concept of the Law of Jante, a listing of ten cultural rules which describe a pattern of group behaviour towards individuals common to Nordic countries. 
He was also an essayist and journalists. For a number of years he had a regular column in the weekly magazine Aktuell. Sandemose was awarded the Dobloug Prize during 1959 and was one of six finalists for the Nobel Prize in Literature in 1963.