John was born in Damascus in the third quarter of the 7th century AD, to a prominent Damascene Christian family known as "Mansoūr". The family was named after John's grandfather, Mansour ibn Sarjun, who had been responsible for the taxes of the region during the reign of Emperor Heraclius. Mansur seems to have played a role in the capitulation of Damascus to the troops of Khalid ibn al-Walid in 635 after securing favorable conditions of surrender. Eutychius, a 10th-century Melkite patriarch, mentions him as one high-ranking official involved in the surrender of the city to the Muslims.
Though information about the tribal background of the Mansour family are absent in contemporary sources, biographer Daniel Sahas speculates the name Mansour could have implied that they belonged to the Arab Christian tribes of Kalb or Taghlib. Moreover, the family name was common among Syrian Christians of Arab origins, and Eutychius noted that the governor of Damascus, who was likely Mansour ibn Sarjun, was an Arab. However, Sahas also asserts that the name does not necessarily imply an Arab background and could have been used by non-Arab, Semitic Syrians. While Sahas and biographers F. H. Chase and Andrew Louth assert that Mansūr was an Arabic name, Raymond le Coz asserts that the "family was without doubt of Syrian origin"; indeed, according to historian Daniel J. Janosik, "Both aspects could be true, for if his family ancestry were indeed Syrian, his grandfather [Mansour] could have been given an Arabic name when the Arabs took over the government." John was raised in Damascus, and Arab Christian folklore holds that during his adolescence, John associated with the future Umayyad caliph Yazid I and the Taghlibi Christian court poet al-Akhtal.
When Syria was conquered by the Muslim Arabs in the 630s, the court at Damascus retained its large complement of Christian civil servants, John's grandfather among them. John's father, Sarjun (Sergius), went on to serve the Umayyad caliphs. According to John of Jerusalem and some later versions of his life, after his father's death, John also served as an official to the caliphal court before leaving to become a monk. This claim, that John actually served in a Muslim court, has been questioned since he is never mentioned in Muslim sources, which however do refer to his father Sarjun (Sergius) as a secretary in the caliphal administration. In addition, John's own writings never refer to any experience in a Muslim court. It is believed that John became a monk at Mar Saba, and that he was ordained as a priest in 735.