Grace was a descendant of the seventeenth-century Richard Grace, whose father, also named Robert Grace, was a feudal Baron of Courtstown. His ancestors had accompanied the Earl of Pembroke in an invasion of Ireland in the later part of the 12th century. They acquired extensive lands in Kilkenny County of Ireland and the family flourished there for more than 500 years. They then lost their lands in the Glorious Revolution of 1688. Some of the Grace families followed the exiled James II of England to France and others became soldiers of fortune. Grace's father was one of the latter and eventually ended up on an estate in Barbados. In 1707 he came to Philadelphia to live for a while. During this time, Grace was born in the city on April 25, 1709. His mother died while he was a baby and his father went back to Barbados and lived out his life there. Grace was raised by his maternal grandmother. Her name was Constance and she was married to Hugh Lowden, a wealthy merchant. Grace grew up in their Philadelphia mansion on High Street.