Mary Ford was born Iris Colleen Summers in El Monte, California, the second daughter of Marshall McKinley Summers (born February 13, 1896, in Ridgway, Illinois; died August 5, 1981, in Los Angeles), a Nazarene minister, who later became a painting contractor, and his wife, Dorothy May White Summers (born April 5, 1897, in Missouri; died February 22, 1988, in South El Monte, California), and was the sister of Byron Fletcher Summers, Esther E. Williams, Carol Jean Corona, Bruce Summers, Eva Wootten and Bob Summers. Ford came from a musical family. Her parents left Missouri, travelling cross-country while singing gospel music and preaching at revival meetings across the United States. They eventually settled in southern California, where they were heard over KPPC-AM, Pasadena's first Christian radio station. Her sisters and brothers were all musicians; Esther, Carol, Fletcher, jazz organist Bruce and film composer Bob Summers.
While still a junior high school student, Summers and Mildred L. "Milly" Watson (later Millie Pace) (born February 26, 1922, in Los Angeles; died August 2, 1976, in Orange County, California), a local girl, performed together in churches in Pasadena, California, and later made gospel recordings with Milly's older brother, Marvin, for which she wrote some songs. In 1939 Summers and Milly won a Pasadena talent contest judged by "several Hollywood notables, including a very young Judy Garland." Hoping to have a musical career, Summers and Milly Watson lost interest in school, played truant frequently and eventually quit school, only to find employment as a cinema attendant.