Gielen was born in Dresden to Rose (née Steuermann) and
Josef Gielen . His father was a theatre and opera director from 1924 at the Staatstheater Dresden, who staged the premiere of Kaiser/Weill's Der Protagonist at the Semperoper in 1926. His mother Rose came from a Jewish family in Sambor (then Austria-Hungary, now Ukraine). She was an actress who had given up acting when their first child Carola was born, but appeared occasionally, for example as a speaker in the premiere of Arnold Schönberg's Pierrot lunaire in Dresden in 1919, rehearsed with her brother Eduard Steuermann. The boy Michael first attended a reformed school from 1934 until it was closed by the Nazis. Both children were baptized and raised Catholic to counter Nazi indoctrination.
Clemens Krauss called Josef Gielen to the Staatsoper Berlin in 1936, where Michael attended primary school for a year, and then the Kaiserin-Augusta-Gymnasium. When his father's contract was dissolved in 1937, he found a position at the Vienna Burgtheater. The family followed there in 1938. Michael attended gymnasium and took piano lessons. Josef Gielen successfully staged at the Teatro Colón in Buenos Aires, Argentina, in 1938 and 1939, and managed to get immigration papers for his wife and the two children. In 1940, the family left for Argentina, leaving most of their belongings behind.