Born in Dresden, Zimmermann was a member of the Dresdner Kreuzchor from 1954 to 1962, when he completed the Abitur. Directed by Rudolf Mauersberger, Zimmermann was immersed in the works of Johann Sebastian Bach and learned vocal expression, which became a focus of his own compositions. He wrote three motets which were performed by the choir, including a "Vaterunserlied" in 1959. Education in the choir fostered a humanitarian attitude which he kept for life.
He continued his music education at the Hochschule für Musik Carl Maria von Weber, studying composition with Johannes Paul Thilman and also voice and conducting. The works composed during these years include Dramatische Impression für Violoncello und Klavier auf den Tod von J. F. Kennedy (Dramatic impression for cello and piano on the death of John F. Kennedy), composed in 1963, Fünf Gesänge für Bariton und Kammerorchester (Wolfgang Borchert) (Five chants for baritone and chamber orchestra after Wolfgang Borchert), written in 1964, and the opera Weiße Rose based on a libretto by his brother
Ingo Zimmermann and composed in 1967/68. The theme of the opera, which he composed as a student, is the White Rose resistance movement of the siblings Hans and Sophie Scholl. From 1968, he studied in Berlin at the Akademie der Künste with Günter Kochan. In the same year he composed Musik für Streicher (Music for strings), his first work including twelve-tone technique and a new organisation of sound processes in levels ("flächig)".
In 1970, Zimmermann became dramaturge of the Staatsoper Dresden. In 1978 he was appointed professor of composition at the Dresdnen Musikhochschule, where he had lectured from 1976. As a conductor, he was invited by major orchestras such as the Berlin Philharmonic, Vienna Symphony, Staatskapelle Dresden, Gewandhausorchester in Leipzig, Orchestre de Radio France in Paris, Tonhalle Orchestra in Zurich, Concertgebouw Orchestra in Amsterdam, Bavarian Radio Symphony Orchestra, NDR Symphony Orchestra in Hamburg, Prague Radio Symphony Orchestra, Warsaw Philharmonic, MDR Symphony Orchestra, and the Berlin Radio Symphony Orchestra. He also appeared as a guest conductor at opera houses in Bonn, Hamburg, Munich and Vienna. He organised productions of his operas in both East and West Germany, and arranged for leading papers to review them.
In 1986, he founded the Dresdner Zentrum für zeitgenössische Musik (Dresden Center for Contemporary Music) as a research center and for concerts and festivals. He returned to his opera topic Weiße Rose and wrote a condensed version for only two voices and ensemble on a text by Wolfgang Willaschek. It premiered at the Opera Stabile, Hamburg, on 27 February 1986, and was staged often. Zimmermann was the artistic director of the Leipzig Opera. During this time, 27 premieres of new works were performed at the house, including several especially for the tricentenary of the opera house. Parts of Stockhausen's Licht were premiered, also Jörg Herchet's nachtwache, staged by Ruth Berghaus, and Dieter Schnebel's "Majakowskis Tod – Totentanz". The house received international attention, presenting Busoni's Doktor Faust staged by Willy Decker, and a cycle of Mozart's operas on librettos by da Ponte, staged by John Dew, among others. From 2001 to 2003 he was general director of the Deutsche Oper Berlin.
Zimmermann directed the series musica viva of contemporary music, run by the broadcaster Bayerischer Rundfunk from 1997 to 2011. He invited notable composers and ensembles to concerts in Munich, many of which were recorded. In 2007/08, he initiated an additional ars musica viva festival, which presented leading radio orchestras and ensembles. The BMW Kompositionspreises, a composition prize for the series, was an award for many new works by young international composers. A total of 175 works were performed, with 161 compositions commissioned by musica viva, and presented in 180 broadcasts. Zimmermann received the broadcaster's Gold Medal for his work over 14 years.
Zimmermann then directed the Europäisches Zentrum der Künste in Dresden-Hellerau (European centre of the arts in Dresden-Hellerau), with a vision of a laboratory for contemporary art ("Labor für zeitgenössische Kunst"), including theatre, dance, architecture, art and media art. He retired from the position in 2008.