Born in London, the son of an industrial chemist and his wife, Rutter grew up living over the Globe pub on London's Marylebone Road. He was educated at Highgate School where fellow pupils included John Tavener, Howard Shelley, Brian Chapple and Nicholas Snowman, and as a chorister there took part in the first (1963) recording of Britten's War Requiem under the composer's baton. He then read music at Clare College, Cambridge, where he was a member of the choir. While still an undergraduate he had his first compositions published, including the "Shepherd's Pipe Carol" which he had written aged 18. He served as director of music at Clare College from 1975 to 1979 and led the choir to international prominence.
In 1981, Rutter founded his own choir, the Cambridge Singers, which he conducts and with which he has made many recordings of sacred choral repertoire (including his own works), particularly under his own label Collegium Records. He resides at Hemingford Abbots in Cambridgeshire and frequently conducts many choirs and orchestras around the world.
In 1980, he was made an honorary Fellow of Westminster Choir College, Princeton, and in 1988 a Fellow of the Guild of Church Musicians. In 1996, the Archbishop of Canterbury conferred a Lambeth Doctorate of Music upon him in recognition of his contribution to church music. In 2008, he was made an honorary Bencher of the Middle Temple while playing a significant role in the 2008 Temple Festival.
From 1985 to 1992, Rutter suffered severely from myalgic encephalomyelitis (ME, or chronic fatigue syndrome), which restricted his output; after 1985 he stopped writing music on commission, as he was unable to guarantee meeting deadlines.
Rutter also works as an arranger and editor. As a young man he collaborated with Sir David Willcocks on five volumes of the extraordinarily successful Carols for Choirs anthology series.
He was inducted as a National Patron of Delta Omicron, an international professional music fraternity in 1985. Rutter is also a Vice President of the Joyful Company of Singers, President of Association of British Choral Directors (ABCD).