The school was founded in February 1572 under a Royal Charter granted by Queen Elizabeth I to John Lyon, a wealthy local farmer. The Charter described this as a re-endowment, and there is some evidence of a grammar school at Harrow in the mid-16th century, but its location and connection with Lyon's foundation are unclear. Evidence for earlier schools, possibly connected with the chantry of St Mary (established in 1324), is weak. In the original charter, six governors were named, including two members of the Gerard family of Flambards, and two members of the Page family of Wembley and Sudbury Court.
Lyon died in 1592, leaving his assets to two causes: the lesser was the School, and by far the greater beneficiary was the maintenance of a road to London, 10 miles (16 km) away. The school owned and maintained this road for many years following Lyon's death, and the whole school still runs along this 10-mile road in an event called "Long Ducker" every November, whilst some 6th formers opt to do 20 miles – to and from the Albert Memorial in London.
It was only after the death of Lyon's wife in 1608 that the construction of the first school building began. It was completed in 1615 and remains to this day, however it is now much larger. At first the primary subject taught was Latin, and the only sport was archery. Both subjects were compulsory; archery was dropped in 1771.
Although most boys were taught for free, their tuition paid for by Lyon's endowment, there were a number of fee-paying "foreigners" (boys from outside the parish). It was their presence that amplified the need for boarding facilities. By 1701 for every local there were two "foreign" pupils; these generated funds for the School as fees increased. By 1876 the ratio was so high that John Lyon Lower School was brought under the authority of the governors of the Upper School so that the School complied with its object of providing education for the boys of the parish. It is now known as The John Lyon School and is a prominent independent school. It maintains close links with Harrow. The majority of the school's boarding houses were constructed in Victorian times, when the number of boys increased dramatically. Between 1872 and 1877, a Speech Room was constructed to the designs of William Burges. The structure is a Grade II* listed building.
The school war memorial, designed by Sir Robert Lorimer, was erected in 1917, marking the already substantial loss of former pupils by that stage of the First World War.
The 20th century saw the innovation of a central dining hall, the demolition of small houses and further modernisation of the curriculum. Currently there are about 850 boys boarding at Harrow.
In 2005, the school was one of fifty of the country's leading independent schools which were found guilty of running an illegal price-fixing cartel, exposed by The Times, which had allowed them to drive up fees for thousands of parents, although the schools said that they had not realised that the change to the law (which had happened only a few months earlier) about the sharing of information had subsequently made it an offence. Each school was required to pay a nominal penalty of £10,000 and all agreed to make ex-gratia payments totalling £3,000,000 into a trust designed to benefit pupils who attended the schools during the period in respect of which fee information was shared. Jean Scott, the head of the Independent Schools Council, said that independent schools had always been exempt from anti-cartel rules applied to business, were following a long-established procedure in sharing the information with each other, and that they were unaware of the change to the law (on which they had not been consulted).
Harrow has expanded overseas, opening additional schools in Beijing, China (Harrow International School Beijing); Shanghai, China (Harrow International School Shanghai), Bangkok, Thailand (Harrow International School, Bangkok); and New Territories, Hong Kong (Harrow International School Hong Kong).