The church is the fourth building on the site on the village green in Bedford. The cornerstone was laid and services celebrating the dedication of the then-most-recently-restored building were held in 1872.
The first minister, Reverend Thomas Denham, settled in 1684, and a meeting house was built in 1689. When Bedford became part of New York, this house was transferred to Rye, New York. Bedford was part of the Anglican Church until the Revolution. A second meeting house was burned by order of a British colonel in 1779. A third house was transferred to
Westmoreland Sanctuary. The fourth building, the present church, was built in 1872, with a Sunday school in the rear. It was described as
an 1872 newcomer, a whitewashed eccentricity with unmatching towers, an unexpected rose window and, within, amen pews flanking the pulpit.
It is part of the Bedford Village Historic District, which was added to the National Register of Historic Places in northern Westchester County in 1973.
The church, which has excellent acoustics, is often used as a venue for sacred music concerts and recitals. The choir The Master Singers of Westchester performed regular concerts such as Britten's A Ceremony of Carols in 1984, his cantata St. Nicholas in 1986, and Bach's Mass in B minor in March 2003. The Westchester Oratorio Society performed its 20th-anniversary concert there on 6 May 2018, performing Mozart's Requiem, with organist Anthony Newman.