The story opens as two New Englanders, John Branner and his brother Griswell, are travelling in the South and spend the night in a deserted plantation manor. Griswell awakens from a troubled sleep to see Branner walking up the stairs in a trance. He is horrified when Branner returns, no longer alive but an animated corpse gripping the bloody axe that had split his skull. Griswell flees the house in a blind panic and runs aimlessly into the woods.
In his headlong flight he meets the county's sheriff, Buckner, who investigates the house and finds Branner cold and motionless on the floor. Griswell is implicated in his brother's murder, but the sheriff gives him the benefit of the doubt and doggedly attempts to clear him. Buckner is inclined to give some credence to Griswell's bizarre tale due to the ominous reputation of the manor. It was once the residence of a family from the West Indies, the Blassenvilles, who were known for their cruelty. One night in 1890 the last of the Blassenvilles, Elizabeth, fled from the house and never returned. The manor has lain deserted ever since and is shunned by the local black folk. The pigeons of the story's title are ghostly birds that sometimes flock mysteriously about the decaying manor. Legend has it that they are the souls of the Blassenvilles let out of Hell.
The following evening Buckner and Griswell go to the hut of an ancient voodoo man, Jacob, seeking information about the house and the Blassenvilles. Jacob tells of the extinct family and of Celia Blassenville, who mistreated her mulatto maid Joan. He then begins to ramble incoherently about voodoo, the god Damballah, and about zombies and their female counterparts, zuvembies. Finally he tells how "she" participated in voodoo rites. While reaching for firewood, Jacob is bitten by a poisonous snake, meeting the very fate he feared would overtake him for revealing the secrets of Damballah. Buckner and Griswell conclude that Joan transformed herself into a zuvembie to exact vengeance on Celia Blassenville and her nieces. They resolve to spend the night in Blassenville Manor to learn the truth.