The initial sessions for what became Heaven and Hell began with Ozzy Osbourne after Sabbath's Never Say Die! tour. The band moved to Los Angeles for eleven months to record a new album; a process that guitarist Tony Iommi described in his autobiography as a "highly frustrating, never-ending process". In his own autobiography, Osbourne states that he had become fed up with the experimentation on the preceding albums Technical Ecstasy and Never Say Die!, preferring the band's earlier, heavier sound. And he admits, "It was clear they'd had enough of my insane behaviour." In his memoir, Iommi revealed that he has a tape featuring Osbourne singing an early version of what would become "Children of the Sea" with a different lyric and a totally different vocal melody.
Ronnie James Dio was introduced to Iommi in 1979 by Sharon Arden, who would later marry Osbourne. Initially, Dio and Iommi discussed forming a new band, rather than a continuation of Black Sabbath. The pair met again by chance at The Rainbow on Sunset Strip in Los Angeles later that year. Both men were in similar situations: Dio was seeking a new project and Iommi requiring a vocalist. "It must have been fate," Dio recalled, "because we connected so instantly." The pair kept in touch via telephone, until Dio arrived at Iommi's Los Angeles house for a relaxed, getting-to-know-you jam session. On that first day, the duo finished "Children of the Sea", a song Iommi had abandoned prior to Osbourne's firing.
"Sabbath was a band that was floundering," Dio observed. "And, with my inclusion in it, we pulled ourselves up by our bootstraps, cared a lot about each other, and knew that we could do it again – especially under the banner of a band that had been so successful."
Sabbath's line-up was in flux before the recording of Heaven and Hell. The band replaced its vocalist, and drummer Bill Ward was battling personal issues that would see him eventually leave the band. Demos for the album featured Geoff Nicholls on bass, as longtime bassist Geezer Butler was going through a divorce and his future with the band was in question. In fact, when Dio joined the band he doubled as bassist and vocalist, having played bass in the band Elf in the early 1970s. At one point Iommi contacted close friend Frank Zappa for help finding a bassist. Zappa offered his bassist for the Heaven and Hell sessions but Iommi preferred a permanent member. Eventually Butler returned and Nicholls stayed on as the band's unofficial keyboardist.
Former Elf and Rainbow bassist Craig Gruber also rehearsed with the band, though the extent of his involvement is unclear. In a 1996 interview, Iommi stated that Gruber participated only "for a bit". Gruber has stated that his contribution was more substantial; he says he cowrote most of Heaven and Hell's songs and that it was he and not Butler who played bass on the album. Despite not being credited for his contributions, Gruber asserted, "We came to a suitable financial arrangement". Iommi stated in his 2011 autobiography that Gruber recorded all the bass parts on Heaven and Hell, but that Butler rerecorded the parts upon his return, without listening to Gruber's bass tracks.
Black Sabbath performing in Cardiff
Heaven and Hell was recorded at Miami's Criteria Studios (in which the band recorded Technical Ecstasy) and Studio Ferber in Paris. Dio suggested the band hire producer Martin Birch. Birch was Sabbath's first outside producer since 1971's Master of Reality; Iommi had primarily produced the albums since then. Iommi stated that the band felt that they were creating something special. In his memoir, he wrote, "Ozzy would sing with the riff. Just listen to 'Iron Man' and you'll catch my drift: his vocal melody line copies the melody of the music. There was nothing wrong with that, but Ronnie liked singing across the riff instead of with it, come up with a melody that was different from that of the music, which musically opens a lot more doors. I don't want to sound like I'm knocking Ozzy, but Ronnie's approach opened up a new way for me to think ..."
"Heaven and Hell for me wasn't a turning point," Ward recalled. "Heaven and Hell was the beginning of a new band of which I had no idea what band I was in… It was almost like Ron was capable of coming up with lyrics that seemed to fit his idea of how Black Sabbath ought to be, and I sensed a kind of... unrealness about the lyrics. My favourite song on Heaven and Hell was a blues song that we did, 'Lonely Is the Word' – and that seemed to be real… But things like 'Lady Evil', they seemed almost like bandwagon-type lyrics… 'Lonely Is the Word', I definitely liked playing that song. And 'Children of the Sea' – I did like to play that too. I thought Ronnie was a very good singer."
During a slow day in the studio, Iommi doused Ward with a solution used by studio technicians to clean the tape heads. He then set light to the solution, which was more flammable than he had anticipated. Ward suffered third degree burns as a result and still has scars on his legs from the incident. Ward has stated that, due to his alcoholism, he has no memory at all of the period in which the album was recorded. His behaviour became erratic; on the Heaven & Hell Tour, Ward began dictating long and rambling press releases to the band's public relations representatives after every show, instructing them to "get that out on the news wires tonight". Ward's personal issues, which included the deaths of both his parents, would soon force him to leave the band. Dio reportedly answered the telephone in his hotel room one morning mid-tour to hear Ward say "I'm off then, Ron", to which Dio replied "That's nice Bill, where are you going?" "No, I'm off mate. I'm at the airport now"; indicating that he was incapable of completing the tour. American drummer Vinny Appice was quickly brought in to replace him. The Heaven and Hell album represents the only Sabbath material recorded during the Dio-era that does not feature Appice on drums.