The first new recording Black Sabbath made after the Heaven and Hell album was a version of the title track "The Mob Rules" for the soundtrack of the film Heavy Metal. The track "E5150" is also heard in the film but not included on the soundtrack. According to guitarist Tony Iommi's autobiography Iron Man: My Journey Through Heaven & Hell with Black Sabbath, the band began writing and rehearsing songs for Mob Rules at a rented house in Toluca Lake in Los Angeles. Initially the band hoped to record in their own studio to save money and actually purchased a sound desk; but, according to Iommi, "We just couldn't get a guitar sound. We tried it in the studio. We tried it in the hallway. We tried it everywhere but it just wasn't working. We'd bought a studio and it wasn't working!" The band eventually recorded the album at the Record Plant in Los Angeles.
Mob Rules was the first Sabbath album to feature Vinny Appice on drums, who had replaced original member Bill Ward in the middle of the Heaven and Hell tour. Asked by Joe Matera in 2007 if working with a new drummer was jarring after so many years, bassist and lyricist Geezer Butler replied, "No, because Vinnie was a big fan of the band and loved Bill's playing. Bill was one of his favourite drummers and so he knew all his parts and my bass parts and he adjusted accordingly to everybody in the band. He was brilliant. He came in and totally filled in Bill's shoes."
In an interview for the concert film Neon Nights: 30 Years of Heaven and Hell, Butler cites "The Sign of the Southern Cross" as his favourite Mob Rules track because "it gave me a chance to experiment with some bass effects". The album was the last time the band worked with producer and engineer Martin Birch, who went on to work with Iron Maiden until his retirement in 1992. Iommi explained to Guitar World in 1992, "We were all going through a lot of problems at that time, most of it related to drugs. Even the producer, Martin Birch, was having drug problems, and it hurt the sound of that record. Once that happens to your producer, you’re really screwed."
Mob Rules would be singer Ronnie James Dio's second and final studio recording with Black Sabbath until the Mob Rules-era line-up reunited for 1992's Dehumanizer. The seeds of discontent appear to have sprouted when Dio was offered a solo deal by Warner Brothers, with Iommi stating in his memoir, "After the (Heaven and Hell) record became such a great success, Warner Brothers extended the contract at the same time, offering Ronnie a solo deal. That felt a bit odd to us, because we were a band and we didn't want to separate anybody." Dio confided in an interview on the Neon Nights: 30 Years of Heaven and Hell DVD that the recording of Mob Rules was far more difficult for him than Heaven and Hell because "we approached the writing very much differently than the first one. Geezer had gone so we wrote in a very controlled environment in a living room with little amplifiers. And with Mob Rules we hired a studio, turned up as loud as possible and smashed through it all. So it made for a different kind of an attitude".
Iommi reflected to Guitar World in 1992, "Mob Rules was a confusing album for us. We started writing songs differently for some reason, and ended up not using a lot of really great material. That line-up was really great, and the whole thing fell apart for very silly reasons — we were all acting like children." The major problem, noted by Mick Wall in his book Black Sabbath: Symptom of the Universe, was that the balance of power within the band had shifted: "With Bill and Ozzy happy to leave the heavy lifting to Tony and Geezer, in terms of songwriting, coming into the studio only when they were called, even as their flair deserted them over the final, dismal Ozzy-era albums, at least everybody knew where they stood. Now, though, the creative chemistry had shifted."
"I still like that album," Iommi reflected in 1997.