Rhoads was born in Santa Monica, California. The youngest of three children, he had a brother named Doug and a sister named Kathy. Doug, who performed under the name "Kelle", is also a musician. Their parents, Delores and William, were both music teachers. In 1958, father William left the family when Randy was 1 year and 5 months old and remarried, and all three children were subsequently raised by Delores, who also opened a music school in North Hollywood called Musonia to support the family. Delores had received a bachelor's degree in music from UCLA and had played piano professionally.
The Rhoads family did not own a stereo and the children created their own music at home to entertain themselves. Rhoads began taking folk and classical guitar lessons at approximately age 7 at his mother's music school. He soon became interested in electric guitar and began taking lessons at Musonia from an instructor named Scott Shelly. Shelly soon approached Delores to inform her that he could no longer teach her son, as Rhoads' knowledge of the electric guitar had exceeded his own. Rhoads also received piano lessons from his mother to build his understanding of music theory.
Rhoads met future bandmate Kelly Garni while attending John Muir Middle School and the two became best friends. According to Garni, the pair were unpopular due to "the way we looked". "Every time we showed up for school it was usually problematic so we pretty much avoided it. We weren't nerds, we weren't jocks, we weren't dopers, we were just on our own". Rhoads taught Garni how to play bass guitar, and together they formed a band called The Whore, rehearsing during the day at Rodney Bingenheimer's English Disco, a 1970s Hollywood nightspot. It was during this period that Rhoads learned to play lead guitar. "When I met him he didn't know how to play lead guitar yet at all. He was just starting to take lessons for it and really just riffing around", said Garni. With this band, Rhoads spent several months playing at backyard parties around the Los Angeles area in the mid-1970s. The pair formed a cover band called Violet Fox (after his mother's middle name, Violet), with his older brother Kelle on drums. Violet Fox, who were together for approximately five months, staged several performances in the Grand Salon at Musonia. Among their setlist was "Mississippi Queen" by Mountain, and songs from the Rolling Stones, Alice Cooper and David Bowie. After Violet Fox dissolved, Rhoads formed various other short-lived bands such as The Katzenjammer Kids and Mildred Pierce. The Katzenjammer Kids featured a lead vocalist who would wear dresses on stage, a situation that typically did not go over well and at times led to violent reactions from the audience.
According to Garni, he and Rhoads would frequently listen to Long Beach radio station KNAC because it was "the only radio station that would play anything of interest to us", and it was through KNAC that Rhoads discovered much of the music that would influence his playing. The home of a neighbourhood friend with a high quality stereo and large record collection became a regular hangout for the pair, and there they would smoke pot and listen to more obscure (at that time) hard rock music such as early Scorpions records. Live bootleg recordings were very popular at that time, and Rhoads began to take note of the differences between studio recordings and the live versions, particularly the different licks guitarists would incorporate when playing live. He began to memorize these licks and teach himself to play them later when he would return home. Rhoads' brother states that a July 11, 1971 Alice Cooper concert at the Long Beach Auditorium that the pair attended was a defining point in the guitarist's life. After the concert was over Kelle said "Randy was mesmerized. He was catatonic just staring at the stage. Later that night Randy said "I can do this. I can look like this. I can be this. Something clicked that night and I think that kind of showed him what he could do with his talent." Garni concurs, calling Rhoads' discovery of Alice Cooper "a game changer". Glen Buxton of Alice Cooper and Mick Ronson were two early rock influences on his playing.
At age 16, Rhoads and Garni formed the band Little Women. At approximately the same time, Rhoads began teaching guitar in his mother's school during the day and playing live gigs at night. He graduated from Burbank High School, participating in a special program that allowed him to condense his studies and graduate early so he could teach guitar and pursue music full-time. Recruiting lead vocalist Kevin DuBrow and drummer Drew Forsyth, the band soon changed its name to Quiet Riot. Forsyth had periodically played with Rhoads and Garni in the past, most notably in Mildred Pierce. DuBrow was an L.A. photographer who was not at all what Rhoads had in mind for his new band, and he was not well liked by his Quiet Riot bandmates, a situation that caused a great deal of tension within the band. Rhoads had envisioned a frontman in the vein of Alice Cooper or David Bowie, but Dubrow was persistent and would not take no for an answer. In the end, Rhoads and Garni decided that if nothing else, DuBrow shared their enthusiasm and he was hired.
Quiet Riot quickly became one of the most popular acts on the Los Angeles club circuit, and by late 1976 were signed to CBS/Sony Records. Rhoads' "polka-dot theme" became the visual focal point of the band, as many fans began showing up at Quiet Riot shows wearing polka-dot bow-ties and vests, emulating what the guitarist wore on stage.
While the band had a strong following in Los Angeles, Quiet Riot and Quiet Riot II were released only in Japan. Of perhaps more concern to Rhoads was the fact that the relationship between DuBrow and Garni had deteriorated completely during the recording of the band's second album, with potentially catastrophic results. After drunkenly firing a handgun through the ceiling and engaging in a fistfight with Rhoads, Garni drunkenly hatched a plan to shoot and kill DuBrow at The Record Plant studio while recording the album. Rhoads was left with no choice but to fire his longtime friend and band co-founder.
In 1979, ex-Black Sabbath vocalist Ozzy Osbourne was in Los Angeles, attempting to form a new band. An acquaintance of Rhoads' from the LA club circuit, future Slaughter bassist Dana Strum, phoned Rhoads relentlessly to coax him into auditioning. Rhoads initially told Quiet Riot bandmate Rudy Sarzo that he was not really interested in auditioning, but finally agreed to go simply to get Strum off his back. Rhoads got the call for the audition just before his final show with Quiet Riot in September 1979. The day before Osbourne was scheduled to return to England, Rhoads agreed to audition for Osbourne at a Los Angeles studio with his Gibson Les Paul guitar and a practice amp and started warming up. Osbourne, who was very inebriated on that day, said of the audition "He played this fucking solo and I'm like, am I that fucking stoned or am I hallucinating or what the fuck is this?!" Osbourne has maintained that he immediately gave him the job. Rhoads recalled later, "I just tuned up and did some riffs, and he said, 'You've got the gig'; I had the weirdest feeling, because I thought, 'You didn't even hear me yet'". After the audition, Rhoads returned to Musonia and told Sarzo that he had never actually met Osbourne, who was drunk and remained in the studio's control room the entire time. According to Rhoads' own account, it was Strum who emerged from the control room to inform him that he had the job. Rhoads was, however, scheduled to meet Osbourne the following night in his hotel room. In the years following, Osbourne has maintained that his first encounter with Rhoads and the subsequent audition took place the following day at the hotel, and it seems that, in his inebriated state, he combined the two events in his mind. The fact that Osbourne immediately began rehearsals with another guitarist upon returning to England, and did not mention Rhoads until after that guitarist had been fired, seems to confirm that his account of events is innacurate.
Over the next couple of days, Rhoads, Osbourne, Strum, and drummer Frankie Banali jammed together before Osbourne returned to England. Disillusioned with Quiet Riot's inability to land an American recording deal, Rhoads discussed with his mother Delores the possibility of joining an already established band. When she asked him if he would accept "an offer like this one", the guitarist replied "Of course!"
Upon returning to England, Osbourne was introduced to ex-Rainbow bassist Bob Daisley by a Jet Records employee named Arthur Sharpe in a pub, and the pair hit it off and decided to work together. Unhappy with the guitarist they were initially working with, Osbourne mentioned to Daisley that he had recently met a talented young guitarist in Los Angeles by the name of Randy Rhoads. The new group's management intended to keep the lineup all-British and was reluctant to hire an unknown American guitarist, but manager Don Arden eventually relented. Rhoads flew to England only to return home a couple of days later, being turned away by English customs at Heathrow Airport when he didn't have the necessary work permit. A representative from Jet Records was dispatched to clear the matter up but he never arrived, and Rhoads spent the night in a holding cell before being handcuffed and put on a plane back to the United States the next day. Osbourne subsequently called him to apologize, and arrangements were made for Rhoads to return to England with the proper paperwork. Rhoads flew to England on November 27, 1979, and met with Osbourne and Daisley at the Jet Records' offices in London. The trio traveled by train to Osbourne's home, Bulrush Cottage, which also housed a rehearsal space. It was here that Rhoads lived with Osbourne, his then-wife Thelma, and their two children, during his first weeks in England. Years later, Osbourne said in his autobiography that he could not understand why a musician as talented as Rhoads would want to get involved with a "bloated alcoholic wreck" like himself.
After a short search, drummer Lee Kerslake completed the new band, then known as The Blizzard of Ozz. The group headed into the studio to record their debut album, titled Blizzard of Ozz. Rhoads' guitar playing had changed due to the level of freedom allowed by Osbourne and Daisley and he was encouraged to play what he wanted. His work with Quiet Riot had been criticized as being "dull" and did not rely on classical scales or arrangements. Propelled by Rhoads' neo-classical guitar work, the album proved an instant hit with rock fans, particularly in the US. They released two singles from the album: "Mr. Crowley" and the hit "Crazy Train". Osbourne said years later, "One day Randy came to me and said that most heavy metal songs are written in an A to E chord structure. He said, 'Let's try to change that' ... so we made a rule that almost every number that we recorded on an album was never played in the same key."
Following a UK tour the band recorded another album, Diary of a Madman. During a break before leaving for their first US tour, both Kerslake and Daisley were suddenly fired by Sharon Arden, the band's manager and Osbourne's future wife. For the US tour, ex-Black Oak Arkansas drummer Tommy Aldridge and bassist Rudy Sarzo – who had been Rhoads' bandmate in Quiet Riot – were hired. Diary of a Madman was released soon after in October 1981, and since Kerslake and Daisley were already out of the band, Aldridge and Sarzo's names and photos appeared on the album sleeve. Disputes over royalties performance and other intellectual property rights became a source of future court battles. Kerslake has maintained that Rhoads almost left Osbourne's band in late 1981 due to his displeasure with the firing of Kerslake and Daisley. "He didn't want to go [on tour with Osbourne]. We told him we were thrown out. He said he was going to leave the band as he did not want to leave us behind. I told him not to be stupid but thanks for the sentiment", the drummer later recalled.
Aldridge, who Rhoads had regarded as his favorite drummer since seeing him perform on television with Black Oak Arkansas in the 1970s, has said that working with a musician as talented as Rhoads "was inspirational. It was life-changing". From a musical standpoint, he has said that playing with Rhoads was the high point of his career, stating "It was very exciting. From a musical perspective, it was probably the high-water mark of my career. Working with people like Randy Rhoads, guys like that, they kind of grab you by the scruff of your neck and lift you up to their level."
Around this time, Rhoads remarked to Osbourne, bandmates Aldridge and Sarzo, and friend Kelly Garni that he was considering leaving rock for a few years to earn a degree in classical guitar at UCLA. In the 1991 documentary film Don't Blame Me, Osbourne confirmed Rhoads' desire to earn the degree and stated that had he lived, he did not believe Rhoads would have stayed in his band. Friend and ex-Quiet Riot bassist Garni has speculated in interviews that if Rhoads had continued to play rock, he might have gone the route of more keyboard-driven rock, which had become popular through the 1980s. It was at this time that Rhoads was beginning to receive recognition for his playing. Just before his death Jackson Guitars created a signature model, the Jackson Randy Rhoads (though Rhoads had originally called his white pinstriped V "the Concorde"). Rhoads received one prototype – a black offset V hardtail that is the base for today's RR line of Jackson guitars – but died before the guitar went into production. Rhoads also received the Best New Talent award from Guitar Player magazine. While on tour with Osbourne, Rhoads would seek out classical guitar tutors for lessons whenever possible.
At the time of his death, Rhoads had already made the decision to part ways with Osbourne once his contractual obligations had been fulfilled. Though he had a good relationship with Osbourne, the vocalist's constant drug and alcohol abuse made day-to-day life on tour difficult for the members of his band. As the Diary of a Madman US tour progressed, Osbourne would often refuse to perform due to the lingering after-effects of the previous night's excesses, and only Sharon could talk him into taking the stage. Many shows were simply canceled, and Rhoads grew tired of the unpredictability. The final straw came when a plan was announced in February 1982 by Osbourne's management and record label to record a live album of Black Sabbath songs at Toronto's Maple Leaf Gardens later that year. Rhoads and bandmate Tommy Aldridge felt that they had established themselves as recording artists, and they regarded an album of cover songs to be a step backwards artistically and professionally. Thus, they refused to participate in the planned live recording. Osbourne viewed this decision as a betrayal, and the relationship between he and Rhoads became quite strained. Already drinking heavily, Osbourne's drinking increased and began to tear the band apart. At one point he drunkenly fired the entire band, including Rhoads, though he later had no memory of doing so. He began taunting Rhoads with claims that the likes of Frank Zappa and Gary Moore were willing to replace him on the proposed live album. Osbourne's unstable and confrontational behavior soon convinced Rhoads to leave the band. He grudgingly agreed to perform on the live album with the stipulation that he would depart after fulfilling his contractual obligations to Jet Records, which consisted of one more studio album and subsequent tour. The proposed live album was scrapped upon the guitarist's sudden death weeks later, though the plan was quickly resurrected with the release of Speak of the Devil in November of that year.