Richardson was born in Newcastle upon Tyne where he played football for Montagu and North Fenham Boys club. He joined Everton as a schoolboy in 1978 and turned professional in 1980. A versatile midfielder, the young Richardson often had to be content with deputising for the likes of Paul Bracewell, Peter Reid and . However, he still carved out a semi-regular place, and played in the Everton teams that lost the to Liverpool after a replay and won the FA Cup in 1984. He picked up a title medal and a Cup Winners' Cup medal in May 1985. However, he did not make the squad for the FA Cup final, which Everton lost 1-0 to Manchester United.
In , he made 18 First Division appearances for the Toffees and scored three goals as they finished runners-up to Liverpool in the league. A week after the title slipped out of Everton's grasp, they lost the FA Cup final 3-1 to Liverpool, but Richardson was not in the squad.
He made 113 appearances in all for the Toffees, scoring 16 goals.
After making one appearance for Everton early in the , he was sold to Watford for £225,000, helping the Hornets finish ninth in the First Division and reach the semi-finals of the FA Cup.
After one season at Vicarage Road, Richardson was sold to Arsenal for £200,000. Richardson succeeded Graham Rix on the Arsenal left wing, and played in the as Arsenal lost to Luton Town, before moving over to the right for Arsenal's title-winning season to cover for the injured . Most notably, Richardson played in Arsenal's dramatic victory at Anfield where they won the title in the last minute of the last game of the season. Now a Championship winner at two different clubs, Richardson continued to be a regular the following season at Arsenal; however, he never totally got on with Gunners manager  and he was transferred to Real Sociedad for £750,000 in the summer of 1990, where he was paired up with new signing Dalian Atkinson and former Liverpool striker John Aldridge. He had made a total of 121 appearances in all competitions for Arsenal, and scored eight goals.
Richardson spent one season in Spain before returning to England in the summer of 1991, signing for Ron Atkinson's Aston Villa for £450,000, and was an ever-present in his first two seasons at the club. Richardson became Villa's captain, and led the club to runners-up spot in the inaugural season of the Premier League and victory over Manchester United in the , in which he won the Man of the Match award. In May 1994, when at the height of his success with Villa, he also won his one and only cap for England, in a 5–0 friendly victory over on 17 May 1994.
Atkinson was sacked as Villa manager in November 1994, with the club struggling near the foot of the Premier League, and his successor signed midfielder the following month. Atkinson was appointed manager of Coventry City three months later, and swiftly made Richardson one of his first signings in a £300,000 deal.
Richardson spent the next two-and-a-half years at the Sky Blues repeatedly fending off relegation under both Atkinson and his successor, Gordon Strachan. He left Coventry in September 1997, having scored once in the League Cup against Hull City, and had spells at Southampton, Barnsley and finally Blackpool. Richardson retired from playing in 2000, his career finishing on a low after Blackpool were relegated to in his final game.