Richardson was born in Newcastle upon Tyne where he played football for Montagu and North Fenham Boys club. He joined Everton as a schoolboy in 1978 and turned professional in 1980. A versatile midfielder, the young Richardson often had to be content with deputising for the likes of Paul Bracewell, Peter Reid and . However, he still carved out a semi-regular place, and played in the Everton teams that lost the to Liverpool after a replay and won the FA Cup in 1984. He won a together with that of UEFA Cup Winners Cup winner's medals the following year. He made 113 appearances in all for the Toffees, scoring 16 goals.
He moved at the start of the to Watford, but only spent a season there before moving on to Arsenal for £200,000. Richardson succeeded Graham Rix on the Arsenal left wing, and played in the as Arsenal lost to Luton Town, before moving over to the right for Arsenal's title-winning season to cover for the injured . Most notably, Richardson played in Arsenal's dramatic victory at Anfield where they won the title in the last minute of the last game of the season. Now a Championship winner at two different clubs, Richardson continued to be a regular the following season at Arsenal; however, he never totally got on with Gunners manager  and he was transferred to Real Sociedad for £750,000 in the summer of 1990. He made a total of 121 appearances in all competitions for Arsenal, and scored eight goals.
Richardson's time at Real was short. He moved back to England in the summer of 1991, signing for Ron Atkinson's Aston Villa for £450,000, and was an ever-present in his first two seasons at the club. Richardson became Villa's captain, and led the club to runners-up spot in the inaugural season of the Premier League and victory over Manchester United in the , in which he won the Man of the Match award. While at Villa, he also won his one and only cap for England, in a 5–0 friendly victory over on 17 May 1994.
Richardson followed his old boss Atkinson to Coventry City in 1995, and spent the next two-and-a-half years at the Sky Blues repeatedly fending off relegation under both Atkinson and his successor, Gordon Strachan. He left Coventry in September 1997, having scored once in the League Cup against Hull City, and had spells at Southampton, Barnsley and finally Blackpool. Richardson retired from playing in 2000, his career finishing on a low after Blackpool were relegated to in his final game.