His career began at Melleruds IF, where he played for three seasons. During this time he played 38 times, scoring 11 goals. It was scoring 10 of these in the final season that prompted Degerfors IF to sign him for the beginning of the 1996 season. Again, he spent three years at the club, regularly securing a place in the team and a modest supply goals. Having moved out of favour with the team at the back end of the 1998 season, he had an unsuccessful spell at AIK, appearing only 12 times in the league, five of these as a sub. In 2000, he moved to Djurgårdens IF, for whom he played until 2004, during which time he saw them claim the Allsvenskan championship in 2002 and 2003, as well as the Swedish Cup in 2002 and 2004. In total he scored 45 goals for Djurgården during 124 appearances.
He moved to Wigan Athletic at the beginning of calendar year 2005, under manager Paul Jewell. His first appearance in the Wigan team came against Stoke City in February 2005, as a substitute replacement for Gary Teale. This would be his only game of the 2004–05 season. He had a successful , as he scored two extra time goals against Watford and helped Wigan reach the final. He played 16 games during the 2005–06 FA Premier League season, and scored goals against Bolton Wanderers, Tottenham Hotspur (2) and Birmingham City. In the last game of that season, Johansson came on against Arsenal at Highbury and got sent off without touching the ball after tackling fellow country man Freddie Ljungberg in the penalty box, allowing Thierry Henry to score the last goal at the old ground on a penalty kick. He played 12 games during the 2005–06 FA Premier League season, and was released from his Wigan contract in summer 2007 by manager Chris Hutchings.
Johansson was quickly snapped up by Danish Superliga club AaB on 17 July 2007. At AaB, he quickly established himself as a playmaking midfielder. With seven goals in 31 games, he led AaB to the 2007–08 Danish Superliga championship, the first AaB championship since 1999. He helped the team reach the group stage of the 2008–09 UEFA Champions League tournament, and scored in the 3–6 defeat to Villarreal CF. AaB finished third in their group, and moved on to the 2008–09 UEFA Cup, where Johansson scored in the aggregate 6–1 defeat of Deportivo La Coruña. He and AaB also reached the 2009 Danish Cup Final, but lost 0–1 to Copenhagen. Following the sale of Thomas Augustinussen, Johansson was named new AaB team captain in July 2009. With eight goals in 32 games he was the AaB top goalscorer of the 2009–10 Danish Superliga season. As his contract expired in the summer 2010, AaB did not have the financial means to extend it.
He scored his first goal in his second spell for Djurgårdens IF in Allsvenskan on 12 May 2013 in the 3–2 victory against Malmö FF.