Daily newspapers Bild, Welt and magazine Kicker claim the 65-year-old is leaving six months ahead of schedule.
Rummenigge’s departure will herald a new era at Bayern for the start of next season.
He joined the club’s management in 1991 as vice-president and then became chairman in 2002, overseeing an incredible period of success.
With Rummenigge on board, Bayern won the Bundesliga title 19 times and lifted the Champions League trophy in 2001, 2013 and 2020.
A former striker who made 95 appearances for West Germany, Rummenigge played for Bayern from 1974 to 1984.
He was part of the team which won the European Cup twice and was voted the Ballon d’Or winner in both 1980 and 1981.