Initial projections show the centre-right CDU is just ahead of the centre-left SPD in Bremen, a state which the Social Democrats has governed for 73 years, reported Welt.
Exit polls for broadcasters ARD and ZDF put support for the SPD at around 24.5% and the centre-right Christian Democratic Union, Chancellor Angela Merkel's party, up to about 25.5%.
The historic losses could see the SPD finish second in the northwestern city-state, Germany's smallest, for the first time since WWII.
The polls put support for the Greens in Bremen up to about 18 percent. That party is currently the Social Democrats' junior coalition partner in Bremen, and could decide whether or not it keeps its hold on the state government.
The exit polls put The Left (Die Linke) at 12 percent, while the far-right Alternative for Germany (AfD) have 7 percent. The pro-business FDP scooped 6 percent, according to the initial projections.
The top candidate of the CDU, Carsten Meyer-Heder after the exit polls. Photo: DPA
The results mean that SPD politician Carsten Sieling, 60, could remain Bremen mayor if his party form a coalition with the Greens and The Left, which would be a first in Bremen. It would also mark the first time The Left was in power in a western German state.
But if that alliance isn't agreed then there's the possibility that CDU candidate Carsten Meyer-Heder, 58, who only joined the party a year ago, could be elected Bremen's first Christian Democratic mayor with the help of the Greens and the FDP in a coalition.
Bremen is the only federal state with a four-year legislative period; all other German states now vote every five years.
As The Local reported, the Bremen elections will likely send shockwaves to the government in Berlin where a shaky coalition between the SPD and the CDU/CSU stands.
Due to the relatively complex electoral system in Bremen, it takes longer for reliable figures to become available.
A preliminary official final result will therefore not be published until Wednesday, officials said.