The Croats were well worth their victory - which came 10 years after the biggest result in their history when they beat the same opponents 3-0 in the World Cup quarter-finals - as they took charge of the group and are now almost guaranteed a place in the next round.
It will be back to the drawing board for the Germans, who ended the match with 10 men.
Skipper Michael Ballack said the Germans had to recognise they must improve. "Our movement was not fluid enough throughout the match and as a consequence we eked out few chances. We played below our level in all departments. Now we absolutely have to win our last match against Austria," said the Chelsea star.
After a cagey and dull opening the game burst into life midway through the opening stanza. Croatia took the lead on 24 minutes with a move of delightful fluency.
Quick, zipped passes and intelligent running off the ball created space for left-back Danijel Pranjic and his pin-point cross picked out Darijo Srna at the back post. He beat the hesitant Marcell Jansen to the ball to prod home with an outstretched leg.
Another sweeping Croatia move on the half hour should have resulted in a goal as Ivan Rakitic chipped the ball into Ivica Olic, who cushioned a header back for Portsmouth's Nico Kranjcar but he slipped as he shot and fired miles over with the German defence opened up like a tin.
Germany hit back with a 25-yard Michael Ballack free-kick that Stipe Pletikosa in the Croat goal palmed to safety before also repelling a shot from Miroslav Klose moments later.
The Germans were pushing for an equaliser and centre-back Christoph Metzelder headed narrowly over from a Torsten Frings corner. But Croatia should have doubled their lead three minutes before the break as Vedran Corluka left Jansen for dead and crossed into the box where Olic teed up Kranjcar once again only for Jens Lehmann to make a stunning point-blank save from the midfielder's left-foot volley.
On the stroke of half-time Germany had the ball in the net but Klose's effort was ruled out as Mario Gomes was penalised for the slightest of contacts with Pletikosa.
Early in the second period Germany didn't really look like getting back into the match despite a Ballack snap shot that sailed over the bar. On 62 minutes the lead was doubled as Rakitic's right wing cross took a huge deflection off Lukas Podolski and beat Lehmann at his near post, coming back off the upright and presenting Olic with a simple tap in.
Germany did at last come alive and Pletikosa had to divert behind a fierce cross-shot from substitute Bastian Schweinsteiger. Podolski gave them a life-line with 11 minutes to go as he thrashed home a loose ball in the box on the half-volley but it wasn't enough.
And Germany's misery was complete in stoppage time when Schweinsteiger was dismissed for a push on substitute Jerko Leko.