The Cologne administrative court said Cologne must ban the dirtiest diesels from its centre and other streets from April 2019 to tackle dangerously high levels of nitrogen dioxide (NO2) emissions.
In the smaller city of Bonn, the restrictions would apply to only two streets.
The ruling, which can be appealed, is the latest victory for German environmental group DUH which has launched a raft of court cases to force local authorities to boost air quality.
Major urban areas including Stuttgart, Frankfurt and the capital Berlin have already been slapped with legal orders to cut emissions, while Hamburg decided of its own accord to expel the worst polluters from some zones.
Unpopular driving bans
Faced with mounting public anger, Chancellor Angela Merkel's government is scrambling to ward off the unpopular driving bans that promise to not only cause transport upheaval but further hammer the resale value of diesels.
Diesel engines have come under intense scrutiny since Volkswagen admitted in 2015 to installing cheating software in millions of cars to dupe emissions tests.
The scam made the cars seem far less polluting in labs than they were on the road, and suspicions of similar trickery have since spread to other carmakers as well.
But the country's mighty car giants have been stubborn in their refusal to foot the bill for cleaning up diesels in order to bring them into compliance with EU law.
The government wants automakers like Daimler, VW and BMW to pay for hardware fixes to retrofit older diesels with more effective exhaust treatment systems.
But carmakers have so far responded mainly by offering software upgrades for newer diesels and trade-in bonuses — which would still leave drivers shelling out thousands for new cars.
Law could exempt cities from bans
Merkel is also backing a proposed law that would exempt cities from diesel bans when only “minor” infractions against legal pollution limits have been registered.
A total of 65 German cities last year recorded NO2 levels that surpassed European norms of 40 microgrammes per cubic metre of air.
In Cologne, the level spiked to 62 microgrammes while Bonn peaked at 47.
Nitrogen oxides (NOx) including NO2 are estimated to cause thousands of premature deaths in Germany each year.
According to the World Health Organization (WHO), the gases aggravate asthma and bronchitis symptoms and are linked to cardiovascular and respiratory disease.