“Protecting family businesses and jobs is essentially a legitimate reason to exempt companies partly or wholly from the tax”, court vice president Ferdinand Kirchhof said.
But the way the tax had been implemented in law was against the basic rules of the German tax system, he said.
In 2012 the state lost out on taxes of almost €40 billion under the current rules, with only €4.3 billion of business inheritance tax being collected by the authorities.
The exemptions passed in 2009 mean that heirs of a family firm can partially or completely escape inheritance tax if they have already been running the business for several years.
A majority of the business's assets must be related to production and the heirs must maintain workers in their jobs. The rules are particularly favourable for companies with fewer than 20 employees.
The judges said that the present rules were unconstitutional because large firms were benefiting excessively and there were many loopholes in the system.
But they said that lawmakers could either fix the problems they identified with the existing exemptions or create a wholly new structure.