The crane fell onto a crowded Aldi supermarket in an industrial area of Bad Homburg, at around 11.30 am.
Around 47 people were in the supermarket at the time - 40 customers and seven staff.
The 74-year-old mother of the victim was seriously injured as was the crane driver who was flown to hospital, police said.
An initial inspection of the site led to suggestions that the crane had been turning when it fell, and that the 20-tonne concrete blocks fixed to the crane for balance, fell onto the building, smashing through the roof.
The Frankfurter Rundschau newspaper said rescue workers had only been able to enter the building via the remains of the crane as the rest of the building was seriously damaged and could easily collapse. The paper said that from what workers had seen, there could be other fatalities. Rescue teams are expected to be on site until Thursday morning.
The fire brigade tried to use another crane to lift up the collapsed one, after attempts to prop up the building failed, but the second crane was not strong enough to do more than prevent further collapse of the building.
A third, special crane was on Wednesday afternoon, being brought to the scene from Frankfurt.
The person who died was trapped in the wreckage of the building, while a spokesman for the Bad Homborg fire brigade said it was possible that more bodies would be found in the wreckage.
The Rundschau reported onlookers were angered as during the rescue operation fresh concrete continued to be delivered to the site.
READ MORE: Cable car fault leaves 76 passengers dangling