The 116-metre high University Tower in Frankfurt-am-Main was demolished on Sunday by almost one tonne of explosives.
But it has left 50,000 tonnes of reinforced concrete behind.
Burkhard Muche, from demolition firm AWR Abbruch, said: “We need to firstly ensure cleanliness and order.”
Muche said on Monday that the clean-up operation would take around two weeks.
Around 30,000 people watched on Sunday as the tower fell.
Built in 1972, it was demolished to make way for two new blocks of offices and apartments