Angelo Massaro from Taranto in the Apulia region, was accused of killing a young man in 1995.
He was sentenced to 24 years in jail due to evidence from an intercepted phonecall and a tip-off from an informant.
A misinterpretation of a dialect word in the phonecall appears to be behind the wrongful conviction.
Massaro was recorded saying "tengo stu muert" on the phone to his wife a week after the murder, which roughly translates as "I've got this dead person".
However, Massaro's lawyer, Salvatore Maggio, said he was using the word 'muert' in the dialectal sense of 'a dead weight', and was actually talking about a snowblower which was attached to his car.
Maggio was also able to prove that his client had an alibi for the time of the murder; he had been at the local health authority seeking advice about drug addiction.
The Catanzaro Court of Appeal has now ruled that Massaro did not commit the crime, and the 51-year-old has been released.
During his time behind bars, he achieved a high-school diploma and a diploma in surveying, and took a university law course which helped him put forward his own appeal.
He and his lawyer first asked for a retrial in 2011, but the request was refused by the local court. However, Italy's Supreme Court overturned this decision, reopening the case in 2015.
Maggio said that his client's "head was spinning" after his release.
"It's not easy after 21 years in a cell to see cars, bars, streets - the world has changed," the lawyer told Italian media on Thursday. "He's really disorientated."
"Studying helped me a lot, but it was yoga, meditation and sport which stopped me going mad," he told Il Corriere TV. "They allowed me to close a mistaken chapter of my life and to survive this persecution from the legal system which I would not wish on anyone."
He added that "nothing will undo the suffering of these past 20 years".
Astonishingly, this was the second time Massaro was falsely convicted of murder.
In 1991, he served the first year of a 21-year sentence for another killing, before being judged innocent and compensated by the Italian state.