London's Metropolitan Police said Domenico Rancadore, 64, was detained on Wednesday on a European arrest warrant.
Considered an important member of the Sicilian mafia group Cosa Nostra, Rancadore has been wanted by Italian police for 19 years and is on Rome's list of most dangerous criminals.
Shocked neighbours in the west London suburb of Uxbridge, where Rancadore was living with his British wife and running a travel agency, described him as "one of the best neighbours you could have".
He has an outstanding prison sentence of seven years in Italy for mafia-related activities, according to the arrest warrant.
"He was taken into police custody and is scheduled to appear at Westminster Magistrates' Court today (August 8th)," police said in a statement.
Rancadore, who went by the nickname "u profissuri" (the teacher) in the mafia, had been living a "comfortable" life in London before his arrest, Italy's interior ministry said in a statement.
The ministry said Italian police had worked with their British counterparts to locate the fugitive.
"The operation is the result of the important relationship of international police cooperation ensured through Interpol," its statement added.
Rancadore is wanted in Italy over mafia activities between 1987 and 1995 in Palermo, the main city on the island of Sicily, according to the arrest warrant.
The Italian interior ministry said several witnesses had described him as a leading member of the Palermo mafia.
"In particular, in the 1990s he played the role of chief of the Cosa Nostra in Trabia, Palermo," it said.
An international arrest warrant for Rancadore was issued in 1998.
A neighbour said he had been known in London as Marc Skinner and had two children with his English wife.
"I know him very well and he's one of the best neighbours you could ever have," said Joan Hills, 76.
"They have been here for years. I've seen the children grow up with my children.
"I don't know the ins and outs of this, but they are the nicest people that you could wish to meet."