The Monarch flight, which had been due to leave the British city of Birmingham at 5.30pm GMT, was delayed for two hours as a result.
The two men were taken off the jet after all the passengers had boarded.
“It was crazy,” Marsha De Salvatore, an American who lives in Rome, told The Local.
“Not only was the plane delayed but I witnessed the sad reality of racism as these men were escorted off a boarded plane because others suspected them to be terrorists.
"The pilot told us 'what happened today was an act of total racism'."
De Salvatore explained that a British couple, who were standing in front of her in the queue to board the flight, became agitated.
“The husband was anxious and kept saying to his wife, ‘I’m not getting on the plane’. I thought perhaps he had a fear of flying. But just as they were about to hand over their boarding pass, they stepped out of the queue.”
About half an hour after the flight was due to leave, the pilot came out of the cockpit to explain to passengers the reasons for the delay.
“He said that while he believed the accusations were totally wrong, the police were called and the two men, and their luggage, had to be taken off the plane because two other passengers, who refused to get on the plane, thought they looked suspicious.”
Monarch, a UK-based airline, confirmed that two passengers “voluntary offloaded” themselves from flight ZB5408 and that “police were in attendance for two further passengers, who subsequently chose to travel today [Monday].
“Following an additional security check, the flight left 90 minutes later,” the company added in a statement.
De Salvatore said that while some other passengers became anxious, there was relatively little panic on board, and the air stewards were very reassuring.
The captain told passengers that the two men would be travelling to Rome the next morning as it was too late for them to make Sunday's flight.
“The pilot was so lovely," she said.
“While I’m happy that safety measures are in place, these people need to be careful about who they wrongly accuse.”
The incident came on the same day passengers on board a Ryanair flight from Norway to Manchester were evacuated after two men were acting suspiciously.
Shortly before the flight was due to take off from Moss airport in Rygge, just outside the Norwegian capital Oslo, passengers were asked to disembark and police carried out a search of the aircraft.
Two men were detained by police for suspicious behaviour but were later released. Police said that the two men, one British and one Sri Lankan, had been heard arguing loudly in the toilet, and some passengers thought they heard the word 'bomb', according to media reports.
Meanwhile, a passenger was removed from an easyJet flight in Pisa in early April after another passenger complained of “suspicious behaviour”.
On the same day, Meghary Yemane-Tesfagiorgis, from London, told Britain’s ITV news that he was the victim of racial profiling after being asked to leave an easyJet plane that was readying for take-off from Rome on March 29th because another passenger “felt uncomfortable”.