The divisions reportedly happened on several occasions on Wednesday and Thursday at Stockholm's City terminal (Cityterminalen), reported newspaper Dagens Nyheter (DN) on Saturday.
The bus passengers were travelling to catch the ferry to the Finnish archipelago island of Åland.
"It was a very unpleasant experience...It felt like we were on our way to Arlanda Airport to be deported from Sweden," one of the passengers, Samer Chatila, told DN.
Chatila, who was born in Sweden, is currently studying mechanical engineering on Åland. He was travelling there with his brother Ahmad and a friend when the three noticed that the bus driver appeared to be dividing passengers based on their looks.
The driver stood between the exit and the bus and pointed the three to one of the buses, following them to make sure they got on board.
At first they thought the driver was being friendly, but they soon began to suspect that something else was going on.
They noticed that all "Swedish-looking" passengers were being directed to a different bus, while everyone who boarded their bus was foreign-looking or dark-skinned.
"We felt insulted, offended and discriminated against," said Chatila.
"I never thought this could happen in our Sweden."
When they challenged the driver, they were told that if they did not like the system they could get off the bus.
The driver claimed that the bus that the three friends were on was a direct route for passengers who had booked their trip in advance, while the other bus would make several stops to pick up more passengers.
But the driver did not check their tickets, Samer and Ahmad had not pre-booked, and both buses stopped to pick up passengers.
The Stockholm-Grisslehamn bus route is operated by People Travel Group on behalf of Eckerölinjen.
The company received several complaints and on Thursday evening, the director, Tomas Karlsson, learned of the alleged incidents.
"I immediately contacted the head of operations at People Travel Group who took the driver off the route," Karlsson said.
"It seems like this was a single driver's decision and it only happened twice, but that is bad enough."
In response, the company is introducing new training for staff starting on Monday, when the driver is expected to appear in front of the company board to explain his actions.
The Local/nr Follow The Local on Twitter