By Simon Johnson and Johan Sennero

STOCKHOLM (Reuters) - Britain cannot expect to have full access to the European Union's single market if it rejects rules which allow European citizens free access to its labor market, Sweden's prime minister said on Wednesday.

Sweden has been a close ally of Britain - its fourth largest export market - in efforts to pull down internal trade barriers within the EU and streamline bureaucracy, but Stefan Lofven said London could not expect the same treatment when it leaves.

"You cannot have free movement of goods and services and capital and not people," Lofven told Reuters. "The Union doesn't work like that and there will be differences if they are not members."

Britain's minister charged with negotiating Brexit said on Monday it was pressing for a "unique" deal with the EU to restore sovereignty, reduce immigration and boost trade with the bloc after their split.

But European leaders have warned Britain it cannot cherry pick which rules it follows if it wants to continue to have unfettered access to Europe's markets.

Britain's vote to leave the European Union after more than four decades sent shockwaves throughout the bloc and some EU politicians have called for a tough line in negotiations over the divorce procedures.

Lofven, head of Sweden's center-left government, said Britain and Europe should work together to make Brexit a success.

"We should make it as easy as possible and as beneficial for both parties as possible," he said.

There is no hurry for negotiations to start, Lofven said, but the hiatus cannot last forever.

"We have other things to do than think about than Brexit," he said.

"We have to make sure we create jobs ... we have to meet the climate challenge, we have security questions, the fight against terrorism. That's what we should focus on."

For both Sweden and the EU, asylum is one of the biggest challenges for the coming years.

More than 1 million migrants and refugees crossed into Europe last year with 163,000 seeking asylum in Sweden.

With new arrivals sleeping in tents, Sweden made a U-turn on decades of generous asylum policies late last year, introducing border controls, temporary resident permits and other tough measures.

The flood has turned into a trickle, with around 20,000 seeking asylum in the January to August period.

Lofven said the tough stance would remain in place until other EU countries shared the burden.

"When we do not have common rules, we have to take the same approach as everyone else," he said.

Sweden's humanitarian reputation may have been dented, but Lofven said it still punches above its weight.

"We are a big humanitarian donor to the countries close to the conflict in Syria, the Middle East and Africa. We are a big player in the U.N.," he said.

"Everyone I meet outside Sweden says that we are a humanitarian superpower and we will remain one."

(Reporting by Simon Johnson and Johan Sennero; Editing by Alison Williams)