STOCKHOLM (Reuters) - Asylum seeker numbers in Sweden are set to drop by around 80 percent this year from a record 163,000 in 2015 as a result of tighter borders and tougher immigration rules, a government agency said on Tuesday.

Sweden took in more asylum seekers than any other European Union state relative to population last year. But even many liberal Swedes are having second thoughts, put off by reports of crime including sexual assaults by asylum seekers and financial strains on the nation's prized cradle-to-grave welfare system.

In a fresh forecast, the state Migration Agency said it expected Sweden to receive 28,000 to 32,000 asylum applications this year. Its previous forecast from July was 30,000 to 50,000.

Factors in the reduced numbers include the EU's deal with Turkey curbing migration from that nation's shores, border clampdowns along the main Balkan corridor to EU territory and the reimposition of selective identity checks at borders within the EU's Schengen passport-free travel zone, it said.

"...It has become harder to travel into and through Europe to reach Sweden," a Migration Agency statement said. Other EU countries including Germany also report a sharp fall in migrant arrivals from the more than one million registered in 2015.

Sweden's reputation for tolerance and stability made it a haven for refugees for decades. But the mood has changed since 2015 with many Swedes unnerved by reports of rising foreigner crime including gang activity in immigrant-heavy cities.

In addition, soaring costs - spending on immigration and asylum measures will account for around 7 percent of the budget this year - are seen by many as a threat to the welfare net.

Tougher rules introduced by the center-left coalition last year, including border checks and limits on family reunion, have cut numbers - and costs - dramatically.

This year, asylum applications are down to about 22,000. But anti-immigrant sentiment may have permanently hardened.

A poll by SIFO in daily Svenska Dagbladet at the weekend showed more than 40 percent of Swedes want the government to take an even tougher stance on immigration. A second poll showed that while a majority still support a multi-cultural society, that number has fallen sharply since last year.

The popular backlash has also had a violent dimension, with dozens of asylum centers burned down in suspected arson attacks. Two centers were assaulted in the Stockholm region last week.

Concerns about immigration have boosted support for the rightist Sweden Democrats, now backed by around 17 percent of voters in polls, up from the 13 percent they received in a general election in 2014.

(Reporting by Simon Johnson and Daniel Dickson; editing by Mark Heinrich)