By Mehreen Zahra-Malik

ISLAMABAD (Reuters) - Pakistan plans talks with Afghanistan and the United Nations refugee agency to move longtime Afghan refugees to camps at home, the foreign office said on Thursday, after the numbers of those returning plunged this year.

Pakistan has the world's second largest refugee population, with more than 1.5 million registered, and about a million unregistered, refugees from neighboring Afghanistan, most of whom fled the Soviet occupation of their country in the 1980s.

The U.N. says the number of Afghans voluntarily returning from Pakistan has fallen to about 6,000, well below last year's 58,211, as violence worsens in Afghanistan, where the government and its U.S. allies are battling a stubborn Taliban insurgency.

Pakistan's foreign ministry said it would immediately approach Afghanistan on the political and diplomatic fronts, while the ministry for frontier regions would engage with the U.N. refugee agency and Afghanistan's ministry of refugees.

The talks would seek ways to ease "early returns as well as the possibility of shifting Afghan refugees gradually from Pakistan to safer and peaceful areas of Afghanistan, where the Afghan government should establish settlements," the foreign office said in a statement.

Hussain Alemi Balkhi, the Afghan minister for refugees and repatriation, said, "We know that the refugees face harassment and hardship, and we are working with Pakistani authorities to address these problems."

He confirmed plans for a three-way meeting on July 19 with Pakistan and the U.N. refugee agency.

On Wednesday, Pakistani Prime Minister Nawaz Sharif allowed the 1.5 million registered refugees to stay on for six more months.

The registration deadline extension came soon after officials told Reuters at least 500 Afghan refugees had been arrested in the northwestern border province of Khyber Pakhtunkhwa and deported as a security risk.

Pakistan's Dawn newspaper said more than 2,000 refugees were arrested in the last month, and 400 deported to Afghanistan.

(Additional Reporting by Hamid Shalizi in Kabul; Writing by Mehreen Zahra-Malik)