By Shihar Aneez and Ranga Sirilal

COLOMBO (Reuters) - Dozens of Sri Lankan nationalists on Thursday protested against a three-day visit by U.N. Secretary General Ban Ki-Moon, demanding he leave the island and stop an investigation into alleged abuses at the end of the civil war.

Ban is in Colombo to assess Sri Lanka's efforts to address post war reconciliation.

Sri Lanka, under the ousted leader Mahinda Rajapaksa, crushed the separatist Tamil Tiger rebels in 2009. The United Nations accuses the military of killing thousands of civilians, mostly Tamils, during the final weeks of the 26-year-war.

It agreed last year, through a U.N. resolution, to address the alleged war crimes and make efforts toward reconciliation. The Tamil Tigers were also accused of violating human rights during the conflict.

Ban met Prime Minister Ranil Wickremesinghe on Wednesday and President Maithripala Sirisena on Thursday after addressing a youth conference in southern city of Galle. He is scheduled to visit former northern war zone of Jaffna on Friday.

However, in the capital Colombo, a few fringe nationalist groups, which still back Rajapaksa, protested in front of the U.N. office and main railway station, demanding he leave. Around 60 people took part in each demonstration.

A group of Buddhist monks carried pictures of their members and civilians they said were killed by the rebels during the war carrying banners saying: "U.N., where were you?".

Other group carried banners saying: "We don't want your help. Go home".

Speaking in Galle, Ban said young people needed to help the island emerge from the trauma of deprivations and injustice, and play a part in creating a better future.

"Please continue to prove that Sri Lanka is emerging from decades of adversity, suspicion and divisiveness. Please lead the way towards rebuilding, reconciliation and an appreciation of diversity in unity," Ban told the gathering.

In Jaffna, relatives of people missing, suspected of having been killed or taken to covert detention centers, told Reuters that wanted to meet Ban.

The government estimates there are around 65,000 people missing in the war and another insurrection, with victims' families and rights groups accusing the government of mostly being responsible for their disappearance during the height of the conflict.

"If we have a chance to talk to him, we'll tell what happened to my husband," said Easwary Mariyathas, a mother of three, who said her husband was arrested by the navy in 2009.

"I am pressurised to accept death certificate. We want justice, we expect a definite reply. I want to tell the truth to my kids. I can't cheat them saying their dad will come tomorrow and day after tomorrow."

The prime minister has said there are no longer any secret detention centers. The government has established an office to find missing people.

(Additional reporting by N.Parameswaran from Jaffna; Editing by Alison Williams)