By Daniel Flynn and Caroline Stauffer

RIO DE JANEIRO (Reuters) - Brazil's expulsion from Olympic venues of spectators carrying banners or shouting slogans against interim President Michel Temer stirred outcry on social media on Sunday but Games organizers threw their support behind the move.

A video of four security forces agents in red berets and camouflage gear surrounding a man holding a 'Temer Out' banner and manhandling him from the stands of the archery event on Saturday prompted a barrage of criticism from Brazilians.

"A gold medal for repression" tweeted several people following the protester's expulsion from Rio de Janeiro's Sambadrome.

A spokesman for Temer, who took office in May when President Dilma Rousseff was placed on trial on charges of breaking budget laws, said the government had not given security forces any instructions regarding political banners during the Games.

The Brazilian agency that provides protection for major events (SESGE) said in a statement its agents were asked to intervene by spectators at the archery venue because the protester was shouting while competitors were taking aim.

Brazil is hosting South America's first Games amid a political crisis that has deeply divided the nation of 200 million people. Rousseff has denied any wrongdoing and accused the conservative Temer of staging a "coup" against her left-leaning government.

A spokesman for the International Olympic Committee (IOC) noted that the Olympic Charter prohibited the presence of political propaganda within venues, as well as any kind of religious or racial messages.

"Those who make political statements in the venues are requested not to. If they resist, they are kindly requested to leave," Mario Andrada told a news conference. "This is a venue for sports. They need to be focused on that."

Andrada said the organizing committee was informing the public at events that no political slogans would be tolerated.

Mark Adams, IOC director of communications, said incidents of this sort arose at every Olympics.

Despite a focus on international sporting excellence, the Games have often had explicitly political overtones, ranging from the 1936 Berlin Olympics in Nazi Germany to the boycotts of the Cold War years in Moscow in 1980 and four years later in Los Angeles.

More recently, the Sochi Winter Olympics was marked by the arrest by Russian authorities' of punk band Pussy Riot amid gay rights demonstrations.

With political feeling in Brazil running high ahead of Rousseff's trial this month, Temer was booed by some members of the crowd during Friday's glittering opening ceremony in the Maracana stadium. The jeers were even louder at a square in a poor area of downtown Rio where thousands watched the ceremony on a giant screen.

Small protests against Temer at the Games began before the opening ceremony, during the soccer tournament that starts two days before.

In the southeast Brazilian city of Belo Horizonte, spectators wearing letters on their T-shirts spelling 'Temer Out' were expelled from the stadium, local media reported.

(Reporting by Daniel Flynn and Caroline Stauffer; Editing by Andrew Hay)