By Tom Polansek

CHICAGO (Reuters) - CME Group Inc <CME.O> plans to strengthen efforts to deter wrongdoing in markets by raising the maximum possible fine for rule breakers, the company said on Thursday, further raising the stakes for rogue traders following the first U.S. criminal convictions for the manipulative practice known as spoofing.

The owner of the Chicago Board of Trade, New York Mercantile Exchange and other markets is preparing to increase the top monetary penalty to $5 million per offense from $1 million starting on Dec. 14, according to a notice sent to clients.

The CME is making that change and others to address the "increasing complexity of disciplinary matters," after reviewing its procedures, the notice said.

The company, a self-regulatory organization, fines firms and individuals and suspends them from its markets for violations ranging from failing to supervise employees to wash trading, in which traders sell contracts to themselves to make a market look more active than it is.

The U.S. Commodity Futures Trading Commission and National Futures Association also oversee market participants.

Lawyers and traders said the rise of computerized and algorithmic trading has complicated regulatory work due to the large quantity and speed of transactions. Regulators have focused recently on spoofing, after a provision against the trading practice was implemented as part of the 2010 Dodd-Frank financial reform, lawyers said.

Spoofing usually involves using algorithms to place orders with the intent to cancel them before execution. By creating an illusion of demand, spoofers can influence prices.

"Previously, traders were on the floor, trading physical products," said Braden Perry, who was formerly a senior trial attorney for the CFTC. "It's a whole new world now."

Raising the maximum fine helps the CME address "the realities of the world that sometimes a $1 million penalty might not be enough to deter a certain kind of behavior," said Perry, a partner at the law firm Kennyhertz Perry in Kansas City, Missouri.

The CME notice said it wanted to make its rules "more applicable to the types of disciplinary cases going through the enforcement process."

A company spokeswoman said the notice referred to all types of cases and that CME was changing disciplinary procedures "to increase the deterrent effect of the penalties."

Bill Harts, chief executive of Modern Markets Initiative, which represents high-speed traders, said algorithmic trading had not made disciplinary matters more complex.

"When trades are executed by computer, there is a robust, immutable audit trail that is simple for regulators to look at, analyze, and act on," he said.

Last month, a British trader, Navinder Sarao, became the second person convicted of spoofing U.S. futures markets. The first, Michael Coscia, is appealing his conviction.

(Reporting by Tom Polansek; Editing by Jacqueline Wong)