By Dan Whitcomb

(Reuters) - New York City police investigating a bombing in Manhattan over the weekend said on Monday they wanted to question two men who appeared to stumble over a second device made from a pressure cooker that had been left inside a bag lying on a city street.

In a lucky break that helped authorities to thwart a second detonation on Saturday, the men walked away with the bag after taking out what turned out to be a homemade bomb and leaving it exposed on the pavement on 27th Street.

Police discovered the device soon after a bomb exploded four blocks away on 23rd Street in the Chelsea neighborhood and left 29 people injured.

The two men, caught on surveillance video footage, are considered potential witnesses, not suspects, in the bombing, said Robert Boyce, chief of detectives for the New York City Police Department.

"They looked like they were two gentlemen just strolling up and down Seventh Avenue at the time. We have no information that would link them to this at all," Boyce said at a briefing. "However, we still want to talk to them, obviously."

Boyce said the two men were seen picking up the bag containing the device, removing it and then leaving with the bag, for reasons that remains unclear. Police provided no specific description of the men who they said took the bag.

"Once they picked up the bag, they seemed incredulous they had actually picked this up off the street and they walked off with it," Boyce said. "So we'll find out, we'll put their images out. Hopefully we can get them identified."

Earlier on Monday an Afghanistan-born American suspected of detonating the bomb in Chelsea and of planting other devices in New York and New Jersey was arrested following a gun battle with police.

Ahmad Khan Rahami, a 28-year-old naturalized U.S. citizen from Elizabeth, New Jersey, was taken into custody hours after authorities identified him as the prime suspect in the Saturday night blast.

Police suspect Rahami was also behind a bomb that exploded in a New Jersey beach town on Saturday, as well as leaving the device found on the sidewalk after the New York blast. On Sunday, five more devices were found in Elizabeth, the suspect's hometown.Police in Linden, New Jersey, which neighbors Elizabeth about 20 miles (32 km) west of New York, captured Rahami after responding to a complaint by a bar owner of a man sleeping in the closed establishment's hallway.

(Reporting by Dan Whitcomb; Editing by Frank McGurty and Mary Milliken)