The population is only 582 but the median home sale price is $4.4 million. Lobster salad goes for $75 a pound at the local deli, so it may come as no surprise that named Sagaponack the most expensive small town in America to live in.

“People are drawn to Sagaponack because of the big open spaces, the sophistication and the beautiful architecture. And the farthest away from the ocean is three blocks,” said Prudential, Douglas and Elliman realtor Melissa Osbourne. “The area is extremely quaint, charming and historic.”

For a little more than $4 million, you can expect to get a 4,200-square-foot home with four bathrooms, four bedrooms, a screened-in porch and a heated swimming pool.

The tony town, located about 95 miles from Manhattan, has only a handful of businesses, includingTownline BBQ, Loaves & Fishes (home of that lobster salad) and a post office. But expect to rub elbows with big-name celebrities — like Bon Jovi, Ralph Lauren, Sarah Jessica Parker and Matthew Broderick — who frequent the hamlet.

Sagaponack is also tiny. There are only 17 students and three teachers in its school.

Nassau and Suffolk counties have more than half of the 50 richest towns in America that made Businessweek’s list.