Joe Segreto was born into the restaurant business, which is one reason he’s so reluctant to leave it.

During his prime as a young man in the 1970s, he ran a French Quarter restaurant that became a magnet for celebrities who gilded his life with their stardust. That has something to do with it, too.

And later still, for the third act of a career that has run in tandem with bold-face names, it was his own restaurant, Eleven 79, that put him back in the game after a long hiatus. It’s hard for him to shake the feeling that he could do it again.

Eleven 79 was known for Creole-Italian cooking anchored by equal parts veal and crabmeat. It was known for its darkly elegant ambiance inside and for the luxury cars jamming the curbs of the surrounding warehouses outside. And most of all it was known for Segreto, for the dapper man wearing a jacket and pocket square who could address by name everyone in the room — if not when they walked in the door then surely by the time they walked out.

Segreto closed Eleven 79 in late April after a 15-year run. But with his history in the business, how can he say his career is over? He can’t, not even as the dining scene has streaked ahead of the style where he hung his hat and as the toll of ill health has crept slowly behind him.

Pondering the future, Segreto is pragmatic but also hopeful, a mix he credits to the nature of being a restaurateur.

“This is a comeback business,” said Segreto, 74, during an interview at his Garden District home. “In other words, once you’re in this business, it’s something you always come back to. You have to. What’s more wonderful than pleasing your guests? What’s better than seeing that people want to come back because of what you provided? It’s like show business. Once you get a taste of that, you’re always coming back.”

A “life in food”

The decision to close Eleven 79 was a tough call, and one Segreto immediately regretted since, as soon as he did, he heard from Tony Bennett, an old friend from his Las Vegas days. Bennett was in town to perform at Jazz Fest and had hoped to bring his new touring partner Lady Gaga around for dinner. Hosting visiting celebrities has long been in Segreto’s wheelhouse, but his malaise at the closing goes beyond disappointing his old friend.

“My whole life is in food,” he said. “It’s one that has been very joyful, but one that’s been very sad, too. It was a sad day when I had to close that place.”

Segreto is ill. He speaks of it frankly. He has trouble with his throat that he attributes to chemotherapy he undertook many years ago to treat a cancer. Now, he has difficulty swallowing and his speech is strained. The health woes provided the final nudge to close a business he knew was on the edge for a while.

“Eleven 79 was a tremendous success in the beginning, but with the advent of all the newer restaurants, things changed. So many places went casual,” he said. “I knew I needed to make it more casual, to change to the times, but I didn’t have the energy after I got ill.”

So the Annunciation Street restaurant is in limbo, and the sturdy shutters that always gave it the feel of a lair now making it feel boarded up. If his health does not improve, Segreto said he’d sell the business, and he has been showing the building to prospective buyers. But he also hopes with better health he could recapitalize and reopen a different sort of Italian restaurant. It would be more casual, he said, less expensive and smaller in scale, and it might be at a different address.

“My enthusiasm and my optimism tell me that if I stick to it, I could come back, no matter how many new restaurants are out there,” he said.

From the Quarter to the Sahara

Born in 1940 in the French Quarter, he was named Salvadore Segreto, after his grandfather, who emigrated from Sicily and established his family in New Orleans a generation earlier. But he has always been known as Joe, after his father, Joseph Segreto.

During World War II, the senior Joseph Segreto opened an Italian restaurant called Tosca’s on Bourbon Street, next door to Galatoire’s Restaurant (the address today is Galatoire’s 33 Bar & Steak). He would open or manage other restaurants and clubs around the French Quarter, including Segreto’s on St. Louis Street, where he befriended the New Orleans-born singer Louis Prima. The younger Joe Segreto was then just coming of age, and he recalls today there was no question that he would get his start in restaurants.

“We were restaurant people,” he said. “Restaurants were what we did. When I started looking for work, I went to places where my father had friends.”

That led him to Brennan’s Restaurant, which had only just opened on Royal Street when he started his job search in the mid-1950s.

“I was 16,” Segreto said. “I started as a dishwasher, then bus boy, then waiter, then cook. I was just a kid, but I had the opportunity at a major restaurant to learn cooking and service at a high level.”

To name his mentors, he rattles off an entire generation of the Brennan family tree and the Brennan’s Restaurant chef of the time, Paul Blangé.

Later, he became a band manager and in 1960 he shipped off for work in Las Vegas. He was tending bar at the storied Sahara Casino at the same time that Louis Prima was performing on its stage. The old New Orleans ties paid off, and by 1961 Segreto was Prima’s manager, a gig he would hold for about 10 years.

“Suddenly, here I am, a guy in my 20s, the manager of one of the greatest acts in show business, and we’re in Vegas when all the greats are there,” he said, listing the Rat Pack lounge act icons of the era.

Some became his good friends, including Bennett, who frequently calls Segreto to check in and who hosted him backstage during his Jazz Fest performance last month.

“He’s a great guy,” Bennett said in a brief interview during one of those calls to Segreto last week. “We’ve been friends for years, we keep up.”

When Segreto returned home from Las Vegas in the 1970s, he got right back to the restaurant business as manager at Elmwood Plantation on River Road. The acclaimed Jefferson Parish restaurant was run by Joseph Marcello and Joe C. Marcello — the brother and son, respectively, of Carlos Marcello, mob boss of New Orleans of the era. To Segreto, they were all old family friends from his childhood in the French Quarter, and they would remain close throughout his life.

Cuisine and star power

Elmwood Plantation was destroyed by fire in 1978, though by then Segreto and the Marcellos were in partnership for another major restaurant in the French Quarter. They bought Broussard’s Restaurant on Conti Street in 1976 and conducted a detailed renovation across its historic mansion and courtyard. The restaurant dated to 1920 and had been in decline, but it quickly became a hot destination under the new ownership.

“It was an exciting time,” Segreto said. “It was a high, high Creole cuisine we were serving, and all the celebrities wanted to come.”

He recalls many of those celebrity visits vividly. Muhammad Ali ordered two duck entrees and after the huge dinner insisted on shaking hands with all of the dishwashers. Actor George Burns wanted Segreto to introduce him to everyone around the dining room. And during one visit, Sammy Davis Jr., another old friend from Las Vegas, lit a match to “sing by candlelight,” as Segreto remembers it, for anyone in the house celebrating a birthday.

In the early 1980s, Segreto expanded to the North Shore, partnering with New Orleans politico Eddie Sapir (then a judge) to develop a Mandeville country home into a restaurant called The Shadows, which is now the events venue Benedict’s Plantation.

The oil bust of the mid-1980s and the hard times it brought for local businesses upended these ventures, Segreto said. He and his partners sold Broussard’s and The Shadows, and he went to work for the Marcellos running a West Bank landfill through much of the 1990s.

Then in 2000, he opened Eleven 79, along with his friend and long time chef, the late Anthony DiPiazza, transforming a split-level Creole cottage that dates to the 1830s into a noble dining room and bar of exposed brick and timbers. It was his return to the business he loved, and it felt like a homecoming as customers from past chapters of his career became regulars once again.

“Joe knew so many people, and they all followed him wherever he went,” said Joe C. Marcello. “They wouldn’t have come if the food was no good. The food was exceptional because Joe always had a hand in the kitchen.”

Would his following turn up for another go ‘round, even now? Segreto isn’t sure. Many of his regulars moved away after Hurricane Katrina, he said, and others have died. But he doesn’t talk like a man settled into the idea of retirement. His phone rings constantly and his elegantly furnished home is piled and strewn with cookbooks, a collection he revisits for “ideas, inspiration and enjoyment,” he said. But if his career has indeed reached an end, he doesn’t have many regrets.

“I’ve served the father, the son and the sons of those sons,” he said. “I’ve served generations of people who have become my friends and customers. You have the opportunity in this business to make your customers and friends happy. You have the opportunity to give other people an opportunity too. You get to take a young kid and show him or her how they can make money and move ahead, and it’s very gratifying when you see them do that.

“As frustrating as this business can be sometimes, the joy overcomes all of that, and when you’re doing well the money overcomes it too.”

Follow Ian McNulty on Twitter @IanMcNultyNOLA.