Mayor Michael Bloomberg is still happy living in his Upper East Side townhouse but the project that could tempt him to try Downtown living is the $50-million, box-shaped condos architect Santiago Calatrava is designing at the South Street Seaport.

“If the Calatrava building is built—a magnificent piece of architecture—I’m not going to move Downtown, but boy you think about it,” Bloomberg told the publisher and editors of Downtown Express, Gay City News, and The Villager on Monday. “It would be spectacular.”

Bloomberg said the sensitivities from some 9/11 families and others about cultural institutions next to the proposed memorial should be extended to the entire World Trade Center site, minimized his differences in recent weeks with World Trade Center developer Larry Silverstein and Governor George Pataki, and indicated that the developer will likely have an easier time finding tenants for Tower 2 than he will for the first office building where construction is expected to begin—the Freedom Tower.

When 9/11 family groups led the successful fight this fall to get two museums bounced from the area near the memorial, Bloomberg criticized the governor for superseding the Lower Manhattan Development Corporation, a federally-funded state-city authority. But on Monday, the mayor said the same sensitivities that led to International Freedom Center and Drawing Center leaving would have to apply to the entire site.

“This is clearly not your average piece of property and different standards may very well apply on that site,” Bloomberg said, citing other exceptions to free speech such as shouting fire in a theater or prohibitions about what you can say in a courtroom.

He said curators could work with people who understand the site’s importance on programming. “That’s to be worked out,” he explained.. There’s no easy answer to this. You want to have freedom of expression. There’s nobody willing to stand up for the arts more than I am, but I will say there is something different about this piece of property.”

Asked about the progress Silverstein is making in rebuilding, the mayor said, “He’s asked for incentives from the state and city—we’ve given him those, now he’s got to rent it.” A few hours earlier, Bloomberg and Silverstein agreed to delay a city vote on the $3.35 billion in tax-free Liberty Bonds the developer wants in order to build five offices at the WTC. Pataki has said that Liberty Bonds should be used at the WTC, but Bloomberg said “Liberty Bonds are government money, people’s money, and we should look to see what’s the best use of the people’s money.”

The mayor did not refer directly to any problems Silverstein may have finding tenants to rent space in the 1,776-foot Freedom Tower, where work is scheduled to start in a few months, but real estate and other business leaders worry that a high-profile building on West Street, a block away from the Trade Center subway station under construction, will be a harder sell than Tower 2, adjacent to the station.

Bloomberg said he’s not convinced 10 million square feet of office space is needed at the site. “I also have cautioned that we should take a look at the marketplace and see what the marketplace wants,” he said. The mayor explained that the reason he appointed Bill Rudin, president of Rudin Management, a real estate firm, and Lawrence Babbio, Jr., president of Verizon, to the Lower Manhattan Development Corporation board is because they know what the market needs.