The problem was that until recent decades, New York harbor was so full of pollution it was overwhelming the oysters. (19 million people's worth of sewage and runoff is a lot.)

A couple of large investments into waste treatment plants and now it is slowly getting back on its feet.