Seen from satellite views on Google Maps, the area looks more like a wide spot in the road, with just a few visible buildings. Swastika was founded in the 1800s, long before the Nazis came to power in Germany.
"We regret that individuals, for out of the area, that lack the knowledge of the history of our community, become offended when they see the name," Jon Douglass, supervisor for the town of Black Brook, told CNN. "To the members of our community, that the board represents, it is the name that their ancestors chose."
The unincorporated area is so small, it doesn't even have a post office. It's located in Clinton County, which abuts the Canadian border.
"I think that's probably, maybe some viewpoint that it's associated with hate," Douglass told NPR. "But then I believe there are others that do not associate it with hate," he said. "Did the Hindus and the [Buddhists] and all them, did they erase it from their religious history because of the Germans?"
The symbol dates back 7,000 years and is sacred in beliefs including Hinduism and Buddhism.