Cops say 37-year-old Marc Lamparello of New Jersey was arrested after he entered the church with two gas cans, lighter fluid and butane lighters.
Police released images of what they say he had with him when he walked into the church at 7:55 p.m. Wednesday night.
He told police he had run out of gas and was just "cutting through" on his way to get gas.
Police said he was stopped by an usher as he tried to enter and some of the gasoline spilled on the floor. The usher called the NYPD's counterterrorism force right away.
"His basic story was that he was cutting through the cathedral to get to Madison Avenue, that his car had run out of gas," NYPD Deputy Commissioner John Miller told reporters. "We took a look at the vehicle. It was not out of gas, and at that point, he was taken into custody and brought to the Midtown North station house."
New York's Cardinal Timothy Dolan lives next door and understands how close they came to a tragedy.
"You have to admit some fear, but I am grateful for our security and the NYPD for good work," he told Inside Edition.
Lamparello has been charged with attempted arson, reckless endangerment and trespassing. Police say he had a one-way ticket to Rome, Italy, booked for Thursday night.
On Monday, Lamparello was arrested at Sacred Heart Cathedral in Newark, New Jersey, after he allegedly refused to leave at closing time. Lamparello was charged with resisting arrest, trespass and disorderly conduct for that incident.
Lamparello reportedly lives in the basement of his parents' New Jersey home and is said to be deeply religious.