The men were arrested Wednesday night in Midtown and will be appear in criminal court in Lower Manhattan later today.
They will be charged under the New York state's anti-terror laws. Federal prosecutors are aware of the case, but declined to charge them under federal terrorism laws.
This is the 13th plot thwarted against the city since the September 11th attacks, and the kind of homegrown, "lone-wolf" scenario that federal law enforcement officials were recently warned of.
Authorities say the suspects talked about attacking synagogues in Manhattan, but investigators believe they were planning out loud and no specific synagogue was selected.
The two men, Muslims of North African descent, had already obtained some guns and were trying to buy more more, as well as a grenade, according to police.
Their attempts were detected by investigators with the NYPD's Intelligence Division, who moved in to set up a sting.
One of the men, in his 20s, lives in Queens. That man was apparently going to sell drugs to buy the guns. He has prior arrests, including for drug possession.
The plan was already in the works at the time of Osama bin Laden's death, and is not believed to be in retaliation for the killing.
Mayor Michael Bloomberg and Police Commissioner Ray Kelly are expected to discuss the arrests this afternoon at City Hall.