I live in USDA zone 6b (Southeastern Pennsylvania), and Pawpaws are native growers here. Cold shouldn't be an issue in 6b, but winter length may have some effect. They don't do well in really long winters.
As for the conditions.
Soil: What I've noticed is that these don't like compacted soil at all. Lawns are not best. Even a mowed meadow can have problems. A wide circle of mulch around the base will help. These are naturally an understory tree so are used to woodland soils (soft and porous). Loamy is (as one would expect) the best type, but they grow well in sandy soils, and if not compacted much, they do great in clay. I haven't seen them in shale.
Light: As an understory/woods edge tree, it is adapted to some amount of shade. It can take full sun, but can burn (not an issue, I'm guessing, in NY). Planting by a building wall can help.
Watering: They like to stay moist, but if you're growing them in an area that gets over 30" a year of precipitation, you'll likely not need to irrigate much. That is, unless the tree is in full sun in dry soil.
Fertilizing: In the wild, these don't get chemical fertilizers at all. I also don't like to use them in plantings, but that's just My opinion. For production, you need to fertilize with something. I use mushroom soil (living near mushroom country) or composted manure, in a layer 1" deep to the dripline each year. Amazing how awesomely plants respond to that. You could probably also use a basic tree fertilizer.
There could be several reasons why they aren't in your area already.
Most plants are not found in every part of a country that they're hardy in. Sometimes their range hasn't spread so far yet.
Sometimes a plant does not naturalize well in an area, even when it can be grown. Like figs, in my area. They can grow, but they will never naturalize. There are several factors that could be at play in this, and it could be a factor in why you don't have Pawpaws in your area.
If you're near the ocean, or another saltwater body, that would definitely be an issue, as Pawpaws have poor salt tolerance. Even road salt from over the winter can damage them, when it runs off into a waterway.
And probably other factors. These trees grow a lot slower than I expected, when I planted them, and took from 5-8 years (from 12-14" whips) to begin bearing fruit. And the first few years of fruiting were a little slow.
Here are a few varieties recommended for growing in New York, quoting from here (Cornell):
Some cultivars for New York include: 'Davis', 'Sunflower', 'Taylor' and 'Taytwo'.