I'm in USDA hardiness zone 6b. I'm growing eastern prickly pears as outdoor perennials, and would like to grow some other cacti as well. From what I gather, cold hardiness has a lot to do with soil moisture levels, as cacti with a better water supply tend to get more cell wall ruptures, inducing rot and killing the plant.
In a cold, arid climate like North Dakota (zones 4-5), there is very little soil moisture going into the winter months, so there are several cacti that are native to the region.
I live in a far wetter climate, so conditions are harder here. But if I plant in a raised bed of mostly sand and organic matter, the drainage might be good enough for quite a few cactus species.
What cacti will grow well in this type of climate, with little or no care over the winter?