Technically, they are perennials, but they tend not to do well over winter. I have managed to over winter them. But I have to grow them in pots and bring them in (to avoid frosts), and then they grow slowly and seem prone to disease, lack of sun, etc. Hence most people (including myself now) grow them as annuals.
They don't tolerate frost at all. But I have several indoor plants 3 years and older, which do very well in spring when light conditions improve.
Of my balcony peppers which I store on the attic stairs during winter usually about half survive it, being very ill tended in that time.