Aphids are killing my Lenten Roses (Hellebores). I have treated them with natural spray (water, vinegar and Dr Bronner's soap) and with Neem oil. They keep reinfecting; I can't get rid of them.
How do I eliminate them permanently?
That looks like Hellebore aphid; you can hose them off, or use the remedies you've tried, but none of these will give lasting control, they will just get rid of what's there when you treat, which means you will need to retreat probably every day or two. If you use neem spray, remember to spray beneath the leaves too, because neem should have more of a residual effect than soapy spray, although still won't work for long. If you want to clear them completely, you will need to use a systemic insecticide (rather than a contact insecticide) which will penetrate the sap stream of the plant and keep the aphids off for usually up to 14 days, depending on the product. Look for a systemic insecticide spray that says it treats for aphids. Some information on Hellebore aphid here https://pnwhandbooks.org/insect/hort/landscape/hosts-pests-landscape-plants/hellebore-helleborus-aphid
I use actual chemical insecticide (responsibly and according to instructions) and it does a really good job at controlling aphids on my roses.
From my personal experience, neem oil does a great job at preventing aphids from becoming established. But once they do become established, I recommend using a pyrethrin based insecticide. Pyrethrins are organic and very powerful. It kills aphids, whiteflies, and other small insects. You can find multiple pyrethrin insecticides on Amazon. Dilute it to what the package says and just spray it on. Almost 99% of the aphids should be killed with one treatment.