The plants themselves have low toxicity, so unless your pets start eating a lot of leaves there shouldn't be any problem with that.
However honey made from the nectar is extremely toxic, and in fact was the cause of one of the earliest accounts of mass poisoning, when Roman troops invaded Turkey in the first century BCE. Regular medical problems still occur in Turkey where people eat "wild honey" in regions where there are many rhododendrons and azaleas growing wild. The fruits and seeds are also highly toxic.
Any bee-keepers in the neighbourhood will not appreciate these plants at all, since they can pollute an entire hive's production of honey into so-called "mad honey," named from the mild form of the symptoms it produces when eaten. Severe symptoms can be life threatening.
There are also reported medical incidents where children mistook the flowers for honeysuckle, and sucked the nectar expecting it to be sweet.
But as benn's answer says, they are beautiful plants so long as you can avoid the risks. You might like the video of a rhododendron garden near where I live, in the UK: https://youtu.be/cVuezDLNI8M