This is most likely to be a scale insect but we can do a little diagnosis tree:
- do they fly if disturbed? => aphids or thrip
- do they look like they are surrounded by a cottony tuft? => mealybug
- are they hard shelled and need to be removed with your finger nail or a toothbrush? => scale
There are many different types of scales but the ones in your picture look similar to these onesones or these onesthese ones.
Control measures are similar for aphids, mealybug and scale. Mix about 5 ml of dish soap to one litre of water. Scrub the rue with a soft toothbrush or cloth then rinse with water. Repeat at five to seven day intervals at least twice.
If the problem returns consider cutting back the plant hard and reapplying soap and water. The plant will regrow and the less area you have to control the easier it is.