It's Nostoc, a type of cyanobacteria - it often occurs in lawns, sometimes in borders, but it's not particularly harmful, in fact it fixes nitrogen, which is good for your plants, despite its rather disgusting appearance. You probably notice it in spring and autumn because that's when the soil's wet enough and temperatures are right for it to be visible, it likes cool, damp weather, which tends to occur at those times of year. Rather than just being one organism, it's actually a collection of cyanobacteria - you're unlikely to be able to get rid of it completely, though you could try clearing some of it away from your primulas. Info here http://askanaturalist.com/what-is-this-green-jelly-slime-in-my-garden/
Not sure I'd put it in the compost bin - I'd be inclined to bag it and bin it. In the compost bin might mean, when you eventually use the compost in the garden, it pops up all over the place in similar weather conditions.