When I recently replaced an herb (by a different kind of herb) in a pot that didn't grow sufficiently anymore, I found that quite a large portion of the soil was held tightly by the many small roots of the herb. Is it beneficial for the growth of the new herb to throw all of that out and replace the volume by new soil, or should the roots along with the soil they hold be kept in the pot, possibly teared apart?
Yes, throw it all out and use fresh compost.
You can throw it on your garden or compost heap, not in the waste bin, of course.
The ecosystem you have in a small pot isn't diverse enough to decompose the old roots, and in any case that would take months or years to complete. For example having earthworms (and smaller soil-dwelling animals) living in a pot is not a good idea, though they are very beneficial "in the wild."
In the outside world that doesn't matter that the decomposition process is slow, because plants can just grow roots somewhere else rather than into the remains of a previous plant, but they can't do that when confined in a pot completely full of old roots!