Is this a desirable plant or a weed?

Question

Found these growing in our flower bed of the home we recently purchased. I didn't really like how they looked, so I started ripping them out. Afterwards, I noticed some in other strategic places and thought maybe I had made a mistake.

Learned earlier today that the stuff in the background is thistle. I'm interested in the stuff with the reddish stem.

This is a picture of the root of one that I dug up.

The stuff growing in another portion of the yard is starting to develop bud at the end. I'm not sure what will come out of it though.

Answer 1

The leaves in the first picture make me think it's a Peony which would make it a good plant to have. A look at the buds on your other plants would help confirm it. This is a picture of one in my yard; notice the globe-shaped flower bud at top right held high above the leaves:

The shoots look like you have the herbaceous type, meaning that they die back every year, regrowing in the Spring. In the right conditions, the plants can live for a long time. Some varieties can be extremely fragrant, and with some the flowers can be so big and heavy that they need support either by staking the stems or with a peony cage. They can be spectacular when they flower as a quick image search will show (image from Wikipedia):

I don't think they like to be disturbed once they've settled into a spot, though, so the ones that you've pulled up may not do well for a while if you decide to replant them.