Triclopyr, often sold as "Stump and Brush Killer" will kill all woody plants, It's easy to use and doesn't require you to physically remove tree roots - you just cut each stem/trunk and paint the chemical straight from the bottle onto the cut surface. This will kill each stem/trunk you cut within a couple of weeks. Quick, easy, and safe, because there's no mixing or spraying of chemicals.
As for the herbaceous perennials, you could apply glyphosate (this, too, is paintable, if diluted first according to directions). Note that this chemical may be carcinogenic. As I understand it, Bayer will be withdrawing RoundUp from the US market in 2023. I have also had success using triclopyr on some herbaceous plants, especially fall-blooming anemones that tried to take over a large bed, so you may want to try it on these, too. In this case, dig around the plant a little below ground level and paint it on as many roots as you can find. Do not cover up the exposed roots.