I live on a sloped lot in Utah (just north of Salt Lake City, USDA zone 7b) with several rock retaining walls (small boulders 3-5' in diameter). After hard rains I've noticed considerable dirt plumes from drainage between the boulders and I worry about erosion. The previous owner had some sort of ground cover plants in between but those pretty much died off when the house sat vacant for one year. I also get weeds growing in the dirt between the rocks. I'm looking for a good, low-maintenance solution to minimize erosion and weeds, even if it's non-organic.
The best solution is to build a retaining wall with a geotextile fabric that will stop soil from being washed out. This might not be as large a job as you might think.
Alternately you can look for native plants that tolerate hot and dry conditions and plant them into the walls. They will stabilize the soil and stop the washout.
Well it depends on how much work you want to do and how much you want to spend In salt lake there are a few good places to get native plants, Cactus and Tropicals has an outdoor area during the warm months. You could do some cool native plants depending on sun and water, you could try wasatch penstamen, the native four o' clock, service berry, choke cherry, mountain mahogany, red dogwood, scarlet Gillia.
Sorry for any of those names that I munched..
USU has a good list here: http://www.hort.usu.edu/plantguide/index.htm
Also if you live near the bench, watch out for rattlesnakes, they love rock piles.