You should look at Red Worm Composting. I've found a lot of useful information there. I was recently looking into this, because I have 6 horses and they're full of. . . worm food. So I was looking into the use of it in my vermicomposting.
The guy that runs the site was posting about it and he was saying to let it age for a few reasons. One is that animal wastes have a high salt content and aging it helps remove some of that and keeps it from harming your worms. Also, like another poster said, it can heat up your worm bin and harm your worms, where it wouldn't if it had been pre-composted. I believe another issue was the gas that freshly decomposing manure gives off inside of what is usually a closed environment with vermicomposting. If you have an open outdoor pile, it's probably not as big of an issue.
He actually said, that you can sometimes find compost worms on farms in old manure piles and you can also add fresh manure to outdoor composts with out much issue, because there is plenty of air flow and the worms can simply move somewhere else in the pile until the manure is perfect for them to begin working on.
Lastly, I may be misremembering and it may have been another animals waste (I know you aren't supposed to use human, cat, or dog), but you had to let it break down to help destroy any bacteria in the feces that are harmful to people. If it isn't probably decomposed and processed, then when you put it in with your garden plant, the water can hit it and splash the bad stuff up onto what you plan to eat.