As Rincewind42 posted in a comment above, one of the reasons for not drinking directly from the hose is that they contain lead. Lead is used as a stabilizer for the PVC in the hose.
This helpful article from the Santa Barbara Independent has a lot of information on hose safety. Another interesting point in that article is that brass hose fittings may contribute to lead contamination.
The advice there -- and that I've seen elsewhere -- is to let water run through the hose for a minute before using it in situations where you really don't want lead contamination, like a children's wading pool or your dog's dish.
If the first flush of water coming out of the hose has 100x the US EPA maximum contaminant level of 0.015 mg/L, then you'd add 1.5 mg/L of lead to your garden if that's where you first spray the hose. So if the first two liters of water are enough to flush out the hose, you'd add 3mg of lead to the garden every time you turn on the hose.
I can't find a definitive reference to maximum soil contamination levels, but several hits on the US EPA website reference a level of 400 ppm as an action level, and I found some mentions of 1000 ppm as a danger level -- don't grow edibles at this level.
So it would seem that if you're:
- using a hose that has contamination at the high end (100x) of the ones tested by the Independent,
- always dumping the first bit of water from the hose on the same spot of ground,
- never add new material (compost, soil, etc) to dilute the old, and
- growing lettuce (which has a high lead uptake) right on that spot, then
- you might have contaminated food.
An easy way to know if you've got lead in your soil is to get a soil test. My results came back with a level of 3 ppm, which is below the natural background soil lead level of 7-20 ppm for some midwestern US states.
So I guess I'll continue using a "leaded" hose on the garden, though I may spray the first bit of water onto the lawn instead of the lettuce (see "Lead in Garden Soils and Plants" in the U Minn link above), and I'll use the kitchen sink instead of the hose to rinse off produce.