It depends on what is making the water dirty. if it was organic debris (leaves, grass clippings, etc.) that fell in and rotted, you should be fine. There will be a possibility of disease spreading, depending what is decomposing in there. even bird droppings can carry disease.
if the barrel looked clean to begin with and nothing fell in, the color may be from a residue in th barrel. }n that case, you should clean it very thoroughly with a grease cutting cleaner, just in case. Although it is probably safe to irrigate with scummy water, it is generally better to keep everything clean and tidy around plants. i always keep my watering supplies squeaky clean anyway, so I don't have to unclog watering cans, sprinklers, hose nozzles, etc..
I recommend you clean it thoroughly, and at least rinse it out once a month to keep things clear. Dirty water really won't help the plants anyway .