I wouldn't bother with insecticides for aphids, although they could get rid of them, probably. I recommend getting a shower nozzle for your hose and showering the foliage every two or three days. Aphids don't tend to stay on regularly showered plants. I didn't have any issues with plants I did that with in 2016, except when I wasn't doing it. It should spray the aphids off, but I don't think that's exactly why it works. The plants seem more resistant to them for some reason. I didn't have whitefly issues either. It also helps against spider mites. It made my plants grow a lot faster, too.
Anyway, if your plants get fungal diseases from humidity and rain easily, this may not be the best suggestion.
Insects can also spread disease, however.
Also, hard water might damage leaves by leaving mineral deposits.
Whatever the case, don't just use a regular hose without a nozzle to spray your plants. It doesn't seem to work and is likely to splash more soil on your foliage, which can hinder growth, and according to many, bring diseases to your foliage.
The Orbit 58993 Control Contractor Turret Pistol Spray Nozzle is the one I use. It's durable and works well, but it lacks a soaker setting (so I use a dedicated soaker piece when I want that).
It's possible that the goji berries aren't very acclimated to your soil and/or growing conditions. This may result in weaker plants. They'll probably do better in the future some time, as long as you get rid of your pests.
In my experience, fertilizing with calcium nitrate (foliar spray) can deter aphids, but make sure you could use the calcium/nitrogen, first. It doesn't deter ladybugs.
Ladybugs and lacewings may also help.