I see what looks like fungal spots on the leaf undersides (where they curl up), which is indicative of inadequate nutrition (and probably poor soil health). Plants deficient in necessary elements are far more likely to have issues like this than are healthy trees, and they can't fight it off on their own.
The best way to solve this is to get your soil tested (send a sample to your county's extension office), and correct any nutrient/mineral deficiency that may come up. Also, one of the best things you can do is to mulch around the tree with a good layer of fine compost (composted manure works well). This not only feeds the soil life and the tree, leading to a much healthier life for the tree, it will hold in soil moisture, and regulate the soil temperature quite a bit.
I seriously doubt the tree will keep on going downhill if you provide the right cultural requirements. While you're at it, you might as well do this for your other trees as well. Prevention is better than cure.