It appears that your apple trees are getting too much nitrogen. Apple trees always prioritize what to do with their available nutrients, and too much nitrogen causes them to abort fruit crops in order to put out new growth with the nitrogen excess. Also, the unbalance caused by the nitrogen excess makes the tree put most of the other available nutrients (Potassium, Phosphorus, and the micronutrients) into the new growth. This also discourages fruit.
I also noted that your trees have never been properly pruned. The tree is very dense in the center where all the fruiting spurs are. Apples don't like growing in the dark. If you don't feel confident pruning your tree, maybe you could consider hiring a professional.
Putting a mulch of high carbon material soaks up nitrogen, especially if it has lots of surface area (like sawdust). Keep in mind the acidifying nature of many of these materials. When you fertilize, use a no-nitrogen fertilizer, like 0-21-15, that has a lot of potassium in it.
If you prune it properly next winter, mulch, fertilize carefully, and keep an eye out for disease, you should meet with more success.
I see only minor damage from the inch-worms. You can spray with carbaryl if you think they will become an issue.