You are correct to be looking for other causes, and waterlogged clay is a prime suspect. It may be most intuitive if you think about where citrus are native. Waterlogged is a condition to which your tree’s ancestors likely didn't have to adapt.
As for leaving on the fruit, it’s highly unlikely that’s the, or even a, problem. From a theoretical perspective whether it might matter depends on whether the plant is determinate or indeterminate. The distinction labels whether they make all their seeds, including any associated fruits, all in one big batch per year or whether they keep trying to start new seeds all season long. Citrus are determinate. They flower once a year (mostly) and their fruit ripens fairly synchronously.
Once indeterminate plants, which do not include citrus, successfully mature some seeds, they may quit trying to make new ones either for the year or forever. Removing IMmature fruits can sometimes trick a plant into putting energy into a new round of seeds and fruit. That’s why deadheading sometimes can extend flowering, but in this case it’s exactly what you don’t want. Deadheading makes a difference only for indeterminate plants, and while the seeds still have not yet fully matured. Neither applies to your ripe lemons.
About the only real risk in leaving on fruit, besides perhaps a mess on the ground, is that a few of a plant’s pest species may overwinter in the fruit and/or seed. Before I worried about that, though, I’d have a specific pest in mind and it would have to have reached troubling levels too. I suggest spending any time you have for the tree working on better drainage before you fussed over picking off fruit you won’t use.
