One guideline would be to look at what's required for organic certification which is in part 36 months with no prohibited substances (In the USA, anyway.) You don't need to find out if you just let it age away (non-organic lawn, not toxic waste dump.)
Assuming you don't have an excess of residual chemicals that would kill your trees, you could go ahead and plant them when ready, but "err on the side of caution" and not eat any fruit (indeed, just pick the flowers once the bees are done with them, or the fruit as soon as it forms and compost it, so the tree spends more energy on being a strong tree) until after the 36 month point (rather than wait 3 years to even plant them.)
Depending on the state of the soils, you might want to take a year of those three and plant various cover crops to till in, or make other improvements to the ground (hauling in organic materials or other soil amendments) before planting your trees.