I have the plants in pots with drainage holes inside plastic decorative pots w/o drainage holes (so that my carpet doesn't get wet) and even with a small amount of watering, over time water builds up in the plastic pot.
Pull those pots with the drain holes up and put 1/4" pieces of tile beneath the pot with holes. When you water allow to drain and then DUMP the water out of the bottom pot acting as a saucer.
Did you use potting soil in your pots? How often do you water?
Don't allow your potted plant base to sit in water for any time at all.
Stormy has the right idea; although, I like to get a gap of at least a half inch myself. I have some slabs of rock core samples (from oil well drilling) that are cut just the right thickness, though I don't think you can go out and buy those. But anything that won't rot (plastic, rock, tile) that you can put under the pot with holes to keep it elevated will work. And, if your items are flat and wide enough to be stable, you can stack a few of them, but don't get carried away. Be careful not to cover the holes with whatever you use for elevation.
Be sure to not over-water, otherwise, any amount of elevation will eventually be not enough.
If most of your water is just running straight through, you might want to re-pot in better potting soil.
My dad always waters his plants that are small enough in the sink and leaves them there for a couple of hours so that anything that runs through immediately will not be a problem. I don't like dirt in my sink though, but he has his own sink in his plant room and one in the utility room, so it works for him (he puts a paper towel over the hole so that he doesn't get dirt down the drain).
My dad has another trick that helps with watering. When he pots his plants, he puts pieces of already broken clay pots in the bottom of the pot, being careful not to cover the holes. The pieces absorb water when the soil is very wet and release it back again if the soil gets very dry.
There are rules to sensible watering of most houseplants;the usual advice of watering when the top of the soil feels just about dry to the touch, then watering well, allowing the pot to drain down freely, and then, after 30 minutes, going back and emptying out any excess water in the outer pot or tray. It sounds like the last step in the procedure is the one you're omitting to do...