According to "Seed to Seed" by Suzanne Ashworth, p29:
Always realize that seed vigor can be lost during storage well before the seeds die completely. For example, if a seed sample germinates at only 20%, that doesn't mean those seeds which sprouted would have grown normally. That sample's seed vigor has diminished to the point that 80% of the seeds aren't even strong enough to sprout, and the remaining 20% will have so little strength that they probably will grow weakly and may not even reach maturity. Constantly keep in mind that your seeds are living entities and will only grow strongly during subsequent seasons if you do everything possible to see that they are stored in truly excellent conditions. Strong plants in the garden are partially a reflection of good seed storage techniques.
So to the second part of your question, yes, you can get reduced germination rates and weaker adult plants.
But if the seeds have been stored well (in a cool place with stable temperature, away from light, and very dry) they may still be strong. If you have enough seed to spare for a test, it's worth doing a germination rate test as described in Peter Turner's answer so that you can get an idea of whether it's worth the effort and space in your garden. Some seed is so cheap (or you need so little) that it's better to toss the old seed and buy fresh.
Re: does it vary from plant to plant? Yes, but it's not based on how "easy" the plants are to grow. See the longevity table in this article. For example, onion seeds do not store well: germination drops off a lot after 1 year. But endive or cucumber can store for 5 years. And if you keep your seeds very dry and cool you can store them for even longer.
Re: "apply these expiry dates cynically, in order to generate extra sales", I don't know of any evidence. But I'm guessing many casual gardeners aren't storing their seeds in ideal conditions, so the "sow by" date you see is probably legitimate. Good seed companies will mark the date and percentage of the last germination test on the packet. Use a chart like the one linked above to decide for yourself (or perform your own germination test) if the seeds are still good.