You can't just leave it out, you have to substitute it with something. Coir, for example. (I use well-composted horse manure + bedding.) If you leave it out, the soil won't hold enough water.
Mike Perry's answer to this question says:
A good "Potting Mix" is made up of 3 basic parts:
* Water retaining material.
* Drainage material.
* Plant food (fertilizer) material.
In your example recipe, the peat moss is for water retention, the vermiculite is for drainage, and you don't have any plant food. (You might leave out fertilizer from a seed-starting mix if you are going to quickly prick out the germinated seedlings into another potting container that has some food.)
You could substitute coir for the peat moss in that recipe, or if you wanted to add plant food to the mix and you had a compost with good water retention you could try using it for double duty (food & water).
See my answer to the same question for the peat-less recipe that I use (which does have soil, so may not suit your application).