The lemon trees are about 4 inches tall, have been in ground for about 4 months, and are doing very well but there is going to be frost soon and don’t know how to keep them warm during winter since they are so small.
Cover them in reemay cloth (white gardening cloth) or a sheet. This will keep the frost from touching the plants. This will keep the plant warmer than the air above it. YOu can also mulch around them with straw to keep the plant warmer than the air around it. Make sure you use straw not hay. You can create a hoop house with the remay cloth keeping the cloth there the entire winter. You can take the remay off in the day, but you can also keep it there until spring. It will help insulate the plant. Take a look at the link I provided.
How long to keep plants covered
I'm not sure where you live, but if you live outside of USDA Zone 8, you should probably be keeping it indoors a in a warm location for winter. Then outside again in the spring. Making sure you mist the leaves often indoors, becuase citrus tend to attract spider mites. Misting the leaves often will keep any population lower. You can also use a kitchen sprayer to wash off the leaves once a month. Keeping the frond humid and washed off keeps spiders mites from taking off.
In spring you can fertilise you lemon with a Palm & Citrus fertiliser. Make sure it is somewhere the rains can get to it.