My lemon trees are about 3 inches tall ( as I planted them from seed ) we have cold winters here and am scared that they will freeze. I planted them in the ground about 3 months ago.
The easiest answer is yes. If you are uncertain if your area can grow citrus, then dig it up. Most Citrus require USDA Zone 9 or warmer. Even so, this is the temperature where the cold will kill the entire plants. You need protect from becoming defoliated or killing your crop in the winter. This is going to be 1 to 2 zones higher. Meaning Zone 10 or 11.
I know some gardeners in border line areas cover them in remay (white garden cloth) and use incandescent lights, like Christmas lights to keep them warm only during the really cold spells, but they also grow them right up again the house. I have a attached a video that gives other options like wrapping it in a sheet or light blanket, using space heater for those cold spell, anything to keep it warm during those cold spell. Options for protection citrus
So, the easiest and best answer is to dig it up. Grow it in a pot. Lemons are very easy to grow in pots. If you keep it inside in the winter mist the leaves often to keep away spider mites. Use the kitchen hand sprayer to hose the leaves tops and bottoms off when you water the plant. This again will help with spider mites. They love citrus plants.