It sounds like you may have an infestation of fungus gnats, especially as your plant was outdoors during summer. The larvae can damage the roots, but their presence in houseplants is usually a sign the plant is kept too damp; fsungus gnats are particularly attracted to moist soil. Allow the top inch or so of the planting medium to dry out before watering, then water thoroughly, tipping away any excess in an outer pot or tray after 30 minutes so the plant is not left sitting in water. I'm assuming there is good drainage from the  pot; there is no need to change the potting soil unless you've had the plant for longer than a year, when it might be a good idea. More information here http://extension.colostate.edu/topic-areas/insects/fungus-gnats-as-houseplant-and-indoor-pests-5-584/