I have an olive tree that was a mail order gift about four years ago. Unusually for mail-order florist "living plant" deliveries, this one has survived more than a few months! It came with two olives on it to prove that it was an olive tree, but I haven't seen any signs of flowers or olives since!
Location: North Texas. We do get winter frosts, so it is in a pot which comes in for winter. The rest of the year it is on the deck and gets a lot of sun from mid-morning to early evening. It is also quite thirsty. Thirst is partly due to the summer heat, but when it dries out it does drop its leaves. This has only happened a couple of times and the leaves come back again, but I wouldn't be surprised if this prohibits flower buds for the year.
When it comes in for winter, it will get some morning sun, but not a huge amount. The house windows are not very good for plants.
It gets a liquid fertilizer perhaps once a year. I gave it some about a month ago, whilst doing the young pepper plants.
So what can I do to encourage flower growth?