This tree has branches like citrus, but the leaves are soft like a chili plant; and developing fruit is red.
In this location I would usually expect a Cumquat. I can return to this location for more data.
In Southern Australia.
Gardening & Landscaping Stack Exchange is a question and answer site for gardeners and landscapers. It only takes a minute to sign up.Sign up to join this community
I think this is a hot pepper plant, one of the many ornamental cultivars of Capsicum frutescens. Although often grown as annuals, most cultivars of this species are short-lived perennials that can grow into a small shrub in frost-free climates. Capsicum frutescens cultivars are commonly used as ornamental plants. They're usually called "ornamental peppers." In addition to being pretty, they would be a good choice for a potted sidewalk plant because they like heat and sun, and tolerate drought.
Although you considered Tamarillo, notice that Tamarillo fruits grow in dangling clusters that hang down, whereas your plant has fruits that point up. Fruits that point up is a distinctive characteristic of Capsicum frutescens. Also notice the difference in fruit shapes, and the relative size of the stem compared to the fruit it supports.
Capsicum frutescens fruit:
(closeup of an image on this page)
Although I couldn't find any photos of pepper plant bark, I've seen similar streaky green and brown bark develop on garden-variety hot pepper plants, probably from the closely related species Capsicum annuum.
The flowers are pretty similar between Tamarillo and Capsicum species, so I wouldn't consider flowers a reliable way of distinguishing them (at least not without a much higher-resolution photo, and possibly some precise measurements of specific flower parts).