We do grow hazelnuts, in large quantity actually, though not so many as we once did, mostly down in Kent, where its called the Kentish cob and/or filbert (there are differences between the two). They grow well in the southern half of Britain, and don't take up too much space.
We also grow some sweet chestnuts, but because we've been in Europe for so many years, we import more chestnuts than we grow - ours are not ready for consumption by Christmas, which is mostly when consumers want them, but chestnuts grown in other countries with more reliable climates are, so we import them, in fact, they're in the shops now, I bought some yesterday. Chestnut production in the UK has, over the years, declined quite a bit because it works out cheaper to import them. They also prefer sandier soil, and a lot of growing areas of the UK have heavy soil.
Walnuts also are grown, but not, so far as I know, commercially - walnut trees are prone to catching late frosts, and the unpredictable nature of the weather here means they're highly likely to catch a late frost, so from a commercial point of view, its been more cost effective to import.
Apart from commercial cost effectiveness, its a question of space - growing space in the southern half of Britain is limited and walnuts, for instance, are pretty large trees, and if the crop is unreliable because of the conditions, then it makes more sense to create, say, an apple orchard instead.
The reasons we don't grow so many nuts now as we once did are simple - economics (money, in other words) and globalisation. If commercial growers find the best conditions for various nuts in other countries, that's where they'll be grown, and will then be shipped out, so although I'm old enough to recall as a child British filberts and chestnuts being sold on street corners and market stalls, economies of scale mean the world has moved on to doing what's cost effective, because its more economic to import some crops than struggle to grow them here, and use the (limited) growing space here for what will grow easily and more productively.
