It depends on the building. If the building wall is made from brick, concrete etc, then a climbing vine is unlikely to damage it. If it has a textured plaster finish, then the vine will damage that. If it is wood, then it will tend to make the wood damp, and this can encourage borer attack.
I think you're better off with a wooden house not to allow vines to attach to the timber. I have a Bougainvillea against mine, and the only damage that happens is from the nails I put into the weatherboards so that I can tie it in place. Since it likes direct sunshine, the boards behind it don't suffer from dampness.