Does whether avoidance is "fine" or not depend oh how you're avoiding it?
Got money in an ISA? Why not put it in a standard B.Soc account and have basic rate tax deducted at source? Putting money in an ISA is avoiding tax. The fact is, the government set these up, with limits, as tax free with the explicit intent of inducing people to use them by making them tax-free.
What about buying shares in company B because that's a growth-oriented company and you have unused CGT allowance, instead of company A where you'd pay your marginal income tax rate on dividends? After all, the reason you have a CGT allowance is because government wants us to have it, so what's wrong with utilising it? Yet, making the decision to invest in B instead of A, for that reason, is tax avoidance.