Some parts of the market are bearing the brunt of a lack of confidence, interest rate rises and investor tax changes.