A mining charity has denied it is “constructing a £30m ivory tower”, after accusations by a Labour MP that it was failing former mineworkers’ communities.