Complex traits are highly polygenic, with heritability explained by many hundreds of common variants of small effect together with rare variants of large effect1. Yet how this genetic architecture varies along the trait continuum has been underexplored, as has the role of natural selection in shaping this variation. Here we developed an approach based on polygenic risk scores that reveals widespread departures from common-variant architecture in one or both of the tails of 74 quantitative traits