Recoil is the factor. Heavy loads, at low velocities, still put a lot of stress on wood. It has been suggested that light loads, at low pressure and most often low velocities, generate low recoil and hence less stress on wood. It all comes back to simple vector physics, for every vector, action or force there is an equal and opposite vector, action or force. If you push a payload out the barrel a force is thrust into and against the stock.