Max,

If it's a beavertail, check the recoil reinforcing screw that
threads into the forend iron from the front of the forend wood.
If it's not tight, this could cause your movement.

Dan