The Nilo shotshell efficiency test of 1972-73 showed these numbers: A load of #4 steel at 40 yards in a 90 degree, 10 mph crosswind moved the pattern 8.36". So, three times that wind velocity would move it three times as much, or 25", but only 18.75" at thirty yards. BBs would deflect even less than the #4s tested. Maybe you underestimated the wind speed.
Number 4 lead loads did not deflect as much, showing 6.56" at the same 40 yd. range.
Not making any accusations, but flinching, or jerking the trigger, on a "squirrel aimed" shotgun shot can account for a lot of pattern movement, and is very easy to do, even by very experienced shotgunners.
SRH